Investigate how noise affects our telemetry metrics |
||||
Issue descriptionWe currently have very little information about how noise in benchmarks affect how we are able to catch regressions.
,
Oct 15 2017
The following revision refers to this bug: https://chromium.googlesource.com/catapult/+/dd5b39ac0cf7621fd1cfa95ef1f514540e123e08 commit dd5b39ac0cf7621fd1cfa95ef1f514540e123e08 Author: rnephew <rnephew@chromium.org> Date: Fri Oct 13 17:07:57 2017 [SoundWave] Add script for fetching timeseries from perf dashboard. Also creates and uses a new library for communicating with the dashboard. Returns the data in the format: TEST_PATH_1 |[revision, value, timestamp, r_commit_pos, r_webkit_rev]...]| TEST_PATH_2 |[revision, value, timestamp, r_commit_pos, r_webkit_rev]...]| It is saved as two different enteries per row. The full test path for the metric and a list of lists that contains the revision, value, timestamp, and commit position for all data points. Bug: chromium:769027 Change-Id: I7d2abd9da47fec4e2aaf144066d6ed2ae5d92cb1 Reviewed-on: https://chromium-review.googlesource.com/698925 Commit-Queue: rnephew <rnephew@chromium.org> Reviewed-by: Juan Antonio Navarro Pérez <perezju@chromium.org> Reviewed-by: Ned Nguyen <nednguyen@google.com> [modify] https://crrev.com/dd5b39ac0cf7621fd1cfa95ef1f514540e123e08/experimental/soundwave/fetch_alert_noise_data [add] https://crrev.com/dd5b39ac0cf7621fd1cfa95ef1f514540e123e08/experimental/soundwave/fetch_timeseries_data_for_benchmark [add] https://crrev.com/dd5b39ac0cf7621fd1cfa95ef1f514540e123e08/experimental/soundwave/dashboard_api.py
,
Nov 2 2017
The following revision refers to this bug: https://chromium.googlesource.com/catapult/+/b73033696d2dad7a5903d483852b4f0695358c44 commit b73033696d2dad7a5903d483852b4f0695358c44 Author: rnephew <rnephew@chromium.org> Date: Thu Nov 02 20:12:57 2017 [Soundwave] Make fetch_alert_data use same dashboard api that fetch_timeseries uses Bug: chromium:769027 Change-Id: I45b5c87766739f0bc7b6261944d3ea2456f7cd54 Reviewed-on: https://chromium-review.googlesource.com/724177 Commit-Queue: rnephew <rnephew@chromium.org> Reviewed-by: Juan Antonio Navarro Pérez <perezju@chromium.org> Reviewed-by: Charlie Andrews <charliea@chromium.org> [delete] https://crrev.com/c6f1c0c98c627dc57866033d32739c4af30825d0/experimental/soundwave/fetch_alert_noise_data [add] https://crrev.com/b73033696d2dad7a5903d483852b4f0695358c44/experimental/soundwave/fetch_alert_data [modify] https://crrev.com/b73033696d2dad7a5903d483852b4f0695358c44/experimental/soundwave/dashboard_api.py
,
Nov 6 2017
The following revision refers to this bug: https://chromium.googlesource.com/catapult/+/a0da5f7b56baee87826a035bb4c64f38cf171c09 commit a0da5f7b56baee87826a035bb4c64f38cf171c09 Author: rnephew <rnephew@chromium.org> Date: Mon Nov 06 18:28:09 2017 [Soundwave] Refactor fetching all timeseries for a benchmark into communicator. This is to support the ability to grab multiple benchmarks worth of data at one time. Bug: chromium:769027 Change-Id: Ifb84f1d28bdb3a5ef0075d5b2362d61375bbe3b9 Reviewed-on: https://chromium-review.googlesource.com/752142 Commit-Queue: rnephew <rnephew@chromium.org> Reviewed-by: Charlie Andrews <charliea@chromium.org> Reviewed-by: Juan Antonio Navarro Pérez <perezju@chromium.org> [modify] https://crrev.com/a0da5f7b56baee87826a035bb4c64f38cf171c09/experimental/soundwave/fetch_timeseries_data_for_benchmark [modify] https://crrev.com/a0da5f7b56baee87826a035bb4c64f38cf171c09/experimental/soundwave/dashboard_api.py
,
Nov 10 2017
The following revision refers to this bug: https://chromium.googlesource.com/catapult/+/16fb1b78744e229c8bbc5d555d2503cf1901de61 commit 16fb1b78744e229c8bbc5d555d2503cf1901de61 Author: rnephew <rnephew@chromium.org> Date: Fri Nov 10 15:05:50 2017 [soundwave] Make fetch_alerts_data interpret days as an int. IT currently passes it as a string, which causes problems later when it is assumed to be an int. Bug: chromium:769027 Change-Id: I12e5b5ede7e4cca2e7d49fca9fcfd9d61f268667 Reviewed-on: https://chromium-review.googlesource.com/763568 Commit-Queue: Charlie Andrews <charliea@chromium.org> Reviewed-by: Charlie Andrews <charliea@chromium.org> [modify] https://crrev.com/16fb1b78744e229c8bbc5d555d2503cf1901de61/experimental/soundwave/fetch_alert_data
,
Aug 1
,
Aug 2
,
Jan 11
Available, but no owner or component? Please find a component, as no one will ever find this without one. |
||||
►
Sign in to add a comment |
||||
Comment 1 by bugdroid1@chromium.org
, Sep 27 2017