New issue
Advanced search Search tips

Issue 769027 link

Starred by 2 users

Issue metadata

Status: Untriaged
Owner: ----
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug



Sign in to add a comment

Investigate how noise affects our telemetry metrics

Project Member Reported by rnep...@chromium.org, Sep 26 2017

Issue description

We currently have very little information about how noise in benchmarks affect how we are able to catch regressions.
 
Project Member

Comment 1 by bugdroid1@chromium.org, Sep 27 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/catapult/+/5d8ddb9f52690dfc67111317e882f24f41634b69

commit 5d8ddb9f52690dfc67111317e882f24f41634b69
Author: rnephew <rnephew@chromium.org>
Date: Wed Sep 27 22:57:10 2017

[Soundwave] Add tool that displays alerts and related noise and bug data.

Bug: chromium:769027
Change-Id: I30edb200e08b5143cb8763169565974c6f88581b
Reviewed-on: https://chromium-review.googlesource.com/688161
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Reviewed-by: rnephew <rnephew@chromium.org>
Commit-Queue: rnephew <rnephew@chromium.org>

[add] https://crrev.com/5d8ddb9f52690dfc67111317e882f24f41634b69/experimental/soundwave/fetch_alert_noise_data

Project Member

Comment 2 by bugdroid1@chromium.org, Oct 15 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/catapult/+/dd5b39ac0cf7621fd1cfa95ef1f514540e123e08

commit dd5b39ac0cf7621fd1cfa95ef1f514540e123e08
Author: rnephew <rnephew@chromium.org>
Date: Fri Oct 13 17:07:57 2017

[SoundWave] Add script for fetching timeseries from perf dashboard.

Also creates and uses a new library for communicating with the dashboard.

Returns the data in the format:
TEST_PATH_1 |[revision, value, timestamp, r_commit_pos, r_webkit_rev]...]|
TEST_PATH_2 |[revision, value, timestamp, r_commit_pos, r_webkit_rev]...]|

It is saved as two different enteries per row. The full test path for the metric
and a list of lists that contains the revision, value, timestamp, and commit
position for all data points.

Bug: chromium:769027
Change-Id: I7d2abd9da47fec4e2aaf144066d6ed2ae5d92cb1
Reviewed-on: https://chromium-review.googlesource.com/698925
Commit-Queue: rnephew <rnephew@chromium.org>
Reviewed-by: Juan Antonio Navarro Pérez <perezju@chromium.org>
Reviewed-by: Ned Nguyen <nednguyen@google.com>

[modify] https://crrev.com/dd5b39ac0cf7621fd1cfa95ef1f514540e123e08/experimental/soundwave/fetch_alert_noise_data
[add] https://crrev.com/dd5b39ac0cf7621fd1cfa95ef1f514540e123e08/experimental/soundwave/fetch_timeseries_data_for_benchmark
[add] https://crrev.com/dd5b39ac0cf7621fd1cfa95ef1f514540e123e08/experimental/soundwave/dashboard_api.py

Project Member

Comment 3 by bugdroid1@chromium.org, Nov 2 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/catapult/+/b73033696d2dad7a5903d483852b4f0695358c44

commit b73033696d2dad7a5903d483852b4f0695358c44
Author: rnephew <rnephew@chromium.org>
Date: Thu Nov 02 20:12:57 2017

[Soundwave] Make fetch_alert_data use same dashboard api that fetch_timeseries uses

Bug: chromium:769027
Change-Id: I45b5c87766739f0bc7b6261944d3ea2456f7cd54
Reviewed-on: https://chromium-review.googlesource.com/724177
Commit-Queue: rnephew <rnephew@chromium.org>
Reviewed-by: Juan Antonio Navarro Pérez <perezju@chromium.org>
Reviewed-by: Charlie Andrews <charliea@chromium.org>

[delete] https://crrev.com/c6f1c0c98c627dc57866033d32739c4af30825d0/experimental/soundwave/fetch_alert_noise_data
[add] https://crrev.com/b73033696d2dad7a5903d483852b4f0695358c44/experimental/soundwave/fetch_alert_data
[modify] https://crrev.com/b73033696d2dad7a5903d483852b4f0695358c44/experimental/soundwave/dashboard_api.py

Project Member

Comment 4 by bugdroid1@chromium.org, Nov 6 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/catapult/+/a0da5f7b56baee87826a035bb4c64f38cf171c09

commit a0da5f7b56baee87826a035bb4c64f38cf171c09
Author: rnephew <rnephew@chromium.org>
Date: Mon Nov 06 18:28:09 2017

[Soundwave] Refactor fetching all timeseries for a benchmark into communicator.

This is to support the ability to grab multiple benchmarks worth of data at one time.

Bug: chromium:769027
Change-Id: Ifb84f1d28bdb3a5ef0075d5b2362d61375bbe3b9
Reviewed-on: https://chromium-review.googlesource.com/752142
Commit-Queue: rnephew <rnephew@chromium.org>
Reviewed-by: Charlie Andrews <charliea@chromium.org>
Reviewed-by: Juan Antonio Navarro Pérez <perezju@chromium.org>

[modify] https://crrev.com/a0da5f7b56baee87826a035bb4c64f38cf171c09/experimental/soundwave/fetch_timeseries_data_for_benchmark
[modify] https://crrev.com/a0da5f7b56baee87826a035bb4c64f38cf171c09/experimental/soundwave/dashboard_api.py

Project Member

Comment 5 by bugdroid1@chromium.org, Nov 10 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/catapult/+/16fb1b78744e229c8bbc5d555d2503cf1901de61

commit 16fb1b78744e229c8bbc5d555d2503cf1901de61
Author: rnephew <rnephew@chromium.org>
Date: Fri Nov 10 15:05:50 2017

[soundwave] Make fetch_alerts_data interpret days as an int.

IT currently passes it as a string, which causes problems later when
it is assumed to be an int.

Bug: chromium:769027
Change-Id: I12e5b5ede7e4cca2e7d49fca9fcfd9d61f268667
Reviewed-on: https://chromium-review.googlesource.com/763568
Commit-Queue: Charlie Andrews <charliea@chromium.org>
Reviewed-by: Charlie Andrews <charliea@chromium.org>

[modify] https://crrev.com/16fb1b78744e229c8bbc5d555d2503cf1901de61/experimental/soundwave/fetch_alert_data

Status: Assigned (was: Untriaged)
Owner: ----
Status: Available (was: Assigned)
Status: Untriaged (was: Available)
Available, but no owner or component? Please find a component, as no one will ever find this without one.

Sign in to add a comment