New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 728389 link

Starred by 1 user

Issue metadata

Status: Archived
Owner:
Closed: Jun 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 3
Type: Bug

Blocked on:
issue 728848



Sign in to add a comment

investigate if notifications of CrOS perf regressions could be sent earlier

Project Member Reported by ihf@chromium.org, May 31 2017

Issue description

https://chromeperf.appspot.com/report?sid=907a0232ad50024ad5409d2168a321d3a3ea36371cc0b5758eb02348fc2edd22

When Chrome doesn't uprev for a while but we have a performance regression in a Chrome OS perf dashboard test the notification is only sent once Chrome revisions uprev again. In other words while the dashboard plots the timeline properly with Chrome OS revisions as secondary keys (and the regression is clearly visible), the notification seems to ignores it. (We did get notifications from DUTs/tests in the PFQ, where Chrome revision goes up every day but not from the CQ where Chrome can be a few days old.)

Sources might be here
https://cs.chromium.org/chromium/src/third_party/catapult/dashboard/dashboard/models/anomaly.py?type=cs&q=%22zero-to-nonzero%22+package:%5Echromium$&l=15
 
So according to the datastore timestamp, the anomaly object was generated early May 30, and the actual regression occurred on May 27. Is that the difference you're seeing?

There's an anomaly config associated with Chrome OS Graphics that sets the min_segment_size to 12, which as I understand it means that it needs to collect 12 points before/after a change point before it considers it.

Comment 2 by pwang@chromium.org, Jun 1 2017

OK, I see. 

It seems like the reason that we get the graphics_Idle alerts but graphics_Sanity is that somehow graphics_Idle generates more points than the other tests.
Cc: sullivan@chromium.org
+sullivan for how to test settings

From offline discussion, Po-Hsien would like to try out different settings but I'm a little unfamiliar with the process of trying those out.

Comment 4 by ihf@chromium.org, Jun 1 2017

Yes, many tests we run at most once a day and waiting 2+ weeks for an alert is a long time.
You can experiment with changing the alert config settings in the debug page. Screenshot of the link to get there attached.

You'll likely want to tweak "min_segment_size". The tradeoff is that with a lower segment size, you get alerts faster, but if there are short jumps due to noise, you may be alerted on those.

Once you have a config you like, use the "Report Issue" menu on dashboard to request a config change.
debug-alerts.png
53.0 KB View Download

Comment 6 by ihf@chromium.org, Jun 1 2017

Very cool page!

Is it possible to set different min_segment_size for different paths/wild cards like ChromeOS_Graphics/*/Failures or is it per master only (same value for all of ChromeOS_Graphics)?
You can set different min_segment_size for different paths/wildcards as long as they don't overlap (if they do, the first one alphabetically will be chosen, which is a bit confusing)

Comment 8 by ihf@chromium.org, Jun 2 2017

Blockedon: 728848
Status: Started (was: Untriaged)

Comment 9 by ihf@chromium.org, Jun 2 2017

Status: Fixed (was: Started)
Thanks everyone!
Labels: VerifyIn-61

Comment 11 by dchan@chromium.org, Jan 22 2018

Status: Archived (was: Fixed)

Sign in to add a comment