investigate if notifications of CrOS perf regressions could be sent earlier |
||||||
Issue descriptionhttps://chromeperf.appspot.com/report?sid=907a0232ad50024ad5409d2168a321d3a3ea36371cc0b5758eb02348fc2edd22 When Chrome doesn't uprev for a while but we have a performance regression in a Chrome OS perf dashboard test the notification is only sent once Chrome revisions uprev again. In other words while the dashboard plots the timeline properly with Chrome OS revisions as secondary keys (and the regression is clearly visible), the notification seems to ignores it. (We did get notifications from DUTs/tests in the PFQ, where Chrome revision goes up every day but not from the CQ where Chrome can be a few days old.) Sources might be here https://cs.chromium.org/chromium/src/third_party/catapult/dashboard/dashboard/models/anomaly.py?type=cs&q=%22zero-to-nonzero%22+package:%5Echromium$&l=15
,
Jun 1 2017
OK, I see. It seems like the reason that we get the graphics_Idle alerts but graphics_Sanity is that somehow graphics_Idle generates more points than the other tests.
,
Jun 1 2017
+sullivan for how to test settings From offline discussion, Po-Hsien would like to try out different settings but I'm a little unfamiliar with the process of trying those out.
,
Jun 1 2017
Yes, many tests we run at most once a day and waiting 2+ weeks for an alert is a long time.
,
Jun 1 2017
You can experiment with changing the alert config settings in the debug page. Screenshot of the link to get there attached. You'll likely want to tweak "min_segment_size". The tradeoff is that with a lower segment size, you get alerts faster, but if there are short jumps due to noise, you may be alerted on those. Once you have a config you like, use the "Report Issue" menu on dashboard to request a config change.
,
Jun 1 2017
Very cool page! Is it possible to set different min_segment_size for different paths/wild cards like ChromeOS_Graphics/*/Failures or is it per master only (same value for all of ChromeOS_Graphics)?
,
Jun 1 2017
You can set different min_segment_size for different paths/wildcards as long as they don't overlap (if they do, the first one alphabetically will be chosen, which is a bit confusing)
,
Jun 2 2017
,
Jun 2 2017
Thanks everyone!
,
Aug 1 2017
,
Jan 22 2018
|
||||||
►
Sign in to add a comment |
||||||
Comment 1 by simonhatch@chromium.org
, Jun 1 2017