Bisects run too many user stories. |
||
Issue descriptionEtienne and I were looking at a potential regression: https://bugs.chromium.org/p/chromium/issues/detail?id=706945 The regression looks more prominent on browse_news. But when I tried to bisect just that user story: https://build.chromium.org/p/tryserver.chromium.perf/builders/winx64_10_perf_bisect/builds/976 The bisect looks like it's running all user stories. Given that it takes 30min to run through all user_stories, only running the relevant user story should speed things up by an order of magnitude.
,
Mar 31 2017
+perezju There's no --story-filter set, so the bisect is just going to run everything. I think there's a command in start_try_job.py:GuessStoryFilter() on the dashboard that fills this in, maybe it doesn't know how to in this case?
,
Mar 31 2017
It looks like the links in the original bug are summary metrics, so the story-filter didn't get prefilled. This wouldn't happen to a perf sheriff, but not sure what to do if someone looks at a summary chart and tries to bisect as in the bug.
,
Mar 31 2017
Ah good point, wonder if we could throw up a warning then that story filter is incompatible with the summary?
,
Mar 31 2017
Alternatively, can we make sure that sheriffs don't bisect on the summary chart? Maybe add a warning "you should pick a story to bisect instead"?
,
Mar 31 2017
I'm not a sheriff. And this regression isn't on the alerts page, because alerts are currently turned off for this test.
,
Apr 3 2017
I agree with Ned, the solution here is probably a warning on the dashboard when trying to start a bisect on a summary metric. |
||
►
Sign in to add a comment |
||
Comment 1 by sullivan@chromium.org
, Mar 30 2017Components: -Speed>Telemetry Speed>Bisection