Bisect ignores repeat_count and max_time_minutes |
||||
Issue descriptionI've run two bisects with different parameters (for the same regression, issue 633193 ): Bisect 1: https://chromeperf.appspot.com/buildbucket_job_status/9005529157844552592 max_time_minutes: 20 repeat_count: 10 Bisect 2: https://chromeperf.appspot.com/buildbucket_job_status/9004900681642967744 max_time_minutes: 100 repeat_count: 100 Nevertheless, the benchmark was run the same number of times in both cases: Bisect 1: https://bugs.chromium.org/p/chromium/issues/detail?id=633193#c3 Revision Mean Std Dev N Good? chromium@407992 464392 1022.11 8 good chromium@408012 463840 0.0 8 bad Bisect 2: https://bugs.chromium.org/p/chromium/issues/detail?id=633193#c7 Revision Mean Std Dev N Good? chromium@407954 463840 0.0 8 good chromium@408012 465062 3456.34 8 bad Given that I increased max_time_minutes 5 times and repeat_count 10 times, I would expect that the second bisect would run each revision ~40 times (5×8).
,
Aug 9 2016
Another example (the same bot in this case): Bisect 1: https://bugs.chromium.org/p/chromium/issues/detail?id=633538#c3 Bisect 2: https://bugs.chromium.org/p/chromium/issues/detail?id=633538#c7
,
Aug 9 2016
,
Aug 9 2016
+dtu The bisect dialog wasn't updated with the new classification algorithm and it's really confusing people. This is logged as https://github.com/catapult-project/catapult/issues/2602 on the catapult side. Dave, Roberto, can you explain how the new classification algorithm works wrt repeats?
,
Sep 13 2017
|
||||
►
Sign in to add a comment |
||||
Comment 1 by petrcermak@chromium.org
, Aug 9 2016