https://bugs.chromium.org/p/chromium/issues/detail?id=664480#c3 https://build.chromium.org/p/tryserver.chromium.perf/builders/win_perf_bisect/builds/7043 https://build.chromium.org/p/tryserver.chromium.perf/builders/win_perf_bisect/builds/7043/steps/Failure%20reason/logs/reason The bug comment lists N/A for the mean of the revisions, but all the steps passed and the failure reason is "Bisect cannot identify a culprit: Bisect failed to reproduce the regression with enough confidence." Did the bisect actually get data to compare? If so, why did it not post to the bug? If not, it seems like there is a pretty big bug in reporting status.
The bisect did not get any data, even after several re-tries. I am working on a patch to fix the error message.
The following revision refers to this bug: https://chromium.googlesource.com/chromium/tools/build.git/+/e042d4dbd0ee62abaa3ad5d74e7dd897519a80bc commit e042d4dbd0ee62abaa3ad5d74e7dd897519a80bc Author: robertocn <robertocn@chromium.org> Date: Fri Nov 11 20:47:42 2016 Better error message when reference range yields no values. The issue is that since catapult code is now parsing values rather than recipe code, failing to find any values in the reference range seems like low confidence. This implements a check to distinguish between those 2 cases. R=sullivan@chromium.org,dtu@chromium.org BUG= 664510 Review-Url: https://codereview.chromium.org/2498693002 [modify] https://crrev.com/e042d4dbd0ee62abaa3ad5d74e7dd897519a80bc/scripts/slave/recipe_modules/auto_bisect_staging/bisector.py
Started bisect jobs https://chromeperf.appspot.com/buildbucket_job_status/8996215073872407920 https://chromeperf.appspot.com/buildbucket_job_status/8996214869675224464 https://chromeperf.appspot.com/buildbucket_job_status/8996214085182740752 https://chromeperf.appspot.com/buildbucket_job_status/8996213944419352352 https://chromeperf.appspot.com/buildbucket_job_status/8996213709775782384
Most jobs failed due to the issue in dashboard (see https://codereview.chromium.org/2501433002/) https://chromeperf.appspot.com/buildbucket_job_status/8996200368754926560 https://chromeperf.appspot.com/buildbucket_job_status/8996200184971116032 https://chromeperf.appspot.com/buildbucket_job_status/8996200070300035360 https://chromeperf.appspot.com/buildbucket_job_status/8996200022383862064
This was a problem with the parsing of metrics in catapult. A successful bisect: https://bugs.chromium.org/p/chromium/issues/detail?id=664100#c15
Comment 1 by sullivan@chromium.org
, Nov 11 2016