Bisect failures hard to diagnose |
|||||||||||
Issue descriptionNow that everything is switched to recipe bisect, the more detailed buildbot output is great, but it's hard to find a high-level summary. The old bisect had a "results" step that was pretty clear: https://uberchromegw.corp.google.com/i/internal.tryserver.clankium/builders/clankium_nexus5_perf_bisect/builds/36/steps/Results/logs/stdio I think that: 1) There should be a doc in chromium/src/tools/perf/docs that explains how to find/read the output for a failure and is linked from perf sheriffing documentation 2) There should be a results step (if there already is, and I'm continuing to miss it, it should be highlighted in the buildbot status page more.) Assigning this to Roberto, but we may decide to give it to someone on the wider team if it proves to be a good "ramping up" task. Some examples where I had a hard time diagnosing: https://build.chromium.org/p/tryserver.chromium.perf/builders/android_s5_perf_bisect/builds/595 - how could I quickly narrow down that this was due to SVN authorization issue? How should I have known that the purple gsutl ls steps are not the major failures? https://build.chromium.org/p/tryserver.chromium.perf/builders/win_x64_perf_bisect/builds/1200 - no toplevel error showing clearly it is a timeout
,
Apr 19 2016
Other related issues: - I find it really hard to figure out where is the stdio log of the benchmarks as they were run during the bisect steps. For example, often the error: "Metric ['foo', 'bar'] was not found in the test output." means that the browser crashed or somehow stopped prematurely while running the benchmark. But I can't figure out the exact reason without looking at the stdio. - The old bisect script had some "partial bisect results" which were useful to look at even if the bisect job eventually went on to fail due to some other random reasons; or to make a call on a possible suspect CL even if it's taking a lot longer to finish the full bisect. Not sure if we have a similar functionality on the new bisect recipe, or where to find that output.
,
Apr 20 2016
,
Apr 26 2016
,
Apr 26 2016
,
May 26 2016
,
May 31 2016
,
Aug 11 2016
Design doc for this feature: https://goto.google.com/bisectoutput Also note that this is blocked on the cleanup of the nested steps feature.
,
Aug 12 2016
,
Aug 12 2016
,
Aug 23 2016
Issue 396354 has been merged into this issue.
,
Feb 3 2017
,
Feb 7 2017
Closing out old bugs, re-open if still valid. |
|||||||||||
►
Sign in to add a comment |
|||||||||||
Comment 1 by robert...@chromium.org
, Apr 15 2016