New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 603937 link

Starred by 4 users

Issue metadata

Status: WontFix
Owner:
Last visit > 30 days ago
Closed: Feb 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: All
Pri: 1
Type: Bug

Blocked on:
issue 636163
issue 542320
issue 592025

Blocking:
issue 614975
issue 615829
issue 619701
issue 620403



Sign in to add a comment

Bisect failures hard to diagnose

Project Member Reported by sullivan@chromium.org, Apr 15 2016

Issue description

Now that everything is switched to recipe bisect, the more detailed buildbot output is great, but it's hard to find a high-level summary. The old bisect had a "results" step that was pretty clear: https://uberchromegw.corp.google.com/i/internal.tryserver.clankium/builders/clankium_nexus5_perf_bisect/builds/36/steps/Results/logs/stdio

I think that:
1) There should be a doc in chromium/src/tools/perf/docs that explains how to find/read the output for a failure and is linked from perf sheriffing documentation
2) There should be a results step (if there already is, and I'm continuing to miss it, it should be highlighted in the buildbot status page more.)

Assigning this to Roberto, but we may decide to give it to someone on the wider team if it proves to be a good "ramping up" task.


Some examples where I had a hard time diagnosing:
https://build.chromium.org/p/tryserver.chromium.perf/builders/android_s5_perf_bisect/builds/595 - how could I quickly narrow down that this was due to SVN authorization issue? How should I have known that the purple gsutl ls steps are not the major failures?
https://build.chromium.org/p/tryserver.chromium.perf/builders/win_x64_perf_bisect/builds/1200 - no toplevel error showing clearly it is a timeout
 
Cc: pras...@chromium.org
I had this concern when the change was made to make the bisect post results to the dashboard.

The information is still available under steps called 'Post bisect result' that are done after every revision and at the end, and the argument was made that the responsibility of formatting the information for human consumption belonged with the dashboard.

My opinion is that at least for now it would be more helpful to declutter the output of the bisect by nesting steps more logically, remove unnecessary repetition, etc.

+Prasad had good ideas about how the end result should look like.
Cc: perezju@chromium.org
Other related issues:

- I find it really hard to figure out where is the stdio log of the benchmarks as they were run during the bisect steps. For example, often the error: "Metric ['foo', 'bar'] was not found in the test output." means that the browser crashed or somehow stopped prematurely while running the benchmark. But I can't figure out the exact reason without looking at the stdio.

- The old bisect script had some "partial bisect results" which were useful to look at even if the bisect job eventually went on to fail due to some other random reasons; or to make a call on a possible suspect CL even if it's taking a lot longer to finish the full bisect. Not sure if we have a similar functionality on the new bisect recipe, or where to find that output.
Status: Assigned (was: Unconfirmed)
Cc: robert...@chromium.org
 Issue 606533  has been merged into this issue.
Labels: -Pri-3 Pri-1
Blocking: 614975
Blocking: 615829
Blockedon: 636163
Design doc for this feature: https://goto.google.com/bisectoutput
Also note that this is blocked on the cleanup of the nested steps feature.
Blocking: 620403
Blocking: 619701
Blockedon: 542320 592025
 Issue 396354  has been merged into this issue.
Components: Speed>Bisection
Status: WontFix (was: Assigned)
Closing out old bugs, re-open if still valid.

Sign in to add a comment