New issue
Advanced search Search tips

Issue 665439 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Jan 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug

Blocking:
issue 664494
issue 661520
issue 664191
issue 669607
issue 671067
issue 675591



Sign in to add a comment

Inconclusive bisect report

Project Member Reported by sullivan@chromium.org, Nov 15 2016

Issue description

It looks like it couldn't figure out where to bisect next, is that correct? If so, error message should be clearer, and status should be consistent with "cannot reproduce".


https://bugs.chromium.org/p/chromium/issues/detail?id=661520#c10


===== BISECT JOB RESULTS =====
Status: completed


===== TESTED REVISIONS =====
Revision                   Mean      Std Dev  N   Good?
android-chrome@688c5e8698  14399463  379675   12  good
android-chrome@bdf0aee84b  14379103  215911   12  good
android-chrome@41ded9d6b7  14428822  498352   18  good
android-chrome@6984a85387  14492357  453238   27  good
android-chrome@3f2e665ead  14520057  487055   27  unknown
android-chrome@3d7660d07c  14544430  493324   27  unknown
android-chrome@4e1223d6a0  14520519  588532   27  unknown
android-chrome@6f8f727ed6  14509075  521898   27  unknown
android-chrome@5a8daab6a4  14523675  576071   27  unknown
android-chrome@1daba1daa7  14565504  316362   18  bad
android-chrome@87c257dd4d  14579310  386425   18  bad

 
Blocking: 664191
Another one:
https://bugs.chromium.org/p/chromium/issues/detail?id=664191#c16
===== BISECT JOB RESULTS =====
Status: completed


===== TESTED REVISIONS =====
Revision                       Mean     Std Dev  N   Good?
chromium@431022                4787623  222748   18  good
chromium@431049                4812721  682320   27  good
chromium@431062                4798846  442090   18  good
chromium@431066                4829685  433505   18  good
chromium@431066,v8@0700bc1bf8  4819817  376403   12  good
chromium@431066,v8@472cb045c5  4766608  665838   27  unknown
chromium@431066,v8@4cfe2e82c6  4691244  92658.5  8   bad
chromium@431067                4731737  403889   18  bad
chromium@431068                4777676  600437   27  unknown
chromium@431069                4745638  694009   27  bad
chromium@431075                4722731  235095   18  bad
The problem is that a single culprit could not be identified. Ideally the bisect would print out the range.
Blocking: 671067
Project Member

Comment 4 by bugdroid1@chromium.org, Dec 5 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build.git/+/a7cdbdfb1d955f04974fe47c2b4e20fe91f5b3ee

commit a7cdbdfb1d955f04974fe47c2b4e20fe91f5b3ee
Author: Roberto Carrillo <robertocn@google.com>
Date: Mon Dec 05 22:35:47 2016

Correct inconclusive bisect messaging.

This re-does https://chromium-review.googlesource.com/#/c/411502/ and
addresses the comments by clarifying the flow of the conditions.

TBR=sullivan@chromium.org,simonhatch@chromium.org
BUG= 665439 

Change-Id: I385b6ed51b324b4bd3dcd7af8ff7f55dec5bdcec
Reviewed-on: https://chromium-review.googlesource.com/413005
Commit-Queue: Roberto Carrillo <robertocn@chromium.org>
Reviewed-by: Roberto Carrillo <robertocn@chromium.org>

[modify] https://crrev.com/a7cdbdfb1d955f04974fe47c2b4e20fe91f5b3ee/scripts/slave/recipe_modules/auto_bisect_staging/bisector.py
[add] https://crrev.com/a7cdbdfb1d955f04974fe47c2b4e20fe91f5b3ee/scripts/slave/recipe_modules/auto_bisect_staging/example.expected/failed_build_inconclusive_1.json
[add] https://crrev.com/a7cdbdfb1d955f04974fe47c2b4e20fe91f5b3ee/scripts/slave/recipe_modules/auto_bisect_staging/example.expected/failed_build_inconclusive_11.json
[modify] https://crrev.com/a7cdbdfb1d955f04974fe47c2b4e20fe91f5b3ee/scripts/slave/recipe_modules/auto_bisect_staging/example.expected/no_values.json
[modify] https://crrev.com/a7cdbdfb1d955f04974fe47c2b4e20fe91f5b3ee/scripts/slave/recipe_modules/auto_bisect_staging/example.py
[modify] https://crrev.com/a7cdbdfb1d955f04974fe47c2b4e20fe91f5b3ee/scripts/slave/recipe_modules/auto_bisect_staging/local_bisect.py
[modify] https://crrev.com/a7cdbdfb1d955f04974fe47c2b4e20fe91f5b3ee/scripts/slave/recipe_modules/auto_bisect_staging/revision_state.py

Blocking: 669607
Project Member

Comment 6 by bugdroid1@chromium.org, Dec 8 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build.git/+/7c86ca265e7d1040175bd012afd6ef6492803ef1

commit 7c86ca265e7d1040175bd012afd6ef6492803ef1
Author: Roberto Carrillo <robertocn@google.com>
Date: Thu Dec 08 18:33:54 2016

Fixing layout of abort reason, removing line breaks from exception.

The details of why a bisect was inconclusive (such as a range of
revisions failing to build) are both the abort reason to be specified
in the bisect report, and the details of the bisect failure. However,
for failing a bisect, we need a string with no newlines as a message
for the StepFailure exception, while for readability we need them in
the bisect report. This change removes the newlines from the message
before raising the failure.

BUG= 665439 
TBR=simonhatch@chromium.org

Change-Id: I1c49040675d3cb7346392590e4ff1278404e06fa
Reviewed-on: https://chromium-review.googlesource.com/417127
Reviewed-by: Roberto Carrillo <robertocn@chromium.org>
Commit-Queue: Roberto Carrillo <robertocn@chromium.org>

[modify] https://crrev.com/7c86ca265e7d1040175bd012afd6ef6492803ef1/scripts/slave/recipe_modules/auto_bisect_staging/bisector.py
[modify] https://crrev.com/7c86ca265e7d1040175bd012afd6ef6492803ef1/scripts/slave/recipe_modules/auto_bisect_staging/example.expected/failed_build_inconclusive_1.json
[modify] https://crrev.com/7c86ca265e7d1040175bd012afd6ef6492803ef1/scripts/slave/recipe_modules/auto_bisect_staging/example.expected/failed_build_inconclusive_11.json
[modify] https://crrev.com/7c86ca265e7d1040175bd012afd6ef6492803ef1/scripts/slave/recipe_modules/auto_bisect_staging/local_bisect.py

Owner: simonhatch@chromium.org
Status: Started (was: Untriaged)
Current plan is to pack the json with relevant data and let the dashboard sort the message out instead of this hybrid approach where part of the message is generated in the recipe and the rest is done when we update the bug.
Project Member

Comment 8 by bugdroid1@chromium.org, Dec 13 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build.git/+/da6a284d86e7f712336c9e19deefdb977be28822

commit da6a284d86e7f712336c9e19deefdb977be28822
Author: Simon <simonhatch@chromium.org>
Date: Tue Dec 13 18:05:04 2016

Bisect - Include failed builds + build ids in revision data.


TBR=robertocn@chromium.org
BUG= 665439 

Change-Id: I0f2c4c1872ea61f54dfa8da747dae6e134df9c28
Reviewed-on: https://chromium-review.googlesource.com/419128
Reviewed-by: Simon Hatch <simonhatch@chromium.org>
Commit-Queue: Simon Hatch <simonhatch@chromium.org>

[modify] https://crrev.com/da6a284d86e7f712336c9e19deefdb977be28822/scripts/slave/recipe_modules/auto_bisect_staging/bisector.py
[modify] https://crrev.com/da6a284d86e7f712336c9e19deefdb977be28822/scripts/slave/recipe_modules/auto_bisect_staging/example.expected/basic_bisect_other_direction.json
[modify] https://crrev.com/da6a284d86e7f712336c9e19deefdb977be28822/scripts/slave/recipe_modules/auto_bisect_staging/example.expected/basic_buildbot_bisect.json
[modify] https://crrev.com/da6a284d86e7f712336c9e19deefdb977be28822/scripts/slave/recipe_modules/auto_bisect_staging/example.expected/basic_linux_bisect.json
[modify] https://crrev.com/da6a284d86e7f712336c9e19deefdb977be28822/scripts/slave/recipe_modules/auto_bisect_staging/example.expected/basic_mac_bisect.json
[modify] https://crrev.com/da6a284d86e7f712336c9e19deefdb977be28822/scripts/slave/recipe_modules/auto_bisect_staging/example.expected/basic_win32_bisect.json
[modify] https://crrev.com/da6a284d86e7f712336c9e19deefdb977be28822/scripts/slave/recipe_modules/auto_bisect_staging/example.expected/basic_win64_bisect.json
[modify] https://crrev.com/da6a284d86e7f712336c9e19deefdb977be28822/scripts/slave/recipe_modules/auto_bisect_staging/example.expected/failed_build.json
[modify] https://crrev.com/da6a284d86e7f712336c9e19deefdb977be28822/scripts/slave/recipe_modules/auto_bisect_staging/example.expected/failed_build_inconclusive_1.json
[modify] https://crrev.com/da6a284d86e7f712336c9e19deefdb977be28822/scripts/slave/recipe_modules/auto_bisect_staging/example.expected/failed_build_inconclusive_11.json
[modify] https://crrev.com/da6a284d86e7f712336c9e19deefdb977be28822/scripts/slave/recipe_modules/auto_bisect_staging/example.expected/failed_buildbucket_get.json
[modify] https://crrev.com/da6a284d86e7f712336c9e19deefdb977be28822/scripts/slave/recipe_modules/auto_bisect_staging/example.expected/gathering_references_no_values.json
[modify] https://crrev.com/da6a284d86e7f712336c9e19deefdb977be28822/scripts/slave/recipe_modules/auto_bisect_staging/example.expected/no_repro.json
[modify] https://crrev.com/da6a284d86e7f712336c9e19deefdb977be28822/scripts/slave/recipe_modules/auto_bisect_staging/example.expected/no_values.json
[modify] https://crrev.com/da6a284d86e7f712336c9e19deefdb977be28822/scripts/slave/recipe_modules/auto_bisect_staging/example.expected/retest_bisect.json
[modify] https://crrev.com/da6a284d86e7f712336c9e19deefdb977be28822/scripts/slave/recipe_modules/auto_bisect_staging/example.expected/return_code.json
[modify] https://crrev.com/da6a284d86e7f712336c9e19deefdb977be28822/scripts/slave/recipe_modules/auto_bisect_staging/example.expected/v8_roll_bisect.json
[modify] https://crrev.com/da6a284d86e7f712336c9e19deefdb977be28822/scripts/slave/recipe_modules/auto_bisect_staging/example.expected/v8_roll_bisect_bis.json
[modify] https://crrev.com/da6a284d86e7f712336c9e19deefdb977be28822/scripts/slave/recipe_modules/auto_bisect_staging/revision_state.py
[modify] https://crrev.com/da6a284d86e7f712336c9e19deefdb977be28822/scripts/slave/recipes/bisection/android_bisect_staging.expected/local_basic_recipe_basic_device.json
[modify] https://crrev.com/da6a284d86e7f712336c9e19deefdb977be28822/scripts/slave/recipes/bisection/android_bisect_staging.expected/local_basic_recipe_disconnected_device.json
[modify] https://crrev.com/da6a284d86e7f712336c9e19deefdb977be28822/scripts/slave/recipes/bisection/android_bisect_staging.expected/local_basic_recipe_failed_device.json

Blocking: 675591
Project Member

Comment 11 by bugdroid1@chromium.org, Dec 20 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/2fea9784e4cd8ce556ab21e24dc83ae07447f34e

commit 2fea9784e4cd8ce556ab21e24dc83ae07447f34e
Author: catapult-deps-roller <catapult-deps-roller@chromium.org>
Date: Tue Dec 20 23:14:25 2016

Roll src/third_party/catapult/ 70f42a7c5..28346a164 (1 commit).

https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/70f42a7c55ca..28346a164856

$ git log 70f42a7c5..28346a164 --date=short --no-merges --format='%ad %ae %s'
2016-12-20 simonhatch Dashboard - Support build failures in bisect output.

BUG=668540, 665439 

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, see:
http://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium#TOC-Failures-due-to-DEPS-rolls

CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel
TBR=catapult-sheriff@chromium.org

Review-Url: https://codereview.chromium.org/2596513003
Cr-Commit-Position: refs/heads/master@{#439915}

[modify] https://crrev.com/2fea9784e4cd8ce556ab21e24dc83ae07447f34e/DEPS

Project Member

Comment 12 by bugdroid1@chromium.org, Dec 22 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build.git/+/3a4b4f28f0e8e866ccacbd8c1327bbf1977fabe8

commit 3a4b4f28f0e8e866ccacbd8c1327bbf1977fabe8
Author: Simon <simonhatch@chromium.org>
Date: Thu Dec 22 01:18:23 2016

Push bisect changes from staging to prod.

These include changes to collapse steps on the build page, packing the json
with extra revision data, and gracefully handling test failures.

https://codereview.chromium.org/2588143002/ must land first.

BUG=668540,  665439 , 672524,  672880 

Change-Id: Id72657a4ff1227ef8e87bd4361db183320bd2236
Reviewed-on: https://chromium-review.googlesource.com/422433
Commit-Queue: Simon Hatch <simonhatch@chromium.org>
Reviewed-by: David Tu <dtu@chromium.org>

[modify] https://crrev.com/3a4b4f28f0e8e866ccacbd8c1327bbf1977fabe8/scripts/slave/recipe_modules/auto_bisect/__init__.py
[modify] https://crrev.com/3a4b4f28f0e8e866ccacbd8c1327bbf1977fabe8/scripts/slave/recipe_modules/auto_bisect/bisector.py
[modify] https://crrev.com/3a4b4f28f0e8e866ccacbd8c1327bbf1977fabe8/scripts/slave/recipe_modules/auto_bisect/example.expected/basic_bisect_other_direction.json
[modify] https://crrev.com/3a4b4f28f0e8e866ccacbd8c1327bbf1977fabe8/scripts/slave/recipe_modules/auto_bisect/example.expected/basic_buildbot_bisect.json
[modify] https://crrev.com/3a4b4f28f0e8e866ccacbd8c1327bbf1977fabe8/scripts/slave/recipe_modules/auto_bisect/example.expected/basic_linux_bisect.json
[modify] https://crrev.com/3a4b4f28f0e8e866ccacbd8c1327bbf1977fabe8/scripts/slave/recipe_modules/auto_bisect/example.expected/basic_mac_bisect.json
[modify] https://crrev.com/3a4b4f28f0e8e866ccacbd8c1327bbf1977fabe8/scripts/slave/recipe_modules/auto_bisect/example.expected/basic_win32_bisect.json
[modify] https://crrev.com/3a4b4f28f0e8e866ccacbd8c1327bbf1977fabe8/scripts/slave/recipe_modules/auto_bisect/example.expected/basic_win64_bisect.json
[modify] https://crrev.com/3a4b4f28f0e8e866ccacbd8c1327bbf1977fabe8/scripts/slave/recipe_modules/auto_bisect/example.expected/failed_build.json
[add] https://crrev.com/3a4b4f28f0e8e866ccacbd8c1327bbf1977fabe8/scripts/slave/recipe_modules/auto_bisect/example.expected/failed_build_inconclusive_1.json
[add] https://crrev.com/3a4b4f28f0e8e866ccacbd8c1327bbf1977fabe8/scripts/slave/recipe_modules/auto_bisect/example.expected/failed_build_inconclusive_11.json
[modify] https://crrev.com/3a4b4f28f0e8e866ccacbd8c1327bbf1977fabe8/scripts/slave/recipe_modules/auto_bisect/example.expected/failed_buildbucket_get.json
[modify] https://crrev.com/3a4b4f28f0e8e866ccacbd8c1327bbf1977fabe8/scripts/slave/recipe_modules/auto_bisect/example.expected/gathering_references_no_values.json
[modify] https://crrev.com/3a4b4f28f0e8e866ccacbd8c1327bbf1977fabe8/scripts/slave/recipe_modules/auto_bisect/example.expected/no_repro.json
[modify] https://crrev.com/3a4b4f28f0e8e866ccacbd8c1327bbf1977fabe8/scripts/slave/recipe_modules/auto_bisect/example.expected/no_values.json
[modify] https://crrev.com/3a4b4f28f0e8e866ccacbd8c1327bbf1977fabe8/scripts/slave/recipe_modules/auto_bisect/example.expected/retest_bisect.json
[modify] https://crrev.com/3a4b4f28f0e8e866ccacbd8c1327bbf1977fabe8/scripts/slave/recipe_modules/auto_bisect/example.expected/return_code.json
[modify] https://crrev.com/3a4b4f28f0e8e866ccacbd8c1327bbf1977fabe8/scripts/slave/recipe_modules/auto_bisect/example.expected/return_code_fail.json
[modify] https://crrev.com/3a4b4f28f0e8e866ccacbd8c1327bbf1977fabe8/scripts/slave/recipe_modules/auto_bisect/example.expected/v8_roll_bisect.json
[modify] https://crrev.com/3a4b4f28f0e8e866ccacbd8c1327bbf1977fabe8/scripts/slave/recipe_modules/auto_bisect/example.expected/v8_roll_bisect_bis.json
[modify] https://crrev.com/3a4b4f28f0e8e866ccacbd8c1327bbf1977fabe8/scripts/slave/recipe_modules/auto_bisect/example.py
[modify] https://crrev.com/3a4b4f28f0e8e866ccacbd8c1327bbf1977fabe8/scripts/slave/recipe_modules/auto_bisect/local_bisect.py
[modify] https://crrev.com/3a4b4f28f0e8e866ccacbd8c1327bbf1977fabe8/scripts/slave/recipe_modules/auto_bisect/revision_state.py
[modify] https://crrev.com/3a4b4f28f0e8e866ccacbd8c1327bbf1977fabe8/scripts/slave/recipes/bisection/android_bisect.expected/local_basic_recipe_basic_device.json
[modify] https://crrev.com/3a4b4f28f0e8e866ccacbd8c1327bbf1977fabe8/scripts/slave/recipes/bisection/android_bisect.expected/local_basic_recipe_disconnected_device.json
[modify] https://crrev.com/3a4b4f28f0e8e866ccacbd8c1327bbf1977fabe8/scripts/slave/recipes/bisection/android_bisect.expected/local_basic_recipe_failed_device.json

Project Member

Comment 13 by bugdroid1@chromium.org, Jan 6 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/64657ddad6ba9f1c432aee54c0282b1c7aca1e73

commit 64657ddad6ba9f1c432aee54c0282b1c7aca1e73
Author: catapult-deps-roller <catapult-deps-roller@chromium.org>
Date: Fri Jan 06 17:50:13 2017

Roll src/third_party/catapult/ 9e6944a8f..fa1926f93 (23 commits).

https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/9e6944a8f310..fa1926f937dc

$ git log 9e6944a8f..fa1926f93 --date=short --no-merges --format='%ad %ae %s'
2017-01-06 sullivan Update perf dashboard tooltips to use logdog logs and show status pages.
2017-01-06 charliea Undo unnecessary disables from http://crrev.com/2236493003/
2017-01-05 simonhatch Dashboard - Add some extra logging info for failed auto-bisect.
2017-01-05 nednguyen Revert of Asserting telemetry story display name length for filename creation. (patchset #3 id:40001 of https://codereview.chromium.org/2488743006/ )
2017-01-05 dtu [pinpoint] Fix error message for missing parameters in /isolated.
2017-01-05 benjhayden Update UserModel to an ES6 class.
2017-01-05 benjhayden Configure chai to include stack traces.
2017-01-05 achuith Revert retries for DNS failures.
2017-01-05 benjhayden Finish renaming InteractionRecords to UserExpectations.
2017-01-05 sullivan Add a cached last_row_timestamp to StoppageAlert.
2017-01-05 eakuefner [Dashboard] Factor business logic in AddPointHandler out into AddData method
2017-01-05 simonhatch Dashboard - Bisect output should include percent change if possible.
2017-01-05 jessimb Changing group reports to use an id for multiple keys.
2017-01-05 charliea Replace iterItems() with Object.entries() in a few files
2017-01-05 charliea Use individual power sample for story:power metric
2017-01-05 eakuefner [Dashboard] Eliminate redundancy in AddPointHandler.post
2017-01-05 mikecase Add logcat markers to begin and end of each story.
2017-01-05 charliea Migrate more files from iterItems() to Object methods
2017-01-05 simonhatch Dashboard - Refactor output and clarify bisect failures.
2017-01-04 charliea [catapult android trybot] Make Telemetry tests run on Android
2017-01-04 eyaich Asserting telemetry story display name length for filename creation.
2017-01-04 achuith Browser startup timeout 4 min on ChromeOS, 1 min elsewhere.
2017-01-04 aiolos Unit refactor cl 1: Rename UnitScale to UnitPrefixScale.

BUG= 662941 , 546625 , 665439 ,678282, 665439 , 662941 ,676742

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, see:
http://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium#TOC-Failures-due-to-DEPS-rolls

CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel
TBR=catapult-sheriff@chromium.org

Review-Url: https://codereview.chromium.org/2613243002
Cr-Commit-Position: refs/heads/master@{#441970}

[modify] https://crrev.com/64657ddad6ba9f1c432aee54c0282b1c7aca1e73/DEPS

Status: Fixed (was: Started)
Gonna close this, the big output refactor is live. Can file new issues as they're found.
Components: Speed>Bisection

Sign in to add a comment