New issue
Advanced search Search tips

Issue 676416 link

Starred by 2 users

Issue metadata

Status: Fixed
Owner:
Closed: Dec 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 0
Type: Bug


Sign in to add a comment

Bisect - Failing to post results to dashboard

Project Member Reported by simonhatch@chromium.org, Dec 21 2016

Issue description

Seeing results like this:

https://build.chromium.org/p/tryserver.chromium.perf/builders/win_perf_bisect/builds/7084

{
  "status_code": 400, 
  "text": "Invalid JSON string.\nrequest_id:585a6aa800ff00ff86d626f74c510001737e6368726f6d65706572660001636c65616e2d73696d6f6e68617463682d376361333539353500010109\n"
}

 
Blocking: 672523 660245 674457
Blocking: 673285
Blocking: 670095
Adding some logging to try and track this down, it's happening a lot, but seems to predate a lot of the json changes.
Project Member

Comment 5 by bugdroid1@chromium.org, Dec 22 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build.git/+/112dae798e847082f98654d247c36e42dd342ee4

commit 112dae798e847082f98654d247c36e42dd342ee4
Author: Simon <simonhatch@chromium.org>
Date: Thu Dec 22 18:37:16 2016

Bisect - Add posted json to debug step.

TBR=dtu@chromium.org
BUG= 676416 

Change-Id: I91e274e68c606e8a0eb1e13b2806c7e51e9ee614
Reviewed-on: https://chromium-review.googlesource.com/422850
Reviewed-by: Simon Hatch <simonhatch@chromium.org>
Commit-Queue: Simon Hatch <simonhatch@chromium.org>

[modify] https://crrev.com/112dae798e847082f98654d247c36e42dd342ee4/scripts/slave/recipe_modules/auto_bisect_staging/bisector.py
[modify] https://crrev.com/112dae798e847082f98654d247c36e42dd342ee4/scripts/slave/recipe_modules/auto_bisect_staging/example.expected/basic_bisect_other_direction.json
[modify] https://crrev.com/112dae798e847082f98654d247c36e42dd342ee4/scripts/slave/recipe_modules/auto_bisect_staging/example.expected/basic_buildbot_bisect.json
[modify] https://crrev.com/112dae798e847082f98654d247c36e42dd342ee4/scripts/slave/recipe_modules/auto_bisect_staging/example.expected/basic_linux_bisect.json
[modify] https://crrev.com/112dae798e847082f98654d247c36e42dd342ee4/scripts/slave/recipe_modules/auto_bisect_staging/example.expected/basic_mac_bisect.json
[modify] https://crrev.com/112dae798e847082f98654d247c36e42dd342ee4/scripts/slave/recipe_modules/auto_bisect_staging/example.expected/basic_win32_bisect.json
[modify] https://crrev.com/112dae798e847082f98654d247c36e42dd342ee4/scripts/slave/recipe_modules/auto_bisect_staging/example.expected/basic_win64_bisect.json
[modify] https://crrev.com/112dae798e847082f98654d247c36e42dd342ee4/scripts/slave/recipe_modules/auto_bisect_staging/example.expected/retest_bisect.json
[modify] https://crrev.com/112dae798e847082f98654d247c36e42dd342ee4/scripts/slave/recipe_modules/auto_bisect_staging/example.expected/return_code.json
[modify] https://crrev.com/112dae798e847082f98654d247c36e42dd342ee4/scripts/slave/recipe_modules/auto_bisect_staging/example.expected/v8_roll_bisect.json
[modify] https://crrev.com/112dae798e847082f98654d247c36e42dd342ee4/scripts/slave/recipe_modules/auto_bisect_staging/example.expected/v8_roll_bisect_bis.json
[modify] https://crrev.com/112dae798e847082f98654d247c36e42dd342ee4/scripts/slave/recipes/bisection/android_bisect_staging.expected/local_basic_recipe_basic_device.json
[modify] https://crrev.com/112dae798e847082f98654d247c36e42dd342ee4/scripts/slave/recipes/bisection/android_bisect_staging.expected/local_basic_recipe_disconnected_device.json

Project Member

Comment 6 by bugdroid1@chromium.org, Dec 22 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build.git/+/c421ff6420f3e05dad803eb7611443c1e7c5390f

commit c421ff6420f3e05dad803eb7611443c1e7c5390f
Author: Simon <simonhatch@chromium.org>
Date: Thu Dec 22 19:40:12 2016

Bisect - Add more debug for posted json.

TBR=dtu@chromium.org
BUG= 676416 

Change-Id: I66957ff8777c54ff16d4f33d6b91fd376851c591
Reviewed-on: https://chromium-review.googlesource.com/423329
Reviewed-by: Simon Hatch <simonhatch@chromium.org>
Commit-Queue: Simon Hatch <simonhatch@chromium.org>

[modify] https://crrev.com/c421ff6420f3e05dad803eb7611443c1e7c5390f/scripts/slave/recipe_modules/auto_bisect_staging/bisector.py
[modify] https://crrev.com/c421ff6420f3e05dad803eb7611443c1e7c5390f/scripts/slave/recipe_modules/auto_bisect_staging/example.expected/basic_bisect_other_direction.json
[modify] https://crrev.com/c421ff6420f3e05dad803eb7611443c1e7c5390f/scripts/slave/recipe_modules/auto_bisect_staging/example.expected/basic_buildbot_bisect.json
[modify] https://crrev.com/c421ff6420f3e05dad803eb7611443c1e7c5390f/scripts/slave/recipe_modules/auto_bisect_staging/example.expected/basic_linux_bisect.json
[modify] https://crrev.com/c421ff6420f3e05dad803eb7611443c1e7c5390f/scripts/slave/recipe_modules/auto_bisect_staging/example.expected/basic_mac_bisect.json
[modify] https://crrev.com/c421ff6420f3e05dad803eb7611443c1e7c5390f/scripts/slave/recipe_modules/auto_bisect_staging/example.expected/basic_win32_bisect.json
[modify] https://crrev.com/c421ff6420f3e05dad803eb7611443c1e7c5390f/scripts/slave/recipe_modules/auto_bisect_staging/example.expected/basic_win64_bisect.json
[modify] https://crrev.com/c421ff6420f3e05dad803eb7611443c1e7c5390f/scripts/slave/recipe_modules/auto_bisect_staging/example.expected/failed_build.json
[modify] https://crrev.com/c421ff6420f3e05dad803eb7611443c1e7c5390f/scripts/slave/recipe_modules/auto_bisect_staging/example.expected/failed_build_inconclusive_1.json
[modify] https://crrev.com/c421ff6420f3e05dad803eb7611443c1e7c5390f/scripts/slave/recipe_modules/auto_bisect_staging/example.expected/failed_build_inconclusive_11.json
[modify] https://crrev.com/c421ff6420f3e05dad803eb7611443c1e7c5390f/scripts/slave/recipe_modules/auto_bisect_staging/example.expected/failed_buildbucket_get.json
[modify] https://crrev.com/c421ff6420f3e05dad803eb7611443c1e7c5390f/scripts/slave/recipe_modules/auto_bisect_staging/example.expected/gathering_references_no_values.json
[modify] https://crrev.com/c421ff6420f3e05dad803eb7611443c1e7c5390f/scripts/slave/recipe_modules/auto_bisect_staging/example.expected/no_repro.json
[modify] https://crrev.com/c421ff6420f3e05dad803eb7611443c1e7c5390f/scripts/slave/recipe_modules/auto_bisect_staging/example.expected/no_values.json
[modify] https://crrev.com/c421ff6420f3e05dad803eb7611443c1e7c5390f/scripts/slave/recipe_modules/auto_bisect_staging/example.expected/retest_bisect.json
[modify] https://crrev.com/c421ff6420f3e05dad803eb7611443c1e7c5390f/scripts/slave/recipe_modules/auto_bisect_staging/example.expected/return_code.json
[modify] https://crrev.com/c421ff6420f3e05dad803eb7611443c1e7c5390f/scripts/slave/recipe_modules/auto_bisect_staging/example.expected/return_code_fail.json
[modify] https://crrev.com/c421ff6420f3e05dad803eb7611443c1e7c5390f/scripts/slave/recipe_modules/auto_bisect_staging/example.expected/v8_roll_bisect.json
[modify] https://crrev.com/c421ff6420f3e05dad803eb7611443c1e7c5390f/scripts/slave/recipe_modules/auto_bisect_staging/example.expected/v8_roll_bisect_bis.json
[modify] https://crrev.com/c421ff6420f3e05dad803eb7611443c1e7c5390f/scripts/slave/recipes/bisection/android_bisect_staging.expected/local_basic_recipe_basic_device.json
[modify] https://crrev.com/c421ff6420f3e05dad803eb7611443c1e7c5390f/scripts/slave/recipes/bisection/android_bisect_staging.expected/local_basic_recipe_disconnected_device.json

Owner: dtu@chromium.org
Trying to track this down by bisecting using the waterfall, so far this is what I've narrowed down to:


BROKEN 359ac007c8fa9be6536223d8c7427c22923db4cc Roll recipe dependencies (trivial).
https://build.chromium.org/p/tryserver.chromium.perf/builders/android_fyi_perf_bisect/builds/1375

a4dc1bd4c41dd72b2fa83bed1047db1072b30554 Move linux_chromium_tsan_rel_ng to main chromium.memory waterfall.
7e5e90522de65e7ca274c8dd7e24f4c2622b25ac Modify the isolate recipe to support exparchive isolates.

BROKEN ef4c1516d5c8c6624eae58fc6a6e07e34679f8f4 Reland of: chromium.perf: Switch 64 bit android bots to chromium recipe
https://build.chromium.org/p/tryserver.chromium.perf/builders/winx64_10_perf_bisect/builds/818

ef6ecbf86db2d433cb8d0eac676ff228bc89af35 [Android] Change extension to uploaded test trace file to html.

BROKEN dfe63d162a2d2f4653638281174cb9b458f6bb2d [Android] Switch to catapult's version of provision_devices. (RELAND)
https://build.chromium.org/p/tryserver.chromium.perf/builders/linux_perf_bisect/builds/6953

768ffedd918411676b0a930450c97d588367949e Add iOS to gatekeeper
cd824dbcd9d77f480ace76286661b39676fdbb15 [perf] [Reland] perf_dashboard module refactor.
32605e50f627c220e9ff3315e5dc19beae9f06ed Revert "Add Layout Tests and multiprocess CTS tests to WebView N bot."

WORKING df3cf941f58d305637209df35f378401e7bd8a4a Distinguish between missing vs large gtest output.
https://build.chromium.org/p/tryserver.chromium.perf/builders/winx64ati_perf_bisect

WORKING 684c1ca07767d7ad7d5c8dec925745299667a39a Compile test targets plus configured additional targets on iOS
https://build.chromium.org/p/tryserver.chromium.perf/builders/android_fyi_perf_bisect/builds/1374


Dave, can you take a look? Looks like you did a bunch of refactoring work (https://chromium-review.googlesource.com/c/421254/) around post_json which seems like it might be the cause.
Project Member

Comment 8 by bugdroid1@chromium.org, Dec 22 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/ec95940936ce43e19b843b15205dc88be4215e56

commit ec95940936ce43e19b843b15205dc88be4215e56
Author: catapult-deps-roller <catapult-deps-roller@chromium.org>
Date: Thu Dec 22 19:53:28 2016

Roll src/third_party/catapult/ 489a5bc40..810f05af2 (1 commit).

https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/489a5bc40e98..810f05af27d2

$ git log 489a5bc40..810f05af2 --date=short --no-merges --format='%ad %ae %s'
2016-12-22 simonhatch Dashboard - Move log statement in /post_bisect_results.

BUG= 676416 

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, see:
http://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium#TOC-Failures-due-to-DEPS-rolls

CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel
TBR=catapult-sheriff@chromium.org

Review-Url: https://codereview.chromium.org/2600553002
Cr-Commit-Position: refs/heads/master@{#440478}

[modify] https://crrev.com/ec95940936ce43e19b843b15205dc88be4215e56/DEPS

Labels: -Pri-3 Pri-1
Issue 677498 has been merged into this issue.
Cc: sullivan@chromium.org simonhatch@chromium.org
Labels: -Pri-1 Pri-0
Status: Assigned (was: Untriaged)
Raising prio as this fails on all internal V8 testing. See duplicate issue 677498.

Comment 14 by dtu@chromium.org, Dec 29 2016

Status: Fixed (was: Assigned)
Components: Speed>Bisection

Sign in to add a comment