New issue
Advanced search Search tips

Issue 870630 link

Starred by 1 user

Issue metadata

Status: Verified
Owner:
Closed: Aug 3
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Bug-Regression



Sign in to add a comment

V8 auto-bisect gets 404 from build-extract

Project Member Reported by machenb...@chromium.org, Aug 3

Issue description

See:
https://logs.chromium.org/logs/v8/buildbucket/cr-buildbucket.appspot.com/8939229168628760800/+/steps/Bisect/0/steps/Fetch_changes/0/stdout

Maybe there's some permission needed with luci for a service account? Or maybe the urls have changed due to luci migration?
 
Actually, maybe CBE might be gone? I guess there should be another way to get the changes on luci or milo now?
Cc: iannucci@chromium.org
Indeed. Looks like the app was deleted: https://pantheon.corp.google.com/appengine/versions?project=chrome-build-extract. Last time I checked it was just a proxy to luci-milo.appspot.com, so perhaps you can use that directly, i.e. instead of a GET request to http://chrome-build-extract.appspot.com:80/p/client.v8/builders/V8%20Win64%20-%20msvc/builds/3967?json=1 send a POST request to https://luci-milo.appspot.com/prpc/milo.Buildbot/GetBuildbotBuildJSON with the following payload

  {"master":"client.v8", "builder":"V8 Win64 - msvc", "buildNum":3967}

and make sure to use Authorization header containing bearer token of the task service account.

Robbie, is there a recipe module that we can use to issue such requests easier, i.e. one that will pre-populate the header?
Actually I am not sure if Authorization header is required. The RPC explorer [1] requires me to sign-in before I can issue the request, but I can still get the data via a normal curl request:

  curl --request POST -H 'Content-Type: application/json' --verbose \
       --data '{"master":"client.v8", "builder":"V8 Win64 - msvc", "buildNum":3967}' \
       https://luci-milo.appspot.com/prpc/milo.Buildbot/GetBuildbotBuildJSON

OTH, I am not quite sure what it returns. Looks like JSON, but also has some weird binary bytes in the beginning and the end. The response header Content-Type says 'application/rpc' - not sure how to parse that. Is there a chance that we have a recipe module for issuing/parsing PRPC requests?

[1]: https://luci-milo.appspot.com/rpcexplorer/services/milo.Buildbot/GetBuildbotBuildJSON?request={%20%20%20%20%22master%22:%20%22client.v8%22,%20%20%20%20%22builder%22:%20%22V8%20Win64%20-%20msvc%22,%20%20%20%20%22buildNum%22:%203967}
After a little more codesearching, I've found that schedule recipe module directly calls prcp command directly [2]. Looks like it's part of PRPC, but not included into CIPD recipe bundle. How does it work?

I've also tried using it locally:

  echo '{"master":"client.v8", "builder":"V8 Win64 - msvc", "buildNum":3967}' | prpc call -format=json luci-milo.appspot.com milo.Buildbot.GetBuildbotBuildJSON

This returns a JSON dict containing base64-encoded data in data key. Decoding that gives another JSON that contains information about the build in a format similar to one that was returned by chrome-build-extact. It would nice to have it all wrapped into a prpc recipe module.

[2]: https://cs.chromium.org/chromium/infra/recipes-py/recipe_modules/scheduler/api.py?l=158&rcl=1f25d6536d9972411f4540f2e883e1560ddeb0ad
Cc: -serg...@chromium.org
Owner: serg...@chromium.org
Status: Started (was: Untriaged)
Andrii just explained to me that PRPC is available on all bots, provided by LUCI as base software.
Project Member

Comment 8 by bugdroid1@chromium.org, Aug 3

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/28343d17006ba57d8d3419daa1ad53f6865769ba

commit 28343d17006ba57d8d3419daa1ad53f6865769ba
Author: Sergiy Byelozyorov <sergiyb@chromium.org>
Date: Fri Aug 03 15:22:21 2018

[V8] Switch from deprecated CBE to Milo API

R=iannucci@chromium.org, machenbach@chromium.org

Bug:  870630 
Change-Id: Ica52e6bf3f8378a247e902abccb9bb9c94aa0185
Reviewed-on: https://chromium-review.googlesource.com/1162232
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>

[modify] https://crrev.com/28343d17006ba57d8d3419daa1ad53f6865769ba/scripts/slave/recipe_modules/v8/api.py
[modify] https://crrev.com/28343d17006ba57d8d3419daa1ad53f6865769ba/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux64___custom_snapshot___debug_slim_bisect_tester_swarming.json
[modify] https://crrev.com/28343d17006ba57d8d3419daa1ad53f6865769ba/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux64___internal_snapshot_test_failures.json
[modify] https://crrev.com/28343d17006ba57d8d3419daa1ad53f6865769ba/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux___predictable_bisect_one_change.json
[modify] https://crrev.com/28343d17006ba57d8d3419daa1ad53f6865769ba/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux64_bisect_tester_swarming.json
[modify] https://crrev.com/28343d17006ba57d8d3419daa1ad53f6865769ba/scripts/slave/README.recipes.md
[modify] https://crrev.com/28343d17006ba57d8d3419daa1ad53f6865769ba/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux___predictable_bisect_recurring_failure.json
[modify] https://crrev.com/28343d17006ba57d8d3419daa1ad53f6865769ba/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux___shared_bisect_swarming.json
[modify] https://crrev.com/28343d17006ba57d8d3419daa1ad53f6865769ba/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux___predictable_bisect.json

Status: Fixed (was: Started)
Status: Started (was: Fixed)
Looks like Milo returns list of changes from newest to oldest, whereas CBE returned them in reverse order: https://ci.chromium.org/swarming/task/3f192ad298539b10.

CL: https://crrev.com/c/1162321
Thanks for working on this. I just re-triggered the failing build with led:
https://ci.chromium.org/swarming/task/3f1963037f56b910?server=chromium-swarm.appspot.com

Will probably fail too, since it was before your new CL.
This one includes my last CL, so should pass: https://ci.chromium.org/swarming/task/3f19608f9d4c7010
Project Member

Comment 13 by bugdroid1@chromium.org, Aug 3

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/b5d98f6e59f11cb0c7958ac12105d91d40ef6fab

commit b5d98f6e59f11cb0c7958ac12105d91d40ef6fab
Author: Sergiy Byelozyorov <sergiyb@chromium.org>
Date: Fri Aug 03 16:32:17 2018

[V8] Sort changes by timestamp to ensure correct order

R=machebach@chromium.org

Bug:  870630 
Change-Id: Ic3b49f6028a2d55569df68ed71e53dccbceb2ead
Reviewed-on: https://chromium-review.googlesource.com/1162321
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>

[modify] https://crrev.com/b5d98f6e59f11cb0c7958ac12105d91d40ef6fab/scripts/slave/recipe_modules/v8/api.py
[modify] https://crrev.com/b5d98f6e59f11cb0c7958ac12105d91d40ef6fab/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux64___custom_snapshot___debug_slim_bisect_tester_swarming.json
[modify] https://crrev.com/b5d98f6e59f11cb0c7958ac12105d91d40ef6fab/scripts/slave/recipe_modules/v8/test_api.py
[modify] https://crrev.com/b5d98f6e59f11cb0c7958ac12105d91d40ef6fab/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux64___internal_snapshot_test_failures.json
[modify] https://crrev.com/b5d98f6e59f11cb0c7958ac12105d91d40ef6fab/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux___predictable_bisect_one_change.json
[modify] https://crrev.com/b5d98f6e59f11cb0c7958ac12105d91d40ef6fab/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux64_bisect_tester_swarming.json
[modify] https://crrev.com/b5d98f6e59f11cb0c7958ac12105d91d40ef6fab/scripts/slave/README.recipes.md
[modify] https://crrev.com/b5d98f6e59f11cb0c7958ac12105d91d40ef6fab/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux___predictable_bisect_recurring_failure.json
[modify] https://crrev.com/b5d98f6e59f11cb0c7958ac12105d91d40ef6fab/scripts/slave/recipes/v8.py
[modify] https://crrev.com/b5d98f6e59f11cb0c7958ac12105d91d40ef6fab/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux___shared_bisect_swarming.json
[modify] https://crrev.com/b5d98f6e59f11cb0c7958ac12105d91d40ef6fab/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux___predictable_bisect_override_changes.json
[modify] https://crrev.com/b5d98f6e59f11cb0c7958ac12105d91d40ef6fab/scripts/slave/recipes/v8.expected/full_client_v8_V8_Linux___predictable_bisect.json

Status: Fixed (was: Started)
Cc: -iannucci@chromium.org
Thanks. Rerunning the task to verify:
https://ci.chromium.org/swarming/task/3f19e550dc3c3010?server=chromium-swarm.appspot.com
Status: Verified (was: Fixed)
Bisect results of last run look good. Thanks for fixing!

Sign in to add a comment