New issue
Advanced search Search tips

Issue 837087 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Apr 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Bug-Regression



Sign in to add a comment

Update recipe engine service_account auth token generation to swarming reality

Project Member Reported by tandrii@chromium.org, Apr 26 2018

Issue description

Lifetimes of only below 5 minutes are guaranteed. Current default is 10, which results in frequent failures to procure token.

Let's remove lifetime parameter completely, encouraging tools to call luci-auth directly for all their retries whenever necessary.
 
Labels: -Restrict-View-Google
CLs:
  https://crrev.com/c/1029370 in recipe engine
  https://crrev.com/c/1029455 in build
Sadly, also this is required: https://crrev.com/i/615209 
Project Member

Comment 4 by bugdroid1@chromium.org, Apr 26 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/066a4878aa8abfcd8575f62447ec9aece2d6a189

commit 066a4878aa8abfcd8575f62447ec9aece2d6a189
Author: Andrii Shyshkalov <tandrii@chromium.org>
Date: Thu Apr 26 02:07:49 2018

recipes: temp fix to ensure auth token can be obtained on LUCI bots.

TBR=vadimsh@chromium.org

Bug:  837087 
Change-Id: I1547afe3591345c1b1138c75fa1a5ac32e794cdc
Reviewed-on: https://chromium-review.googlesource.com/1029458
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>

[modify] https://crrev.com/066a4878aa8abfcd8575f62447ec9aece2d6a189/recipes/recipes/recipe_roll_tryjob.expected/diff_train_fail.json
[modify] https://crrev.com/066a4878aa8abfcd8575f62447ec9aece2d6a189/recipes/recipes/recipe_roll_tryjob.expected/diff_train_fail_ack_engine_checkout.json
[modify] https://crrev.com/066a4878aa8abfcd8575f62447ec9aece2d6a189/recipes/recipes/recipe_roll_tryjob.expected/with_patch_train_fail.json
[modify] https://crrev.com/066a4878aa8abfcd8575f62447ec9aece2d6a189/recipes/recipes/recipe_roll_tryjob.expected/bypass.json
[modify] https://crrev.com/066a4878aa8abfcd8575f62447ec9aece2d6a189/recipes/recipes/recipe_roll_tryjob.expected/without_patch_test_fail.json
[modify] https://crrev.com/066a4878aa8abfcd8575f62447ec9aece2d6a189/recipes/recipes/recipe_roll_tryjob.expected/diff_test_fail.json
[modify] https://crrev.com/066a4878aa8abfcd8575f62447ec9aece2d6a189/recipes/recipes/recipe_roll_tryjob.expected/diff_test_fail_ack.json
[modify] https://crrev.com/066a4878aa8abfcd8575f62447ec9aece2d6a189/recipes/recipes/recipe_roll_tryjob.expected/without_patch_train_fail.json
[modify] https://crrev.com/066a4878aa8abfcd8575f62447ec9aece2d6a189/recipes/recipes/recipe_roll_tryjob.expected/with_patch_test_fail.json
[modify] https://crrev.com/066a4878aa8abfcd8575f62447ec9aece2d6a189/recipes/recipes/recipe_roll_tryjob.py
[modify] https://crrev.com/066a4878aa8abfcd8575f62447ec9aece2d6a189/recipes/recipes/recipe_roll_tryjob.expected/basic.json
[modify] https://crrev.com/066a4878aa8abfcd8575f62447ec9aece2d6a189/recipes/recipes/recipe_roll_tryjob.expected/diff_train_fail_ack.json

Status: Started (was: Assigned)
OK, so the order of CLs to land:
(1) land clank https://crrev.com/i/615209 
  and make let it stay in prod for a day.
(2) land https://crrev.com/c/1029455 in build
    and revert temp workaround above by landing cl https://crrev.com/c/1029610
(3) finally land https://crrev.com/c/1029370
Project Member

Comment 6 by bugdroid1@chromium.org, Apr 26 2018

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chrome/tools/build_limited/scripts/slave/+/350347e15ee7badc76b36b392bd115b03abae8c6

commit 350347e15ee7badc76b36b392bd115b03abae8c6
Author: Andrii Shyshkalov <tandrii@chromium.org>
Date: Thu Apr 26 02:21:13 2018

The code in vl (1) https://crrev.com/i/615209  appears dead, since builders running this recipe don't have the step.
Project Member

Comment 9 by bugdroid1@chromium.org, Apr 26 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/2c06700006b955606ad47926fc55ad3f060eeefa

commit 2c06700006b955606ad47926fc55ad3f060eeefa
Author: Andrii Shyshkalov <tandrii@chromium.org>
Date: Thu Apr 26 05:39:39 2018

Revert "recipes: temp fix to ensure auth token can be obtained on LUCI bots."

This reverts commit 066a4878aa8abfcd8575f62447ec9aece2d6a189.

Reason for revert: will be superseded by 3 other CLs about to be landed.

Original change's description:
> recipes: temp fix to ensure auth token can be obtained on LUCI bots.
> 
> TBR=vadimsh@chromium.org
> 
> Bug:  837087 
> Change-Id: I1547afe3591345c1b1138c75fa1a5ac32e794cdc
> Reviewed-on: https://chromium-review.googlesource.com/1029458
> Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
> Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>

TBR=vadimsh@chromium.org,tandrii@chromium.org

Bug:  837087 
Change-Id: I9b5f9e7f21147af50135b865afc9e981933613ce
Reviewed-on: https://chromium-review.googlesource.com/1029610
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>

[modify] https://crrev.com/2c06700006b955606ad47926fc55ad3f060eeefa/recipes/recipes/recipe_roll_tryjob.expected/diff_train_fail.json
[modify] https://crrev.com/2c06700006b955606ad47926fc55ad3f060eeefa/recipes/recipes/recipe_roll_tryjob.expected/diff_train_fail_ack_engine_checkout.json
[modify] https://crrev.com/2c06700006b955606ad47926fc55ad3f060eeefa/recipes/recipes/recipe_roll_tryjob.expected/with_patch_train_fail.json
[modify] https://crrev.com/2c06700006b955606ad47926fc55ad3f060eeefa/recipes/recipes/recipe_roll_tryjob.expected/bypass.json
[modify] https://crrev.com/2c06700006b955606ad47926fc55ad3f060eeefa/recipes/recipes/recipe_roll_tryjob.expected/without_patch_test_fail.json
[modify] https://crrev.com/2c06700006b955606ad47926fc55ad3f060eeefa/recipes/recipes/recipe_roll_tryjob.expected/diff_test_fail.json
[modify] https://crrev.com/2c06700006b955606ad47926fc55ad3f060eeefa/recipes/recipes/recipe_roll_tryjob.expected/diff_test_fail_ack.json
[modify] https://crrev.com/2c06700006b955606ad47926fc55ad3f060eeefa/recipes/recipes/recipe_roll_tryjob.expected/without_patch_train_fail.json
[modify] https://crrev.com/2c06700006b955606ad47926fc55ad3f060eeefa/recipes/recipes/recipe_roll_tryjob.expected/with_patch_test_fail.json
[modify] https://crrev.com/2c06700006b955606ad47926fc55ad3f060eeefa/recipes/recipes/recipe_roll_tryjob.py
[modify] https://crrev.com/2c06700006b955606ad47926fc55ad3f060eeefa/recipes/recipes/recipe_roll_tryjob.expected/basic.json
[modify] https://crrev.com/2c06700006b955606ad47926fc55ad3f060eeefa/recipes/recipes/recipe_roll_tryjob.expected/diff_train_fail_ack.json

Project Member

Comment 10 by bugdroid1@chromium.org, Apr 26 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/recipes-py/+/0a82af179bce717e9be1240286497312309487d0

commit 0a82af179bce717e9be1240286497312309487d0
Author: Andrii Shyshkalov <tandrii@chromium.org>
Date: Thu Apr 26 05:51:42 2018

service_account: remove ability to vary lifetime and set it to 180s.

It was 10 minutes before, which longer than LUCI_CONTEXT on swarming bot
can guarantee to mint (5 minute, currently). Due to delays in RPCs,
slowness of Python, etc, 3 minutes will likely be available by the time
the token is given back to recipe code.

Also, make token procurement an infra step.

TBR=estaab@chromium.org

Bug:  837087 
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Change-Id: Ia9477c12c2525012d2210afbb6782df837ea770c
Reviewed-on: https://chromium-review.googlesource.com/1029370
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>

[modify] https://crrev.com/0a82af179bce717e9be1240286497312309487d0/recipe_modules/service_account/examples/full.expected/windows.json
[modify] https://crrev.com/0a82af179bce717e9be1240286497312309487d0/recipe_modules/service_account/examples/full.expected/json_key.json
[modify] https://crrev.com/0a82af179bce717e9be1240286497312309487d0/recipe_modules/service_account/api.py
[modify] https://crrev.com/0a82af179bce717e9be1240286497312309487d0/README.recipes.md
[modify] https://crrev.com/0a82af179bce717e9be1240286497312309487d0/recipe_modules/service_account/examples/full.expected/default.json
[rename] https://crrev.com/0a82af179bce717e9be1240286497312309487d0/recipe_modules/service_account/examples/full.expected/custom_scopes.json
[modify] https://crrev.com/0a82af179bce717e9be1240286497312309487d0/recipe_modules/service_account/examples/full.py
[modify] https://crrev.com/0a82af179bce717e9be1240286497312309487d0/recipe_modules/service_account/examples/full.expected/no_authutil.json

Status: Fixed (was: Started)
all rolls completed.
Project Member

Comment 12 by bugdroid1@chromium.org, May 4 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/depot_tools/+/142a92ce8b834a412c16c0451ed8d434ea323e58

commit 142a92ce8b834a412c16c0451ed8d434ea323e58
Author: Andrii Shyshkalov <tandrii@chromium.org>
Date: Fri May 04 19:26:56 2018

auth: request refresh tokens with expiry <3 minutes.

Due to swarming, 5 minutes is hard upper limit,
and by the time token arrives to the bot it may be <5 minutes,
like here
https://logs.chromium.org/v/?s=chromium%2Fbuildbucket%2Fcr-buildbucket.appspot.com%2F8947432403218547136%2F%2B%2Fsteps%2Fpresubmit%2F0%2Fstdout

R=vadimsh@chromium.org

Bug:  837087 
Change-Id: I98fbe3f0f85c9f62bc984f439614c3f88a16ed06
Reviewed-on: https://chromium-review.googlesource.com/1044779
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>

[modify] https://crrev.com/142a92ce8b834a412c16c0451ed8d434ea323e58/auth.py

Project Member

Comment 13 by bugdroid1@chromium.org, May 4 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/afd725a7192f4340b59faeb543aaeaa2f3ba06d2

commit afd725a7192f4340b59faeb543aaeaa2f3ba06d2
Author: depot-tools-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com <depot-tools-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Date: Fri May 04 21:11:34 2018

Roll src/third_party/depot_tools/ 157591213..142a92ce8 (1 commit)

https://chromium.googlesource.com/chromium/tools/depot_tools.git/+log/15759121361d..142a92ce8b83

$ git log 157591213..142a92ce8 --date=short --no-merges --format='%ad %ae %s'
2018-05-04 tandrii auth: request refresh tokens with expiry <3 minutes.

Created with:
  roll-dep src/third_party/depot_tools
BUG= chromium:837087 


The AutoRoll server is located here: https://depot-tools-chromium-roll.skia.org

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.


TBR=agable@chromium.org

Change-Id: Iecb1a28f1e5ffa180639739105193b8f4e5e26d5
Reviewed-on: https://chromium-review.googlesource.com/1044948
Commit-Queue: depot-tools-chromium-autoroll <depot-tools-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Reviewed-by: depot-tools-chromium-autoroll <depot-tools-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#556189}
[modify] https://crrev.com/afd725a7192f4340b59faeb543aaeaa2f3ba06d2/DEPS

Sign in to add a comment