WPT importer bot failing because of no GitHub credentials |
||||||
Issue descriptionslave271-c1 is a new wpt-importer builder (as of https://chromium-review.googlesource.com/#/c/chromium/tools/build/+/1066183). It's failing to import now since it seems to lack GitHub credentials: e.g for https://ci.chromium.org/buildbot/chromium.infra.cron/wpt-importer/17893, the error is: 2018-05-18 12:03:34,215 - Credentials JSON file not found at /creds/json/wpt-import.json. 2018-05-18 12:03:34,215 - You have not set your GitHub credentials. This script may fail with a network error when making an API request to GitHub.
,
May 18 2018
,
May 23 2018
,
May 23 2018
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/48648cd880147ed373dd85db1c9c8564ac8dd913 commit 48648cd880147ed373dd85db1c9c8564ac8dd913 Author: Robert Ma <robertma@chromium.org> Date: Wed May 23 17:14:42 2018 [WPT import] Set global git user name/email This CL modifies the recipe for wpt-importer to set the Blink autoroller account as the *global* git user instead of just the repo-specific one. This change will hopefully resolve issue 844563 which started after we got some a bot with a default global git user. wpt-exporter already sets the global git user in its recipe. Bug: 844563 Change-Id: Iccde3b24207d713fef64f9b57c2d3666b0027f17 Reviewed-on: https://chromium-review.googlesource.com/1069840 Reviewed-by: Quinten Yearsley <qyearsley@chromium.org> Reviewed-by: Ben Pastene <bpastene@chromium.org> Commit-Queue: Robert Ma <robertma@chromium.org> [modify] https://crrev.com/48648cd880147ed373dd85db1c9c8564ac8dd913/recipes/recipes/wpt_import.expected/wpt-import-with-issue.json [modify] https://crrev.com/48648cd880147ed373dd85db1c9c8564ac8dd913/recipes/recipes/wpt_import.expected/wpt-import-without-issue.json [modify] https://crrev.com/48648cd880147ed373dd85db1c9c8564ac8dd913/recipes/README.recipes.md [modify] https://crrev.com/48648cd880147ed373dd85db1c9c8564ac8dd913/recipes/recipes/wpt_import.py
,
May 23 2018
Git deployed on bots automatically detects GCE instance credentials. We have env var "SKIP_GCE_AUTH_FOR_GIT" [1] to override this. However, you should really switch your bot to LUCI. You'd need a new service account (@chromium.org one won't work) -- I can create it for you, but you'd need to grant it the same permissions your current account has (again, I can help doing git admin stuff, but you have to tell me which repo and permissions to change). And btw, you shouldn't do "git config --global'. Run it in the repo you are working with instead. [1] https://cs.chromium.org/chromium/src/third_party/depot_tools/gerrit_util.py?q=gerrit_util.py+GCE&sq=package:chromium&g=0&l=238
,
May 23 2018
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/7a6b7fcaa6e83e123ab8fc00b1140beac7ba4c5d commit 7a6b7fcaa6e83e123ab8fc00b1140beac7ba4c5d Author: Robert Ma <robertma@chromium.org> Date: Wed May 23 19:15:10 2018 [WPT sync] Fix the git user setup WPT importer & exporter now run on new GCE-based bots, which have default git credentials. This CL sets SKIP_GCE_AUTH_FOR_GIT to override it (only in importer; we don't use `git cl` in exporter). Besides, we stop changing the global git user, as suggested. The proper, long-term solution is to migrate to LUCI ( issue 803111 ). This CL is just a workaround to get the bots working for now. Bug: 844563 Change-Id: I025fd27748210801795bd6564deee74f986e89e3 Reviewed-on: https://chromium-review.googlesource.com/1070489 Commit-Queue: Robert Ma <robertma@chromium.org> Reviewed-by: Quinten Yearsley <qyearsley@chromium.org> Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org> [modify] https://crrev.com/7a6b7fcaa6e83e123ab8fc00b1140beac7ba4c5d/recipes/recipes/wpt_import.expected/wpt-import-with-issue.json [modify] https://crrev.com/7a6b7fcaa6e83e123ab8fc00b1140beac7ba4c5d/recipes/recipes/wpt_export.py [modify] https://crrev.com/7a6b7fcaa6e83e123ab8fc00b1140beac7ba4c5d/recipes/README.recipes.md [modify] https://crrev.com/7a6b7fcaa6e83e123ab8fc00b1140beac7ba4c5d/recipes/recipes/wpt_import.py [modify] https://crrev.com/7a6b7fcaa6e83e123ab8fc00b1140beac7ba4c5d/recipes/recipes/wpt_import.expected/wpt-import-without-issue.json [modify] https://crrev.com/7a6b7fcaa6e83e123ab8fc00b1140beac7ba4c5d/recipes/recipes/wpt_export.expected/wpt-export.json
,
May 23 2018
Thanks for the help, Andrii. This is still not quite working. I've verified the environment variable is set correctly, but I'm seeing the following error message right after `git cl upload`: [W2018-05-23 13:07:18,657 9978 139648127891264 gerrit_util.py] need more than 1 value to unpack And then the uploader is still the GCE service account. Presumably the error message above has something to do with it? The error message doesn't have a line number and it's a built-in Python error so I don't know where exactly in gerrit_util.py this is happening.
,
May 23 2018
Looking at gerrit_util.py, it seems to try to use gitcookies first: https://cs.chromium.org/chromium/tools/depot_tools/gerrit_util.py?l=209&rcl=ebdd0db493b20f0abeab8960e6ea0ceb7c6b379a On GCE instances, we have a gitcookie cache for the service account (/home/chrome-bot/.git-credential-cache/cookie). `git cl` finds it so it'll stop looking into .netrc. Any thoughts on how to solve this? Can we just remove that cache file prior to executing the script?
,
May 23 2018
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/fc3b724985517242bb18d1d6a9f79b590335aac8 commit fc3b724985517242bb18d1d6a9f79b590335aac8 Author: Robert Ma <robertma@chromium.org> Date: Wed May 23 23:46:20 2018 [WPT import] Create a dummy home directory for importer `git cl` tries to use the cached cookies (in the home directory) by default, which is the GCE service account, regardless of the git user.email setting. This CL creates a dummy home directory without the cached cookies for the importer so that the `git cl` invocation will look up the token for the importer account from .netrc. One benefit of using a dummy HOME instead of removing the cached cookies is to prevent potential unintended effects on other processes or runs. Bug: 844563 Change-Id: Idf8426b478686601ccdf41ca21b24dca577d4101 Reviewed-on: https://chromium-review.googlesource.com/1070591 Commit-Queue: Robert Ma <robertma@chromium.org> Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org> [modify] https://crrev.com/fc3b724985517242bb18d1d6a9f79b590335aac8/recipes/recipes/wpt_import.expected/wpt-import-with-issue.json [modify] https://crrev.com/fc3b724985517242bb18d1d6a9f79b590335aac8/recipes/recipes/wpt_import.expected/wpt-import-without-issue.json [modify] https://crrev.com/fc3b724985517242bb18d1d6a9f79b590335aac8/recipes/README.recipes.md [modify] https://crrev.com/fc3b724985517242bb18d1d6a9f79b590335aac8/recipes/recipes/wpt_import.py
,
May 24 2018
,
May 24 2018
The CL in #9 seems to do the trick, finally... We had one successful automatic import: https://crrev.com/c/1070881
,
May 24 2018
Thanks Robert and Andrii \o/ |
||||||
►
Sign in to add a comment |
||||||
Comment 1 by robertma@chromium.org
, May 18 2018