Migrate "wpt-importer" to LUCI |
|||||||||||||||
Issue descriptionMigrate builder chromium.infra.cron:wpt-importer to LUCI. Buildbot: https://ci.chromium.org/buildbot/chromium.infra.cron/wpt-importer LUCI: https://ci.chromium.org/buildbucket/luci.infra.cron/wpt-importer Migration app will be posting updates on changes of the migration status. For the latest status, see https://luci-migration.appspot.com/masters/chromium.infra.cron/builders/wpt-importer Migration app will close this bug when the builder is entirely migrated from Buildbot to LUCI.
,
Jan 18 2018
,
Jan 30 2018
,
Feb 13 2018
,
Feb 21 2018
,
Apr 23 2018
Should this still be P2? There's no status update at https://luci-migration.appspot.com/masters/chromium.infra.cron/builders/wpt-importer/
,
Apr 30 2018
This would most likely be migrated together with wpt-exporter ( bug 790570 ). First step for both of these is to modify the recipe to check the api.runtime.is_experimental recipe attribute, and if the builder is experimental, just do a dry run. Then we'll add the LUCI builder, check that it runs as expected with dry runs and then flip "LUCI is prod". This will make it easy to roll back if something unexpected happens.
,
May 19 2018
,
Jul 10
,
Jul 10
,
Jul 10
,
Jul 10
,
Sep 8
,
Oct 1
,
Oct 4
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/2f1e7a9d203304f6e14570ff17fc332d62649800 commit 2f1e7a9d203304f6e14570ff17fc332d62649800 Author: Nodir Turakulov <nodir@google.com> Date: Thu Oct 04 22:26:17 2018 luci.infra.cron: connect to migration app Connect luci.infra.cron to migration app because not all builders were migrated to LUCI yet. Bug: 790503 Change-Id: I0ae2a67d17f50955ee869d3540f80dabae20454b Reviewed-on: https://chromium-review.googlesource.com/c/1262919 Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org> Commit-Queue: Nodir Turakulov <nodir@chromium.org> [modify] https://crrev.com/2f1e7a9d203304f6e14570ff17fc332d62649800/cr-buildbucket.cfg
,
Oct 12
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/167121ea09987452572723a78b2d8987b770b751 commit 167121ea09987452572723a78b2d8987b770b751 Author: Robert Ma <robertma@chromium.org> Date: Fri Oct 12 13:49:45 2018 [WPT import] Add a flag to upload but not submit CL We used to have only one flag, --auto-update, which controls both uploading and landing the import CL. This makes testing new setup of the bot (i.e. LUCI in this case) hard. This change adds one more flag, --auto-upload, to exercise more steps (namely uploading and rebaselining), which will be used on the experimental LUCI builder. Bug: 790503 Change-Id: I4ce07fc9ddf15a34c9cb25ba4723d250cad26977 Reviewed-on: https://chromium-review.googlesource.com/c/1277587 Commit-Queue: Robert Ma <robertma@chromium.org> Reviewed-by: Quinten Yearsley <qyearsley@chromium.org> Cr-Commit-Position: refs/heads/master@{#599197} [modify] https://crrev.com/167121ea09987452572723a78b2d8987b770b751/third_party/blink/tools/blinkpy/w3c/test_importer.py
,
Oct 12
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/ec5a8d649238fad935a09d6ae1f260bde7171b6b commit ec5a8d649238fad935a09d6ae1f260bde7171b6b Author: Robert Ma <robertma@chromium.org> Date: Fri Oct 12 13:51:45 2018 [WPT import] Add --auto-upload to importer on LUCI To test two more critical steps in the importer: uploading CL and rebaselining. The flag is being added in https://crrev.com/c/1277587 . Bug: 790503 Change-Id: Id3abeafe573e876019da0c5e06236728053a1168 Reviewed-on: https://chromium-review.googlesource.com/c/1277625 Reviewed-by: Quinten Yearsley <qyearsley@chromium.org> Commit-Queue: Robert Ma <robertma@chromium.org> Cr-Commit-Position: refs/heads/master@{#18280} [modify] https://crrev.com/ec5a8d649238fad935a09d6ae1f260bde7171b6b/recipes/recipes/wpt_import.expected/wpt-import-without-issue_buildbot_experimental.json [modify] https://crrev.com/ec5a8d649238fad935a09d6ae1f260bde7171b6b/recipes/README.recipes.md [modify] https://crrev.com/ec5a8d649238fad935a09d6ae1f260bde7171b6b/recipes/recipes/wpt_import.py
,
Oct 30
Build looks to be working as intended and matches Buildbot results. Can the LUCI builder be flipped to prod?
,
Oct 30
Hi Eric, sorry I was travelling in the last two weeks. Yes, could you please flip the switch to set LUCI as prod? Thanks!
,
Oct 31
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/72b85dbf39aee46060072351c2057bd18913356e commit 72b85dbf39aee46060072351c2057bd18913356e Author: Robert Ma <robertma@chromium.org> Date: Wed Oct 31 19:51:15 2018 [WPT Import] Stop creating CLs on experimental bot The experimental builds on LUCI have proven to work as intended and we are flipping prod to LUCI, so we don't need to create CLs from the experimental bot anymore. This will save some unnecessary try jobs and reduce the wait time of *-blink-rel bots. Bug: 790503 Change-Id: I7e983165a4df3c327278df0ec77618ce96a7ca45 Reviewed-on: https://chromium-review.googlesource.com/c/1310523 Auto-Submit: Robert Ma <robertma@chromium.org> Commit-Queue: Ryan Tseng <hinoka@chromium.org> Reviewed-by: Ryan Tseng <hinoka@chromium.org> Cr-Commit-Position: refs/heads/master@{#18702} [modify] https://crrev.com/72b85dbf39aee46060072351c2057bd18913356e/recipes/recipes/wpt_import.expected/wpt-import-without-issue_buildbot_experimental.json [modify] https://crrev.com/72b85dbf39aee46060072351c2057bd18913356e/recipes/README.recipes.md [modify] https://crrev.com/72b85dbf39aee46060072351c2057bd18913356e/recipes/recipes/wpt_import.py
,
Nov 1
Imports are currently failing on chromium_presubmit due to the account being used. See e.g. https://chromium-review.googlesource.com/c/chromium/src/+/1312618/ and https://ci.chromium.org/p/chromium/builders/luci.chromium.try/chromium_presubmit/269758: ** Presubmit ERRORS ** wpt-autoroller@chops-service-accounts.iam.gserviceaccount.com is not in AUTHORS file. If you are a new contributor, please visit https://www.chromium.org/developers/contributing-code and read the "Legal" section If you are a chromite, verify the contributor signed the CLA.
,
Nov 2
The issue above seems to have been fixed, but the new account seems to be having trouble with Monorail. From https://ci.chromium.org/p/infra/builders/luci.infra.cron/wpt-importer/b8931004603591793696: 2018-11-01 22:18:25,607 - URL being requested: GET https://monorail-prod.appspot.com/_ah/api/discovery/v1/apis/monorail/v1/rest 2018-11-01 22:18:25,743 - URL being requested: POST https://monorail-prod.appspot.com/_ah/api/monorail/v1/projects/chromium/issues?alt=json 2018-11-01 22:18:25,743 - Attempting refresh to obtain initial access_token 2018-11-01 22:18:25,799 - Refreshing due to a 401 (attempt 1/2) 2018-11-01 22:18:25,827 - Refreshing due to a 401 (attempt 2/2) Traceback (most recent call last): File "/b/swarming/w/ir/kitchen-workdir/src/third_party/blink/tools/wpt_import.py", line 26, in <module> main() File "/b/swarming/w/ir/kitchen-workdir/src/third_party/blink/tools/wpt_import.py", line 19, in main host.exit(importer.main()) File "/b/swarming/w/ir/kitchen-workdir/src/third_party/blink/tools/blinkpy/w3c/test_importer.py", line 186, in main if not self.send_notifications(local_wpt, options.auto_file_bugs, options.monorail_auth_json): File "/b/swarming/w/ir/kitchen-workdir/src/third_party/blink/tools/blinkpy/w3c/test_importer.py", line 667, in send_notifications dry_run=not auto_file_bugs, service_account_key_json=monorail_auth_json) File "/b/swarming/w/ir/kitchen-workdir/src/third_party/blink/tools/blinkpy/w3c/import_notifier.py", line 73, in main self.file_bugs(bugs, dry_run, service_account_key_json) File "/b/swarming/w/ir/kitchen-workdir/src/third_party/blink/tools/blinkpy/w3c/import_notifier.py", line 264, in file_bugs response = api.insert_issue(bug) File "/b/swarming/w/ir/kitchen-workdir/src/third_party/blink/tools/blinkpy/w3c/monorail.py", line 137, in insert_issue return self.api.issues().insert(projectId=issue.project_id, body=body).execute() File "/b/swarming/w/ir/cache/vpython/ae31aa/lib/python2.7/site-packages/oauth2client/_helpers.py", line 133, in positional_wrapper return wrapped(*args, **kwargs) File "/b/swarming/w/ir/cache/vpython/ae31aa/lib/python2.7/site-packages/googleapiclient/http.py", line 840, in execute raise HttpError(resp, content, uri=self.uri) googleapiclient.errors.HttpError: <HttpError 401 when requesting https://monorail-prod.appspot.com/_ah/api/monorail/v1/projects/chromium/issues?alt=json returned "Auth error: Neither client ID '116719108335393041424' nor email 'swarming@chromecompute.google.com.iam.gserviceaccount.com' is whitelisted"> step returned non-zero exit code: 1
,
Nov 6
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/d86841e2fe2b3eda485a350a7ef44a489f15009e commit d86841e2fe2b3eda485a350a7ef44a489f15009e Author: Robert Ma <robertma@chromium.org> Date: Tue Nov 06 18:20:38 2018 [Import Notifier] Get Monorail OAuth token from luci-auth The application default credential belongs to the default Swarming bot account, not the dedicated service account for wpt-importer. As suggested by LUCI task force, we need to use `luci-auth token` instead. Bug: 790503 Change-Id: I3257db9a49196fd427c6f2371d00dcabab7ee3a6 Reviewed-on: https://chromium-review.googlesource.com/c/1315931 Commit-Queue: Robert Ma <robertma@chromium.org> Reviewed-by: Quinten Yearsley <qyearsley@chromium.org> Cr-Commit-Position: refs/heads/master@{#605746} [add] https://crrev.com/d86841e2fe2b3eda485a350a7ef44a489f15009e/third_party/blink/tools/blinkpy/common/net/luci_auth.py [modify] https://crrev.com/d86841e2fe2b3eda485a350a7ef44a489f15009e/third_party/blink/tools/blinkpy/w3c/import_notifier.py [modify] https://crrev.com/d86841e2fe2b3eda485a350a7ef44a489f15009e/third_party/blink/tools/blinkpy/w3c/import_notifier_unittest.py [modify] https://crrev.com/d86841e2fe2b3eda485a350a7ef44a489f15009e/third_party/blink/tools/blinkpy/w3c/monorail.py
,
Nov 8
,
Dec 17
|
|||||||||||||||
►
Sign in to add a comment |
|||||||||||||||
Comment 1 by robertma@chromium.org
, Jan 17 2018Cc: robertma@chromium.org foolip@chromium.org
Components: Blink>Infra>Ecosystem