Issue metadata
Sign in to add a comment
|
led get-build stopped working after automatic cache task slices were enabled |
||||||||||||||||||||||
Issue descriptionE.g. when running led get-build 8938012618933164688 | led launch the resulting task fails immediately with NO_RESOURCE: https://ci.chromium.org/swarming/task/3f5db6f78f375110?server=chrome-swarming.appspot.com
,
Aug 23
A workaround for me is to pipe in: led edit -d caches ''
,
Aug 23
I thought I fixed this; can you confirm you're using the latest version of led?
,
Aug 24
How do I check what version is the latest?
,
Aug 24
Currently deployed version (in depot_tools) is `git_revision:9a931a5307c46b16b1c12e01e8239d4a73830b89`. If you run `led version`, you should see one of these instance ids, depending on your platform: $ cipd resolve infra/tools/luci/led/ -version git_revision:9a931a5307c46b16b1c12e01e8239d4a73830b89 Packages: infra/tools/luci/led/linux-386:iIpQ5iwa7WGuykl4VFgAGmS2HnyVCUAwz26294PUAq4C infra/tools/luci/led/linux-amd64:ERsRSgZMwHDgzcZe2527X_hK6aAbyGBsfuNvsIKSppEC infra/tools/luci/led/linux-arm64:3tXD-fay5juhpK3mUH0ToIMLcW3U9ZXs-sjjv5F18_IC infra/tools/luci/led/linux-armv6l:twLjhS4AGVQnOjKxYSTkTfIUaVfT4l0NwWcGkW0zDnAC infra/tools/luci/led/linux-mips64:wbddq9_7ZFc7Vg2hu4cQqvPHd3edbdpvdpN-7ZlFIZYC infra/tools/luci/led/linux-mips64le:_3_uv-hjsh-29CecuNTsjkGkzVOGu7mW9eXggrog4b4C infra/tools/luci/led/linux-mipsle:pY10citugyciz_H6UFYJ-cAgeU1lkq41sm0yCJJJBwEC infra/tools/luci/led/linux-ppc64:0WsLUOPPDjJCk_j6erufPMHKhCurg1Ha-gYr7XS_E4kC infra/tools/luci/led/linux-ppc64le:KUve06pRLqtllRI-sPo6WzrUo5KROm6bymk6QcSuWc0C infra/tools/luci/led/linux-s390x:dOAtQ73ZmJKMSKc7khOs9S_6I8wooQTRoPgTtAbTuqYC infra/tools/luci/led/mac-amd64:CViiquK9ekBErdpJfg3yOGRF8Td8NEhADRlbK0SlANQC infra/tools/luci/led/windows-386:gsuQsh_HYEQlNAL5XZu76_GoOlue9LYQMo1lBD7ENJ8C infra/tools/luci/led/windows-amd64:NQC9vH3suHP2sFpYaluO75AJqB3UMCpPRDqRS9-cMIIC
,
Aug 25
Neither version 'latest' nor specifying that git_revision fixes the issue. This command produced a swarming task that has caches specified and won't schedule: led get-builder 'luci.fuchsia.ci:zircon-x64-clang-qemu_kvm' | led edit-recipe-bundle | led edit -p 'tryjob=true' | led launch https://chromium-swarm.appspot.com/task?id=3f873efa62fa7410&refresh=10
,
Aug 25
Ok, phosek@google.com ran into this and I realized what the issue is :). led drops everything after slice 0. Looks like I need to rework led a bit.
,
Aug 27
I am curious what the issue was. Can you share more details please?
,
Aug 29
It's a combination: - Buildbucket started using multiple task slices (vs only one before). - LED assumed only one task slice and used the first one (which is the warm cache path).
,
Sep 5
Workaround in #2 works great for this. One tiny caveat: for some asks it may also be necessary to edit the job manually to increase the expiration timeout - I could not find a led command to do this without manual editing. Without this change, led task expires in just 2 minutes if no bots are available atm.
,
Sep 5
setting expiration_secs from command line led get-whatever | jq '.systemland.swarming_task.task_slices[0].expiration_secs = "600"'
,
Sep 10
,
Sep 10
Planning to fix this tomorrow
,
Sep 12
Later in the day than I hoped, but CL: https://chromium-review.googlesource.com/c/infra/infra/+/1220602
,
Sep 12
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/b71c243dd3e086edf914e1bd3411c3f557505829 commit b71c243dd3e086edf914e1bd3411c3f557505829 Author: Robert Iannucci <iannucci@chromium.org> Date: Wed Sep 12 18:02:29 2018 [led] Fix led to properly handle tasks with many slices. This changes the JobDefinition intermediate JSON format to handle multiple slices. Incoming tasks which didn't use slices are converted to tasks with a single slice. `led edit*` operations now apply to every slice in the JobDefinition. R=hinoka@chromium.org, nodir@chromium.org, tandrii@chromium.org Bug: 875089 Change-Id: Iaa60bd16ec62c14e87337f5af4179f324302a0ca Reviewed-on: https://chromium-review.googlesource.com/1220602 Commit-Queue: Robbie Iannucci <iannucci@chromium.org> Reviewed-by: Nodir Turakulov <nodir@chromium.org> Cr-Commit-Position: refs/heads/master@{#17535} [modify] https://crrev.com/b71c243dd3e086edf914e1bd3411c3f557505829/go/src/infra/tools/led/job_userland.go [modify] https://crrev.com/b71c243dd3e086edf914e1bd3411c3f557505829/go/src/infra/tools/led/job.go [modify] https://crrev.com/b71c243dd3e086edf914e1bd3411c3f557505829/go/src/infra/tools/led/cmd_get_swarm.go [modify] https://crrev.com/b71c243dd3e086edf914e1bd3411c3f557505829/go/src/infra/tools/led/job_edit.go [modify] https://crrev.com/b71c243dd3e086edf914e1bd3411c3f557505829/go/src/infra/tools/led/job_def.go [modify] https://crrev.com/b71c243dd3e086edf914e1bd3411c3f557505829/go/src/infra/tools/led/job_systemland.go
,
Sep 13
The following revision refers to this bug: https://chromium.googlesource.com/chromium/tools/depot_tools/+/dfce68bcddba7508d74016405a95e7865cbf47ba commit dfce68bcddba7508d74016405a95e7865cbf47ba Author: Robert Iannucci <iannucci@chromium.org> Date: Thu Sep 13 01:23:56 2018 [led] Roll led to handle jobs with multiple swarming TaskSlices. R=hinoka@chromium.org, nodir@chromium.org, tandrii@chromium.org Bug: 875089 Change-Id: I6abb7a5d0aebf53c4c3ab6be309ed43e652278c0 Reviewed-on: https://chromium-review.googlesource.com/1222883 Commit-Queue: Robbie Iannucci <iannucci@chromium.org> Commit-Queue: Ryan Tseng <hinoka@chromium.org> Auto-Submit: Robbie Iannucci <iannucci@chromium.org> Reviewed-by: Ryan Tseng <hinoka@chromium.org> [modify] https://crrev.com/dfce68bcddba7508d74016405a95e7865cbf47ba/cipd_manifest.txt
,
Sep 13
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/de2a0a7df36d477a8a6398488c7c8d785e90373d commit de2a0a7df36d477a8a6398488c7c8d785e90373d Author: depot-tools-chromium-autoroll <depot-tools-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Date: Thu Sep 13 04:39:15 2018 Roll src/third_party/depot_tools 0425ebd2b395..dfce68bcddba (1 commits) https://chromium.googlesource.com/chromium/tools/depot_tools.git/+log/0425ebd2b395..dfce68bcddba git log 0425ebd2b395..dfce68bcddba --date=short --no-merges --format='%ad %ae %s' 2018-09-13 iannucci@chromium.org [led] Roll led to handle jobs with multiple swarming TaskSlices. Created with: gclient setdep -r src/third_party/depot_tools@dfce68bcddba The AutoRoll server is located here: https://autoroll.skia.org/r/depot-tools-chromium-autoroll Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, please contact the current sheriff, who should be CC'd on the roll, and stop the roller if necessary. BUG= chromium:875089 TBR=agable@chromium.org Change-Id: I55fa403d9f88e95b2c70380bde69e6d1cf22fd9b Reviewed-on: https://chromium-review.googlesource.com/1222202 Reviewed-by: depot-tools-chromium-autoroll <depot-tools-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Commit-Queue: depot-tools-chromium-autoroll <depot-tools-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Cr-Commit-Position: refs/heads/master@{#590925} [modify] https://crrev.com/de2a0a7df36d477a8a6398488c7c8d785e90373d/DEPS
,
Sep 13
Looks like recent changes broke led in a different way: $ led get-build 8935510276360715312 | led edit -S chrome-swarming.appspot.com | led edit-recipe-bundle | led launch [I 2018-09-13 11:35:25] getting build definition [I 2018-09-13 11:35:25] bundling recipes [I 2018-09-13 11:35:26] getting build definition: done [I 2018-09-13 11:35:26] getting task definition: "3feb0bf0dda5ba10" [I 2018-09-13 11:35:26] getting task definition: done [W 2018-09-13 11:35:26] waiting for JobDefinition on stdin... [I 2018-09-13 11:35:27] bundling recipes: done [I 2018-09-13 11:35:27] isolating recipes [found] [hashed/size/to hash] [looked up/to lookup] [uploaded/size/to upload/size] [1778] [1778/12.6Mib/1778] [1728/1778] [0/0b/0/0b] 1.6s [I 2018-09-13 11:35:29] isolating recipes: done [found] [hashed/size/to hash] [looked up/to lookup] [uploaded/size/to upload/size] [I 2018-09-13 11:35:29] building swarming task [I 2018-09-13 11:35:29] building swarming task: done [I 2018-09-13 11:35:29] launching swarming task [E 2018-09-13 11:35:29] original error: googleapi: Error 400: missing expiration_secs I've checked the source of the generated task (after led edit-recipe-bundle) and it has two slices, both with expiration_secs. Not sure why it fails...
,
Sep 13
Same error for $ led get-builder luci.v8-internal.try:v8_pixel2_perf_try | led edit-cr-cl https://chromium-review.googlesource.com/c/v8/v8/+/1204092 | led edit -S chrome-swarming.appspot.com | led edit-recipe-bundle | led launch
,
Sep 14
This is also affecting me. Same error as sergiyb. """ original error: googleapi: Error 400: missing expiration_secs """
,
Sep 20
Same here after update_depot_tools. My workaround is to install an old version of led: cipd install infra/tools/luci/led/mac-amd64 CViiquK9ekBErdpJfg3yOGRF8Td8NEhADRlbK0SlANQC
,
Sep 20
Issue 887552 has been merged into this issue.
,
Sep 24
I've tried workaround from #21, but still get "Error 400: missing expiration_secs". I've done it on linux, so I've used the following command: cipd install infra/tools/luci/led/linux-amd64 gT9UxxL939aCQmOX-Mwn4G3FtVUedMSntDnf6Cw2yiQC -root $HOME/depot_tools/.cipd_bin Looks like the command above downloads wrong binary, since manually downloading the binary from [1] and then placing it into $HOME/depot_tools/.cipd_bin worked. [1]: https://chrome-infra-packages.appspot.com/p/infra/tools/luci/led/linux-amd64/+/gT9UxxL939aCQmOX-Mwn4G3FtVUedMSntDnf6Cw2yiQC
,
Sep 24
Sorry I dropped the ball on this; CL to actually fix this is https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1241677
,
Sep 24
The following revision refers to this bug: https://chromium.googlesource.com/chromium/tools/depot_tools/+/64b61330ec7f3632d5eb1ea97133ce11f3bf6627 commit 64b61330ec7f3632d5eb1ea97133ce11f3bf6627 Author: Robert Iannucci <iannucci@chromium.org> Date: Mon Sep 24 22:54:13 2018 [led] Roll led to fix missing expiration_secs issue. TBR=tandrii@chromium.org, vadimsh@chromium.org Bug: 875089 Change-Id: Ia48c7c67df8bdbd9bc0573406bd9f56f6cfe5d49 Reviewed-on: https://chromium-review.googlesource.com/1241677 Reviewed-by: Robbie Iannucci <iannucci@chromium.org> Reviewed-by: Vadim Shtayura <vadimsh@chromium.org> Commit-Queue: Robbie Iannucci <iannucci@chromium.org> [modify] https://crrev.com/64b61330ec7f3632d5eb1ea97133ce11f3bf6627/cipd_manifest.versions [modify] https://crrev.com/64b61330ec7f3632d5eb1ea97133ce11f3bf6627/cipd_manifest.txt
,
Sep 24
Finally fixed... so sorry for the extended breakage :(.
,
Sep 25
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/5e1f96f2b6b7c9e947ee8165fad27e60082929a6 commit 5e1f96f2b6b7c9e947ee8165fad27e60082929a6 Author: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com> Date: Tue Sep 25 00:36:21 2018 Roll src/third_party/depot_tools 482d615b83e1..64b61330ec7f (1 commits) https://chromium.googlesource.com/chromium/tools/depot_tools.git/+log/482d615b83e1..64b61330ec7f git log 482d615b83e1..64b61330ec7f --date=short --no-merges --format='%ad %ae %s' 2018-09-24 iannucci@chromium.org [led] Roll led to fix missing expiration_secs issue. Created with: gclient setdep -r src/third_party/depot_tools@64b61330ec7f The AutoRoll server is located here: https://autoroll.skia.org/r/depot-tools-chromium-autoroll Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, please contact the current sheriff, who should be CC'd on the roll, and stop the roller if necessary. BUG= chromium:875089 TBR=agable@chromium.org Change-Id: I27e0bdd4fb470ecd7304c86c2c4563d2c0623c1e Reviewed-on: https://chromium-review.googlesource.com/1242054 Reviewed-by: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com> Commit-Queue: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com> Cr-Commit-Position: refs/heads/master@{#593776} [modify] https://crrev.com/5e1f96f2b6b7c9e947ee8165fad27e60082929a6/DEPS
,
Oct 15
The following revision refers to this bug: https://chromium.googlesource.com/chromium/tools/depot_tools/+/2f727917ac2531cd26c4b4d4fc805398c0bbd132 commit 2f727917ac2531cd26c4b4d4fc805398c0bbd132 Author: Andrii Shyshkalov <tandrii@chromium.org> Date: Mon Oct 15 17:02:33 2018 git cl: add reviewers and ccs to git push command if possible. Also fix a typo in ValidateEmail function which didn't support email addresses with '-' in them, e.g., infra-dev@chromium.org. R=ehmaldonado Bug: 875089 Change-Id: I2d73c1473527c9bf62e25e9f88250196b783fcb0 Reviewed-on: https://chromium-review.googlesource.com/c/1242849 Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org> Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org> [modify] https://crrev.com/2f727917ac2531cd26c4b4d4fc805398c0bbd132/scm.py [modify] https://crrev.com/2f727917ac2531cd26c4b4d4fc805398c0bbd132/tests/git_cl_test.py [modify] https://crrev.com/2f727917ac2531cd26c4b4d4fc805398c0bbd132/git_cl.py
,
Oct 15
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/4eb8a3969e2dcf2603615ebcc933cff0f3d42c5d commit 4eb8a3969e2dcf2603615ebcc933cff0f3d42c5d Author: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com> Date: Mon Oct 15 21:01:02 2018 Roll src/third_party/depot_tools 9f274436bd0d..2f727917ac25 (1 commits) https://chromium.googlesource.com/chromium/tools/depot_tools.git/+log/9f274436bd0d..2f727917ac25 git log 9f274436bd0d..2f727917ac25 --date=short --no-merges --format='%ad %ae %s' 2018-10-15 tandrii@chromium.org git cl: add reviewers and ccs to git push command if possible. Created with: gclient setdep -r src/third_party/depot_tools@2f727917ac25 The AutoRoll server is located here: https://autoroll.skia.org/r/depot-tools-chromium-autoroll Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, please contact the current sheriff, who should be CC'd on the roll, and stop the roller if necessary. BUG= chromium:875089 TBR=agable@chromium.org Change-Id: I3ecf74c74a21847cb52ffb91bd7d66687e93738d Reviewed-on: https://chromium-review.googlesource.com/c/1281202 Reviewed-by: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com> Commit-Queue: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com> Cr-Commit-Position: refs/heads/master@{#599746} [modify] https://crrev.com/4eb8a3969e2dcf2603615ebcc933cff0f3d42c5d/DEPS |
|||||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||||
Comment 1 by serg...@chromium.org
, Aug 17