New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 863238 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Sep 26
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug

Blocked on:
issue 868608



Sign in to add a comment

Add new builder to test chromium_tests recipe changes

Project Member Reported by martiniss@chromium.org, Jul 12

Issue description

This builder would use the `led` tool (https://chromium.googlesource.com/infra/infra/+/master/go/src/infra/tools/led) to run a chromium builder with the modified recipe.

Design doc (internal for now) is at https://docs.google.com/document/d/1-wbWhzJWDtDtp9w6MMElWOOzLDcAZA2gaz_UDK3cosY/edit
 

Comment 1 Deleted

Comment 2 Deleted

Project Member

Comment 3 by bugdroid1@chromium.org, Jul 16

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/f705d73767f4596e4c19090fb2c1d2d921014b64

commit f705d73767f4596e4c19090fb2c1d2d921014b64
Author: Stephen Martinis <martiniss@chromium.org>
Date: Mon Jul 16 20:07:08 2018

led: Add option to dump task json when launching

led doesn't output any structured data about the swarming task it
launched. Recipes need structured data, so add some for the launch
command.

Bug:  863238 
Change-Id: I5caf7a14de2b5041b25f195be87d8ec708dff676
Reviewed-on: https://chromium-review.googlesource.com/1135964
Commit-Queue: Stephen Martinis <martiniss@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>

[modify] https://crrev.com/f705d73767f4596e4c19090fb2c1d2d921014b64/go/src/infra/tools/led/cmd_launch.go

Project Member

Comment 4 by bugdroid1@chromium.org, Jul 19

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/fd3247f14fad5b9b0322ebb8b36245f408824647

commit fd3247f14fad5b9b0322ebb8b36245f408824647
Author: Stephen Martinis <martiniss@chromium.org>
Date: Thu Jul 19 22:02:39 2018

led: Dump swarming task json to stdout

This makes led easier to interact with programatically.

Bug:  863238 
Change-Id: Ice877f72d91e7f27da242f44b30f44c11f82c451
Reviewed-on: https://chromium-review.googlesource.com/1141206
Commit-Queue: Stephen Martinis <martiniss@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>

[modify] https://crrev.com/fd3247f14fad5b9b0322ebb8b36245f408824647/go/src/infra/tools/led/cmd_launch.go

Project Member

Comment 6 by bugdroid1@chromium.org, Jul 25

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/40e4c5a7aac9102bd636e034a9a061c85165a0f8

commit 40e4c5a7aac9102bd636e034a9a061c85165a0f8
Author: Stephen Martinis <martiniss@chromium.org>
Date: Wed Jul 25 00:05:51 2018

Add 'led_recipes_tester' recipe

This can be used to test recipe changes, by running a chromium builder
build using the modified recipes, using led. The plan is to add CQ
tryjobs to the build and depot_tools repos which use this recipe.

Bug:  863238 
Change-Id: I53bd0f4b097da018af4049d44772f39a63f03b7b
Reviewed-on: https://chromium-review.googlesource.com/1139187
Commit-Queue: Stephen Martinis <martiniss@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>

[modify] https://crrev.com/40e4c5a7aac9102bd636e034a9a061c85165a0f8/scripts/slave/README.recipes.md
[add] https://crrev.com/40e4c5a7aac9102bd636e034a9a061c85165a0f8/scripts/slave/recipes/led_recipes_tester.expected/basic.json
[add] https://crrev.com/40e4c5a7aac9102bd636e034a9a061c85165a0f8/scripts/slave/recipes/led_recipes_tester.py

Project Member

Comment 7 by bugdroid1@chromium.org, Jul 25

Labels: merge-merged-config
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/1f19cde60789183e0392038105f4da6eada6c775

commit 1f19cde60789183e0392038105f4da6eada6c775
Author: Stephen Martinis <martiniss@chromium.org>
Date: Wed Jul 25 18:14:15 2018

Add Build recipes tester

Bug:  863238 
Change-Id: I04854a93dde485710a71b165472d03c8ae2051f1
Reviewed-on: https://chromium-review.googlesource.com/1149326
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Stephen Martinis <martiniss@chromium.org>

[modify] https://crrev.com/1f19cde60789183e0392038105f4da6eada6c775/cr-buildbucket.cfg

Project Member

Comment 8 by bugdroid1@chromium.org, Jul 25

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infradata/config/+/4d8a1807a85bd4f61f737cd3950b171f2d98aa92

commit 4d8a1807a85bd4f61f737cd3950b171f2d98aa92
Author: Stephen Martinis <martiniss@google.com>
Date: Wed Jul 25 19:06:07 2018

Project Member

Comment 9 by bugdroid1@chromium.org, Jul 25

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/627e36bee411cc04b12e964be8c7696f624ff396

commit 627e36bee411cc04b12e964be8c7696f624ff396
Author: Stephen Martinis <martiniss@chromium.org>
Date: Wed Jul 25 19:11:04 2018

Use recipes tester service account for recipes tester

Bug:  863238 
Change-Id: I3355ba254db0316d0b71f99d8b4edae65122a80c
Reviewed-on: https://chromium-review.googlesource.com/1150563
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
Commit-Queue: Stephen Martinis <martiniss@chromium.org>

[modify] https://crrev.com/627e36bee411cc04b12e964be8c7696f624ff396/cr-buildbucket.cfg

Labels: Foundation-Troopers
Owner: ----
Status: Available (was: Assigned)
I'm running into some permissions errors with the new service account.

#1: When running `git cl try`, I get Auth error while calling swarming on behalf of user:martiniss@chromium.org: {"error": {"message": "Service account \"infra-try-recipes-tester@chops-service-accounts.iam.gserviceaccount.com\" is not allowed in the pool \"luci.flex.try\", see pools.cfg"}}.


I tried adding the service account to project-infra-try-task-accounts, but it then gave me this error:
Auth error while calling swarming on behalf of user:martiniss@chromium.org: {"error": {"message": "service account \"infra-try-recipes-tester@chops-service-accounts.iam.gserviceaccount.com\" matches 2 rules in the config rev 4d8a1807a85bd4f61f737cd3950b171f2d98aa92: \"luci.infra.try builders\", \"luci.infra.try recipe testers\"\n"}}.


I then removed it from that group, and magically it worked for a bit and scheduled https://chromium-swarm.appspot.com/task?id=3eebcb7129656410&refresh=10&show_raw=1&wide_logs=true

I tried git cl try again after a bit, and it failed with the original error.

#2: In the task listed above, it can't seem to upload the recipe to the isolate server. I'm not sure where those permissions are managed, but I probably need to add the service account to something.

Foundation trooper, do you know who can help with these issues?
Re "project-infra-try-task-accounts". We should add infra-try-recipes-tester@ to this group and delete https://chrome-internal.googlesource.com/infradata/config/+/master/configs/luci-token-server/service_accounts.cfg#158 (since the account would be covered by this rule already: https://chrome-internal.googlesource.com/infradata/config/+/master/configs/luci-token-server/service_accounts.cfg#151)

Sorry, should have noticed earlier.

Once we do this, Isolate should start working too, since project-infra-try-task-accounts is in isolateserver-full-access group.
Project Member

Comment 12 by bugdroid1@chromium.org, Jul 25

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/9fef0cfbbee9067b71a2977553448ed1c157b0df

commit 9fef0cfbbee9067b71a2977553448ed1c157b0df
Author: Stephen Martinis <martiniss@chromium.org>
Date: Wed Jul 25 20:03:56 2018

led_recipes_tester: Set is_experimental

These tryjobs shouldn't affect production traffic, so set the
is_experimental flag to try to avoid this.

Bug:  863238 
Change-Id: I0cb2044a5a3c67fefe33221da0e0b5b3af7e8189
Reviewed-on: https://chromium-review.googlesource.com/1150575
Commit-Queue: Stephen Martinis <martiniss@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>

[modify] https://crrev.com/9fef0cfbbee9067b71a2977553448ed1c157b0df/scripts/slave/README.recipes.md
[modify] https://crrev.com/9fef0cfbbee9067b71a2977553448ed1c157b0df/scripts/slave/recipes/led_recipes_tester.expected/basic.json
[modify] https://crrev.com/9fef0cfbbee9067b71a2977553448ed1c157b0df/scripts/slave/recipes/led_recipes_tester.py

Project Member

Comment 13 by bugdroid1@chromium.org, Jul 25

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infradata/config/+/d04d633bf3fadd11304687d78f6c0b71b0b79ffc

commit d04d633bf3fadd11304687d78f6c0b71b0b79ffc
Author: Stephen Martinis <martiniss@google.com>
Date: Wed Jul 25 20:05:48 2018

What was proposed in #11 is done.

Tasks are now failing because they can't seem to mint tokens for the service account. https://ci.chromium.org/p/infra/builders/luci.infra.try/Build%20Recipes%20Tester/b8939999145977762592 is a sample task
Labels: -Foundation-Troopers
Owner: martiniss@chromium.org
Status: Assigned (was: Available)
Seems like those errors went away after a bit: https://ci.chromium.org/p/infra/builders/luci.infra.try/Build%20Recipes%20Tester/b8939998915401340704 seems to be ok now
Cc: vadimsh@chromium.org
I added infra-try-recipes-tester@chops-service-accounts.iam.gserviceaccount.com to the 'chromium-led-users' group, which should allow it to schedule tasks in the luci.chromium.try pool. Not sure if that's the best practice?
Project Member

Comment 17 by bugdroid1@chromium.org, Jul 26

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/71f4c437210b151dea71f7944af26a2e139db265

commit 71f4c437210b151dea71f7944af26a2e139db265
Author: Stephen Martinis <martiniss@chromium.org>
Date: Thu Jul 26 02:34:52 2018

led_recipes_tester: Fix swarming.py path

Bug:  863238 
Change-Id: I08ce1d4a21bbdbb7d6618e44380d2f05505b555b
Reviewed-on: https://chromium-review.googlesource.com/1150630
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Stephen Martinis <martiniss@chromium.org>

[modify] https://crrev.com/71f4c437210b151dea71f7944af26a2e139db265/scripts/slave/recipes/led_recipes_tester.expected/basic.json
[modify] https://crrev.com/71f4c437210b151dea71f7944af26a2e139db265/scripts/slave/recipes/led_recipes_tester.py

Project Member

Comment 18 by bugdroid1@chromium.org, Jul 26

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/3f4e29fc90249e1565e6ddf489bd281dcaee6bc2

commit 3f4e29fc90249e1565e6ddf489bd281dcaee6bc2
Author: Stephen Martinis <martiniss@chromium.org>
Date: Thu Jul 26 19:41:20 2018

led_recipes_tester: Fix swarming.py path for real

TBR=iannucci

Bug:  863238 
Change-Id: Ie90fe5072493da3a10ff050e572f9fd9b736094d
Reviewed-on: https://chromium-review.googlesource.com/1151959
Commit-Queue: Stephen Martinis <martiniss@chromium.org>
Reviewed-by: Stephen Martinis <martiniss@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>

[modify] https://crrev.com/3f4e29fc90249e1565e6ddf489bd281dcaee6bc2/scripts/slave/recipes/led_recipes_tester.expected/basic.json
[modify] https://crrev.com/3f4e29fc90249e1565e6ddf489bd281dcaee6bc2/scripts/slave/recipes/led_recipes_tester.py

Project Member

Comment 19 by bugdroid1@chromium.org, Jul 26

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infradata/config/+/c115c1f5d4025c65f903424f1c78175aa93d9fa5

commit c115c1f5d4025c65f903424f1c78175aa93d9fa5
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Thu Jul 26 20:07:04 2018

Per #19, removed infra-try-recipes-tester@chops-service-accounts.iam.gserviceaccount.com from chromium-led-users.
Project Member

Comment 21 by bugdroid1@chromium.org, Jul 26

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/recipes-py/+/36cdb3033498ecf5b5834dbc58eca2542164cbed

commit 36cdb3033498ecf5b5834dbc58eca2542164cbed
Author: Stephen Martinis <martiniss@google.com>
Date: Thu Jul 26 20:47:53 2018

led: Add task link when launching a task

Recipe-Nontrivial-Roll: build
Bug:  863238 
Change-Id: I699b464b5f879f13befca5aa27c4b7d4fd3d84ca
Reviewed-on: https://chromium-review.googlesource.com/1152101
Commit-Queue: Stephen Martinis <martiniss@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>

[modify] https://crrev.com/36cdb3033498ecf5b5834dbc58eca2542164cbed/recipe_modules/led/tests/full.py
[modify] https://crrev.com/36cdb3033498ecf5b5834dbc58eca2542164cbed/recipe_modules/led/tests/full.expected/basic.json
[modify] https://crrev.com/36cdb3033498ecf5b5834dbc58eca2542164cbed/recipe_modules/led/api.py

Project Member

Comment 22 by bugdroid1@chromium.org, Jul 27

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/bd9ac73abfb78aff978380abf88dcacf4d109174

commit bd9ac73abfb78aff978380abf88dcacf4d109174
Author: Stephen Martinis <martiniss@chromium.org>
Date: Fri Jul 27 22:33:33 2018

led_recipes_tester: Modify collect call

Add some relevant arguments. Also add a link to the swarming task.

Bug:  863238 
Change-Id: Icfef85e215f2e1cb96f44eca0188703880a1e205
Reviewed-on: https://chromium-review.googlesource.com/1152117
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Stephen Martinis <martiniss@chromium.org>

[modify] https://crrev.com/bd9ac73abfb78aff978380abf88dcacf4d109174/scripts/slave/recipes/led_recipes_tester.expected/basic.json
[modify] https://crrev.com/bd9ac73abfb78aff978380abf88dcacf4d109174/scripts/slave/recipes/led_recipes_tester.py

Blockedon: 868608
Project Member

Comment 24 by bugdroid1@chromium.org, Jul 31

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/7418d15231806a6a6cd06dacc49e77920ed80211

commit 7418d15231806a6a6cd06dacc49e77920ed80211
Author: Stephen Martinis <martiniss@chromium.org>
Date: Tue Jul 31 19:44:40 2018

Increase execution timeout for recipes tester

It's running full chromium builds, so it needs longer than 30 minutes.

Bug:  863238 
Change-Id: I9582c3f4258a74ef01d1846adeedb2460312fbb8
Reviewed-on: https://chromium-review.googlesource.com/1157171
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Commit-Queue: Stephen Martinis <martiniss@chromium.org>

[modify] https://crrev.com/7418d15231806a6a6cd06dacc49e77920ed80211/cr-buildbucket.cfg

The bot works, and I'm using it manually.

Before I add it to the CQ, it needs  issue 868608  to be fixed.
Project Member

Comment 29 by bugdroid1@chromium.org, Sep 13

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/recipes-py/+/241d879a3dee0f9f1a0dc8933dc67f24e0d33391

commit 241d879a3dee0f9f1a0dc8933dc67f24e0d33391
Author: Stephen Martinis <martiniss@google.com>
Date: Thu Sep 13 22:09:44 2018

analyze: Output field default values

This will mean the 'recipes' field will always be present. The recipe
using this command expects this.

Bug:  863238 
Change-Id: I99a18af4d4a1b82d4effb9fe63712312aff18b47
Reviewed-on: https://chromium-review.googlesource.com/1222235
Auto-Submit: Stephen Martinis <martiniss@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>

[modify] https://crrev.com/241d879a3dee0f9f1a0dc8933dc67f24e0d33391/recipe_engine/analyze.py

Project Member

Comment 30 by bugdroid1@chromium.org, Sep 14

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/recipes-py/+/b8db432be1add133ef005e17d5729c537349f2c9

commit b8db432be1add133ef005e17d5729c537349f2c9
Author: Stephen Martinis <martiniss@google.com>
Date: Fri Sep 14 18:50:06 2018

analyze.py: Actually include default fields

The other CL I landed removed the argument being passed. The default is
still False, so no defaults were being printed.

TBR=iannucci

Bug:  863238 
Change-Id: I540a9001b123d1b491c082ef2429f84f4f1093ee
Reviewed-on: https://chromium-review.googlesource.com/1227058
Commit-Queue: Stephen Martinis <martiniss@chromium.org>
Reviewed-by: Stephen Martinis <martiniss@chromium.org>

[modify] https://crrev.com/b8db432be1add133ef005e17d5729c537349f2c9/recipe_engine/analyze.py
[modify] https://crrev.com/b8db432be1add133ef005e17d5729c537349f2c9/unittests/analyze_test.py

Project Member

Comment 31 by bugdroid1@chromium.org, Sep 14

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/46197069449ee2552baba9a55fae41e3b71e2876

commit 46197069449ee2552baba9a55fae41e3b71e2876
Author: Stephen Martinis <martiniss@chromium.org>
Date: Fri Sep 14 21:29:27 2018

Add recipes tester to cq

Only experimental for now

Bug:  863238 
Change-Id: Ia659f7722f355f23791160d71ac5dc0213166bc1
Reviewed-on: https://chromium-review.googlesource.com/1154129
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Stephen Martinis <martiniss@chromium.org>

[modify] https://crrev.com/46197069449ee2552baba9a55fae41e3b71e2876/infra/config/cq.cfg

Project Member

Comment 32 by bugdroid1@chromium.org, Sep 26

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/1c44f42f4396aed358d39f088ecbb3428afec278

commit 1c44f42f4396aed358d39f088ecbb3428afec278
Author: Stephen Martinis <martiniss@chromium.org>
Date: Wed Sep 26 19:43:10 2018

Make "Build Recipes Tester" a blocking CQ builder

Bug:  863238 
Change-Id: Ic3d5e5cf5066ba6c10bff699f7cc8cbcc3e19493
Reviewed-on: https://chromium-review.googlesource.com/1243887
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Stephen Martinis <martiniss@chromium.org>

[modify] https://crrev.com/1c44f42f4396aed358d39f088ecbb3428afec278/infra/config/cq.cfg

Status: Fixed (was: Assigned)
This is live. Closing.

Sign in to add a comment