HTTP 500 was due to a bug https://chromium-review.googlesource.com/c/infra/infra/+/806418
fixed now

the problem was that the service account wasn't in the swarming-users group. I've added it to project-v8-tryjob-access and included project-v8-tryjob-access in swarming-users.

Note, however, that the LUCI bot should not using pre-deployed credentials. Instead the builder should specify the service account (example: https://chromium.googlesource.com/chromium/src/+/infra/config/cr-buildbucket.cfg#264). See how it is used in recipes
https://chromium-review.googlesource.com/c/chromium/tools/build/+/767768/18/scripts/slave/recipe_modules/chromium_tests/api.py

> the problem was that the service account wasn't in the swarming-users group. I've added it to project-v8-tryjob-access and included project-v8-tryjob-access in swarming-users.

Thank you. But how did it work on Buildbot?

> Note, however, that the LUCI bot should not using pre-deployed credentials. Instead the builder should specify the service account (example: https://chromium.googlesource.com/chromium/src/+/infra/config/cr-buildbucket.cfg#264).

Should we use the same service account for V8? Should we create a new one? Must it be on chops-service-accounts?

> See how it is used in recipes
https://chromium-review.googlesource.com/c/chromium/tools/build/+/767768/18/scripts/slave/recipe_modules/chromium_tests/api.py

I don't see any service accounts specified in this code. Will buildbucket module use one specified in cr-buildbucket.cfg by default?

>> the problem was that the service account wasn't in the swarming-users group. I've added it to project-v8-tryjob-access and included project-v8-tryjob-access in swarming-users.
> Thank you. But how did it work on Buildbot?

I've realized that the error is about Buildbucket not being able to trigger LUCI builds (swarming tasks) using v8-bot service account. On Buildbot we were scheduling Buildbot builds. Now I understand the difference.

OTH, the builds are still failing to trigger other builds [1][2] even though you've mentioned that you've added v8-bot to swarming-users.

[1]: https://logs.chromium.org/v/?s=v8%2Fbuildbucket%2Fcr-buildbucket.appspot.com%2F8961118436103853312%2F%2B%2Fsteps%2Ftrigger%2F0%2Flogs%2Fjson.output%2F0
[2]: https://logs.chromium.org/v/?s=v8%2Fbuildbucket%2Fcr-buildbucket.appspot.com%2F8961118440424342272%2F%2B%2Fsteps%2Ftrigger%2F0%2Flogs%2Fjson.output%2F0

> Should we use the same service account for V8? Should we create a new one? Must it be on chops-service-accounts?


see "Authentication" section https://docs.google.com/document/d/1dGXRvz1QJh-tNWppG5xElckG25ag0rQw7JJ5F8zOKyc/edit#heading=h.c31nfh2j8gjl

> I don't see any service accounts specified in this code. Will buildbucket module use one specified in cr-buildbucket.cfg by default?

ah, right. Buildbucket binary now automatically picks up the auth token floating in the LUCI environment. It will be there once the service account is specified in the builder

> OTH, the builds are still failing to trigger other builds [1][2] even though you've mentioned that you've added v8-bot to swarming-users

fixing: https://chrome-internal-review.googlesource.com/c/infradata/config/+/520799

I've created a service account for V8, but when I've added it to the cr-buildbucket.cfg, CQ stopped being able to trigger LUCI builds:

HttpError 403 when requesting https://cr-buildbucket.appspot.com/api/buildbucket/v1/builds/batch?alt=json returned "Auth error while calling swarming on behalf of user:5071639625-1lppvbtck1morgivc6sq4dul7klu27sd@developer.gserviceaccount.com: Failed to call https://chromium-swarm.appspot.com/_ah/api/swarming/v1/tasks/new: HTTP 403"

I thought this service account is only used by the swarming task. Why does it affect CQ?

Reland: https://crrev.com/c/808844 (but we shouldn't land it until it's clear how to make sure that it does not break CQ)

P.S. After looking at your CL, I've realized that you've also created service accounts for V8, so I've deleted the one I created and updated the re-land CL to use yours.

P.P.S. Hmm, after looking at https://pantheon.corp.google.com/iam-admin/serviceaccounts/project?project=chops-service-accounts, looks like you have not created the accounts yet. I'll create them.

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infradata/config/+/47722e85c5f79d9073902036664f7f3dc83f872b

commit 47722e85c5f79d9073902036664f7f3dc83f872b
Author: Nodir Turakulov <nodir@google.com>
Date: Tue Dec 05 15:13:10 2017

just landed https://chrome-internal-review.googlesource.com/c/infradata/config/+/520799
i guess i was waiting for Vadim's approval

It seems this has also affected talking to the isolate server and "isolate" step is now failing on LUCI builders:
 - https://ci.chromium.org/p/v8/builds/b8961034537322898400
 - https://ci.chromium.org/p/v8/builds/b8961034395239410704
 - https://ci.chromium.org/p/v8/builds/b8961034362103782656
 - https://ci.chromium.org/p/v8/builds/b8961034534216977760

Do I need to add this new account to some group for access to the isolate server?

done https://chrome-infra-auth.appspot.com/auth/change_log?auth_db_rev=6590

After this it failed with the following error:

  User user:v8-try-builder@chops-service-accounts.iam.gserviceaccount.com cannot add builds to bucket luci.v8.try

so I've added it to the role in https://crrev.com/c/808975, but it now fails [1]with

  Auth error while calling swarming on behalf of user:v8-try-builder@chops-service-accounts.iam.gserviceaccount.com: Failed to call https://chromium-swarm.appspot.com/_ah/api/swarming/v1/tasks/new: HTTP 403

Not sure what this means...

[1]: https://logs.chromium.org/v/?s=v8%2Fbuildbucket%2Fcr-buildbucket.appspot.com%2F8961029246912113504%2F%2B%2Fsteps%2Ftrigger%2F0%2Flogs%2Fjson.output%2F0

Btw, I am collecting all steps that we've taken so far in the LUCI onboarding doc: https://docs.google.com/document/d/1dGXRvz1QJh-tNWppG5xElckG25ag0rQw7JJ5F8zOKyc/edit#heading=h.fwcobzars7c2 (see the bullet-point list in the bottom of the section).

i've added the new service accounts to project-v8-tryjob-access. This should fix the problem#2 in c#13.

reverting https://crrev.com/c/808975: see reasons in https://chromium-review.googlesource.com/c/v8/v8/+/809345

The following revision refers to this bug:
  https://chromium.googlesource.com/v8/v8.git/+/a6790e0d6426fdc06fa6e943c0c793b9add0be45

commit a6790e0d6426fdc06fa6e943c0c793b9add0be45
Author: Sergiy Byelozyorov <sergiyb@chromium.org>
Date: Wed Dec 06 09:06:39 2017

Whitespace CL

TBR=machenbach@chromium.org

Bug:  chromium:791583 
Change-Id: I66e5ae2f5d0c8d26bf2a9dbe0a2cc6022a4c2b36
Reviewed-on: https://chromium-review.googlesource.com/808973
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49881}
[modify] https://crrev.com/a6790e0d6426fdc06fa6e943c0c793b9add0be45/tools/whitespace.txt

> This should fix the problem#2 in c#13.

It's still fails with the same error: https://logs.chromium.org/v/?s=v8%2Fbuildbucket%2Fcr-buildbucket.appspot.com%2F8960965322985357776%2F%2B%2Fsteps%2Ftrigger%2F0%2Flogs%2Fjson.output%2F0.

The following revision refers to this bug:
  https://chromium.googlesource.com/v8/v8.git/+/f9aacf158404e65e5316c652066ca00eb60b226d

commit f9aacf158404e65e5316c652066ca00eb60b226d
Author: Sergiy Byelozyorov <sergiyb@chromium.org>
Date: Wed Dec 06 11:30:18 2017

Whitespace CL

R=machenbach@chromium.org

Bug:  chromium:791583 
Change-Id: I84e2949b4b6eb79308218bd1fa073abbb58eaf84
Reviewed-on: https://chromium-review.googlesource.com/809109
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49889}
[modify] https://crrev.com/f9aacf158404e65e5316c652066ca00eb60b226d/tools/whitespace.txt

i was staring at ACLs, groups and logs for a half an hour and still cannot understand why v8-try-builder@chops-service-accounts.iam.gserviceaccount.com cannot schedule builds on luci.v8.try
Vadim, please help.

one concern:
https://pantheon.corp.google.com/logs/viewer?project=chromium-swarm&minLogLevel=0&expandAll=false&timestamp=2017-12-06T12%3A37%3A48.848383000Z&dateRangeStart=2017-12-06T07%3A52%3A14.065Z&dateRangeEnd=2017-12-06T13%3A52%3A14.065Z&interval=PT6H&resource=gae_app&logName=projects%2Fchromium-swarm%2Flogs%2Fappengine.googleapis.com%252Frequest_log&filters=status%3A403&filters=text%3Av8-try-builder%40chops-service-accounts.iam.gserviceaccount.com&advancedFilter=resource.type%3D%22gae_app%22%0Aresource.labels.zone%3D%22us9%22%0Aresource.labels.project_id%3D%22chromium-swarm%22%0Aresource.labels.version_id%3D%223254-d863d1f%22%0Aresource.labels.module_id%3D%22default%22%0Atimestamp%3D%222017-12-06T12%3A15%3A57.868844000Z%22%0AinsertId%3D%225a27df7e000260e609779d07%22

does not say WHY access is denied

No additional messages like that mean the request failed on the outer most coarse grain check. I think v8-try-builder@chops-service-accounts.iam.gserviceaccount.com should be added to https://chrome-infra-auth.appspot.com/auth/groups/chromium-swarm-users

Re #1: I don't see project-v8-tryjob-access in chromium-swarm-users, that's why it doesn't work. I added it now in https://chrome-infra-auth.appspot.com/auth/change_log?auth_db_rev=6620, so presumably it should work.

I added project-v8-tryjobs-access to deprecated swarming-users by habit

Ugh... I didn't know it still exists. We should kill it...

Deleted deprecated groups.

FWIU this is fixed, please reopen if it is not

The following revision refers to this bug:
  https://chromium.googlesource.com/v8/v8.git/+/aa670efb40ca42a5471971a9c248dfa086f9cd55

commit aa670efb40ca42a5471971a9c248dfa086f9cd55
Author: Sergiy Byelozyorov <sergiyb@chromium.org>
Date: Thu Dec 07 17:45:10 2017

Whitespace CL

TBR=sergiyb@chromium.org

Bug:  chromium:791583 
Change-Id: I34ca6f966276277a69194c52fabb475826a034ae
Reviewed-on: https://chromium-review.googlesource.com/813854
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49941}
[modify] https://crrev.com/aa670efb40ca42a5471971a9c248dfa086f9cd55/tools/whitespace.txt