Tryjobs no longer sending email |
||||
Issue descriptionThere have been a series of changes, both to the Chromite libraries as well as luci-notify, related to tryjobs and emails. We discovered yesterday, in testing, that we are no longer seeing email reports for trjobs. Now that email templates have been rolled out in luci-notify, need to pinpoint the root cause and resolve so we can being utilizing the new functionality.
,
Jun 1 2018
An example tryjob from TOT (no email expected): https://cros-goldeneye.corp.google.com/chromeos/healthmonitoring/buildDetails?buildbucketId=8944904522481847056 It's buildbucket entry includes: "parameters_json": "{\"builder_name\": \"Prod\", \"email_notify\": [{\"email\": \"dgarrett@google.com\"}], \"properties\": {\"cbb_branch\": \"master\", \"cbb_config\": \"success-build\", \"cbb_display_label\": \"tryjob\", \"cbb_email\": \"dgarrett@google.com\", \"cbb_extra_args\": [\"--remote-trybot\", \"-b\", \"master\"]}}", "id": "8944904522481847056", Here is an example tryjob that DID generate email: https://cros-goldeneye.corp.google.com/chromeos/healthmonitoring/buildDetails?buildbucketId=8946229564617436496 "parameters_json": "{\"builder_name\": \"Try\", \"email_notify\": [{\"email\": \"dgarrett@google.com\"}], \"properties\": {\"cbb_branch\": \"master\", \"cbb_config\": \"chromeos-infra-go-tryjob\", \"cbb_display_label\": \"tryjob\", \"cbb_email\": \"dgarrett@google.com\", \"cbb_extra_args\": [\"--remote-trybot\", \"-b\", \"master\"]}}",
,
Jun 1 2018
Two more jobs for comparison: This one is from April and did send email: https://apis-explorer.appspot.com/apis-explorer/?base=https://cr-buildbucket.appspot.com/_ah/api#p/buildbucket/v1/buildbucket.get?id=8950050549497544672&_h=1& This one was from yesterday and did not send email: https://apis-explorer.appspot.com/apis-explorer/?base=https://cr-buildbucket.appspot.com/_ah/api#p/buildbucket/v1/buildbucket.get?id=8944907622828644976&_h=3& -- Mike
,
Jun 1 2018
The following revision refers to this bug: https://chromium.googlesource.com/infra/luci/luci-go.git/+/7057582a2a99cbcd89108b3841d54d8a678f52ac commit 7057582a2a99cbcd89108b3841d54d8a678f52ac Author: Nodir Turakulov <nodir@google.com> Date: Fri Jun 01 19:06:41 2018 [luci-notify] Log build id in pubsub handler Log id of the build being processed in a pubsub handler. Bug: 848781 Change-Id: Ie7d69659debd3b41edcaf9236a735253bef1dedd Reviewed-on: https://chromium-review.googlesource.com/1082957 Reviewed-by: Mike Nichols <mikenichols@chromium.org> Commit-Queue: Nodir Turakulov <nodir@chromium.org> [modify] https://crrev.com/7057582a2a99cbcd89108b3841d54d8a678f52ac/luci_notify/notify/pubsub.go
,
Jun 1 2018
luci-notify now requires luci-notify.cfg to be in the project, even if it does not define notifiers. This is how it determines that luci-notify is enabled for the project. Please add one, perhaps with a comment. Sorry I didn't realize chromeos does no have luci-notify.cfg
,
Jun 1 2018
documentation update: https://chromium-review.googlesource.com/#/c/1082984/
,
Jun 1 2018
The following revision refers to this bug: https://chromium.googlesource.com/infra/luci/luci-go.git/+/a26863935bedd43c7183298b9dd6ccadb521070f commit a26863935bedd43c7183298b9dd6ccadb521070f Author: Nodir Turakulov <nodir@google.com> Date: Fri Jun 01 20:03:31 2018 [luci-notify] require luci-notify.cfg Update docs to explain that luci-notify.cfg is required for email templates to work, even if it does not define any notifiers. Bug: 848781 Change-Id: I5613288592adf526be47199898af5af8d8caae11 Reviewed-on: https://chromium-review.googlesource.com/1082984 Reviewed-by: Mike Nichols <mikenichols@chromium.org> Commit-Queue: Nodir Turakulov <nodir@chromium.org> [modify] https://crrev.com/a26863935bedd43c7183298b9dd6ccadb521070f/luci_notify/doc/email_templates.md
,
Jun 1 2018
The following revision refers to this bug: https://chrome-internal.googlesource.com/chromeos/manifest-internal/+/d5772ce62dec4dbd135cc2732f54aee13cc5d6b3 commit d5772ce62dec4dbd135cc2732f54aee13cc5d6b3 Author: Mike Nichols <mikenichols@chromium.org> Date: Fri Jun 01 22:12:40 2018
,
Jun 2 2018
I was hoping that implementing the luci-notify.cfg would resolve the issue, but the latest tryjob that I kicked off never delivered a notification: https://cros-goldeneye.corp.google.com/chromeos/healthmonitoring/buildDetails?buildbucketId=8944883052375054160 https://ci.chromium.org/p/chromeos/builds/b8944883052375054160# https://apis-explorer.appspot.com/apis-explorer/?base=https://cr-buildbucket.appspot.com/_ah/api#p/buildbucket/v1/buildbucket.get?id=8944883052375054160&_h=1& I checked both the request log and activity log for luci-notify but never found a matching entry for the buildbucket ID. Will continue to dig and see what I can find. -- Mike
,
Jun 4 2018
,
Jun 4 2018
Nodir discovered a permissions issue with the Chromeos config (chromeos is internal) and has updating the permissions to allow luci-config to access. Another job has been kicked off to test. -- Mike
,
Jun 4 2018
This is now resolved. There is an issue with the default template, that I just pushed a change to fix, but otherwise this is now working as expected. Thanks Nodir for your assistance in getting this resolved! -- Mike |
||||
►
Sign in to add a comment |
||||
Comment 1 by jclinton@chromium.org
, Jun 1 2018