New issue
Advanced search Search tips

Issue 843718 link

Starred by 2 users

Issue metadata

Status: Available
Owner: ----
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Feature

Blocking:
issue 843720



Sign in to add a comment

Buildbucket PubSub in v2 format

Project Member Reported by no...@chromium.org, May 16 2018

Issue description

Add a new global pubsub topic that uses v2 format for build. 
Each pubsub message data must be a protobuf message in binary format.
The message must contain a buildbucket.v2.Build message.

Relevant code:
https://chromium.googlesource.com/infra/infra/+/76252a520b2b120034df2ad5f73a7c6ae9e6db78/appengine/cr-buildbucket/notifications.py#72
 

Comment 1 by no...@chromium.org, May 16 2018

Blocking: 843720

Comment 2 by no...@chromium.org, May 16 2018

Labels: buildbucket-v2

Comment 3 by no...@chromium.org, May 18 2018

Blocking: 816652
Cc: jclinton@chromium.org
Unrelated to OKRs, CrOS very urgently (P0.5) needs a feature in luci-notify that I’m working on now, and it occurred to me that this bug blocks the feature because that feature needs steps in PubSub notifications. jchinlee, I’m fine with either of us implementing PubSub v2 in the next few days. I have cycles for this. WDYT?
If you have cycles, feel free. I was under the impression that there were some questions in how to proceed regarding pubsub v2; are we settled on that?

Comment 5 by no...@chromium.org, May 18 2018

Cc: jchin...@chromium.org
Owner: no...@chromium.org
Are you referring to the security concerns in bug 843720? They are specific to user-defined PubSub topics and aren’t very related to the message format. This bug is not blocked by that

Comment 6 by no...@chromium.org, May 21 2018

Status: Started (was: Assigned)
Project Member

Comment 7 by bugdroid1@chromium.org, May 22 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/1d3b9f75c9b8e8de8a96c4429b50f2ac455989e8

commit 1d3b9f75c9b8e8de8a96c4429b50f2ac455989e8
Author: Nodir Turakulov <nodir@google.com>
Date: Tue May 22 16:45:56 2018

[milo] Check Buildbucket PubSub version

https://chromium-review.googlesource.com/c/infra/infra/+/1069124
introduces "version" pubsub message attribute that indicates the version of the
pubsub message data. In v2, it is a binary protobuf.

Check version in Milo. Ignore v2 until crbug.com/826006 is fixed.

Bug: 843718
Change-Id: I6cf1d6596af57b3d09e4b76e1b78171a26f0d2a0
Reviewed-on: https://chromium-review.googlesource.com/1069221
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Ryan Tseng <hinoka@chromium.org>

[modify] https://crrev.com/1d3b9f75c9b8e8de8a96c4429b50f2ac455989e8/milo/buildsource/buildbucket/pubsub.go

Project Member

Comment 8 by bugdroid1@chromium.org, May 22 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/6680792bf79956789636765444b20bc2c0ad17bb

commit 6680792bf79956789636765444b20bc2c0ad17bb
Author: Nodir Turakulov <nodir@google.com>
Date: Tue May 22 17:37:05 2018

[luci-migration] Ignore v2 PubSub messages

https://chromium-review.googlesource.com/c/infra/infra/+/1069124
duplicates all pubsub messages in the global PubSub topic in two formats.
Ignore v2 messages.

Bug: 843718
Change-Id: Iedb3be6c227bada8b96adfa4e170949fecff9036
Reviewed-on: https://chromium-review.googlesource.com/1069253
Reviewed-by: Jao-ke Chin-Lee <jchinlee@chromium.org>
Commit-Queue: Nodir Turakulov <nodir@chromium.org>

[modify] https://crrev.com/6680792bf79956789636765444b20bc2c0ad17bb/go/src/infra/appengine/luci-migration/app/handlers.go

Project Member

Comment 9 by bugdroid1@chromium.org, May 22 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/4abc5ab08202bf1ac2fdb2d374afcf720f23a91a

commit 4abc5ab08202bf1ac2fdb2d374afcf720f23a91a
Author: Nodir Turakulov <nodir@google.com>
Date: Tue May 22 21:18:27 2018

[luci-notify] Ignore v2 PubSub messages.

https://chromium-review.googlesource.com/c/infra/infra/+/1069124
duplicates all pubsub messages in the global PubSub topic in two formats.
Ignore v2 messages until v2 is fully deployed.

While at it, also add pubsub message deduplication.

Bug: 843718
Change-Id: I5228e99f9f789643c1ff6a29fd422769afb18208
Reviewed-on: https://chromium-review.googlesource.com/1069289
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
Commit-Queue: Nodir Turakulov <nodir@chromium.org>

[modify] https://crrev.com/4abc5ab08202bf1ac2fdb2d374afcf720f23a91a/luci_notify/frontend/main.go
[modify] https://crrev.com/4abc5ab08202bf1ac2fdb2d374afcf720f23a91a/luci_notify/notify/pubsub.go

Comment 10 by no...@chromium.org, May 23 2018

Blocking: -816652

Comment 11 by no...@chromium.org, May 23 2018

Blocking: -843720
Cc: -jclinton@chromium.org no...@chromium.org
Labels: -Pri-1 Pri-2
Owner: ----
Status: Available (was: Started)
this is no longer urgent because
- https://chromium-review.googlesource.com/c/infra/infra/+/1068687/8#message-5232aafbf83d8cdeaca2d856f4ab30e94ed2c14b
- https://bugs.chromium.org/p/chromium/issues/detail?id=816652#c53

Comment 12 by no...@chromium.org, May 23 2018

Blocking: 843720

Comment 13 by st...@chromium.org, May 23 2018

Cc: st...@chromium.org

Comment 14 by no...@chromium.org, Jun 22 2018

Components: Infra>Platform>Buildbucket

Comment 15 by no...@chromium.org, Jun 22 2018

Components: -Infra>Platform>Buildbucket>Swarmbucket

Sign in to add a comment