New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 706095 link

Starred by 8 users

Issue metadata

Status: Fixed
Owner:
Closed: May 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Feature

Blocking:
issue v8:6416
issue 813196
issue 826731



Sign in to add a comment

Give buildbucket plugin ability to fetch all builders in a given bucket

Project Member Reported by aga...@chromium.org, Mar 28 2017

Issue description

Rietveld simply lists all builders in a list of specified buckets. This is much lower maintenance than manually listing all of the builders in the bucket that should be manually triggerable.

Perhaps a syntax like
buildbucket.config:
[bucket "master.tryserver.chromium.win"]
  all = true


This can be done for buildbot (master.*) buckets via go/buildermap and chrome-infra-botmap.appspot.com (which could grow a simple new api). This can be done for luci buckets via https://apis-explorer.appspot.com/apis-explorer/?base=https://cr-buildbucket.appspot.com/_ah/api#p/swarmbucket/v1/swarmbucket.get_builders
 

Comment 1 by no...@chromium.org, Mar 28 2017

Labels: -Type-Bug Type-Feature
pro tip: strip "_ah/" from the API URL when running in prod

Comment 2 by rmis...@google.com, Mar 28 2017

[+1 with many 0s] for this feature request to reduce buildbucket.config maintenance pain
Blocking: v8:6416

Comment 4 by aga...@chromium.org, May 31 2017

 Issue 727684  has been merged into this issue.
 Issue 739549  has been merged into this issue.
Labels: -Milestone-Afterglow
Removing Milestone-Afterglow, as it has ceased to have meaning. More refined milestones may be added back in the near future.

Comment 7 by no...@chromium.org, Feb 22 2018

Blocking: 813196
Cc: dpranke@chromium.org
Owner: no...@chromium.org
Status: Started (was: Available)
starting LUCI-portion of this bug for  bug 813196 
Project Member

Comment 8 by bugdroid1@chromium.org, Feb 22 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/gerrit-plugins/buildbucket/+/c6d551e5a353161f64bf4ddc91f309600393a5df

commit c6d551e5a353161f64bf4ddc91f309600393a5df
Author: Nodir Turakulov <nodir@google.com>
Date: Thu Feb 22 18:58:33 2018

[gerrit-buildbucket] generalize baseURL

Remove /_ah/api/buildbucket/v1 from base URL so that the same baseURL
can be used for /_ah/api/swarmbucket/v1

Bug:  706095 
Change-Id: I658dd97c00ee144dbb191ad6753bf2449d0aa1d1
Reviewed-on: https://chromium-review.googlesource.com/932011
Reviewed-by: Aaron Gable <agable@chromium.org>

[modify] https://crrev.com/c6d551e5a353161f64bf4ddc91f309600393a5df/src/main/resources/static/cr-buildbucket-client.js
[modify] https://crrev.com/c6d551e5a353161f64bf4ddc91f309600393a5df/src/main/resources/static/cr-buildbucket-view.js

Comment 9 by no...@chromium.org, Feb 22 2018

Owner: ----
Status: Available (was: Started)
Today I will work on BQ export in Buildbucket, not this.
Labels: LUCI-Backlog
This is Gerrit plugin specific, but IMO it should be on LUCI backlog because it'll make config easier manage. For chromium, for example, config would just say 

bucket "luci.chromium.try"
and maybe "luci.chrome.try"
and won't require maintenance.

Comment 11 by no...@chromium.org, Mar 28 2018

Status: Started (was: Available)

Comment 12 by no...@chromium.org, Mar 28 2018

Blocking: 826731

Comment 13 by efoo@chromium.org, Mar 31 2018

Labels: -LUCI-Backlog LUCI-Chromium-CQSets LUCI-Blocker-Chromium-CQSets

Comment 14 by efoo@chromium.org, Mar 31 2018

Labels: -Pri-2 Pri-1
Project Member

Comment 16 by bugdroid1@chromium.org, Apr 18 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/gerrit-plugins/buildbucket/+/6e624621a1436c79168e0ceb242cd69ef38770ca

commit 6e624621a1436c79168e0ceb242cd69ef38770ca
Author: Nodir Turakulov <nodir@google.com>
Date: Wed Apr 18 23:48:42 2018

[buildbucket-gerrit] resort builds by id

The displayed builds may include builds from multiple patchsets
if the later patchsets have only trivial changes;
we want builds from each builder to be shown in time order,
even if they're from different patchsets.

An array formed by concatenating sorted arrays is not
guaranteed to be sorted. Sort it again.

Bug:  706095 
Change-Id: Ibf4a95fbbf215c6d242f3109a47361a1cde09d4f
Reviewed-on: https://chromium-review.googlesource.com/1017233
Reviewed-by: Quinten Yearsley <qyearsley@chromium.org>

[modify] https://crrev.com/6e624621a1436c79168e0ceb242cd69ef38770ca/test/cr-buildbucket-client_test.html
[modify] https://crrev.com/6e624621a1436c79168e0ceb242cd69ef38770ca/src/main/resources/static/cr-buildbucket-client.js
[modify] https://crrev.com/6e624621a1436c79168e0ceb242cd69ef38770ca/src/main/resources/static/cr-buildbucket-view.js

Project Member

Comment 17 by bugdroid1@chromium.org, Apr 19 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/gerrit-plugins/buildbucket/+/430662940fdced40e0b69fb1e567b491657e4862

commit 430662940fdced40e0b69fb1e567b491657e4862
Author: Nodir Turakulov <nodir@google.com>
Date: Thu Apr 19 16:05:05 2018

[buildbucket-gerrit] plumb plugin to client

cr-tryjob-picker relies on cr-buildbucket-client that requires a Gerrit
plugin object, but the picker doesn't initialize client's plugin.
Plumb plugin to client when used inside picker.

Bug:  706095 
Change-Id: I2e1a7d2833bc3d1c657ccf2cba11dada011a7cbf
Reviewed-on: https://chromium-review.googlesource.com/1018492
Reviewed-by: Quinten Yearsley <qyearsley@chromium.org>

[modify] https://crrev.com/430662940fdced40e0b69fb1e567b491657e4862/src/main/resources/static/cr-tryjob-picker.html
[modify] https://crrev.com/430662940fdced40e0b69fb1e567b491657e4862/src/main/resources/static/cr-buildbucket-view.html
[modify] https://crrev.com/430662940fdced40e0b69fb1e567b491657e4862/src/main/resources/static/cr-tryjob-picker.js

Project Member

Comment 18 by bugdroid1@chromium.org, Apr 19 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/gerrit-plugins/buildbucket/+/50ed25c6f5edfb5366b56dd9d2d2b038c94ed2c5

commit 50ed25c6f5edfb5366b56dd9d2d2b038c94ed2c5
Author: Nodir Turakulov <nodir@google.com>
Date: Thu Apr 19 16:05:11 2018

[buildbucket-gerrit] fix html import in picker

cr-tryjob-picker_test.html, that doesn't load cr-buildbucket-view.html,
fails because cr-buildbucket-client.html is not imported.
Add html-import of cr-buildbucket.client.html to cr-tryjob-picker.html

Bug:  706095 
Change-Id: If738f327450552c693866764a1e9d266b25a8d6f
Reviewed-on: https://chromium-review.googlesource.com/1018495
Reviewed-by: Quinten Yearsley <qyearsley@chromium.org>

[modify] https://crrev.com/50ed25c6f5edfb5366b56dd9d2d2b038c94ed2c5/src/main/resources/static/cr-tryjob-picker.html

Project Member

Comment 19 by bugdroid1@chromium.org, Apr 19 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/gerrit-plugins/buildbucket/+/682f3bc7cefa035d905daca3e9035a1ae92c3623

commit 682f3bc7cefa035d905daca3e9035a1ae92c3623
Author: Nodir Turakulov <nodir@google.com>
Date: Thu Apr 19 16:05:16 2018

[buildbucket-gerrit] fix _fetch

_fetch helper ignores method and payload. Fix that.

In case of an error, assume the response is JSON and use it as a promise
rejection reason.

Improve tests. Use console.warn instead of console.error because wct
tracks it and fails tests. Returns promises instead of accepting done().

Bug:  706095 
Change-Id: I994887ce191ebdeab427e9c2518f0ef8b2ad5be8
Reviewed-on: https://chromium-review.googlesource.com/1018496
Reviewed-by: Quinten Yearsley <qyearsley@chromium.org>

[modify] https://crrev.com/682f3bc7cefa035d905daca3e9035a1ae92c3623/src/main/resources/static/cr-buildbucket-client.js
[modify] https://crrev.com/682f3bc7cefa035d905daca3e9035a1ae92c3623/test/cr-buildbucket-client_test.html

Project Member

Comment 20 by bugdroid1@chromium.org, Apr 19 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/gerrit-plugins/buildbucket/+/d4f45887c6e831c0dab6cd0b54af85ad454c10d1

commit d4f45887c6e831c0dab6cd0b54af85ad454c10d1
Author: Nodir Turakulov <nodir@google.com>
Date: Thu Apr 19 18:04:34 2018

[buildbucket-gerrit] refactor build scheduling

Rename trigger -> schedule.

Change scheduling API to deal with build requests and results,
not HTTP JSON request/response payloads.

Use _fetch to schedule builds.

Do not specify builder tag when scheduling a build.
Buildbucket server adds it automatically.

In async tests, return promises instead of calling done().

Bug:  706095 
Change-Id: Iac77dacba565919766fca138a21ff94c0ce08410
Reviewed-on: https://chromium-review.googlesource.com/1017915
Reviewed-by: Quinten Yearsley <qyearsley@chromium.org>

[modify] https://crrev.com/d4f45887c6e831c0dab6cd0b54af85ad454c10d1/src/main/resources/static/cr-buildbucket-client.js
[modify] https://crrev.com/d4f45887c6e831c0dab6cd0b54af85ad454c10d1/test/cr-tryjob-picker_test.html
[modify] https://crrev.com/d4f45887c6e831c0dab6cd0b54af85ad454c10d1/src/main/resources/static/cr-tryjob-picker.js

Comment 21 by no...@chromium.org, Apr 19 2018

Owner: no...@chromium.org
Project Member

Comment 22 by bugdroid1@chromium.org, Apr 20 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/0fab3e035afc10fc48487ac00ba24821eb3c6a0f

commit 0fab3e035afc10fc48487ac00ba24821eb3c6a0f
Author: Nodir Turakulov <nodir@google.com>
Date: Fri Apr 20 20:05:19 2018

[buildbucket] get_builders: add bucket filtering

Gerrit-buildbucket needs to load all builders of few buckets.
Add bucket filtering to the get_builders RPC.

R=tandrii@chromium.org

Bug:  706095 
Change-Id: I612b8e26b5bc7b2e908f3c03ef63954e2a200c9d
Reviewed-on: https://chromium-review.googlesource.com/1020076
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Nodir Turakulov <nodir@chromium.org>

[modify] https://crrev.com/0fab3e035afc10fc48487ac00ba24821eb3c6a0f/appengine/cr-buildbucket/swarming/test/swarmbucket_api_test.py
[modify] https://crrev.com/0fab3e035afc10fc48487ac00ba24821eb3c6a0f/appengine/cr-buildbucket/config.py
[modify] https://crrev.com/0fab3e035afc10fc48487ac00ba24821eb3c6a0f/appengine/cr-buildbucket/swarming/swarmbucket_api.py
[modify] https://crrev.com/0fab3e035afc10fc48487ac00ba24821eb3c6a0f/appengine/cr-buildbucket/test/config_test.py

Comment 23 by no...@chromium.org, Apr 24 2018

Cc: estaab@chromium.org tandrii@chromium.org iannucci@chromium.org efoo@chromium.org
 Issue 826731  has been merged into this issue.
Project Member

Comment 24 by bugdroid1@chromium.org, Apr 25 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/gerrit-plugins/buildbucket/+/5277eec06ab255b28244c8348f2300486c016c13

commit 5277eec06ab255b28244c8348f2300486c016c13
Author: Nodir Turakulov <nodir@google.com>
Date: Wed Apr 25 03:17:25 2018

[buildbucket-gerrit] load builders from server

LUCI builders are defined in a Buildbucket server configuration
and Buildbucket exposes them via an API. Instead of requiring
users to duplicate builders in gerrit-buildbucket plugin and
buildbucket server, use the API to load the builders.

Load the list of builders as soon as we know the buckets from
the plugin config. Cache the list of builders per bucket for
10m in localStorage.

Some of the builders returned by the API are not prod yet.
Whether a builder is prod or not can be determined by talking
to luci-migration.appspot.com. This will be done in a separate CL.

Bug:  706095 
Change-Id: I87e1ccdfb011923a8763962d8f1a2800ccc5bf67
Reviewed-on: https://chromium-review.googlesource.com/1022030
Reviewed-by: Quinten Yearsley <qyearsley@chromium.org>

[modify] https://crrev.com/5277eec06ab255b28244c8348f2300486c016c13/test/cr-buildbucket-client_test.html
[modify] https://crrev.com/5277eec06ab255b28244c8348f2300486c016c13/src/main/resources/static/cr-tryjob-picker.html
[modify] https://crrev.com/5277eec06ab255b28244c8348f2300486c016c13/src/main/resources/static/cr-buildbucket-client.js
[modify] https://crrev.com/5277eec06ab255b28244c8348f2300486c016c13/test/cr-tryjob-picker_test.html
[modify] https://crrev.com/5277eec06ab255b28244c8348f2300486c016c13/src/main/resources/static/cr-tryjob-picker.js

Status: Fixed (was: Started)
Buildbucket-Gerrit now displays LUCI builders of all buckets in buildbucket.config
@nodir - does that mean we can delete the entries in the buildbucket.config (and I can delete the code to generate that from MB)?
yes! but let's test it on infra.git first https://chromium-review.googlesource.com/c/infra/infra/+/1047625

Sign in to add a comment