New issue
Advanced search Search tips

Issue 851585 link

Starred by 2 users

Issue metadata

Status: Assigned
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Feature

Blocked on:
issue 917851
issue 853450

Blocking:
issue 854916
issue 904271
issue 912703
issue 851474



Sign in to add a comment

Index builds by commit position

Project Member Reported by no...@chromium.org, Jun 11 2018

Issue description

Extend SearchBuilds API to allow ordering builds by commit position.
Ingest commit positions from source manifests and include in buildbucket.v2.Build.output.gitiles_commits.

One of the usages is milo console. This enables displaying builds by commits that were actually checked out, as opposed to specified at build creation time. 

Design doc TBD.
 

Comment 1 by no...@chromium.org, Jun 11 2018

Blocking: 851474

Comment 2 by no...@chromium.org, Jun 16 2018

Labels: -Pri-2 Pri-1
Status: Started (was: Assigned)
Design: go/buildbucket-blame

Comment 3 by no...@chromium.org, Jun 16 2018

Blockedon: 853450

Comment 4 by no...@chromium.org, Jun 17 2018

Cc: st...@chromium.org

Comment 5 by no...@chromium.org, Jun 22 2018

Blocking: 854916
Project Member

Comment 6 by bugdroid1@chromium.org, Jun 29 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/ad39d954caf25286b018d97f4c520c63bc48b8f4

commit ad39d954caf25286b018d97f4c520c63bc48b8f4
Author: Nodir Turakulov <nodir@google.com>
Date: Fri Jun 29 20:23:52 2018

[buildbucket] Add BuildPredicate.build: BuildRange

Add a build range to BuildPredicate. Here range is defined on the order in
context, which is usually newest-to-oldest, but can be extended to order by
commit position. The new field works for both and can be used to find the
"previous" build for the purpose of computing blamelist.

Bug: 851585
Change-Id: Ifbfef84a5cf8711e940111bc4d0ac0849d87c8b9
Reviewed-on: https://chromium-review.googlesource.com/1120433
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Nodir Turakulov <nodir@chromium.org>

[modify] https://crrev.com/ad39d954caf25286b018d97f4c520c63bc48b8f4/buildbucket/proto/pb.discovery.go
[modify] https://crrev.com/ad39d954caf25286b018d97f4c520c63bc48b8f4/buildbucket/proto/rpc.pb.go
[modify] https://crrev.com/ad39d954caf25286b018d97f4c520c63bc48b8f4/buildbucket/proto/rpc.proto

Project Member

Comment 7 by bugdroid1@chromium.org, Jul 9

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/4b6fcd80de9b6bc4c356c23ab47469a730e71603

commit 4b6fcd80de9b6bc4c356c23ab47469a730e71603
Author: Nodir Turakulov <nodir@google.com>
Date: Mon Jul 09 18:56:52 2018

[buildbucket] Add Build.Output.gitiles_commit

Add a Build field that specifies the commit that was actually checked out and executed
against. This supersedes got_revision property. When GitilesCommit grows
"position" field, it will supersede got_cp as well.

This is different from go/buildbucket-blame: this CL does not include deps.
Using a map of gitiles commits makes it difficult to figure out which of
output commits correspond to the input commit, and overall which of output
commits to use for blamelist computation. It could be mitigated with
something like Build.Output.main_checkout field, but it brings its own
complexity: different builds are allowed to specify different values,
it is impossible to specify the main checkout in a field mask, so that
RPC response includes only the main checkout. Instead we start with simpler
singular commit. When/if we need to expose DEPS, later we can add
gitiles_commit_deps.

Single Build.Output.gitiles_commit enables blamelist computation: it is
git log between current and previous builds' output commits. The previous
build can be determined using SearchBuilds RPC. Today SearchBuilds RPC supports
only build creation order, which enables static blamelist computation.
Dynamic blamelist computation can be enabled by adding support for
commit position order in SearchBuilds RPC.

Allow filtering by output commit id or output commit ref.
Filtering by ref is important for builders that want to be able to build
for a arbitrary refs. For such builds, when computing blamelist, the previous
build should be determined with ref filter.

There is no known use case for filtering on input commit.

Bug: 851585

Change-Id: I0da37cc8ed38bfa9dc86beb1caac67db88127f66
Reviewed-on: https://chromium-review.googlesource.com/1119231
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>

[modify] https://crrev.com/4b6fcd80de9b6bc4c356c23ab47469a730e71603/buildbucket/proto/build.pb.go
[modify] https://crrev.com/4b6fcd80de9b6bc4c356c23ab47469a730e71603/buildbucket/proto/build.proto
[modify] https://crrev.com/4b6fcd80de9b6bc4c356c23ab47469a730e71603/buildbucket/proto/common.pb.go
[modify] https://crrev.com/4b6fcd80de9b6bc4c356c23ab47469a730e71603/buildbucket/proto/common.proto
[modify] https://crrev.com/4b6fcd80de9b6bc4c356c23ab47469a730e71603/buildbucket/proto/pb.discovery.go
[modify] https://crrev.com/4b6fcd80de9b6bc4c356c23ab47469a730e71603/buildbucket/proto/rpc.pb.go
[modify] https://crrev.com/4b6fcd80de9b6bc4c356c23ab47469a730e71603/buildbucket/proto/rpc.proto
[modify] https://crrev.com/4b6fcd80de9b6bc4c356c23ab47469a730e71603/buildbucket/proto/step.pb.go

Cc: chanli@chromium.org
Project Member

Comment 9 by bugdroid1@chromium.org, Aug 2

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/e0f06b32eb5175b4bbf8f619842b4e99c8240ce8

commit e0f06b32eb5175b4bbf8f619842b4e99c8240ce8
Author: Nodir Turakulov <nodir@google.com>
Date: Thu Aug 02 19:13:56 2018

[buildbucket] Implement SearchBuildsRequest.predicate.build

Implement build range filtering with creation time order.

R=tandrii@chromiumorg

Bug: 851585
Change-Id: I43a3d6dd998f074cd98736054ee474b8dc1b3c31
Reviewed-on: https://chromium-review.googlesource.com/1121249
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>

[modify] https://crrev.com/e0f06b32eb5175b4bbf8f619842b4e99c8240ce8/appengine/cr-buildbucket/v2/test/validation_test.py
[modify] https://crrev.com/e0f06b32eb5175b4bbf8f619842b4e99c8240ce8/appengine/cr-buildbucket/test/search_test.py
[modify] https://crrev.com/e0f06b32eb5175b4bbf8f619842b4e99c8240ce8/appengine/cr-buildbucket/model.py
[modify] https://crrev.com/e0f06b32eb5175b4bbf8f619842b4e99c8240ce8/appengine/cr-buildbucket/v2/api.py
[modify] https://crrev.com/e0f06b32eb5175b4bbf8f619842b4e99c8240ce8/appengine/cr-buildbucket/v2/test/api_test.py
[modify] https://crrev.com/e0f06b32eb5175b4bbf8f619842b4e99c8240ce8/appengine/cr-buildbucket/search.py
[modify] https://crrev.com/e0f06b32eb5175b4bbf8f619842b4e99c8240ce8/appengine/cr-buildbucket/v2/validation.py
[modify] https://crrev.com/e0f06b32eb5175b4bbf8f619842b4e99c8240ce8/appengine/cr-buildbucket/proto/rpc_pb2.py
[modify] https://crrev.com/e0f06b32eb5175b4bbf8f619842b4e99c8240ce8/appengine/cr-buildbucket/proto/rpc_prpc_pb2.py

Project Member

Comment 11 by bugdroid1@chromium.org, Aug 23

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/bfaf7c2feb3dec1c69bdb3fef3de79ccace62eff

commit bfaf7c2feb3dec1c69bdb3fef3de79ccace62eff
Author: Nodir Turakulov <nodir@google.com>
Date: Thu Aug 23 18:55:15 2018

[buildbucket] Add SearchBuildsPredicate.output_gitiles_commit validation

R=tandrii@chromium.org

Bug: 851585
Change-Id: Ifefc2d8450730926908998398f793ea9eb42163e
Reviewed-on: https://chromium-review.googlesource.com/1185882
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Nodir Turakulov <nodir@chromium.org>

[modify] https://crrev.com/bfaf7c2feb3dec1c69bdb3fef3de79ccace62eff/appengine/cr-buildbucket/v2/test/validation_test.py
[modify] https://crrev.com/bfaf7c2feb3dec1c69bdb3fef3de79ccace62eff/appengine/cr-buildbucket/v2/validation.py

Status: Assigned (was: Started)
currently not working on this because it is blocked on  issue 853450 
Blocking: 904271
Cc: no...@chromium.org iannu...@google.com
 Issue 721185  has been merged into this issue.
Blocking: 912703
Blockedon: 917851
Project Member

Comment 17 by bugdroid1@chromium.org, Jan 4

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/recipes-py/+/c1b615b6808b8aa8001b5e4e1d5d234233ad8266

commit c1b615b6808b8aa8001b5e4e1d5d234233ad8266
Author: Nodir Turakulov <nodir@google.com>
Date: Fri Jan 04 23:17:33 2019

[buildbucket] Add set_output_gitiles_commit

Add a recipe function to set output gitiles commit of a build.
It is currently implemented as an output property with a dummy step, because
there is currently no better way to propagate this to kitchen.
Most likely the impl will be replaced when recipe engine speaks build proto,
but the public API will remain the same.

R=iannucci@chromium.org

Bug: 851585
Change-Id: I7477919c5e8e1d7ca2f319774af89f2ebbb9a89d
Reviewed-on: https://chromium-review.googlesource.com/c/1390771
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>

[modify] https://crrev.com/c1b615b6808b8aa8001b5e4e1d5d234233ad8266/recipe_modules/buildbucket/examples/full.expected/basic-try.json
[modify] https://crrev.com/c1b615b6808b8aa8001b5e4e1d5d234233ad8266/README.recipes.md
[modify] https://crrev.com/c1b615b6808b8aa8001b5e4e1d5d234233ad8266/recipe_modules/buildbucket/api.py
[modify] https://crrev.com/c1b615b6808b8aa8001b5e4e1d5d234233ad8266/recipe_modules/buildbucket/examples/full.expected/basic-ci-win.json
[modify] https://crrev.com/c1b615b6808b8aa8001b5e4e1d5d234233ad8266/recipe_modules/buildbucket/examples/full.py

Sign in to add a comment