context: bug 832733 this bug is scoped to full implementation of SearchBuilds RPC
Issue 590475 has been merged into this issue.
The following revision refers to this bug: https://chromium.googlesource.com/infra/luci/luci-go.git/+/8fe9c7388dae6155fc58ba2546a4430aedbcbae9 commit 8fe9c7388dae6155fc58ba2546a4430aedbcbae9 Author: Nodir Turakulov <nodir@google.com> Date: Thu May 24 23:31:52 2018 [buildbucket][v2] Add status_reason fields. Add Build.status_reason fields, CancelationReason and InfraFailureReason messages. Bug: 843712 Change-Id: I005fd3a0d678ef4e5d1ac5636c2e5a6688816e49 Reviewed-on: https://chromium-review.googlesource.com/1072793 Commit-Queue: Nodir Turakulov <nodir@chromium.org> Reviewed-by: Vadim Shtayura <vadimsh@chromium.org> [modify] https://crrev.com/8fe9c7388dae6155fc58ba2546a4430aedbcbae9/buildbucket/proto/build.pb.go [modify] https://crrev.com/8fe9c7388dae6155fc58ba2546a4430aedbcbae9/buildbucket/proto/build.proto [modify] https://crrev.com/8fe9c7388dae6155fc58ba2546a4430aedbcbae9/buildbucket/proto/pb.discovery.go
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/3f5294472241ef835c3bd9b627c41d892ee21649 commit 3f5294472241ef835c3bd9b627c41d892ee21649 Author: Nodir Turakulov <nodir@google.com> Date: Fri May 25 17:04:04 2018 [buildbucket] Add V2->V1 status conversion. The new fields are being added in https://chromium-review.googlesource.com/c/infra/luci/luci-go/+/1072793 Bug: 843712 Change-Id: Icfc1791a80e60c43be4abd41679f7a69fcab26e6 Reviewed-on: https://chromium-review.googlesource.com/1072876 Reviewed-by: Vadim Shtayura <vadimsh@chromium.org> Commit-Queue: Nodir Turakulov <nodir@chromium.org> [modify] https://crrev.com/3f5294472241ef835c3bd9b627c41d892ee21649/appengine/cr-buildbucket/proto/build_pb2.py [modify] https://crrev.com/3f5294472241ef835c3bd9b627c41d892ee21649/appengine/cr-buildbucket/v2/test/builds_test.py [modify] https://crrev.com/3f5294472241ef835c3bd9b627c41d892ee21649/appengine/cr-buildbucket/v2/builds.py
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/1a6e6342a460c51e90b3e6ff8d48d169388207e9 commit 1a6e6342a460c51e90b3e6ff8d48d169388207e9 Author: Nodir Turakulov <nodir@google.com> Date: Fri May 25 20:46:24 2018 [buildbucket] Inline v2/errors.py into v2/builds.py errors.py is overloaded. There is another errors.py in buildbucket and other v2 scripts need to import it. Bug: 843712 Change-Id: I3eaa5fec6dd4350136c06abe5c17023b8fabc3f4 Reviewed-on: https://chromium-review.googlesource.com/1073094 Reviewed-by: Vadim Shtayura <vadimsh@chromium.org> Commit-Queue: Nodir Turakulov <nodir@chromium.org> [modify] https://crrev.com/1a6e6342a460c51e90b3e6ff8d48d169388207e9/appengine/cr-buildbucket/v2/__init__.py [modify] https://crrev.com/1a6e6342a460c51e90b3e6ff8d48d169388207e9/appengine/cr-buildbucket/v2/builds.py [modify] https://crrev.com/1a6e6342a460c51e90b3e6ff8d48d169388207e9/appengine/cr-buildbucket/v2/test/builds_test.py [modify] https://crrev.com/1a6e6342a460c51e90b3e6ff8d48d169388207e9/appengine/cr-buildbucket/bq.py [delete] https://crrev.com/cd80bece6959060fc97b71f04212e55095b9be09/appengine/cr-buildbucket/v2/errors.py
The following revision refers to this bug: https://chromium.googlesource.com/infra/luci/luci-go.git/+/0d5f34970ea158c53f555356c3e8f8658b3300ca commit 0d5f34970ea158c53f555356c3e8f8658b3300ca Author: Nodir Turakulov <nodir@google.com> Date: Fri May 25 20:49:23 2018 [scheduler] Index builds by git hash Buildbucket SearchBuilds V2 RPC supports searching by git commit id, but not (gitiles_host, gitiles_project, commit_hash). Add new tag "buildset:commit/git/{commit hash}". ScheduleBuild V2 will take care of this automatically and transparently. Bug: 843712 Change-Id: I4456aa0ef30f276f3e9a39745e63a8aca5be4ea6 Reviewed-on: https://chromium-review.googlesource.com/1073506 Commit-Queue: Nodir Turakulov <nodir@chromium.org> Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org> [modify] https://crrev.com/0d5f34970ea158c53f555356c3e8f8658b3300ca/scheduler/appengine/engine/policy/request_builder.go [modify] https://crrev.com/0d5f34970ea158c53f555356c3e8f8658b3300ca/scheduler/appengine/engine/policy/request_builder_test.go
The following revision refers to this bug: https://chromium.googlesource.com/infra/luci/luci-go.git/+/70abb2032a6d62bd83578f611dfd8e381a0eea37 commit 70abb2032a6d62bd83578f611dfd8e381a0eea37 Author: Nodir Turakulov <nodir@google.com> Date: Fri May 25 22:23:33 2018 [buildbucket][v2] Add Search RPC v2 to proto Add Search RPC v2 and the messages it needs. It will be implemented in Python. Bug: 843712 Change-Id: Ic794d4b09e30582d5e8ea8409d2f3fa660e8e5f1 Reviewed-on: https://chromium-review.googlesource.com/1072633 Reviewed-by: Vadim Shtayura <vadimsh@chromium.org> Commit-Queue: Nodir Turakulov <nodir@chromium.org> [modify] https://crrev.com/70abb2032a6d62bd83578f611dfd8e381a0eea37/buildbucket/proto/build.pb.go [modify] https://crrev.com/70abb2032a6d62bd83578f611dfd8e381a0eea37/buildbucket/proto/common.pb.go [modify] https://crrev.com/70abb2032a6d62bd83578f611dfd8e381a0eea37/buildbucket/proto/common.proto [modify] https://crrev.com/70abb2032a6d62bd83578f611dfd8e381a0eea37/buildbucket/proto/pb.discovery.go [modify] https://crrev.com/70abb2032a6d62bd83578f611dfd8e381a0eea37/buildbucket/proto/rpc.pb.go [modify] https://crrev.com/70abb2032a6d62bd83578f611dfd8e381a0eea37/buildbucket/proto/rpc.proto
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/fb823fd6c8cdf5efa104b57189c36cda6e307959 commit fb823fd6c8cdf5efa104b57189c36cda6e307959 Author: Nodir Turakulov <nodir@google.com> Date: Wed May 30 18:13:57 2018 [buildbucket][v2] Implement build search by V2 status. status_v2->status_v1 function depends on infra_failure_reason. In particular, V2 status==INFRA_FAILURE might correspond V1 cancelation_reason=TIMEOUT. This means it is impossible to take a V2 status and translate it to Datastore query over V1 status. Index Build entities with status_v2. We might not need builds.status_to_v1, it might have been a mistake. Bug: 843712 Change-Id: I6e3adc0c57122c302cfd69aa7ff2a20059cacf7c Reviewed-on: https://chromium-review.googlesource.com/1073497 Commit-Queue: Nodir Turakulov <nodir@chromium.org> Reviewed-by: Vadim Shtayura <vadimsh@chromium.org> [modify] https://crrev.com/fb823fd6c8cdf5efa104b57189c36cda6e307959/appengine/cr-buildbucket/service.py [modify] https://crrev.com/fb823fd6c8cdf5efa104b57189c36cda6e307959/appengine/cr-buildbucket/test/service_test.py [modify] https://crrev.com/fb823fd6c8cdf5efa104b57189c36cda6e307959/appengine/cr-buildbucket/v2/__init__.py [modify] https://crrev.com/fb823fd6c8cdf5efa104b57189c36cda6e307959/appengine/cr-buildbucket/model.py [modify] https://crrev.com/fb823fd6c8cdf5efa104b57189c36cda6e307959/appengine/cr-buildbucket/v2/builds.py
The following revision refers to this bug: https://chromium.googlesource.com/infra/luci/luci-py.git/+/1d170bd88b6e905bb4bc680c1d9dfc6d31b5ad1d commit 1d170bd88b6e905bb4bc680c1d9dfc6d31b5ad1d Author: Nodir Turakulov <nodir@google.com> Date: Thu May 31 20:36:35 2018 [protoutil] add field_masks.Mask.submask Add field_masks.Mask.submask that returns a descendant by path. Bug: 843712 Change-Id: I56c3921e9a6a7ab5c844dd3f142e38125b362052 Reviewed-on: https://chromium-review.googlesource.com/1081153 Commit-Queue: Nodir Turakulov <nodir@chromium.org> Reviewed-by: Vadim Shtayura <vadimsh@chromium.org> [modify] https://crrev.com/1d170bd88b6e905bb4bc680c1d9dfc6d31b5ad1d/appengine/components/components/protoutil/field_masks.py [modify] https://crrev.com/1d170bd88b6e905bb4bc680c1d9dfc6d31b5ad1d/appengine/components/components/protoutil/field_masks_test.py
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/c3a1311956d78084416e2e9f60b4b1707e7804d1 commit c3a1311956d78084416e2e9f60b4b1707e7804d1 Author: Nodir Turakulov <nodir@google.com> Date: Sat Jun 02 02:11:19 2018 [buildbucket][v2] Implement SearchBuilds V2 RPC This adds SearchBuild V2 RPC based on the existing V1 implementation. Responses include steps if the request field mask asks for them. Later we will replace this implementation with Go. It will be faster. Move default field masks to default_field_masks.py. Keep them there as a dict {response message class -> protoutil.Mask}. Remove default_mask parameter from api_method@ decorator. Use _v1 and _v2 suffixes to clearly indicate whether things are v1 or v2. Allow STATUS_UNSPECIFIED in SearchQuery.status. Treat it as None. Bug: 843712 Change-Id: I1b5cc679384e012e6f964bd32b29127d0c3de8a8 Reviewed-on: https://chromium-review.googlesource.com/1080988 Commit-Queue: Nodir Turakulov <nodir@chromium.org> Reviewed-by: Vadim Shtayura <vadimsh@chromium.org> [modify] https://crrev.com/c3a1311956d78084416e2e9f60b4b1707e7804d1/appengine/cr-buildbucket/service.py [modify] https://crrev.com/c3a1311956d78084416e2e9f60b4b1707e7804d1/appengine/cr-buildbucket/v2/api.py [modify] https://crrev.com/c3a1311956d78084416e2e9f60b4b1707e7804d1/appengine/cr-buildbucket/buildtags.py [modify] https://crrev.com/c3a1311956d78084416e2e9f60b4b1707e7804d1/appengine/cr-buildbucket/v2/test/api_test.py [add] https://crrev.com/c3a1311956d78084416e2e9f60b4b1707e7804d1/appengine/cr-buildbucket/v2/default_field_masks.py
jinjingl, please give it a try: https://cr-buildbucket.appspot.com/rpcexplorer/services/buildbucket.v2.Builds/SearchBuilds?request={%20%20%20%20%22predicate%22:%20{%20%20%20%20%20%20%20%20%22builder%22:%20{%20%20%20%20%20%20%20%20%20%20%20%20%22project%22:%20%22chromeos%22,%20%20%20%20%20%20%20%20%20%20%20%20%22bucket%22:%20%22general%22,%20%20%20%20%20%20%20%20%20%20%20%20%22builder%22:%20%22Prod%22%20%20%20%20%20%20%20%20}%20%20%20%20},%20%20%20%20%22fields%22:%20%22builds.*.id,builds.*.steps,builds.*.input%22} - don't forget to login - try without "fields" - more docs: https://chromium.googlesource.com/infra/luci/luci-go/+/f59c9de50df31833ed2d33d48d17bb55a1873bf7/buildbucket/proto/rpc.proto#51 filter by tags: https://cr-buildbucket.appspot.com/rpcexplorer/services/buildbucket.v2.Builds/SearchBuilds?request={%20%20%20%20%22predicate%22:%20{%20%20%20%20%20%20%20%20%22builder%22:%20{%20%20%20%20%20%20%20%20%20%20%20%20%22project%22:%20%22chromeos%22,%20%20%20%20%20%20%20%20%20%20%20%20%22bucket%22:%20%22general%22,%20%20%20%20%20%20%20%20%20%20%20%20%22builder%22:%20%22Prod%22%20%20%20%20%20%20%20%20},%20%20%20%20%20%20%20%20%22tags%22:%20[%20%20%20%20%20%20%20%20%20%20%20%20{%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%22key%22:%20%22cbb_config%22,%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%22value%22:%20%22som-dispatcher%22%20%20%20%20%20%20%20%20%20%20%20%20}%20%20%20%20%20%20%20%20]%20%20%20%20},%20%20%20%20%22fields%22:%20%22builds.*.id,builds.*.tags,builds.*.steps,builds.*.input%22}
The following revision refers to this bug: https://chromium.googlesource.com/infra/luci/luci-go.git/+/544edc8774a10f378f3a1f10c6ba5d3d755dc0bb commit 544edc8774a10f378f3a1f10c6ba5d3d755dc0bb Author: Nodir Turakulov <nodir@google.com> Date: Mon Jun 04 21:30:14 2018 [buildbucket] Allow searching for ended builds SearchBuilds RPC does support finding completed builds regardless of build result. This is a regression from v1. Expand Status_COMPLETED_STATUS_MASK to represent a union of all terminal statuses and allow it in BuildPredicate. Rename it to ENDED. Use "end" as opposed to "complete" because it is the antonym of "start". Bug: 843712 Change-Id: Iad3bfcd99a50427bd204e1f08bd4569a66451f94 Reviewed-on: https://chromium-review.googlesource.com/1084074 Commit-Queue: Nodir Turakulov <nodir@chromium.org> Reviewed-by: Vadim Shtayura <vadimsh@chromium.org> [modify] https://crrev.com/544edc8774a10f378f3a1f10c6ba5d3d755dc0bb/buildbucket/proto/common.go [modify] https://crrev.com/544edc8774a10f378f3a1f10c6ba5d3d755dc0bb/buildbucket/proto/common.pb.go [modify] https://crrev.com/544edc8774a10f378f3a1f10c6ba5d3d755dc0bb/buildbucket/proto/common.proto [modify] https://crrev.com/544edc8774a10f378f3a1f10c6ba5d3d755dc0bb/buildbucket/proto/pb.discovery.go [modify] https://crrev.com/544edc8774a10f378f3a1f10c6ba5d3d755dc0bb/milo/buildsource/buildbot/buildstore/buildbucket.go
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/270281386532f5f9e2b7e73b2ec8e1373d9eacbd commit 270281386532f5f9e2b7e73b2ec8e1373d9eacbd Author: Nodir Turakulov <nodir@google.com> Date: Thu Jun 07 19:48:48 2018 [buildbucket][v2] Search ended builds Implement search of ended builds. See https://chromium-review.googlesource.com/c/infra/luci/luci-go/+/1084074 Bug: 843712 Change-Id: I96e6679cfeed7c7142f87a84c1a7c4b62671beca Reviewed-on: https://chromium-review.googlesource.com/1084059 Commit-Queue: Nodir Turakulov <nodir@chromium.org> Reviewed-by: Vadim Shtayura <vadimsh@chromium.org> [modify] https://crrev.com/270281386532f5f9e2b7e73b2ec8e1373d9eacbd/appengine/cr-buildbucket/v2/api.py [modify] https://crrev.com/270281386532f5f9e2b7e73b2ec8e1373d9eacbd/appengine/cr-buildbucket/proto/common_pb2.py [modify] https://crrev.com/270281386532f5f9e2b7e73b2ec8e1373d9eacbd/appengine/cr-buildbucket/proto/rpc_prpc_pb2.py
Comment 1 by no...@chromium.org
, May 16 2018