Issue 848363  has been merged into this issue.

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/a8d014753ff8a76f89e80480719e9c0266b3a637

commit a8d014753ff8a76f89e80480719e9c0266b3a637
Author: Nodir Turakulov <nodir@google.com>
Date: Thu May 31 21:58:50 2018

[buildbucket] Log number of skipped entities

Search is currently slow.
Log number of entities we fetch from Datastore, but skip.

Bug:  848346 
Change-Id: I821e11a897d6f7de0b177c195aa28be887f1b1e3
Reviewed-on: https://chromium-review.googlesource.com/1081230
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Commit-Queue: Nodir Turakulov <nodir@chromium.org>

[modify] https://crrev.com/a8d014753ff8a76f89e80480719e9c0266b3a637/appengine/cr-buildbucket/service.py

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/51eb2db17189b6f0b64df89f860b55ef6703b88a

commit 51eb2db17189b6f0b64df89f860b55ef6703b88a
Author: Nodir Turakulov <nodir@google.com>
Date: Thu May 31 22:43:20 2018

[buildbucket] Tune search

Query-based search implementation has an "optimization" related to experimental builds.
It turned out to do more harm (queries are slow) than good. Remove it.

Bug:  848346 
Change-Id: I5feac148272afdbcb82415444318c7997a074f81
Reviewed-on: https://chromium-review.googlesource.com/1081512
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Commit-Queue: Nodir Turakulov <nodir@chromium.org>

[modify] https://crrev.com/51eb2db17189b6f0b64df89f860b55ef6703b88a/appengine/cr-buildbucket/service.py

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/9ffd7a773ada3b828459a43aacab44cf2136c681

commit 9ffd7a773ada3b828459a43aacab44cf2136c681
Author: Nodir Turakulov <nodir@google.com>
Date: Fri Jun 01 18:33:18 2018

[buildbucket] Add more logging in query search

Datastore query-based search can take a lot of time, e.g. 20s. Log more.

Bug:  848346 
Change-Id: I5f41e2fd21db9690bf972646b394b865ccb8b851
Reviewed-on: https://chromium-review.googlesource.com/1081975
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Ryan Tseng <hinoka@chromium.org>

[modify] https://crrev.com/9ffd7a773ada3b828459a43aacab44cf2136c681/appengine/cr-buildbucket/service.py

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/5e54d90b018aef8e8cbef568789217999ee495da

commit 5e54d90b018aef8e8cbef568789217999ee495da
Author: Nodir Turakulov <nodir@google.com>
Date: Fri Jun 01 21:14:18 2018

[buildbucket] Add two composite build indexes

Milo builder page searches builds by builder and status. Add two composite
indexes to optimize such queries.

Bug:  848346 
Change-Id: If602177c9a45e70d1787a92c6c7d06068c0c9468
Reviewed-on: https://chromium-review.googlesource.com/1083315
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Commit-Queue: Nodir Turakulov <nodir@chromium.org>

[modify] https://crrev.com/5e54d90b018aef8e8cbef568789217999ee495da/appengine/cr-buildbucket/index.yaml

the existing API was optimized as much as possible without breaking changes. The remaining requests that still take >5s have large response size:

https://pantheon.corp.google.com/logs/viewer?project=cr-buildbucket&minLogLevel=0&expandAll=false&timestamp=2018-06-01T23:28:54.541000000Z&customFacets=&limitCustomFacetWidth=true&dateRangeStart=2018-06-01T22:31:08.814Z&dateRangeEnd=2018-06-01T23:31:08.814Z&interval=PT1H&resource=gae_app%2Fmodule_id%2Fdefault%2Fversion_id%2F15808-361cd11&logName=projects%2Fcr-buildbucket%2Flogs%2Fappengine.googleapis.com%252Frequest_log&scrollTimestamp=2018-06-01T23:31:03.063626000Z&advancedFilter=resource.type%3D%22gae_app%22%0Aresource.labels.module_id%3D%22default%22%0AlogName%3D%22projects%2Fcr-buildbucket%2Flogs%2Fappengine.googleapis.com%252Frequest_log%22%0AprotoPayload.latency%3E%3D%225.010824s%22%0A%0AprotoPayload.resource:%22search%22

displays response size >=1MB

This can be further improved by switch go Search API v2 which enables further optimizations: bug 848960

specifically, it enables not fetching properties from datastore because builder view doesn't need them, for example

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/b43a72b9574ff24e1527036c96d92c9d67863f9f

commit b43a72b9574ff24e1527036c96d92c9d67863f9f
Author: Nodir Turakulov <nodir@google.com>
Date: Mon Jun 04 17:41:09 2018

[buildbucket] Index by experimental

Most buildbucket searches do not include experimental builds, so
(experimental=False) filter is added to datastore queries. This means
that the recently added indexes are only partially useful in most of queries.

Update the indexes to include experimental property.
This will speedup Milo's API Emulation.

TBR=hinoka@chromium.org
Bug:  848346 
Change-Id: I18cf37fce4f519cf05584103bb6504bda074f95e
Reviewed-on: https://chromium-review.googlesource.com/1084078
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Commit-Queue: Nodir Turakulov <nodir@chromium.org>

[modify] https://crrev.com/b43a72b9574ff24e1527036c96d92c9d67863f9f/appengine/cr-buildbucket/index.yaml

a few optimizations and a bug fix were applied. Reopen if it is still a problem.