New issue
Advanced search Search tips

Issue 687625 link

Starred by 1 user
Status: Fixed
Owner:
d...@chromium.org
Closed: Feb 2017
Cc:
hinoka@chromium.org
efoo@chromium.org
no...@chromium.org
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Bug



Sign in to add a comment

Add a build info pRPC endpoint to Milo.

Project Member Reported by d...@chromium.org, Feb 1 2017 
A build info pRPC endpoint should be added to Milo. Milo is the layer in the LUCI stack that is aware of the bigger picture of builds and the various methods that builds are executed.

The build info RPC would return the build as a Milo annotation protobuf. The protobuf would be primarily loaded from LogDog's annotation stream, emitted by Annotee/Recipes. We would augment it with distributor-specific information (e.g., Swarming tags, BuildBot properties, etc.) and return it to the user.

The build info RPC would also return the LogDog prefix for the build, which would allow log resolution.

This lets Milo act as a one-stop point for resolving "I want information about this build". This includes the oft-requested ability to grab log streams for builds without doing the full LogDog query dance.
Project Member

Comment 1 by bugdroid1@chromium.org, Feb 2 2017 

The following revision refers to this bug:
  https://chromium.googlesource.com/external/github.com/luci/luci-go.git/+/149de756b08fe70a72945361d12c8353bc8656fa

commit 149de756b08fe70a72945361d12c8353bc8656fa
Author: dnj <dnj@chromium.org>
Date: Thu Feb 02 21:52:18 2017

Update LogDog client to export its host.

Make the LogDog client host field accessible to the client.

Defensively confirm to assert that the descriptor is present in the
server's response.

BUG= chromium:687625 
TEST=None

Review-Url: https://codereview.chromium.org/2671633002

[modify] https://crrev.com/149de756b08fe70a72945361d12c8353bc8656fa/logdog/client/coordinator/client.go
[modify] https://crrev.com/149de756b08fe70a72945361d12c8353bc8656fa/logdog/client/coordinator/stream.go
Project Member

Comment 2 by bugdroid1@chromium.org, Feb 2 2017 

The following revision refers to this bug:
  https://chromium.googlesource.com/external/github.com/luci/luci-go.git/+/6126eaafade03c4ff12640b1602c78fdbf0c09fb

commit 6126eaafade03c4ff12640b1602c78fdbf0c09fb
Author: dnj <dnj@chromium.org>
Date: Thu Feb 02 22:37:31 2017

milo: Export LogDog annotation stream, remove host

Update the LogDog annotation stream and associated LogDog methods to be
public. This is in anticipation of other Milo modules using it to load
annotatons.

Update URL generation to use the "viewer" package.

Remove arbitrary host specification from the LogDog endpoint (security
bug).

BUG= chromium:687625 
TEST=None

Review-Url: https://codereview.chromium.org/2674603002

[modify] https://crrev.com/6126eaafade03c4ff12640b1602c78fdbf0c09fb/milo/appengine/frontend/milo.go
[modify] https://crrev.com/6126eaafade03c4ff12640b1602c78fdbf0c09fb/milo/appengine/logdog/build.go
[modify] https://crrev.com/6126eaafade03c4ff12640b1602c78fdbf0c09fb/milo/appengine/logdog/http.go
Project Member

Comment 3 by bugdroid1@chromium.org, Feb 3 2017 

The following revision refers to this bug:
  https://chromium.googlesource.com/external/github.com/luci/luci-go.git/+/f640dfb81c083686c36758c43176f983c9351d38

commit f640dfb81c083686c36758c43176f983c9351d38
Author: dnj <dnj@chromium.org>
Date: Fri Feb 03 22:16:33 2017

milo: Use service interface for swarming.

Milo now uses an internal "swarmingService" interface for swarming
operations. This allows the debug code to be moved into test files,
removing it from production. It also makes room for additional Swarming
service operations to be added in parallel.

BUG= chromium:687625 
TEST=unit

Review-Url: https://codereview.chromium.org/2675493003

[modify] https://crrev.com/f640dfb81c083686c36758c43176f983c9351d38/milo/appengine/frontend/expectations/buildbot-swarming.TestableBuild-build-canceled.html
[modify] https://crrev.com/f640dfb81c083686c36758c43176f983c9351d38/milo/appengine/frontend/expectations/buildbot-swarming.TestableBuild-build-exception.html
[modify] https://crrev.com/f640dfb81c083686c36758c43176f983c9351d38/milo/appengine/frontend/expectations/buildbot-swarming.TestableBuild-build-expired.html
[modify] https://crrev.com/f640dfb81c083686c36758c43176f983c9351d38/milo/appengine/frontend/expectations/buildbot-swarming.TestableBuild-build-link.html
[modify] https://crrev.com/f640dfb81c083686c36758c43176f983c9351d38/milo/appengine/frontend/expectations/buildbot-swarming.TestableBuild-build-nested.html
[modify] https://crrev.com/f640dfb81c083686c36758c43176f983c9351d38/milo/appengine/frontend/expectations/buildbot-swarming.TestableBuild-build-patch-failure.html
[modify] https://crrev.com/f640dfb81c083686c36758c43176f983c9351d38/milo/appengine/frontend/expectations/buildbot-swarming.TestableBuild-build-pending.html
[modify] https://crrev.com/f640dfb81c083686c36758c43176f983c9351d38/milo/appengine/frontend/expectations/buildbot-swarming.TestableBuild-build-running.html
[modify] https://crrev.com/f640dfb81c083686c36758c43176f983c9351d38/milo/appengine/frontend/expectations/buildbot-swarming.TestableBuild-build-timeout.html
[modify] https://crrev.com/f640dfb81c083686c36758c43176f983c9351d38/milo/appengine/frontend/expectations/buildbot-swarming.TestableBuild-build-unicode.html
[modify] https://crrev.com/f640dfb81c083686c36758c43176f983c9351d38/milo/appengine/swarming/build.go
[modify] https://crrev.com/f640dfb81c083686c36758c43176f983c9351d38/milo/appengine/swarming/buildLog.go
[modify] https://crrev.com/f640dfb81c083686c36758c43176f983c9351d38/milo/appengine/swarming/build_test.go
[modify] https://crrev.com/f640dfb81c083686c36758c43176f983c9351d38/milo/appengine/swarming/expectations/build-canceled.json
[modify] https://crrev.com/f640dfb81c083686c36758c43176f983c9351d38/milo/appengine/swarming/expectations/build-exception.json
[modify] https://crrev.com/f640dfb81c083686c36758c43176f983c9351d38/milo/appengine/swarming/expectations/build-expired.json
[modify] https://crrev.com/f640dfb81c083686c36758c43176f983c9351d38/milo/appengine/swarming/expectations/build-link.json
[modify] https://crrev.com/f640dfb81c083686c36758c43176f983c9351d38/milo/appengine/swarming/expectations/build-nested.json
[modify] https://crrev.com/f640dfb81c083686c36758c43176f983c9351d38/milo/appengine/swarming/expectations/build-patch-failure.json
[modify] https://crrev.com/f640dfb81c083686c36758c43176f983c9351d38/milo/appengine/swarming/expectations/build-pending.json
[modify] https://crrev.com/f640dfb81c083686c36758c43176f983c9351d38/milo/appengine/swarming/expectations/build-running.json
[modify] https://crrev.com/f640dfb81c083686c36758c43176f983c9351d38/milo/appengine/swarming/expectations/build-timeout.json
[modify] https://crrev.com/f640dfb81c083686c36758c43176f983c9351d38/milo/appengine/swarming/expectations/build-unicode.json
[modify] https://crrev.com/f640dfb81c083686c36758c43176f983c9351d38/milo/appengine/swarming/html.go
[modify] https://crrev.com/f640dfb81c083686c36758c43176f983c9351d38/milo/appengine/swarming/html_data.go
Project Member

Comment 5 by bugdroid1@chromium.org, Feb 3 2017 

The following revision refers to this bug:
  https://chromium.googlesource.com/external/github.com/luci/luci-go.git/+/96163a8561d90a9784ae79e12683d7440a457f04

commit 96163a8561d90a9784ae79e12683d7440a457f04
Author: dnj <dnj@chromium.org>
Date: Fri Feb 03 23:51:39 2017

milo: Use struct for BuildBot properties.

Use a struct for BuildBot properties internally. This will
Marshal/Unmarshal to the previous []interface{} construction, but
internally it will make the properties more accessible.

BUG= chromium:687625 
TEST=unit

Review-Url: https://codereview.chromium.org/2676503002

[modify] https://crrev.com/96163a8561d90a9784ae79e12683d7440a457f04/milo/appengine/buildbot/build.go
[modify] https://crrev.com/96163a8561d90a9784ae79e12683d7440a457f04/milo/appengine/buildbot/structs.go
Project Member

Comment 6 by bugdroid1@chromium.org, Feb 4 2017 

The following revision refers to this bug:
  https://chromium.googlesource.com/external/github.com/luci/luci-go.git/+/cf475ef7566a1fb123eae4993888614c6a121532

commit cf475ef7566a1fb123eae4993888614c6a121532
Author: dnj <dnj@chromium.org>
Date: Sat Feb 04 00:06:45 2017

milo: Add BuildInfo implementation for BuildBot.

This adds a BuildInfo endpoint implementation for BuildBot builds. These
builds will load the Milo annotation protobuf from LogDog, merge it with
the BuildBot build information, and export that via pRPC endpoint.

BUG= chromium:687625 
TEST=unit

Review-Url: https://codereview.chromium.org/2674513003

[add] https://crrev.com/cf475ef7566a1fb123eae4993888614c6a121532/milo/appengine/buildbot/buildinfo.go
[add] https://crrev.com/cf475ef7566a1fb123eae4993888614c6a121532/milo/appengine/buildbot/buildinfo_test.go
[modify] https://crrev.com/cf475ef7566a1fb123eae4993888614c6a121532/milo/appengine/buildinfo/service.go
Project Member

Comment 7 by bugdroid1@chromium.org, Feb 4 2017 

The following revision refers to this bug:
  https://chromium.googlesource.com/external/github.com/luci/luci-go.git/+/cf475ef7566a1fb123eae4993888614c6a121532

commit cf475ef7566a1fb123eae4993888614c6a121532
Author: dnj <dnj@chromium.org>
Date: Sat Feb 04 00:06:45 2017

milo: Add BuildInfo implementation for BuildBot.

This adds a BuildInfo endpoint implementation for BuildBot builds. These
builds will load the Milo annotation protobuf from LogDog, merge it with
the BuildBot build information, and export that via pRPC endpoint.

BUG= chromium:687625 
TEST=unit

Review-Url: https://codereview.chromium.org/2674513003

[add] https://crrev.com/cf475ef7566a1fb123eae4993888614c6a121532/milo/appengine/buildbot/buildinfo.go
[add] https://crrev.com/cf475ef7566a1fb123eae4993888614c6a121532/milo/appengine/buildbot/buildinfo_test.go
[modify] https://crrev.com/cf475ef7566a1fb123eae4993888614c6a121532/milo/appengine/buildinfo/service.go
Project Member

Comment 9 by bugdroid1@chromium.org, Feb 10 2017 

The following revision refers to this bug:
  https://chromium.googlesource.com/external/github.com/luci/luci-go.git/+/e5aca54945978b7f4397c02b71609574004850dc

commit e5aca54945978b7f4397c02b71609574004850dc
Author: dnj <dnj@chromium.org>
Date: Fri Feb 10 21:22:00 2017

Accommodate Swarming try number into log stream.

Incorporate the Swarming try number into the LogDog log stream
calculation, which uses the *run ID*, not the task ID, to generate its
stream path.

BUG= chromium:687625 
TEST=unit

Review-Url: https://codereview.chromium.org/2681393008

[modify] https://crrev.com/e5aca54945978b7f4397c02b71609574004850dc/milo/appengine/swarming/buildinfo.go
[modify] https://crrev.com/e5aca54945978b7f4397c02b71609574004850dc/milo/appengine/swarming/buildinfo_test.go

Comment 10 by d...@chromium.org, Feb 10 2017

Status: Fixed (was: Started)
Uploaded, stable, and 100%! Yay!
Project Member

Comment 11 by bugdroid1@chromium.org, Feb 16 2017 

The following revision refers to this bug:
  https://chromium.googlesource.com/external/github.com/luci/luci-go.git/+/ddca4ffe013b628ca436a6c7a3df7d565ba2e8b0

commit ddca4ffe013b628ca436a6c7a3df7d565ba2e8b0
Author: dnj <dnj@chromium.org>
Date: Thu Feb 16 02:19:45 2017

Fix LogDog client queries without state.

LogDog is allowed to issue queries without retrieving state. However,
the client library considered a lack of state an error. Update it to
permit this.

Update Milo's tests to properly reflect the missing state for its
queries so the client library would have properly failed before, and now
properly succeeds.

BUG= chromium:687625 
TEST=unit
R=hinoka@chromium.org

Review-Url: https://codereview.chromium.org/2697223002

[modify] https://crrev.com/ddca4ffe013b628ca436a6c7a3df7d565ba2e8b0/logdog/client/coordinator/list.go
[modify] https://crrev.com/ddca4ffe013b628ca436a6c7a3df7d565ba2e8b0/logdog/client/coordinator/query.go
[modify] https://crrev.com/ddca4ffe013b628ca436a6c7a3df7d565ba2e8b0/logdog/client/coordinator/stream.go
[modify] https://crrev.com/ddca4ffe013b628ca436a6c7a3df7d565ba2e8b0/milo/appengine/buildbot/buildinfo_test.go
►

Sign in to add a comment