New issue
Advanced search Search tips

Issue 780129 link

Starred by 2 users

Issue metadata

Status: Available
Owner: ----
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Feature

Blocked on:
issue 782369



Sign in to add a comment

Source commit index to store metadata about commits

Project Member Reported by iannucci@chromium.org, Oct 31 2017

Issue description

Partial design doc:

https://docs.google.com/document/d/1xDIwTby2PxPLyEJMwCxv7n3gcRjgMAoRf5tOHlcDkqA/edit#

The basic premise here is that we currently organize almost all of our data by Build/Test/Whatever and then (poorly) by Commit, but:
  * commits have an order which is difficult to query, including going 'up' towards a ref. This means queries like "next build" are extremely difficult to implement.
  * Every form of additional data we need to associate with a commit requires a new service and/or new datastore models. Each service has to re-invent solutions to these problems.

Instead, we could potentially have a service to index commits (bi-directionally, including correct git parentage information), and then allow other authenticated services to associate data with the commits (i.e. builds, test results, etc.).

Implementations to consider:
  * An appengine app
  * A GKE app (maybe with spanner datastore)
  * A hybrid of those two (like logdog)
  * Git notes (potentially too low bandwidth and complex addition procedure, doesn't solve history forward/back indexing)
 

Comment 1 by efoo@chromium.org, Nov 7 2017

Components: Infra>Platform
Labels: LUCI-Afterglow
Owner: ----
Status: Available (was: Assigned)

Comment 2 by efoo@chromium.org, Nov 7 2017

Blockedon: 782369
request on luci-eng [https://groups.google.com/a/google.com/d/msg/luci-eng/FElE6S4PsfI/imoQHygoCgAJ]

"""
Hi all, 

I am working on getting Fuchsia performance tests up and running on our CI service.  I'm planning on logging performance test results from a recipe so that they show up under "steps and logfiles" on a builder's details page: https://screenshot.googleplex.com/jAXfbasDkYp

Others on my team are interested in being able to download this data to do their own local analysis.  Typically they'd be looking for the data from a particular commit and/or date.  Does LUCI have an API for fetching data like this, or is the only solution at the moment to navigate to a builder's page, click the build in question, and download the file by hand.
"""

Sign in to add a comment