cmtice@ would this be a reasonable fix for the buildbot API that's going away? If not, this will be very low priority.

We actually need two pieces from the build log/record for any particular trybot run (using our script) are:

- The build number/build id
- The final build status number (0 = success; 1 = warnings; 6 = retry;  all others = failure)

If your fix can get us both pieces of information that would be fine, but if it can't get us the build id, then we've still got a problem (we currently find & parse the build record for this).

Which build number is that? buildbot, CIDB, or buildbucket? The tryjob command can only know the buildbucket id without a lot of extra work. Also, the buildbot id will go away as we move to swarming (it's the one shown next to a build in the waterfall UI).

I'm not sure if cbuildbot's exit code is in the buildbucket result currently. Even if it's not, that's a reasonable request.

The best way to explain what I need is by example.  We had a job that ran on Monday.  To look at the job, I go to 

https://uberchromegw.corp.google.com/i/chromiumos.tryserver/builders/release/builds/15298

In this case, the build id I need is "15298", the number at the end of the URL. (So I can find the correct URL!)

That is also the number that gets tacked onto the end of the artifacts which get uploaded (which is how we find the artifacts, which we also need, but which we can find via the build id).

We are rolling out build details pages based on the buildbucket id. When they work (soon), we can use that.

And the cros tryjob command will start outputting the exact URLs for the builds it's running.

Maybe that's all you really need?

I'm going to have to solve the artifacts id problem also. I'd forgotten about that.

Our basic workflow is to launch a trybot job that will build a ChromeOS image.  We then want to download the built ChromeOS image and run performance tests on it, comparing it with the nearest 'vanilla image'.  But we only want to download and run tests if the build succeeded.  So what we really need to know (and I don't really care how we get it is):

- When the trybot job finishes (we don't want to try to download/test the image before it's finished building).
- Did the trybot job finish successfully (create an image we can download and test)
- The actual name/location of the image that was uploaded, so we know what to download and test.  Normally we have been constructing this from the name of the buildbot job we launched plus the build id we recovered from the build record.  Our current code is:  command = ('gsutil ls gs://chromeos-image-archive/trybot-%s/*b%s/chromiumos_test_image.tar.xz' % (build, build_id), where build is the name of the trybot (e.g. 'falco-release') and build_id is what we've been discussing.

- The trybot job URL (as given in comment 4), so if the job did not finish successfully, we can go look at it and figure out why.  This, also, we have been constructing from the build id.

Just to a understand, why not run the performance tests on the builder?

Our Golo builders have inconsistent physical specs, but all of our GCE builders should be identical.

Then this could be a new group of builders instead of a special set of tryjobs with wrapper scripts.

1). To the best of my knowledege, the builders do not allow you to run arbitrary autotest tests -- you can only specify suites of tests to run.
2). The builders do not run A vs B comparision tests, which our scripts do, allowing us to run the performance tests comparing the performance of 2 (sometimes 3) different ChromeOS images, and generating a unified report comparing the performance of the various ChromeOS Images.

Referring to point 1 in comment 11, we also use quite a few special flags & tweaks when we run the tests, to get good results and reduce the noise.  Again, I'm not sure the builders have the flexibility we would need.

They can run arbitrary tests if we write the new test stages needed, but comparing multiple builds against each other seems more suited to something run outside of the builds themselves.

Thanks for explaining!

Just as a note for when this work gets picked up.

cros tryjob would have easy access to the values in "result_details_json" from:

https://apis-explorer.appspot.com/apis-explorer/?base=https://cr-buildbucket.appspot.com/_ah/api#p/buildbucket/v1/buildbucket.get?id=8967268633312483040&_h=1&

(insert any buildbucket id in there to compare with other types of builds)