Here is what I find looking at a few sample builds:

master-paladin: 8969165543551753664
   "bot_id:cros-wimpy0-c2",
   "builder:master-paladin",
   "scheduler_invocation_id:9134595604352158960",
   "scheduler_job_id:chromiumos-chromite/master-paladin",
   "user_agent:luci-scheduler"

beaglebone-paladin: 8969156136093863936
   "board:beaglebone",
   "build_type:paladin",
   "builder:beaglebone-paladin",
   "buildset:cbuildbot/master/master-paladin/1825531",
   "cbb_branch:master",
   "cbb_config:beaglebone-paladin",
   "cbb_master_build_id:1825531",
   "master:False",
   "master_config:master-paladin"

Manual tryjob: 8969613204938988608
   "bot_id:build67-m2",
   "build_type:tryjob",
   "builder:pre_cq"

PreCQ tryjob: 8969157360832698608
   "build_type:tryjob",
   "builder:no_vmtest_pre_cq"


The builder name is ALMOST what we want. It currently refers to the vertical buildbot build column, which may or may not have anything to do the the cbuildbot config name.

A test swarming request (fairly old now): 8979304166465571488

  "tags": [
   "build_type:tryjob",
   "builder:Generic",
   "swarming_dimension:pool:ChromeOS",
   "swarming_hostname:chrome-swarming.appspot.com",
   "swarming_tag:build_type:tryjob",
   "swarming_tag:buildbucket_bucket:luci.chromeos.general",
   "swarming_tag:buildbucket_build_id:8979304166465571488",
   "swarming_tag:buildbucket_hostname:cr-buildbucket.appspot.com",
   "swarming_tag:buildbucket_template_canary:false",
   "swarming_tag:buildbucket_template_revision:f6e51070e8834b168923ce27015c89f846fb156e",
   "swarming_tag:builder:Generic",
   "swarming_tag:log_location:logdog://luci-logdog.appspot.com/chromeos/buildbucket/cr-buildbucket.appspot.com/8979304166465571488/+/annotations",
   "swarming_tag:luci_project:chromeos",
   "swarming_tag:pool:ChromeOS",
   "swarming_tag:priority:30",
   "swarming_tag:recipe_name:cros/cbuildbot",
   "swarming_tag:recipe_repository:https://chromium.googlesource.com/chromium/tools/build",
   "swarming_tag:recipe_revision:HEAD",
   "swarming_tag:service_account:none",
   "swarming_tag:user:",
   "swarming_task_id:3631a6f941e3dd10"
  ],

After discussion with jinjingl@, we're planning to add the following tags to all ChromeOS buildbucket requests.

"cbb_branch:master",
"cbb_config:beaglebone-paladin",
"cbb_master_build_id:1825531",
"cbb_prodution:true",
"cbb_email:dgarrett@chromium.org",     // Only present for tryjobs.


We think this is enough to identify all cases we care about.

Any build with "cbb_production:false" is a tryjob. "cbb_email" shows who submitted it.

Any build with "cbb_master_build_id:0" is a master builder, and MIGHT have slave builds that point at it.

cbb_config, cbb_branch, cbb_master_build_id:0, cbb_production:false are enough to uniquely identify master/slave groups over time.

This is reasonably manageable for all entities we expect to schedule builds.

LUCI Scheduler / UI Force build button, need to know cbb_config, and cbb_branch (needed no matter what), and can hard code cbb_master_build_id:0 and cbb_production:true.

Master builders have to know branch (usually their own), config, master_build_id (their own id), and hard code production.

tryjob requests have to know branch/config/email, and can hardcode the rest.


jinjingl@ is going to write up a design doc where we can track this as it evolves over time, which it surely will.

 Issue 762640  has been merged into this issue.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/741f047aa8c25d52babc5422e80e677353d7121f

commit 741f047aa8c25d52babc5422e80e677353d7121f
Author: Don Garrett <dgarrett@google.com>
Date: Thu Oct 05 17:36:15 2017

cros tryjob: Create additional buildbucket tags.

To support GE's UI effort, we need to populate additional buildbucket
tags to allow indexed build lookups. This CL adds those tags for
tryjobs, but not build slaves, scheduled builds, or UI triggered
builds.

BUG= chromium:755334 
TEST=run_tests
     cros tryjob --yes lumpy-compile-only-pre-cq
     cros tryjob --swarming --yes lumpy-compile-only-pre-cq

Change-Id: Id594417674970e9a9e279957c06e5e9b6a613821
Reviewed-on: https://chromium-review.googlesource.com/688491
Commit-Ready: Don Garrett <dgarrett@chromium.org>
Tested-by: Don Garrett <dgarrett@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>

[modify] https://crrev.com/741f047aa8c25d52babc5422e80e677353d7121f/cli/cros/cros_tryjob_unittest.py
[modify] https://crrev.com/741f047aa8c25d52babc5422e80e677353d7121f/lib/remote_try.py
[modify] https://crrev.com/741f047aa8c25d52babc5422e80e677353d7121f/lib/remote_try_unittest.py
[modify] https://crrev.com/741f047aa8c25d52babc5422e80e677353d7121f/cli/cros/cros_tryjob.py

I believe this is fixed for tryjobs. Here are a couple of buildbucket examples to confirm with:

  Swarming: 8966539335535856464
  Buildbot: 8966539232529285104

If they look good, we can consider this fixed.

Well, fixed for tryjobs.

Still need to add them for:
  master -> slave
  luci scheduler

The two tryjob ones look good to me. 

One has more swarming tags? (I assume this does not affect anything though) 

One of them was run as a swarming job, and one of them as a buildbot tryjob. Other than that, the two came from identical requests.

 Issue 755331  has been merged into this issue.

Based on feedback from today's meeting, I'm renaming the tag 'cbb_display_group' to 'cbb_build_group'.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/60eb880dcf3607494594921d1d9febe5cf22781c

commit 60eb880dcf3607494594921d1d9febe5cf22781c
Author: Don Garrett <dgarrett@google.com>
Date: Sat Oct 21 03:04:14 2017

remote_try: Rename cbb_display_group -> cbb_build_group.

Rename the buildbucket tag that will group related builds in the new
UI to a more descriptive name. In particular, this tag may be
associated with more UI behaviors than just which builds are grouped
together.

Also, remove the cidb_production tag, which is no longer needed.

BUG= chromium:755334 
TEST=Unittests

Change-Id: Ib35f20d7230671dfb570afbf800d1cdefa2f627f
Reviewed-on: https://chromium-review.googlesource.com/727187
Commit-Ready: Don Garrett <dgarrett@chromium.org>
Tested-by: Don Garrett <dgarrett@chromium.org>
Reviewed-by: Jacob Kopczynski <jkop@chromium.org>

[modify] https://crrev.com/60eb880dcf3607494594921d1d9febe5cf22781c/lib/remote_try.py
[modify] https://crrev.com/60eb880dcf3607494594921d1d9febe5cf22781c/lib/remote_try_unittest.py
[modify] https://crrev.com/60eb880dcf3607494594921d1d9febe5cf22781c/cli/cros/cros_tryjob.py

Are tags populated in Buildbucket for most builds? Or these are for tryjobs :-) 

Oh.... sorry, only for tryjobs. I guess I've been super focused on those.

Not for LUCI scheduler builder, or slave builds.

Working on LUCI Scheduler now.

Here is a current example buildbucket request for "master-paladin":

{
  "build": {
    "bucket": "master.chromeos",
    "canary_preference": "AUTO",
    "created_by": "user:luci-scheduler@appspot.gserviceaccount.com",
    "created_ts": "1510781611291990",
    "id": "8962824377117679328",
    "parameters_json": "{\"builder_name\": \"master-paladin\", \"properties\": {}}",
    "project": "chromiumos-chromite",
    "result_details_json": "null",
    "status": "SCHEDULED",
    "status_changed_ts": "1510781611297200",
    "tags": [
      "builder:master-paladin",
      "scheduler_invocation_id:9128254437945262400",
      "scheduler_job_id:chromiumos-chromite/master-paladin",
      "user_agent:luci-scheduler"
    ],
    "updated_ts": "1510781611297230",
    "utcnow_ts": "1510781611414700"
  }
}

I believe these are missing tags:
            'cbb_branch:master',
            'cbb_display_label:cq',
            'cbb_config:master-paladin',
            'cbb_email:',
            'cbb_master_build_id:',

Note that there is no email set because it's not associated with any user, and no master_build_id, since it is a master builder.

Cool. Thanks!

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/ca64c8e76891066b959f7911747071fa65d0967f

commit ca64c8e76891066b959f7911747071fa65d0967f
Author: Don Garrett <dgarrett@google.com>
Date: Fri Nov 17 10:00:33 2017

scheduler_stages: Add buildbucket tags for GE.

The GE UI uses buildbucket tags to search for builds, so populate the
needed tags for all builders scheduled via a master builder on the
ChromeOS and ChromiumOS waterfalls.

BUG= chromium:755334 
TEST=run_tests

Change-Id: Ia1fd7fa20d8d9ccbfbd9ccc2133cc1c39e67e125
Reviewed-on: https://chromium-review.googlesource.com/773628
Commit-Ready: Don Garrett <dgarrett@chromium.org>
Tested-by: Don Garrett <dgarrett@chromium.org>
Reviewed-by: Jacob Kopczynski <jkop@chromium.org>

[modify] https://crrev.com/ca64c8e76891066b959f7911747071fa65d0967f/cbuildbot/stages/scheduler_stages.py
[modify] https://crrev.com/ca64c8e76891066b959f7911747071fa65d0967f/lib/remote_try.py
[modify] https://crrev.com/ca64c8e76891066b959f7911747071fa65d0967f/cbuildbot/stages/scheduler_stages_unittest.py

Oh, I lied earlier. This is NOT fixed for the case where a master builder schedules slave builds.

Seems I did not lie. These tags should also be present for slave builds. I believe this bug is resolved for all buildbucket scheduled builds.


Example release build:

https://apis-explorer.appspot.com/apis-explorer/?base=https://cr-buildbucket.appspot.com/_ah/api#p/buildbucket/v1/buildbucket.get?id=8957763934295226656&_h=2&