Is this really something we want to do? AFAICT the "friendly names" has been the case for lots of years. I'd *love* to rename them to builder names, as it would remove yet more magic from the BuildBot code.

This operation is manually intensive, as preserving history across builder rename will require manually renaming the builder's history directory on the master system. Let's make absolutely sure that this is what we want to do before going through with it.

This is a request that came out of the Chrome Gardening code yellow as a way to reduce confusion (which I think we all agree with).

I hadn't thought about losing history. Is it a total non-starter to just lose it? How far back does it go anyway.... the last 200 builds?

Losing history might actually be a non-issue. It looks like Past Me already told BuildBot to name history directories after the config rather than the friendly name.

Does loss of history mean "you can't see those logs anymore", or "those logs are still available, but under the old name"?

I don't think a one-time loss of history is a big deal. Especially in the latter case.

Oh, sweet.

It would effectively mean "you can't see the logs anymore", since even if they still existed on the master, there would be no link to them nor system serving them.

I just did a local test though and it looks like it won't be a problem. I did apparently decouple the builder's state directory name from the builder name.

CL ready to do this, it will obviously require a master restart to take effect: https://codereview.chromium.org/1757553002

LMK if I have the go-ahead to land.

Let's discuss a bit more first (I'll rope in chromeos infra) but sounds good to me.

I think that I can safely speak for other gardeners when saying that the "user friendly" names are anything but :)

The primary reason the "user friendly" names cause confusion is when trying to find the associated name in 'cbuildbot --list', where, e.g. "peach_pit (chrome)" actually corresponds to "peach_pit-tot-chrome-pfq-informational".

+abodenha@ FYI

It is worth noting that BuildBot's URL interface (/builders/<whatever>/builds/1234) uses the display name of the builder as "<whatever>". Consequently, any tooling or links that use hard-coded builder names in URLs (including JSON endpoint hits) will likely break.

That would break links in bugs. This one, for example:

https://uberchromegw.corp.google.com/i/chromeos/builders/pineview%20group%20canary/builds/2157

How painful would it be to set up redirects? FWIW, those escaped paths are also kind of a pain, e.g. https://uberchromegw.corp.google.com/i/chromeos.chrome/builders/Lumpy%20%28chrome%29

I say that those links (while useful) have always been temporary. I say we go forward with this, if the deputy (primary and secondary) sign off on it.

Adding deputies.

Redirects would be fairly painful.

I would agree that old links are not critical to preserve, so long as any
new links (e.g. from the PFQ master to failed builders) are correct.

On Tue, Mar 1, 2016 at 12:48 PM, dnj@chromium.org via Monorail <
monorail@chromium.org> wrote:

That's a good point - Chromite generates builder links. It would need to be updated to use the config names directly.

(Which is probably a good thing).

Simran just gave the go ahead.

Which becomes a roll out problem since it would be generating invalid links until the waterfall is restarted.

Maybe I should prep a CL that removes config names, and see how much it affects. Before we land the waterfall side of this.

Yeah, there would definitely be a race.

Worst case, though, we get a round or two of builds with unclickable links with the silver lining that they will become clickable after the restart. I don't think that's too horrible - no data loss or anything. We probably want to CQ the change, though, meaning that this will likely happen to some extent.

If you wanted to prevent this, we could pass a master schema ID from master to cbuildbot and have cbuildbot choose how to generate links based on the schema version. This isn't a bad idea in general, but would take some time, and probably isn't worth it for this specific event.

I was just chatting with +jparent@, and she mentioned that we should probably send out an email to chrome-infrastructure-team@ (and maybe chromeos-infra-eng@?) as an FYI/heads-up in case this breaks anything else (e.g. sheriff-o-matic which we are starting to populate with chromeos builders). It seems OK to fix any problems as they come up, but a heads-up is always a good thing :)

dnj@ - assigning to you since you already put together a CL :)

Actually handing to dgarrett@ since he's working up the Chromite side of this. Please pass to me when we're ready for the BuildBot part.

I'm confused. Chromite-generated urls are done programatically based on the builder name that was passed in to the run. Chromite-generated urls should be fine prior to the restart, and after the restart. (but links from pre-restart will be broken after the restart)

Is that true for master lists? Chromite emits links to builders that failed, so those builder names wouldn't have been passed to the master builder. Are the slave builders registering their builder names with the database that the master uses?

Yes, the slaves register themselves with cidb, including the name of their builder. This is what the master uses to construct the url to the slave build.

Excellent!

Also... looking at the chromite side, I was thinking that the 'Description' field matched the friendly builder names, but that's obviously not true.

In combination with that Aviv just said, I don't think we need any other chromite changes for this.

Okay. We're getting to go a good drain/restart window, want to flip the switch here?

Let's do it.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build.git/+/44bff35f736334573a94463d0a0ed15f2f9ba6d9

commit 44bff35f736334573a94463d0a0ed15f2f9ba6d9
Author: dnj@chromium.org <dnj@chromium.org>
Date: Wed Mar 02 01:01:15 2016

CrOS: Builder names == target config names.

BUG= chromium:591117 
TEST=None

Review URL: https://codereview.chromium.org/1757553002

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/build@299039 0039d316-1c4b-4281-b951-d872f2087c98

[modify] https://crrev.com/44bff35f736334573a94463d0a0ed15f2f9ba6d9/scripts/master/cros/builder_config.py

PSA sent.

Looks like we hit a round of builders failing b/c their names changed. I've kicked off a restart, will hit once the current canary run finishes.

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infradata/master-manager.git/+/aea8bbd76c7498bc20d2fb0c792a83cfd4109bbd

commit aea8bbd76c7498bc20d2fb0c792a83cfd4109bbd
Author: dnj <dnj@google.com>
Date: Wed Mar 02 01:09:34 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build.git/+/b749d12e388dc1c1f469f92bfbc3ecdf6b5aefd5

commit b749d12e388dc1c1f469f92bfbc3ecdf6b5aefd5
Author: dnj@chromium.org <dnj@chromium.org>
Date: Wed Mar 02 01:09:52 2016

CrOS: Update Chromite pin.

Update ChromeOS Chromite pins.
- [master]
  ae46034b4c736d33dc7def9967265f125f195fea =>
     0b18a845107ad0ab34368bfe616d8767c48bd996

BUG= chromium:591117 
TBR=bpastene@chromium.org

Review URL: https://codereview.chromium.org/1753063002

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/build@299040 0039d316-1c4b-4281-b951-d872f2087c98

[modify] https://crrev.com/b749d12e388dc1c1f469f92bfbc3ecdf6b5aefd5/scripts/common/cros_chromite_pins.json

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infradata/master-manager.git/+/5b341ad5865e0c3984974e6fbbecb826e39728cb

commit 5b341ad5865e0c3984974e6fbbecb826e39728cb
Author: dnj <dnj@google.com>
Date: Wed Mar 02 01:45:59 2016

This is complete. Passing back to dgarrett@ for follow-up and verification.

So far, inter-waterfall links appear to be working.

Waiting for a full set of builds to finish so we can see if master->slave links work.

Can we get this on https://uberchromegw.corp.google.com/i/chromeos.chrome/waterfall too?

Obviously we can't do it for https://uberchromegw.corp.google.com/i/chromiumos.tryserver/waterfall

I wonder also if there's a way to do this for https://uberchromegw.corp.google.com/i/chromeos_release/waterfall

chromeos.chrome is a bit more work, as it isn't part of the new configuration scheme and several builders are actually not directly tied to a specific instantiation of a cbuildbot target.

chromeos_release has already been updated.

Could this have broken the build status links on goldeneye?
https://cros-goldeneye.corp.google.com/console/listBuild#/ ->
https://uberchromegw.corp.google.com/i/chromeos_release/builders/beltino-a-release-group/builds/1947

The "chromeos_release" link there is weird to me. All "chromeos_release" builders have their branches appended as a suffix. For example:

https://uberchromegw.corp.google.com/i/chromeos_release/builders/auron-release-group%20release-R50-7978.B

That link is more in-line with "chromeos" waterfall, which is de facto tip-of-tree. Changing "chromeos_release" to "chromeos" actually yields a working link:
https://uberchromegw.corp.google.com/i/chromeos/builders/beltino-a-release-group/builds/1947

In this case, I don't think this name change had that impact, as a builder name change within a waterfall should not affect one waterfall being confused with another. Then again, I don't know where Goldeneye generates this link from.

Filed b/27444383 for the goldeneye team.

In order to fix GoldenEye links we needed to update our assumption of ToT builder names having "(canary)" in them. But since we know the bot config name, we could use it instead of the builder name in order to not break old links (which we guess it'd be cool for users). So we have some questions:

  1. Can we assume all builders in "chromeos" have the same name as their config? If we use that instead we can provide back-compat.
  2. Are "chromeos_release" and "chromeos.branch" affected by this change? Maybe not now, but after the next branch point?
  3. Can we safely assume branch builders will always have the branch name appended at the end of their name? We deduct it from there since we have no way to ingest it (https://bugs.chromium.org/p/chromium/issues/detail?id=407876).

Thank you!

1) Yes. All translation has been removed.
2) Yes and yes. "chromiumos" is also affected. NOTE: (1) is not true for all of these, as some still have their branch names appended, but they all will begin their config.
3) That seems reasonable.

Question: Can these builders also be renamed?

https://build.chromium.org/p/chromiumos.chromium/builders

These get a lot more verbose, e.g.:
X86 (chromium) -> x86-generic-tot-chromium-pfq-informational

But I still think is -much- more helpful than the "user friendly" name which is pretty meaningless.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build.git/+/8334e36a0b53db43da8754786039e9e768d34e03

commit 8334e36a0b53db43da8754786039e9e768d34e03
Author: dnj@chromium.org <dnj@chromium.org>
Date: Thu Mar 03 00:31:01 2016

CrOS: Rename chromiumos.chromium builders.

TBR=stip@google.com
BUG= chromium:591117 
TEST=None

Review URL: https://codereview.chromium.org/1763583002

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/build@299066 0039d316-1c4b-4281-b951-d872f2087c98

[modify] https://crrev.com/8334e36a0b53db43da8754786039e9e768d34e03/masters/master.chromiumos.chromium/master_asan_cfg.py
[modify] https://crrev.com/8334e36a0b53db43da8754786039e9e768d34e03/masters/master.chromiumos.chromium/master_chromium_cfg.py
[modify] https://crrev.com/8334e36a0b53db43da8754786039e9e768d34e03/masters/master.chromiumos.chromium/master_perf_cfg.py
[modify] https://crrev.com/8334e36a0b53db43da8754786039e9e768d34e03/masters/master.chromiumos.chromium/slaves.cfg

RE #48: Okay done.

Also did this for chromeos.chrome where applicable.

I am trying to find binary pfq artifacts. Are they supposed to be here?
** BUILD ARTIFACTS FOR THIS BUILD CAN BE FOUND AT:
**  nyan: https://storage.cloud.google.com/chromeos-image-archive/nyan-chrome-pfq/R51-8001.0.0-rc1/index.html
@@@STEP_LINK@Artifacts[nyan]: nyan-chrome-pfq/R51-8001.0.0-rc1@https://storage.cloud.google.com/chromeos-image-archive/nyan-chrome-pfq/R51-8001.0.0-rc1/index.html@@@

I believe that's from a build that failed very early on, and so didn't upload anything.

RE #53, looking at the bucket:  https://storage.cloud.google.com/chromeos-image-archive/nyan-chrome-pfq

All of the artifacts are already there. This implies that the artifact upload has always used the config name. Therefore, it is unlikely that this builder rename has affected artifact upload paths at all.

Thanks for checking. I also had the feeling that it may have not uploaded due to how early it failed.

The following revision refers to this bug:
  http://goto.ext.google.com/viewvc/chrome-internal?view=rev&revision=84650

------------------------------------------------------------------
r84650 | dnj@google.com | 2016-03-02T01:11:00.180920Z

-----------------------------------------------------------------

The following revision refers to this bug:
  http://goto.ext.google.com/viewvc/chrome-internal?view=rev&revision=84690

------------------------------------------------------------------
r84690 | dnj@google.com | 2016-03-03T00:57:27.883054Z

-----------------------------------------------------------------

Bulk verified