These sound like good steps. In (b) I would advocate to try to merge together the test suites. There were configuration differences between some of the bots that made no practical sense from looking at them, and some of the tests that run on a subset of bots should really run on all bots. This could be a larger project, and I think we should work with the owners of both the various waterfalls and test suites to achieve it.

We should also add:

c) Document the generator script

which I'll start doing now.

Agreed about (b).

By "document the generator script" I'm guessing you mean "document how the .pyl files work", not "document the source code of generate_buildbot_json.py", because the latter is pretty straightforward, right?

Yes, that's what I mean. :)

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/33d04f3a650ac2e8d1ea9005c9e94628b6d597b7

commit 33d04f3a650ac2e8d1ea9005c9e94628b6d597b7
Author: Dirk Pranke <dpranke@chromium.org>
Date: Sat Dec 16 00:05:27 2017

Run the blink unit tests on the remaining main bots.

This CL adds the blink unit tests (blink_heap_unittests,
blink_platform_unittests, webkit_unit_tests, webkit_python_unittests,
wtf_unittests) to a bunch of mac, linux, win, chromeos, and fyi
bots where they should've been running but weren't, by removing a
bunch of test_suite_exceptions. All of these tests are fast so there
should be no significant impact on capacity or cycle time.

There are still a bunch of exceptions for android, memory, and clang
bots that we'll address in a follow-up CL.

R=jbudorick@chromium.org, foolip@chromium.org
BUG= 560435 ,  795306 

Change-Id: I38c5b72b49e47014017cf671dd22c4bf52b5869d
Reviewed-on: https://chromium-review.googlesource.com/829874
Commit-Queue: Dirk Pranke <dpranke@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Cr-Commit-Position: refs/heads/master@{#524531}
[modify] https://crrev.com/33d04f3a650ac2e8d1ea9005c9e94628b6d597b7/testing/buildbot/chromium.chromiumos.json
[modify] https://crrev.com/33d04f3a650ac2e8d1ea9005c9e94628b6d597b7/testing/buildbot/chromium.clang.json
[modify] https://crrev.com/33d04f3a650ac2e8d1ea9005c9e94628b6d597b7/testing/buildbot/chromium.fyi.json
[modify] https://crrev.com/33d04f3a650ac2e8d1ea9005c9e94628b6d597b7/testing/buildbot/chromium.linux.json
[modify] https://crrev.com/33d04f3a650ac2e8d1ea9005c9e94628b6d597b7/testing/buildbot/chromium.mac.json
[modify] https://crrev.com/33d04f3a650ac2e8d1ea9005c9e94628b6d597b7/testing/buildbot/chromium.win.json
[modify] https://crrev.com/33d04f3a650ac2e8d1ea9005c9e94628b6d597b7/testing/buildbot/test_suite_exceptions.pyl

Is there a particular person that this bug can be assigned to?

I think it's ok if this one is left as available for now. Multiple people are working on cleaning things up in here.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/e165e971cc9205d7f3151cb4a089efd2584f1952

commit e165e971cc9205d7f3151cb4a089efd2584f1952
Author: Kenneth Russell <kbr@chromium.org>
Date: Mon Jan 01 23:45:26 2018

Document how to use generate_buildbot_json.py.

Describes the various data files the tool consumes, order of precedence
for various aspects of a test's description, and suggestions on adding
new tests, new bots, and new features to the tool.

BUG= 795306 

Change-Id: I70357cc061f34e0c2cb4d07ee89a593f95891c38
Reviewed-on: https://chromium-review.googlesource.com/846292
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Dirk Pranke <dpranke@chromium.org>
Cr-Commit-Position: refs/heads/master@{#526431}
[modify] https://crrev.com/e165e971cc9205d7f3151cb4a089efd2584f1952/testing/buildbot/README.md

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/6616b929d801905185a24b339f89668b36d61e64

commit 6616b929d801905185a24b339f89668b36d61e64
Author: Caleb Rouleau <crouleau@chromium.org>
Date: Thu Jan 04 02:38:18 2018

Clearly point to the correct script for gpu waterfall.

It took me a while to figure out that there was another script named
generate_buildbot_json.py in an entirely separate folder used for gpu.
Figured this would make it less confusing for the next person.

Bug:  795306 
Cq-Include-Trybots: master.tryserver.chromium.android:android_optional_gpu_tests_rel;master.tryserver.chromium.linux:linux_optional_gpu_tests_rel;master.tryserver.chromium.mac:mac_optional_gpu_tests_rel;master.tryserver.chromium.win:win_optional_gpu_tests_rel
Change-Id: I9268786cdbb8db3d84296a740a3039d0b72d3600
Reviewed-on: https://chromium-review.googlesource.com/848249
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Commit-Queue: Kenneth Russell <kbr@chromium.org>
Cr-Commit-Position: refs/heads/master@{#526903}
[modify] https://crrev.com/6616b929d801905185a24b339f89668b36d61e64/content/test/gpu/generate_buildbot_json.py
[modify] https://crrev.com/6616b929d801905185a24b339f89668b36d61e64/testing/buildbot/chromium.gpu.fyi.json
[modify] https://crrev.com/6616b929d801905185a24b339f89668b36d61e64/testing/buildbot/chromium.gpu.json
[modify] https://crrev.com/6616b929d801905185a24b339f89668b36d61e64/testing/buildbot/client.v8.fyi.json

This issue has been Available for over a year. If it's no longer important or seems unlikely to be fixed, please consider closing it out. If it is important, please re-triage the issue.

Sorry for the inconvenience if the bug really should have been left as Available.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot

thakis@ did a ton of cleanups in the test_suites.pyl and waterfalls.pyl files in other bugs; should link this bug to those.

Maybe easiest to just dupe?

If you disagree, undupe and marked as blocked on instead :-)