Adding Richard, who owns the process for ensuring that our hardware coverage for both the PFQ and CQ is correct. (Feel free to take this bug or close it if this is already something that you do.)

I would at least like us to have relatively painless method for a gardener or TPM to inspect/audit the builders. Goldeneye helps a lot, but I still can't easily answer:

1. Why do we have both veyron-minnie and veyron-pinky? (they are both grouped under 'pinky')?
2. Why do we have 'falco' and 'peppy' (both grouped under 'slippy')?
3. Why are these board groups are tested and not others?
4. What is the status of the chromium builder coverage (which is not reported in Goldeneye)?

I believe that we have a test in chromite that ensures that all important configurations are covered, but I don't recall the details. I would really like to add a summary of "why we need this" to the builder output (maybe there is one and I don't know or have forgotten about it?)
and maybe populate Goldeneye with this info (for the chrome-pfq builders anyway).

1: Removed veyron_pinky:  issue 703841 
2. Removed falco:  issue 703435 

We still have the following matching prebuilts, but these at least have some level of difference between the boards, documented here (in the PFQ section):

https://docs.google.com/spreadsheets/d/1Gdb2Rkfl7-RwhlHnrJ4-WxMTTWj5Dxp4wIueMn2vfnU/edit#gid=0

chromium:


arm-generic / daisy

_CompatId(arch='arm', useflags=('accessibility', 'autotest', 'build_tests', 'buildcheck', 'chrome_debug', 'chrome_remoting', 'clang', 'cups', 'evdev_gestures', 'fonts', 'gold', 'hardfp', 'highdpi', 'nacl', 'neon', 'opengles', 'ozone_platform_default_gbm', 'ozone_platform_gbm', 'runhooks', 'v4l2_codec', 'xkbcommon'), cflags=('-O2', '-O2', '-pipe', '-march=armv7-a', '-mtune=cortex-a15', '-mfpu=neon', '-mfloat-abi=hard', '-g', '-fno-exceptions', '-fno-unwind-tables', '-fno-asynchronous-unwind-tables'))


chrome:

nyan / peach_pit / daisy_skate

_CompatId(arch='arm', useflags=('accessibility', 'autotest', 'build_tests', 'buildcheck', 'chrome_debug', 'chrome_internal', 'chrome_remoting', 'clang', 'cups', 'evdev_gestures', 'fonts', 'gold', 'hardfp', 'highdpi', 'nacl', 'neon', 'opengles', 'ozone_platform_default_gbm', 'ozone_platform_gbm', 'runhooks', 'v4l2_codec', 'xkbcommon'), cflags=('-O2', '-O2', '-pipe', '-march=armv7-a', '-mtune=cortex-a15', '-mfpu=neon', '-mfloat-abi=hard', '-g', '-fno-exceptions', '-fno-unwind-tables', '-fno-asynchronous-unwind-tables'))


cyan / peppy / tricky

_CompatId(arch='amd64', useflags=('accessibility', 'authpolicy', 'autotest', 'build_tests', 'buildcheck', 'chrome_debug', 'chrome_internal', 'chrome_remoting', 'clang', 'cups', 'evdev_gestures', 'fonts', 'gold', 'highdpi', 'nacl', 'opengles', 'ozone_platform_default_gbm', 'ozone_platform_gbm', 'runhooks', 'v4l2_codec', 'vaapi', 'xkbcommon'), cflags=('-O2', '-pipe', '-O2', '-pipe', '-march=corei7', '-g', '-fno-exceptions', '-fno-unwind-tables', '-fno-asynchronous-unwind-tables'))

Can we add reks for N coverage (with VMTest)?

Is "we have exactly one PFQ builder for each combination of USE flags used to build the chromeos-chrome package" the main goal here, or is it more subtle than that?

The next step is probably for someone who's familiar with the current audit/management process to document how it works.

I think that is pretty much it. We need at least one builder per use flag combination to create the prebuilt binaries, and we don't want more than that to make gardening life easier. Except when we have special new hardware that Chrome uses (say special video codecs), then we may want to add extra builders.

> I think that is pretty much it. We need at least one builder
> per use flag combination to create the prebuilt binaries, and
> we don't want more than that to make gardening life easier.

It is a requirement for at least one builder per use flag combination.
However, not adding more to make gardening life easier isn't
necessarily the best trade-off.  We need enough variety in the
hardware coverage to make sure we find Chrome bugs before they hit
the canaries.  So, we should make sure to have coverage across
the following:
  * Relevant hardware families/leaderboards.
  * A sampling of Chromeboxes (i.e. hardware with an external
    monitor).  There's a history of occasional Chrome bugs that
    only manifest on Chromeboxes (mostly because they run headless).

Also, regarding this:
> Is "we have exactly one PFQ builder for each combination of USE
> flags used to build the chromeos-chrome package" the main goal
> here, or is it more subtle than that?

The PFQ serves two key functions:
 1) To create Chrome pre-builts so that the CQ can run faster, and
    doesn't depend on the Chrome source base.
 2) To find and block Chrome bugs before they hit the canaries.

The "each combination of USE flags" requirement exists to serve
item 1).  Coverage requirements exist to serve item 2).

A TPM should own computing the minimal hardware coverage set that gives us the business coverage that we want. Josafat, can you own finding someone to do that?

In general, we need to have hardware for each of the architecture we support, including any special hardware variants 

+bhthompson,cindyb to come up with a proposal/audit on this