Please try running the generate_buildbot_json.py script locally and see if any diffs show up in the JSON file.

If multiple people CQ changes to the generated JSON files at the same time, they're usually applied incorrectly because the file is so repetitive.

Jamie, please re-confirm this. I don't think there's a bug in the JSON parsing, but rather that a patch was applied incorrectly.

I can confirm this is still happening.

https://luci-milo.appspot.com/buildbot/tryserver.chromium.win/win_optional_gpu_tests_rel/6194

See any build from the optional builders (search the above log for angle_deqp_gles3). You will see they *only* pull in the AMD configs, and don't even test NVIDIA, which we have had on the ANGLE CQ for a long time.

Any advice?

Thanks Jamie for pointing this out.

One strangeness in the configuration of win_optional_gpu_tests_rel is that it mirrors the (not actually existing) "Optional Win7 Release (NVIDIA)", but also "Win7 Release (AMD)", so any test that goes on the Win7 Release (AMD) bot will also go on the optional GPU tryserver. This is set up here:

https://cs.chromium.org/chromium/build/scripts/slave/recipe_modules/chromium_tests/trybots.py?l=586

I'm still digging through the JSON file to figure out what's going on for the NVIDIA side of things.

OK, looking at the report, the main problem is the abovementioned one. Taking this.

Note that this will have the side-effect of ceasing to run the WebGL 1.0 conformance tests on Win/AMD, as well as many of the other tests that are only specified for the main waterfall bots and not the "optional" tryservers. However, since this bot was already running the WebGL 2.0 tests on Win/AMD, the loss of coverage should be minimal.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build.git/+/450af73ed85c75c5cd58bb59302d9a5db18dc996

commit 450af73ed85c75c5cd58bb59302d9a5db18dc996
Author: Kenneth Russell <kbr@chromium.org>
Date: Wed Dec 21 21:57:36 2016

Change win_optional_gpu_tests_rel to mirror optional AMD bot.

It was previously mirroring the waterfall bot "Win7 Release (AMD)".
Make it mirror the virtual bot "Optional Win7 Release (AMD)" as
expected.

This will have the side-effect of making this bot stop running the
WebGL 1.0 conformance tests, but since it's already running the WebGL
2.0 conformance tests, the loss of coverage should be minimal. (Note
that win_chromium_rel_ng does not run tests on Win/AMD because there
isn't enough capacity.)

BUG= 672903 

Change-Id: I59ce9978e195a2526866e920661d02af104e36b0

TBR=dpranke@chromium.org

Change-Id: I59ce9978e195a2526866e920661d02af104e36b0
Reviewed-on: https://chromium-review.googlesource.com/422941
Commit-Queue: Kenneth Russell <kbr@chromium.org>
Reviewed-by: Kenneth Russell <kbr@chromium.org>

[modify] https://crrev.com/450af73ed85c75c5cd58bb59302d9a5db18dc996/scripts/slave/recipe_modules/chromium_tests/trybots.py

This is fixed and will take effect in the next runs of win_optional_gpu_tests_rel. It looks like the AMD bots behind this are being fixed in Issue 673475 so when they come back online let's look at them and verify.