Also breaks here:
https://build.chromium.org/p/chromium.gpu/builders/Win7%20Debug%20%28NVIDIA%29/builds/50161
Gives much better regression range revision 456763 - revision 456781

sugoi's already got a patch up for review fixing this in https://chromium-review.googlesource.com/455464/ .

A couple of questions about this regression:

1) Why is it only happening on the Debug Windows bots and not the Release bots?

2) The usual way to handle regressions like this is to revert the breaking change, and then update the tests and re-land the now-no-longer-breaking change -- especially if bots on the main chromium.gpu waterfall are broken (which they are).

I think it would be more appropriate to revert the DEPS roll until the tests are fixed, since there are concerns with the above CL, and question (1) is concerning.

Hi Ken,

  I'm not 100% certain of what's causing this issue yet, but it doesn't look like it's the DEPS roll (it's not in the range of commits when the failures started.

  It's likely because of this change:
https://codereview.chromium.org/2715563002/diff/220001/ui/gl/BUILD.gn

  Since SwiftShader can now be enabled on all non chrome branded builds, bots running chromium can now use SwiftShader, which they didn't before. There must be a bug somewhere that causes the GPU process to crash and the test to still succeed in Release.

  My suggestion would be to land the fix for debug builds so that the bots can turn green and open a separate one for us to investigate the release only issue (why they aren't failing as expected). Nicolas and I will investigate and try to figure it out by the end of next week. If we can't figure it out by then, we can revert the file mentioned above to effectively disable SwiftShader on all non chrome branded builds.

I noticed that with --disable-gpu, the GPU process (with SwiftShader) doesn't start until I go to chrome:gpu, or a WebGL page. The GpuProcess_no_gpu_process test opens about:blank, so that would explain why the test doesn't fail on a Release build. That said, GpuProcess_skip_gpu_process opens chrome:gpu, so that one I still expected to fail.

Canary runs WebGL when using --disable-gpu, and I've verified there's a GPU process and it loaded the SwiftShader libraries. So something else must be causing the test to not see the process.

I've updated https://chromium-review.googlesource.com/455464/ to only apply to Debug. I definitely plan to have tests that check that SwiftShader is loaded correctly, but at the moment I'm not confident that the test itself would work correctly. We're going to need a bit of time to figure out why we're getting these weird test results. I don't think there's a big risk of missing any other regression by skipping the test on Win Debug, especially considering that there is likely something else broken that SwiftShader simply exposed.

chrome:gpu kills the GPU process after it has collected GPU data, so we might simply be seeing a timing difference between Debug and Release, at least for the GpuProcess_skip_gpu_process test.

Thanks for your analysis sugoi@ and capn@.

If there's a race condition in the GpuProcess_skip_gpu_process then let's fix it. I've uploaded https://codereview.chromium.org/2753873002/ and explicitly stated that it should be disabled on platforms where SwiftShader is used. Let's see whether it fails on Windows in Release mode.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/3497883d50bbe727116906995bab97d360a52a0d

commit 3497883d50bbe727116906995bab97d360a52a0d
Author: Nicolas Capens <capn@google.com>
Date: Wed Mar 15 19:14:49 2017

Skip no GPU process tests on Windows.

Chrome on Windows creates a GPU process that uses SwiftShader when using
either --disable-gpu or a blacklisted GPU.

BUG= 701625 

Change-Id: I9b51aa33315f935e8134be7326881d184bac6bc0
Reviewed-on: https://chromium-review.googlesource.com/455464
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Reviewed-by: Alexis Hétu <sugoi@chromium.org>
Commit-Queue: Nicolas Capens <nicolascapens@google.com>
Cr-Commit-Position: refs/heads/master@{#457159}
[modify] https://crrev.com/3497883d50bbe727116906995bab97d360a52a0d/content/test/gpu/gpu_tests/gpu_process_expectations.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/a04f321efbf30ec25a5eb8592a549e901ed8ffe7

commit a04f321efbf30ec25a5eb8592a549e901ed8ffe7
Author: kbr <kbr@chromium.org>
Date: Thu Mar 16 02:52:08 2017

Fix race condition in GpuProcess_skip_gpu_process.

chrome:gpu may be spinning up a GPU process regardless of
--disable-gpu, causing a race condition. Instead, load a page
containing WebGL content.

This test will not work on platforms where SwiftShader is enabled as a
software fallback. It should be disabled there, and removed once
SwiftShader is deployed everywhere.

BUG= 701625 
CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel;master.tryserver.chromium.linux:linux_optional_gpu_tests_rel;master.tryserver.chromium.mac:mac_optional_gpu_tests_rel;master.tryserver.chromium.win:win_optional_gpu_tests_rel

Review-Url: https://codereview.chromium.org/2753873002
Cr-Commit-Position: refs/heads/master@{#457331}

[modify] https://crrev.com/a04f321efbf30ec25a5eb8592a549e901ed8ffe7/content/test/data/gpu/functional_files/context.js
[modify] https://crrev.com/a04f321efbf30ec25a5eb8592a549e901ed8ffe7/content/test/gpu/gpu_tests/gpu_process_expectations.py
[modify] https://crrev.com/a04f321efbf30ec25a5eb8592a549e901ed8ffe7/content/test/gpu/gpu_tests/gpu_process_integration_test.py

The Debug Windows bots are rolling green after Nicolas' patch above, and the new SwiftShader/WebGL test passed the CQ, so it seems to be working successfully on Windows.

FYI the Debug Windows bots are red again, but this time due to GpuProcess_swiftshader_for_webgl which was added in https://codereview.chromium.org/2753873002/. We already have a patch in the works: https://bugs.chromium.org/p/swiftshader/issues/detail?id=32#c3

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/12281bd8755744a457842d926b0df999dfd5db9e

commit 12281bd8755744a457842d926b0df999dfd5db9e
Author: capn <capn@chromium.org>
Date: Wed Apr 12 23:26:05 2017

Re-enable GpuProcess_no_gpu_process on Windows Debug.

Debug builds of SwiftShader were creating a startup dialog for attaching a
debugger, causing the GPU process created by chrome://gpu not to exit. This has
been fixed, so the suppressed test can be enabled again.

BUG= 701625 

Change-Id: I623ba7e0ac9e187831ffb2756300f9e2e99372cf
CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel;master.tryserver.chromium.linux:linux_optional_gpu_tests_rel;master.tryserver.chromium.mac:mac_optional_gpu_tests_rel;master.tryserver.chromium.win:win_optional_gpu_tests_rel

Review-Url: https://codereview.chromium.org/2801243003
Cr-Commit-Position: refs/heads/master@{#464205}

[modify] https://crrev.com/12281bd8755744a457842d926b0df999dfd5db9e/content/test/gpu/gpu_tests/gpu_process_expectations.py