Jamie: looks like potentially a 10/200 (5%) flake rate of webgl_conformance_vulkan_passthrough_tests on win_optional_gpu_tests_rel right now:

https://ci.chromium.org/p/chromium/builders/luci.chromium.try/win_optional_gpu_tests_rel?limit=200

Could you please take a look and add flaky suppressions if needed? Thanks. Also, do you think we should upgrade the graphics driver on the Win10 NVIDIA machines to pick up a newer / more stable Vulkan version?

Ken, looking at https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Win10%20FYI%20Release%20%28NVIDIA%29?limit=200 I see 4 Vulkan specific failures. It makes sense there would be fewer failures than win_optional_gpu_tests_rel because of test de-duplication.

Also looking at the original failure for this bug I see it seemed to have a random problem initializing the rendering context for Vulkan:

[4676:5632:1220/030356.681:ERROR:gl_surface_egl.cc(537)] EGL Driver message (Error) eglCreateWindowSurface: Internal Vulkan error: Initialization of an object could not be completed for implementation-specific reasons., in ../../third_party/angle/src/libANGLE/renderer/vulkan/SurfaceVk.cpp, initializeImpl:467.
[4676:5632:1220/030356.681:ERROR:gl_surface_egl.cc(1057)] eglCreateWindowSurface failed with error EGL_BAD_SURFACE
[4676:5632:1220/030357.258:ERROR:in_process_command_buffer.cc(482)] ContextResult::kSurfaceFailure: Failed to create surface.

Looking at https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Win10%20FYI%20Exp%20Release%20%28NVIDIA%29?limit=200 however I see zero failures.

Several other failures seem to be due to lost device in the Vulkan back-end. See issue 907544. That makes me think this issue and issue 907544 will be solved by the NVIDIA driver upgrade in issue 887241. 

Cc'ing Adam who was helping with the driver upgrad.e

Can we try to mitigate this problem somehow until the driver update's done? Could you add a blanket Flaky expectation for "*" to webgl_conformance_expectations.py for ['win', 'nvidia', 'passthrough', 'vulkan']? We need to stamp out flakiness on the trybot because it blocks other Chromium developers. Thanks.

Yeah. I'm traveling and will be taking time off. If I don't get to it please just TBR a total removal of the vulkan WebGL tests to myself or Geoff.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/8837121fe4fc086eb66e50e46f4546528631fcf5

commit 8837121fe4fc086eb66e50e46f4546528631fcf5
Author: Jamie Madill <jmadill@chromium.org>
Date: Fri Dec 21 15:50:22 2018

Mark Vulkan WebGL tests as flaky.

Until the drivers can be upgraded.

Bug:  917046 
Tbr: kbr@chromium.org
Change-Id: I7696701e71dbef72159ff8eab392729e3326037e
Reviewed-on: https://chromium-review.googlesource.com/c/1388545
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Commit-Queue: Jamie Madill <jmadill@chromium.org>
Cr-Commit-Position: refs/heads/master@{#618538}
[modify] https://crrev.com/8837121fe4fc086eb66e50e46f4546528631fcf5/content/test/gpu/gpu_tests/webgl_conformance_expectations.py

What's the point of marking the tests as flaky rather than removing them via Skip()? [since we intend to ignore the results]

Right now, the tests can still timeout, causing builds to fail even though they're marked as flaky in webgl_conformance_expectations, right?

The results are not ignored. In the Telemetry-based GPU test suites, a Flaky expectation means "run the test up to three times, and if it fails all three times, report a failure".

If Jamie's flaky expectation for these tests doesn't result in them running reliably then we'll temporarily take them off this bot.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/187f4c9551bac0c7b38047d41772f262fc4267d3

commit 187f4c9551bac0c7b38047d41772f262fc4267d3
Author: Jamie Madill <jmadill@chromium.org>
Date: Fri Jan 11 20:31:08 2019

Remove flaky Win/Vulkan/NV WebGL suppression.

The drivers have been upgraded. Try running the test suites again
to see if the device lost problems have gone away.

Bug:  917046 
Change-Id: I8a8b92e5cdb0f26f7b6f502cbd99c52308767ecd
Tbr: kbr@chromium.org
Reviewed-on: https://chromium-review.googlesource.com/c/1407089
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Commit-Queue: Jamie Madill <jmadill@chromium.org>
Cr-Commit-Position: refs/heads/master@{#622124}
[modify] https://crrev.com/187f4c9551bac0c7b38047d41772f262fc4267d3/content/test/gpu/gpu_tests/webgl_conformance_expectations.py

Don't see any Vulkan flakes in the last 200 builds in 

https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Win10%20FYI%20Release%20%28NVIDIA%29?limit=200

Tentatively closing this out. Will rearrange the bug tags to keep the issues associated with the driver upgrade. <3 labs team for the upgrade.

Fantastic work Jamie driving this to completion!