New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 888134 link

Starred by 2 users

Issue metadata

Status: Fixed
Owner:
Closed: Sep 24
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Windows
Pri: 1
Type: Bug



Sign in to add a comment

Flaky GPU tests: GPU Windows 10 Debug+Release NVIDIA

Project Member Reported by weiliangc@chromium.org, Sep 21

Issue description

Builders:
https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Win10%20Debug%20%28NVIDIA%29?limit=200
https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Win10%20Release%20%28NVIDIA%29?limit=200

Failure test lists:
screenshot_sync_tests on NVIDIA GPU on Windows on Windows-10 Run on OS: 'Windows-10'
* gpu_tests.screenshot_sync_integration_test.ScreenshotSyncIntegrationTest.ScreenshotSync_GPURasterWithCanvas
* gpu_tests.screenshot_sync_integration_test.ScreenshotSyncIntegrationTest.ScreenshotSync_GPURasterWithDivs
* gpu_tests.screenshot_sync_integration_test.ScreenshotSyncIntegrationTest.ScreenshotSync_SWRasterWithCanvas
* gpu_tests.screenshot_sync_integration_test.ScreenshotSyncIntegrationTest.ScreenshotSync_SWRasterWithDivs

pixel_test on NVIDIA GPU on Windows on Windows-10 Run on OS: 'Windows-10'

Unexpected Failures:
* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_2DCanvasWebGL
* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_BackgroundImage
* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_CSS3DBlueBox
* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_Canvas2DRedBox
* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_Canvas2DUntagged
* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_CanvasDisplayLinearRGBAccelerated2D
* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_CanvasDisplayLinearRGBUnaccelerated2DGPUCompositing
* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_CanvasLowLatency2D
* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_CanvasLowLatencyWebGL
* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_DirectComposition_ComplexOverlays
* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_DirectComposition_Nonroot
* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_DirectComposition_Underlay
* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_GpuRasterization_BlueBox
* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_GpuRasterization_ConcavePaths
* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_OffscreenCanvas2DResizeOnWorker
* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_OffscreenCanvasTransferAfterStyleResize
* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_OffscreenCanvasTransferBeforeStyleResize
* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_OffscreenCanvasTransferToImageBitmap
* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_OffscreenCanvasTransferToImageBitmapWorker
* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_OffscreenCanvasWebGLDefault
* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_OffscreenCanvasWebGLDefaultWorker
* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_OffscreenCanvasWebGLPaintAfterResize
* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_OffscreenCanvasWebglResizeOnWorker
* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_ScissorTestWithPreserveDrawingBuffer
* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_SolidColorBackground
* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_WebGL2_BlitFramebuffer_Result_Displayed
* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_WebGL2_ClearBufferfv_Result_Displayed
* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_WebGLGreenTriangle_AA_Alpha
* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_WebGLGreenTriangle_AA_NoAlpha
* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_WebGLGreenTriangle_NoAA_Alpha
* ... 3 more (33 total) ...


 
When the tests fail, all of the tests are failing.

https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Win10%20Debug%20%28NVIDIA%29/7692
https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Win10%20Debug%20%28NVIDIA%29/7685
https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Win10%20Debug%20%28NVIDIA%29/7679
https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Win10%20Debug%20%28NVIDIA%29/7664

Looking at the failing shards, it looks to me like two bots have broken KVM connections.

https://chromium-swarm.appspot.com/bot?id=build85-m4&sort_stats=total%3Adesc
https://chromium-swarm.appspot.com/bot?id=build89-m4&sort_stats=total%3Adesc

I've clicked the "shut down gracefully" and filed Issue 888140 about having them inspected.

This failure mode looks different: the browser failed to start. It didn't happen again on that bot and there was no minidump so I'm not sure how to make progress on it.

https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Win10%20Debug%20%28NVIDIA%29/7667

(Linking to the failing builds and doing this follow-up is the standard operating procedure. If https://chromium.googlesource.com/chromium/src/+/master/docs/gpu/pixel_wrangling.md should be updated, please suggest changes to it. Thanks!)

Status: Fixed (was: Assigned)
The lab bug has been fixed and hasn't seen more flakes over the weekend, marking fixed.
 Issue 887202  has been merged into this issue.
 Issue 887203  has been merged into this issue.

Sign in to add a comment