New issue
Advanced search Search tips

Issue 844849 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: May 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Windows
Pri: 1
Type: Bug-Regression

Blocking:
issue 844262



Sign in to add a comment

pixel_test and screenshot_sync_tests failing on Win7

Project Member Reported by jdarpinian@chromium.org, May 19 2018

Issue description

All the Win7 FYI bots are failing. The first failing build is here:

https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Win7%20FYI%20Release%20%28AMD%29/1236

There are a lot of potential causes in the blamelist and nothing immediately stands out. I tried bisecting with bisect_builds.py but the builds it downloads don't repro the problem. Currently running some tryjobs to see if that repros.

Here are the tests that are failing:

* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_CSS3DBlueBox
* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_DirectComposition_Video_VP9
* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_GpuRasterization_ConcavePaths
* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_OffscreenCanvasAccelerated2D
* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_OffscreenCanvasTransferAfterStyleResize
* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_OffscreenCanvasTransferToImageBitmap
* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_SolidColorBackground
* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_Video_VP9
* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_WebGLGreenTriangle_AA_Alpha
* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_WebGLGreenTriangle_NoAA_Alpha
* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_WebGL_PremultipliedAlpha_False
* gpu_tests.screenshot_sync_integration_test.ScreenshotSyncIntegrationTest.ScreenshotSync_GPURasterWithCanvas
* gpu_tests.screenshot_sync_integration_test.ScreenshotSyncIntegrationTest.ScreenshotSync_SWRasterWithCanvas
* gpu_tests.screenshot_sync_integration_test.ScreenshotSyncIntegrationTest.ScreenshotSync_SWRasterWithDivs

 

Comment 1 by kbr@chromium.org, May 19 2018

Cc: danakj@chromium.org sunn...@chromium.org kylec...@chromium.org
Labels: -Type-Bug -Pri-2 Pri-1 Type-Bug-Regression
Yikes. This is the second breakage of the Win7 GPU tests in two weeks. Maybe we need to have Win7 tests on the CQ after all.

The new "gpu_manual_try_win7_nvidia_rel" trybot can be used to send tryjobs to the Win7 bots (I see you've found that already per https://chromium-review.googlesource.com/1067072 , for example), but there's very little capacity.

Agree that there are a lot of suspect CLs. The failure mode is that the rendering and snapshotting is desynchronized. See for example these pixel failures:
http://chromium-browser-gpu-tests.commondatastorage.googleapis.com/view_test_results.html?11526d32023303a84a17b46e295033434fc4e66a_Win7_FYI_Release_NVIDIA__telemetry

from this build:
https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Win7%20FYI%20Release%20%28NVIDIA%29/1008

(The link is in the pixel_test logs)

This indicates to me it's likely a GPU side change. These stand out:

Remove idle work polling interval override
https://chromium-review.googlesource.com/1064981

Do not consider pending queries as idle work in passthrough decoder
https://chromium-review.googlesource.com/1065232

I asked Sunny about those and he didn't think they would cause this, but I agree they sound suspicious. I'll try reverting them anyway. BTW do you know how to use git cl try? I thought the flag "-b gpu_manual_try_win7_nvidia_rel" would allow running a single bot but it didn't work, it just scheduled a full CQ run instead.

Comment 3 by kbr@chromium.org, May 19 2018

Sorry, not 100% sure how or whether "git cl try -b" works at this point. The best way to try a job on a single bot is to use the Gerrit UI.

SGTM to try reverting anything that seems suspicious. The top priority is to get the bot green again. Please add "Bug:  844849 " to all the reverts if possible so they're all linked here and so that we can more easily find what to reland later.

Found it, it was:
 Remove idle work polling interval override
https://chromium-review.googlesource.com/1064981

Here's the revert:
https://chromium-review.googlesource.com/c/chromium/src/+/1067083

Comment 5 by kbr@chromium.org, May 19 2018

Very nice work James. LGTM'd and CQ'd.

Project Member

Comment 6 by bugdroid1@chromium.org, May 19 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/7bac98f5d17817cdee6de59d5bdc1f5e8426131c

commit 7bac98f5d17817cdee6de59d5bdc1f5e8426131c
Author: James Darpinian <jdarpinian@chromium.org>
Date: Sat May 19 22:30:51 2018

Revert "Remove idle work polling interval override"

This reverts commit db2744fe6d080718ac44c3810dee794e4d3400d7.

Caused failures on Win7 FYI trybots:
https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Win7%20FYI%20Release%20%28AMD%29/1236


* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_CSS3DBlueBox
* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_DirectComposition_Video_VP9
* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_GpuRasterization_ConcavePaths
* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_OffscreenCanvasAccelerated2D
* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_OffscreenCanvasTransferAfterStyleResize
* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_OffscreenCanvasTransferToImageBitmap
* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_SolidColorBackground
* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_Video_VP9
* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_WebGLGreenTriangle_AA_Alpha
* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_WebGLGreenTriangle_NoAA_Alpha
* gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_WebGL_PremultipliedAlpha_False
* gpu_tests.screenshot_sync_integration_test.ScreenshotSyncIntegrationTest.ScreenshotSync_GPURasterWithCanvas
* gpu_tests.screenshot_sync_integration_test.ScreenshotSyncIntegrationTest.ScreenshotSync_SWRasterWithCanvas
* gpu_tests.screenshot_sync_integration_test.ScreenshotSyncIntegrationTest.ScreenshotSync_SWRasterWithDivs

Bug:  844849 

Cq-Include-Trybots: luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel
Change-Id: I57da87a7242d332af1d3642e4161c61fb20b7836
Reviewed-on: https://chromium-review.googlesource.com/1067083
Commit-Queue: James Darpinian <jdarpinian@chromium.org>
Commit-Queue: Kenneth Russell <kbr@chromium.org>
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Cr-Commit-Position: refs/heads/master@{#560183}
[modify] https://crrev.com/7bac98f5d17817cdee6de59d5bdc1f5e8426131c/gpu/ipc/service/command_buffer_stub.cc

Status: Fixed (was: Started)

Comment 8 by kbr@chromium.org, May 22 2018

Blocking: 844262

Sign in to add a comment