New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 921279 link

Starred by 2 users

Issue metadata

Status: Assigned
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Bug
Flaky-Test: gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_DirectComposition_ComplexOverlays



Sign in to add a comment

gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_DirectComposition_ComplexOverlays is flaky

Project Member Reported by Findit, Jan 12

Issue description

Owner: enne@chromium.org
Status: Assigned (was: Untriaged)
could you please take a look at this?
Cc: enne@chromium.org
Components: Tests>Flaky Internals>GPU
Labels: Hotlist-PixelWrangler
Owner: senorblanco@chromium.org
Assigning to current pixel wrangler to disable.
This hasn't flaked in the last few days. If it flakes again by the end of my shift, I'll disable it.
Labels: -Sheriff-Chromium
Removing bug from SoM.

Comment 7 by senorblanco@chromium.org, Jan 16 (6 days ago)

FYI, CL waiting for OWNERS review here: https://chromium-review.googlesource.com/c/chromium/src/+/1414873

Comment 8 by dalecur...@chromium.org, Jan 16 (6 days ago)

That CL is still WIP so OWNERS can't see it.

Comment 9 by dalecur...@chromium.org, Jan 16 (6 days ago)

Cc: kbr@chromium.org zmo@chromium.org

Comment 10 by zmo@chromium.org, Jan 16 (6 days ago)

Owner: magchen@chromium.org
Maggie: can you take a look at the failed runs' logs and see what's going on there?

Comment 11 by magchen@chromium.org, Jan 17 (6 days ago)

Reference image is not available form the link. The rotated four-color frame doesn't not show in the generated image. Only see the black square at the upper right corner. 

Comment 12 by magchen@chromium.org, Jan 17 (6 days ago)

The error message shows the value from the read pixel is [255, 255, 255], which matches the generated image where most pixels are white except for the upper left corner.

From the failing log:

  Traceback (most recent call last):
    _RunGpuTest at content\test\gpu\gpu_tests\gpu_integration_test.py:155
      self.RunActualGpuTest(url, *args)
    RunActualGpuTest at content\test\gpu\gpu_tests\pixel_integration_test.py:146
      self.RunLegacyPixelTest(test_path, do_page_action, args)
    RunLegacyPixelTest at content\test\gpu\gpu_tests\pixel_integration_test.py:168
      tab, page.name, screenshot, page.expected_colors, dpr)
    _ValidateScreenshotSamples at content\test\gpu\gpu_tests\cloud_storage_integration_test_base.py:398
      self.GetParsedCommandLineOptions().test_machine_name)
    _CompareScreenshotSamples at content\test\gpu\gpu_tests\cloud_storage_integration_test_base.py:197
      str(actual_color.b) + "]")
    fail at .swarming_module\bin\Lib\unittest\case.py:410
      raise self.failureException(msg)
  AssertionError: Expected pixel at [60, 10] (actual pixel (60, 10))  to be [255, 255, 15] but got [255, 255, 255]
  
  Locals:
    msg : 'Expected pixel at [60, 10] (actual pixel (60, 10))  to be [255, 255, 15] but got [255, 255, 255]'

Comment 13 by magchen@chromium.org, Jan 17 (6 days ago)

I have created another CL to set the expected result of this test to be "Flaky" when tested on Window nVidia, instead of removing the test completely. So we can continue running this test on Intel.
Project Member

Comment 14 by bugdroid1@chromium.org, Jan 17 (6 days ago)

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/83dd146194b6b2604330994a30e53bac1166c32e

commit 83dd146194b6b2604330994a30e53bac1166c32e
Author: Maggie Chen <magchen@chromium.org>
Date: Thu Jan 17 04:40:06 2019

Temporarily disable Pixel_DirectComposition_ComplexOverlays pixel test on Windows nVidia machines

This test has been flaky on nVidia for a few days. We should re-enable it once the bad CL is found.

Bug: 921279
Change-Id: I36a4371910b96c7a2858b6114a81d225e3f7cb0e
Reviewed-on: https://chromium-review.googlesource.com/c/1416135
Commit-Queue: Zhenyao Mo <zmo@chromium.org>
Reviewed-by: Zhenyao Mo <zmo@chromium.org>
Cr-Commit-Position: refs/heads/master@{#623580}
[modify] https://crrev.com/83dd146194b6b2604330994a30e53bac1166c32e/content/test/gpu/gpu_tests/pixel_expectations.py

Comment 15 by iclell...@chromium.org, Jan 17 (5 days ago)

Labels: -Sheriff-Chromium
Going to remove this from the sheriff queue. If it flakes again, Findit will notify sheriffs then.

Comment 16 by magchen@chromium.org, Jan 18 (4 days ago)

I am working on a solution to improve the test stability. The length of this video clip will be extended to 5 seconds and the screen shot will be taken 2 seconds after the playback starts.
Project Member

Comment 17 by bugdroid1@chromium.org, Jan 19 (4 days ago)

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/5193113e7fea7f9d5b61e6169411b3577c761b5f

commit 5193113e7fea7f9d5b61e6169411b3577c761b5f
Author: Maggie Chen <magchen@chromium.org>
Date: Sat Jan 19 06:10:49 2019

Improve Pixel_DirectComposition_ComplexOverlays test stability

This test is flaky. We have already seen some failures in bots. When it fails,
there is no data in the captured frame. It's possible the video playback has not
started at the time of screen capture.

This CL uses a different way to detect if the video is ready for screen shot.
With HTMLMediaElement, when the video currentTime > 0, a video frame should be
available for test.

Bug:921279

Change-Id: Icda3a29fe813d4d1be383e873dfb2d729c5d9f79
Reviewed-on: https://chromium-review.googlesource.com/c/1417351
Reviewed-by: Zhenyao Mo <zmo@chromium.org>
Reviewed-by: Dale Curtis <dalecurtis@chromium.org>
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Reviewed-by: Frank Liberato <liberato@chromium.org>
Commit-Queue: Maggie Chen <magchen@chromium.org>
Cr-Commit-Position: refs/heads/master@{#624452}
[modify] https://crrev.com/5193113e7fea7f9d5b61e6169411b3577c761b5f/content/test/data/gpu/pixel_video_complex_overlays.html

Comment 18 by senorblanco@chromium.org, Today (14 hours ago)

Should we remove the Flaky expectation in pixel_expectations.py now?

Comment 19 by magchen@chromium.org, Today (12 hours ago)

Yes, we should. But I would like to monitor the bot status for two more weeks to ensure the new method works. Then I will remove the Flaky expectation code.

Sign in to add a comment