Issue metadata
Sign in to add a comment
|
gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_DirectComposition_ComplexOverlays is flaky |
||||||||||||||||||||||
Issue descriptiongpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_DirectComposition_ComplexOverlays is flaky. Findit has detected 3 flake occurrences of this test within the past 24 hours. List of all flake occurrences can be found at: https://findit-for-me.appspot.com/flake/occurrences?key=ag9zfmZpbmRpdC1mb3ItbWVykQELEgVGbGFrZSKFAWNocm9taXVtQHRlbGVtZXRyeV9ncHVfaW50ZWdyYXRpb25fdGVzdEBncHVfdGVzdHMucGl4ZWxfaW50ZWdyYXRpb25fdGVzdC5QaXhlbEludGVncmF0aW9uVGVzdC5QaXhlbF9EaXJlY3RDb21wb3NpdGlvbl9Db21wbGV4T3ZlcmxheXMM. Unless the culprit CL is found and reverted, please disable this test first within 30 minutes then find an appropriate owner. If the result above is wrong, please file a bug using this link: https://bugs.chromium.org/p/chromium/issues/entry?status=Unconfirmed&labels=Pri-1,Test-Findit-Wrong&components=Tools%3ETest%3EFindit%3EFlakiness&summary=%5BFindit%5D%20Flake%20Detection%20-%20Wrong%20result%3A%20gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_DirectComposition_ComplexOverlays&comment=Link%20to%20flake%20details%3A%20https://findit-for-me.appspot.com/flake/occurrences?key=ag9zfmZpbmRpdC1mb3ItbWVykQELEgVGbGFrZSKFAWNocm9taXVtQHRlbGVtZXRyeV9ncHVfaW50ZWdyYXRpb25fdGVzdEBncHVfdGVzdHMucGl4ZWxfaW50ZWdyYXRpb25fdGVzdC5QaXhlbEludGVncmF0aW9uVGVzdC5QaXhlbF9EaXJlY3RDb21wb3NpdGlvbl9Db21wbGV4T3ZlcmxheXMM Automatically posted by the findit-for-me app (https://goo.gl/Ot9f7N).
,
Jan 14
Assigning to current pixel wrangler to disable.
,
Jan 14
This hasn't flaked in the last few days. If it flakes again by the end of my shift, I'll disable it.
,
Jan 14
Removing bug from SoM.
,
Jan 16
Seems like it's still flaking? https://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=pixel_test&tests=gpu_tests%2Fpixel_integration_test%2FPixelIntegrationTest%2FPixel_DirectComposition_ComplexOverlays Caveat I might have a real bug, but those failures aren't from my tries... https://chromium-review.googlesource.com/c/chromium/src/+/1410382
,
Jan 16
(6 days ago)
gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_DirectComposition_ComplexOverlays is flaky. Findit has detected 3 new flake occurrences of this test. List of all flake occurrences can be found at: https://findit-for-me.appspot.com/flake/occurrences?key=ag9zfmZpbmRpdC1mb3ItbWVykQELEgVGbGFrZSKFAWNocm9taXVtQHRlbGVtZXRyeV9ncHVfaW50ZWdyYXRpb25fdGVzdEBncHVfdGVzdHMucGl4ZWxfaW50ZWdyYXRpb25fdGVzdC5QaXhlbEludGVncmF0aW9uVGVzdC5QaXhlbF9EaXJlY3RDb21wb3NpdGlvbl9Db21wbGV4T3ZlcmxheXMM. Since these tests are still flaky, this issue has been moved back onto the Sheriff Bug Queue if it hasn't already. If the result above is wrong, please file a bug using this link: https://bugs.chromium.org/p/chromium/issues/entry?status=Unconfirmed&labels=Pri-1,Test-Findit-Wrong&components=Tools%3ETest%3EFindit%3EFlakiness&summary=%5BFindit%5D%20Flake%20Detection%20-%20Wrong%20result%3A%20gpu_tests.pixel_integration_test.PixelIntegrationTest.Pixel_DirectComposition_ComplexOverlays&comment=Link%20to%20flake%20details%3A%20https://findit-for-me.appspot.com/flake/occurrences?key=ag9zfmZpbmRpdC1mb3ItbWVykQELEgVGbGFrZSKFAWNocm9taXVtQHRlbGVtZXRyeV9ncHVfaW50ZWdyYXRpb25fdGVzdEBncHVfdGVzdHMucGl4ZWxfaW50ZWdyYXRpb25fdGVzdC5QaXhlbEludGVncmF0aW9uVGVzdC5QaXhlbF9EaXJlY3RDb21wb3NpdGlvbl9Db21wbGV4T3ZlcmxheXMM Automatically posted by the findit-for-me app (https://goo.gl/Ot9f7N).
,
Jan 16
(6 days ago)
FYI, CL waiting for OWNERS review here: https://chromium-review.googlesource.com/c/chromium/src/+/1414873
,
Jan 16
(6 days ago)
That CL is still WIP so OWNERS can't see it.
,
Jan 16
(6 days ago)
,
Jan 16
(6 days ago)
Maggie: can you take a look at the failed runs' logs and see what's going on there?
,
Jan 17
(6 days ago)
Reference image is not available form the link. The rotated four-color frame doesn't not show in the generated image. Only see the black square at the upper right corner.
,
Jan 17
(6 days ago)
The error message shows the value from the read pixel is [255, 255, 255], which matches the generated image where most pixels are white except for the upper left corner.
From the failing log:
Traceback (most recent call last):
_RunGpuTest at content\test\gpu\gpu_tests\gpu_integration_test.py:155
self.RunActualGpuTest(url, *args)
RunActualGpuTest at content\test\gpu\gpu_tests\pixel_integration_test.py:146
self.RunLegacyPixelTest(test_path, do_page_action, args)
RunLegacyPixelTest at content\test\gpu\gpu_tests\pixel_integration_test.py:168
tab, page.name, screenshot, page.expected_colors, dpr)
_ValidateScreenshotSamples at content\test\gpu\gpu_tests\cloud_storage_integration_test_base.py:398
self.GetParsedCommandLineOptions().test_machine_name)
_CompareScreenshotSamples at content\test\gpu\gpu_tests\cloud_storage_integration_test_base.py:197
str(actual_color.b) + "]")
fail at .swarming_module\bin\Lib\unittest\case.py:410
raise self.failureException(msg)
AssertionError: Expected pixel at [60, 10] (actual pixel (60, 10)) to be [255, 255, 15] but got [255, 255, 255]
Locals:
msg : 'Expected pixel at [60, 10] (actual pixel (60, 10)) to be [255, 255, 15] but got [255, 255, 255]'
,
Jan 17
(6 days ago)
I have created another CL to set the expected result of this test to be "Flaky" when tested on Window nVidia, instead of removing the test completely. So we can continue running this test on Intel.
,
Jan 17
(6 days ago)
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/83dd146194b6b2604330994a30e53bac1166c32e commit 83dd146194b6b2604330994a30e53bac1166c32e Author: Maggie Chen <magchen@chromium.org> Date: Thu Jan 17 04:40:06 2019 Temporarily disable Pixel_DirectComposition_ComplexOverlays pixel test on Windows nVidia machines This test has been flaky on nVidia for a few days. We should re-enable it once the bad CL is found. Bug: 921279 Change-Id: I36a4371910b96c7a2858b6114a81d225e3f7cb0e Reviewed-on: https://chromium-review.googlesource.com/c/1416135 Commit-Queue: Zhenyao Mo <zmo@chromium.org> Reviewed-by: Zhenyao Mo <zmo@chromium.org> Cr-Commit-Position: refs/heads/master@{#623580} [modify] https://crrev.com/83dd146194b6b2604330994a30e53bac1166c32e/content/test/gpu/gpu_tests/pixel_expectations.py
,
Jan 17
(5 days ago)
Going to remove this from the sheriff queue. If it flakes again, Findit will notify sheriffs then.
,
Jan 18
(4 days ago)
I am working on a solution to improve the test stability. The length of this video clip will be extended to 5 seconds and the screen shot will be taken 2 seconds after the playback starts.
,
Jan 19
(4 days ago)
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/5193113e7fea7f9d5b61e6169411b3577c761b5f commit 5193113e7fea7f9d5b61e6169411b3577c761b5f Author: Maggie Chen <magchen@chromium.org> Date: Sat Jan 19 06:10:49 2019 Improve Pixel_DirectComposition_ComplexOverlays test stability This test is flaky. We have already seen some failures in bots. When it fails, there is no data in the captured frame. It's possible the video playback has not started at the time of screen capture. This CL uses a different way to detect if the video is ready for screen shot. With HTMLMediaElement, when the video currentTime > 0, a video frame should be available for test. Bug:921279 Change-Id: Icda3a29fe813d4d1be383e873dfb2d729c5d9f79 Reviewed-on: https://chromium-review.googlesource.com/c/1417351 Reviewed-by: Zhenyao Mo <zmo@chromium.org> Reviewed-by: Dale Curtis <dalecurtis@chromium.org> Reviewed-by: Kenneth Russell <kbr@chromium.org> Reviewed-by: Frank Liberato <liberato@chromium.org> Commit-Queue: Maggie Chen <magchen@chromium.org> Cr-Commit-Position: refs/heads/master@{#624452} [modify] https://crrev.com/5193113e7fea7f9d5b61e6169411b3577c761b5f/content/test/data/gpu/pixel_video_complex_overlays.html
,
Today
(14 hours ago)
Should we remove the Flaky expectation in pixel_expectations.py now?
,
Today
(12 hours ago)
Yes, we should. But I would like to monitor the bot status for two more weeks to ensure the new method works. Then I will remove the Flaky expectation code. |
|||||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||||
Comment 1 by battre@chromium.org
, Jan 14Status: Assigned (was: Untriaged)