Idle PNG/JPEG encoding times out |
|||||||||||||||||||||
Issue descriptionThis test is super flaky: canvas-colorManaged-convertToBlob-roundtrip.html https://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=webkit_layout_tests&tests=fast%2Fcanvas%2Fcolor-space%2Fcanvas-colorManaged-convertToBlob-roundtrip.html In addition to the flaky timeouts (tracked in https://crbug.com/851746 ), this test also fails flakily. I can reproduce this locally with: third_party/blink/tools/run_web_tests.py --debug fast/canvas/color-space/canvas-colorManaged-convertToBlob-roundtrip.html --iterations=100 --no-retry-failures When this fails for me, I get the following: FAIL Test canvas convertToBlob(): mimeType: image/png, blobColorSpace: srgb, blobPixelFormat: 8-8-8-8, source color space: srgb, pixel format: 8-8-8-8, alpha: 1 assert_approx_equals: expected 27 +/- 10 but got 0
,
Jul 6
,
Jul 6
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/f6b163d797fcff563c54fd9718511830cd271b63 commit f6b163d797fcff563c54fd9718511830cd271b63 Author: Reza.Zakerinasab <zakerinasab@chromium.org> Date: Fri Jul 06 15:52:33 2018 Disable canvas-colorManaged-convertToBlob-roundtrip.html These two tests fail on different platforms: fast/canvas/color-space/canvas-colorManaged-convertToBlob-roundtrip.html virtual/gpu/fast/canvas/color-space/canvas-colorManaged-convertToBlob-roundtrip.html Flakiness dashboard: https://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=webkit_layout_tests&tests=fast%2Fcanvas%2Fcolor-space%2Fcanvas-colorManaged-convertToBlob-roundtrip.html TBR=fserb@chromium.org NOTRY=true Bug: 860706 Change-Id: I23f4661dedc1d3f6b5c70c851432b5ac26bb595f Reviewed-on: https://chromium-review.googlesource.com/1127927 Reviewed-by: Mohammad Reza Zakerinasab <zakerinasab@chromium.org> Commit-Queue: Mohammad Reza Zakerinasab <zakerinasab@chromium.org> Cr-Commit-Position: refs/heads/master@{#572980} [modify] https://crrev.com/f6b163d797fcff563c54fd9718511830cd271b63/third_party/WebKit/LayoutTests/TestExpectations
,
Jul 6
,
Jul 6
Pixel mismatches might be Skia-related, but timeouts seem to be a scheduling issue in idle encoding code path.
,
Jul 10
@zakerinasab, there are a bunch of timeouts in other tests too. I'm going to merge the failures into this bug. Are the timeouts also going to be fixed by the skia refactoring? If not, can you triage this to the owner of the idle encoding codepath?
,
Jul 10
Issue 861972 has been merged into this issue.
,
Jul 10
Issue 861880 has been merged into this issue.
,
Jul 10
Issue 861860 has been merged into this issue.
,
Jul 10
Issue 861858 has been merged into this issue.
,
Jul 10
Issue 861859 has been merged into this issue.
,
Jul 10
Issue 861879 has been merged into this issue.
,
Jul 10
@6: I believe timeouts are not related to Skia refactoring. IIUC the problem is that when deadline is met and PostIdleTask() is called in CanvasAsyncBlobCreator::IdleEncodeRows(), we don't get the function called again to continue decoding and resolve the promise. If we change kSlackBeforeDeadline from 1 ms to 0.1 ms we see less timeouts, but only because we less hit the deadline constraint. I'm OOO this week but I'll look into this when I get back.
,
Jul 10
@zakerinasab, I worry that sheriffs will end up skipping many canvas tests due to this. Is there another person who could look into the timeouts while you're out?
,
Jul 10
Assigning to fserb@ since I'm OOO.
,
Jul 10
Test is marked as flaky in TestExpectations, removing from sheriff queue.
,
Jul 10
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/2f87aeb83b5e20a141f091d8dd0b63c19cc0432a commit 2f87aeb83b5e20a141f091d8dd0b63c19cc0432a Author: Dirk Pranke <dpranke@chromium.org> Date: Tue Jul 10 21:19:03 2018 Adjust bug number for a flaky test in TestExpectations. Bug 851746 was duped into bug 860706 , so this updates the entry. TBR=fserb@chromium.org NOTRY=true BUG= 851746 , 860706 Change-Id: I8f1e9e4b2c5e606f20459cc0121ca8c2c65211ee Reviewed-on: https://chromium-review.googlesource.com/1132322 Commit-Queue: Dirk Pranke <dpranke@chromium.org> Reviewed-by: Dirk Pranke <dpranke@chromium.org> Cr-Commit-Position: refs/heads/master@{#573910} [modify] https://crrev.com/2f87aeb83b5e20a141f091d8dd0b63c19cc0432a/third_party/WebKit/LayoutTests/TestExpectations
,
Jul 11
Detected 4 new flakes for test/step "css3/filters/effect-blur-hw.html". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyKwsSBUZsYWtlIiBjc3MzL2ZpbHRlcnMvZWZmZWN0LWJsdXItaHcuaHRtbAw. This message was posted automatically by the chromium-try-flakes app. Since flakiness is ongoing, the issue was moved back into Sheriff Bug Queue (unless already there).
,
Jul 11
Hm. The notes in #c18 are for a different test ...
,
Jul 12
Detected 4 new flakes for test/step "virtual/gpu-rasterization/images/color-profile-mask-image-svg.html". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyTQsSBUZsYWtlIkJ2aXJ0dWFsL2dwdS1yYXN0ZXJpemF0aW9uL2ltYWdlcy9jb2xvci1wcm9maWxlLW1hc2staW1hZ2Utc3ZnLmh0bWwM. This message was posted automatically by the chromium-try-flakes app. Since flakiness is ongoing, the issue was moved back into Sheriff Bug Queue (unless already there).
,
Jul 12
,
Jul 12
Detected 3 new flakes for test/step "css3/filters/filter-repaint-composited-fallback-crash.html". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyRQsSBUZsYWtlIjpjc3MzL2ZpbHRlcnMvZmlsdGVyLXJlcGFpbnQtY29tcG9zaXRlZC1mYWxsYmFjay1jcmFzaC5odG1sDA. This message was posted automatically by the chromium-try-flakes app. Since flakiness is ongoing, the issue was moved back into Sheriff Bug Queue (unless already there).
,
Jul 12
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/f0d8f15b89a20e5d4f113a064ae4ea651c069ee9 commit f0d8f15b89a20e5d4f113a064ae4ea651c069ee9 Author: Kim Paulhamus <kpaulhamus@chromium.org> Date: Thu Jul 12 22:58:26 2018 Disable a couple flaky LayoutTests - css3/filters/filter-repaint-composited-fallback-crash.html - virtual/gpu-rasterization/images/color-profile-mask-image-svg.html Tbr: tkent@chromium.org Bug: 860706 Change-Id: Iaedb2f2960eb8ed975562db4d0d0e05dfbdc4cfb Reviewed-on: https://chromium-review.googlesource.com/1135815 Commit-Queue: Kim Paulhamus <kpaulhamus@chromium.org> Reviewed-by: Kim Paulhamus <kpaulhamus@chromium.org> Cr-Commit-Position: refs/heads/master@{#574763} [modify] https://crrev.com/f0d8f15b89a20e5d4f113a064ae4ea651c069ee9/third_party/WebKit/LayoutTests/TestExpectations
,
Jul 12
,
Jul 14
Detected 3 new flakes for test/step "css3/filters/effect-blur-hw.html". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyKwsSBUZsYWtlIiBjc3MzL2ZpbHRlcnMvZWZmZWN0LWJsdXItaHcuaHRtbAw. This message was posted automatically by the chromium-try-flakes app. Since flakiness is ongoing, the issue was moved back into Sheriff Bug Queue (unless already there).
,
Jul 14
Reporting to stale-flakes-reports@google.com to investigate why this issue has been in the appropriate queue 5 times or more.
,
Jul 16
Issue 862907 has been merged into this issue.
,
Jul 16
Adding brianosman@ and mtklein@ since they are working on the Skia refactoring.
,
Jul 16
,
Jul 16
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/7f5bf99be754b97e24f63be6c3620cef970c730d commit 7f5bf99be754b97e24f63be6c3620cef970c730d Author: Reza.Zakerinasab <zakerinasab@chromium.org> Date: Mon Jul 16 14:34:31 2018 Adjust the bug numbers for flaky tests This CL adjusts the bug numbers for flaky tests that belong to crbug.com/860706 TBR=fserb@chromium.org NOTRY=true BUG= 860706 Change-Id: I585d047a6b631d8fc6643c3990973d4c78b554df Reviewed-on: https://chromium-review.googlesource.com/1138184 Reviewed-by: Mohammad Reza Zakerinasab <zakerinasab@chromium.org> Commit-Queue: Mohammad Reza Zakerinasab <zakerinasab@chromium.org> Cr-Commit-Position: refs/heads/master@{#575251} [modify] https://crrev.com/7f5bf99be754b97e24f63be6c3620cef970c730d/third_party/WebKit/LayoutTests/TestExpectations
,
Jul 16
,
Jul 16
,
Jul 16
Issue 863544 has been merged into this issue.
,
Jul 16
Issue 863550 has been merged into this issue.
,
Jul 16
Issue 863462 has been merged into this issue.
,
Jul 16
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/575d896934aafa29f7c867f773488e118dfcc1e9 commit 575d896934aafa29f7c867f773488e118dfcc1e9 Author: Reza.Zakerinasab <zakerinasab@chromium.org> Date: Mon Jul 16 15:00:48 2018 Adjust bug numbers for flaky tests This CL adjusts the bug numbers for flaky tests that belong to crbug.com/860706 TBR=fserb@chromium.org NOTRY=true BUG= 860706 Change-Id: Iacf8abbf1fefe41fa24cb443c1b34015bbeb4310 Reviewed-on: https://chromium-review.googlesource.com/1138335 Commit-Queue: Mohammad Reza Zakerinasab <zakerinasab@chromium.org> Reviewed-by: Mohammad Reza Zakerinasab <zakerinasab@chromium.org> Cr-Commit-Position: refs/heads/master@{#575265} [modify] https://crrev.com/575d896934aafa29f7c867f773488e118dfcc1e9/third_party/WebKit/LayoutTests/TestExpectations
,
Jul 16
To separate timeouts from failures, which seem to have different roots, failure report moved to https://bugs.chromium.org/p/chromium/issues/detail?id=863906.
,
Jul 16
Adding gab@ to cc since the source of timeout issues seems to be MessageLoop dropping the idle encoding task when we get close enough to the deadline and repost the idle task in https://cs.chromium.org/chromium/src/third_party/blink/renderer/core/html/canvas/canvas_async_blob_creator.cc?sq=package:chromium&g=0&l=428. gab@, can you take a look and see if this can be a side effect of your recent CLs?
,
Jul 16
Issue 864222 has been merged into this issue.
,
Jul 16
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/c31fce6a69e3ccc34f598cb6ef2b82595bbfcc92 commit c31fce6a69e3ccc34f598cb6ef2b82595bbfcc92 Author: Xiaocheng Hu <xiaochengh@chromium.org> Date: Mon Jul 16 22:55:50 2018 Mark virtual/gpu/fast/canvas/canvas-shadow-source-in.html as flaky timeout on Mac Bug: 860706 Change-Id: I763095091eead566f92aacf421e6a45cd7a4bc25 Tbr: xiaochengh@chromium.org NoTry: True Reviewed-on: https://chromium-review.googlesource.com/1138998 Reviewed-by: Xiaocheng Hu <xiaochengh@chromium.org> Commit-Queue: Xiaocheng Hu <xiaochengh@chromium.org> Cr-Commit-Position: refs/heads/master@{#575460} [modify] https://crrev.com/c31fce6a69e3ccc34f598cb6ef2b82595bbfcc92/third_party/WebKit/LayoutTests/TestExpectations
,
Jul 17
Issue 864297 has been merged into this issue.
,
Jul 17
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/f6f14207e2af48683d8b5905f31dcf0ec47f86bf commit f6f14207e2af48683d8b5905f31dcf0ec47f86bf Author: Xiaocheng Hu <xiaochengh@chromium.org> Date: Tue Jul 17 01:35:38 2018 Mark virtual/gpu/fast/canvas/canvas-blending-pattern-over-pattern.html as flaky timeout on Mac Change-Id: I17dfd6ca8e94db3b610cca28e0feca2b63bb3d01 Tbr: xiaochengh@chromium.org NoTry: True Bug: 860706 Reviewed-on: https://chromium-review.googlesource.com/1139435 Reviewed-by: Xiaocheng Hu <xiaochengh@chromium.org> Commit-Queue: Xiaocheng Hu <xiaochengh@chromium.org> Cr-Commit-Position: refs/heads/master@{#575516} [modify] https://crrev.com/f6f14207e2af48683d8b5905f31dcf0ec47f86bf/third_party/WebKit/LayoutTests/TestExpectations
,
Jul 17
Re. 38 : idle tasks are not a concept of base::MessageLoop (so no, not caused by my recent MessageLoop CLs). The Blink scheduler handles idle tasks, not sure why reposting on deadline would result in the reposted task getting lost). +skyostil,altimin,alexclare ^^^
,
Jul 17
#38: Do you mean the newly posted idle task never runs? Idle tasks always run at the lowest priority, so if some other repeating task keeps hogging the thread, then the idle work will never run. If it's critical that we get to that work, it sounds like we should post a fallback delayed task that ensures the work gets done.
,
Jul 17
Layout tests do not play nicely with idle tasks — you have to call testRunner.runIdleTasks manually to run them.
,
Jul 17
#44: I don't think anything else is hogging the thread. We are posting delayed tasks to the thread to check the state of the idle encoding, but I'm not sure if the selected timeouts respect the timeout limit of the layout test runner or not. I'll investigate. #45: IIUC the scenario is different here. Instead of asking the developer to pass the toBlob call inside a requestIdleCallback and then call testRunner.runIdleTasks manually, developer just calls toBlob and we do idle PNG/JPEG encoding inside chromium. Still the issue seems only happening in test runner. Calling the test in a loop 1000x in content_shell does not cause timeout.
,
Jul 18
Detected 3 new flakes for test/step "virtual/gpu/fast/canvas/canvas-blending-image-over-pattern.html". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNySgsSBUZsYWtlIj92aXJ0dWFsL2dwdS9mYXN0L2NhbnZhcy9jYW52YXMtYmxlbmRpbmctaW1hZ2Utb3Zlci1wYXR0ZXJuLmh0bWwM. This message was posted automatically by the chromium-try-flakes app. Since flakiness is ongoing, the issue was moved back into Sheriff Bug Queue (unless already there).
,
Jul 20
Issue 865829 has been merged into this issue.
,
Jul 20
Issue 865624 has been merged into this issue.
,
Jul 20
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/5fe22ac286f213ef8bad7fa4fc19261114bdb087 commit 5fe22ac286f213ef8bad7fa4fc19261114bdb087 Author: Reza.Zakerinasab <zakerinasab@chromium.org> Date: Fri Jul 20 15:29:23 2018 Disable idle encoding for layout tests This CL disables idle encoding in CanvasAsyncBlobCreator for layout tests. The current timeout deadlines make it possible to wait more than 6 seconds for the idle encoding to finish, which results in layout tests timeout. This change also modifies the way slack time before idle task deadline is calculated, trying to allow idle encoding proceed if we have enough time to encode a smaller image row. Bug: 860706 Change-Id: I7fdebcb84cc5ae7840a34126a0b9f1a6c0037f9e Reviewed-on: https://chromium-review.googlesource.com/1142136 Reviewed-by: Justin Novosad <junov@chromium.org> Commit-Queue: Mohammad Reza Zakerinasab <zakerinasab@chromium.org> Cr-Commit-Position: refs/heads/master@{#576871} [modify] https://crrev.com/5fe22ac286f213ef8bad7fa4fc19261114bdb087/third_party/WebKit/LayoutTests/TestExpectations [modify] https://crrev.com/5fe22ac286f213ef8bad7fa4fc19261114bdb087/third_party/blink/renderer/core/html/canvas/canvas_async_blob_creator.cc [modify] https://crrev.com/5fe22ac286f213ef8bad7fa4fc19261114bdb087/third_party/blink/renderer/core/html/canvas/canvas_async_blob_creator.h [modify] https://crrev.com/5fe22ac286f213ef8bad7fa4fc19261114bdb087/third_party/blink/renderer/core/html/canvas/canvas_async_blob_creator_test.cc [modify] https://crrev.com/5fe22ac286f213ef8bad7fa4fc19261114bdb087/third_party/blink/renderer/platform/runtime_enabled_features.json5
,
Jul 20
The landed CL should fix the issue for the layout tests that were affected by the flaky PNG/JPEG idle encoding behavior. I close this bug for now and monitor the waterfall to see if the timeouts return, in that case we would need a new bug to find out the root cause. |
|||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||
Comment 1 by zakerinasab@chromium.org
, Jul 6