Debug flaky layout tests with --enable-display-compositor-pixel-dump enabled |
|
Issue descriptionWhen --enable-display-compositor-pixel-dump is enabled, it used to also add the --run-all-compositor-stages-before-draw flag, which extended the frame deadline to infinity. The intention for this was that we should allow as much time as needed to draw all content prior to capturing the pixel output. This CL enables the compositor pixel dumps but removes the infinite deadline, by default: https://crrev.com/c/1213864 The problem is that when both of these flags are enabled, many layout tests become flaky. On a local run, the list below all showed flakyness or outright failure. Many of them are svg-attribute-interpolation related, all of which share a single .js test runner. Those tests typically show a repeating pattern of timeout-pass-timeout-pass. These need to be debugged to see why they become flaky. In almost all of these cases, if the --run-all-compositor-stages-before-draw flag is removed, the flakiness goes away completely. virtual/android/fullscreen/rendering/backdrop-video.html virtual/android/rootscroller/rootscroller-during-fullscreen.html virtual/android/rootscroller/set-root-scroller.html virtual/android/rootscroller/set-rootscroller-before-load.html virtual/android/url-bar/bottom-and-top-fixed-sticks-to-top.html virtual/paint-touchaction-rects/fast/events/touch/gesture/gesture-scrollbar-touchscreen-fling.html virtual/scroll_customization/fast/events/touch/gesture/touch-gesture-scroll-input-field.html virtual/threaded/animations/responsive/viewport-unit-transform-responsive.html virtual/threaded/animations/responsive/viewport-unit-translate-responsive.html virtual/threaded/animations/svg-attribute-composition/svg-slope-composition.html virtual/threaded/animations/svg-attribute-interpolation/svg-amplitude-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-azimuth-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-baseFrequency-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-bias-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-calc-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-class-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-clipPathUnits-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-cx-cy-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-d-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-diffuseConstant-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-divisor-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-dx-dy-length-list-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-dx-dy-number-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-edgeMode-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-elevation-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-exponent-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-filterUnits-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-fx-fy-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-gradientTransform-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-gradientUnits-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-height-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-href-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-in-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-intercept-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-k1-k2-k3-k4-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-kernelMatrix-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-kernelUnitLength-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-lengthAdjust-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-limitingConeAngle-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-markerHeight-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-markerUnits-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-markerWidth-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-maskContentUnits-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-maskUnits-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-method-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-mode-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-numOctaves-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-offset-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-operator-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-order-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-orient-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-pathLength-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-patternContentUnits-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-patternTransform-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-patternUnits-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-points-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-pointsAtX-pointsAtY-pointsAtZ-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-preserveAlpha-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-preserveAspectRatio-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-primitiveUnits-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-r-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-radius-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-refX-refY-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-result-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-rotate-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-rx-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-scale-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-seed-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-slope-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-spacing-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-specularConstant-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-spreadMethod-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-startOffset-interpolation-001.html virtual/threaded/animations/svg-attribute-interpolation/svg-startOffset-interpolation-002.html virtual/threaded/animations/svg-attribute-interpolation/svg-stdDeviation-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-stitchTiles-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-surfaceScale-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-tableValues-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-target-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-targetX-targetY-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-textLength-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-transform-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-type-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-values-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-viewBox-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-width-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-x-y-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-x1-x2-y1-y2-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-xChannelSelector-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-y-list-interpolation.html virtual/threaded/animations/svg-attribute-interpolation/svg-z-interpolation.html virtual/threaded/animations/timing/animation-duration-infinite.html virtual/threaded/fast/events/pinch/gesture-pinch-zoom-prevent-in-handler.html virtual/threaded/fast/events/pinch/scroll-visual-viewport-send-boundary-events.html virtual/threaded/fast/scrolling/editor-command-scroll-page-scale.html virtual/threaded/fast/scrolling/scrollbar-prevent-default.html virtual/threaded/http/tests/devtools/tracing/decode-resize.js virtual/threaded/http/tests/devtools/tracing/frame-model-instrumentation.js virtual/threaded/http/tests/devtools/tracing/timeline-paint/layer-tree.js virtual/threaded/http/tests/devtools/tracing/timeline-paint/paint-profiler-update.js virtual/threaded/http/tests/devtools/tracing/timeline-paint/timeline-paint-and-multiple-style-invalidations.js virtual/threaded/http/tests/devtools/tracing/timeline-paint/timeline-paint-image.js virtual/threaded/http/tests/devtools/tracing/timeline-paint/timeline-paint-with-layout-invalidations-on-deleted-node.js virtual/threaded/http/tests/devtools/tracing/timeline-paint/timeline-paint-with-layout-invalidations.js virtual/threaded/http/tests/devtools/tracing/timeline-paint/timeline-paint-with-style-recalc-invalidations.js virtual/threaded/http/tests/devtools/tracing/timeline-paint/timeline-paint.js virtual/threaded/http/tests/devtools/tracing/timeline-paint/update-layer-tree.js virtual/threaded/printing/quirks-percentage-height-body.html virtual/threaded/printing/quirks-percentage-height.html virtual/threaded/printing/simultaneous-position-float-change.html virtual/threaded/printing/standards-percentage-heights.html virtual/threaded/printing/subframes-percentage-height.html virtual/threaded/printing/webgl-repeated-printing-preservedrawingbuffer.html virtual/threaded/printing/webgl-repeated-printing.html virtual/threaded/synthetic_gestures/synthetic-pinch-zoom-gesture-touchscreen-desktop.html
,
Oct 26
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/c41b2d08296c7d720343f716641e40c9c2f68ad0 commit c41b2d08296c7d720343f716641e40c9c2f68ad0 Author: Mason Freed <masonfreed@chromium.org> Date: Fri Oct 26 17:20:21 2018 Enable display compositor pixel dumps by default. SHERIFFS: PLEASE DO NOT REVERT THIS CL BECAUSE OF A SMALL AMOUNT OF LAYOUT TEST FLAKINESS. If a few layout tests begin to show flakiness after this CL lands, please add them to TestExpectations and email me (masonfreed@) to investigate. Though I tried to identify all potentially flaky tests, a few may have slipped through. With this CL, the --enable-display-compositor-pixel-dump flag becomes the default for content_shell. With this flag in place, layout test pixel dumps are performed from the browser side, instead of from the renderer side. Note that to avoid a significant amount of layout test flakiness, another change was also made to not add the --run-all-compositor-stages-before-draw flag by default. There is a bug (crbug.com/894613) tracking that problem separately. With the flip of this switch, several modifications had to be made to the TestExpectations file. First, there are a number of tests that change their appearance slightly when being captured from the browser, and these tests need to be rebaselined. These are summarized below, and will be rebaselined as a separate CL, once this one lands and has had time to stabilize. These bugs track the items added to TestExpectations: - crbug.com/887140: HDR support - crbug.com/881040 : Media controls now contain an overflow menu. - crbug.com/667551 : A bunch of tests are listed under this bug, and just require rebaselining to fix non-material single-pixel antialiasing failures. - crbug.com/891427: These either start failing, or become flaky, when the --enable-display-compositor-pixel-dump flag is enabled. They need to be debugged prior to re-enabling. - crbug.com/895556 : These tests double their background size when the flag is enabled. They need to be fixed or rebaselined. Bug: 667551 , 891427, 881040 , 887140, 894613, 895556 Cq-Include-Trybots: luci.chromium.try:android_optional_gpu_tests_rel Change-Id: I46946e6377f659c9dedc0dfaa20e7658e8cc519d Reviewed-on: https://chromium-review.googlesource.com/c/1213864 Commit-Queue: Mason Freed <masonfreed@chromium.org> Reviewed-by: Philip Jägenstedt <foolip@chromium.org> Reviewed-by: Peter Beverloo <peter@chromium.org> Reviewed-by: Stephen Chenney <schenney@chromium.org> Cr-Commit-Position: refs/heads/master@{#603127} [modify] https://crrev.com/c41b2d08296c7d720343f716641e40c9c2f68ad0/content/shell/app/shell_main_delegate.cc [modify] https://crrev.com/c41b2d08296c7d720343f716641e40c9c2f68ad0/docs/testing/writing_layout_tests.md [modify] https://crrev.com/c41b2d08296c7d720343f716641e40c9c2f68ad0/third_party/WebKit/LayoutTests/TestExpectations |
|
►
Sign in to add a comment |
|
Comment 1 by masonfreed@chromium.org
, Oct 16