Create a sustainable approach for testing slimming paint v2 |
||||
Issue descriptionWe currently mark all spv2 tests as skip with specific tests marked as passing or failing. This doesn't work with the rebaseline bot and will be confusing to other teams as we pass more tests. We should switch this around.
,
Jun 8 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/b06c412aba54b65354d24bc1291d1acae745f686 commit b06c412aba54b65354d24bc1291d1acae745f686 Author: pdr <pdr@chromium.org> Date: Wed Jun 08 03:16:46 2016 Add initial baselines for enable-slimming-paint-v2 FlagExpectations This patch adds an initial set of baselines (linux-debug only) for tests run with --additional-driver-flag=--enable-slimming-paint-v2. All but the compositing, fast, and svg directories have been skipped. To run tests for spv2: run-webkit-tests --debug --additional-driver-flag=--enable-slimming-paint-v2 BUG= 601275 Review-Url: https://codereview.chromium.org/2048823003 Cr-Commit-Position: refs/heads/master@{#398470} [add] https://crrev.com/b06c412aba54b65354d24bc1291d1acae745f686/third_party/WebKit/LayoutTests/FlagExpectations/enable-slimming-paint-v2
,
Jul 22 2016
Pretty much just waiting on crbug.com/622865 which is under active development. Marking as a blocker.
,
Jul 28 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/5335696e1522503d52c438c73e0d52ca1caa4dae commit 5335696e1522503d52c438c73e0d52ca1caa4dae Author: pdr <pdr@chromium.org> Date: Thu Jul 28 21:36:22 2016 Update spv2 expectations for linux in preparation for the new trybot This patch updates the spv2 expectations to what they are today on linux. Previously, I generated this list from my mac. No attempt has been made to triage these results. BUG= 601275 Review-Url: https://codereview.chromium.org/2188923002 Cr-Commit-Position: refs/heads/master@{#408490} [modify] https://crrev.com/5335696e1522503d52c438c73e0d52ca1caa4dae/third_party/WebKit/LayoutTests/FlagExpectations/enable-slimming-paint-v2
,
Jul 29 2016
The next step is to trigger our new trybot using a presubmit.py to check for changes to specific directories.
,
Aug 2 2016
,
Aug 9 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/515107379c3c624bf9d2dcb36578ffa7e2ca7004 commit 515107379c3c624bf9d2dcb36578ffa7e2ca7004 Author: pdr <pdr@chromium.org> Date: Tue Aug 09 22:33:27 2016 Add suppressions for more spv2-specific tests This patch adds the last suppressions needed for turning on the spv2 trybot in https://codereview.chromium.org/2200003002. BUG= 601275 Review-Url: https://codereview.chromium.org/2231513002 Cr-Commit-Position: refs/heads/master@{#410858} [modify] https://crrev.com/515107379c3c624bf9d2dcb36578ffa7e2ca7004/third_party/WebKit/LayoutTests/FlagExpectations/enable-slimming-paint-v2
,
Aug 10 2016
This will be landing shortly. To keep everything linked, here's the paint-dev post: https://groups.google.com/a/chromium.org/forum/#!topic/paint-dev/s2pcooZuJRw
,
Aug 10 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/b02023adefda06d9afa8ec55ff892221b073e7bd commit b02023adefda06d9afa8ec55ff892221b073e7bd Author: pdr <pdr@chromium.org> Date: Wed Aug 10 02:50:16 2016 Automatically trigger spv2 tryjobs on paint-related changes This patch adds a presubmit hook for adding a spv2 cq tryjob to all patches affecting "paint-related" directories which currently includes core/paint, core/layout, platform/graphics/paint, and platform/graphics/compositing. BUG= 601275 Review-Url: https://codereview.chromium.org/2200003002 Cr-Commit-Position: refs/heads/master@{#410932} [modify] https://crrev.com/b02023adefda06d9afa8ec55ff892221b073e7bd/third_party/WebKit/PRESUBMIT.py
,
Aug 12 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/07fd19897e1811586bbdcce52a1f331dec3086af commit 07fd19897e1811586bbdcce52a1f331dec3086af Author: pdr <pdr@chromium.org> Date: Fri Aug 12 21:14:36 2016 Remove all but a few spv2 virtual tests Per the new slimming paint v2 (spv2) testing plan [1], we will only use virtual tests to catch basic errors on the main CQ. This approach is simpler for the rest of the blink team which may not understand our unique virtual skip-and-pass testing approach. This patch removes all but a few spv2 virtual tests which should catch people completely breaking spv2 but not much more. [1] https://docs.google.com/document/d/1QCM912Dr6u38DqyQqd7pxQxDy8FFOoWMMDq7uAXqKdA/view BUG= 601275 Review-Url: https://codereview.chromium.org/2238123004 Cr-Commit-Position: refs/heads/master@{#411770} [modify] https://crrev.com/07fd19897e1811586bbdcce52a1f331dec3086af/third_party/WebKit/LayoutTests/TestExpectations
,
Aug 12 2016
Fixed! This also caught a real failure today in https://codereview.chromium.org/2238853005. For posterity, the current spv2 linux-release numbers: Total "paint" tests: 16,851 (svg, compositing, and fast) Total spv2 failures: 2,716
,
Aug 30 2016
Just FYI, if you're the command-line type, this bot can be triggered manually with: git cl try -b linux_layout_tests_slimming_paint_v2 -m master.tryserver.chromium.linux |
||||
►
Sign in to add a comment |
||||
Comment 1 by pdr@chromium.org
, Jun 7 2016Owner: pdr@chromium.org
Status: Started (was: Available)
Summary: Create a sustainable approach for testing slimming paint v2 (was: Switch spv2 layouttests to be pass by default and mark failures)