New issue
Advanced search Search tips

Issue 601275 link

Starred by 2 users

Issue metadata

Status: Fixed
Owner:
Closed: Aug 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug

Blocked on:
issue 622865
issue 633707



Sign in to add a comment

Create a sustainable approach for testing slimming paint v2

Project Member Reported by pdr@chromium.org, Apr 7 2016

Issue description

We currently mark all spv2 tests as skip with specific tests marked as passing or failing. This doesn't work with the rebaseline bot and will be confusing to other teams as we pass more tests. We should switch this around.
 

Comment 1 by pdr@chromium.org, Jun 7 2016

Labels: -Pri-3 Pri-2
Owner: pdr@chromium.org
Status: Started (was: Available)
Summary: Create a sustainable approach for testing slimming paint v2 (was: Switch spv2 layouttests to be pass by default and mark failures)
Updated plan:
https://docs.google.com/document/d/1QCM912Dr6u38DqyQqd7pxQxDy8FFOoWMMDq7uAXqKdA/view

We're going to switch to using FlagExpectations and manually run tests with --additional-driver-flag=--enable-slimming-paint-v2. FlagExpectations works like TestExpectations.
Project Member

Comment 2 by bugdroid1@chromium.org, Jun 8 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/b06c412aba54b65354d24bc1291d1acae745f686

commit b06c412aba54b65354d24bc1291d1acae745f686
Author: pdr <pdr@chromium.org>
Date: Wed Jun 08 03:16:46 2016

Add initial baselines for enable-slimming-paint-v2 FlagExpectations

This patch adds an initial set of baselines (linux-debug only) for
tests run with --additional-driver-flag=--enable-slimming-paint-v2.
All but the compositing, fast, and svg directories have been skipped.

To run tests for spv2:
run-webkit-tests --debug --additional-driver-flag=--enable-slimming-paint-v2

BUG= 601275 

Review-Url: https://codereview.chromium.org/2048823003
Cr-Commit-Position: refs/heads/master@{#398470}

[add] https://crrev.com/b06c412aba54b65354d24bc1291d1acae745f686/third_party/WebKit/LayoutTests/FlagExpectations/enable-slimming-paint-v2

Comment 3 by pdr@chromium.org, Jul 22 2016

Blockedon: 622865
Pretty much just waiting on crbug.com/622865 which is under active development. Marking as a blocker.
Project Member

Comment 4 by bugdroid1@chromium.org, Jul 28 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/5335696e1522503d52c438c73e0d52ca1caa4dae

commit 5335696e1522503d52c438c73e0d52ca1caa4dae
Author: pdr <pdr@chromium.org>
Date: Thu Jul 28 21:36:22 2016

Update spv2 expectations for linux in preparation for the new trybot

This patch updates the spv2 expectations to what they are today on
linux. Previously, I generated this list from my mac. No attempt has
been made to triage these results.

BUG= 601275 

Review-Url: https://codereview.chromium.org/2188923002
Cr-Commit-Position: refs/heads/master@{#408490}

[modify] https://crrev.com/5335696e1522503d52c438c73e0d52ca1caa4dae/third_party/WebKit/LayoutTests/FlagExpectations/enable-slimming-paint-v2

Comment 5 by pdr@chromium.org, Jul 29 2016

The next step is to trigger our new trybot using a presubmit.py to check for changes to specific directories.

Comment 6 by pdr@chromium.org, Aug 2 2016

Blockedon: 633707
Project Member

Comment 7 by bugdroid1@chromium.org, Aug 9 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/515107379c3c624bf9d2dcb36578ffa7e2ca7004

commit 515107379c3c624bf9d2dcb36578ffa7e2ca7004
Author: pdr <pdr@chromium.org>
Date: Tue Aug 09 22:33:27 2016

Add suppressions for more spv2-specific tests

This patch adds the last suppressions needed for turning on the spv2
trybot in https://codereview.chromium.org/2200003002.

BUG= 601275 

Review-Url: https://codereview.chromium.org/2231513002
Cr-Commit-Position: refs/heads/master@{#410858}

[modify] https://crrev.com/515107379c3c624bf9d2dcb36578ffa7e2ca7004/third_party/WebKit/LayoutTests/FlagExpectations/enable-slimming-paint-v2

Comment 8 by pdr@chromium.org, Aug 10 2016

This will be landing shortly. To keep everything linked, here's the paint-dev post:
https://groups.google.com/a/chromium.org/forum/#!topic/paint-dev/s2pcooZuJRw
Project Member

Comment 9 by bugdroid1@chromium.org, Aug 10 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/b02023adefda06d9afa8ec55ff892221b073e7bd

commit b02023adefda06d9afa8ec55ff892221b073e7bd
Author: pdr <pdr@chromium.org>
Date: Wed Aug 10 02:50:16 2016

Automatically trigger spv2 tryjobs on paint-related changes

This patch adds a presubmit hook for adding a spv2 cq tryjob to all
patches affecting "paint-related" directories which currently includes
core/paint, core/layout, platform/graphics/paint, and
platform/graphics/compositing.

BUG= 601275 

Review-Url: https://codereview.chromium.org/2200003002
Cr-Commit-Position: refs/heads/master@{#410932}

[modify] https://crrev.com/b02023adefda06d9afa8ec55ff892221b073e7bd/third_party/WebKit/PRESUBMIT.py

Project Member

Comment 10 by bugdroid1@chromium.org, Aug 12 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/07fd19897e1811586bbdcce52a1f331dec3086af

commit 07fd19897e1811586bbdcce52a1f331dec3086af
Author: pdr <pdr@chromium.org>
Date: Fri Aug 12 21:14:36 2016

Remove all but a few spv2 virtual tests

Per the new slimming paint v2 (spv2) testing plan [1], we will only
use virtual tests to catch basic errors on the main CQ. This approach
is simpler for the rest of the blink team which may not understand our
unique virtual skip-and-pass testing approach.

This patch removes all but a few spv2 virtual tests which should catch
people completely breaking spv2 but not much more.

[1] https://docs.google.com/document/d/1QCM912Dr6u38DqyQqd7pxQxDy8FFOoWMMDq7uAXqKdA/view

BUG= 601275 

Review-Url: https://codereview.chromium.org/2238123004
Cr-Commit-Position: refs/heads/master@{#411770}

[modify] https://crrev.com/07fd19897e1811586bbdcce52a1f331dec3086af/third_party/WebKit/LayoutTests/TestExpectations

Comment 11 by pdr@chromium.org, Aug 12 2016

Status: Fixed (was: Started)
Fixed! This also caught a real failure today in https://codereview.chromium.org/2238853005.

For posterity, the current spv2 linux-release numbers:
Total "paint" tests: 16,851 (svg, compositing, and fast)
Total spv2 failures: 2,716

Comment 12 by pdr@chromium.org, Aug 30 2016

Just FYI, if you're the command-line type, this bot can be triggered manually with:
git cl try -b linux_layout_tests_slimming_paint_v2 -m master.tryserver.chromium.linux

Sign in to add a comment