New issue
Advanced search Search tips

Issue 807751 link

Starred by 2 users

Issue metadata

Status: Assigned
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug

Blocked on:
issue 835690



Sign in to add a comment

Switch the layout tests to the new test expectations format

Project Member Reported by dpranke@chromium.org, Jan 31 2018

Issue description

We want to change the test expectations format that the layout tests use so that it is more generic and can be used for other test formats (like telemetry and gtests).

The new format is specced at bit.ly/chromium-test-list-format (at the moment; this should move into a markdown doc in-repo).

This bug will be used to track the conversion work. 
 

Comment 1 by ajuma@chromium.org, Apr 3 2018

Just checking for an update, since this marked P2.
No update, due to other demands on my time. If someone else wanted to work on this I'd be happy to hand it off.
Labels: -Pri-2 Pri-1
Blockedon: 835690
Components: Build
Labels: Build-Tools-TYP
Owner: ----
Status: Available (was: Assigned)
Update: I've posted the first full attempt at implementing the new syntax in typ in https://crrev.com/c/1023650 . Even once that lands, though, there will still be additional work to do to migrate blinkpy to typ's parser and migrate the existing expectations files over.

I'm not going to mark this `started` until I've actually started on that work.

I'm actually also going to un-assign myself, on the random chance that someone else will want to work on this before I get to it :). 
dpranke@, nobody else did get to work on this, should you be assigned again? Or is it not a P1?
Labels: -Pri-1 Pri-2
Owner: nedngu...@google.com
I will take this one
Cc: rmhasan@google.com dpranke@chromium.org
 Issue 905411  has been merged into this issue.
Status: Started (was: Available)
Project Member

Comment 11 by bugdroid1@chromium.org, Nov 21

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/d6ca4cfdd36c92bcb7b8d63a5628c68d09aebcbb

commit d6ca4cfdd36c92bcb7b8d63a5628c68d09aebcbb
Author: Ned Nguyen <nednguyen@google.com>
Date: Wed Nov 21 19:44:21 2018

Update logic to merge platforms in wpt_expectations_updater

This updates wpt_expectations_updater to merge specifiers of same
platform type to a simpler specifier when possible, but still keep separate
expectation lines for different specifiers.

This is needed for later CLs which will update Layout test expectation formats
to no longer have mutual exclusive specifiers on the same expectation line
(CL https://chromium-review.googlesource.com/c/chromium/src/+/1336439)

Bug: 807751
Change-Id: Icd47c6d5d3f35a789ed266ee20fbb60dca382ac3
Reviewed-on: https://chromium-review.googlesource.com/c/1336912
Commit-Queue: Ned Nguyen <nednguyen@google.com>
Reviewed-by: Robert Ma <robertma@chromium.org>
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Cr-Commit-Position: refs/heads/master@{#610174}
[modify] https://crrev.com/d6ca4cfdd36c92bcb7b8d63a5628c68d09aebcbb/third_party/blink/tools/blinkpy/w3c/wpt_expectations_updater.py
[modify] https://crrev.com/d6ca4cfdd36c92bcb7b8d63a5628c68d09aebcbb/third_party/blink/tools/blinkpy/w3c/wpt_expectations_updater_unittest.py

Project Member

Comment 12 by bugdroid1@chromium.org, Nov 22

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/f7343f668b24c4445680edb4416877ca2e729587

commit f7343f668b24c4445680edb4416877ca2e729587
Author: Ned Nguyen <nednguyen@google.com>
Date: Thu Nov 22 18:54:33 2018

Update TestExpectation to no longer include lines with mutual exclusive specifiers

This also update test expectation linter to error out on lines with mutual
exclusive specifiers

The Expectation files are updated using the script in https://chromium-review.googlesource.com/c/chromium/src/+/1336387

This is the first part of the plan to migrate WebkitLayout test expectations to the new format.

Bug: 807751
Change-Id: Iad2296a99804fb16d9b474229b68c0d47d3fe1eb
Reviewed-on: https://chromium-review.googlesource.com/c/1336439
Commit-Queue: Ned Nguyen <nednguyen@google.com>
Reviewed-by: Robert Ma <robertma@chromium.org>
Cr-Commit-Position: refs/heads/master@{#610468}
[modify] https://crrev.com/f7343f668b24c4445680edb4416877ca2e729587/third_party/WebKit/LayoutTests/NeverFixTests
[modify] https://crrev.com/f7343f668b24c4445680edb4416877ca2e729587/third_party/WebKit/LayoutTests/SlowTests
[modify] https://crrev.com/f7343f668b24c4445680edb4416877ca2e729587/third_party/WebKit/LayoutTests/TestExpectations
[modify] https://crrev.com/f7343f668b24c4445680edb4416877ca2e729587/third_party/blink/tools/blinkpy/web_tests/models/test_expectations.py
[modify] https://crrev.com/f7343f668b24c4445680edb4416877ca2e729587/third_party/blink/tools/blinkpy/web_tests/models/test_expectations_unittest.py

Owner: estaab@chromium.org
I no longer work on this. Reassign to Erik to triage.

Erik: this may make sense to let Rakib drives this once he's done with migrating gpu tests to typ expectation format
Components: Test>WebTests
Status: Assigned (was: Started)

Sign in to add a comment