New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 594216 link

Starred by 1 user

Issue metadata

Status: Available
Owner: ----
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: All
Pri: 3
Type: Bug



Sign in to add a comment

Virtual layout tests should inherit SLOW expectations from their non-virtual counterparts.

Project Member Reported by alex...@chromium.org, Mar 11 2016

Issue description

In  https://crbug.com/594189 , I discovered that the test virtual/spv2/fast/overflow/lots-of-sibling-inline-boxes.html was frequently timing out, even though its non-virtual counterpart, fast/overflow/lots-of-sibling-inline-boxes.html has the SLOW expectation.  lukasza@ and I were surprised to find that SLOW wasn't being inherited here, even though it seems like it should be.

There are similarly duplicated expectations for other tests as well:
 crbug.com/420008  inspector/tracing/ [ Slow ]
 crbug.com/420008  virtual/threaded/inspector/tracing/ [ Slow ]

phajdan.jr@, dpranke@, jochen@: any thoughts on whether this might be worth fixing?
 
Cc: tansell@chromium.org qyears...@chromium.org ojan@chromium.org
Components: Blink>Tools>Test
Labels: b
Status: Available (was: Untriaged)
Yes, I think this is probably worth fixing.

There's also the related problem that if you specify 

inspector/tracing/ [ Slow ]
inspector/tracing/foo.html [ Failure ]

the [ Slow ] is dropped from foo.html. Some times this is the right thing to do, but most of the time it isn't, and is probably why we have so many flaky timeouts for inspector tests. 

(I've been thinking about this myself recently).

Unfortunately, we don't have a [ Fast ] equivalent to [ Pass ]; we probably either  need to add one or make the propagation rules for Slow work entirely differently than Failure.
Labels: -b

Comment 3 by ojan@chromium.org, Mar 11 2016

Having virtual tests inherit the slowness of the non virtual test suite seems reasonable to me. It's a bit magic, but it's what you want 99% of the time and the consequences of marking too many things as Slow isn't that bad for the remaining 1%.

Naina recently added lint checks that you can only use Slow to the SlowTests file and can't put anything other than Slow in the SlowTests file. The files actually merge their expectations. So, I think the related problem Dirk pointed out is not actually a problem, right?

Long ago when I was working on this code, if a file had a specific expectations line, it would override *any* expectations on enclosing directories (and it had to).

Unless we've specifically added code to *not* do that for Slow tests -- and I don't think we have based on what've seen over the past week -- I think it's still a problem.

Comment 5 by ojan@chromium.org, Mar 11 2016

I don't think that's how it works if we're merging across files. I might be wrong though.

I think if you have:
SlowTests:
foo [ Slow ]

TestExpectations:
foo/bar.html [ Failure ]

Then foo/bar.html will get [ Slow Failure ].
Hm. At first test, it looks like you're right, but now I'm puzzled by why I'm seeing so many inspector timeouts, then ...

Comment 7 by sshru...@google.com, Mar 18 2016

Components: -Blink>Tools>Test Blink>ToolsTest
Fixing component name after Monorail migration
Cc: -qyears...@chromium.org
Owner: qyears...@chromium.org
Status: Assigned (was: Available)

Comment 9 by sshru...@google.com, Apr 18 2016

Components: -Blink>ToolsTest Blink>Infra
Blink>ToolsTest renamed to Blink>Infra
Labels: Test-Layout
Components: -Blink>LayoutTests
Deprecating component:Blink>LayoutTests, to use label Test=Layout instead.
Status: Available (was: Assigned)
Summary: Virtual layout tests should inherit SLOW expectations from their non-virtual counterparts. (was: Virtual layout tests should inherit SLOW expectations from their non-virtual counterparts)
Cc: qyears...@chromium.org
Owner: ----
Hmmm... Alex found out that lack of inheritance appears to be by design according to https://chromium.googlesource.com/chromium/src/+/master/docs/testing/layout_tests.md

I am still not sure if I agree with this design...
Cc: yhirano@chromium.org
FWIW, the problem of lack of inheritance can be seen in https://crrev.com/2505933002 and in https://crrev.com/2547733002
Yes. Feel free to change the implementation if you find that we reduce the total number of expectations by doing so. I don't think it's clear a priori which is better.

Comment 17 by ojan@chromium.org, Mar 7 2017

Cc: -ojan@chromium.org
Cc: -tansell@chromium.org
This is a P2 but hasn't seen any activity in over a year. Is it really a P2, and if so should someone be assigned? FWIW, seems like a P3 to me.
P3 is fine.
Labels: -Pri-2 Pri-3

Sign in to add a comment