Automate updating LayoutTests/SlowTests |
||||
Issue descriptionThis bug is a follow-up to the comment in https://crbug.com/874695#c8 It would be great if populating LayoutTests/SlowTests could be automated - for example a test could be marked as [ Slow ] whenever a bot notices that the test passes, but took more than X seconds.
,
Aug 17
,
Aug 17
Just out of curious: if we don't specify that a layout test is slow, what are the consequences? The test could pass or fail randomly?
,
Aug 17
Without the [ Slow ] expectation, the test would flakily time out more often. Marking a test as [ Slow ] increases the timeout - this means that a test is less likely to hit the timeout due to random variations in the test runtime. From third_party/WebKit/LayoutTests/SlowTests: # This file should only mark tests as Slow. If they are failing in some other way, that listing should be in TestExpectations. # Lines listed in both files will have the *union* of the expectations. # # In general, we should list tests in this file if they are slower than ~30% of the default timeout time, as we # can see a 3x variance or more in test times on the bots. In practice, this means that any test that takes longer # than 2 seconds in Release mode or 6 seconds in Debug mode should be listed here. # # Further documentation: # https://chromium.googlesource.com/chromium/src/+/master/docs/testing/layout_test_expectations.md
,
Nov 8
@ned - feel free to either take this or triage further?
,
Dec 17
It's unlikely for me to get to this bug, so unassign myself |
||||
►
Sign in to add a comment |
||||
Comment 1 by lukasza@chromium.org
, Aug 17