New issue
Advanced search Search tips

Issue 875419 link

Starred by 2 users

Issue metadata

Status: Untriaged
Owner: ----
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug



Sign in to add a comment

Automate updating LayoutTests/SlowTests

Project Member Reported by lukasza@chromium.org, Aug 17

Issue description

This bug is a follow-up to the comment in https://crbug.com/874695#c8

It would be great if populating LayoutTests/SlowTests could be automated - for example a test could be marked as [ Slow ] whenever a bot notices that the test passes, but took more than X seconds.
 
Owner: dpranke@chromium.org
dpranke@ - could you PTAL and help find a good owner for this?
Components: -Blink>Infra Infra>Flakiness
Just out of curious: if we don't specify that a layout test is slow, what are the consequences? The test could pass or fail randomly?
Without the [ Slow ] expectation, the test would flakily time out more often. 
 Marking a test as [ Slow ] increases the timeout - this means that a test is less likely to hit the timeout due to random variations in the test runtime.

From third_party/WebKit/LayoutTests/SlowTests:

# This file should only mark tests as Slow. If they are failing in some other way, that listing should be in TestExpectations.
# Lines listed in both files will have the *union* of the expectations.
#
# In general, we should list tests in this file if they are slower than ~30% of the default timeout time, as we
# can see a 3x variance or more in test times on the bots. In practice, this means that any test that takes longer
# than 2 seconds in Release mode or 6 seconds in Debug mode should be listed here.
#
# Further documentation:
# https://chromium.googlesource.com/chromium/src/+/master/docs/testing/layout_test_expectations.md
Cc: robertma@chromium.org
Owner: nedngu...@google.com
@ned - feel free to either take this or triage further?
Owner: ----
It's unlikely for me to get to this bug, so unassign myself

Sign in to add a comment