New issue
Advanced search Search tips

Issue 894569 link

Starred by 4 users

Issue metadata

Status: Assigned
Owner: ----
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug



Sign in to add a comment

Make sure experimental features does not apply toward all the layout test

Project Member Reported by nedngu...@google.com, Oct 11

Issue description

Today, experimental features in blink is applicable toward all the layout tests, whereas only a subset of tests rely on experimental features.

This is problematic, because:
i) It mysteriously changes behavior due to turning on certain features, leading to lots of delay while debugging (from chrishtr@)
ii) There are a number of abandoned features in the "experimental" or "test" state, and turning them off takes work because it causes some tests to fail. (also from chrishtr@)
iii) It also make the test binary diverges from what we ship to users (due to the behavior change mentioned in (i)

The thread in https://groups.google.com/a/chromium.org/d/msg/blink-dev/1hgFn1oTEvU/adSPLugsCAAJ proposes two solutions that are favorable by blink devs:

1) Put strict limits on the time a feature is allowed to be tested-but-not-shipped. Proposal: should ship in the same Chrome major version in which tests were flipped to test the feature. Any earlier testing should be done with techniques such as virtual layout test suites or RuntimeEnabledFlag override/TEST_P patterns in unittests.

2) Separate "test" from "experimental". "experimental" would merely mean --enable-experimental-web-platform-features turns on the feature. "test" means on for testing. They can be turned on separately.
 
Cc: robertma@chromium.org
* For me personally, I would prefer solution (2) which only enable --enable-experimental-web-platform-features for a subset of tests. My experience is putting limit of a doing X for developers don't work well in the past (e.g: deadline to fix need-manual-rebaseline tests, fixing failing tests...) and requires lots of handholding. 

While I generally agree with the philosophy of there should be a time limit on features that are not shipped, I would like to decouple that from making webkit layout test closers to what we ship to users.
Status: Assigned (was: Untriaged)
Cc: sullivan@chromium.org
The perf tests currently run with experimental features enabled but not test features:
https://cs.chromium.org/chromium/src/tools/perf/benchmarks/blink_perf.py?l=255

We should consider updating the perf benchmarks to take the same approach that we do with layout tests.
Owner: nedngu...@google.com
Cc: estaab@chromium.org
Owner: ----
It's unlikely that I can take this effort given I have little left working on this project. Unassign myself so other folks can take this 

Sign in to add a comment