Make sure experimental features does not apply toward all the layout test |
|||||
Issue descriptionToday, experimental features in blink is applicable toward all the layout tests, whereas only a subset of tests rely on experimental features. This is problematic, because: i) It mysteriously changes behavior due to turning on certain features, leading to lots of delay while debugging (from chrishtr@) ii) There are a number of abandoned features in the "experimental" or "test" state, and turning them off takes work because it causes some tests to fail. (also from chrishtr@) iii) It also make the test binary diverges from what we ship to users (due to the behavior change mentioned in (i) The thread in https://groups.google.com/a/chromium.org/d/msg/blink-dev/1hgFn1oTEvU/adSPLugsCAAJ proposes two solutions that are favorable by blink devs: 1) Put strict limits on the time a feature is allowed to be tested-but-not-shipped. Proposal: should ship in the same Chrome major version in which tests were flipped to test the feature. Any earlier testing should be done with techniques such as virtual layout test suites or RuntimeEnabledFlag override/TEST_P patterns in unittests. 2) Separate "test" from "experimental". "experimental" would merely mean --enable-experimental-web-platform-features turns on the feature. "test" means on for testing. They can be turned on separately.
,
Oct 11
* For me personally, I would prefer solution (2) which only enable --enable-experimental-web-platform-features for a subset of tests. My experience is putting limit of a doing X for developers don't work well in the past (e.g: deadline to fix need-manual-rebaseline tests, fixing failing tests...) and requires lots of handholding. While I generally agree with the philosophy of there should be a time limit on features that are not shipped, I would like to decouple that from making webkit layout test closers to what we ship to users.
,
Oct 11
,
Oct 12
The perf tests currently run with experimental features enabled but not test features: https://cs.chromium.org/chromium/src/tools/perf/benchmarks/blink_perf.py?l=255 We should consider updating the perf benchmarks to take the same approach that we do with layout tests.
,
Oct 12
,
Dec 17
It's unlikely that I can take this effort given I have little left working on this project. Unassign myself so other folks can take this |
|||||
►
Sign in to add a comment |
|||||
Comment 1 by robertma@chromium.org
, Oct 11