WebView also?

Nope the webview team added their own version of these tests long ago: https://cs.chromium.org/chromium/src/android_webview/tools/system_webview_shell/test/data/webexposed/, AFAIK they're adequate.

May be relevant: https://bugs.chromium.org/p/chromium/issues/detail?id=667069#c14

Testing CL: https://codereview.chromium.org/2578253003/

Note that LayoutTests don't currently run on Android L+ (issue 567947), but we do have a KitKat bot at https://build.chromium.org/p/chromium.webkit/builders/WebKit%20Android%20(Nexus4)

It looks like there's no CQ coverage though, so when I add these to SmokeTests it means CLs that ship new APIs on Android without updating webexposed test results (eg. PaymentRequest) will pass CQ but then get reverted for breaking the tree.  jbudorick/aelias: Is that OK or should we explore adding Android LayoutTests to the CQ as well?

In my opinion it's not worth the burden.  As you've just observed with http://crbug.com/675027, most layout tests are designed to isolate out platform-specific behaviors, and we have very few #if OS(ANDROID) in Blink, so there's not much increased coverage overall.

> so when I add these to SmokeTests it means CLs that ship new APIs on Android without updating webexposed test results (eg. PaymentRequest)

The best solution to this kind of thing is to make the PaymentRequest layout tests run on Linux.  There's probably no strong reason why they don't.

I think the best systematic improvement here would be to simply delete the Android layout test system, which would force this to happen, instead of making teams add this kind of rarely run, flaky test coverage and then wrongly consider their system properly tested.

Personally I think the entire idea of multi-platform layout test runners is a legacy of WebKit and with the uniformity we've gotten with Blink, we would be better off running them only on a single platform.  And this especially holds true for Android.

I can see that argument for LayoutTests generally.  But this is really a special case about tracking what API surface area we're shipping on different platforms.  In the specific issue that led to this bug (https://bugs.chromium.org/p/chromium/issues/detail?id=652148#c4) we do already have tests that run on Linux under a status=experimental RuntimeEnabledFeature.  But there's Android-specific code that enables the feature by default.  So the ONLY way to see what APIs are enabled/disabled by default is to run something on a real Android build.

You could argue we should re-write the entire webexposed test suite to be content browser tests instead of LayoutTests or something.  But JS LayoutTests with -expected.txt files are a pretty convenient way to write this sort of test IMHO.

OK.  The problem is that layout tests are really, really slow on a real phone.  So I'd prefer a solution along the lines of a virtual/ suite on Linux that simulates the Android environment more accurately.

Ok, I'm not going to redesign the whole webexposed test system right now for this - returning to Assigned.

Do we know WHY layout tests are "really, really slow" on a real phone?  They're just running web pages (i.e. the #1 job of Chrome) - seems like something that's pretty important to be able to reliably and efficiently test, right?  I.e. if our testing is constrained by the fact that loading web pages is too slow/unreliable to do on Android at scale, then I think we might have a bigger problem to worry about ;-)

Related to  bug 412732 .

Moving back to available (and assigning a hotlist).