Several not-very-related WebView instrumentation tests failing in a cluster |
|||||||||
Issue description"org.chromium.android_webview.test.AwSettingsTest#testUseWideViewportLayoutWidthNoQuirks" is flaky. This issue was created automatically by the chromium-try-flakes app. Please find the right owner to fix the respective test/step and assign this issue to them. If the step/test is infrastructure-related, please add Infra-Troopers label and change issue status to Untriaged. When done, please remove the issue from Sheriff Bug Queue by removing the Sheriff-Chromium label. We have detected 3 recent flakes. List of all flakes can be found at https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyYgsSBUZsYWtlIldvcmcuY2hyb21pdW0uYW5kcm9pZF93ZWJ2aWV3LnRlc3QuQXdTZXR0aW5nc1Rlc3QjdGVzdFVzZVdpZGVWaWV3cG9ydExheW91dFdpZHRoTm9RdWlya3MM. Flaky tests should be disabled within 30 minutes unless culprit CL is found and reverted. Please see more details here: https://sites.google.com/a/chromium.org/dev/developers/tree-sheriffs/sheriffing-bug-queues#triaging-auto-filed-flakiness-bugs
,
Apr 26 2017
The file with the test itself was touched 3 weeks ago.
,
Apr 26 2017
I can't identify the issue, I am going to add @RetryOnFailure.
,
Apr 26 2017
,
Apr 26 2017
torne@ (as an OWNER of android_webview), could you have a look? ntfschr@ (as the last one to touch that file), could you have a look?
,
Apr 26 2017
Issue 715562 has been merged into this issue.
,
Apr 27 2017
boliu@ suggested that the tests does not flake anymore and torne@ suggested that because the reports ( issue 715494 , issue 715492 and issue 715491 ) contain same 3 patches, it is unlikely that this is a flake. Thus, we are not disabling them. torne@, please have look or reassign.
,
Apr 27 2017
So this and the related issues have a couple of things in common: 1) Failed in just a couple of recent builds, mostly the same exact builds (or significantly overlapping). Historical flake data has few/no other failures for these tests. 2) All failures are in builds where 10+ not-very-related tests failed, but with similar failures: incorrect display scale, incorrect width, or just timeout waiting for events. It's not always the same exact set of tests, but there is overlap. 3) In all the logs I've looked at, even though it's sharded across a number of test devices the majority (usually all, or all but one or two) of the failures happen on the same device. This looks awfully like something going wrong with the device configuration during the tests; one guess (without much evidence) would be that the screen isn't being kept awake and the tests which fail are the ones which run during a period where the screen has turned off and which actually care about that. The times at which they ran are pretty spread out through the entire test duration, however, but the sharding makes it hard to judge whether this is plausible or not. Whether that's right or not, it doesn't appear to be an issue with any particular test given the pattern of failures, so adding @RetryOnFailure isn't likely to resolve it unless we add it to a large number of tests with similar characteristics. Questions: a) Bo, am I remembering correctly that the screen being off affects how we determine dimensions/scale? b) John, do you think it's possible our screen-on forcing isn't working?
,
Apr 27 2017
,
Apr 27 2017
Issue 715492 has been merged into this issue.
,
Apr 27 2017
Issue 715625 has been merged into this issue.
,
Apr 27 2017
> a) Bo, am I remembering correctly that the screen being off affects how we determine dimensions/scale? Not directly. But it's hard to say what things stop when app is not in the foreground or if screen is not on, since a lot of systems are throttled in that case, and what effects that will have.. so then say if blink pauses all layout, then not having size updates makes sense?
,
Apr 27 2017
OK, so it's a bit more complex than I was recalling but that's roughly what I was thinking of: if there's something up on the device (something else stealing focus, screen off, other related things) then it seems plausible that these tests that are checking sizes/scales would all fail in a cluster.
,
Apr 27 2017
Seeing some possibly similar failures in other suites over the last day or so. Investigating.
,
Jun 27 2017
Recently failed on: https://build.chromium.org/p/chromium.linux/builders/Android%20Tests/builds/43170 org.chromium.android_webview.test.AwSettingsTest#testUseWideViewportLayoutWidthNoQuirks org.chromium.android_webview.test.AwLegacyQuirksTest#testTargetDensityDpi In contrast to comment #1, the actual value is not 0 this time. junit.framework.AssertionFailedError: Expected: 640, Actual: 598 at org.chromium.android_webview.test.AwSettingsTest.useWideViewportLayoutWidthTest(AwSettingsTest.java:2579) at org.chromium.android_webview.test.AwSettingsTest.testUseWideViewportLayoutWidthNoQuirks(AwSettingsTest.java:2622) at java.lang.reflect.Method.invokeNative(Native Method) at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:214) at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:199) at android.test.ActivityInstrumentationTestCase2.runTest(ActivityInstrumentationTestCase2.java:192) at org.chromium.base.test.BaseTestResult.runParameterized(BaseTestResult.java:161) at org.chromium.base.test.BaseTestResult.run(BaseTestResult.java:124) at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:191) at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:176) at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:554) at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1701)
,
Jun 28 2017
I'm not actively investigating this but somebody probably should - I'm about to go on vacation though :/
,
Nov 8 2017
Closing as obsolete.
,
Dec 28 2017
Issue 797009 has been merged into this issue.
,
Dec 28 2017
The test is still flaky
,
Apr 9 2018
This might be a hard one. ctzsm@, could you take a look?
,
Apr 9 2018
Why was this reopened? The original cause for the failures was because the screen turned off on a tester device (comment #8), and WebView tests expect the screen to be on. I don't think the original issue was a WebView problem, it was an infra problem. If this is because of issue 797009 , then perhaps there's something specific to AwSettingsTest#testUseWideViewportLayoutWidthNoQuirks, or maybe we're just seeing more screen-off issues.
,
Apr 9 2018
From comment #18, it looks like this is reopened because of issue 797009 . We probably should try to enable AwSettingsTest#testUseWideViewportLayoutWidthNoQuirks to see what happens now.
,
Apr 14 2018
This crbug was for tracking the infra failure. We should keep it closed as the root cause is different from issue 797009 . |
|||||||||
►
Sign in to add a comment |
|||||||||
Comment 1 by vitaliii@chromium.org
, Apr 26 2017