WebappSplashScreenTest#testUmaWhenSplashHides flaky on Lollipop Low-end Tester |
||||
Issue descriptionWebappSplashScreenTest#testUmaWhenSplashHides is failing about 1/10 runs on Lollipop Low-end Tester Flakiness dashboard: https://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=chrome_public_test_apk&tests=WebappSplashScreenTest%23testUmaWhenSplashHides Example build failure: https://build.chromium.org/p/chromium.android/builders/Lollipop%20Low-end%20Tester/builds/3807 C 1166.113s Main [FAIL] org.chromium.chrome.browser.webapps.WebappSplashScreenTest#testUmaWhenSplashHides: C 1166.113s Main junit.framework.AssertionFailedError C 1166.113s Main at org.chromium.chrome.browser.webapps.WebappSplashScreenTest.testUmaWhenSplashHides(WebappSplashScreenTest.java:207) C 1166.113s Main at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:214) C 1166.113s Main at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:199) C 1166.113s Main at android.test.ActivityInstrumentationTestCase2.runTest(ActivityInstrumentationTestCase2.java:192) C 1166.113s Main at org.chromium.chrome.test.ChromeActivityTestCaseBase.runTest(ChromeActivityTestCaseBase.java:765) C 1166.113s Main at org.chromium.base.test.BaseTestResult.runParameterized(BaseTestResult.java:161) C 1166.113s Main at org.chromium.base.test.BaseTestResult.run(BaseTestResult.java:124) C 1166.114s Main at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:191) C 1166.114s Main at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:176) C 1166.114s Main at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:555) C 1166.114s Main at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1853)
,
Sep 9 2016
,
Sep 12 2016
,
Sep 12 2016
Any updates here?
,
Sep 13 2016
This is still in my queue. I'm not sure how reliably I can follow up though, as I don't have a low-end Lollipop device to test with.
,
Sep 14 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/d19c087b9bfdf76a8af91e2f0a0e791bf2ffcafa commit d19c087b9bfdf76a8af91e2f0a0e791bf2ffcafa Author: dominickn <dominickn@chromium.org> Date: Wed Sep 14 00:27:15 2016 Speculatively deflake some web app tests on low-end Lollipop. This attempts to deflake two tests which flake on the Lollipop low-end tester. WebappSplashScreenIconTest#setUp decodes a string into a bitmap, which is then encoded back into a string. Remove both of these steps so that the string is directly inserted into WebappDataStorage. This speculatively deflakes WebappSplashScreenIconTest#testShowSplashScreen, which has a criteria takes too long timeout. WebappSplashScreenTest#testUmaWhenSplashHides fails an assert for a histogram value representing the number of milliseconds that the splashscreen was shown. On the theory that the slow device is showing the splash screen for longer, the speculative fix is to check more of the histogram buckets. BUG= 636633 Review-Url: https://codereview.chromium.org/2337843003 Cr-Commit-Position: refs/heads/master@{#418438} [modify] https://crrev.com/d19c087b9bfdf76a8af91e2f0a0e791bf2ffcafa/chrome/android/java/src/org/chromium/chrome/browser/webapps/WebappDataStorage.java [modify] https://crrev.com/d19c087b9bfdf76a8af91e2f0a0e791bf2ffcafa/chrome/android/javatests/src/org/chromium/chrome/browser/webapps/WebappSplashScreenIconTest.java [modify] https://crrev.com/d19c087b9bfdf76a8af91e2f0a0e791bf2ffcafa/chrome/android/javatests/src/org/chromium/chrome/browser/webapps/WebappSplashScreenTest.java
,
Sep 14 2016
WebappSplashScreenIconTest#testShowSplashScreen is still flaking unfrotunately, so it looks like my fix didn't quite help for that - https://build.chromium.org/p/chromium.android/builders/Lollipop%20Low-end%20Tester/builds/4006 However, WebappSplashScreenTest#testUmaWhenSplashHides hasn't flaked yet, but there haven't been a lot of cycles since the patch landed.
,
Sep 15 2016
No flakes for WebappSplashScreenTest#testUmaWhenSplashHides since https://crrev.com/d19c087b9bfdf76a8af91e2f0a0e791bf2ffcafa - the longest such period in some time. Plus, the longer 6s, 7s runs of the test are now passing and not flaking. I'm going to declare the deflaking a success and close this. WebappSplashScreenIconTest#testShowSplashScreen can be looked at separately if needed. |
||||
►
Sign in to add a comment |
||||
Comment 1 by dominickn@chromium.org
, Aug 11 2016