WebApkIntegrationTest#testLaunchAndNavigateOffOrigin is flaky |
||
Issue descriptionhttps://chromium-review.googlesource.com/c/chromium/src/+/1003552 is likely the culprit. Failures: https://ci.chromium.org/buildbot/chromium.android/KitKat%20Phone%20Tester%20%28dbg%29/5480 https://ci.chromium.org/buildbot/chromium.android/KitKat%20Phone%20Tester%20%28dbg%29/5478 https://ci.chromium.org/buildbot/chromium.android/KitKat%20Tablet%20Tester/11833 java.lang.AssertionError: Criteria not met in allotted time. at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.assertTrue(Assert.java:41) at org.chromium.content.browser.test.util.CriteriaHelper.pollInstrumentationThread(CriteriaHelper.java:93) at org.chromium.content.browser.test.util.CriteriaHelper.pollUiThread(CriteriaHelper.java:172) at org.chromium.content.browser.test.util.CriteriaHelper.pollUiThread(CriteriaHelper.java:194) at org.chromium.chrome.browser.webapps.WebApkIntegrationTest.testLaunchAndNavigateOffOrigin(WebApkIntegrationTest.java:138) at java.lang.reflect.Method.invokeNative(Native Method) at java.lang.reflect.Method.invoke(Method.java:515) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:52) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.chromium.base.test.ScreenshotOnFailureStatement.evaluate(ScreenshotOnFailureStatement.java:37) at org.chromium.chrome.test.ChromeActivityTestRule$1.evaluate(ChromeActivityTestRule.java:113) at android.support.test.internal.statement.UiThreadStatement.evaluate(UiThreadStatement.java:55) at android.support.test.rule.ActivityTestRule$ActivityStatement.evaluate(ActivityTestRule.java:270) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55) at org.junit.rules.RunRules.evaluate(RunRules.java:20) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) at org.chromium.base.test.BaseJUnit4ClassRunner.runChild(BaseJUnit4ClassRunner.java:201) at org.chromium.base.test.BaseJUnit4ClassRunner.runChild(BaseJUnit4ClassRunner.java:43) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.chromium.base.test.BaseJUnit4ClassRunner.run(BaseJUnit4ClassRunner.java:183) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at android.support.test.internal.runner.TestExecutor.execute(TestExecutor.java:59) at android.support.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:262) at org.chromium.base.test.BaseChromiumAndroidJUnitRunner.onStart(BaseChromiumAndroidJUnitRunner.java:120) at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1701)
,
Apr 19 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/58ffe2a0623c201fde8be22578f5062cd9a3cb15 commit 58ffe2a0623c201fde8be22578f5062cd9a3cb15 Author: Peter Kotwicz <pkotwicz@chromium.org> Date: Thu Apr 19 00:10:31 2018 [Android Refactor] Simplify WebApkIntegrationTest This CL removes the redirection check in testLaunchAndNavigateOffOrigin() The redirection check is no longer needed because the test does not load the real www.google.com page due to --host-resolver-rules command line flag BUG= 834466 Change-Id: I1c94800d1ffee8b71d5e159bd1c36899a71d2f54 Reviewed-on: https://chromium-review.googlesource.com/1017599 Commit-Queue: Yaron Friedman <yfriedman@chromium.org> Reviewed-by: Yaron Friedman <yfriedman@chromium.org> Cr-Commit-Position: refs/heads/master@{#551888} [modify] https://crrev.com/58ffe2a0623c201fde8be22578f5062cd9a3cb15/chrome/android/javatests/src/org/chromium/chrome/browser/webapps/WebApkIntegrationTest.java
,
Apr 19 2018
Peter's CL didn't seem to help. Failing pretty consistently on K: https://ci.chromium.org/buildbot/chromium.android/KitKat%20Tablet%20Tester/ https://ci.chromium.org/buildbot/chromium.android/KitKat%20Phone%20Tester%20%28dbg%29/ And flaky on other Android versions: https://ci.chromium.org/buildbot/chromium.android/Marshmallow%20Tablet%20Tester/12435 https://ci.chromium.org/buildbot/chromium.android/Lollipop%20Phone%20Tester/20238 Given that the flakiness was introduced by https://chromium-review.googlesource.com/c/chromium/src/+/1003552, should we revert and reland with the flakiness fixed? Otherwise I'll mark it as flaky. Flakiness dashboard: https://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=chrome_public_test_apk&tests=org.chromium.chrome.browser.webapps.WebApkIntegrationTest%23testLaunchAndNavigateOffOrigin
,
Apr 19 2018
Well we were trying to get this on m67 :/ Let's mark as flaky and keep investigating
,
Apr 19 2018
I have a local fix which seems to fix the flakiness.
,
Apr 19 2018
This might end up being a multi patch CL though
,
Apr 19 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/42791f4250fbe790a554990c1d5b84aad413c921 commit 42791f4250fbe790a554990c1d5b84aad413c921 Author: Eric Stevenson <estevenson@chromium.org> Date: Thu Apr 19 18:26:09 2018 Android: Mark testLaunchAndNavigateOffOrigin as flaky. WebApkIntegrationTest#testLaunchAndNavigateOffOrigin started flaking heavily after https://chromium-review.googlesource.com/c/chromium/src/+/1003552. That CL is necessary for M67 though, so mark as flaky while we investigate. Bug: 834466 Change-Id: Idef70484302ba0e6dc9335ab2324531f3ae16c33 Reviewed-on: https://chromium-review.googlesource.com/1019543 Reviewed-by: Peter Kotwicz <pkotwicz@chromium.org> Commit-Queue: Eric Stevenson <estevenson@chromium.org> Cr-Commit-Position: refs/heads/master@{#552090} [modify] https://crrev.com/42791f4250fbe790a554990c1d5b84aad413c921/chrome/android/javatests/src/org/chromium/chrome/browser/webapps/WebApkIntegrationTest.java
,
Apr 19 2018
Issue 834944 has been merged into this issue.
,
Apr 19 2018
,
Apr 20 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/fb0e69759a22fdaf96cea73450df034dcd15bc3a commit fb0e69759a22fdaf96cea73450df034dcd15bc3a Author: Peter Kotwicz <pkotwicz@chromium.org> Date: Fri Apr 20 19:26:30 2018 Fix WebApkIntegrationTest#testLaunchAndNavigateOffOrigin flakiness This CL changes the test to navigate to a non-404 URL. This change make the 404 interstitial no longer show. I suspect that the interstitial was causing the test to be flaky. Navigating to a non-404 URL was accomplished by: - specifying the target port via the --host-resolver-rules command line flag - navigating to non-https URLs (navigating to an https URL causes an interstitial about certificate validation to show) This CL introduces a new test WebAPK - "http_webapk" - which has a http scope BUG= 834466 Change-Id: I9e9dda7079e3bf2fb62f57d2ecf3ccaad5dbd10d Reviewed-on: https://chromium-review.googlesource.com/1020229 Reviewed-by: Yaron Friedman <yfriedman@chromium.org> Commit-Queue: Peter Kotwicz <pkotwicz@chromium.org> Cr-Commit-Position: refs/heads/master@{#552427} [modify] https://crrev.com/fb0e69759a22fdaf96cea73450df034dcd15bc3a/chrome/android/BUILD.gn [modify] https://crrev.com/fb0e69759a22fdaf96cea73450df034dcd15bc3a/chrome/android/javatests/src/org/chromium/chrome/browser/webapps/WebApkIntegrationTest.java [modify] https://crrev.com/fb0e69759a22fdaf96cea73450df034dcd15bc3a/chrome/android/webapk/shell_apk/BUILD.gn [add] https://crrev.com/fb0e69759a22fdaf96cea73450df034dcd15bc3a/chrome/android/webapk/shell_apk/http_manifest_config.json
,
Apr 23 2018
looks like Peter's CL addressed it
,
Apr 23 2018
Thanks for the quick fix Peter!
,
Apr 24 2018
|
||
►
Sign in to add a comment |
||
Comment 1 by yfried...@chromium.org
, Apr 18 2018