BookmarkTest#testAddBookmark is very flaky on KitKat Phone Tester |
||||||
Issue descriptionFlakiness dashboard link: https://test-results.appspot.com/dashboards/flakiness_dashboard.html#tests=org.chromium.chrome.browser.bookmarks.BookmarkTest%23testAddBookmark&testType=chrome_public_test_apk You can see 1 in 3 runs seem to result in failure Failure output: java.lang.RuntimeException: Exception occurred while waiting for runnable at org.chromium.base.ThreadUtils.runOnUiThreadBlocking(ThreadUtils.java:81) at org.chromium.chrome.browser.bookmarks.BookmarkTest.testAddBookmark(BookmarkTest.java:163) at java.lang.reflect.Method.invokeNative(Native Method) at java.lang.reflect.Method.invoke(Method.java:515) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:52) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.chromium.base.test.ScreenshotOnFailureStatement.evaluate(ScreenshotOnFailureStatement.java:37) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.chromium.chrome.test.ChromeActivityTestRule$1.evaluate(ChromeActivityTestRule.java:113) at android.support.test.internal.statement.UiThreadStatement.evaluate(UiThreadStatement.java:55) at android.support.test.rule.ActivityTestRule$ActivityStatement.evaluate(ActivityTestRule.java:270) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55) at org.junit.rules.RunRules.evaluate(RunRules.java:20) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) at org.chromium.base.test.BaseJUnit4ClassRunner.runChild(BaseJUnit4ClassRunner.java:196) at org.chromium.base.test.BaseJUnit4ClassRunner.runChild(BaseJUnit4ClassRunner.java:42) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.chromium.base.test.BaseJUnit4ClassRunner.run(BaseJUnit4ClassRunner.java:178) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at android.support.test.internal.runner.TestExecutor.execute(TestExecutor.java:59) at android.support.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:262) at org.chromium.base.test.BaseChromiumAndroidJUnitRunner.onStart(BaseChromiumAndroidJUnitRunner.java:99) at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1701) Caused by: java.util.concurrent.ExecutionException: java.lang.AssertionError: The test page is not added as bookmark: at java.util.concurrent.FutureTask.report(FutureTask.java:93) at java.util.concurrent.FutureTask.get(FutureTask.java:163) at org.chromium.base.ThreadUtils.runOnUiThreadBlocking(ThreadUtils.java:79) ... 40 more Caused by: java.lang.AssertionError: The test page is not added as bookmark: at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.assertTrue(Assert.java:41) at org.chromium.chrome.browser.bookmarks.BookmarkTest$3.run(BookmarkTest.java:169) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:422) at java.util.concurrent.FutureTask.run(FutureTask.java:237) at android.os.Handler.handleCallback(Handler.java:733) at android.os.Handler.dispatchMessage(Handler.java:95) at android.os.Looper.loop(Looper.java:136) at android.app.ActivityThread.main(ActivityThread.java:5001) at java.lang.reflect.Method.invokeNative(Native Method) at java.lang.reflect.Method.invoke(Method.java:515) at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:785) at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:601) at dalvik.system.NativeStart.main(Native Method)
,
Feb 1 2018
Code review for disable: https://chromium-review.googlesource.com/c/chromium/src/+/896632
,
Feb 1 2018
,
Feb 1 2018
,
Feb 1 2018
Is this only flaky on one bot? If so, can we please disable on KitKat but leave enabled on other platforms until I get a chance to investigate?
,
Feb 1 2018
,
Feb 1 2018
+wychen@ could this be related to the recent changes around partner bookmark loading? The test asserts the bookmark model isn't loaded, simulates a click on the the bookmark menu item to bookmark the current page, waits for the bookmark model to load, then checks if a bookmark with the expected ID exists.
,
Feb 1 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/4276c9a4ab9a4e0987106aa89c34203e3f052f6c commit 4276c9a4ab9a4e0987106aa89c34203e3f052f6c Author: Maria Khomenko <mariakhomenko@chromium.org> Date: Thu Feb 01 17:27:03 2018 Disable BookmarkTest#addBookmark because it's flaky. Bug: 807807 Change-Id: Ibaff48b7c5d998da98526c10d5996b5920b199c6 Reviewed-on: https://chromium-review.googlesource.com/896632 Reviewed-by: Theresa <twellington@chromium.org> Commit-Queue: Theresa <twellington@chromium.org> Cr-Commit-Position: refs/heads/master@{#533713} [modify] https://crrev.com/4276c9a4ab9a4e0987106aa89c34203e3f052f6c/chrome/android/javatests/src/org/chromium/chrome/browser/bookmarks/BookmarkTest.java
,
Feb 1 2018
There haven't been any recent bookmarks changes that are jumping out at me.. but it's possibly this test has been flaky for a while and it just took some time or a sheriff to notice. The flakiness dashboard only goes back a couple of days. I tried to repro locally and the test passed 75/75 times on an N5 running KTU84P.
,
Feb 2 2018
We could use longer timeout in the pollUiThread() call and see if this changes the flakiness. If it's just that the execution time on the KK devices have a longer tail, this might well be a valid solution. The debug build is more flaky than the release build on KK phones.
,
Feb 2 2018
The two bots, KitKat Phone Tester (rel) and KitKat Phone Tester (dbg) didn't run this test before 2018-01-31, so this flakiness might be due to a recent introduction of these new bots. |
||||||
►
Sign in to add a comment |
||||||
Comment 1 by mariakho...@chromium.org
, Feb 1 2018