New issue
Advanced search Search tips

Issue 807807 link

Starred by 1 user

Issue metadata

Status: Assigned
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 2
Type: Bug



Sign in to add a comment

BookmarkTest#testAddBookmark is very flaky on KitKat Phone Tester

Project Member Reported by mariakho...@chromium.org, Feb 1 2018

Issue description

Flakiness dashboard link: https://test-results.appspot.com/dashboards/flakiness_dashboard.html#tests=org.chromium.chrome.browser.bookmarks.BookmarkTest%23testAddBookmark&testType=chrome_public_test_apk

You can see 1 in 3 runs seem to result in failure

Failure output:

java.lang.RuntimeException: Exception occurred while waiting for runnable
	at org.chromium.base.ThreadUtils.runOnUiThreadBlocking(ThreadUtils.java:81)
	at org.chromium.chrome.browser.bookmarks.BookmarkTest.testAddBookmark(BookmarkTest.java:163)
	at java.lang.reflect.Method.invokeNative(Native Method)
	at java.lang.reflect.Method.invoke(Method.java:515)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:52)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.chromium.base.test.ScreenshotOnFailureStatement.evaluate(ScreenshotOnFailureStatement.java:37)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.chromium.chrome.test.ChromeActivityTestRule$1.evaluate(ChromeActivityTestRule.java:113)
	at android.support.test.internal.statement.UiThreadStatement.evaluate(UiThreadStatement.java:55)
	at android.support.test.rule.ActivityTestRule$ActivityStatement.evaluate(ActivityTestRule.java:270)
	at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.chromium.base.test.BaseJUnit4ClassRunner.runChild(BaseJUnit4ClassRunner.java:196)
	at org.chromium.base.test.BaseJUnit4ClassRunner.runChild(BaseJUnit4ClassRunner.java:42)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.chromium.base.test.BaseJUnit4ClassRunner.run(BaseJUnit4ClassRunner.java:178)
	at org.junit.runners.Suite.runChild(Suite.java:128)
	at org.junit.runners.Suite.runChild(Suite.java:27)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
	at android.support.test.internal.runner.TestExecutor.execute(TestExecutor.java:59)
	at android.support.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:262)
	at org.chromium.base.test.BaseChromiumAndroidJUnitRunner.onStart(BaseChromiumAndroidJUnitRunner.java:99)
	at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1701)
Caused by: java.util.concurrent.ExecutionException: java.lang.AssertionError: The test page is not added as bookmark:
	at java.util.concurrent.FutureTask.report(FutureTask.java:93)
	at java.util.concurrent.FutureTask.get(FutureTask.java:163)
	at org.chromium.base.ThreadUtils.runOnUiThreadBlocking(ThreadUtils.java:79)
	... 40 more
Caused by: java.lang.AssertionError: The test page is not added as bookmark:
	at org.junit.Assert.fail(Assert.java:88)
	at org.junit.Assert.assertTrue(Assert.java:41)
	at org.chromium.chrome.browser.bookmarks.BookmarkTest$3.run(BookmarkTest.java:169)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:422)
	at java.util.concurrent.FutureTask.run(FutureTask.java:237)
	at android.os.Handler.handleCallback(Handler.java:733)
	at android.os.Handler.dispatchMessage(Handler.java:95)
	at android.os.Looper.loop(Looper.java:136)
	at android.app.ActivityThread.main(ActivityThread.java:5001)
	at java.lang.reflect.Method.invokeNative(Native Method)
	at java.lang.reflect.Method.invoke(Method.java:515)
	at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:785)
	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:601)
	at dalvik.system.NativeStart.main(Native Method)

 
Labels: android-sheriff
Labels: -android-sheriff sheriff-android
Is this only flaky on one bot? If so, can we please disable on KitKat but leave enabled on other platforms until I get a chance to investigate?
Cc: huayinz@chromium.org
Components: UI>Browser>Bookmarks
Labels: -sheriff-android
Status: Available (was: Untriaged)
Cc: wychen@chromium.org
+wychen@ could this be related to the recent changes around partner bookmark loading?

The test asserts the bookmark model isn't loaded, simulates a click on the the bookmark menu item to bookmark the current page, waits for the bookmark model to load, then checks if a bookmark with the expected ID exists.
Project Member

Comment 8 by bugdroid1@chromium.org, Feb 1 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/4276c9a4ab9a4e0987106aa89c34203e3f052f6c

commit 4276c9a4ab9a4e0987106aa89c34203e3f052f6c
Author: Maria Khomenko <mariakhomenko@chromium.org>
Date: Thu Feb 01 17:27:03 2018

Disable BookmarkTest#addBookmark because it's flaky.

Bug: 807807
Change-Id: Ibaff48b7c5d998da98526c10d5996b5920b199c6
Reviewed-on: https://chromium-review.googlesource.com/896632
Reviewed-by: Theresa <twellington@chromium.org>
Commit-Queue: Theresa <twellington@chromium.org>
Cr-Commit-Position: refs/heads/master@{#533713}
[modify] https://crrev.com/4276c9a4ab9a4e0987106aa89c34203e3f052f6c/chrome/android/javatests/src/org/chromium/chrome/browser/bookmarks/BookmarkTest.java

There haven't been any recent bookmarks changes that are jumping out at me.. but it's possibly this test has been flaky for a while and it just took some time or a sheriff to notice. The flakiness dashboard only goes back a couple of days.

I tried to repro locally and the test passed 75/75 times on an N5 running KTU84P.
Cc: -wychen@chromium.org
Owner: wychen@chromium.org
Status: Assigned (was: Available)
We could use longer timeout in the pollUiThread() call and see if this changes the flakiness. If it's just that the execution time on the KK devices have a longer tail, this might well be a valid solution. The debug build is more flaky than the release build on KK phones.
The two bots, KitKat Phone Tester (rel) and KitKat Phone Tester (dbg) didn't run this test before 2018-01-31, so this flakiness might be due to a recent introduction of these new bots.

Sign in to add a comment