New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 848089 link

Starred by 3 users

Issue metadata

Status: Fixed
Owner:
Closed: Jun 2018
Cc:
Components:
EstimatedDays: ----
NextAction: 2018-06-05
OS: Android
Pri: 1
Type: Bug
Q2

Blocking:
issue 835450
issue 846619



Sign in to add a comment

org.chromium.chrome.browser.contextual_suggestions.ContextualSuggestionsTest#testMultiInstanceMode failing on KitKat on CQ and waterfall

Project Member Reported by kbr@chromium.org, May 30 2018

Issue description

This test is failing frequently on the android-kitkat-arm-rel trybot, blocking the CQ:

https://ci.chromium.org/p/chromium/builders/luci.chromium.try/android-kitkat-arm-rel?limit=200

for example:

https://ci.chromium.org/p/chromium/builders/luci.chromium.try/android-kitkat-arm-rel/1086

See the flakiness dashboard:

https://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=chrome_public_test_apk&tests=org.chromium.chrome.browser.contextual_suggestions.ContextualSuggestionsTest#testMultiInstanceMode

and this waterfall bot, where it's failing frequently:

https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/KitKat%20Phone%20Tester%20%28dbg%29

It looks like this might have been suppressed by the sheriffs but I can't tell for sure from Sheriff-O-Matic. Can someone please triage this? It's blocking the CQ. Thanks.

 
It's a bit hard to tell from the flakiness dashboard whether this just started failing.... based on KitKat Phone Tester, I'm guessing this was a recent regression.
https://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=chrome_public_test_apk&tests=ContextualSuggestionsTest%23testMultiInstanceMode

If the sheriff's can look for a culprit, that'd be fantastic. Otherwise we can disable this test on KitKat for now (please restrict to this OS version). Multi-instance mode is only available on N+ anyway, so we don't lose much.

I'm on the shuttle home, so can't address till the morning, but happy to investigate more then.

Failure trace:
earching for Chrome symbols from within: /b/swarming/w/ir/out/Release/lib.unstripped:/b/swarming/w/ir/out/Release
java.lang.AssertionError: Second model has incorrect number of items. expected:<6> but was:<3>
	at org.junit.Assert.fail(Assert.java:88)
	at org.junit.Assert.failNotEquals(Assert.java:834)
	at org.junit.Assert.assertEquals(Assert.java:645)
	at org.chromium.chrome.browser.contextual_suggestions.ContextualSuggestionsTest.testMultiInstanceMode(ContextualSuggestionsTest.java:497)
	at java.lang.reflect.Method.invokeNative(Method.java)
	at java.lang.reflect.Method.invoke(Method.java:515)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:52)
	at org.chromium.base.test.params.ParameterizedFrameworkMethod.invokeExplosively(ParameterizedFrameworkMethod.java:47)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.chromium.base.test.ScreenshotOnFailureStatement.evaluate(ScreenshotOnFailureStatement.java:37)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.chromium.chrome.test.ChromeActivityTestRule$1.evaluate(ChromeActivityTestRule.java:115)
	at android.support.test.internal.statement.UiThreadStatement.evaluate(UiThreadStatement.java:55)
	at android.support.test.rule.ActivityTestRule$ActivityStatement.evaluate(ActivityTestRule.java:270)
	at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
	at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
	at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
	at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
	at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
	at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.chromium.base.test.BaseJUnit4ClassRunner.runChild(BaseJUnit4ClassRunner.java:238)
	at org.chromium.base.test.BaseJUnit4ClassRunner.runChild(BaseJUnit4ClassRunner.java:49)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.chromium.base.test.BaseJUnit4ClassRunner.run(BaseJUnit4ClassRunner.java:221)
	at org.junit.runners.Suite.runChild(Suite.java:128)
	at org.junit.runners.Suite.runChild(Suite.java:27)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.junit.runners.Suite.runChild(Suite.java:128)
	at org.junit.runners.Suite.runChild(Suite.java:27)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
	at android.support.test.internal.runner.TestExecutor.execute(TestExecutor.java:59)
	at android.support.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:262)
	at org.chromium.base.test.BaseChromiumAndroidJUnitRunner.onStart(BaseChromiumAndroidJUnitRunner.java:122)
	at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1701)
Cc: bauerb@chromium.org
Components: -UI>Browser>ContentSuggestions UI>Browser>ContentSuggestions>Explore
Owner: bauerb@chromium.org
Status: Assigned (was: Untriaged)
bauerb@ - can you see whether this is related to your change to migrate to use Use ListObserver instead of NodeParent to notify about changes?

It's listed on the first KitKat Phone Tester flake identified on the flakiness dashboard:
https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/KitKat%20Phone%20Tester%20%28dbg%29/7348

Based on results details, the test flaked twice on that run.

Comment 3 by bauerb@chromium.org, May 31 2018

Status: Started (was: Assigned)
Ugh, and of course I can't reproduce this locally without a KitKat device :-/
In the part of the test that's failing, we wait for a itemRangeInsertedCallback then assert the model has the correct number of items:

        itemRangeInsertedCallback.waitForCallback(0);
        assertEquals("Second model has incorrect number of items.",
                (int) FakeContextualSuggestionsSource.TOTAL_ITEM_COUNT,
                mModel2.getClusterList().getItemCount());


I suspect we're getting multiple itemRangeInsertedCallback calls and that to fix this we could just wait for more callbacks or change the callback to wait until there are 6 total items.

Comment 5 by bau...@google.com, May 31 2018

Oh right, and in addition we access the cluster list on the instrumentation thread, so if we finish waiting after the first callback, we have a race condition between reading the number of items on the instrumentation thread and inserting additional items on the UI thread. Thanks for the investigation :)
Do you have time to look into this before you're off for the day? If not, I can pick it up or at least disable the test for now to unblock the CQ.

Comment 7 by bau...@google.com, May 31 2018

Well, I have a notion of how to fix it, but I'm not going to get around to doing it today. I guess we can disable the test for now?
Owner: twelling...@chromium.org
I have a CL out for review: https://chromium-review.googlesource.com/c/chromium/src/+/1081105
Project Member

Comment 9 by bugdroid1@chromium.org, May 31 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/7c3409cd24a81b560477e558664ea71e6cbcbdff

commit 7c3409cd24a81b560477e558664ea71e6cbcbdff
Author: Theresa <twellington@chromium.org>
Date: Thu May 31 19:36:44 2018

Fix ContextualSuggestionsTest#testMultiInstanceMode

onItemRangeInserted() can be called multiple times. Wait until the
expected number of items have been inserted before notifying the
callback helper. Also get the total model count on the UI thread to
avoid a race between waiting for the callback on the instrumentation
thread and updating the model state on the UI thread.

BUG= 848089 

Change-Id: Ie4ae5cfeef41176d7ea800842c5e106e3bfa8511
Reviewed-on: https://chromium-review.googlesource.com/1081105
Commit-Queue: Theresa <twellington@chromium.org>
Reviewed-by: Becky Zhou <huayinz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#563351}
[modify] https://crrev.com/7c3409cd24a81b560477e558664ea71e6cbcbdff/chrome/android/javatests/src/org/chromium/chrome/browser/contextual_suggestions/ContextualSuggestionsTest.java

Status: Fixed (was: Started)
Labels: O-EoC-Launch KR-Complete-Implementation Q2
NextAction: 2018-06-05
Cc: lizeb@chromium.org
 Issue 848132  has been merged into this issue.
The NextAction date has arrived: 2018-06-05

Sign in to add a comment