ContextualSearchManagerTest#testQuickActionCaptionAndImage flaky on Android |
||||||||||
Issue descriptionhttps://build.chromium.org/p/chromium.linux/builders/Android%20Tests?numbuilds=100 Contexual Search related tests started failing randomly since around master@{#437965}. I first suspected and reverted master@{#437959} but that did not work. CCing random people I found from git-blame. ***** Instance #1 ***** org.chromium.chrome.browser.contextualsearch.ContextualSearchManagerTest#testQuickActionCaptionAndImage (run #1): junit.framework.AssertionFailedError: Values did not match. Expected: Search, actual: null at org.chromium.content.browser.test.util.CriteriaHelper.pollInstrumentationThread(CriteriaHelper.java:74) at org.chromium.chrome.browser.contextualsearch.ContextualSearchManagerTest.waitForSelectionToBe(ContextualSearchManagerTest.java:214) at org.chromium.chrome.browser.contextualsearch.ContextualSearchFakeServer$FakeTapSearch.simulate(ContextualSearchFakeServer.java:206) at org.chromium.chrome.browser.contextualsearch.ContextualSearchManagerTest.simulateTapSearch(ContextualSearchManagerTest.java:296) at org.chromium.chrome.browser.contextualsearch.ContextualSearchManagerTest.testQuickActionCaptionAndImage(ContextualSearchManagerTest.java:2743) at java.lang.reflect.Method.invokeNative(Native Method) at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:214) at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:199) at android.test.ActivityInstrumentationTestCase2.runTest(ActivityInstrumentationTestCase2.java:192) at org.chromium.chrome.test.ChromeActivityTestCaseBase.runTest(ChromeActivityTestCaseBase.java:726) at org.chromium.base.test.BaseTestResult.runParameterized(BaseTestResult.java:161) at org.chromium.base.test.BaseTestResult.run(BaseTestResult.java:124) at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:191) at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:176) at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:554) at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1701) ***** Instance #2 ***** org.chromium.chrome.browser.contextualsearch.ContextualSearchManagerTest#testPreventHandlingCurrentSelectionModification (run #1): junit.framework.AssertionFailedError: Panel did not enter PEEKED state. Instead, the current state is CLOSED. at org.chromium.content.browser.test.util.CriteriaHelper.pollInstrumentationThread(CriteriaHelper.java:74) at org.chromium.chrome.browser.contextualsearch.ContextualSearchManagerTest.waitForPanelToEnterState(ContextualSearchManagerTest.java:691) at org.chromium.chrome.browser.contextualsearch.ContextualSearchManagerTest.waitForPanelToPeek(ContextualSearchManagerTest.java:657) at org.chromium.chrome.browser.contextualsearch.ContextualSearchManagerTest.longPressNode(ContextualSearchManagerTest.java:197) at org.chromium.chrome.browser.contextualsearch.ContextualSearchManagerTest.testPreventHandlingCurrentSelectionModification(ContextualSearchManagerTest.java:2026) at java.lang.reflect.Method.invokeNative(Native Method) at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:214) at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:199) at android.test.ActivityInstrumentationTestCase2.runTest(ActivityInstrumentationTestCase2.java:192) at org.chromium.chrome.test.ChromeActivityTestCaseBase.runTest(ChromeActivityTestCaseBase.java:726) at org.chromium.base.test.BaseTestResult.runParameterized(BaseTestResult.java:161) at org.chromium.base.test.BaseTestResult.run(BaseTestResult.java:124) at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:191) at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:176) at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:554) at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1701)
,
Dec 13 2016
The patch is going to collide with issue 673684 , which is handling testPreventHandlingCurrentSelectionModification #testQuickActionCaptionControl() should get the @RetryOnFailure annotation instead of @Disabled if it is flaky. Based on the upstream flakiness dashboard, it flakes very, very infrequently: https://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=chrome_public_test_apk&tests=ContextualSearchManagerTest%23testQuickActionCaptionAndImage nednguyen@ -- can you please include links to the suspected first failed build and the CL you tried to revert?
,
Dec 13 2016
,
Dec 13 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/00ec53a017409e5f1b13cc6b5dbaf9ac196f8178 commit 00ec53a017409e5f1b13cc6b5dbaf9ac196f8178 Author: nednguyen <nednguyen@google.com> Date: Tue Dec 13 16:01:03 2016 Disable flaky tests in ContextualSearchManagerTest.java suite BUG= 673681 TBR=donnd@chromium.org,twellington@chromium.org Review-Url: https://codereview.chromium.org/2571783002 Cr-Commit-Position: refs/heads/master@{#438180} [modify] https://crrev.com/00ec53a017409e5f1b13cc6b5dbaf9ac196f8178/chrome/android/javatests/src/org/chromium/chrome/browser/contextualsearch/ContextualSearchManagerTest.java
,
Dec 13 2016
,
Dec 13 2016
,
Dec 13 2016
The test wasn't actually disabled (it was marked @RetryOnFailure), so why is the Tests-Disabled annotation needed?
,
Dec 14 2016
It should not have Tests-Disabled, removed. Also removed sheriff label.
,
Mar 3 2017
All of our contextual search tests are now marked as @RetryOnFailure. Donn -- do we have a master bug this can be marked as blocking?
,
Mar 3 2017
,
Jan 24 2018
,
Jan 24 2018
,
Oct 18
This hasn't reported any flakes recently that I'm aware of. |
||||||||||
►
Sign in to add a comment |
||||||||||
Comment 1 by nedngu...@google.com
, Dec 13 2016