Regression ranges:
chromium:
453363 - 453481

failures:
org.chromium.chrome.browser.contextualsearch.ContextualSearchManagerTest#testQuickActionIntent
org.chromium.chrome.browser.contextualsearch.ContextualSearchManagerTest#testLivePrefetchFailoverRequestMadeAfterOpen
org.chromium.chrome.browser.contextualsearch.ContextualSearchManagerTest#testPromoTapCount
org.chromium.chrome.browser.contextualsearch.ContextualSearchManagerTest#testPrefetchFailoverRequestMadeAfterOpen
org.chromium.chrome.browser.contextualsearch.ContextualSearchManagerTest#testChainedSearchContentVisibility
org.chromium.chrome.browser.contextualsearch.ContextualSearchManagerTest#testQuickActionUrl
org.chromium.chrome.browser.contextualsearch.ContextualSearchManagerTest#testExpandBeforeSearchTermResolution
org.chromium.chrome.browser.contextualsearch.ContextualSearchManagerTest#testSearchTermResolutionError
org.chromium.chrome.browser.contextualsearch.ContextualSearchManagerTest#testTapCausesOneLowPriorityRequest

+ OWNERS of /src/chrome/android/javatests/src/org/chromium/chrome/browser/contextualsearch


The regression range is fairly large, but I haven't spotted any obvious culprit yet.

Example failure (from https://uberchromegw.corp.google.com/i/chromium.android/builders/Lollipop%20Tablet%20Tester/builds/6861/steps/chrome_public_test_apk.chrome_public_test_apk/logs/stdio):


I 3783.749s run_tests_on_device(09078485)  detected failure in org.chromium.chrome.browser.contextualsearch.ContextualSearchManagerTest#testExpandBeforeSearchTermResolution. raw output:
I 3783.749s run_tests_on_device(09078485)    INSTRUMENTATION_STATUS: numtests=1
I 3783.749s run_tests_on_device(09078485)    INSTRUMENTATION_STATUS: stream=
I 3783.749s run_tests_on_device(09078485)    org.chromium.chrome.browser.contextualsearch.ContextualSearchManagerTest:
I 3783.749s run_tests_on_device(09078485)    INSTRUMENTATION_STATUS: id=InstrumentationTestRunner
I 3783.749s run_tests_on_device(09078485)    INSTRUMENTATION_STATUS: test=testExpandBeforeSearchTermResolution
I 3783.749s run_tests_on_device(09078485)    INSTRUMENTATION_STATUS: class=org.chromium.chrome.browser.contextualsearch.ContextualSearchManagerTest
I 3783.749s run_tests_on_device(09078485)    INSTRUMENTATION_STATUS: current=1
I 3783.749s run_tests_on_device(09078485)    INSTRUMENTATION_STATUS_CODE: 1
I 3783.749s run_tests_on_device(09078485)    INSTRUMENTATION_STATUS: numtests=1
I 3783.749s run_tests_on_device(09078485)    INSTRUMENTATION_STATUS: stream=
I 3783.749s run_tests_on_device(09078485)    Failure in testExpandBeforeSearchTermResolution:
I 3783.749s run_tests_on_device(09078485)    junit.framework.AssertionFailedError: Panel did not enter EXPANDED state. Instead, the current state is PEEKED.
I 3783.749s run_tests_on_device(09078485)    	at org.chromium.content.browser.test.util.CriteriaHelper.pollInstrumentationThread(CriteriaHelper.java:79)
I 3783.749s run_tests_on_device(09078485)    	at org.chromium.chrome.browser.contextualsearch.ContextualSearchManagerTest.waitForPanelToEnterState(ContextualSearchManagerTest.java:763)
I 3783.750s run_tests_on_device(09078485)    	at org.chromium.chrome.browser.contextualsearch.ContextualSearchManagerTest.waitForPanelToExpand(ContextualSearchManagerTest.java:738)
I 3783.750s run_tests_on_device(09078485)    	at org.chromium.chrome.browser.contextualsearch.ContextualSearchManagerTest.tapPeekingBarToExpandAndAssert(ContextualSearchManagerTest.java:992)
I 3783.750s run_tests_on_device(09078485)    	at org.chromium.chrome.browser.contextualsearch.ContextualSearchManagerTest.testExpandBeforeSearchTermResolution(ContextualSearchManagerTest.java:1721)
I 3783.750s run_tests_on_device(09078485)    	at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:214)
I 3783.750s run_tests_on_device(09078485)    	at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:199)
I 3783.750s run_tests_on_device(09078485)    	at android.test.ActivityInstrumentationTestCase2.runTest(ActivityInstrumentationTestCase2.java:192)
I 3783.750s run_tests_on_device(09078485)    	at org.chromium.chrome.test.ChromeActivityTestCaseBase.runTest(ChromeActivityTestCaseBase.java:752)
I 3783.750s run_tests_on_device(09078485)    	at org.chromium.base.test.BaseTestResult.runParameterized(BaseTestResult.java:161)
I 3783.750s run_tests_on_device(09078485)    	at org.chromium.base.test.BaseTestResult.run(BaseTestResult.java:124)
I 3783.750s run_tests_on_device(09078485)    	at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:191)
I 3783.750s run_tests_on_device(09078485)    	at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:176)
I 3783.750s run_tests_on_device(09078485)    	at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:555)
I 3783.750s run_tests_on_device(09078485)    	at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1853)
I 3783.750s run_tests_on_device(09078485)    
I 3783.750s run_tests_on_device(09078485)    INSTRUMENTATION_STATUS: id=InstrumentationTestRunner
I 3783.750s run_tests_on_device(09078485)    INSTRUMENTATION_STATUS: test=testExpandBeforeSearchTermResolution
I 3783.750s run_tests_on_device(09078485)    INSTRUMENTATION_STATUS: class=org.chromium.chrome.browser.contextualsearch.ContextualSearchManagerTest
I 3783.750s run_tests_on_device(09078485)    INSTRUMENTATION_STATUS: stack=junit.framework.AssertionFailedError: Panel did not enter EXPANDED state. Instead, the current state is PEEKED.
I 3783.750s run_tests_on_device(09078485)    	at org.chromium.content.browser.test.util.CriteriaHelper.pollInstrumentationThread(CriteriaHelper.java:79)
I 3783.750s run_tests_on_device(09078485)    	at org.chromium.chrome.browser.contextualsearch.ContextualSearchManagerTest.waitForPanelToEnterState(ContextualSearchManagerTest.java:763)
I 3783.750s run_tests_on_device(09078485)    	at org.chromium.chrome.browser.contextualsearch.ContextualSearchManagerTest.waitForPanelToExpand(ContextualSearchManagerTest.java:738)
I 3783.750s run_tests_on_device(09078485)    	at org.chromium.chrome.browser.contextualsearch.ContextualSearchManagerTest.tapPeekingBarToExpandAndAssert(ContextualSearchManagerTest.java:992)
I 3783.750s run_tests_on_device(09078485)    	at org.chromium.chrome.browser.contextualsearch.ContextualSearchManagerTest.testExpandBeforeSearchTermResolution(ContextualSearchManagerTest.java:1721)
I 3783.750s run_tests_on_device(09078485)    	at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:214)
I 3783.750s run_tests_on_device(09078485)    	at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:199)
I 3783.750s run_tests_on_device(09078485)    	at android.test.ActivityInstrumentationTestCase2.runTest(ActivityInstrumentationTestCase2.java:192)
I 3783.750s run_tests_on_device(09078485)    	at org.chromium.chrome.test.ChromeActivityTestCaseBase.runTest(ChromeActivityTestCaseBase.java:752)
I 3783.751s run_tests_on_device(09078485)    	at org.chromium.base.test.BaseTestResult.runParameterized(BaseTestResult.java:161)
I 3783.751s run_tests_on_device(09078485)    	at org.chromium.base.test.BaseTestResult.run(BaseTestResult.java:124)
I 3783.751s run_tests_on_device(09078485)    	at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:191)
I 3783.751s run_tests_on_device(09078485)    	at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:176)
I 3783.751s run_tests_on_device(09078485)    	at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:555)
I 3783.751s run_tests_on_device(09078485)    	at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1853)
I 3783.751s run_tests_on_device(09078485)    
I 3783.751s run_tests_on_device(09078485)    INSTRUMENTATION_STATUS: current=1
I 3783.751s run_tests_on_device(09078485)    INSTRUMENTATION_STATUS_CODE: -2
I 3783.751s run_tests_on_device(09078485)    INSTRUMENTATION_RESULT: stream=
I 3783.751s run_tests_on_device(09078485)    Test results for ChromeInstrumentationTestRunner=.F
I 3783.751s run_tests_on_device(09078485)    Time: 19.571
I 3783.751s run_tests_on_device(09078485)    
I 3783.751s run_tests_on_device(09078485)    FAILURES!!!
I 3783.751s run_tests_on_device(09078485)    Tests run: 1,  Failures: 1,  Errors: 0

Since these are failing consistently, it should be easy to reproduce and debug.  hartmanng@ - do you have some cycles today as Sheriff to look into this or do you need Donn or I to investigate?

Regression range link from sheriff-o-matic: https://chromium.googlesource.com/chromium/src/+log/70a5e24e2309d63e582757e0606da685fc12bfd1%5E..08c892f77027cc70fd4a26bb218e88fe1e30161e?pretty=fuller

I think that regression range is from the Marshmallow 64-bit bot that's also failing consistently.

The regular Marshmallow Tablet and Lollipop Tablet bots started failing more recently with this regression range:
https://chromium.googlesource.com/chromium/src/+log/104b71bdc9e24311f7a63f67b183b0b176f94bb5..d492a16c9bbbe163d48cb0b2976304787f21c5d1?pretty=fuller&n=10000

The more narrow regression range for the M-tablet and L-tablet bots is a subset of the sheriff-o-matic identified regression range.

Note that some of the quick action tests are failing with different stack traces, but the root cause is likely the same. The bar should be receiving a tap to either expand or activate the quick action and it appears that is not happening.

C 3917.071s Main  [FAIL] org.chromium.chrome.browser.contextualsearch.ContextualSearchManagerTest#testQuickActionUrl:
C 3917.071s Main  junit.framework.AssertionFailedError: Page did not load.  Tab information at time of failure -- url: http://127.0.0.1:42810/chrome/test/data/android/google.html, final URL: http://127.0.0.1:42810/chrome/test/data/android/contextualsearch/tap_test.html, load progress: 100, is loading: false
C 3917.071s Main  	at org.chromium.chrome.test.util.ChromeTabUtils.waitForTabPageLoaded(ChromeTabUtils.java:84)
C 3917.071s Main  	at org.chromium.chrome.browser.contextualsearch.ContextualSearchManagerTest.testQuickActionUrl(ContextualSearchManagerTest.java:2906)
C 3917.071s Main  	at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:214)
C 3917.071s Main  	at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:199)
C 3917.071s Main  	at android.test.ActivityInstrumentationTestCase2.runTest(ActivityInstrumentationTestCase2.java:192)
C 3917.071s Main  	at org.chromium.chrome.test.ChromeActivityTestCaseBase.runTest(ChromeActivityTestCaseBase.java:752)
C 3917.071s Main  	at org.chromium.base.test.BaseTestResult.runParameterized(BaseTestResult.java:161)
C 3917.071s Main  	at org.chromium.base.test.BaseTestResult.run(BaseTestResult.java:124)
C 3917.071s Main  	at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:191)
C 3917.071s Main  	at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:176)
C 3917.071s Main  	at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:555)
C 3917.071s Main  	at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1853)


C 3917.071s Main  [FAIL] org.chromium.chrome.browser.contextualsearch.ContextualSearchManagerTest#testQuickActionIntent:
C 3917.071s Main  junit.framework.AssertionFailedError: expected:<1> but was:<0>
C 3917.071s Main  	at org.chromium.chrome.browser.contextualsearch.ContextualSearchManagerTest.testQuickActionIntent(ContextualSearchManagerTest.java:2881)
C 3917.071s Main  	at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:214)
C 3917.071s Main  	at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:199)
C 3917.071s Main  	at android.test.ActivityInstrumentationTestCase2.runTest(ActivityInstrumentationTestCase2.java:192)
C 3917.071s Main  	at org.chromium.chrome.test.ChromeActivityTestCaseBase.runTest(ChromeActivityTestCaseBase.java:752)
C 3917.071s Main  	at org.chromium.base.test.BaseTestResult.runParameterized(BaseTestResult.java:161)
C 3917.071s Main  	at org.chromium.base.test.BaseTestResult.run(BaseTestResult.java:124)
C 3917.071s Main  	at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:191)
C 3917.071s Main  	at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:176)
C 3917.071s Main  	at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:555)
C 3917.071s Main  	at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1853)

I may have some free cycles later in the day to investigate, but I'm dealing with a few separate failures right now. If you two have some spare cycles to investigate, that might help get it fixed faster - otherwise I'll try to look into it in a few hours.

I can start looking into it.

I can reproduce this on a Nexus 9 in landscape but not portrait. I think it's likely that these tests have always been bad and the devices recently got changed from portrait to landscape.

The issue is that ContextualSearchManagerTest#clickPanelBar() is clicking outside of the activity. #clickPanelBar() is using the decor view, which includes the status bar and Android controls, to calculate the Y position. When the N9 is in landscape mode, the panel Y offset is 640dp and its height is 56dp. The simulated tap is at 729.6dp, which is below the activity inside the decor view.

We should be using the actual position of the contextual search panel to calculate where to click. I have an in-flight CL here: https://codereview.chromium.org/2726643002

Thanks for taking care of this Theresa!

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/af45f47f94de96e49d4e93ad2dfbe806b9a3e3b0

commit af45f47f94de96e49d4e93ad2dfbe806b9a3e3b0
Author: twellington <twellington@chromium.org>
Date: Tue Feb 28 22:16:58 2017

[Contextual Search] Use panel location to determine test touch target

BUG= 697055 

Review-Url: https://codereview.chromium.org/2726643002
Cr-Commit-Position: refs/heads/master@{#453725}

[modify] https://crrev.com/af45f47f94de96e49d4e93ad2dfbe806b9a3e3b0/chrome/android/javatests/src/org/chromium/chrome/browser/contextualsearch/ContextualSearchManagerTest.java