Flakiness in shipping tests |
||||||
Issue descriptionThis test has been failing consistently since build 3578 https://uberchromegw.corp.google.com/i/chromium.android/builders/Lollipop%20Low-end%20Tester/builds/3578 I 831.878s run_tests_on_device(AG86044202E68GC) detected failure in org.chromium.chrome.browser.payments.PaymentRequestDynamicShippingSingleAddressTest#testAddAddressAndPay. raw output: I 831.878s run_tests_on_device(AG86044202E68GC) INSTRUMENTATION_STATUS: numtests=1 I 831.878s run_tests_on_device(AG86044202E68GC) INSTRUMENTATION_STATUS: stream= I 831.878s run_tests_on_device(AG86044202E68GC) org.chromium.chrome.browser.payments.PaymentRequestDynamicShippingSingleAddressTest: I 831.878s run_tests_on_device(AG86044202E68GC) INSTRUMENTATION_STATUS: id=InstrumentationTestRunner I 831.878s run_tests_on_device(AG86044202E68GC) INSTRUMENTATION_STATUS: test=testAddAddressAndPay I 831.878s run_tests_on_device(AG86044202E68GC) INSTRUMENTATION_STATUS: class=org.chromium.chrome.browser.payments.PaymentRequestDynamicShippingSingleAddressTest I 831.878s run_tests_on_device(AG86044202E68GC) INSTRUMENTATION_STATUS: current=1 I 831.878s run_tests_on_device(AG86044202E68GC) INSTRUMENTATION_STATUS_CODE: 1 I 831.878s run_tests_on_device(AG86044202E68GC) INSTRUMENTATION_STATUS: numtests=1 I 831.879s run_tests_on_device(AG86044202E68GC) INSTRUMENTATION_STATUS: stream= I 831.879s run_tests_on_device(AG86044202E68GC) Error in testAddAddressAndPay: I 831.879s run_tests_on_device(AG86044202E68GC) java.util.concurrent.TimeoutException: waitForCallback timed out! I 831.879s run_tests_on_device(AG86044202E68GC) at org.chromium.content.browser.test.util.CallbackHelper.waitForCallback(CallbackHelper.java:184) I 831.879s run_tests_on_device(AG86044202E68GC) at org.chromium.content.browser.test.util.CallbackHelper.waitForCallback(CallbackHelper.java:192) I 831.879s run_tests_on_device(AG86044202E68GC) at org.chromium.content.browser.test.util.CallbackHelper.waitForCallback(CallbackHelper.java:198) I 831.879s run_tests_on_device(AG86044202E68GC) at org.chromium.chrome.browser.payments.PaymentRequestTestBase.clickInEditorAndWait(PaymentRequestTestBase.java:182) I 831.879s run_tests_on_device(AG86044202E68GC) at org.chromium.chrome.browser.payments.PaymentRequestDynamicShippingSingleAddressTest.testAddAddressAndPay(PaymentRequestDynamicShippingSingleAddressTest.java:86) I 831.879s run_tests_on_device(AG86044202E68GC) at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:214) I 831.879s run_tests_on_device(AG86044202E68GC) at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:199) I 831.879s run_tests_on_device(AG86044202E68GC) at android.test.ActivityInstrumentationTestCase2.runTest(ActivityInstrumentationTestCase2.java:192) I 831.879s run_tests_on_device(AG86044202E68GC) at org.chromium.chrome.test.ChromeActivityTestCaseBase.runTest(ChromeActivityTestCaseBase.java:765) I 831.879s run_tests_on_device(AG86044202E68GC) at org.chromium.base.test.BaseTestResult.runParameterized(BaseTestResult.java:161) I 831.879s run_tests_on_device(AG86044202E68GC) at org.chromium.base.test.BaseTestResult.run(BaseTestResult.java:124) I 831.879s run_tests_on_device(AG86044202E68GC) at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:191) I 831.879s run_tests_on_device(AG86044202E68GC) at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:176) I 831.880s run_tests_on_device(AG86044202E68GC) at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:555) I 831.880s run_tests_on_device(AG86044202E68GC) at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1853) rouslan@ it seems this is your area, could this be caused by your change https://codereview.chromium.org/2093363002 ?
,
Jul 8 2016
,
Jul 11 2016
The tests are still failing. Other bots started failing in similar ways a day later on the other Lollipop tester bots. chrome_public_test_apk failing on https://uberchromegw.corp.google.com/i/chromium.android/builders/Lollipop%20Phone%20Tester?numbuilds=50 and... https://uberchromegw.corp.google.com/i/chromium.android/builders/Lollipop%20Tablet%20Tester
,
Jul 22 2016
,
Jul 22 2016
Looks like waiting for afterTextChanged() [1] is timing out after calling setText() from tests [2]. [1]https://cs.chromium.org/chromium/src/chrome/android/java/src/org/chromium/chrome/browser/payments/ui/EditorTextField.java?rcl=0&l=71 [2]https://cs.chromium.org/chromium/src/chrome/android/javatests/src/org/chromium/chrome/browser/payments/PaymentRequestTestBase.java?rcl=0&l=362
,
Jul 22 2016
Going to install L on my Nexus 4. Is that low-end enough?
,
Jul 22 2016
No, you need a Svelte device setup. You can do it on a Nexus 4, but I don't remember what ROM option that shows up as.
,
Jul 22 2016
gsennton@: The test was added in https://codereview.chromium.org/2093363002. I will try to reproduce the flakiness locally to fix it.
,
Jul 22 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/d577494f52c4d5a5347b5eee564769de34dffeda commit d577494f52c4d5a5347b5eee564769de34dffeda Author: tedchoc <tedchoc@chromium.org> Date: Fri Jul 22 17:49:24 2016 Disable PaymentRequestDynamicShippingSingleAddressTest#testAddAddressAndPay on svelte Has never passed since it was introduced. Disabling on low end for now. BUG=626289 Review-Url: https://codereview.chromium.org/2170183004 Cr-Commit-Position: refs/heads/master@{#407196} [modify] https://crrev.com/d577494f52c4d5a5347b5eee564769de34dffeda/chrome/android/javatests/src/org/chromium/chrome/browser/payments/PaymentRequestDynamicShippingSingleAddressTest.java
,
Aug 8 2016
,
Aug 8 2016
,
Aug 8 2016
Issue 623244 has been merged into this issue.
,
Aug 11 2016
Issue 636631 has been merged into this issue.
,
Aug 11 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/6894c53caec3edba7893a04061adcfcc6bc7edd8 commit 6894c53caec3edba7893a04061adcfcc6bc7edd8 Author: twellington <twellington@chromium.org> Date: Thu Aug 11 02:24:54 2016 Mark two PaymentRequestNoShipping tests as flaky PaymentRequestNoShippingTest#testAddInvalidCardNumberAndCancel and #testAddEmptyNameOnCardAndCancel have been flaking frequently on a few bots. BUG=626289 TBR=rouslan@chromium.org, dfalcantara@chromium.org Review-Url: https://codereview.chromium.org/2234213002 Cr-Commit-Position: refs/heads/master@{#411228} [modify] https://crrev.com/6894c53caec3edba7893a04061adcfcc6bc7edd8/chrome/android/javatests/src/org/chromium/chrome/browser/payments/PaymentRequestNoShippingTest.java
,
Aug 11 2016
#testAddressAndPay is also flaky on Jelly Bean (it was disabled for svelte in #9). The stack trace is the same as in the original bug report. This test had flaked in 5/20 runs. https://test-results.appspot.com/dashboards/flakiness_dashboard.html#tests=org.chromium.chrome.browser.payments.PaymentRequestDynamicShippingSingleAddressTest%23testAddAddressAndPay&testType=chrome_public_test_apk rouslan@ - should I restrict it to KK+? Mark as flaky?
,
Aug 11 2016
Mark as flaky is a heavier hammer, as I understand it, so let's use that. I feel bad for the android sheriffs.
,
Aug 11 2016
Okay, marking as flaky. Thanks!
,
Aug 11 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/7ac59fab1b5ea2b43d742c2a0d980f81e92dabe2 commit 7ac59fab1b5ea2b43d742c2a0d980f81e92dabe2 Author: twellington <twellington@chromium.org> Date: Thu Aug 11 19:15:00 2016 Mark PaymentRequestDynamicShippingSingleAddressTest#testAddAddressAndPay flaky BUG=626289 TBR=dfalcantara@chromium.org Review-Url: https://codereview.chromium.org/2243573003 Cr-Commit-Position: refs/heads/master@{#411394} [modify] https://crrev.com/7ac59fab1b5ea2b43d742c2a0d980f81e92dabe2/chrome/android/javatests/src/org/chromium/chrome/browser/payments/PaymentRequestDynamicShippingSingleAddressTest.java
,
Aug 17 2016
Issue 638537 has been merged into this issue.
,
Sep 22 2016
Issue 649073 has been merged into this issue.
,
Jun 22 2017
,
Jun 27 2017
|
||||||
►
Sign in to add a comment |
||||||
Comment 1 by gsennton@chromium.org
, Jul 7 2016Status: Assigned (was: Untriaged)