The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/a9eef7ca67094604a2f325eb699cb2ef686206eb

commit a9eef7ca67094604a2f325eb699cb2ef686206eb
Author: mattcary <mattcary@chromium.org>
Date: Tue May 30 08:38:34 2017

Disable flaky PaymentRequestMetricsTest.testShownLoggedOnlyOnce.

This is also break anroid M 64 bit build bot.

BUG=727558
TBR=wuandy@chromium.org,gogerald@chromium.org

Review-Url: https://codereview.chromium.org/2913763002
Cr-Commit-Position: refs/heads/master@{#475467}

[modify] https://crrev.com/a9eef7ca67094604a2f325eb699cb2ef686206eb/chrome/android/javatests/src/org/chromium/chrome/browser/payments/PaymentRequestMetricsTest.java

Don't see the obvious relationship to your CL, but let's increase the priority for bug,

cc'ed sebsg@ for ideas,

org.chromium.chrome.browser.payments.PaymentRequestMetricsTest#testSelectedPaymentMethod_CreditCard also just failed on M 64 bit:

https://uberchromegw.corp.google.com/i/chromium.android/builders/Marshmallow%2064%20bit%20Tester/builds/12829

It appears to be occasionally flaky, not just on this bot:

https://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=chrome_public_test_apk&showAllRuns=true&tests=org.chromium.chrome.browser.payments.PaymentRequestMetricsTest%23testSelectedPaymentMethod_CreditCard

The problem is that these tests are timing out before the callback is called sometimes.. e.g.

java.util.concurrent.TimeoutException: waitForCallback timed out!
	at org.chromium.base.test.util.CallbackHelper.waitForCallback(CallbackHelper.java:191)
	at org.chromium.base.test.util.CallbackHelper.waitForCallback(CallbackHelper.java:227)
	at org.chromium.chrome.browser.payments.PaymentRequestTestCommon.clickNodeAndWait(PaymentRequestTestCommon.java:197)
	at org.chromium.chrome.browser.payments.PaymentRequestTestCommon.openPageAndClickNodeAndWait(PaymentRequestTestCommon.java:166)
	at org.chromium.chrome.browser.payments.PaymentRequestTestCommon.triggerUIAndWait(PaymentRequestTestCommon.java:182)
	at org.chromium.chrome.browser.payments.PaymentRequestTestRule.triggerUIAndWait(PaymentRequestTestRule.java:138)
	at org.chromium.chrome.browser.payments.PaymentRequestMetricsTest.testSelectedPaymentMethod_CreditCard(PaymentRequestMetricsTest.java:287)

I will add a @RetryOnFailure for now.

..Perhaps a better 'fix' would be to increase the timeout though.. or refactor the tests so they don't rely on something happening within a timeout.

CL adding the retries is in flight at https://codereview.chromium.org/2927793002 

Promote the priority since this is causing multiple tests flaky,

Just noticed there is a potentially interesting error message in the post-test screenshot for the failure in #3 - "NotSupportedError - The payment method is not supported" - is this expected? A similar error message also showed up when org.chromium.chrome.browser.payments.PaymentRequestAbortTest#testAbortBeforePayClicked failed here:

https://uberchromegw.corp.google.com/i/chromium.android/builders/Marshmallow%2064%20bit%20Tester/builds/12830 - but with the error message "Cannot read property 'abort' of undefined" 

However there are no screenshots for when these flaky tests pass so I don't know if these messages are correlated to the test failures.

PaymentRequestAbortTest also flaked recently, I see now that all the PaymentRequest tests wait for timeouts so are liable to similar flakiness.

Found https://bugs.chromium.org/p/chromium/issues/detail?id=709491 and https://bugs.chromium.org/p/chromium/issues/detail?id=710911 for similar issues, maybe the fix in 710911 didn't actually fix things?

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/e58ee8dcf36c801caf96cf04f9f5eb6d618b2df8

commit e58ee8dcf36c801caf96cf04f9f5eb6d618b2df8
Author: awdf <awdf@chromium.org>
Date: Wed Jun 07 16:08:58 2017

Retry flaky PaymentRequestMetricTests on failure

- Various of these tests were occasionally timing out on some builders,
in particular in the browser_side_navigation_chrome_public_test_apk step
on the Marshmallow 64 bit Tester.

- Added @RetryOnFailure to all tests that involve waiting for a timeout.

- Leaving the one disabled test disabled because that one seems to
take the longest in the flakiness dashboard (usually >4s) so more prone
to flaking (testShownLoggedOnlyOnce), even with a retry.

BUG=727558
TBR=sebsg@chromium.org

Review-Url: https://codereview.chromium.org/2927793002
Cr-Commit-Position: refs/heads/master@{#477668}

[modify] https://crrev.com/e58ee8dcf36c801caf96cf04f9f5eb6d618b2df8/chrome/android/javatests/src/org/chromium/chrome/browser/payments/PaymentRequestMetricsTest.java

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/10b7258ae51826ad1961ce90429f435b9c1a0ada

commit 10b7258ae51826ad1961ce90429f435b9c1a0ada
Author: awdf <awdf@chromium.org>
Date: Wed Jun 07 16:55:26 2017

Retry flaky PaymentRequestAbortTests on failure

BUG=727558
TBR=gogerald@chromium.org

Review-Url: https://codereview.chromium.org/2924053002
Cr-Commit-Position: refs/heads/master@{#477683}

[modify] https://crrev.com/10b7258ae51826ad1961ce90429f435b9c1a0ada/chrome/android/javatests/src/org/chromium/chrome/browser/payments/PaymentRequestAbortTest.java