The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/edf4c2a2a9a387ac62c66391e63e93c70658cbc3

commit edf4c2a2a9a387ac62c66391e63e93c70658cbc3
Author: mvanouwerkerk <mvanouwerkerk@chromium.org>
Date: Tue May 24 13:19:34 2016

Disable testUrlDisplay on M and higher.

BUG= 614338 
TBR=dfalcantara

Review-Url: https://codereview.chromium.org/2007553004
Cr-Commit-Position: refs/heads/master@{#395585}

[modify] https://crrev.com/edf4c2a2a9a387ac62c66391e63e93c70658cbc3/chrome/android/javatests/src/org/chromium/chrome/browser/webapps/WebappUrlBarTest.java

Do your M testers have the same hardware as your KitKat and Lollipop testers?
We've seen the exact same failure on various of our KitKat and Lollipop devices as well - mostly on Debug builds and almost exclusively on less expensive devices.

Could the fact that this only failed on M be coincidence based on the performance of those devices?

I'm relatively convinced that this is simply a timeout which isn't long enough that's causing the problem.
With one of our affected devices and num_retries=0, I can get this to fail 9 out of 10 test runs.

Increasing the timeout in WebappActivityTestBase.waitUntilIdle leads to 0 out of 10 test runs failing.
I'll experiment some more to see if there's a specific reason on why startup is that slow, but unless I find another root cause I'd propose simply increasing the timeout and re-enabling those tests on M
--> I'll update (and most likely post a CL) as soon as I have additional information.

I looked into it some more.
When I look at the device without an increased timeout, it's clearly visible that it is still loading when it hits the timeout most of the time.
I couldn't find anything suspicious that took overly long - it's just going over the three seconds.

I'd therefore recommend just increasing the timeout.
Proposed fix: https://codereview.chromium.org/2108913002

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/6db590fb7126cbf0c9a606b2da0aaf1c9968c687

commit 6db590fb7126cbf0c9a606b2da0aaf1c9968c687
Author: kraush <kraush@amazon.com>
Date: Wed Jun 29 02:30:35 2016

Increase timeout in WebappActivityTestBase.waitUntilIdle

This change increases the timeout in
WebappActivityTestBase.waitUntilIdle and re-enables one test
that was failing on M testers due to a too short timeout.

BUG= 614338 

Review-Url: https://codereview.chromium.org/2108913002
Cr-Commit-Position: refs/heads/master@{#402678}

[modify] https://crrev.com/6db590fb7126cbf0c9a606b2da0aaf1c9968c687/chrome/android/javatests/src/org/chromium/chrome/browser/webapps/WebappActivityTestBase.java
[modify] https://crrev.com/6db590fb7126cbf0c9a606b2da0aaf1c9968c687/chrome/android/javatests/src/org/chromium/chrome/browser/webapps/WebappUrlBarTest.java

Hopefully fixed by kraush's patch.  Thanks!  Can reopen this if it starts flaking again.