Yoland, if you're around maybe you can fix this, otherwise I'll rollback https://chromiumcodereview.appspot.com/2818593002/

That CL hasn't landed?

Oops, Looks like https://chromiumcodereview.appspot.com/2818593002/ has not yet landed -- maybe we just need to land that, so I will LGTM and check the commit box to see if that works.

I don't think that's the right fix, either.

Trying to land https://chromiumcodereview.appspot.com/2818593002/ now.

Thanks John.

Double-oops: looks like there may just be a screen problem with this recent set of multiple failures because I see LOTS of these messages in the log(https://build.chromium.org/p/chromium.android/builders/Lollipop%20Phone%20Tester/builds/11272) :

    java.lang.AssertionError: Many tests will fail if the screen is not on.

e.g.

I  127.987s run_tests_on_device(0282040600515a3b)    java.lang.AssertionError: Many tests will fail if the screen is not on.
I  127.987s run_tests_on_device(0282040600515a3b)    	at org.junit.Assert.fail(Assert.java:88)
I  127.987s run_tests_on_device(0282040600515a3b)    	at org.junit.Assert.assertTrue(Assert.java:41)
I  127.987s run_tests_on_device(0282040600515a3b)    	at org.chromium.content_shell_apk.ContentShellTestCommon.assertScreenIsOn(ContentShellTestCommon.java:65)
I  127.988s run_tests_on_device(0282040600515a3b)    	at org.chromium.content_shell_apk.ContentShellTestBase.setUp(ContentShellTestBase.java:47)
I  127.988s run_tests_on_device(0282040600515a3b)    	at org.chromium.content.browser.VideoFullscreenOrientationLockTest.setUp(VideoFullscreenOrientationLockTest.java:116)
I  127.988s run_tests_on_device(0282040600515a3b)    	at org.chromium.base.test.BaseTestResult.runParameterized(BaseTestResult.java:161)
I  127.988s run_tests_on_device(0282040600515a3b)    	at org.chromium.base.test.BaseTestResult.run(BaseTestResult.java:124)
I  127.988s run_tests_on_device(0282040600515a3b)    	at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:191)
I  127.988s run_tests_on_device(0282040600515a3b)    	at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:176)
I  127.988s run_tests_on_device(0282040600515a3b)    	at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:555)
I  127.988s run_tests_on_device(0282040600515a3b)    	at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1853)

https://chromiumcodereview.appspot.com/2818593002/ is not going to help on these tests, 2818593002 only limit the tests to run on phone, not tablets.

I took a look at the content test failure. Based on the 2 logs with failures, one specific device (0282040600515a3b) would have screen not on issue and fail all the test that throws assertion error if the screen is off

I m not familiar with how provision device is done on the swarming bots, but it should set the screen to be alway-on at the beginning of each build on the bot.

Maybe it's just a device issue? or maybe before content test starts, some other test cause 0282040600515a3b to be turned off?

+jbudorick, any idea on how this might be happening?

screen-off on a swarming bot is probably due to drift between swarming's provisioning logic and provision_devices.py. bpastene has a fix (https://chrome-internal-review.googlesource.com/c/353609/), though beware that it'll require more than just landing the CL.

Issue 711233 has been merged into this issue.

I'm thinking the "screen is off" message was a red herring. The bot started rolling green again early this morning:
https://build.chromium.org/p/chromium.android/builders/Lollipop%20Phone%20Tester

As far as I can tell, nothing changed wrt to how the devices are provisioned, either before or after it started failing.