failures:
org.chromium.chrome.browser.vr.WebXrVrInputTest#testScreenTapsNotRegistered_WebXr__WebappActivity
org.chromium.chrome.browser.vr.WebXrVrTransitionTest#testWebVrDisabledWithoutFlagSet__WebappActivity
org.chromium.chrome.browser.vr.WebXrVrInputTest#testScreenTapsNotRegistered__WebappActivity
org.chromium.chrome.browser.vr.WebXrVrTransitionTest#testWindowRafFiresDuringNonImmersiveSession__WebappActivity
org.chromium.chrome.browser.vr.WebXrVrInputTest#testWebXrGamepadNotReturnedWithoutGamepadSupport_Cardboard__WebappActivity
org.chromium.chrome.browser.vr.WebXrVrInputTest#testPresentationLocksFocus_WebXr__WebappActivity
org.chromium.chrome.browser.vr.WebXrVrInputTest#testScreenTapsRegisteredOnCardboard__WebappActivity
org.chromium.chrome.browser.vr.WebXrVrInputTest#testWebXrGamepadNotReturnedWithoutAnyFeatures_Cardboard__WebappActivity
org.chromium.chrome.browser.vr.WebXrVrInputTest#testScreenTapsRegisteredOnCardboard_WebXr__WebappActivity
org.chromium.chrome.browser.vr.WebXrVrInputTest#testPresentationLocksFocus__WebappActivity
org.chromium.chrome.browser.vr.WebXrVrTransitionTest#testNonImmersiveStopsDuringImmersive__WebappActivity
org.chromium.chrome.browser.vr.WebXrVrTransitionTest#testRequestSessionEntersVr__WebappActivity

Example failure link:
https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Android%20CFI/3347

https://logs.chromium.org/logs/chromium/buildbucket/cr-buildbucket.appspot.com/8932221535319382704/+/steps/chrome_public_test_vr_apk_on_Android_device_Nexus_5X/0/logs/org.chromium.chrome.browser.vr.WebXrVrInputTest_testScreenTapsNotRegistered_WebXr__WebappActivity/0

I see these errors occur sporadically but enough to show up in the 'consistent failures' section of the sheriffing tool.

I have a CL blocked for several days on this problem. I've tried local testing with an M device and can't get head to pass; looking at the flakiness dashboard the XrVr tests are failing a lot on K and M.

It seems that most CLs don't trigger these tests, so the CQ doesn't block most people.

I'm going to disable them on pre-N devices. It's a little discouraging that nothing's been done about this bug for 3 months.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/d5ae3d0b3a001b5af0e2e6d9e7404a673e7de8f8

commit d5ae3d0b3a001b5af0e2e6d9e7404a673e7de8f8
Author: Matthew Cary <mattcary@chromium.org>
Date: Thu Jan 17 11:01:18 2019

XrVr: disable WebXrVr tests on pre-N.

Bug: 897259
Change-Id: Iac5a81cf6108fb27a9c3e1f3685278497fbd38c7

TBR: bsheedy@chromium.org
Change-Id: Iac5a81cf6108fb27a9c3e1f3685278497fbd38c7
Reviewed-on: https://chromium-review.googlesource.com/c/1417451
Reviewed-by: Matthew Cary <mattcary@chromium.org>
Commit-Queue: Matthew Cary <mattcary@chromium.org>
Cr-Commit-Position: refs/heads/master@{#623640}
[modify] https://crrev.com/d5ae3d0b3a001b5af0e2e6d9e7404a673e7de8f8/chrome/android/javatests/src/org/chromium/chrome/browser/vr/VrInstallUpdateInfoBarTest.java
[modify] https://crrev.com/d5ae3d0b3a001b5af0e2e6d9e7404a673e7de8f8/chrome/android/javatests/src/org/chromium/chrome/browser/vr/WebXrVrDeviceTest.java
[modify] https://crrev.com/d5ae3d0b3a001b5af0e2e6d9e7404a673e7de8f8/chrome/android/javatests/src/org/chromium/chrome/browser/vr/WebXrVrInputTest.java
[modify] https://crrev.com/d5ae3d0b3a001b5af0e2e6d9e7404a673e7de8f8/chrome/android/javatests/src/org/chromium/chrome/browser/vr/WebXrVrTabTest.java
[modify] https://crrev.com/d5ae3d0b3a001b5af0e2e6d9e7404a673e7de8f8/chrome/android/javatests/src/org/chromium/chrome/browser/vr/WebXrVrTransitionTest.java

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/3bac194f6b515796389eb9e23f1818e4bb4a0715

commit 3bac194f6b515796389eb9e23f1818e4bb4a0715
Author: Matthew Cary <mattcary@chromium.org>
Date: Thu Jan 17 13:12:44 2019

Vr: Disable VrBrowser tests on pre-N.

Bug: 897259
Change-Id: I48194d4c01a3139db0183c3b7e843418510a51b3

TBR: bsheedy@chromium.org
Change-Id: I48194d4c01a3139db0183c3b7e843418510a51b3
Reviewed-on: https://chromium-review.googlesource.com/c/1417532
Reviewed-by: Matthew Cary <mattcary@chromium.org>
Commit-Queue: Matthew Cary <mattcary@chromium.org>
Cr-Commit-Position: refs/heads/master@{#623661}
[modify] https://crrev.com/3bac194f6b515796389eb9e23f1818e4bb4a0715/chrome/android/javatests/src/org/chromium/chrome/browser/vr/VrBrowserCompositorViewHolderTest.java
[modify] https://crrev.com/3bac194f6b515796389eb9e23f1818e4bb4a0715/chrome/android/javatests/src/org/chromium/chrome/browser/vr/VrBrowserTransitionTest.java

Sorry about that; team's been in transition and it looks like this issue got lost. We'll get on it.

Did some initial investigation. While it does look like a bit of flakiness has crept into the VR tests on the CQ (which is definitely something that should be fixed), it's only in a small number of tests and almost always resolved with the automatic retries.

Looking at runs for mattcary@'s patch that was blocked, every test is segfaulting 100% of the time, e.g. https://luci-logdog.appspot.com/logs/chromium/android/swarming/logcats/42743ff814e43811/+/logcat_logcat_org.chromium.chrome.browser.vr.WebXrVrInputTest.testWebXrGamepadReturnedWithGamepadSupport_Cardboard__WebappActivity_20190117T105920-UTC_01e68704a2bdc28f.

I believe this is a legitimate issue that's getting caught in the CQ, and something that should be resolved before submitting the patch. If you're having trouble reproducing, I can work with you to try to get that resolved, or worst case, help you run the test on swarming where it will reproduce.

Talked with mattcary@ offline, and the reason they thought the tests were broken on ToT was that the VR tests require some magic incantations to run properly (see https://chromium.googlesource.com/chromium/src/+/HEAD/chrome/android/javatests/src/org/chromium/chrome/browser/vr/README.md).

I'll re-enable the tests and investigate the small amount of flakiness that's present.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/71a6362fb2aa31fdc058160b9babdf735a221ebc

commit 71a6362fb2aa31fdc058160b9babdf735a221ebc
Author: bsheedy <bsheedy@chromium.org>
Date: Thu Jan 17 19:24:40 2019

Re-enable VR cardboard tests

Re-enables the tests that were disabled in https://crrev.com/c/1417451
and https://crrev.com/1417532. These were though to be failing 100% of
the time on ToT, but turned out to be an issue with the local repro
environment not being set up properly for VR tests.

TBR=billorr@chromium.org

Bug: 897259
Change-Id: Ia8c1bd154231398774134ab701523e230c858681
Reviewed-on: https://chromium-review.googlesource.com/c/1418077
Reviewed-by: Brian Sheedy <bsheedy@chromium.org>
Commit-Queue: Brian Sheedy <bsheedy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#623790}
[modify] https://crrev.com/71a6362fb2aa31fdc058160b9babdf735a221ebc/chrome/android/javatests/src/org/chromium/chrome/browser/vr/VrBrowserCompositorViewHolderTest.java
[modify] https://crrev.com/71a6362fb2aa31fdc058160b9babdf735a221ebc/chrome/android/javatests/src/org/chromium/chrome/browser/vr/VrBrowserTransitionTest.java
[modify] https://crrev.com/71a6362fb2aa31fdc058160b9babdf735a221ebc/chrome/android/javatests/src/org/chromium/chrome/browser/vr/VrInstallUpdateInfoBarTest.java
[modify] https://crrev.com/71a6362fb2aa31fdc058160b9babdf735a221ebc/chrome/android/javatests/src/org/chromium/chrome/browser/vr/WebXrVrDeviceTest.java
[modify] https://crrev.com/71a6362fb2aa31fdc058160b9babdf735a221ebc/chrome/android/javatests/src/org/chromium/chrome/browser/vr/WebXrVrInputTest.java
[modify] https://crrev.com/71a6362fb2aa31fdc058160b9babdf735a221ebc/chrome/android/javatests/src/org/chromium/chrome/browser/vr/WebXrVrTabTest.java
[modify] https://crrev.com/71a6362fb2aa31fdc058160b9babdf735a221ebc/chrome/android/javatests/src/org/chromium/chrome/browser/vr/WebXrVrTransitionTest.java