It looks like the issue is caused by the immersive mode confirmation that pops up when you enter VR - on K, it requires that the user hit "OK" before going away, whereas on newer versions, it's just a toast-like notification.

I think in most cases, we're sending the second screen tap before the confirmation appears, meaning that it gets sent to Chrome, and the test works fine. In rare cases, the confirmation appears on top of Chrome before we send the second tap, and when we try to, it tries to send it to the confirmation and fails since it's technically not Chrome.

There are a few solutions to this:

1. Make Android think that immersive mode has already been confirmed by editing the stored content provider entry (uri content://settings/secure, name immersive_mode_confirmations). This can be done via adb, but that isn't really an option for swarming tests atm, as we don't have direct access to ADB and the UI Automator executeShellCommand which acts like ADB has a minimum SDK version of 21. Plus, editing it this way seems to have a habit of rebooting the device. Editing via an implementation of Android's ContentProvider would likely run in to permission issues.

2. Use UI Automator to click on the "OK" button since it can interact with apps besides Chrome. However, once immersive mode has been confirmed for an app on a device, this won't need to be done again, so we can't just say "wait for the confirmation to appear then close it". Sleeping for a certain amount of time before checking whether the confirmation exists would work, but sleeps are highly discouraged since they're prone to flakiness.

3. Disable the test on K - any bugs that prevent cardboard clicks from registering *should* show up regardless of OS.

Seems to be an issue on L as well...

It seems there is a race condition issue between the second screen tap we send to chrome and the confirmation dialog. Will the confirmation dialog show up always if we don't send the second screen tap to Chrome? Is it expected?

If yes, how about waiting until the confirmation dialog shows up and close it, then send the second screen tap to Chrome?

The confirmation dialog will only show up if it's never been confirmed before on the device.

However, it looks like the device provisioning script somehow disables it, as it no longer appears during tests after provisioning. I would guess that it's this setting https://cs.chromium.org/chromium/src/third_party/catapult/devil/devil/android/settings.py?q=package:%5E(chromium)$+file:(/%7C%5E)devil/android/settings(%5C.(swig%7Cpy%7Cspt)$%7C/(__init__%5C.(swig%7Cpy%7Cspt))?$)&dr&l=83. However, since it's disabled during tests, I'm not sure why this is happening at all, as all the swarming devices should be provisioned properly. I'll check with John on this.

This is starting to look like a swarming problem. I found the bot/devices that this failed on at one point and ran a script as a swarming test that dumps the content://settings/secure table from the devices: https://isolateserver.appspot.com/browse?namespace=default-gzip&hash=7c196ea8594436e9d5b4100e105176eb88359a37

The first device has anr_show_background set to 0, but the rest don't. I'll double check that the devices it does pass on have it set to make sure that's actually the issue, then file a bug to get the devices re-provisioned.

Confirmed to be a swarming issue in Issue 711035 - apparently anr_show_background isn't currently set during swarming provisioning https://chrome-internal.googlesource.com/infradata/config/+/master/configs/chromium-swarm/scripts/android.py#227. bpastene@ has agreed to add it to the list of content://settings/secure settings to set during provisioning, so I'll mark this as closed after that goes in.

Closing this since anr_show_background setting has been added to swarming bots' provisioning.