New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 844822 link

Starred by 4 users

Issue metadata

Status: Assigned
Owner:
Last visit > 30 days ago
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 3
Type: Bug
Proj-XR



Sign in to add a comment

Remove necessity of user gesture for VR gamepads

Reported by artyo...@gmail.com, May 19 2018

Issue description

Steps to reproduce the problem:
VR controllers in WebVR are represented via Gamepad API. Gamepad API requires user gesture to reveal the controller to a page. This is not necessary for VR gamepads (such as Daydream controller) and that requirement could be lifted, since user already has committed the user gesture - entered VR.

What is the expected behavior?
VR gamepads should become available right after entering VR.

What went wrong?
File /device/gamepad/gamepad_user_gesture.cc could be modified by the following way:

if (pad.display_id != 0) return true;

display_id is set for VR controllers only.

Did this work before? N/A 

Does this work in other browsers? Yes

Chrome version: master  Channel: n/a
OS Version: 
Flash Version:
 
Status: Untriaged (was: Unconfirmed)
Labels: -Pri-2 Proj-VR Pri-3
Status: Available (was: Untriaged)
The issue of needing a user gesture for input after we already have one for the VR is addressed in the latest version of the spec, https://immersive-web.github.io/webxr/#dom-xrsession-getinputsources. At present we have no plans to implement this fix for the Gamepad api devices in Chromium, but would welcome patches.

Comment 3 by artyo...@gmail.com, May 31 2018

It is really not, if you use WebXR (or WebVR) + gamepad API. All what you need to add is one line "if (pad.display_id != 0) return true;" at the beginning of the method GamepadsHaveUserGesture(...) in /device/gamepad/gamepad_user_gesture.cc
Artem is right, it really is a one-line fix and it should be safe as we only report VR devices (which are the only devices that have a display id) when in WebVR presentation.

The only reason not to bother is that WebVR is being replaced and WebXR has a different solution, but given how trivial this fix is I think we might as well improve WebVR while it's around.
Owner: mthiesse@chromium.org
Status: Started (was: Available)
Labels: M-68
Also might as well get this into M-68 as it's really just a safe quality of life improvement.
Cc: bajones@chromium.org mattreynolds@chromium.org
The gamepad spec mentions that devices are connected when the first button/axis change happens.  If we are diverging from that for VR, should we have a spec discussion or ensure other browsers will be making similar behavior changes (and update the spec accordingly)?  The web platform should ideally be consistent between browsers, and have the behavior documented in specs.

Arguably, the axis of the VR controller is always changing, and that's really the most useful part of the controller.
Project Member

Comment 9 by bugdroid1@chromium.org, May 31 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/b1ea93b5daf3553add97b635e36c15b955062546

commit b1ea93b5daf3553add97b635e36c15b955062546
Author: Michael Thiessen <mthiesse@chromium.org>
Date: Thu May 31 17:59:54 2018

Bypass 'gesture' requirement for using VR controllers in VR presentation

Bug: 844822
Change-Id: I348593a0406ce1f8e6a839d92d2df2420c4bea74
Reviewed-on: https://chromium-review.googlesource.com/1080847
Commit-Queue: Michael Thiessen <mthiesse@chromium.org>
Reviewed-by: Brandon Jones <bajones@chromium.org>
Cr-Commit-Position: refs/heads/master@{#563313}
[modify] https://crrev.com/b1ea93b5daf3553add97b635e36c15b955062546/device/gamepad/gamepad_user_gesture.cc

> Arguably, the axis of the VR controller is always changing, and that's really the most useful part of the controller.

True, but the point is to avoid exposing more surface for fingerprinting until the user has made some indication they want to use the device. Was this change reviewed by chrome-privacy-core@? We should make sure they are aware of the change since this feature has been of particular interest in the past.

It sounds like we have a strong indication since the user is already in WebVR presentation mode. What sort of interaction is required to enter that mode? If it involves putting on a headset or explicitly enabling VR features for a particular origin I'd guess it's probably fine to expose any connected gamepads as well.
At minimum it requires a user activation event (click/touch) on the page. Also requires that the page be HTTPS, since this is an origin trial only.
Labels: -M-68 M-69
However, there are downstream user agents using this code that do not require HTTPS.

I think the way this was implemented seems fragile since it makes assumptions about when VR Gamepads get added. The fact that it also affects non-VR Gamepads is concerning. Also, the timeline for removal is unclear since we will still have WebXR Gamepad Support for some time.
> At minimum it requires a user activation event (click/touch) on the page. Also requires that the page be HTTPS, since this is an origin trial only.

Also, on Android you have to go through headset insertion and controller pairing/wakeup before the page can present, so the user is very much aware and expecting their controller to be used.
> However, there are downstream user agents using this code that do not require HTTPS.

Artem can correct me here if I'm wrong, but I get the impression that downstream browsers have already done this.

> I think the way this was implemented seems fragile since it makes assumptions about when VR Gamepads get added.

I don't think it's really that fragile. It allows access when the device has a display ID, which will only be the case when it's provided to a session.
FWIW, Firefox does give out controllers immediately without a user gesture, even in magic window mode.  Haven't tried Edge.  It would be ideal if the spec matched reality.  Rigorous accurate specs prevent developers from testing on one browser and creating defacto standards by accident.

The impl is somewhat fragile in that it assumes vr gamepads are only added when presentation starts - if we initialized them immediately on Chrome startup, this would effectively remove the gesture requirement if there is a vr gamepad connected.  In fact, openvr and oculus gamepads are added before presentation starts - when the provider initializes for magic window usage.  When VR gamepads get added should also be part of the spec.

We are still giving out more information with WebVR than gamepad, so I'm not concerned with the privacy change for this specific change.  I'm not tremendously concerned about spec deviations since webvr is in origin trial and will be deprecated in favor of webxr.


Comment 16 Deleted

For security reasons we shouldn't initialize the gamepad API without user interaction. Gamepads are accessible from any visible tab, and could be used to communicate between tabs.
Well we definitely have user interaction with the page to request an exclusive VR session. On Android we definitely have interaction with the controller as well. On Windows I'm not sure what the initialization flow is for the various devices.
Components: Blink>WebXR
Owner: ddorwin@chromium.org
Status: Assigned (was: Started)
Assigning to David to figure out what we want here long-term. Right now the gesture requirement is bypassed.
Windows VR gamepads now only update while presenting, so most concerns are addressed.
Owner: cwilso@chromium.org
Labels: BlinkWebXR
Removing Blink>WebVR component and assigning to Blink>WebXR 
Labels: -BlinkWebXR
Removing Blink>WebVR component and assigning to Blink>WebXR 
Components: -Blink>WebVR

Sign in to add a comment