New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 623104 link

Starred by 13 users

Issue metadata

Status: Duplicate
Merged: issue 657733
Owner:
Closed: Feb 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: All
Pri: 3
Type: Bug

Blocked on:
issue 543997

Blocking:
issue 290161



Sign in to add a comment

getUserMedia device selection strategy does not satisfy the W3C spec

Project Member Reported by chfremer@chromium.org, Jun 24 2016

Issue description

The W3C document specifies in Section 10.2.1 [1] how devices should be selected
when getUserMedia is called.

It basically says that first, one needs to find all devices that can satisfy
the given required constraints. Then it is up to the user agent to decide how
to choose between those eligible devices. The decision may or may not involve
the optional constraints and may or may not involve asking the user.

The current implementation does the following:
  1. If a required deviceId is given in the constraints, select the
     corresponding device.
  2. If a list of preferred deviceIds is given in the constraints (via the
     "advanced" section), select the first valid one in order of preference.
  3. If no deviceId constraints are given (or no valid optional ones are found),
     select the first device.

Here are two examples where the current implementation leads to undesired
behavior:

Example 1:
  See also Bug entry [2].
  On a device with a front-facing and a back-facing camera, getUserMedia() is
  called with the following constraints:
    {
      audio: false,
      video: { facingMode: 'environment' }
    }
  Current behavior in Chrome is that the first device is opened. On a Nexus 7,
  for example, the first device is the front-facing camera.

Example 2:
  A device such as the Nexus 7 has two cameras with different native
  resolutions. Camera 1 has 1280x720 and Camera 2 has 1920x1080.
  getUserMedia() is called with the following constraints:
    {
      audio: false,
      video: { width: {min: 1920} }
    }
  Current behavior in Chrome is that Camera 1 is opened, then the constraint is
  applied and this results in an Overconstrained error, even though Camera 2
  would be capable of satisfying the constraint.

[1] https://w3c.github.io/mediacapture-main/#methods-5
[2] https://bugs.chromium.org/p/chromium/issues/detail?id=290161

 
Cc: perkj@chromium.org hta@chromium.org
AFAIK, this is more of a missing feature than a bug. Adding more people to confirm that.


Comment 2 by mcasas@chromium.org, Jun 24 2016

Blocking: 290161

Comment 3 by hta@chromium.org, Jun 24 2016

The second error is probably based on incorrect information given to Chrome (or assumed by Chrome) about the capability of the cameras.

Again, these are not generic errors, they are very specific ones - in this case, that we select a camera that is not able to satisfy the constraints requested.

(Query: Which Chrome version were you using, and which flags were you using? Support for the spec form of constraints was unflagged last week, and is not even in beta yet. Until you have that version, the constraints in the standard form will be completely ignored.)

Comment 4 by hta@chromium.org, Jun 24 2016

Blockedon: 543997
I tested on a local debug build of Chromium based on the source repository. Last pull from origin/master was no longer than 2 days ago. I believe I enabled the experimental web platform flag.

In my analysis, I found that only the deviceId constraints are taken into account for device selection. All other constraints (including things such as facingMode or width/height for video) are stripped out for the device selection logic. Only after device selection is complete and streams have been generated do we apply the original full set of constraints.

I believe we need to change it so that the full set of constraints is taken into account for the device selection. 

Comment 6 by perkj@chromium.org, Jun 27 2016

Also note that there is a chrome://settings/content setting for selecting the default camera. 

So yes, Chrome pick the camera first and then check the rest of the constraints. I agree that example 1 is a bug but example 2 is not necessarily a bug, at least not on desktop. There are capture cards that a JS should not be able to use just by matching the constraints. 


#6: Interesting point about the capture cards on desktop. Please see some questions regarding that below. But regardless of whether or not there are different classes of capture devices that should be handled differently, if there is more than one "regular" capture device, according to the spec [1], the expected behavior is that all required constraints are taken into account for device selection. Both Example 1 and Example 2 are symptoms of our current implementation not doing that.

Regarding the capture cards on desktops:
Does this mean that the goal is to have different classes of capture devices behave differently in terms of how they can be selected? For example a desktop capture card would be discoverable through device enumeration and selectable through the returned device ID, but not by being the best match for a getUserMedia() request that does not specify its device ID explicitly.

The spec [1] does currently not mention anything about which devices should or should not be visible to JS. All it says is the following:

"For each possible source for that media type, construct an unconstrained MediaStreamTrack with that source as its source."

If we want to put restrictions on which capture devices JS should be able to use, do we need to be explicit about this in the spec or is this intended to be an interpretation of what "possible source" means? Who decides which capture devices JS should be able to use?

[1] https://w3c.github.io/mediacapture-main/#methods-5

Comment 8 by perkj@webrtc.org, Jun 28 2016

I would say its up to hta to decide if the current implementation is right or wrong.
(example 1 is clearly a bug.)
 
But yes, currently as far as I am aware, we first select the device and then match the rest of the constraints. 
ie. first match device id from the constraints. If none is specified, chrome settings are used to select the device. 
After that, the other constraints are matched. 

To me, it make sense that we use the device from chrome settings unless JS specifies a device id (or facingMode:)  to use even though the spec says otherwise. 
Owner: hta@chromium.org
Status: Assigned (was: Untriaged)
[triage]: assigning to hta to figure this out as per #8.

Comment 10 by hta@chromium.org, Aug 8 2016

Owner: guidou@chromium.org
Assigning to Guido, who's currently reimplementing this.
I think we have another similar bug; Guido to decide which one is a duplicate.

Cc: vrk@chromium.org tommi@chromium.org magjed@chromium.org
 Issue 273052  has been merged into this issue.
Project Member

Comment 12 by bugdroid1@chromium.org, Jan 24 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/52a3de796ac586a2ab7d6123663b01415e34966f

commit 52a3de796ac586a2ab7d6123663b01415e34966f
Author: guidou <guidou@chromium.org>
Date: Tue Jan 24 10:37:29 2017

Add functionality to get default media device from user preferences.

The motivation for this is that constraint algorithms for getUserMedia
and related functions need this information.
The existing mechanism to get the default device (requesting permission)
is insufficient as some algorithms, such as device selection, need this
information before requesting permission.

The WebContentsDelegate::GetDefaultMediaDeviceID is used in follow-up CL https://codereview.chromium.org/2629383002/.

BUG= 623104 
CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.linux:linux_site_isolation

Review-Url: https://codereview.chromium.org/2629383002
Cr-Commit-Position: refs/heads/master@{#445692}

[modify] https://crrev.com/52a3de796ac586a2ab7d6123663b01415e34966f/chrome/browser/media/webrtc/media_capture_devices_dispatcher.cc
[modify] https://crrev.com/52a3de796ac586a2ab7d6123663b01415e34966f/chrome/browser/media/webrtc/media_capture_devices_dispatcher.h
[modify] https://crrev.com/52a3de796ac586a2ab7d6123663b01415e34966f/chrome/browser/ui/browser.cc
[modify] https://crrev.com/52a3de796ac586a2ab7d6123663b01415e34966f/chrome/browser/ui/browser.h
[modify] https://crrev.com/52a3de796ac586a2ab7d6123663b01415e34966f/chrome/browser/ui/browser_browsertest.cc
[modify] https://crrev.com/52a3de796ac586a2ab7d6123663b01415e34966f/content/public/browser/web_contents_delegate.cc
[modify] https://crrev.com/52a3de796ac586a2ab7d6123663b01415e34966f/content/public/browser/web_contents_delegate.h

Mergedinto: 657733
Status: Duplicate (was: Assigned)
Merging with  issue 657733 , which is a more general bug about constraints, not just for device selection.
FWIW, spec-compliant device selection for cameras landed in M58, although the settings used to configure the device are not chosen with the spec-compliant algorithm.

Sign in to add a comment