New issue
Advanced search Search tips

Issue 630161 link

Starred by 3 users

Issue metadata

Status: WontFix
Owner:
Closed: Jul 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: All
Pri: 2
Type: Feature



Sign in to add a comment

chrome.desktopCapture.chooseDesktopMedia callback should include dimensions of window

Reported by fri...@gmail.com, Jul 21 2016

Issue description

UserAgent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_5) AppleWebKit/601.6.17 (KHTML, like Gecko) Version/9.1.1 Safari/601.6.17

Steps to reproduce the problem:
When calling navigator.webkitGetUserMedia with no dimensions or max dimensions, the image tends to be smaller than the actual window. So the strategy is for the constraints to use large values for the max dimensions. This works for the most part, however, smaller windows are captured larger than their actual size. The result is that small windows might be captured at 200% of the actual size.

If the callback included the width and height, then you can capture the window at the correct size and know that you are screen sharing the window at the actual size.

What is the expected behavior?

What went wrong?
chrome.desktopCapture.chooseDesktopMedia only provides the window id, but not the width and height of the window selected by the user

Did this work before? N/A 

Chrome version: 51.0.2704.106 (64-bit)  Channel: stable
OS Version: OS X 10.11.5
Flash Version:
 
Components: Platform>DevTools
Labels: M-54
Status: Untriaged (was: Unconfirmed)
Untriaged the issue so as to triage it further.

Comment 2 by caseq@chromium.org, Jul 22 2016

Components: -Platform>DevTools

Comment 3 by tapted@chromium.org, Jul 25 2016

Components: Blink>WebRTC
Labels: -OS-Mac OS-All
[mac triage]
Cc: niklase@chromium.org

Comment 5 by mcasas@chromium.org, Jul 29 2016

Components: -Blink>WebRTC Blink>GetUserMedia>Desktop
Labels: -M-54 M-55
niklase@ what's your take on this?
Labels: Needs-Feedback
I don't think it should be necessary, but having no constraints should give a stream with the true window resolution. Can you give us some details on how that fails? 

Comment 8 by fri...@gmail.com, Aug 31 2016

If you do not provide constraints, then small windows are true window resolutions, but larger windows are limited in size.

It would be great if the windows are always using true window resolution with a max specified resolution.  However, today small windows are larger than life when using large max dimensions.  This is more likely to happen on HDPI displays (such as the MacBook Pro).
Can you give a concrete example with app shared and numbers for resolution?

Comment 10 by fri...@gmail.com, Aug 31 2016

I am on a macbook pro 15" with Retina display.  The settings are on scaled for more space.  The resolution on the display reported: 1920x1200.

Now I make safari as small as possible (708x278) and share that.  
- If I use a max resolution of 1280x1024, then the shared video dimensions are: 1280x502

- If I use a max resolution of 1920x1200, then the shared video dimensions are: 1416x556

I am uploading 2 files.
- safari.png is a screen shot of the safari window
- test2.png is what was captured in test2.
safari.png
65.2 KB View Download
test2.png
190 KB View Download

Comment 11 by fri...@gmail.com, Aug 31 2016

These issues only appear to happen on HDPI displays (both mac and windows).  When on older non-HDPI display, it seems to work as expected.  

If you specify the correct dimensions then it works on HDPI, but since we don't know the dimensions in advance, we cannot do that.

An alternative fix would be to make this work on HDPI displays...  There are useful scenarios where knowing the dimensions of a window allows you to decide if you want to scale it and by how much (you can be more explicit about it) to get better performance and legibility.
What you see here comes from that we capture the actual screen resolution. When your window is rendered in a Retina display it's upscaled a factor 2, to 1416x556. We try to send that except when the max constraints prevents it. This is working as intended. What about the other case where you have no max and get a smaller resolution?

Comment 13 by fri...@gmail.com, Aug 31 2016

How will I build a screen shot application without knowing the dimensions or the scaling factor?

Comment 14 by fri...@gmail.com, Aug 31 2016

The other issue is that when I send the screen capture video over the network with WebRTC, the participants sees it in the extra large video.  How would I solve that?
Labels: -Needs-Feedback
niklase: Any response to #14 or can we loop in somebody else?
Labels: Needs-Feedback
Not sure what #14 is about. You can make the play <video> any size you want to.

Comment 17 by fri...@gmail.com, Sep 12 2016

Since it is a desktop share, the video is of an application and users expects to see the application video at the original resolution and not larger than life.  This makes it more readable and application windows are not unexpected sizes to users.

Since I have no idea if the video is scaled larger, I cannot reduce it in size for the users... if you give me some indication I can do that.  (even though sending larger video, would still use more bandwidth than needed and slow down application sharing).

Also, when I do a screenshot application, users expect the screenshot to be the exact size that they see.  When I let them email the screen shot, it is twice as big as it should be.
friksa@ I think it would be easiest if you can create a minimal reproduction case and put it up at https://jsfiddle.net/ to showcase when you think the size of the captured images are incorrect and/or width+height information is missing. Then we can try it on a regular density display and a HDPI one.

Comment 19 by fri...@gmail.com, Oct 3 2016

@niklase already explained that Chrome captures the video at the native screen resolution.  That is exactly the issue on HDPI displays.  

When you build a screen shot application and you take a screen shot at the native HDPI resolution, then you show it in the browser or save it to a PNG, it is 1.5x bigger than what the user expects.

The same is true for video when doing screen sharing or record the screen.
It didn't make it into M55 so moving to M56.
Labels: -M-55 M-56

Comment 22 by fri...@gmail.com, Oct 18 2016

Thanks so much!!

Since @niklase clarified that the capture on HDPI displays are done at the native resolution, we also need to know the scaling factor so that the dimensions can be translated. Otherwise we will not know how much smaller it should be made to look life-size to the user.
Cc: sergeyu@chromium.org
friksa@ #22 is confusing, is there still something 
that needs to be done for this issue?

Comment 24 by fri...@gmail.com, Oct 24 2016

The API callback should include the dimensions of the video and the scaling factor used.  On non-hdpi displays, the scaling will be 1.  On HDPI displays, it might be 1.5 or 2.

Please take a look at the images I posted in #10.
Labels: -M-56 M-57
Bumping to M57. Please update if that's wrong.
Labels: -Needs-Feedback -M-57 Needs-Review M-58
Bumping this to M58. Please correct if that's wrong.
Labels: -Needs-Review
Cleaning up sheriffbot label "Needs-Review" label as a part of modified "Needs-Feedback" sheriffbot rule. [ref bug for cleanup 684919]
Labels: -M-58 M-59
Cc: -niklase@chromium.org
Owner: niklase@chromium.org
Status: Assigned (was: Untriaged)
Labels: -M-59 M-60
Bumping to M60. Please correct if that's wrong.
Labels: -M-60 M-61
Status: WontFix (was: Assigned)
This should be discussed here:https://w3c.github.io/mediacapture-screen-share/, not in a chrome bug.

Sign in to add a comment