getUserMedia interprets differently video width depending on phone orientation |
|||||||
Issue descriptionChrome: 60.0.3089.0 Device name: Google Pixel Phone 7.1.2 Steps to reproduce: 1. Hold device in landscape mode 2. Go to https://beaufortfrancois.github.io/sandbox/media/hd.html 3. getUserMedia Constraints are 1920x1080 and I get a beautiful 1920x1080 video. Yeah! 4. Hold device in portrait mode now 5. New getUserMedia constraints are 1080x1920... Expected result: I should get 1080x1920 video Actual result: I get a 1080x1080 video Note that Chrome Stable (58.0.3029.83) currently returns smaller videos (640x480 and 480x640).
,
May 5 2017
This doesn't look like a bug.
If your constraints say {height: 1920, width: 1080} and the resolution is 1080x1920, you will get 1080x1080 since you say you want width closest to 1080 (you get it) and height closest to 1920 (you get 1080, which is as close as you can get).
Perhaps you can try something like:
{advanced: [{height: 1920, width: 1080}, {height:1080, width: 1920}]}
Then it will first try exactly 1920x1080, if not possible, exactly 1080x1920. If not possible, it will try the default, which is 640x480.
You can also use:
{basic: {height:1920, width:1080}, advanced: [{height: 1920, width: 1080}, {height:1080, width: 1920}]}
Note that these "naked" values are interpreted as ideal in the basic section and as exact in the advanced section.
It will first try 1920x1080 exact, then 1080x1920, then you'll get whatever is closest to 1920x1080.
You can add more advanced sections to try other resolutions.
,
May 5 2017
I see that the problem is that 1080x1920 is not reported as a supported resolution. On phones that do not report supported resolution we use a fallback list and that list only includes the landscape resolutions. I'll prepare a patch to add portrait resolutions on Android.
,
May 10 2017
I've updated https://beaufortfrancois.github.io/sandbox/media/hd.html to include track settings.
,
May 10 2017
Here are some stats I've gathered on my Pixel phone: Firefox 53.0.2 -------------- Portrait: Video element: 768x1024 Track settings: 1024x768 Landscape: Video element: 1024x768 Track settings: 1920x1080 Chrome Canary 60.0.3094.0 ------------------------- Portrait: Video element: 1080x1080 Track settings: 1080x1080 Landscape: Video element: 1920x1080 Track settings: 1920x1080 Note: Chrome Beta and Chrome Dev are the same results as Chrome Canary. Chrome Stable 58.0.3029.83 -------------------------- Portrait: Video element: 480x640 Track settings: N/A Landscape: Video element: 640x480 Track settings: N/A
,
May 10 2017
Firefox Beta 54.0b6 ------------------- Portrait: Video element: 1080x1920 Track settings: 1920x1080 Landscape: Video element: 1920x1080 Track settings: 1920x1080
,
May 10 2017
Please disregard my comment #3. You should not use 1080x1920, but just use 1920x1080. The implementation should deal with the rotation. Because you are using 1080x1920, the track settings in Firefox are 1024x768 and 1080x1080 in Chrome. Since 1080x1920 those are the closest resolutions they can give you, given that you specify a maximum width of 1080. The bug in Chromium is in VideoTrackAdapter::CalculateTargetSize, which is not using the rotation to make the computation. When the phone is rotated the input frame is 1080x1920, but CalculateTargetSize thinks the maximum it should be is 1920x1080 and crops it to 1080x1080. The fix is to incorporate the rotation to CalculateTargetSize.
,
May 10 2017
I tried to make a quick fix by looking at the rotation metadata in the frame, but they do not have the rotation in the metadata. mcasas/chfremer: Do you know a good way to pass the rotation information (frame metadata or other mechanism) ?
,
May 10 2017
I tried to make a quick fix by looking at the rotation metadata in the frame, but they do not have the rotation in the metadata. mcasas/chfremer: Do you know a good way to pass the rotation information (frame metadata or other mechanism) ?
,
May 10 2017
I believe the rotation information is available in VideoCaptureDeviceClient at [1], but is not included alongside the frame from there on its path to the consumers. Where in the path does the constraint processing happen? I guess one option would be to pass the information alongside the frame all the way from VideoCaptureDeviceClient. Assuming that the rotation information originates from a device API, would it be a viable alternative to have the constraint processing module obtain the rotation directly from the device API and assume that it matches with the rotation of incoming frames? [1] https://cs.chromium.org/chromium/src/media/capture/video/video_capture_device_client.cc?dr=CSs&l=138
,
May 10 2017
Thanks for the information. The rotation is not necessary for processing constraints, but for processing frames after the track has started. The rotation information is required in VideoTrackAdapter. I'll try to come up with a patch tomorrow and will get back to you.
,
May 10 2017
FTR, Rotation in Android is read in https://cs.chromium.org/chromium/src/media/capture/video/android/java/src/org/chromium/media/VideoCapture.java?sq=package:chromium&dr=CSs&l=140
,
May 11 2017
After consulting with foolip@, passing the rotation in the frames would be wrong because that would mean the frames need to be rotated to be displayed correctly, which is not the case here. Using a device API like chfremer@ suggests would be better, but I think I can produce something that works by passing the expected native resolution of the frames to the VideoTrackAdapter.
,
May 16 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/0ac21bec152a95d8edb8e759f765a512d552357c commit 0ac21bec152a95d8edb8e759f765a512d552357c Author: guidou <guidou@chromium.org> Date: Tue May 16 10:18:51 2017 Detect frames from rotated devices in VideoTrackAdapter on Android. On Android, when a device is rotated, frames from the camera arrive with width and height swapped. The VideoTrackAdapter was unable to detect this and instead cropped them incorrectly because it treated the rotated dimension as a violation of the maximum size settings. This CL provides a simple mechanism to detect this problem on Android. If the frame arrives from the camera with width and height swapped from what was expected, treat it as a valid frame that does not need to be adjusted. BUG= 718823 Review-Url: https://codereview.chromium.org/2870413004 Cr-Commit-Position: refs/heads/master@{#472057} [modify] https://crrev.com/0ac21bec152a95d8edb8e759f765a512d552357c/content/renderer/media/media_stream_constraints_util.cc [modify] https://crrev.com/0ac21bec152a95d8edb8e759f765a512d552357c/content/renderer/media/media_stream_constraints_util.h [modify] https://crrev.com/0ac21bec152a95d8edb8e759f765a512d552357c/content/renderer/media/media_stream_constraints_util_unittest.cc [modify] https://crrev.com/0ac21bec152a95d8edb8e759f765a512d552357c/content/renderer/media/media_stream_constraints_util_video_content.cc [modify] https://crrev.com/0ac21bec152a95d8edb8e759f765a512d552357c/content/renderer/media/media_stream_constraints_util_video_device.cc [modify] https://crrev.com/0ac21bec152a95d8edb8e759f765a512d552357c/content/renderer/media/media_stream_video_source.cc [modify] https://crrev.com/0ac21bec152a95d8edb8e759f765a512d552357c/content/renderer/media/media_stream_video_source_unittest.cc [modify] https://crrev.com/0ac21bec152a95d8edb8e759f765a512d552357c/content/renderer/media/media_stream_video_track_unittest.cc [modify] https://crrev.com/0ac21bec152a95d8edb8e759f765a512d552357c/content/renderer/media/video_track_adapter.cc [modify] https://crrev.com/0ac21bec152a95d8edb8e759f765a512d552357c/content/renderer/media/video_track_adapter.h
,
May 16 2017
,
May 22 2017
,
May 22 2017
Your change meets the bar and is auto-approved for M59. Please go ahead and merge the CL to branch 3071 manually. Please contact milestone owner if you have questions. Owners: amineer@(Android), cmasso@(iOS), gkihumba@(ChromeOS), Abdul Syed@(Desktop) For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
May 22 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/a110813cbb7d0f8ff2395a6a4138183e97946ea5 commit a110813cbb7d0f8ff2395a6a4138183e97946ea5 Author: Guido Urdaneta <guidou@chromium.org> Date: Mon May 22 15:42:01 2017 Detect frames from rotated devices in VideoTrackAdapter on Android. On Android, when a device is rotated, frames from the camera arrive with width and height swapped. The VideoTrackAdapter was unable to detect this and instead cropped them incorrectly because it treated the rotated dimension as a violation of the maximum size settings. This CL provides a simple mechanism to detect this problem on Android. If the frame arrives from the camera with width and height swapped from what was expected, treat it as a valid frame that does not need to be adjusted. BUG= 718823 Review-Url: https://codereview.chromium.org/2870413004 Cr-Original-Commit-Position: refs/heads/master@{#472057} Review-Url: https://codereview.chromium.org/2897993002 . Cr-Commit-Position: refs/branch-heads/3071@{#652} Cr-Branched-From: a106f0abbf69dad349d4aaf4bcc4f5d376dd2377-refs/heads/master@{#464641} [modify] https://crrev.com/a110813cbb7d0f8ff2395a6a4138183e97946ea5/content/renderer/media/media_stream_constraints_util.cc [modify] https://crrev.com/a110813cbb7d0f8ff2395a6a4138183e97946ea5/content/renderer/media/media_stream_constraints_util.h [modify] https://crrev.com/a110813cbb7d0f8ff2395a6a4138183e97946ea5/content/renderer/media/media_stream_constraints_util_unittest.cc [modify] https://crrev.com/a110813cbb7d0f8ff2395a6a4138183e97946ea5/content/renderer/media/media_stream_constraints_util_video_content.cc [modify] https://crrev.com/a110813cbb7d0f8ff2395a6a4138183e97946ea5/content/renderer/media/media_stream_constraints_util_video_device.cc [modify] https://crrev.com/a110813cbb7d0f8ff2395a6a4138183e97946ea5/content/renderer/media/media_stream_video_source.cc [modify] https://crrev.com/a110813cbb7d0f8ff2395a6a4138183e97946ea5/content/renderer/media/media_stream_video_source_unittest.cc [modify] https://crrev.com/a110813cbb7d0f8ff2395a6a4138183e97946ea5/content/renderer/media/media_stream_video_track_unittest.cc [modify] https://crrev.com/a110813cbb7d0f8ff2395a6a4138183e97946ea5/content/renderer/media/video_track_adapter.cc [modify] https://crrev.com/a110813cbb7d0f8ff2395a6a4138183e97946ea5/content/renderer/media/video_track_adapter.h
,
May 23 2017
Verified fix on Pixel/ N running Chrome "60.0.3107.3"
,
May 31 2017
|
|||||||
►
Sign in to add a comment |
|||||||
Comment 1 by fbeaufort@chromium.org
, May 5 2017Components: Blink>GetUserMedia>Webcam