New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 786880 link

Starred by 6 users

Issue metadata

Status: Fixed
Owner:
Closed: Mar 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Windows
Pri: 2
Type: Bug



Sign in to add a comment

Rotate frames according to camera orientation on Windows

Project Member Reported by huib@chromium.org, Nov 20 2017

Issue description

Similar to  https://crbug.com/684375  
To support devices like the Lenovo Yoga X1 we should rotate camera frames according to device orientation.

 
Cc: jcliang@chromium.org

Comment 2 by cvint...@google.com, Nov 20 2017

Cc: cvintila@chromium.org kotah@chromium.org

Comment 3 by huib@chromium.org, Nov 20 2017

Owner: niklase@chromium.org
Niklas, can you take a look at this?
Owner: chfremer@chromium.org
Status: Assigned (was: Untriaged)
Cc: braveyao@chromium.org
I am currently backed up with other work, so it is unclear when I could start working on this.

Since I don't want this work to be blocked on me, maybe we can delegate to someone else?

braveyao@: Would you happen to have time to look into this? I think you also happen to have a Lenovo Yoga X1.
Yes I have a Yoga X1.
And I have my plate full with some critical issues which are too hard to give any timeline. From the email thread, this feature is pushed by customer but still have a loose timeline, i.e. Q2'2018? In that case I can put it into my queue :)
Cc: -braveyao@chromium.org chfremer@chromium.org
Owner: braveyao@chromium.org
I'm taking care of this now.

Did some research and found this is actually a very complicated feature. The reason is all about the possibilities that external camera/monitor can be connected to the window tablet and related configurations can be changed on the fly. Skype doesn't handle all the cases correctly.

Now found ways to handle multi-monitors case. We can detect internal and external monitors and their status. And only do rotation when the active internal panel changes its orientation.

But there seems being lack of APIs to identify if a camera is internal( front/back) or external USB one for desktop apps(win32 apps). MS added such info since Windows 8.1 for UWP platform, which should not be available in chromium yet.
No idea how often user will rotate tablet with external USB camera. I guess it may be OK to assume built-in camera is the major concern in tablet-rotate-case. At least we can move on with listing this as an known-issue until related UWP APIs is hooked up in Chromium.

Comment 8 by huib@chromium.org, Jan 5 2018

Leaving some known issues for later is reasonable, since it's still a big improvement over the current situation.
Would it be safe to assume the first camera is always internal?
Most likely the internal camera will be connected first but this is not guaranteed! It depends on the order in which the OS queries the various USB hubs.
Project Member

Comment 10 by bugdroid1@chromium.org, Jan 12 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/02bfcdc0ff540f1e3c47960e829145dc0f68fdf0

commit 02bfcdc0ff540f1e3c47960e829145dc0f68fdf0
Author: braveyao <braveyao@chromium.org>
Date: Fri Jan 12 21:41:43 2018

[Video capture] Rotate frames according to camera orientation on Windows.

To support devices like Lenovo Yoga X1, we should rotate camera frames
according to device orientation on Windows.

The difficulties are there will be external cameras and monitors connect
or disconnected to the window machine at any time and also the relevant
configurations. So we need to only do rotation when internal camera and
internal display panel are currently working.

Known issue: without UWP APIs, we can't identify the camera location,
e.g. front/back or external USB camera. For now we assume it's always
front camera when AutoRotation is enabled on the device. Rotate device with
external USB camera will get wrong orientation.

Notes: If user chooses to display on external monitor only, we can't query
the orientation of internal display panel, so the camera will be treated
as external one in such circumstance and will not do rotation.

Bug:  786880 
Change-Id: I553e17d1a945f5a5560f1fbfcc25a0c1a0ddaadc
Reviewed-on: https://chromium-review.googlesource.com/855318
Reviewed-by: Christian Fremerey <chfremer@chromium.org>
Commit-Queue: Weiyong Yao <braveyao@chromium.org>
Cr-Commit-Position: refs/heads/master@{#529073}
[modify] https://crrev.com/02bfcdc0ff540f1e3c47960e829145dc0f68fdf0/media/capture/BUILD.gn
[modify] https://crrev.com/02bfcdc0ff540f1e3c47960e829145dc0f68fdf0/media/capture/video/win/video_capture_device_mf_win.cc
[add] https://crrev.com/02bfcdc0ff540f1e3c47960e829145dc0f68fdf0/media/capture/video/win/video_capture_device_utils_win.cc
[add] https://crrev.com/02bfcdc0ff540f1e3c47960e829145dc0f68fdf0/media/capture/video/win/video_capture_device_utils_win.h
[modify] https://crrev.com/02bfcdc0ff540f1e3c47960e829145dc0f68fdf0/media/capture/video/win/video_capture_device_win.cc

With the cl in #10, we can support device rotation of Win tablet for most of the working scenario, with knowing issue that rotating device with external USB camera in working will get wrong orientation.
I'll follow up the camera-location-detection stuff. It seems that we at least can support it on Win10, if there is no universal way for all windows versions. Will report back soon.
Project Member

Comment 12 by bugdroid1@chromium.org, Mar 20 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/03cd475d231eab645bf38b86a511bb4e5d20186a

commit 03cd475d231eab645bf38b86a511bb4e5d20186a
Author: braveyao <braveyao@chromium.org>
Date: Tue Mar 20 21:18:43 2018

[Video Capture Win] Enumerate camera location with UWP APIs

Enumerate camera locations with UWP APIs on Windows10 and save them into
|VideoCaptureDeviceDescriptors| . The location info can be used for
frame rotation when a windows tablet is rotated during a video call.
Only frames captured by an internal camera will be rotated accordingly.

PS: on Win8/7, which doesn't support UWP APIs, we'll treat all cameras,
internal or USB external, as internal one in frame rotation processing
to cover most user cases.

Bug:  786880 
Change-Id: Idb3b5477159bf07f9158ebbd03677cf99c7b5680
Reviewed-on: https://chromium-review.googlesource.com/955897
Reviewed-by: Christian Fremerey <chfremer@chromium.org>
Commit-Queue: Weiyong Yao <braveyao@chromium.org>
Cr-Commit-Position: refs/heads/master@{#544520}
[modify] https://crrev.com/03cd475d231eab645bf38b86a511bb4e5d20186a/media/capture/video/fake_video_capture_device_factory.cc
[modify] https://crrev.com/03cd475d231eab645bf38b86a511bb4e5d20186a/media/capture/video/fake_video_capture_device_factory.h
[modify] https://crrev.com/03cd475d231eab645bf38b86a511bb4e5d20186a/media/capture/video/file_video_capture_device_factory.cc
[modify] https://crrev.com/03cd475d231eab645bf38b86a511bb4e5d20186a/media/capture/video/file_video_capture_device_factory.h
[modify] https://crrev.com/03cd475d231eab645bf38b86a511bb4e5d20186a/media/capture/video/video_capture_device_factory.cc
[modify] https://crrev.com/03cd475d231eab645bf38b86a511bb4e5d20186a/media/capture/video/video_capture_device_factory.h
[modify] https://crrev.com/03cd475d231eab645bf38b86a511bb4e5d20186a/media/capture/video/video_capture_system_impl.cc
[modify] https://crrev.com/03cd475d231eab645bf38b86a511bb4e5d20186a/media/capture/video/video_capture_system_impl.h
[modify] https://crrev.com/03cd475d231eab645bf38b86a511bb4e5d20186a/media/capture/video/win/video_capture_device_factory_win.cc
[modify] https://crrev.com/03cd475d231eab645bf38b86a511bb4e5d20186a/media/capture/video/win/video_capture_device_factory_win.h
[modify] https://crrev.com/03cd475d231eab645bf38b86a511bb4e5d20186a/media/capture/video/win/video_capture_device_mf_win.cc
[modify] https://crrev.com/03cd475d231eab645bf38b86a511bb4e5d20186a/media/capture/video/win/video_capture_device_mf_win.h
[modify] https://crrev.com/03cd475d231eab645bf38b86a511bb4e5d20186a/media/capture/video/win/video_capture_device_mf_win_unittest.cc
[modify] https://crrev.com/03cd475d231eab645bf38b86a511bb4e5d20186a/media/capture/video/win/video_capture_device_utils_win.cc
[modify] https://crrev.com/03cd475d231eab645bf38b86a511bb4e5d20186a/media/capture/video/win/video_capture_device_utils_win.h
[modify] https://crrev.com/03cd475d231eab645bf38b86a511bb4e5d20186a/media/capture/video/win/video_capture_device_win.cc
[modify] https://crrev.com/03cd475d231eab645bf38b86a511bb4e5d20186a/services/video_capture/test/mock_device_factory.cc
[modify] https://crrev.com/03cd475d231eab645bf38b86a511bb4e5d20186a/services/video_capture/test/mock_device_factory.h

Status: Fixed (was: Assigned)
With the latest Cl, we can detect the camera location on Windows 10 and do the frame rotation accordingly. 
As to Win8/7, there is no API to get camera location. So we keep assuming that it's the built-in facing-front camera working during auto-rotation is ON, which should be the major cases.

Mark this as Fixed.
Cc: qiangchen@chromium.org
 Issue 645937  has been merged into this issue.
Cc: pbomm...@chromium.org
 Issue 746000  has been merged into this issue.

Sign in to add a comment