New issue
Advanced search Search tips

Issue 913203 link

Starred by 9 users

Issue metadata

Status: Started
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: 2019-01-18
OS: Android
Pri: 2
Type: Bug



Sign in to add a comment

Bad image quality from getUserMedia with default settings

Project Member Reported by malteubl@google.com, Dec 8

Issue description

Chrome Version: 71.0.3578.83
OS: Android on Pixel 3

What steps will reproduce the problem?
(0) I tested only with Pixel 3 XL. Not sure it is device specific.
(1) Capture image with getUserMedia
(2) Point at slightly bright background (like the sky or a white wall)

I tested with various apps and sample apps. The results are consistent.

This is one of them https://simpl.info/imagecapture/

What is the expected result?

Image quality is identical or at least comparable to device's camera app.

What happens instead?

Image has off white-balance, is over-exposed, or something else (I'm not an expert in photography) that makes it look bad.

The last attached image was taken with the default camera app for comparison.
 
Screenshot_20181207-103804 (1).png
3.1 MB View Download
Screenshot_20181207-151559.png
2.7 MB View Download
camera-app.jpg
3.5 MB View Download
Components: Blink>GetUserMedia
Components: -Blink>GetUserMedia Blink>GetUserMedia>Webcam
Did you use takePhoto()? takePhoto() should give you "native camera" quality while grabFrame() will not. Just in case you are not satisfied with the default auto settings, there are a host of manual controls we are working on[1,2], like manual settings for whitebalance/color temperature, exposureTime and focusDistance.

To fix too much brightness or darkness, changing exposure time helps.

If you want to try exposureTime/focusDistance etc to recreate the "pro" controls you find on native camera app, here is some WIP demo I am working on (please use chrome canary/chrome72):
https://riju.github.io/WebCamera/samples/

[1] https://crbug.com/823316
[2] https://crbug.com/732807
I was just playing with several demos, and one production app (not my own). They all produced equally bad results, so I'm wondering whether the defaults are bad.
Cc: mcasas@chromium.org rijubrat...@intel.com
I don't have a Pixel 3 with me right now, but my guess is that the native camera app does use some computational photography / AI / post processing, etc before the actual image is saved. For takePhoto() on Android, generally, camera preview images are sent to SurfaceView and  the capture of JPEG images are done with ImageReader with the JPEG. So maybe on Pixel 3, the defaults don't give you the same "look" as the AI powered native camera.

I'm comparing against the "video" that is shown on the screen before taking a photo (The view finder). While it doesn't perfectly match the post-processed image, it still looks good and comparable to the photo. The direct capture in Chrome looks really bad.

More apples to apples attached:

The first screenshot shows the view finder and the takePhoto on Pixel 3. Both look horrible.
The second screenshot shows the native camera app (screenshot, so this isn't post-processed).

Chrome's behavior is unusably bad.
Screenshot_20181213-074108.png
1.8 MB View Download
Screenshot_20181213-074124.png
2.3 MB View Download
I quickly tried https://simpl.info/imagecapture/ on my Samasung Galaxy S8+, attaching the pics. I can take check again tomorrow with bright/similar conditions as yours.
screenshot_takePhoto.jpg
1.6 MB View Download
Image_NativeCamera.jpg
2.8 MB View Download
I was shooting in a bus away from the sun. My initial shots were indoors as well.
Cc: chfremer@google.com
We're seeing the same issue of over-saturated, over-blown photos when taken with the takePhoto API. Currently we can only reproduce on Pixel 3, Pixel 1 seems to work as expected. This appears to be a regression, as we were not seeing this behavior before.
Also, the stream from getUserMedia is also affected by the over-saturation, so its not just the ImageCapture API.
Cc: -chfremer@google.com chfremer@chromium.org
Do we already know if this affects every Pixel 3 device on M71+, or have we seen any cases where it behaved normally?

Do we already know if this is a regression from M70 to M71?
I just tested with Chrome 66.0.3359.158 on a Pixel 3 and I still see the over-exposed stream.

Another observation: if stream width/height/framerate constraints provided, the over-exposed issue seems to go away.

Example constraints that make problem appear to go away: {
  width: {
    ideal: 1280,
    max: 1920,
  },
  height: {
    ideal: 720,
    max: 1080,
  },
  frameRate: 30,
  facingMode: 'environment',
}
Interesting. 
I will check how the different constraints settings translate to differences in the calls that Chrome makes into the Android camera APIs. Unless we find that Chrome somehow ends up making unreasonable or incorrect calls to those APIs, we may have to report this to the Android camera team.
Cc: -chfremer@chromium.org
Owner: chfremer@chromium.org
Status: Started (was: Untriaged)
I managed to get my hands on a Pixel 3 and was able to reproduce the issue by navigating to https://webrtc.github.io/samples/src/content/devices/input-output/ and manually switching there to the back facing camera. When pointing the camera to my screen showing a white background with black text, the image on the phone would be pretty much all white instead of showing the text.

The issue does not seem to affect the front facing camera.

Since Chrome does a lot of camera configuration settings, I tried to systematically disable some of them to see if it helps. It turns out that the setting that makes the difference for this issue is CONTROL_AE_TARGET_FPS_RANGE, see [1] for documentation, and [2] for the code site where Chromium sets this. 

In the above example, Chromium sets this control to a value range 15-60, which looks reasonable. When commenting out that line of code, i.e. not applying this setting at all, the issue disappears.

I'll see if we can get someone from Android camera or Pixel teams to look at this and check if this is an issue on their end.

[1] https://developer.android.com/reference/android/hardware/camera2/CaptureRequest
[2] https://cs.chromium.org/chromium/src/media/capture/video/android/java/src/org/chromium/media/VideoCaptureCamera2.java?q=VideoCaptureCamera2&dr=CSs&l=1088
Project Member

Comment 16 by bugdroid1@chromium.org, Jan 16 (6 days ago)

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/eb2ad611dbc0f7b0505fff3f9954dc832a017be9

commit eb2ad611dbc0f7b0505fff3f9954dc832a017be9
Author: Christian Fremerey <chfremer@chromium.org>
Date: Wed Jan 16 21:44:32 2019

[Video Capture, Android] Do not set CONTROL_AE_TARGET_FPS_RANGE on Pixel 3

This is a workaround for an issue on Pixel 3 phones, where
setting this control causes video to be overexposed.

Bug: 913203
Change-Id: I3e9b8f7a12075239f75c4f31552faa4a1ffb76fe
Reviewed-on: https://chromium-review.googlesource.com/c/1412017
Reviewed-by: Miguel Casas <mcasas@chromium.org>
Commit-Queue: Christian Fremerey <chfremer@chromium.org>
Cr-Commit-Position: refs/heads/master@{#623380}
[modify] https://crrev.com/eb2ad611dbc0f7b0505fff3f9954dc832a017be9/media/capture/video/android/java/src/org/chromium/media/VideoCaptureCamera2.java

Comment 17 by chfremer@google.com, Jan 17 (5 days ago)

Labels: Merge-Request-72
Requesting merge of #16 to M72 beta.
I verified on today's Canary that the workaround is effective.
Project Member

Comment 18 by sheriffbot@chromium.org, Jan 17 (5 days ago)

Labels: -Merge-Request-72 Merge-Review-72 Hotlist-Merge-Review
This bug requires manual review: We are only 11 days from stable.
Please contact the milestone owner if you have questions.
Owners: govind@(Android), kariahda@(iOS), djmm@(ChromeOS), abdulsyed@(Desktop)

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot

Comment 19 by gov...@chromium.org, Jan 17 (5 days ago)

Before we approve merge to M72, please answer followings:
* Is this M72 regression? Is it critical? (This is reported on M71 so not M72 regression)
* Is the change well baked/verified in Canary, having enough automation tests coverage and safe to merge to M72
* Any other important details to justify the merge.

Please note M72 is going to stable soon, so merge bar is very high. Thank you.

Comment 20 by chfremer@google.com, Jan 17 (5 days ago)

re #19: 
* Is this M72 regression? 
No. It is a workaround for an issue with a relatively newly released product, i.e. Pixel 3.

* Is it critical? 
See b/122484329 #10

* Is the change well baked/verified in Canary, having enough automation tests coverage and safe to merge to M72? 
I verified on today's Canary build that the workaround is effective and does not cause regression. The general feature of video and image capture is covered with automated testing. Since the only functional change is to not set an optional setting, this is a very safe change.

Comment 21 by gov...@chromium.org, Jan 17 (5 days ago)

Labels: -Merge-Review-72 Merge-Approved-72
NextAction: 2019-01-18
Approving merge to M72 branch 3626 based on comment #20. Pls merge tomorrow morning if change continue to look good in canary. Thank you.

Comment 22 by monor...@bugs.chromium.org, Jan 18 (4 days ago)

The NextAction date has arrived: 2019-01-18
Project Member

Comment 23 by bugdroid1@chromium.org, Jan 18 (4 days ago)

Labels: -merge-approved-72 merge-merged-3626
The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/4547f7c932955528e3327390b884eb032ee5583c

commit 4547f7c932955528e3327390b884eb032ee5583c
Author: Christian Fremerey <chfremer@chromium.org>
Date: Fri Jan 18 19:54:40 2019

[Video Capture, Android] Do not set CONTROL_AE_TARGET_FPS_RANGE on Pixel 3

This is a workaround for an issue on Pixel 3 phones, where
setting this control causes video to be overexposed.

Bug: 913203
Change-Id: I3e9b8f7a12075239f75c4f31552faa4a1ffb76fe
Reviewed-on: https://chromium-review.googlesource.com/c/1412017
Reviewed-by: Miguel Casas <mcasas@chromium.org>
Commit-Queue: Christian Fremerey <chfremer@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#623380}(cherry picked from commit eb2ad611dbc0f7b0505fff3f9954dc832a017be9)
Reviewed-on: https://chromium-review.googlesource.com/c/1422880
Reviewed-by: Christian Fremerey <chfremer@chromium.org>
Cr-Commit-Position: refs/branch-heads/3626@{#734}
Cr-Branched-From: d897fb137fbaaa9355c0c93124cc048824eb1e65-refs/heads/master@{#612437}
[modify] https://crrev.com/4547f7c932955528e3327390b884eb032ee5583c/media/capture/video/android/java/src/org/chromium/media/VideoCaptureCamera2.java

Project Member

Comment 24 by cr-audit...@appspot.gserviceaccount.com, Jan 18 (4 days ago)

Labels: Merge-Merged-72-3626
The following revision refers to this bug: 
https://chromium.googlesource.com/chromium/src.git/+/4547f7c932955528e3327390b884eb032ee5583c

Commit: 4547f7c932955528e3327390b884eb032ee5583c
Author: chfremer@chromium.org
Commiter: chfremer@chromium.org
Date: 2019-01-18 19:54:40 +0000 UTC

[Video Capture, Android] Do not set CONTROL_AE_TARGET_FPS_RANGE on Pixel 3

This is a workaround for an issue on Pixel 3 phones, where
setting this control causes video to be overexposed.

Bug: 913203
Change-Id: I3e9b8f7a12075239f75c4f31552faa4a1ffb76fe
Reviewed-on: https://chromium-review.googlesource.com/c/1412017
Reviewed-by: Miguel Casas <mcasas@chromium.org>
Commit-Queue: Christian Fremerey <chfremer@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#623380}(cherry picked from commit eb2ad611dbc0f7b0505fff3f9954dc832a017be9)
Reviewed-on: https://chromium-review.googlesource.com/c/1422880
Reviewed-by: Christian Fremerey <chfremer@chromium.org>
Cr-Commit-Position: refs/branch-heads/3626@{#734}
Cr-Branched-From: d897fb137fbaaa9355c0c93124cc048824eb1e65-refs/heads/master@{#612437}

Sign in to add a comment