New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 878811 link

Starred by 6 users

Issue metadata

Status: Duplicate
Merged: issue 866390
Owner:
Closed: Oct 23
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 2
Type: Bug



Sign in to add a comment

captured audio quality is worse with getUserMedia() than with HTML MediaCapture

Project Member Reported by joelriley@google.com, Aug 29

Issue description

Chrome Version       : 68.0.3440.91
URLs (if applicable) :
OS version               : Android 9
Network (such as Cable/DSL/Dial up etc):
Audio/Video format (if applicable):
Special chrome flags (if applicable):

Behavior in Safari (if known): n/a
Behavior in Firefox (if known): n/a

Video issue, Audio issue, both, neither? Audio

Flash or HTML5?  HTML5

What steps will reproduce the problem?
(1) Capture video through getUserMedia/MediaRecorder in environment with loud audio
(2) Receive webm video Blob
(3) Play webm video (on device or VLC on mac) and audio quality is poor.

What is the expected result?
Audio quality on video is consistent with video captured through native device Camera.

What is the actual result?

Compared to a video captured using HTML media capture (<input accept="video/*" capture="camcorder"), the audio quality of the webm video captured with MediaRecorder is poor:

+ Sounds levels tend to randomly dip with mediarecorder
+ There is a 'warbliness' to the audio with mediarecorder, while native camera is more crisp and consistent
+ Volume is quieter than native camera

All these tests were on Pixel 1 with Chrome:
default codec: https://drive.google.com/open?id=1xZq1k9G2tk26NV_1c3lo7UI_bV2SjHEb
v8,pcm codec: https://drive.google.com/open?id=1vqp5YRUVPS6uNvs--AwpO00VnrQrHGYT
html media capture: https://drive.google.com/open?id=1-F8mI9XsX6E5ok6mDg-y5Pal8SeEQOvy

Constraint used for video capture:
{
  width: {
    ideal: 1280,
    max: 1920,
  },
  height: {
    ideal: 720,
    max: 1080,
  },
  frameRate: 30,
}

Audio constraint: {echoCancellation: false}

MediarRecorder Timeslice value: 5000
 
Cc: emir...@chromium.org mcasas@chromium.org
Components: -Blink>MediaRecording Blink>GetUserMedia>Mic
Summary: captured audio quality is worse with getUserMedia() than with HTML MediaCapture (was: Poor audio quality for videos captured via MediaRecorder API)
guidou@ can you triage this bug please?
 (note that when recording with PCM, there's no encoding happening).

Status: Untriaged (was: Unconfirmed)
Cc: olka@chromium.org
olka@: Can you take a look or help triage further?
Cc: dalecur...@chromium.org niklase@chromium.org
Labels: Performance-Media
joelriley@:

- For the getUserMedia/MediaRecorder case: if you then press the volume control on the Pixel 1, what volume is then affected? Do you see a Speaker icon (Media) or a Phone icon (Communication)? Can you compare with HTML.

dalecurtis@: are the getUserMedia/MediaRecorder client and the HTML media capture client using different audio input stacks in Chrome on Android? If not, one possible reason could be that the "HTML" case uses media mode and no built-in effects, while the getUserMedia/MediaRecorder uses communication mode and built-in effects (AEC, AGC etc.). AFAIK; using Audio constraint: {echoCancellation: false} will only disable the software based (WebRTC) AEC but not built-in AECs when available.

Status: ExternalDependency (was: Untriaged)
Owner: kwiberg@chromium.org
Why is it ExternalDependency?
re #5: During getUserMedia/MediaRecorder capture, volume control affects communication (I see phone icon). During HTML media capture, the volume controls are overridden and used as shutter buttons.
> Why is it ExternalDependency?

Because we were waiting for answers (comment 5). Am I misusing that status value?
Seems like performing audio capture in 'communication' mode may be the issue here. Is there a way to disable this with the API or reasonable to add the ability to disable when requesting a stream?
Labels: Needs-Feedback
Owner: joelriley@google.com
Status: Assigned (was: ExternalDependency)
Cc: kwiberg@chromium.org
(I think "needs feedback" is done like this, external dependency is usually if there is a problem in 3rd party code or something)
I think I gave feedback in comment #8. Is anything else needed from me?
Owner: olka@chromium.org
Owner: henrika@chromium.org
Henrik - could you PTAL?
Cc: braveyao@chromium.org solenberg@chromium.org
Labels: -Pri-2 Pri-3
I think I have a an understanding of what happens here but there is no simple solution to the problem.

joelriley@: if you can build Chrome for Android I could point to to some things you could try out to confirm my current theory.

The long story is that WebRTC in Chrome on Android is not up-to-date with the desktop version given that usage on Android is only a fraction of the usage on desktop platforms and I don't think we have resources at this stage to improve this area.


Cc: ossu@chromium.org
Did you try disabling the other two constraints as well? I.e. noiseSuppression and autoGainControl[1]? It definitely sounds like such processing could be involved in this case.

[1] https://www.w3.org/TR/mediacapture-streams/#media-track-constraints  
Cc: guidou@chromium.org
Now that I'm digging into it, I'm unsure of which constraints are implemented at which levels - there are some older, "goog" prefixed constraints as well. I'm going to CC guidou@ and hope he knows the answers to the following:

echoCancellation: false used to turn all processing off, but I'm unable to find where that happens. Are the standard noiseSuppression and autoGainControl constraints implemented? Are the old ones supported when specifying constraints the modern way?
Yes, noiseSuppression and autoGainControl are implemented.
If echoCancellation is set to off, noiseSuppression and autoGainControl are off by default too.
If echoCancellation is set to true or not set, noiseSuppression and autoGainControl are on by default.
Of course, it is possible to explicitly set noiseSuppression and autoGainControl to values different from echoCancellation.

It happens here: https://cs.chromium.org/chromium/src/content/renderer/media/stream/media_stream_constraints_util_audio.cc?q=media_stream_constraints_util_audio&sq=package:chromium&dr=C&l=479
One of the main issues here is that the said constraint does not affect the lower layer audio parts on Android which always tries to enable any available built-in AEC. Hence, software based components might be disabled, but the built-in parts (AEC, NS) are still enabled.
Labels: -Pri-3 M-72 Pri-2
Mergedinto: 866390
Status: Duplicate (was: Assigned)
Issue 897881 has been merged into this issue.

Sign in to add a comment