Hello, do you know what API the TestNav app is using to record audio? Also, can you send feedback for this issue by pressing alt + shift + i and writing "Pearson TestNav 899272" into the comments?

Hello, 

We are using the older WebRTC API call getUserMedia to setup the connection to the microphone. We create a MediaStreamAudioSourceNode off our audio context with the returned media stream. We then capture PCM audio data via a ScriptProcessorNode created from the same audio context. Finally, we encode the raw audio data in a Web Worker using an emscripten compiled version of libopus.

This is not using the MediaRecorder: "we encode the raw audio data in a Web
Worker using an emscripten compiled version of libopus." (#2). It also uses
WebAudio

Correct, in production we are not using the MediaRecorder API. However, we are experimenting with that API. It would dramatically cut down on our complexity for recording/encoding/decoding.

Also we see the same issue with both navigator.getUserMedia & navigator.mediaDevices.getUserMedia. Not sure if that matters.

The issue is triggered by playing/pausing the audio data after it's been recorded/encoded & set in our audio player as a b64 string. If I rapidly click the play/pause button for about 30 seconds, it seems to severely degrade the performance the next time recording occurs with the script processor node.

I have attached a simple HTML page with no dependencies using the MediaRecorder API for recording & an <audio> tag for playing back the recording. The page can also be viewed at this url: https://codepen.io/arousset/pen/bQxBKK 

After about ~20 calls to audio.load() the next recording is cutoff. In the attached HTML file on lines 67-72 you can see how we've setup playback. 

I've tried only calling audio.load() once per recorded blob, but after enough successive recordings the issue starts to appear again. I've also tried different ways of setting the audio data in the <audio> tag using the src attribute with an object url from the blob and also a base64 data string. I've tried both of these approaches in a child <source> node as well. All of these different ways of setting the data show the same issue.

Also note, this has only been detected on ChromeOS in both the browser and our app. Chrome on any other platform is working fine.

Let me know if you have further questions, thanks!

Note: The steps to reproduce the issue are included in the sample page both at the codepen & in the attached file.

Deleted: cros-issue.html 3.7 KB

cros-issue.html 3.7 KB View Download

The comment #8 and the repro case provided by andrew.rousset@ suggest this is MediaRecorder and/or getUserMedia issue. The new repro case in #8 does not use Web Audio API at all.

To respond the comment #2:

> We then capture PCM audio data via a ScriptProcessorNode created from the same audio context. Finally, we encode the raw audio data in a Web Worker using an emscripten compiled version of libopus.

Have you tried AudioWorklet instead of ScriptProcessorNode? The SPN is a common cause of choppy audio. See more info here:
https://developers.google.com/web/updates/2018/06/audio-worklet-design-pattern

Yes, it's broken for both. I see the same issue with our Web Audio backed recorder & the MediaRecorder example I gave in the above comment using the same steps.

#8: bump " has only been detected on ChromeOS"

The thing that surprises me is that in the code/codepen in #8, play/stop
only plays back the Blob generated _after_ recording, i.e. mediaRecorder.onstop
has been called and a new Blob generated out of the recorded chunks and
put into audio.src, so: step 4 "Click Play/Stop repeatedly (10-20 times 
is usually enough)", should not affect MR at all (I'm the MR dev so I'm
looking at that part in particular). IOW MR only gets engaged when we
click "Record", not when clicking on "Play/Stop" (and to get back to "Record"
we have to click "Delete").  For further triaging we could try forgetting
about the play-stop cycle and just do a number of Record - Stop (then auto
playback the recorded chunks) cycles, and see if the cutoffs are present.

To me it looks like it is tied to loading the audio. In the simpler example we are explicitly telling the audio element to load every time we click play.

We deployed a potential fix recently that just let the browser load the audio by streaming it after calling play() on the audio element and skipping the load call altogether.

However, the issue still appears. So my guess would be that loading audio is affecting the open media stream.

Changing Components based on #13

mlamouri@ could you triage plz?

I'm not sure how loading the audio would affect media stream but maybe the videostack team would know. Definitely not a blink issue I would say.

Seems like part of the old audio isn't being flushed? Does this happen if 
you create a new mediastream+element each time you want to record?

We are in the process of testing a workaround that creates a new media stream & a new script processor node for capture from the stream each time we record.

In my own testing, that looks to have sidestepped the issue.

When is this issue going to get fixed?
 This is very critical for us, regardless of whether the workaround works, the original issue needs to be fixed.

+MediaStream folks since this seems like an issue deep in media streams.

There's no createStream() call, so removing the component 
Blink>MediaStream>CaptureFromElement

emircan@ seeing that you work on a lot of Blink>MediaStream issues, could you take a look at triaging this further? Thanks.