Missing audio data with getUserMedia
Reported by
nick.lam...@gmail.com,
Aug 13 2016
|
||||||||||||||||||||||||
Issue descriptionUserAgent: Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36 Example URL: https://gist.github.com/Nlammertyn/c7194a7099d9fb243ee0331fb7b0d570 Steps to reproduce the problem: 1. Record long webm files using getUserMedia and MediaRecorder What is the expected behavior? Clean and consistent audio What went wrong? We're developing an electron application (desktop nodejs in conjunction with chromium) that does webcam recording. We've noticed corruption of audio during long recordings (1+ hours) from time to time. After testing and trying out a myriad of things for almost two months we still can't figure out what might be causing this. There are no errors and corruption starts seemingly at random (might be 5 minutes into a recording, sometimes only after 6 hours) The reproduction url has the relevant code without any extraneous parts. Looking and listening to the problem it's clear that audio data is missing in the resulting blob. Normal audio is alternated with silence. The duration of these silent (empty) parts is somewhere between 125 and 150ms. It feels like an issue with buffers but it's hard to pinpoint. Testing was done on numerous machines including older i3's and latest generation i5's with both microsoft lifecams and logitech c920's. Both can cause the corruption to occur. Initially we thought it was cpu related because it only showed up on laptops, but after debugging and optimizing we got cpu usage down a lot (45-55% peak total cpu, 25-30% chromium process), so there's a lot of headroom. Reducing cpu usage did seem to make it occur somewhat less frequently. We've been trying a bunch of workarounds, the latest was to start the mediarecorder with a chunksize of 30 seconds and every 5 minutes stop the mediarecorder and stream, create a new getUserMedia stream and MediaRecorder, start that again. (destroy everything and recreate it essentially) When this 'reset' is done it does recover, but not always and the audio corruption sets in again eventually. Attached are waveform pictures of background noise and voice during long recordings, both normal and corrupted versions. Also mp3's of the actual audio. I had to record these using audacity's wasapi (loopback) as when extracting the audio from the webm files results in the silences being removed and the actual audio data pushed together, so the result is a file that is shorter than the actual duration. Did this work before? N/A Is it a problem with Flash or HTML5? HTML5 Does this work in other browsers? N/A Chrome version: 52.0.2743.82 Channel: stable OS Version: 10 Flash Version: Shockwave Flash 22.0 r0 Any help or insight would be greatly appreciated. I'm of course available to try and test anything.
,
Aug 17 2016
,
Aug 17 2016
[triage]: qiangchen/niklase: can you have a look?
,
Aug 17 2016
We'll take a look, thanks for reporting this. Can you also attach the webm files for a corrupt recording? It's worth checking if there's discontinuous timestamps in the resulting files. Attach a drive/dropbox link if the files are too large for the bug tracker.
,
Aug 17 2016
Thanks for checking in so soon niklase, here are a few webm files (on s3), I've got a lot of them so if you need anything else, do let me know! Subsequent files, not sure where it starts exactly: https://vidtext-temp.s3.amazonaws.com/1-1.webm https://vidtext-temp.s3.amazonaws.com/1-2.webm A few other ones: https://vidtext-temp.s3.amazonaws.com/2.webm https://vidtext-temp.s3.amazonaws.com/3.webm In this one it starts when the light is turned on, so this might indicate something going wrong when a cpu spike is encountered: https://vidtext-temp.s3.amazonaws.com/light-on-glitch.webm Thanks! Nick
,
Aug 24 2016
,
Aug 24 2016
(meant to add a component tag, not replace it)
,
Aug 24 2016
mcasas@, I took a look at one of the files with problems (light-on-glitch.webm). What I see is that when the audio distortion occurs the audio track timestamps start to increase with a different rate than before. Before the distortion they are updated with 60 on average, but during the distortion the increase with 80. What is the value supposed to be?
,
Aug 24 2016
Looking at audio_track_recorder 60 is the correct value.
,
Aug 29 2016
grunell@, For this bug it seems like the audio track data start coming in with a slower rate, and the audio buffer timestamp gets much noiser as well. Any ideas how we can check for glitches on the recording side?
,
Aug 30 2016
Glitches on the browser/render process border can be seen in the log: https://cs.chromium.org/chromium/src/content/browser/renderer_host/media/audio_input_sync_writer.cc?sq=package:chromium&l=115 There's no data for "OS glitches" as we have for Mac. If you enable WebRTC debug recordings, there will be the AEC recordings, as well as the mic input captured close to the OS (in AudioInputController).
,
Aug 30 2016
Good point, I didn't realize that would work without a peerConnection. Nick, can you repro this with AEC recording active? - Open a separate tab with chrome://webrtc-internals/ - Expand "Create Dump" - Check "Enable diagnostic audio recordings" and choose a file location. You'll get 2 files, one of them is a wav file. It would be very interesting to see if the glitches are there as well.
,
Aug 31 2016
Hi niklase, I've compiled a zip for you to review: https://vidtext-temp.s3.amazonaws.com/debug%20run%203.zip So after doing a few tests the audio corruption showed on them all, I've picked the smallest one to show. It also contains the diagnostic wav file. First of I learned that the silent gap isn't necessarily 120-150ms. It is in the transcoded files, but I presume ffmpeg uses a different buffer size and thus the silent parts don't match exactly. In the webm files the silent parts are much smaller but also more frequent. The recording is around 15 minutes long. The zip contains .wav files for the portion between 5 and 10 minutes (the 2nd webm file more or less) of both the webm and the transcoded part. Also waveform images of a small part to show the difference in length and frequency of the silent/missing parts. From what I can gather the diagnostic wav file is clean, which also shows on the waveform image (the waveform images are all with the same zoom factor so the x-axis is the same for all). While recording it was playing brown noise through the speakers, but if necessary I can redo a test with some other source, music or voices if that would make it clearer. Let me know if there's anything else I can do Kind regards Nick
,
Aug 31 2016
Thanks. Sorry that I have to ask for more, but can you redo the test when playing a sine tone in the room? What I suspect is happening is that 10ms blocks are missing from the microphone side, but that won't be audible in the wav file since all blocks will be concatenated - and for pure noise you can't hear the discontinuities. For a sine tone it will definitely be audible. You're right that different players handles the audio drift differently. They all play the audio continuously until the timestamp error is too large, but the threshold seems to vary between players.
,
Sep 1 2016
Hi niklase, With pleasure, here's a short test with a sine wave: https://vidtext-temp.s3.amazonaws.com/sine%20debug%201.zip What I noticed was that the length of the recorded wasapi (what's coming out of the speakers) files didn't correspond to the raw wav. When I align the waveforms so that some clear sounds are lined up the recorded files are longer. This made me think (with your explanation about timestamping) that the silent parts could have been inserted while the actual audio would still be there. Next I imported the webm and transcoded mp4 files directly into audacity, as before the silent parts don't get imported so all the actual data is concatenated. This resulted in identical (more or less) waveforms and they sound normal. I do have to say that I tried it with older recordings with voices and doing the same, importing the webm, results in continuous audio, but there's still some corruption audible and it sounds sped up, so audio data missing probably, an example is attached as well. I've put in the waveform images and .wav's of all the files. We're waiting to launch into production because of the bug, so anything I can do, just let me know! Kind regards Nick
,
Sep 2 2016
Right, so Audacity just decodes and concatenates the data without looking at the timestamp (no video to align to), and then you end up with a much shorter clip, which sounds speeded up for voice. This means that the timestamps are correct but that we lose data before encoding. Grunell, can you take a look at this? Since MediaRecorder use 60ms block for Opus the audio encode time could be higher than what's normally the case for WebRTC. Could that lead to dropped audio frames?
,
Sep 5 2016
On the input side, there is extra buffering when doing IPC to hold up to roughly 1 s of audio if 10 ms buffers are used over IPC, if needed. If buffered, it will catch up when next buffer with audio comes. So if data is dropped, the system must be under heavy load which doesn't seem to be the case here. A few consecutive 60 ms encodings that take too long is not a problem.
,
Sep 5 2016
To chime in, could it be possible that a spike of cpu usage might cause the situation you mentioned above? We've seen recordings where the audio corruption starts at the same time when the video image degrades slightly as well (becomes more blurry, less sharp). It seems from what I can gather that this might be a dynamic scaling of the video quality when the cpu can't handle it at the default encoding quality. If so then the audio corruption is probably certainly related to cpu usage.
,
Sep 6 2016
In terms of dropped data at the process boundary, yes it could be caused by a CPU usage spike.
,
Sep 6 2016
OK, I think we need to repro this to make more progress. Over to chfremer to test this on a windows laptop.
,
Sep 8 2016
Just a quick status report on this. I have been trying to reproduce this using the following setup: * Dell Latitude E7440 * Windows 8.1 * Using the built-in camera * Chromium build from tip of tree While trying to do long recordings, I ran into the same issue [1] as Nick with Blobs not being freed and being unable to record anything past 512MB of data. I got recordings with a little below 30 minutes duration, but did not observe the audio dropout issue. I already have a small node.js server for writing incoming data chunks to a local file. With the workaround for the Blobs issue described in [1], I should be able to do longer recordings, and then hopefully be able to reproduce the issue. If it does not work with the setup described above, I may need to try something closer to what Nick has been using. [1] https://bugs.chromium.org/p/chromium/issues/detail?id=468718
,
Sep 9 2016
,
Sep 9 2016
@chfremer Yeah, the e.data.close() after you've transmitted to a server should definitely solve the 512MB limit. I've compared cpu's, yours has a passmark score of 3741, the one we're using for testing where we can reproduce it often has 2456 (https://www.cpubenchmark.net/cpu.php?cpu=Intel+Core+i3-3227U+%40+1.90GHz) (also doesn't has clock speed bursting) I test remotely using teamviewer which takes up around 15% of the cpu, so we're really starving the cpu while testing (we have seen the problem without teamviewer as well). Perhaps running some cpu benchmark software on the same cpu core might strain the cpu enough to uncover the bug faster on your machine? Let me know if I can help in any way Nick
,
Sep 13 2016
I still had no success in getting this reproduced. I have started testing on a (beefy) Dell Precision M3800 Laptop running Windows 10, Chrome 52.0.2743.116 64-bit and a Logitech C930 Webcam connected via USB. I let a program run in the background to get CPU and Memory usage to 100%, but was not able to provoke it so far. However, after having analyzed your example files one more time, I have developed an interesting theory that may be worth testing. Since the audio with the silence parts removed sound clean in the sine, and since it matches up with the diagnostic audio captured close to the OS, it may be worth considering that the issue is not with the audio but with the video. Could it be that under some circumstances (e.g. high load) the camera driver decides to reduce the frame rate from 24 to 18? If the camera driver delivers frames at 18fps while the capture code still assumes we are getting them at 24fps, this would explain the factor 8/6 of mismatch we are seeing in the timestamps between audio and video exactly. To test this we could do the following. Put a clock in front of the camera while doing another capture. Then let us have a look at the video when the issue happens. Just a wild theory at this point, but may be worth trying.
,
Sep 14 2016
I was finally able to provoke the issue, even though I still don't have a repro that triggers the issue 100 % of the time. My setup is as follows: * Dell Precision M3800 Laptop running Windows 10 * Local Debug build of Chromium from tip of tree * Logitech C930 Webcam via USB To provoke the issue I am covering the camera before using it in Chromium in order to make the video content appear completely black. This makes the camera reduce the frame rate at which it captures. Then I let it record for about 20 seconds, and then I uncover the camera. The frame rate increases, and _sometimes_ the audio dropouts start appearing shortly after. So far I was unable to make this happen on the Chrome 52.0.2743.116 64-bit release build. Maybe the "slowness" of the debug build helps here. I produced a log file of a recording where the issue appears that has timestamps of arriving video frames and audio packets, and I am analyzing it now to see if it can help narrow down the root cause. So far I am seeing the same symptoms as in Nick's files, with encoded audio packets changing their temporal spacing from 60ms to 80ms, while the duration of each packet stays constant at 60ms.
,
Sep 15 2016
Here is what I found so far. I followed the audio buffers all the way down to where they are delivered to us from WASAPI in WASAPIAudioInputStream::Run() [1]. When everything is normal, WASAPI delivers 480 frames on every buffer, every 10ms. As soon as the issue starts, on each third buffer, WASAPI delivers only 96 frames instead of 480. This means that for every 30ms of time, we get only 22ms worth of audio samples. What exactly causes WASAPI to get into that state is currently not clear to me. It seems to correlate with load on the system, which is probably why I am able to provoke it the way I did. Without deeper analysis I am unable to tell whether there is a bug in our usage of WASAPI or in the WASAPI implementation itself. I think it would make sense to have someone from the audio team have a closer look at this. [1] https://cs.chromium.org/chromium/src/media/audio/win/audio_low_latency_input_win.cc?q=WASAPI&sq=package:chromium&l=296
,
Sep 15 2016
grunell, back to you to comment on this finding.
,
Sep 16 2016
Interesting. Nothings rings a bell though. * Can this be reproduced with a WebRTC call (e.g. with AppRTC) or something else using getUserMedia? Or only with MediaRecorder? * Are you certain about the correlation with system load? * Does it happen with Windows 7? * Do you have a test page I can try to repro with?
,
Sep 16 2016
* Can this be reproduced with a WebRTC call (e.g. with AppRTC) or something else using getUserMedia? Or only with MediaRecorder? I am able to reproduce this without MediaRecorder. My latest repro uses a html video element to play back a live stream obtained via getUserMedia(). * Are you certain about the correlation with system load? Well, based on my attempts to repro, I found the following: + On my test machine, I can provoke the issue with high success rate on a debug build of Chromium from the tip of tree. To provoke, I have the test website send a large array of data to a local node.js server. While sending the data, the website typically freezes. Chance of success appears to be higher when the JavaScript console is open in Chromium. + Issue appears with both a Logitech C930 USB webcam and the internal webcam from the Laptop. + I am unable to provoke the issue on Chrome 52.0.2743.116 64-bit release build. + I am currently building a local release build from tip of tree to test whether release/debug makes the difference or the Chrome version. + Issue seems to repro with higher success rate when system is under load (e.g. compiling Chromium in the background) * Does it happen with Windows 7? I currently don't have a Windows 7 machine to test. My coworker has a Win8.1 Laptop that I will try to repro on. If needed, I can try to find a Windows 7 machine as well. * Do you have a test page I can try to repro with? Yes, uploaded to [1]. Here are the steps that work for me: + Run server4.js in node.js on the local machine. + In the browser under test, open localhost:3000 to get to the test page. + Use headphones for audio playback to avoid a feedback loop + Click the Unmute Audio button to hear the live audio from the camera + Click the Send Dummy Data button to send a large array to the server, which should put the browser under load. Note, that there may be easier ways to provoke the issue, but this is the simplest one I have so far. [1] https://drive.google.com/drive/folders/0B5qdfn5hLMy1SHA1RURhMmUtX1U?usp=sharing
,
Sep 16 2016
Quick followup on debug vs. release build. I was not able to provoke the issue on a local release build from tip of tree. This suggests that the reason it did not repro for me on Chrome 52.0.2743.116 64-bit release build is due to the fact that it is a release build (as opposed to that it is an older version).
,
Sep 19 2016
I tried to repro on my (powerful) Windows 7 desktop with the steps in comment #29, but couldn't. If the number of frames in comment #26 refers to "num_frames_to_read" in WASAPIAudioInputStream::Run(), I can't repro that either. (I.e. it's always 441.) I also loaded it massively with a Prime95 torture test on all cores, still always 441. Several missed IPC deadlines on playout side though, and occasional choppiness, probably due to that. That's expected though under that load. This issue should happen under moderate load according to the report though. Any other suggestions for repro'ing?
,
Sep 19 2016
Perhaps limit the chrome process to 1-2 cores with core affinity and run prime95 with the same core affinities? I have to admit I haven't been able to reproduce it myself on my personal desktop (w8.1) with stress testing. The common factor is either windows 10 or laptops in general.
,
Sep 19 2016
#31: Yes, the number of frames refers to "num_frames_to_read". I am surprised that you are getting 441 frames, since the test script is requesting audio at 48kHz. Could it be that your audio device does not support 48kHz? Not sure if that makes a difference for the repro though. Did you try to repro on a debug or release build? So far, I have only been able to repro on a debug build running on Windows 10. If I understand correctly, Nick is seeing the issue also only on Windows 10 so far, but probably using release builds. I will try on a few other machines today and let you know what I find.
,
Sep 19 2016
Tested on two more machines.
1.) Beefy Windows 10 desktop workstation with Core i7 6700 CPU and 32GB RAM.
I was able to provoke the issue on the Debug build.
2.) Dell Latitude E7440 running Windows 8.1.
Unable to provoke the issue on the same Debug build.
Since grunell@ did not see the issue on Windows 7, it could be that the issue only appears on Windows 10.
grunell@, do you have a Windows 10 machine where you could try to repro one more time?
,
Sep 21 2016
I tried to repro on a 4 core Win 10 laptop, but failed there too. It's a debug build. Note that if you run a WebRTC loopback test as in comment #29, and hear glitches, it can be due to something else than the originally observed problem. (E.g. it could happen on playout.) I will run some longer tests, but will not have time until next week. Does the problem persist if another device is used? Long shot - maybe test to change the device sample rate?
,
Sep 26 2016
grunell@: Please let me know if there is anything I can do to help narrow things down further. I agree that audible glitches alone are no guarantee that we are seeing the originally observed problem. On the Windows 10 Laptop where I tested on, I confirmed the issue by looking at logs produced in WASAPIAudioInputStream::Run(). On the Windows 10 Desktop, I did not look at logs, but am fairly confident that the issue is the same, because it results in a similar type of glitches, i.e. audio drops at a fixed period and it does not recover. Since I currently suspect some issue with WASAPI, my best idea for a next step to narrow it down further would be build a small test app exercising the WASAPI code in isolation and seeing if we can somehow provoke the issue. Please let me know if you think this would be reasonable to try. I am also happy to help with testing any alternative ideas.
,
Oct 6 2016
Changing summary based on comment #29. Widening set of components, since it affects several parts and the root cause isn't found yet.
,
Nov 16 2016
Hi guys, Any updates on this? Thanks! Nick
,
Nov 17 2016
I haven't tried to repro again. chfremer@ has not been able to repro with a release build, and a debug build is not to trust since it adds a lot more load on the CPU which can cause audio problem. The original report stats that there quite a lot of CPU headroom. I'll have to get back to thinking about how we can move forward on this.
,
Jan 18 2017
,
Jan 18 2017
Bulk move Blink>MediaStream>Recording ---> Blink>MediaRecording
,
Feb 22 2017
grunell@, chfremer@, any new information on this bug?
,
Mar 3 2017
No, I haven't had the time to look more at this.
,
Mar 18 2017
,
Jun 14 2017
grunell@, chfremer@ – I thought I'd chime in with some additional information about this that we've seen in a deployment of an internal tool that makes use of getUserMedia. Unfortunately we haven't succeeded at a reliably reproducing this issue either, but some of what we've learned might be of interest to you. In our case, we're using WebRTC to manage a live audio call, and we're recording the participant on each end by piping the the data from a ScriptProcessor's onaudioprocess calls into a client-side audio encoder. We've observed this issue sporadically for the past year. While it doesn't happen reliably, it seems to affect *some* users repeatedly, and others never. Our deployment involves users running Chrome on a number of different operating systems, including Mac, Windows and Chrome OS, and we've observed observed the issue across all operating systems. It seems vaguely correlated with less powerful machines (as reported above), and as a result it seems like we've seen it with slightly more frequency on less powerful Chromebooks. That being said, we've never succeeded in duplicating it by trying to compete for processor cycles or anything of that nature. As part of our work to diagnose & mitigate this issue, we have tracked the time delta between sequential pairs of onaudioprocess calls. Our audio buffer is configured to be 372ms long, and typically those calls occur within a few milliseconds of this value. When audio gets dropped, however, the onaudioprocess call is *late* by the amount of missing audio. Because we're recording two sides of a conversation, and because our recordings stop and start simultaneously, we know that these two recordings should be almost identical in length. In cases where one participant is experiencing the audio loss issue, we've found that adding back small snippets of silence where we noted late onaudioprocess calls (each of a duration corresponding to how late the call was) restores the duration of their recording to the precise expected duration.
,
Jun 15 2017
douggerhardt@: Thanks a lot for the input! This could be a different problem than what is reported in this bug, although the symptoms are similar. Would you mind filing a separate bug? If we later determine it's the same it can be duped.
,
Jun 17 2017
grunell@: Sure thing. I've filed it separately (see 734290); my hunch, reading the above, is that we're experiencing the same issue in both cases.
,
Jun 19 2017
,
Jun 19 2017
The original report has been tracked to too few frames in Chrome's WASAPI interfacing implementation, see comment #26. Since comment #45 (now issue 734290 ) is seen on other platforms too, I suspect they are different. The new report could for example be a problem with WebAudio.
,
Jun 19 2017
This issue (the original report) could possibly be the same as or related to https://bugs.chromium.org/p/webrtc/issues/detail?id=5775.
,
Sep 29 2017
Hi, we encountered the same or very similar issue and found a bug in WASAPIAudioInputStream::Run(). The current Microsoft example and a msdn blog post recommends draining the buffers (links below). Kind of hard to reproduce, we have one machine where on the Logitech G430 Gaming Headset if we set the output to 44.1 kHz we reliably saw (after 9 minutes and 30 seconds of streaming): 441 frames, 441 frames, 88 frames, 441 frames, 441 frames, 88 frames The buffer with the 88 frames would also signal a discontinuity. The patch below fixed this, and we still see the single discontinuity after 9 minutes and 30 seconds (I assume we are being pre-empted by something), but then we recover after draining the buffer. Sorry for the ugly debug log lines, basically if there is no line "NO MORE AUDIO" we drain the buffer. [25548:28504:0927/111111.628:INFO:audio_low_latency_input_win.cc(477)] Microphone (Logitech G430 Gaming Headset) GOT frames: 441 [25548:28504:0927/111111.628:INFO:audio_low_latency_input_win.cc(447)] Microphone (Logitech G430 Gaming Headset) NO MORE AUDIO [25548:28504:0927/111111.638:INFO:audio_low_latency_input_win.cc(477)] Microphone (Logitech G430 Gaming Headset) GOT frames: 441 [25548:28504:0927/111111.638:INFO:audio_low_latency_input_win.cc(447)] Microphone (Logitech G430 Gaming Headset) NO MORE AUDIO [25548:28504:0927/111111.647:INFO:audio_low_latency_input_win.cc(477)] Microphone (Logitech G430 Gaming Headset) GOT frames: 441 [25548:28504:0927/111111.668:INFO:audio_low_latency_input_win.cc(477)] Microphone (Logitech G430 Gaming Headset) GOT frames: 441 [25548:28504:0927/111111.668:INFO:audio_low_latency_input_win.cc(474)] Microphone (Logitech G430 Gaming Headset) GOT frames: 88 with DISCONTINUITY [25548:28504:0927/111111.668:INFO:audio_low_latency_input_win.cc(447)] Microphone (Logitech G430 Gaming Headset) NO MORE AUDIO [25548:28504:0927/111111.668:INFO:audio_low_latency_input_win.cc(447)] Microphone (Logitech G430 Gaming Headset) NO MORE AUDIO [25548:28504:0927/111111.677:INFO:audio_low_latency_input_win.cc(477)] Microphone (Logitech G430 Gaming Headset) GOT frames: 441 [25548:28504:0927/111111.677:INFO:audio_low_latency_input_win.cc(447)] Microphone (Logitech G430 Gaming Headset) NO MORE AUDIO [25548:28504:0927/111111.687:INFO:audio_low_latency_input_win.cc(477)] Microphone (Logitech G430 Gaming Headset) GOT frames: 441 [25548:28504:0927/111111.687:INFO:audio_low_latency_input_win.cc(447)] Microphone (Logitech G430 Gaming Headset) NO MORE AUDIO [25548:28504:0927/111111.697:INFO:audio_low_latency_input_win.cc(477)] Microphone (Logitech G430 Gaming Headset) GOT frames: 441 https://blogs.msdn.microsoft.com/matthew_van_eerde/2014/11/05/draining-the-wasapi-capture-buffer-fully/ https://github.com/Microsoft/Windows-universal-samples/blob/master/Samples/WindowsAudioSession/cpp/WASAPICapture.cpp diff --git a/media/audio/win/audio_low_latency_input_win.cc b/media/audio/win/audio_low_latency_input_win.cc index 60f4865c30aa..ff2092919859 100644 --- a/media/audio/win/audio_low_latency_input_win.cc +++ b/media/audio/win/audio_low_latency_input_win.cc @@ -396,6 +425,37 @@ void WASAPIAudioInputStream::Run() { break; case WAIT_OBJECT_0 + 1: { TRACE_EVENT0("audio", "WASAPIAudioInputStream::Run_0"); + + + // Quote from Microsofts WindowsAudioSession Sample: + // + // A word on why we have a loop here; + // Suppose it has been 10 milliseconds or so since the last time + // this routine was invoked, and that we're capturing 48000 samples per second. + // + // The audio engine can be reasonably expected to have accumulated about that much + // audio data - that is, about 480 samples. + // + // However, the audio engine is free to accumulate this in various ways: + // a. as a single packet of 480 samples, OR + // b. as a packet of 80 samples plus a packet of 400 samples, OR + // c. as 48 packets of 10 samples each. + // + // In particular, there is no guarantee that this routine will be + // run once for each packet. + // + // So every time this routine runs, we need to read ALL the packets + // that are now available; + // + // We do this by calling IAudioCaptureClient::GetNextPacketSize + // over and over again until it indicates there are no more packets remaining. + // + // In our case we loop / wait - but the same applies - there are many + // devices which occasionally have extra data and they don't handle it + // gracefully if we don't read all data, specifically he have seen the + // Logitech G430 go into a mode where it never recovers + + while (true) { // |audio_samples_ready_event_| has been set. BYTE* data_ptr = NULL; UINT32 num_frames_to_read = 0; @@ -409,10 +469,15 @@ void WASAPIAudioInputStream::Run() { hr = audio_capture_client_->GetBuffer(&data_ptr, &num_frames_to_read, &flags, &device_position, &first_audio_frame_timestamp); + if (hr == AUDCLNT_S_BUFFER_EMPTY) { + // LOG(INFO) << friendly_name_ << " NO MORE AUDIO"; + break; + } if (FAILED(hr)) { @@ -480,6 +552,7 @@ void WASAPIAudioInputStream::Run() { delay_frames = 0; } } + } } break; default: error = true;
,
Oct 2 2017
Re #51: thanks for the pointer. I agree that it looks like it could be the issue, I'll try to allocate time soon to have a closer look at this.
,
Oct 16 2017
+Marina FYI
,
Nov 23 2017
Any update?
,
Nov 24 2017
No work done on this yet.
,
Nov 24 2017
@grunell any chance for a Chrome/Chromium build for Windows 7 with the patch in #51? This will help me test the patch on our system.
,
Nov 28 2017
pablo.platt: If you have the possibility, you could build Chromium yourself and test with that patch.
,
Nov 28 2017
@grunell Chromium master build is broken with recent VS-2017. I had to install VS-2017 version 15.3.5. @florian do you think we need the same fix for video input from camera as well?
,
Jan 17 2018
Any update? This was reported 1.5 years ago and a possible patch was submitted 4 months ago.
,
Jan 23 2018
I haven't been able to work on this yet. Feel free to upload a patch for review.
,
Feb 23 2018
,
Feb 23 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/9aecd396345fd6c629d58ccfbb9c866e3b129ec1 commit 9aecd396345fd6c629d58ccfbb9c866e3b129ec1 Author: Henrik Grunell <grunell@chromium.org> Date: Fri Feb 23 19:31:21 2018 Drain input audio device buffer completely on Windows. * Move the code pulling and pushing data to its own function. * Add a loop that runs until the audio client reports empty buffer. * Remove unnecessary GetVolume() call; it's overwritten by GetAgcVolume() call later. Bug: 637558 Cq-Include-Trybots: master.tryserver.chromium.android:android_optional_gpu_tests_rel;master.tryserver.chromium.linux:linux_optional_gpu_tests_rel;master.tryserver.chromium.mac:mac_optional_gpu_tests_rel;master.tryserver.chromium.win:win_optional_gpu_tests_rel Change-Id: Ie06d862592143d76508a7d4d709efd42c83a1d9c Reviewed-on: https://chromium-review.googlesource.com/931554 Commit-Queue: Henrik Grunell <grunell@chromium.org> Reviewed-by: Dale Curtis <dalecurtis@chromium.org> Cr-Commit-Position: refs/heads/master@{#538859} [modify] https://crrev.com/9aecd396345fd6c629d58ccfbb9c866e3b129ec1/media/audio/win/audio_low_latency_input_win.cc [modify] https://crrev.com/9aecd396345fd6c629d58ccfbb9c866e3b129ec1/media/audio/win/audio_low_latency_input_win.h
,
Mar 5 2018
florian.p.nierhaus and pablo.platt, could you test the latest Chrome Canary or Dev and see if the change in #62 fixes your problems?
,
Mar 9 2018
,
Mar 14 2018
nick.lammertyn: could you also test Chrome Canary or Dev to see if this fixes your problem?
,
Mar 14 2018
,
Mar 20 2018
I'm closing this. The original report and the the report in comment #51 are describing the same symptoms and the the fix in comment #62 is doing what was tested and verified to fix the problem (also #51). nick.lammertyn, florian.p.nierhaus and pablo.platt: could you still please verify that the problem is fixed? The fix is now in Beta.
,
Apr 9 2018
Hello - I had a slightly similar issue (https://bugs.chromium.org/p/chromium/issues/detail?id=768404) -- however it looks like with Canary 67 the issue I had been experiencing has gone away? Are the two relatable - if so, please address my case and you may wish to close it. Otherwise, it may be good to know which issue my case was related to and now seems fixed. Thanks
,
Apr 10 2018
Thanks for the report. Yes, that bug very much looks similar to this. (I'll update that bug with some more details.) In Chrome 66 this bug was fixed. Let's continue the discussion on your problem in that bug.
,
Apr 13 2018
Issue 768404 has been merged into this issue. |
||||||||||||||||||||||||
►
Sign in to add a comment |
||||||||||||||||||||||||
Comment 1 by yini...@chromium.org
, Aug 16 2016