Simultaneous audio capture and speech recognition does not recognize speech |
|||||||
Issue descriptionUserAgent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36 Steps to reproduce the problem: 1. Go to https://jsfiddle.net/Lx51bz2t/5/ on an Android device. For example, LG G 4. Alternatively, go to https://jsfiddle.net/Lx51bz2t/6/ 2. Approve the audio capture permission. Essentially - // Capture audio. navigator.mediaDevices.getUserMedia({audio: true}).then( stream => { // Recognize speech. const speech = new webkitSpeechRecognition(); speech.continuous = true; // Restart the recognition as long as we record. speech.addEventListener("end", () => speech.start()); speech.start(); // Record stream. const recorder = new MediaRecorder(stream); recorder.start(); }); What is the expected behavior? Simultaneous audio capture, speech recognition and recording without beeps. What went wrong? LG G 4 plays rapid beeps repeatedly and does not transcribe almost at all. I once got "no no no no no no" somehow. Did this work before? N/A Does this work in other browsers? N/A Chrome version: 67.0.3396.99 Channel: stable OS Version: 6.1 (Windows 7, Windows Server 2008 R2) Flash Version: It works pretty well on desktop. No beeping.
,
Jul 23
phistuck@ I was told in the past that encoding (into Opus) would throw
off any speech recognition so, off the bat could you try instantiating
the MediaRecorder like:
const recorder = new MediaRecorder(stream, { mimeType: "audio/webm;codecs=pcm" });
and see if the recognition gets any better?
OTOH we should diagnose if the beeps are natural or introduced in the
capture or the recording parts. Do they show up if you just record ?
(I.e. minus the speech reco part).
,
Jul 23
#2 - no, this is definitely coming due to the speech recognition. My guess is that it beeps as an indication that speech recognition has started and because I also capture (or record, I have not diagnosed it yet, I was focused on making my main goal work which is recording the captured audio. Speech recognition was just a bonus), it immediately stops the recognition and because I keep restarting it when it stops, it just beeps again to indicate it starts again. In case PCM works, recording to PCM is not a great option, it will weigh a lot and I will have to compress it somehow anyway. On the other hand, worst case, I guess I can compress it later with web audio and media recorder. Still, on a mobile device, taking large amounts of memory (either on disk or not) is not a great way to roll.
,
Aug 13
,
Aug 13
phistuck@ we're in a bit of a conundrum: Opus might throw off the Speech recognition, and PCM would be too bandwidth-hungry :-/ Could you try PCM nonetheless and see if that improves the recognitinon plz?
,
Aug 13
,
Aug 13
MediaRecorder is redundant here (it may add up, but it is not necessary). It is capturing audio combined with speech recognition that creates the conflict. https://jsfiddle.net/Lx51bz2t/14/ // Capture audio. navigator.mediaDevices.getUserMedia({audio: true}).then( stream => { // Recognize speech. const speech = new webkitSpeechRecognition(); speech.continuous = true; // Restart the recognition as long as we record. speech.addEventListener("end", () => speech.start()); speech.start(); });
,
Aug 13
Speech recognition alone works fine - https://jsfiddle.net/Lx51bz2t/17/ I confirm the problem on LG G3, too.
,
Aug 13
Adjusting Components. guidou@ could you please route to the audio folks plz?
,
Aug 14
[Triage] Setting to assigned since it has an owner.
,
Aug 14
One more data point that sort of suggests that this combination is not greatly supported - on Windows 7, the tab sometimes crashes when I capture, recognize and record (might not be necessary) simultaneously (perhaps only when the Developer Tools feature is open? Not sure). Once I removed the speech recognition from the mix, it either never crashed, or crashed once in a very long while. Also, LG G3 software details - Android 6.0 Security patch level - 2016-09-01 Baseband - MPSS.DI.2.0.1.c1.13-00111-M8974AAAAANPZM-1.44479.1 Kernel - 3.4.0 Build - MRA58K Software - V30b-425-01 |
|||||||
►
Sign in to add a comment |
|||||||
Comment 1 by phistuck@chromium.org
, Jul 21Labels: -OS-Windows OS-Android
Status: Untriaged (was: Unconfirmed)