CRAS: hotword stream support |
||||||||||
Issue descriptionThe wake on voice (hotword) feature has been broken for a while. Creating this bug to track the effort to fix it, and make sure it works with the continuous hotwording requirement. (1) Audio thread timing for wake on voice device/stream. For 'wake on voice' input device, when key phrase is detected, a large chunk of data is sent to CRAS. We need to schedule audio thread wake up in a way different than the normal input stream, otherwise it's not fast enough to transfer input data to client, and cause large latency. For example, when device on rate 16000, full device buffer 65535 frames equals to roughly 4 seconds of delay. (2) Normal input + hotword input scenario. For rt5514-spi driver, open device for normal recording will cause hotword DSP to disarm. But closing device won't re-arm DSP. We'll look at how and where to do the re-arm job. (3) Language model loading. Codec/machine driver should handle that.
,
Aug 17 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/adhd/+/f165b6f9a069ca02d91e2cdbd13875e5821ecc4a commit f165b6f9a069ca02d91e2cdbd13875e5821ecc4a Author: Hsin-Yu Chao <hychao@chromium.org> Date: Thu Aug 17 14:31:37 2017 CRAS: client - schedule wake by dev level for hotword stream When wake on voice device has burst data after key phrase triggered CRAS should schedule faster wake up to transfer data to client. However client might not be able to process data fast enough, adding the client reply and poll for that signal to schedule the next wake. BUG= chromium:754581 TEST=cras_test_client --listen /tmp/1 to verify the recorded data captures full voice command. Change-Id: If52bb5d7b1b38cec8101d120a4db5ceeab21aa2b Reviewed-on: https://chromium-review.googlesource.com/608736 Commit-Ready: Hsinyu Chao <hychao@chromium.org> Tested-by: Hsinyu Chao <hychao@chromium.org> Reviewed-by: Cheng-Yi Chiang <cychiang@chromium.org> [modify] https://crrev.com/f165b6f9a069ca02d91e2cdbd13875e5821ecc4a/cras/src/tests/timing_unittest.cc [modify] https://crrev.com/f165b6f9a069ca02d91e2cdbd13875e5821ecc4a/cras/src/server/dev_io.c [modify] https://crrev.com/f165b6f9a069ca02d91e2cdbd13875e5821ecc4a/cras/src/server/dev_stream.c [modify] https://crrev.com/f165b6f9a069ca02d91e2cdbd13875e5821ecc4a/cras/src/server/audio_thread.c [modify] https://crrev.com/f165b6f9a069ca02d91e2cdbd13875e5821ecc4a/cras/src/common/cras_messages.h [modify] https://crrev.com/f165b6f9a069ca02d91e2cdbd13875e5821ecc4a/cras/src/libcras/cras_client.c
,
Aug 25 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/adhd/+/e204059b3bc8015a11b5c13309cfa01306678d51 commit e204059b3bc8015a11b5c13309cfa01306678d51 Author: Hsin-Yu Chao <hychao@chromium.org> Date: Fri Aug 25 21:21:34 2017 CRAS: iodev_list - suspend/resume hotword streams Some codecs don't allow DSP recording and normal recording to run at the same time. This patch adds CRAS support to temporarily suspend hotword streams so that input hardware can be used for normal recording, and resume hotword streams/device when normal recording is finished. BUG= chromium:754581 TEST=unittest Change-Id: If426f7a18e58461ad14cbe2eafb45322055d5915 Reviewed-on: https://chromium-review.googlesource.com/625515 Commit-Ready: Hsinyu Chao <hychao@chromium.org> Tested-by: Hsinyu Chao <hychao@chromium.org> Reviewed-by: Hsinyu Chao <hychao@chromium.org> [modify] https://crrev.com/e204059b3bc8015a11b5c13309cfa01306678d51/cras/src/server/cras_empty_iodev.h [modify] https://crrev.com/e204059b3bc8015a11b5c13309cfa01306678d51/cras/src/server/cras_iodev_list.c [modify] https://crrev.com/e204059b3bc8015a11b5c13309cfa01306678d51/cras/src/common/cras_types.h [modify] https://crrev.com/e204059b3bc8015a11b5c13309cfa01306678d51/cras/src/server/cras_iodev_list.h [modify] https://crrev.com/e204059b3bc8015a11b5c13309cfa01306678d51/cras/src/server/cras_empty_iodev.c [modify] https://crrev.com/e204059b3bc8015a11b5c13309cfa01306678d51/cras/src/tests/iodev_list_unittest.cc
,
Aug 25 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/adhd/+/702919e3f96d14990402121ef9c8370c28096395 commit 702919e3f96d14990402121ef9c8370c28096395 Author: Hsin-Yu Chao <hychao@chromium.org> Date: Fri Aug 25 21:21:34 2017 CRAS: alsa_io - Allow input iodev to preempt hotword recording When UCM has 'PreemptHotword' specified for an input node, assign the pre_open hook to suspend all hotword streams, and post_close hook to resume hotword streams. BUG= chromium:754581 TEST=Manual test on Eve, open hotword stream and normal input recording to verify that: (1) when normal input is running, hotword trigger doesn't work. (2) when normal input ends, hotword trigger just work without reopen. Change-Id: Ic548f31360fb5746fe740b23a0ab1076720c5600 Reviewed-on: https://chromium-review.googlesource.com/625596 Commit-Ready: Hsinyu Chao <hychao@chromium.org> Tested-by: Hsinyu Chao <hychao@chromium.org> Reviewed-by: Hsinyu Chao <hychao@chromium.org> [modify] https://crrev.com/702919e3f96d14990402121ef9c8370c28096395/cras/src/tests/alsa_io_unittest.cc [modify] https://crrev.com/702919e3f96d14990402121ef9c8370c28096395/cras/src/server/cras_iodev.h [modify] https://crrev.com/702919e3f96d14990402121ef9c8370c28096395/cras/src/server/cras_alsa_ucm.c [modify] https://crrev.com/702919e3f96d14990402121ef9c8370c28096395/cras/src/server/cras_iodev.c [modify] https://crrev.com/702919e3f96d14990402121ef9c8370c28096395/cras/src/server/cras_alsa_io.c [modify] https://crrev.com/702919e3f96d14990402121ef9c8370c28096395/cras/src/server/cras_alsa_ucm.h
,
Aug 28 2017
Request merge to R61 for all CLs above.
,
Aug 28 2017
This bug requires manual review: We are only 7 days from stable. Please contact the milestone owner if you have questions. Owners: amineer@(Android), cmasso@(iOS), ketakid@(ChromeOS), govind@(Desktop) For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Aug 30 2017
Approving merge to M61.
,
Aug 31 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/adhd/+/06c74d82b267ef4a61a27a81ed6ff4759cb65f04 commit 06c74d82b267ef4a61a27a81ed6ff4759cb65f04 Author: Hsin-Yu Chao <hychao@chromium.org> Date: Thu Aug 31 04:24:06 2017 CRAS: client - schedule wake by dev level for hotword stream When wake on voice device has burst data after key phrase triggered CRAS should schedule faster wake up to transfer data to client. However client might not be able to process data fast enough, adding the client reply and poll for that signal to schedule the next wake. BUG= chromium:754581 TEST=cras_test_client --listen /tmp/1 to verify the recorded data captures full voice command. Change-Id: If52bb5d7b1b38cec8101d120a4db5ceeab21aa2b Reviewed-on: https://chromium-review.googlesource.com/608736 Commit-Ready: Hsinyu Chao <hychao@chromium.org> Tested-by: Hsinyu Chao <hychao@chromium.org> Reviewed-by: Cheng-Yi Chiang <cychiang@chromium.org> (cherry picked from commit f165b6f9a069ca02d91e2cdbd13875e5821ecc4a) Reviewed-on: https://chromium-review.googlesource.com/644997 Reviewed-by: Hsinyu Chao <hychao@chromium.org> Commit-Queue: Hsinyu Chao <hychao@chromium.org> [modify] https://crrev.com/06c74d82b267ef4a61a27a81ed6ff4759cb65f04/cras/src/tests/timing_unittest.cc [modify] https://crrev.com/06c74d82b267ef4a61a27a81ed6ff4759cb65f04/cras/src/server/dev_io.c [modify] https://crrev.com/06c74d82b267ef4a61a27a81ed6ff4759cb65f04/cras/src/server/dev_stream.c [modify] https://crrev.com/06c74d82b267ef4a61a27a81ed6ff4759cb65f04/cras/src/server/audio_thread.c [modify] https://crrev.com/06c74d82b267ef4a61a27a81ed6ff4759cb65f04/cras/src/common/cras_messages.h [modify] https://crrev.com/06c74d82b267ef4a61a27a81ed6ff4759cb65f04/cras/src/libcras/cras_client.c
,
Aug 31 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/adhd/+/0cdd94229ecc5c43e418e3e20b5eb5383e23ffc3 commit 0cdd94229ecc5c43e418e3e20b5eb5383e23ffc3 Author: Hsin-Yu Chao <hychao@chromium.org> Date: Thu Aug 31 04:24:15 2017 CRAS: iodev_list - suspend/resume hotword streams Some codecs don't allow DSP recording and normal recording to run at the same time. This patch adds CRAS support to temporarily suspend hotword streams so that input hardware can be used for normal recording, and resume hotword streams/device when normal recording is finished. BUG= chromium:754581 TEST=unittest Change-Id: If426f7a18e58461ad14cbe2eafb45322055d5915 Reviewed-on: https://chromium-review.googlesource.com/625515 Commit-Ready: Hsinyu Chao <hychao@chromium.org> Tested-by: Hsinyu Chao <hychao@chromium.org> Reviewed-by: Hsinyu Chao <hychao@chromium.org> (cherry picked from commit e204059b3bc8015a11b5c13309cfa01306678d51) Reviewed-on: https://chromium-review.googlesource.com/645002 Commit-Queue: Hsinyu Chao <hychao@chromium.org> [modify] https://crrev.com/0cdd94229ecc5c43e418e3e20b5eb5383e23ffc3/cras/src/server/cras_empty_iodev.h [modify] https://crrev.com/0cdd94229ecc5c43e418e3e20b5eb5383e23ffc3/cras/src/server/cras_iodev_list.c [modify] https://crrev.com/0cdd94229ecc5c43e418e3e20b5eb5383e23ffc3/cras/src/common/cras_types.h [modify] https://crrev.com/0cdd94229ecc5c43e418e3e20b5eb5383e23ffc3/cras/src/server/cras_iodev_list.h [modify] https://crrev.com/0cdd94229ecc5c43e418e3e20b5eb5383e23ffc3/cras/src/server/cras_empty_iodev.c [modify] https://crrev.com/0cdd94229ecc5c43e418e3e20b5eb5383e23ffc3/cras/src/tests/iodev_list_unittest.cc
,
Aug 31 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/adhd/+/95612c31e0a2e5f12106f410ac88abd8900ec0b4 commit 95612c31e0a2e5f12106f410ac88abd8900ec0b4 Author: Hsin-Yu Chao <hychao@chromium.org> Date: Thu Aug 31 04:26:31 2017 CRAS: alsa_io - Allow input iodev to preempt hotword recording When UCM has 'PreemptHotword' specified for an input node, assign the pre_open hook to suspend all hotword streams, and post_close hook to resume hotword streams. BUG= chromium:754581 TEST=Manual test on Eve, open hotword stream and normal input recording to verify that: (1) when normal input is running, hotword trigger doesn't work. (2) when normal input ends, hotword trigger just work without reopen. Change-Id: Ic548f31360fb5746fe740b23a0ab1076720c5600 Reviewed-on: https://chromium-review.googlesource.com/625596 Commit-Ready: Hsinyu Chao <hychao@chromium.org> Tested-by: Hsinyu Chao <hychao@chromium.org> Reviewed-by: Hsinyu Chao <hychao@chromium.org> (cherry picked from commit 702919e3f96d14990402121ef9c8370c28096395) Reviewed-on: https://chromium-review.googlesource.com/645003 Commit-Queue: Hsinyu Chao <hychao@chromium.org> [modify] https://crrev.com/95612c31e0a2e5f12106f410ac88abd8900ec0b4/cras/src/tests/alsa_io_unittest.cc [modify] https://crrev.com/95612c31e0a2e5f12106f410ac88abd8900ec0b4/cras/src/server/cras_iodev.h [modify] https://crrev.com/95612c31e0a2e5f12106f410ac88abd8900ec0b4/cras/src/server/cras_alsa_ucm.c [modify] https://crrev.com/95612c31e0a2e5f12106f410ac88abd8900ec0b4/cras/src/server/cras_iodev.c [modify] https://crrev.com/95612c31e0a2e5f12106f410ac88abd8900ec0b4/cras/src/server/cras_alsa_io.c [modify] https://crrev.com/95612c31e0a2e5f12106f410ac88abd8900ec0b4/cras/src/server/cras_alsa_ucm.h
,
Sep 1 2017
Changes merged to R61 9765.51.0
,
Sep 4 2017
This issue has been approved for a merge. Please merge the fix to any appropriate branches as soon as possible! If all merges have been completed, please remove any remaining Merge-Approved labels from this issue. Thanks for your time! To disable nags, add the Disable-Nags label. For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Sep 5 2017
,
Jan 22 2018
,
Jan 23 2018
|
||||||||||
►
Sign in to add a comment |
||||||||||
Comment 1 by hychao@chromium.org
, Aug 11 2017