New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 851959 link

Starred by 3 users

Issue metadata

Status: Started
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Linux , Windows , Mac
Pri: 1
Type: Feature

Blocked on:
issue 850878

Blocking:
issue 874784


Participants' hotlists:
AudioService-APM
Audio-Service


Sign in to add a comment

Move the APM to the audio service

Project Member Reported by maxmorin@chromium.org, Jun 12 2018

Issue description

Will require sandboxing ( issue 850878 ), shedding old code (e.g. beamforming), improving fuzzer coverage, and of course implementation.

Putting myself as owner here, but this issue will be worked on in parallel by several people.
 
Project Member

Comment 1 by bugdroid1@chromium.org, Jun 13 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/aa4131d301e2353a932e7adcddedbd64cea14664

commit aa4131d301e2353a932e7adcddedbd64cea14664
Author: Max Morin <maxmorin@chromium.org>
Date: Wed Jun 13 17:50:59 2018

Remove implementation of googBeamforming.

Removes the implementation, leaves the web-facing stuff.

Bug:  851940 , 851959
Change-Id: I9a7ef1f14ecd67a6d56d7f63bf35c16813ee626c
Reviewed-on: https://chromium-review.googlesource.com/1097134
Reviewed-by: Guido Urdaneta <guidou@chromium.org>
Commit-Queue: Max Morin <maxmorin@chromium.org>
Cr-Commit-Position: refs/heads/master@{#566899}
[modify] https://crrev.com/aa4131d301e2353a932e7adcddedbd64cea14664/content/renderer/media/stream/media_stream_audio_processor.cc
[modify] https://crrev.com/aa4131d301e2353a932e7adcddedbd64cea14664/content/renderer/media/stream/media_stream_audio_processor_unittest.cc

Project Member

Comment 2 by bugdroid1@chromium.org, Jun 25 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/040f2ad0aed73dba73467bc500e01c50a3adb628

commit 040f2ad0aed73dba73467bc500e01c50a3adb628
Author: Max Morin <maxmorin@chromium.org>
Date: Mon Jun 25 11:58:00 2018

Fork AudioInputController to the audio service.

Also migrate the audio service to it. We want to add audio processing
capabilities to the audio service, and the media/ code cannot depend
on services/audio/, so we need the AudioInputController code in the
audio service. There's a very sketched design doc for processing in the
audio service at
https://docs.google.com/document/d/1HJnii4kuBXshVM202TVPJt_9iDmlaCX-VhatjRxRbC4/edit#.

Bug: 851959
Change-Id: Icabef727e6d309f86472d2fb401d7383051fa0ae
Reviewed-on: https://chromium-review.googlesource.com/1104464
Commit-Queue: Max Morin <maxmorin@chromium.org>
Reviewed-by: Olga Sharonova <olka@chromium.org>
Cr-Commit-Position: refs/heads/master@{#570014}
[modify] https://crrev.com/040f2ad0aed73dba73467bc500e01c50a3adb628/services/audio/BUILD.gn
[add] https://crrev.com/040f2ad0aed73dba73467bc500e01c50a3adb628/services/audio/input_controller.cc
[add] https://crrev.com/040f2ad0aed73dba73467bc500e01c50a3adb628/services/audio/input_controller.h
[add] https://crrev.com/040f2ad0aed73dba73467bc500e01c50a3adb628/services/audio/input_controller_unittest.cc
[modify] https://crrev.com/040f2ad0aed73dba73467bc500e01c50a3adb628/services/audio/input_stream.cc
[modify] https://crrev.com/040f2ad0aed73dba73467bc500e01c50a3adb628/services/audio/input_stream.h
[add] https://crrev.com/040f2ad0aed73dba73467bc500e01c50a3adb628/services/audio/input_sync_writer.cc
[add] https://crrev.com/040f2ad0aed73dba73467bc500e01c50a3adb628/services/audio/input_sync_writer.h
[add] https://crrev.com/040f2ad0aed73dba73467bc500e01c50a3adb628/services/audio/input_sync_writer_unittest.cc
[modify] https://crrev.com/040f2ad0aed73dba73467bc500e01c50a3adb628/services/audio/loopback_stream.cc
[modify] https://crrev.com/040f2ad0aed73dba73467bc500e01c50a3adb628/services/audio/loopback_stream.h
[modify] https://crrev.com/040f2ad0aed73dba73467bc500e01c50a3adb628/services/audio/loopback_stream_unittest.cc

Project Member

Comment 3 by bugdroid1@chromium.org, Jun 25 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/36818434a5be94d42702991733526ce02c9a6492

commit 36818434a5be94d42702991733526ce02c9a6492
Author: Peng Huang <penghuang@chromium.org>
Date: Mon Jun 25 15:50:31 2018

Revert "Fork AudioInputController to the audio service."

This reverts commit 040f2ad0aed73dba73467bc500e01c50a3adb628.

Reason for revert: <INSERT REASONING HERE>

Original change's description:
> Fork AudioInputController to the audio service.
> 
> Also migrate the audio service to it. We want to add audio processing
> capabilities to the audio service, and the media/ code cannot depend
> on services/audio/, so we need the AudioInputController code in the
> audio service. There's a very sketched design doc for processing in the
> audio service at
> https://docs.google.com/document/d/1HJnii4kuBXshVM202TVPJt_9iDmlaCX-VhatjRxRbC4/edit#.
> 
> Bug: 851959
> Change-Id: Icabef727e6d309f86472d2fb401d7383051fa0ae
> Reviewed-on: https://chromium-review.googlesource.com/1104464
> Commit-Queue: Max Morin <maxmorin@chromium.org>
> Reviewed-by: Olga Sharonova <olka@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#570014}

TBR=olka@chromium.org,maxmorin@chromium.org

Change-Id: I3727f7454ec28718e327335baec3960365afbe93
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 851959, 856215 
Reviewed-on: https://chromium-review.googlesource.com/1113837
Reviewed-by: Peng Huang <penghuang@chromium.org>
Commit-Queue: Peng Huang <penghuang@chromium.org>
Cr-Commit-Position: refs/heads/master@{#570050}
[modify] https://crrev.com/36818434a5be94d42702991733526ce02c9a6492/services/audio/BUILD.gn
[delete] https://crrev.com/26dcff3452ceb026bcb3b56643e85379abaca52b/services/audio/input_controller.cc
[delete] https://crrev.com/26dcff3452ceb026bcb3b56643e85379abaca52b/services/audio/input_controller.h
[delete] https://crrev.com/26dcff3452ceb026bcb3b56643e85379abaca52b/services/audio/input_controller_unittest.cc
[modify] https://crrev.com/36818434a5be94d42702991733526ce02c9a6492/services/audio/input_stream.cc
[modify] https://crrev.com/36818434a5be94d42702991733526ce02c9a6492/services/audio/input_stream.h
[delete] https://crrev.com/26dcff3452ceb026bcb3b56643e85379abaca52b/services/audio/input_sync_writer.cc
[delete] https://crrev.com/26dcff3452ceb026bcb3b56643e85379abaca52b/services/audio/input_sync_writer.h
[delete] https://crrev.com/26dcff3452ceb026bcb3b56643e85379abaca52b/services/audio/input_sync_writer_unittest.cc
[modify] https://crrev.com/36818434a5be94d42702991733526ce02c9a6492/services/audio/loopback_stream.cc
[modify] https://crrev.com/36818434a5be94d42702991733526ce02c9a6492/services/audio/loopback_stream.h
[modify] https://crrev.com/36818434a5be94d42702991733526ce02c9a6492/services/audio/loopback_stream_unittest.cc

Project Member

Comment 4 by bugdroid1@chromium.org, Jun 26 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/00f2282b4475a52477c450562b4a62903201a59e

commit 00f2282b4475a52477c450562b4a62903201a59e
Author: Max Morin <maxmorin@chromium.org>
Date: Tue Jun 26 10:39:12 2018

Reland "Fork AudioInputController to the audio service."

This is a reland of 040f2ad0aed73dba73467bc500e01c50a3adb628

Test was flaky even before landing this.

Original change's description:
> Fork AudioInputController to the audio service.
>
> Also migrate the audio service to it. We want to add audio processing
> capabilities to the audio service, and the media/ code cannot depend
> on services/audio/, so we need the AudioInputController code in the
> audio service. There's a very sketched design doc for processing in the
> audio service at
> https://docs.google.com/document/d/1HJnii4kuBXshVM202TVPJt_9iDmlaCX-VhatjRxRbC4/edit#.
>
> Bug: 851959
> Change-Id: Icabef727e6d309f86472d2fb401d7383051fa0ae
> Reviewed-on: https://chromium-review.googlesource.com/1104464
> Commit-Queue: Max Morin <maxmorin@chromium.org>
> Reviewed-by: Olga Sharonova <olka@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#570014}

Tbr: Olka
Bug: 851959,  856215 
Change-Id: I2e99e92937f8ccd73fb2ea3bd09d4175ab21419d
Reviewed-on: https://chromium-review.googlesource.com/1114618
Reviewed-by: Max Morin <maxmorin@chromium.org>
Commit-Queue: Max Morin <maxmorin@chromium.org>
Cr-Commit-Position: refs/heads/master@{#570366}
[modify] https://crrev.com/00f2282b4475a52477c450562b4a62903201a59e/services/audio/BUILD.gn
[add] https://crrev.com/00f2282b4475a52477c450562b4a62903201a59e/services/audio/input_controller.cc
[add] https://crrev.com/00f2282b4475a52477c450562b4a62903201a59e/services/audio/input_controller.h
[add] https://crrev.com/00f2282b4475a52477c450562b4a62903201a59e/services/audio/input_controller_unittest.cc
[modify] https://crrev.com/00f2282b4475a52477c450562b4a62903201a59e/services/audio/input_stream.cc
[modify] https://crrev.com/00f2282b4475a52477c450562b4a62903201a59e/services/audio/input_stream.h
[add] https://crrev.com/00f2282b4475a52477c450562b4a62903201a59e/services/audio/input_sync_writer.cc
[add] https://crrev.com/00f2282b4475a52477c450562b4a62903201a59e/services/audio/input_sync_writer.h
[add] https://crrev.com/00f2282b4475a52477c450562b4a62903201a59e/services/audio/input_sync_writer_unittest.cc
[modify] https://crrev.com/00f2282b4475a52477c450562b4a62903201a59e/services/audio/loopback_stream.cc
[modify] https://crrev.com/00f2282b4475a52477c450562b4a62903201a59e/services/audio/loopback_stream.h
[modify] https://crrev.com/00f2282b4475a52477c450562b4a62903201a59e/services/audio/loopback_stream_unittest.cc

Project Member

Comment 5 by bugdroid1@chromium.org, Jun 26 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/766269e156e642dcf2fc6fcac5844e7fc2fb1cdd

commit 766269e156e642dcf2fc6fcac5844e7fc2fb1cdd
Author: Max Morin <maxmorin@chromium.org>
Date: Tue Jun 26 15:38:45 2018

Simplify audio::InputController

 * Remove the unused TestAudioInputControllerFactory.
 * Declare media::AudioInputController as final, just as a sanity check.
 * Declare InputController as final and get rid of ref counting.
 * Change type of some durations from int to TimeDelta.
 * Get rid of the CreateForStream factory method of audio::InputController.
 * Force audio::InputController to be used on the audio manager thread.
   As a result, get rid of the trampolining methods an unparametrize test.

Bug: 851959
Cq-Include-Trybots: luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel
Change-Id: I26fbf0e51432b8bfd5ee545a68d1474f22046732
Reviewed-on: https://chromium-review.googlesource.com/1105761
Commit-Queue: Max Morin <maxmorin@chromium.org>
Reviewed-by: Olga Sharonova <olka@chromium.org>
Cr-Commit-Position: refs/heads/master@{#570413}
[modify] https://crrev.com/766269e156e642dcf2fc6fcac5844e7fc2fb1cdd/media/audio/BUILD.gn
[modify] https://crrev.com/766269e156e642dcf2fc6fcac5844e7fc2fb1cdd/media/audio/audio_input_controller.h
[delete] https://crrev.com/26630b8363528f7cf5ff28fb310a571cc0449495/media/audio/test_audio_input_controller_factory.cc
[delete] https://crrev.com/26630b8363528f7cf5ff28fb310a571cc0449495/media/audio/test_audio_input_controller_factory.h
[modify] https://crrev.com/766269e156e642dcf2fc6fcac5844e7fc2fb1cdd/services/audio/input_controller.cc
[modify] https://crrev.com/766269e156e642dcf2fc6fcac5844e7fc2fb1cdd/services/audio/input_controller.h
[modify] https://crrev.com/766269e156e642dcf2fc6fcac5844e7fc2fb1cdd/services/audio/input_controller_unittest.cc
[modify] https://crrev.com/766269e156e642dcf2fc6fcac5844e7fc2fb1cdd/services/audio/input_stream.cc
[modify] https://crrev.com/766269e156e642dcf2fc6fcac5844e7fc2fb1cdd/services/audio/input_stream.h
[modify] https://crrev.com/766269e156e642dcf2fc6fcac5844e7fc2fb1cdd/services/audio/input_sync_writer.h

Cc: -ossu@chromium.org maxmorin@chromium.org
Labels: -M-69 -Target-69 Target-70 M-70
Owner: ossu@chromium.org
Oh, thanks! It's what I've always wanted! :)
Project Member

Comment 8 by bugdroid1@chromium.org, Aug 8

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/6f15ce79ad144f8eec9347da7dea3da02f34843e

commit 6f15ce79ad144f8eec9347da7dea3da02f34843e
Author: Oskar Sundbom <ossu@chromium.org>
Date: Wed Aug 08 10:13:43 2018

Templatize GroupCoordinator

For audio processing, we want to connect an output stream to an
audio processing module (as a reference signal for echo cancellation).
Reusing the GroupCoordinator makes sense, so this CL makes the
GroupMember a template parameter.

For an outline of the project this CL is part of, see: https://docs.google.com/document/d/1u4POff_ts_1LE3WDLA_wDDFnUswdlsuHL5DsiTE0a3U/edit?usp=sharing
It's accessible to everyone @chromium.org.

Bug: 851959
Change-Id: I0ef16f5d72199ba4d72321d27b978d8e9b26772f
Reviewed-on: https://chromium-review.googlesource.com/1116542
Commit-Queue: Oskar Sundbom <ossu@chromium.org>
Reviewed-by: Yuri Wiitala <miu@chromium.org>
Cr-Commit-Position: refs/heads/master@{#581511}
[modify] https://crrev.com/6f15ce79ad144f8eec9347da7dea3da02f34843e/services/audio/BUILD.gn
[add] https://crrev.com/6f15ce79ad144f8eec9347da7dea3da02f34843e/services/audio/group_coordinator-impl.h
[delete] https://crrev.com/6a519c561be7b2bd29b53a26a9e63e81e288cb97/services/audio/group_coordinator.cc
[modify] https://crrev.com/6f15ce79ad144f8eec9347da7dea3da02f34843e/services/audio/group_coordinator.h
[modify] https://crrev.com/6f15ce79ad144f8eec9347da7dea3da02f34843e/services/audio/group_coordinator_unittest.cc
[delete] https://crrev.com/6a519c561be7b2bd29b53a26a9e63e81e288cb97/services/audio/group_member.h
[modify] https://crrev.com/6f15ce79ad144f8eec9347da7dea3da02f34843e/services/audio/local_muter.cc
[modify] https://crrev.com/6f15ce79ad144f8eec9347da7dea3da02f34843e/services/audio/local_muter.h
[modify] https://crrev.com/6f15ce79ad144f8eec9347da7dea3da02f34843e/services/audio/local_muter_unittest.cc
[add] https://crrev.com/6f15ce79ad144f8eec9347da7dea3da02f34843e/services/audio/loopback_coordinator.cc
[add] https://crrev.com/6f15ce79ad144f8eec9347da7dea3da02f34843e/services/audio/loopback_coordinator.h
[add] https://crrev.com/6f15ce79ad144f8eec9347da7dea3da02f34843e/services/audio/loopback_group_member.h
[modify] https://crrev.com/6f15ce79ad144f8eec9347da7dea3da02f34843e/services/audio/loopback_stream.cc
[modify] https://crrev.com/6f15ce79ad144f8eec9347da7dea3da02f34843e/services/audio/loopback_stream.h
[modify] https://crrev.com/6f15ce79ad144f8eec9347da7dea3da02f34843e/services/audio/loopback_stream_unittest.cc
[add] https://crrev.com/6f15ce79ad144f8eec9347da7dea3da02f34843e/services/audio/muteable.h
[modify] https://crrev.com/6f15ce79ad144f8eec9347da7dea3da02f34843e/services/audio/output_controller.cc
[modify] https://crrev.com/6f15ce79ad144f8eec9347da7dea3da02f34843e/services/audio/output_controller.h
[modify] https://crrev.com/6f15ce79ad144f8eec9347da7dea3da02f34843e/services/audio/output_controller_unittest.cc
[modify] https://crrev.com/6f15ce79ad144f8eec9347da7dea3da02f34843e/services/audio/output_stream.cc
[modify] https://crrev.com/6f15ce79ad144f8eec9347da7dea3da02f34843e/services/audio/output_stream.h
[add] https://crrev.com/6f15ce79ad144f8eec9347da7dea3da02f34843e/services/audio/snoopable.h
[modify] https://crrev.com/6f15ce79ad144f8eec9347da7dea3da02f34843e/services/audio/snooper_node.h
[modify] https://crrev.com/6f15ce79ad144f8eec9347da7dea3da02f34843e/services/audio/snooper_node_unittest.cc
[modify] https://crrev.com/6f15ce79ad144f8eec9347da7dea3da02f34843e/services/audio/stream_factory.h
[delete] https://crrev.com/6a519c561be7b2bd29b53a26a9e63e81e288cb97/services/audio/test/fake_group_member.cc
[delete] https://crrev.com/6a519c561be7b2bd29b53a26a9e63e81e288cb97/services/audio/test/fake_group_member.h
[add] https://crrev.com/6f15ce79ad144f8eec9347da7dea3da02f34843e/services/audio/test/fake_loopback_group_member.cc
[add] https://crrev.com/6f15ce79ad144f8eec9347da7dea3da02f34843e/services/audio/test/fake_loopback_group_member.h
[add] https://crrev.com/6f15ce79ad144f8eec9347da7dea3da02f34843e/services/audio/test/mock_group_coordinator.cc
[add] https://crrev.com/6f15ce79ad144f8eec9347da7dea3da02f34843e/services/audio/test/mock_group_coordinator.h
[modify] https://crrev.com/6f15ce79ad144f8eec9347da7dea3da02f34843e/services/audio/test/mock_group_member.cc
[modify] https://crrev.com/6f15ce79ad144f8eec9347da7dea3da02f34843e/services/audio/test/mock_group_member.h

Project Member

Comment 9 by bugdroid1@chromium.org, Aug 15

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/396a9099333993e4c1da130ed0ce46865bdb6c12

commit 396a9099333993e4c1da130ed0ce46865bdb6c12
Author: Oskar Sundbom <ossu@chromium.org>
Date: Wed Aug 15 14:08:31 2018

APM move: Add interfaces for Audio Processing in Audio Service

Adds some mojo interfaces for configuring audio processing, intended for
both WebRTC and system-level echo cancellation, though the latter will
be less configurable.

There are two parts to this change:
1. AudioProcessingSettings in media/audio, used to configure audio
   processing when done in the audio service. At one point, the similar
   AudioProcessingParameters in content will be replaced by this one.
2. AudioProcessorControls in media/webrtc; an interface used to control
   auxiliary audio processor functionality, specifically getting stats
   and starting/stopping AEC dumps.

For an outline of the project this CL is part of, see: https://docs.google.com/document/d/1u4POff_ts_1LE3WDLA_wDDFnUswdlsuHL5DsiTE0a3U/edit?usp=sharing
It's accessible to everyone @chromium.org.

Bug: 851959
Tbr: tommi@webrtc.org
Cq-Include-Trybots: luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel
Change-Id: I0c8c934b35b93aa728a0d1b60ae76063e925ee6c
Reviewed-on: https://chromium-review.googlesource.com/1167848
Commit-Queue: Tommi <tommi@chromium.org>
Reviewed-by: Tommi <tommi@chromium.org>
Reviewed-by: Dale Curtis <dalecurtis@chromium.org>
Cr-Commit-Position: refs/heads/master@{#583239}
[modify] https://crrev.com/396a9099333993e4c1da130ed0ce46865bdb6c12/media/DEPS
[modify] https://crrev.com/396a9099333993e4c1da130ed0ce46865bdb6c12/media/audio/BUILD.gn
[add] https://crrev.com/396a9099333993e4c1da130ed0ce46865bdb6c12/media/audio/audio_processing.h
[add] https://crrev.com/396a9099333993e4c1da130ed0ce46865bdb6c12/media/webrtc/BUILD.gn
[add] https://crrev.com/396a9099333993e4c1da130ed0ce46865bdb6c12/media/webrtc/DEPS
[add] https://crrev.com/396a9099333993e4c1da130ed0ce46865bdb6c12/media/webrtc/OWNERS
[add] https://crrev.com/396a9099333993e4c1da130ed0ce46865bdb6c12/media/webrtc/audio_processor_controls.h

Project Member

Comment 10 by bugdroid1@chromium.org, Aug 16

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/c56406b79464871b4017cd0dfba022be010e616a

commit c56406b79464871b4017cd0dfba022be010e616a
Author: Oskar Sundbom <ossu@chromium.org>
Date: Thu Aug 16 15:08:56 2018

APM move: Move EchoInformation into media/webrtc

It will be used both by content::MediaStreamAudioProcessor and the
upcoming media::AudioProcessor.

For an outline of the project this CL is part of, see: https://docs.google.com/document/d/1u4POff_ts_1LE3WDLA_wDDFnUswdlsuHL5DsiTE0a3U/edit?usp=sharing
It's accessible to everyone @chromium.org.

Bug: 851959
Change-Id: I794558e46ab01fd2f367e1df2fababdefb712c4f
Reviewed-on: https://chromium-review.googlesource.com/1170841
Commit-Queue: Oskar Sundbom <ossu@chromium.org>
Reviewed-by: Dale Curtis <dalecurtis@chromium.org>
Cr-Commit-Position: refs/heads/master@{#583653}
[modify] https://crrev.com/c56406b79464871b4017cd0dfba022be010e616a/content/renderer/BUILD.gn
[modify] https://crrev.com/c56406b79464871b4017cd0dfba022be010e616a/content/renderer/media/stream/media_stream_audio_processor.cc
[modify] https://crrev.com/c56406b79464871b4017cd0dfba022be010e616a/content/renderer/media/stream/media_stream_audio_processor.h
[modify] https://crrev.com/c56406b79464871b4017cd0dfba022be010e616a/content/renderer/media/stream/media_stream_audio_processor_options.cc
[modify] https://crrev.com/c56406b79464871b4017cd0dfba022be010e616a/content/renderer/media/stream/media_stream_audio_processor_options.h
[modify] https://crrev.com/c56406b79464871b4017cd0dfba022be010e616a/media/webrtc/BUILD.gn
[modify] https://crrev.com/c56406b79464871b4017cd0dfba022be010e616a/media/webrtc/audio_processor_controls.h
[add] https://crrev.com/c56406b79464871b4017cd0dfba022be010e616a/media/webrtc/echo_information.cc
[add] https://crrev.com/c56406b79464871b4017cd0dfba022be010e616a/media/webrtc/echo_information.h

Project Member

Comment 11 by bugdroid1@chromium.org, Aug 16

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/bebdf1c8bfd486871e213834a6bf57e999a74293

commit bebdf1c8bfd486871e213834a6bf57e999a74293
Author: Oskar Sundbom <ossu@chromium.org>
Date: Thu Aug 16 15:27:02 2018

APM move: Add AudioSinkParameters to AudioDeviceFactory et al

By putting these parameters in a struct, future changes to parameters
accepted by audio sinks can be made easier, without changing so many
dependencies.

This CL also adds an optional 'processing_id' parameter to audio
sinks. It will be used to associate input and output streams for
audio processing, when it's running in the audio service. At this
point, the id doesn't get transferred along IPC. That's part of
the next CL.

For an outline of the project this CL is part of, see: https://docs.google.com/document/d/1u4POff_ts_1LE3WDLA_wDDFnUswdlsuHL5DsiTE0a3U/edit?usp=sharing
It's accessible to everyone @chromium.org.

Bug: 851959

Tbr: kinuko@chromium.org
Cq-Include-Trybots: luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel
Change-Id: I59a467d83d1516965bbbf06cd4fc49a3198c6cfb
Reviewed-on: https://chromium-review.googlesource.com/1163884
Commit-Queue: Oskar Sundbom <ossu@chromium.org>
Reviewed-by: Dale Curtis <dalecurtis@chromium.org>
Cr-Commit-Position: refs/heads/master@{#583659}
[modify] https://crrev.com/bebdf1c8bfd486871e213834a6bf57e999a74293/content/renderer/media/audio/audio_device_factory.cc
[modify] https://crrev.com/bebdf1c8bfd486871e213834a6bf57e999a74293/content/renderer/media/audio/audio_device_factory.h
[modify] https://crrev.com/bebdf1c8bfd486871e213834a6bf57e999a74293/content/renderer/media/audio/audio_renderer_sink_cache_impl.cc
[modify] https://crrev.com/bebdf1c8bfd486871e213834a6bf57e999a74293/content/renderer/media/audio/audio_renderer_sink_cache_impl.h
[modify] https://crrev.com/bebdf1c8bfd486871e213834a6bf57e999a74293/content/renderer/media/audio/audio_renderer_sink_cache_unittest.cc
[modify] https://crrev.com/bebdf1c8bfd486871e213834a6bf57e999a74293/content/renderer/media/audio/mock_audio_device_factory.h
[modify] https://crrev.com/bebdf1c8bfd486871e213834a6bf57e999a74293/content/renderer/media/media_factory.cc
[modify] https://crrev.com/bebdf1c8bfd486871e213834a6bf57e999a74293/content/renderer/media/renderer_webaudiodevice_impl.cc
[modify] https://crrev.com/bebdf1c8bfd486871e213834a6bf57e999a74293/content/renderer/media/renderer_webaudiodevice_impl_unittest.cc
[modify] https://crrev.com/bebdf1c8bfd486871e213834a6bf57e999a74293/content/renderer/media/stream/track_audio_renderer.cc
[modify] https://crrev.com/bebdf1c8bfd486871e213834a6bf57e999a74293/content/renderer/media/webrtc/webrtc_audio_renderer.cc
[modify] https://crrev.com/bebdf1c8bfd486871e213834a6bf57e999a74293/content/renderer/media/webrtc/webrtc_audio_renderer_unittest.cc
[modify] https://crrev.com/bebdf1c8bfd486871e213834a6bf57e999a74293/content/renderer/media/webrtc_local_audio_source_provider.cc
[modify] https://crrev.com/bebdf1c8bfd486871e213834a6bf57e999a74293/content/renderer/pepper/pepper_plugin_instance_impl.cc
[modify] https://crrev.com/bebdf1c8bfd486871e213834a6bf57e999a74293/content/renderer/render_frame_impl.cc
[modify] https://crrev.com/bebdf1c8bfd486871e213834a6bf57e999a74293/content/renderer/renderer_blink_platform_impl.cc
[modify] https://crrev.com/bebdf1c8bfd486871e213834a6bf57e999a74293/content/test/renderer_audio_output_stream_factory_context_impl_unittest.cc
[modify] https://crrev.com/bebdf1c8bfd486871e213834a6bf57e999a74293/media/audio/BUILD.gn
[modify] https://crrev.com/bebdf1c8bfd486871e213834a6bf57e999a74293/media/audio/audio_output_device.cc
[modify] https://crrev.com/bebdf1c8bfd486871e213834a6bf57e999a74293/media/audio/audio_output_device.h
[modify] https://crrev.com/bebdf1c8bfd486871e213834a6bf57e999a74293/media/audio/audio_output_device_unittest.cc
[add] https://crrev.com/bebdf1c8bfd486871e213834a6bf57e999a74293/media/audio/audio_sink_parameters.cc
[add] https://crrev.com/bebdf1c8bfd486871e213834a6bf57e999a74293/media/audio/audio_sink_parameters.h

Project Member

Comment 12 by bugdroid1@chromium.org, Aug 17

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/46876cce4bebc46a3df81fbb4f3a1f81a82aa1d7

commit 46876cce4bebc46a3df81fbb4f3a1f81a82aa1d7
Author: Oskar Sundbom <ossu@chromium.org>
Date: Fri Aug 17 11:50:55 2018

APM move: Add AudioSourceParameters to AudioDeviceFactory et al

Putting parameters in a struct like this will make it easier to change
parameters going forward, as it should be less necessary to update
implementations that don't care about certain parameters.

This CL also adds processing configuration as a new, optional
audio source parameter to this struct. It is used to configure audio
processing when it's supported directly in the source, as will be the
case in the audio service.

For an outline of the project this CL is part of, see: https://docs.google.com/document/d/1u4POff_ts_1LE3WDLA_wDDFnUswdlsuHL5DsiTE0a3U/edit?usp=sharing
It's accessible to everyone @chromium.org.

Bug: 851959

Cq-Include-Trybots: luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel
Change-Id: Ibdcddf1bfc741467dcab68776872c6fcda0f43ad
Reviewed-on: https://chromium-review.googlesource.com/1167849
Commit-Queue: Oskar Sundbom <ossu@chromium.org>
Reviewed-by: Nasko Oskov <nasko@chromium.org>
Reviewed-by: Dale Curtis <dalecurtis@chromium.org>
Reviewed-by: Yuri Wiitala <miu@chromium.org>
Cr-Commit-Position: refs/heads/master@{#584025}
[modify] https://crrev.com/46876cce4bebc46a3df81fbb4f3a1f81a82aa1d7/content/renderer/media/audio/audio_device_factory.cc
[modify] https://crrev.com/46876cce4bebc46a3df81fbb4f3a1f81a82aa1d7/content/renderer/media/audio/audio_device_factory.h
[modify] https://crrev.com/46876cce4bebc46a3df81fbb4f3a1f81a82aa1d7/content/renderer/media/audio/audio_input_ipc_factory.cc
[modify] https://crrev.com/46876cce4bebc46a3df81fbb4f3a1f81a82aa1d7/content/renderer/media/audio/audio_input_ipc_factory.h
[modify] https://crrev.com/46876cce4bebc46a3df81fbb4f3a1f81a82aa1d7/content/renderer/media/audio/mock_audio_device_factory.cc
[modify] https://crrev.com/46876cce4bebc46a3df81fbb4f3a1f81a82aa1d7/content/renderer/media/audio/mock_audio_device_factory.h
[modify] https://crrev.com/46876cce4bebc46a3df81fbb4f3a1f81a82aa1d7/content/renderer/media/renderer_webaudiodevice_impl_unittest.cc
[modify] https://crrev.com/46876cce4bebc46a3df81fbb4f3a1f81a82aa1d7/content/renderer/media/stream/local_media_stream_audio_source.cc
[modify] https://crrev.com/46876cce4bebc46a3df81fbb4f3a1f81a82aa1d7/content/renderer/media/stream/processed_local_audio_source.cc
[modify] https://crrev.com/46876cce4bebc46a3df81fbb4f3a1f81a82aa1d7/content/renderer/media/webrtc/webrtc_audio_renderer_unittest.cc
[modify] https://crrev.com/46876cce4bebc46a3df81fbb4f3a1f81a82aa1d7/content/renderer/pepper/pepper_platform_audio_input.cc
[modify] https://crrev.com/46876cce4bebc46a3df81fbb4f3a1f81a82aa1d7/media/audio/BUILD.gn
[modify] https://crrev.com/46876cce4bebc46a3df81fbb4f3a1f81a82aa1d7/media/audio/audio_input_device.cc
[modify] https://crrev.com/46876cce4bebc46a3df81fbb4f3a1f81a82aa1d7/media/audio/audio_input_ipc.cc
[modify] https://crrev.com/46876cce4bebc46a3df81fbb4f3a1f81a82aa1d7/media/audio/audio_input_ipc.h
[add] https://crrev.com/46876cce4bebc46a3df81fbb4f3a1f81a82aa1d7/media/audio/audio_source_parameters.cc
[add] https://crrev.com/46876cce4bebc46a3df81fbb4f3a1f81a82aa1d7/media/audio/audio_source_parameters.h
[modify] https://crrev.com/46876cce4bebc46a3df81fbb4f3a1f81a82aa1d7/media/base/audio_capturer_source.h

Cc: -maxmorin@chromium.org ossu@chromium.org
Owner: maxmorin@chromium.org
Project Member

Comment 14 by bugdroid1@chromium.org, Aug 24

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/754f801e374500694c546d2e2f598c035b96290d

commit 754f801e374500694c546d2e2f598c035b96290d
Author: Max Morin <maxmorin@chromium.org>
Date: Fri Aug 24 13:58:35 2018

APM move: Plumb processing_id through to AudioService

The processing_id added to AudioSinkParameters in a previous CL is now
plumbed all the way into the Audio Service. It is used to register as
an observer of a StreamMonitor group with the appropiate ID. The output
controller will check when new (processed) input streams join the group
and will notify them of its existence, as well as when it goes away.

The next CL will connect InputControllers in a similar manner.

For an outline of the project this CL is part of, see: https://docs.google.com/document/d/1u4POff_ts_1LE3WDLA_wDDFnUswdlsuHL5DsiTE0a3U/edit?usp=sharing
It's accessible to everyone @chromium.org.

Bug: 851959
Cq-Include-Trybots: luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel
Change-Id: Id1dcaf882b59b963abd7c157fe7259e2b762660e
Reviewed-on: https://chromium-review.googlesource.com/1174833
Reviewed-by: Max Morin <maxmorin@chromium.org>
Reviewed-by: Dale Curtis <dalecurtis@chromium.org>
Reviewed-by: Kinuko Yasuda <kinuko@chromium.org>
Commit-Queue: Max Morin <maxmorin@chromium.org>
Cr-Commit-Position: refs/heads/master@{#585826}
[modify] https://crrev.com/754f801e374500694c546d2e2f598c035b96290d/content/browser/media/audio_output_stream_broker.cc
[modify] https://crrev.com/754f801e374500694c546d2e2f598c035b96290d/content/browser/media/audio_output_stream_broker.h
[modify] https://crrev.com/754f801e374500694c546d2e2f598c035b96290d/content/browser/media/audio_output_stream_broker_unittest.cc
[modify] https://crrev.com/754f801e374500694c546d2e2f598c035b96290d/content/browser/media/audio_stream_broker.cc
[modify] https://crrev.com/754f801e374500694c546d2e2f598c035b96290d/content/browser/media/audio_stream_broker.h
[modify] https://crrev.com/754f801e374500694c546d2e2f598c035b96290d/content/browser/media/forwarding_audio_stream_factory.cc
[modify] https://crrev.com/754f801e374500694c546d2e2f598c035b96290d/content/browser/media/forwarding_audio_stream_factory.h
[modify] https://crrev.com/754f801e374500694c546d2e2f598c035b96290d/content/browser/media/forwarding_audio_stream_factory_unittest.cc
[modify] https://crrev.com/754f801e374500694c546d2e2f598c035b96290d/content/browser/renderer_host/media/old_render_frame_audio_output_stream_factory_unittest.cc
[modify] https://crrev.com/754f801e374500694c546d2e2f598c035b96290d/content/browser/renderer_host/media/render_frame_audio_output_stream_factory.cc
[modify] https://crrev.com/754f801e374500694c546d2e2f598c035b96290d/content/browser/renderer_host/media/render_frame_audio_output_stream_factory_unittest.cc
[modify] https://crrev.com/754f801e374500694c546d2e2f598c035b96290d/content/renderer/media/audio/mojo_audio_output_ipc.cc
[modify] https://crrev.com/754f801e374500694c546d2e2f598c035b96290d/content/renderer/media/audio/mojo_audio_output_ipc.h
[modify] https://crrev.com/754f801e374500694c546d2e2f598c035b96290d/content/renderer/media/audio/mojo_audio_output_ipc_unittest.cc
[modify] https://crrev.com/754f801e374500694c546d2e2f598c035b96290d/content/renderer/pepper/pepper_platform_audio_output.cc
[modify] https://crrev.com/754f801e374500694c546d2e2f598c035b96290d/content/renderer/pepper/pepper_platform_audio_output_dev.cc
[modify] https://crrev.com/754f801e374500694c546d2e2f598c035b96290d/media/audio/audio_output_device.cc
[modify] https://crrev.com/754f801e374500694c546d2e2f598c035b96290d/media/audio/audio_output_device_unittest.cc
[modify] https://crrev.com/754f801e374500694c546d2e2f598c035b96290d/media/audio/audio_output_ipc.h
[modify] https://crrev.com/754f801e374500694c546d2e2f598c035b96290d/media/mojo/interfaces/audio_output_stream.mojom
[modify] https://crrev.com/754f801e374500694c546d2e2f598c035b96290d/media/mojo/services/mojo_audio_output_stream_provider.cc
[modify] https://crrev.com/754f801e374500694c546d2e2f598c035b96290d/media/mojo/services/mojo_audio_output_stream_provider.h
[modify] https://crrev.com/754f801e374500694c546d2e2f598c035b96290d/media/mojo/services/mojo_audio_output_stream_provider_unittest.cc
[modify] https://crrev.com/754f801e374500694c546d2e2f598c035b96290d/services/audio/BUILD.gn
[modify] https://crrev.com/754f801e374500694c546d2e2f598c035b96290d/services/audio/output_controller.cc
[modify] https://crrev.com/754f801e374500694c546d2e2f598c035b96290d/services/audio/output_controller.h
[modify] https://crrev.com/754f801e374500694c546d2e2f598c035b96290d/services/audio/output_controller_unittest.cc
[modify] https://crrev.com/754f801e374500694c546d2e2f598c035b96290d/services/audio/output_stream.cc
[modify] https://crrev.com/754f801e374500694c546d2e2f598c035b96290d/services/audio/output_stream.h
[modify] https://crrev.com/754f801e374500694c546d2e2f598c035b96290d/services/audio/output_stream_unittest.cc
[modify] https://crrev.com/754f801e374500694c546d2e2f598c035b96290d/services/audio/public/cpp/fake_stream_factory.h
[modify] https://crrev.com/754f801e374500694c546d2e2f598c035b96290d/services/audio/public/cpp/input_ipc_unittest.cc
[modify] https://crrev.com/754f801e374500694c546d2e2f598c035b96290d/services/audio/public/cpp/output_device.cc
[modify] https://crrev.com/754f801e374500694c546d2e2f598c035b96290d/services/audio/public/cpp/output_device_unittest.cc
[modify] https://crrev.com/754f801e374500694c546d2e2f598c035b96290d/services/audio/public/mojom/stream_factory.mojom
[modify] https://crrev.com/754f801e374500694c546d2e2f598c035b96290d/services/audio/stream_factory.cc
[modify] https://crrev.com/754f801e374500694c546d2e2f598c035b96290d/services/audio/stream_factory.h
[add] https://crrev.com/754f801e374500694c546d2e2f598c035b96290d/services/audio/stream_monitor.h
[add] https://crrev.com/754f801e374500694c546d2e2f598c035b96290d/services/audio/stream_monitor_coordinator.cc
[add] https://crrev.com/754f801e374500694c546d2e2f598c035b96290d/services/audio/stream_monitor_coordinator.h
[modify] https://crrev.com/754f801e374500694c546d2e2f598c035b96290d/services/audio/test/fake_loopback_group_member.cc

Project Member

Comment 15 by bugdroid1@chromium.org, Aug 27

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/db5f2b6e74d6d2e2a9c6f530199067c1d6eff7b6

commit db5f2b6e74d6d2e2a9c6f530199067c1d6eff7b6
Author: Max Morin <maxmorin@chromium.org>
Date: Mon Aug 27 09:45:24 2018

Make TimeDelta TriviallyCopyable for std::atomic.

We're having some awkwardness with working around the issue with stuff
like std::atomic<int64_t> and conversions back and forth. It would be
preferable to use std::atomic<TimeDelta> directly.

Removes the workaround added for  bug 635974 , since that was a bug in
vs 2015 and we use clang now.

Also fixes a couple of lint issues in time.h.

Bug:  635974 , 851959, 761570
Change-Id: I4683f960b0c348748c5f0aaf222da4dda40256ec
Reviewed-on: https://chromium-review.googlesource.com/1184781
Commit-Queue: Max Morin <maxmorin@chromium.org>
Reviewed-by: Yuri Wiitala <miu@chromium.org>
Reviewed-by: Bruce Dawson <brucedawson@chromium.org>
Cr-Commit-Position: refs/heads/master@{#586219}
[modify] https://crrev.com/db5f2b6e74d6d2e2a9c6f530199067c1d6eff7b6/base/time/time.h
[modify] https://crrev.com/db5f2b6e74d6d2e2a9c6f530199067c1d6eff7b6/base/time/time_unittest.cc

Project Member

Comment 16 by bugdroid1@chromium.org, Aug 29

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/e1731a3ae43e3a2af05e461fcf617b51e174bbd7

commit e1731a3ae43e3a2af05e461fcf617b51e174bbd7
Author: Max Morin <maxmorin@chromium.org>
Date: Wed Aug 29 09:37:43 2018

APM move: Implementing the APM for use in the audio service

I've placed it in media/webrtc as that made the most sense. I considered
making it its own taget (i.e. //media/webrtc:audio_processor) but it was
a bit cumbersome and other parts of media don't seem to do that.

This change includes moving some flags from content:: to media:: since
they need to be accessible by both audio processor implementations.

The new AudioProcessor code is similar to MediaStreamAudioProcessor, but
I've tried to simplify it as much as possible. It seems like the decisions
made for the old implementation were made under different circumstances
than we have now. The old implementation will run whenever the "APM in
Audio Service" feature isn't active. This'll ensure we're not changing
any behavior outside of the experiment.

The AudioProcessor is protected by a build flag to make sure it isn't built
on platforms where we don't have a sandboxed audio service.

For an outline of the project this CL is part of, see: https://docs.google.com/document/d/1u4POff_ts_1LE3WDLA_wDDFnUswdlsuHL5DsiTE0a3U/edit?usp=sharing
It's accessible to everyone @chromium.org.

Bug: 851959
Cq-Include-Trybots: luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel
Change-Id: Iec4cd2dd983b484921d5d0701d4a08bca21793b4
Reviewed-on: https://chromium-review.googlesource.com/1169463
Commit-Queue: Max Morin <maxmorin@chromium.org>
Reviewed-by: Dale Curtis <dalecurtis@chromium.org>
Reviewed-by: John Abd-El-Malek <jam@chromium.org>
Cr-Commit-Position: refs/heads/master@{#587050}
[modify] https://crrev.com/e1731a3ae43e3a2af05e461fcf617b51e174bbd7/content/browser/BUILD.gn
[modify] https://crrev.com/e1731a3ae43e3a2af05e461fcf617b51e174bbd7/content/browser/renderer_host/render_process_host_impl.cc
[modify] https://crrev.com/e1731a3ae43e3a2af05e461fcf617b51e174bbd7/content/browser/utility_process_host.cc
[modify] https://crrev.com/e1731a3ae43e3a2af05e461fcf617b51e174bbd7/content/public/common/content_features.cc
[modify] https://crrev.com/e1731a3ae43e3a2af05e461fcf617b51e174bbd7/content/public/common/content_features.h
[modify] https://crrev.com/e1731a3ae43e3a2af05e461fcf617b51e174bbd7/content/public/common/content_switches.cc
[modify] https://crrev.com/e1731a3ae43e3a2af05e461fcf617b51e174bbd7/content/public/common/content_switches.h
[modify] https://crrev.com/e1731a3ae43e3a2af05e461fcf617b51e174bbd7/content/renderer/media/stream/media_stream_audio_processor.cc
[modify] https://crrev.com/e1731a3ae43e3a2af05e461fcf617b51e174bbd7/media/BUILD.gn
[modify] https://crrev.com/e1731a3ae43e3a2af05e461fcf617b51e174bbd7/media/audio/audio_processing.h
[modify] https://crrev.com/e1731a3ae43e3a2af05e461fcf617b51e174bbd7/media/webrtc/BUILD.gn
[modify] https://crrev.com/e1731a3ae43e3a2af05e461fcf617b51e174bbd7/media/webrtc/DEPS
[add] https://crrev.com/e1731a3ae43e3a2af05e461fcf617b51e174bbd7/media/webrtc/audio_processing.gni
[add] https://crrev.com/e1731a3ae43e3a2af05e461fcf617b51e174bbd7/media/webrtc/audio_processor.cc
[add] https://crrev.com/e1731a3ae43e3a2af05e461fcf617b51e174bbd7/media/webrtc/audio_processor.h
[add] https://crrev.com/e1731a3ae43e3a2af05e461fcf617b51e174bbd7/media/webrtc/audio_processor_unittest.cc
[add] https://crrev.com/e1731a3ae43e3a2af05e461fcf617b51e174bbd7/media/webrtc/webrtc_switches.cc
[add] https://crrev.com/e1731a3ae43e3a2af05e461fcf617b51e174bbd7/media/webrtc/webrtc_switches.h
[modify] https://crrev.com/e1731a3ae43e3a2af05e461fcf617b51e174bbd7/services/audio/BUILD.gn
[modify] https://crrev.com/e1731a3ae43e3a2af05e461fcf617b51e174bbd7/services/audio/DEPS

Project Member

Comment 17 by bugdroid1@chromium.org, Aug 30

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/157168603a362901c065703f7e0640de948f54c9

commit 157168603a362901c065703f7e0640de948f54c9
Author: Max Morin <maxmorin@chromium.org>
Date: Thu Aug 30 21:45:54 2018

APM move: Plumb processing settings from renderer into audio service.

Similar to the output CL, processing settings are plumbed into the audio
service. InputControllers join the processing group and get notified by
the appropriate output controller is started. It can then reconfigure
itself as necessary; to either start monitoring the output stream or to
ask what device it is using and call SetOutputDeviceForAec on its
AudioInputStream.

For an outline of the project this CL is part of, see: https://docs.google.com/document/d/1u4POff_ts_1LE3WDLA_wDDFnUswdlsuHL5DsiTE0a3U/edit?usp=sharing
It's accessible to everyone @chromium.org.

Bug: 851959
Cq-Include-Trybots: luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel
Change-Id: I138cd290295821467e836a7e7e6eb7d7ad364bdc
Reviewed-on: https://chromium-review.googlesource.com/1174835
Commit-Queue: Max Morin <maxmorin@chromium.org>
Reviewed-by: Kinuko Yasuda <kinuko@chromium.org>
Reviewed-by: Robert Sesek <rsesek@chromium.org>
Reviewed-by: Olga Sharonova <olka@chromium.org>
Reviewed-by: Max Morin <maxmorin@chromium.org>
Cr-Commit-Position: refs/heads/master@{#587792}
[modify] https://crrev.com/157168603a362901c065703f7e0640de948f54c9/content/browser/media/audio_input_stream_broker.cc
[modify] https://crrev.com/157168603a362901c065703f7e0640de948f54c9/content/browser/media/audio_input_stream_broker.h
[modify] https://crrev.com/157168603a362901c065703f7e0640de948f54c9/content/browser/media/audio_input_stream_broker_unittest.cc
[modify] https://crrev.com/157168603a362901c065703f7e0640de948f54c9/content/browser/media/audio_stream_broker.cc
[modify] https://crrev.com/157168603a362901c065703f7e0640de948f54c9/content/browser/media/audio_stream_broker.h
[modify] https://crrev.com/157168603a362901c065703f7e0640de948f54c9/content/browser/media/forwarding_audio_stream_factory.cc
[modify] https://crrev.com/157168603a362901c065703f7e0640de948f54c9/content/browser/media/forwarding_audio_stream_factory.h
[modify] https://crrev.com/157168603a362901c065703f7e0640de948f54c9/content/browser/media/forwarding_audio_stream_factory_unittest.cc
[modify] https://crrev.com/157168603a362901c065703f7e0640de948f54c9/content/browser/renderer_host/media/old_render_frame_audio_input_stream_factory.cc
[modify] https://crrev.com/157168603a362901c065703f7e0640de948f54c9/content/browser/renderer_host/media/old_render_frame_audio_input_stream_factory.h
[modify] https://crrev.com/157168603a362901c065703f7e0640de948f54c9/content/browser/renderer_host/media/old_render_frame_audio_input_stream_factory_unittest.cc
[modify] https://crrev.com/157168603a362901c065703f7e0640de948f54c9/content/browser/renderer_host/media/render_frame_audio_input_stream_factory.cc
[modify] https://crrev.com/157168603a362901c065703f7e0640de948f54c9/content/browser/renderer_host/media/render_frame_audio_input_stream_factory.h
[modify] https://crrev.com/157168603a362901c065703f7e0640de948f54c9/content/browser/renderer_host/media/render_frame_audio_input_stream_factory_unittest.cc
[modify] https://crrev.com/157168603a362901c065703f7e0640de948f54c9/content/common/BUILD.gn
[modify] https://crrev.com/157168603a362901c065703f7e0640de948f54c9/content/common/media/renderer_audio_input_stream_factory.mojom
[modify] https://crrev.com/157168603a362901c065703f7e0640de948f54c9/content/renderer/BUILD.gn
[modify] https://crrev.com/157168603a362901c065703f7e0640de948f54c9/content/renderer/media/audio/audio_input_ipc_factory.cc
[modify] https://crrev.com/157168603a362901c065703f7e0640de948f54c9/content/renderer/media/audio/mojo_audio_input_ipc.cc
[modify] https://crrev.com/157168603a362901c065703f7e0640de948f54c9/content/renderer/media/audio/mojo_audio_input_ipc.h
[modify] https://crrev.com/157168603a362901c065703f7e0640de948f54c9/content/renderer/media/audio/mojo_audio_input_ipc_unittest.cc
[modify] https://crrev.com/157168603a362901c065703f7e0640de948f54c9/media/audio/audio_processing.h
[modify] https://crrev.com/157168603a362901c065703f7e0640de948f54c9/services/audio/BUILD.gn
[modify] https://crrev.com/157168603a362901c065703f7e0640de948f54c9/services/audio/input_controller.cc
[modify] https://crrev.com/157168603a362901c065703f7e0640de948f54c9/services/audio/input_controller.h
[modify] https://crrev.com/157168603a362901c065703f7e0640de948f54c9/services/audio/input_controller_unittest.cc
[modify] https://crrev.com/157168603a362901c065703f7e0640de948f54c9/services/audio/input_stream.cc
[modify] https://crrev.com/157168603a362901c065703f7e0640de948f54c9/services/audio/input_stream.h
[modify] https://crrev.com/157168603a362901c065703f7e0640de948f54c9/services/audio/input_stream_unittest.cc
[add] https://crrev.com/157168603a362901c065703f7e0640de948f54c9/services/audio/public/cpp/audio_processing.typemap
[add] https://crrev.com/157168603a362901c065703f7e0640de948f54c9/services/audio/public/cpp/audio_processing_mojom_traits.cc
[add] https://crrev.com/157168603a362901c065703f7e0640de948f54c9/services/audio/public/cpp/audio_processing_mojom_traits.h
[modify] https://crrev.com/157168603a362901c065703f7e0640de948f54c9/services/audio/public/cpp/fake_stream_factory.h
[modify] https://crrev.com/157168603a362901c065703f7e0640de948f54c9/services/audio/public/cpp/input_ipc.cc
[modify] https://crrev.com/157168603a362901c065703f7e0640de948f54c9/services/audio/public/cpp/input_ipc_unittest.cc
[modify] https://crrev.com/157168603a362901c065703f7e0640de948f54c9/services/audio/public/cpp/typemaps.gni
[modify] https://crrev.com/157168603a362901c065703f7e0640de948f54c9/services/audio/public/mojom/BUILD.gn
[add] https://crrev.com/157168603a362901c065703f7e0640de948f54c9/services/audio/public/mojom/audio_processing.mojom
[modify] https://crrev.com/157168603a362901c065703f7e0640de948f54c9/services/audio/public/mojom/stream_factory.mojom
[modify] https://crrev.com/157168603a362901c065703f7e0640de948f54c9/services/audio/stream_factory.cc
[modify] https://crrev.com/157168603a362901c065703f7e0640de948f54c9/services/audio/stream_factory.h

Project Member

Comment 18 by bugdroid1@chromium.org, Aug 30

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/86d4f131d29c49e761ee20d9b1c5026b1172ef50

commit 86d4f131d29c49e761ee20d9b1c5026b1172ef50
Author: Max Morin <maxmorin@chromium.org>
Date: Thu Aug 30 21:51:13 2018

APM move: Configure and use APM in Audio Service

ProcessedLocalAudioSource is changed to either create a
MediaStreamAudioProcessor or an AudioServiceAudioProcessorProxy depending
on whether the WebRtcApmInAudioService flag is set.

AudioServiceAudioProcessorProxy proxies GetStats and AECDump calls to
the remote audio processor. Although the JavaScript getStats call is
asynchronous, we currently collect stats synchronously inside Chrome.
Ideally, this would be changed. For now, the proxy overcomes this
mismatch by polling the remote audio processor for stats at regular
intervals. It uses a heuristic to determine the rate at which the user
is calling getStats and tries to match that, within some reasonable
limits. 878764 has been filed to fix this.

For AECDumps, we already get a file-handle from the browser, so it can
just be sent along to the audio service. So the audio service does not
need to be able to create files for this to work.

For an outline of the project this CL is part of, see: https://docs.google.com/document/d/1u4POff_ts_1LE3WDLA_wDDFnUswdlsuHL5DsiTE0a3U/edit?usp=sharing
It's accessible to everyone @chromium.org.

No-try since the test timing out is unrelated.

No-Try: true
Bug: 851959, 878764, 879133,  879243 , 879296
Cq-Include-Trybots: luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel
Change-Id: I7473c068aa691d69f6ba90dce2550534c9cb3d8a
Reviewed-on: https://chromium-review.googlesource.com/1169471
Commit-Queue: Max Morin <maxmorin@chromium.org>
Reviewed-by: Robert Sesek <rsesek@chromium.org>
Reviewed-by: Kinuko Yasuda <kinuko@chromium.org>
Reviewed-by: Olga Sharonova <olka@chromium.org>
Reviewed-by: Max Morin <maxmorin@chromium.org>
Cr-Commit-Position: refs/heads/master@{#587796}
[modify] https://crrev.com/86d4f131d29c49e761ee20d9b1c5026b1172ef50/content/browser/media/media_internals.cc
[modify] https://crrev.com/86d4f131d29c49e761ee20d9b1c5026b1172ef50/content/browser/renderer_host/media/audio_output_delegate_impl_unittest.cc
[modify] https://crrev.com/86d4f131d29c49e761ee20d9b1c5026b1172ef50/content/browser/webrtc/webrtc_audio_browsertest.cc
[modify] https://crrev.com/86d4f131d29c49e761ee20d9b1c5026b1172ef50/content/renderer/BUILD.gn
[modify] https://crrev.com/86d4f131d29c49e761ee20d9b1c5026b1172ef50/content/renderer/media/audio/mojo_audio_input_ipc.cc
[add] https://crrev.com/86d4f131d29c49e761ee20d9b1c5026b1172ef50/content/renderer/media/stream/audio_service_audio_processor_proxy.cc
[add] https://crrev.com/86d4f131d29c49e761ee20d9b1c5026b1172ef50/content/renderer/media/stream/audio_service_audio_processor_proxy.h
[modify] https://crrev.com/86d4f131d29c49e761ee20d9b1c5026b1172ef50/content/renderer/media/stream/media_stream_audio_processor_options.cc
[modify] https://crrev.com/86d4f131d29c49e761ee20d9b1c5026b1172ef50/content/renderer/media/stream/media_stream_audio_processor_options.h
[modify] https://crrev.com/86d4f131d29c49e761ee20d9b1c5026b1172ef50/content/renderer/media/stream/processed_local_audio_source.cc
[modify] https://crrev.com/86d4f131d29c49e761ee20d9b1c5026b1172ef50/content/renderer/media/stream/processed_local_audio_source.h
[modify] https://crrev.com/86d4f131d29c49e761ee20d9b1c5026b1172ef50/content/renderer/media/webrtc/webrtc_audio_device_impl.cc
[modify] https://crrev.com/86d4f131d29c49e761ee20d9b1c5026b1172ef50/content/renderer/media/webrtc/webrtc_audio_device_impl.h
[modify] https://crrev.com/86d4f131d29c49e761ee20d9b1c5026b1172ef50/content/renderer/media/webrtc/webrtc_audio_renderer.cc
[modify] https://crrev.com/86d4f131d29c49e761ee20d9b1c5026b1172ef50/content/renderer/media/webrtc/webrtc_audio_renderer_unittest.cc
[modify] https://crrev.com/86d4f131d29c49e761ee20d9b1c5026b1172ef50/content/renderer/media/webrtc/webrtc_audio_sink.cc
[modify] https://crrev.com/86d4f131d29c49e761ee20d9b1c5026b1172ef50/content/renderer/media/webrtc/webrtc_audio_sink.h
[modify] https://crrev.com/86d4f131d29c49e761ee20d9b1c5026b1172ef50/content/renderer/media/webrtc/webrtc_media_stream_track_adapter.cc
[modify] https://crrev.com/86d4f131d29c49e761ee20d9b1c5026b1172ef50/content/test/BUILD.gn
[modify] https://crrev.com/86d4f131d29c49e761ee20d9b1c5026b1172ef50/media/audio/BUILD.gn
[modify] https://crrev.com/86d4f131d29c49e761ee20d9b1c5026b1172ef50/media/audio/audio_logging.h
[add] https://crrev.com/86d4f131d29c49e761ee20d9b1c5026b1172ef50/media/audio/audio_processing.cc
[modify] https://crrev.com/86d4f131d29c49e761ee20d9b1c5026b1172ef50/media/audio/audio_processing.h
[modify] https://crrev.com/86d4f131d29c49e761ee20d9b1c5026b1172ef50/media/audio/audio_source_parameters.h
[modify] https://crrev.com/86d4f131d29c49e761ee20d9b1c5026b1172ef50/media/audio/fake_audio_log_factory.cc
[modify] https://crrev.com/86d4f131d29c49e761ee20d9b1c5026b1172ef50/media/mojo/interfaces/audio_logging.mojom
[modify] https://crrev.com/86d4f131d29c49e761ee20d9b1c5026b1172ef50/services/audio/input_controller.cc
[modify] https://crrev.com/86d4f131d29c49e761ee20d9b1c5026b1172ef50/services/audio/input_stream.cc
[modify] https://crrev.com/86d4f131d29c49e761ee20d9b1c5026b1172ef50/services/audio/log_adapter.cc
[modify] https://crrev.com/86d4f131d29c49e761ee20d9b1c5026b1172ef50/services/audio/log_adapter.h
[modify] https://crrev.com/86d4f131d29c49e761ee20d9b1c5026b1172ef50/services/audio/log_factory_manager_unittest.cc
[modify] https://crrev.com/86d4f131d29c49e761ee20d9b1c5026b1172ef50/services/audio/test/mock_log.h

Should we change the milestone label to M71?
Blocking: 874784
Project Member

Comment 21 by bugdroid1@chromium.org, Sep 10

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/a072b983b33cd2e8c022a20e6b2a2702e5940ff6

commit a072b983b33cd2e8c022a20e6b2a2702e5940ff6
Author: Max Morin <maxmorin@chromium.org>
Date: Mon Sep 10 14:27:14 2018

Sanitize audio data passed to APM in audio service.

The APM expects data in the range [-1, 1] only. This CL also avoids
sending bitstream stuff to snoopers, since they would misunderstand it
anyways. The sanitation is only carried out when using the APM, to
ensure that ordinary playback isn't impacted.

Bug: 851959, 882360
Change-Id: I8b51af68f3a3e53383eb6630f73b88205faac887
Reviewed-on: https://chromium-review.googlesource.com/1215865
Reviewed-by: Oskar Sundbom <ossu@chromium.org>
Commit-Queue: Max Morin <maxmorin@chromium.org>
Cr-Commit-Position: refs/heads/master@{#589905}
[modify] https://crrev.com/a072b983b33cd2e8c022a20e6b2a2702e5940ff6/services/audio/output_controller.cc

Project Member

Comment 22 by bugdroid1@chromium.org, Sep 14

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/df36306dcbed91e2477ee5020ca8ae8024d58911

commit df36306dcbed91e2477ee5020ca8ae8024d58911
Author: Oskar Sundbom <ossu@chromium.org>
Date: Fri Sep 14 09:07:59 2018

AudioServiceAudioProcessorProxy: Register as Aec Dump delegate later

There's no point in registering before having gotten an
AudioProcessorControls to proxy to. Registering too early means it'll
discard the request to start an AEC dump.

I also simplified SetControls, as it should only be called once. The
current structure was left behind from when the plan was to call it
with nullptr to detach from the controls.

Bug: 851959
Change-Id: I4470a127fb6e22aa1b6fe4dd8a0b9bd0866514a4
Reviewed-on: https://chromium-review.googlesource.com/1224930
Reviewed-by: Max Morin <maxmorin@chromium.org>
Commit-Queue: Oskar Sundbom <ossu@chromium.org>
Cr-Commit-Position: refs/heads/master@{#591309}
[modify] https://crrev.com/df36306dcbed91e2477ee5020ca8ae8024d58911/content/renderer/media/stream/audio_service_audio_processor_proxy.cc
[modify] https://crrev.com/df36306dcbed91e2477ee5020ca8ae8024d58911/content/renderer/media/stream/audio_service_audio_processor_proxy.h

Labels: Merge-Request-70
Requesting merge for CL in comment 22. It ensures AEC dumps work, which is helps immensely in debugging issues with the APM. The change itself is small and safe and will only be executed if running with the "APM in Audio Service" flag enabled.
Project Member

Comment 24 by sheriffbot@chromium.org, Sep 17

Labels: -Merge-Request-70 Merge-Review-70 Hotlist-Merge-Review
This bug requires manual review: DEPS changes referenced in bugdroid comments.
Please contact the milestone owner if you have questions.
Owners: benmason@(Android), kariahda@(iOS), geohsu@(ChromeOS), abdulsyed@(Desktop)

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
Labels: -Merge-Review-70 Merge-Approved-70
Project Member

Comment 26 by bugdroid1@chromium.org, Sep 18

Labels: -merge-approved-70 merge-merged-3538
The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/b94fdaa6d8613705bec2b0c836bacb43cf6f121a

commit b94fdaa6d8613705bec2b0c836bacb43cf6f121a
Author: Oskar Sundbom <ossu@chromium.org>
Date: Tue Sep 18 10:55:10 2018

M70: AudioServiceAudioProcessorProxy: Register as Aec Dump delegate later

There's no point in registering before having gotten an
AudioProcessorControls to proxy to. Registering too early means it'll
discard the request to start an AEC dump.

I also simplified SetControls, as it should only be called once. The
current structure was left behind from when the plan was to call it
with nullptr to detach from the controls.

Bug: 851959
Change-Id: I4470a127fb6e22aa1b6fe4dd8a0b9bd0866514a4
Reviewed-on: https://chromium-review.googlesource.com/1224930
Reviewed-by: Max Morin <maxmorin@chromium.org>
Commit-Queue: Oskar Sundbom <ossu@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#591309}(cherry picked from commit df36306dcbed91e2477ee5020ca8ae8024d58911)
Reviewed-on: https://chromium-review.googlesource.com/1230021
Reviewed-by: Oskar Sundbom <ossu@chromium.org>
Cr-Commit-Position: refs/branch-heads/3538@{#482}
Cr-Branched-From: 79f7c91a2b2a2932cd447fa6f865cb6662fa8fa6-refs/heads/master@{#587811}
[modify] https://crrev.com/b94fdaa6d8613705bec2b0c836bacb43cf6f121a/content/renderer/media/stream/audio_service_audio_processor_proxy.cc
[modify] https://crrev.com/b94fdaa6d8613705bec2b0c836bacb43cf6f121a/content/renderer/media/stream/audio_service_audio_processor_proxy.h

Sign in to add a comment