Update of my investigation:

The mechanism that we mute the original tab is not calling tab->Mute() but:
In AudioMirroringManager, when StartMirroring is called, we divert the audio stream from "going to speaker" to "going to WebContentsAudioInputStream". 
When stopMirroring is called, we divert the audio stream back.

If you exit recorder normally, StopMirroring will be called, and the sound will get back. If you kill the recorder, StopMirroring will not be called, then the sound will no longer be back, unless you close the tab, and reopen it.

I'll look for a way on how to duplicate the audio stream.

cc miu@ for more ideas as this is the same code path with audio part of tabCapture:

The main functionality we want to implement is that during capturing, we still want to hear the original sound.

The main difficulty duplicating the audio stream is here:
In AudioOutputController, its |sync_reader_| is the audio provider, and the target for transmitting audio to is the |stream_|. By switching |stream_| to speaker or a virtual stream, we can either play out the sound or record the audio.

The difficulty for |sync_reader_| to serve two output |stream_| is here:
it is the |stream_| that pulls data from |sync_reader_|. So if we simply start two |stream_| and use one |sync_reader_| serving them, then both |stream_| will get part of the audio data. 

I've written a design doc, as the change would be non-trivial.

https://docs.google.com/document/d/19iJv7bitqk9DmVlPIie6KPY14enF9q7_N5qVaXELK_E

+dalecurtis

Is recording audio at the browser level what you want here? I thought mcasas@ was working on copying audio at the tag level?

#6: s/tag/tab/: I'm eavesdropping the audio from a single
WMPlayer element, whereas, I think, qiangchen@ is interested
in recording all tab sounds and noises - IIUC that's "after"
(in audio flow terms) the renderer's AudioRendererMixer.

Re #6: Essentially yes.
What we need to do is to generate the audio stream for a given tab.
The current functionality doing so is by redirecting the stream to a given destination. The price is that you cannot hear the sound of the tab during this capturing (because the stream is not going to the speaker any more). 

And yes, the workflow is in the browser process. AudioOutputController is the class doing the redirection. 
I think we can make a copy of the audio data to serve those virtual destinations. But audio data flow is a pull model (it is those destinations that initiates requests for audio from AudioOutputController, not the AudioOutputController initiates a data push to destinations), and miu@ points out several critical issues. Essentially the problem is that how we can reconcile clock rate difference between destinations.

You can see #4 for the design doc, in which there are more details.

Update of Design:

After my code investigation, updated my design with AudioShifter use for clock rate difference reconciliation.

https://codereview.chromium.org/1839723002/

is an experimental CL illustrating the design idea.

Patch 1 is the core part -- Change of AudioMirroringManager and AudioOutputController to make sound duplication work.

Patch 2 adds the code path that propagates the duplicate-or-divert info to the core class.

PTAL.

Thanks.

Updated the design with deeper look into WebContentsAudioInputStream.
Updated the design with miu@'s suggestion for push method.

However, during my experimental implementation, I found a difficulty, namely the consumer-provider audio parameter discrepancy.

Will this make it to M51? If not please update the milestone label.

Due to the complexity, we defer it to M-52

Moving this nonessential bug to the next milestone.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/092d94c06544ef160955147dce1527da9699c81b

commit 092d94c06544ef160955147dce1527da9699c81b
Author: qiangchen <qiangchen@chromium.org>
Date: Thu Jun 02 01:27:14 2016

Unmute Tab Audio For Desktop Share

We did an enhancement to support audio for desktop share.
However, using the existing framework, we found a deficiency:
During the tab sharing with audio, on the sender side, we
cannot hear the sound.
The main reason is that we divert the audio stream to
WebContentsAudioInputStream, and no longer serve the speaker.

In this CL, we make the code path capable of duplicating
audio data.

BUG= 595428 

Review-Url: https://codereview.chromium.org/1897953003
Cr-Commit-Position: refs/heads/master@{#397288}

[modify] https://crrev.com/092d94c06544ef160955147dce1527da9699c81b/content/browser/media/capture/audio_mirroring_manager.cc
[modify] https://crrev.com/092d94c06544ef160955147dce1527da9699c81b/content/browser/media/capture/audio_mirroring_manager.h
[modify] https://crrev.com/092d94c06544ef160955147dce1527da9699c81b/content/browser/media/capture/audio_mirroring_manager_unittest.cc
[modify] https://crrev.com/092d94c06544ef160955147dce1527da9699c81b/content/browser/media/capture/web_contents_audio_input_stream.cc
[modify] https://crrev.com/092d94c06544ef160955147dce1527da9699c81b/content/browser/media/capture/web_contents_audio_input_stream.h
[modify] https://crrev.com/092d94c06544ef160955147dce1527da9699c81b/content/browser/media/capture/web_contents_audio_input_stream_unittest.cc
[modify] https://crrev.com/092d94c06544ef160955147dce1527da9699c81b/content/browser/media/capture/web_contents_audio_muter.cc
[modify] https://crrev.com/092d94c06544ef160955147dce1527da9699c81b/media/audio/BUILD.gn
[modify] https://crrev.com/092d94c06544ef160955147dce1527da9699c81b/media/audio/audio_output_controller.cc
[modify] https://crrev.com/092d94c06544ef160955147dce1527da9699c81b/media/audio/audio_output_controller.h
[modify] https://crrev.com/092d94c06544ef160955147dce1527da9699c81b/media/audio/audio_output_controller_unittest.cc
[modify] https://crrev.com/092d94c06544ef160955147dce1527da9699c81b/media/audio/audio_source_diverter.h
[modify] https://crrev.com/092d94c06544ef160955147dce1527da9699c81b/media/audio/virtual_audio_input_stream.cc
[modify] https://crrev.com/092d94c06544ef160955147dce1527da9699c81b/media/audio/virtual_audio_input_stream.h
[modify] https://crrev.com/092d94c06544ef160955147dce1527da9699c81b/media/audio/virtual_audio_output_stream.cc
[modify] https://crrev.com/092d94c06544ef160955147dce1527da9699c81b/media/audio/virtual_audio_output_stream_unittest.cc
[add] https://crrev.com/092d94c06544ef160955147dce1527da9699c81b/media/audio/virtual_audio_sink.cc
[add] https://crrev.com/092d94c06544ef160955147dce1527da9699c81b/media/audio/virtual_audio_sink.h
[modify] https://crrev.com/092d94c06544ef160955147dce1527da9699c81b/media/media.gyp

The core part of unmuting the audio during sharing is done.
Now we need to hook up the functionality with use cases.
After some investigation, I found several approaches:

1. Set |is_duplication| to true for WebContentsAudioInputStream, if the device type is MEDIA_DESKTOP_AUDIO_CAPTURE, which means this audio request is from desktop capture picker. (For cast case, the device type is MEDIA_TAB_AUDIO_CAPTURE, and thus unaffected by this check.)

2. Let audio constraints control the value of |is_duplication|.
2.a I found there is an existing audio constraint called "googAudioMirroring", which is not used in desktop or tab capture case. Not sure appropriate to simply use "googAudioMirroring" to control |is_duplication|?

2.b Create another constraint entry say "googAudioDuplication" to control |is_duplication|.

Pros and Cons:
1. Simple, but kind of hard code, as the web developer has no control on whether the source sound is muted or not.

2. Flexible, but we need to notify web developers of this change, and they may struggle if the default setting is not what they want.
2.b For this plan, we will have to do some blink developement.

Any suggestions?

pin miu@ and niklase@ to take a look at #16.
Can you give me some input on the implementation plan?

I think this will be a complicated switch to expose to web developers, so I vote for 1. I see 2 typical scenarios:

- Sharing tab audio from your laptop to a video meeting with another display in the same room. In this case you have to mute the system audio to not play it twice.

- In a video call with someone else, and you want to share another tab with audio. In this case you will be forced to hear the tab audio yourself if you want to hear the other party - I think this is fine.

niklase: In c#18, I'm not sure I understand your second bullet point. This "is_duplication" setting only controls whether the tab that is being captured will have its audio locally emitted or muted. So, if you're in a VC in one tab and want to share a different tab, the user is *not* forced to hear the tab if they want to continue to hear the other party.

Re c#16: What do you see are the most typical use cases for this? Is this something, for example, that is going to be initiated from a VC application like Hangouts? If so, it might be a good idea to mute the tab locally since the user would hear its audio from the VC. If the local audio were not muted, echo cancellation would not suffice and the user will get annoying feedback loops (amp@ recently encountered a problem like this with his recent project).

I've been told of another use case where the purpose of having tab capture was to record a user's session for quality assurance purposes, and in this case local audio should be emitted to avoid diminishing their normal user experience.

So, in order to be clear about the API, what do you see are the typical use cases? Also, IIUC, tab capture will be initiated through the same picker dialog as desktop/window capture, so what is currently the default behavior there?

Re#20: I do not think there could be feedback loops, unless the user is sharing the tab running VC application itself. For example, if within Hangouts, you are sharing YouTube, YouTube sound goes to the other peer, and suppose the peer's microphone speaker position is close, then the sound goes into microphone, and thus is streamed back to your Hangouts tab, not your YouTube tab. That's it, it will not go to the peer again.

The current behavior is "mute the source audio", because in the current code, we never plug true into "is_duplication" when creating WCAIS. We are discussing the condition on when we should plug in true.

If using audio constraint, do you think we can use "googAudioMirroring"? Or create a new entry?

Sorry, I wasn't precise: I meant, what does the existing desktop capture do: Is local audio muted or unmuted while desktop capturing is active?

I agree it'd be best to just have a new boolean constraint that says whether local audio will be muted or not for the captured tab. We can't re-use googAudioMirroring because that means something totally different (and is actually not related to screen capture at all!). Also, if this new boolean constraint is not specified by the client, the default should probably be the same as what desktop capture does today.

Oh, BTW--Constraints are in blink code here: https://cs.chromium.org/chromium/src/third_party/WebKit/public/platform/WebMediaConstraints.h?q=WebMediaC&sq=package:chromium&l=31

Re #22: Did you mean screen share?
Local audio is unmuted, and I think actually it is out of chrome's control.

Yeah desktop audio isn't muted and we can't do it. 

Ok. Sounds like tab capture should not be muted by default, then. So, then, we'd just want a boolean constraint like googMuteCapturedSource (true/false) that defaults to false.

Then I think we have different default behavior for chrome.desktopCapture and chrome.tabCapture.

Can we implement plan 1 first, and make plan 2 as a future enhancement?

re #c27: SGTM.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/4534e544153d04f43759bcdba32a03fb7cb45c92

commit 4534e544153d04f43759bcdba32a03fb7cb45c92
Author: qiangchen <qiangchen@chromium.org>
Date: Tue Jun 21 18:44:19 2016

Unmute Tab Audio For Desktop Share By Default

We did an enhancement to support audio for desktop share.
However, using the existing framework, we found a deficiency:
During the tab sharing with audio, on the sender side, we
cannot hear the sound.
The main reason is that we divert the audio stream to
WebContentsAudioInputStream, and no longer serve the speaker.

In previous CL, we make the code path capable of duplicating
audio data. In this CL, we make the duplication to be the default
behavior for audio type MEDIA_DESKTOP_AUDIO_CAPTURE.

BUG= 595428 

Review-Url: https://codereview.chromium.org/2060963002
Cr-Commit-Position: refs/heads/master@{#401054}

[modify] https://crrev.com/4534e544153d04f43759bcdba32a03fb7cb45c92/content/browser/media/capture/web_contents_audio_input_stream.cc
[modify] https://crrev.com/4534e544153d04f43759bcdba32a03fb7cb45c92/content/browser/media/capture/web_contents_audio_input_stream.h
[modify] https://crrev.com/4534e544153d04f43759bcdba32a03fb7cb45c92/content/browser/renderer_host/media/audio_input_renderer_host.cc

srcv@: We've solved the tab audio issue. So now during tab sharing with audio, both peers can hear the audio. Namely the sender and receiver can both hear audio.

The development made a significant change. Please test the functionality thoroughly.