New issue
Advanced search Search tips
Note: Color blocks (like █ or █) mean that a user may not be available. Tooltip shows the reason.

Issue 597876 link

Starred by 1 user
Status: Fixed
Owner:
emir...@chromium.org
Closed: Mar 2016
Cc:
niklase@chromium.org
m...@chromium.org
mcasas@chromium.org
hta@chromium.org
█ phil...@opera.com
guidou@chromium.org
Components:
EstimatedDays: ----
NextAction: ----
OS: All
Pri: 2
Type: Bug

Blocking:
issue 587789



Sign in to add a comment

MediaStream behavior when a new sink is added

Project Member Reported by emir...@chromium.org, Mar 25 2016 
MediaStream does not hold onto the last video frame. Therefore, when a new sink is added, we do not send anything down until a new frame is provided by the source.

This caused issues in canvas capture interoperability with Firefox, as it has been discussed on  issue 587789  and https://github.com/w3c/mediacapture-fromelement/issues/29#issuecomment-194277907. In addition, this caused issues in Tab/Desktop capture, pointed by miu and some work can be seen on https://codereview.chromium.org/1826643003/. So, I wanted to move the discussion to a seperate bug.

As pointed out by many people, VideoFrame holding onto a video frame is a risky approach. Instead, I propose adding a new functionality such that MediaStreamTrack would ask for the source to send a new frame when a new sink is added. This is a completely Chrome side solution, and doesn't cause any state changes in MediaStream or blink. The initial CL can be seen on https://codereview.chromium.org/1829193003/. Let me know WDYT.
Project Member

Comment 1 by bugdroid1@chromium.org, Mar 26 2016 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/dbe511aa9eea169722b239f82174850df2185eff

commit dbe511aa9eea169722b239f82174850df2185eff
Author: emircan <emircan@chromium.org>
Date: Sat Mar 26 00:46:01 2016

Request frame for new sinks from MediaStreamVideoTrack

This CL adds a series of RequstFrame() calls propagated from
MediaStreamVideoTrack when a new sink is added.
MediaStreamVideoTrack
-> MediaStreamVideoCapturerSource which is a MediaStreamVideoSource
-> media::VideoCapturerSource

This addresses the issues when a new sink is added but doesn't
receive a frame because it wasn't added on time.

BUG= 597876 ,  587789 ,  486274 

Review URL: https://codereview.chromium.org/1829193003

Cr-Commit-Position: refs/heads/master@{#383421}

[modify] https://crrev.com/dbe511aa9eea169722b239f82174850df2185eff/content/renderer/media/canvas_capture_handler.cc
[modify] https://crrev.com/dbe511aa9eea169722b239f82174850df2185eff/content/renderer/media/canvas_capture_handler.h
[modify] https://crrev.com/dbe511aa9eea169722b239f82174850df2185eff/content/renderer/media/media_stream_video_capturer_source.cc
[modify] https://crrev.com/dbe511aa9eea169722b239f82174850df2185eff/content/renderer/media/media_stream_video_capturer_source.h
[modify] https://crrev.com/dbe511aa9eea169722b239f82174850df2185eff/content/renderer/media/media_stream_video_capturer_source_unittest.cc
[modify] https://crrev.com/dbe511aa9eea169722b239f82174850df2185eff/content/renderer/media/media_stream_video_source.h
[modify] https://crrev.com/dbe511aa9eea169722b239f82174850df2185eff/content/renderer/media/media_stream_video_track.cc
[modify] https://crrev.com/dbe511aa9eea169722b239f82174850df2185eff/content/renderer/media/media_stream_video_track_unittest.cc
[modify] https://crrev.com/dbe511aa9eea169722b239f82174850df2185eff/content/renderer/media/mock_media_stream_video_source.cc
[modify] https://crrev.com/dbe511aa9eea169722b239f82174850df2185eff/content/renderer/media/mock_media_stream_video_source.h
[modify] https://crrev.com/dbe511aa9eea169722b239f82174850df2185eff/media/base/video_capturer_source.h

Comment 2 by emir...@chromium.org, Mar 28 2016

Status: Fixed (was: Started)

Comment 3 by phil...@opera.com, Mar 29 2016 

Asking here since there's a good change it'll be lost in the review: Is there an issue to make the matching change to the spec? If this CL does what I think it does, it's the opposite of how I thought https://github.com/w3c/mediacapture-fromelement/issues/29 was going to be resolved.

Comment 4 by emir...@chromium.org, Mar 29 2016 

Note that this doesn't hold onto last frame in MediaStreamTrack or change state. It propagates RequestRefreshFrame() calls down to the Source when a new Sink is added, and Source decides to send a refresh frame or not. It was a part of the plan on  issue 486274  about tab capture. I decided to take the same approach to canvas capture so that it solves interoperability issues with Firefox in those examples and we don't have the side effects of holding onto the last frame in the MediaStreamTrack. 

I think this solution is the ideal to address at this point, unless MediaStream holding onto frame is defined in the spec and Firefox or Chrome changes implementation. I can easily change RequestRefreshFrame() overrides in canvas capture to change the behavior then.

There isn't an issue to make the change in the spec, since AFAIK there isn't any description of MediaStreamTrack behavior, holding onto frames or adding sinks on mediacapture-main spec. I am not sure if tab capture follows a spec, maybe miu@ can help on that.
►

Sign in to add a comment