New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 249427 link

Starred by 52 users

Issue metadata

Status: Fixed
Owner:
Last visit > 30 days ago
Closed: Sep 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: All
Pri: 2
Type: Bug



Sign in to add a comment

Implement HTMLMediaElement.audioTracks & HTMLMediaElement.videoTracks

Project Member Reported by acolwell@chromium.org, Jun 13 2013

Issue description

Placeholder bug for all changes related to adding support for the HTMLMediaElement.audioTracks & HTMLMediaElement.videoTracks attributes, associated objects, and track selection.
 
Blocking: chromium:249428

Comment 2 by ycheo@chromium.org, Nov 6 2013

@scherkus,
Do you have any plans or timeline on this features?
This is also one of items that Android TV partners are interested in.
ycheo: no timeline yet

it seems like you have a few requirements -- perhaps it'd be better to discuss offline to understand the bigger picture
 Issue 319991  has been merged into this issue.
I'm interested in doing this, see the description on  Issue 319991 .
This is going to be a pretty big patch, so I'm trying to do some refactoring first to reduce the size of the final patch. The first two are completely independent, so they can be reviewed in any order.

Renaming addTrack and removeTrack functions to addTextTrack and removeTextTrack: https://codereview.chromium.org/92763002

Move as much code as possible into TrackBase: https://codereview.chromium.org/92783002
I want to refactor a bunch of code into a TrackListBase, but I'm waiting on a review of the "change" and "removetrack" events before doing that: https://codereview.chromium.org/83073006/
I think I have the Blink patch ready, but it's not really testable without code in Chromium (and won't apply until I can get the other patches accepted). The Chromium side will be a bit more interesting, since I can't just create something new, since there's already VideoDecoderConfig.
I'm just waiting on those patches to get an OWNERS lgtm..
Which patches are you referring to? I'm a core OWNER now in Blink so I should be help them along if they are stalled.
The two you already reviewed. Should I be able to just commit them now since you already wrote lgtm?
You should be able to land https://codereview.chromium.org/92783002 now since that only has changes in Source/core and my OWNERS powers apply there.

For https://codereview.chromium.org/92763002/ , you'll need an OWNERS review from abarth, eseidel, or jamesr for the Source/platform & Source/web changes.
Ok, I'll rebase them to make sure they still apply. I'm not sure how to get more attention on that one that still needs another reviewer..
Labels: M-35
Owner: acolwell@chromium.org
Status: Assigned
Project Member

Comment 16 by bugdroid1@chromium.org, Feb 25 2014

The following revision refers to this bug:
    http://src.chromium.org/viewvc/blink?view=rev&rev=167826

------------------------------------------------------------------------
r167826 | acolwell@chromium.org | 2014-02-25T21:40:41.603685Z

Changed paths:
   M http://src.chromium.org/viewvc/blink/trunk/Source/platform/graphics/media/MediaPlayer.h?r1=167826&r2=167825&pathrev=167826
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/html/track/TextTrackList.cpp?r1=167826&r2=167825&pathrev=167826
   M http://src.chromium.org/viewvc/blink/trunk/Source/web/WebMediaPlayerClientImpl.cpp?r1=167826&r2=167825&pathrev=167826
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/html/HTMLMediaElement.cpp?r1=167826&r2=167825&pathrev=167826
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/html/track/TextTrackList.h?r1=167826&r2=167825&pathrev=167826
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/html/HTMLMediaElement.h?r1=167826&r2=167825&pathrev=167826
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/html/HTMLTrackElement.cpp?r1=167826&r2=167825&pathrev=167826

Rename text track methods to make way for audio & video tracks.

- Rename various text track related methods to explicitly indicate that
  they apply to text tracks. This is to avoid confusion when audio &
  video track logic is landed.

- Renamed TextTrackList::clearOwner() to clearOwnerAndClients() and moved
  some the client clearing out of HTMLMediaElement and into TextTrackList.
  This doesn't introduce any change in behavior. It just restores some
  encapsulation.

BUG= 249427 
TEST=All LayoutTests still pass.

Review URL: https://codereview.chromium.org/180173002
------------------------------------------------------------------------
Project Member

Comment 17 by bugdroid1@chromium.org, Feb 26 2014

The following revision refers to this bug:
    http://src.chromium.org/viewvc/blink?view=rev&rev=167841

------------------------------------------------------------------------
r167841 | haraken@chromium.org | 2014-02-26T01:22:52.283546Z

Changed paths:
   M http://src.chromium.org/viewvc/blink/trunk/Source/platform/graphics/media/MediaPlayer.h?r1=167841&r2=167840&pathrev=167841
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/html/track/TextTrackList.cpp?r1=167841&r2=167840&pathrev=167841
   M http://src.chromium.org/viewvc/blink/trunk/Source/web/WebMediaPlayerClientImpl.cpp?r1=167841&r2=167840&pathrev=167841
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/html/HTMLMediaElement.cpp?r1=167841&r2=167840&pathrev=167841
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/html/track/TextTrackList.h?r1=167841&r2=167840&pathrev=167841
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/html/HTMLMediaElement.h?r1=167841&r2=167840&pathrev=167841
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/html/HTMLTrackElement.cpp?r1=167841&r2=167840&pathrev=167841

Revert 167826 "Rename text track methods to make way for audio &..."

This CL broke a lot of Media tests and Track tests in Linux bots.
http://build.chromium.org/p/chromium.webkit/builders/WebKit%20Linux%2032

> Rename text track methods to make way for audio & video tracks.
> 
> - Rename various text track related methods to explicitly indicate that
>   they apply to text tracks. This is to avoid confusion when audio &
>   video track logic is landed.
> 
> - Renamed TextTrackList::clearOwner() to clearOwnerAndClients() and moved
>   some the client clearing out of HTMLMediaElement and into TextTrackList.
>   This doesn't introduce any change in behavior. It just restores some
>   encapsulation.
> 
> BUG= 249427 
> TEST=All LayoutTests still pass.
> 
> Review URL: https://codereview.chromium.org/180173002

TBR=acolwell@chromium.org

Review URL: https://codereview.chromium.org/177213015
------------------------------------------------------------------------
Project Member

Comment 18 by bugdroid1@chromium.org, Feb 26 2014

The following revision refers to this bug:
    http://src.chromium.org/viewvc/blink?view=rev&rev=167841

------------------------------------------------------------------------
r167841 | haraken@chromium.org | 2014-02-26T01:22:52.283546Z

Changed paths:
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/html/HTMLMediaElement.h?r1=167841&r2=167840&pathrev=167841
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/html/HTMLTrackElement.cpp?r1=167841&r2=167840&pathrev=167841
   M http://src.chromium.org/viewvc/blink/trunk/Source/platform/graphics/media/MediaPlayer.h?r1=167841&r2=167840&pathrev=167841
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/html/track/TextTrackList.cpp?r1=167841&r2=167840&pathrev=167841
   M http://src.chromium.org/viewvc/blink/trunk/Source/web/WebMediaPlayerClientImpl.cpp?r1=167841&r2=167840&pathrev=167841
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/html/HTMLMediaElement.cpp?r1=167841&r2=167840&pathrev=167841
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/html/track/TextTrackList.h?r1=167841&r2=167840&pathrev=167841

Revert 167826 "Rename text track methods to make way for audio &..."

This CL broke a lot of Media tests and Track tests in Linux bots.
http://build.chromium.org/p/chromium.webkit/builders/WebKit%20Linux%2032

> Rename text track methods to make way for audio & video tracks.
> 
> - Rename various text track related methods to explicitly indicate that
>   they apply to text tracks. This is to avoid confusion when audio &
>   video track logic is landed.
> 
> - Renamed TextTrackList::clearOwner() to clearOwnerAndClients() and moved
>   some the client clearing out of HTMLMediaElement and into TextTrackList.
>   This doesn't introduce any change in behavior. It just restores some
>   encapsulation.
> 
> BUG= 249427 
> TEST=All LayoutTests still pass.
> 
> Review URL: https://codereview.chromium.org/180173002

TBR=acolwell@chromium.org

Review URL: https://codereview.chromium.org/177213015
------------------------------------------------------------------------
Project Member

Comment 19 by bugdroid1@chromium.org, Feb 26 2014

The following revision refers to this bug:
    http://src.chromium.org/viewvc/blink?view=rev&rev=167842

------------------------------------------------------------------------
r167842 | haraken@chromium.org | 2014-02-26T01:32:27.165667Z

Changed paths:
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/html/HTMLMediaElement.h?r1=167842&r2=167841&pathrev=167842
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/html/HTMLTrackElement.cpp?r1=167842&r2=167841&pathrev=167842
   M http://src.chromium.org/viewvc/blink/trunk/Source/platform/graphics/media/MediaPlayer.h?r1=167842&r2=167841&pathrev=167842
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/html/track/TextTrackList.cpp?r1=167842&r2=167841&pathrev=167842
   M http://src.chromium.org/viewvc/blink/trunk/Source/web/WebMediaPlayerClientImpl.cpp?r1=167842&r2=167841&pathrev=167842
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/html/HTMLMediaElement.cpp?r1=167842&r2=167841&pathrev=167842
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/html/track/TextTrackList.h?r1=167842&r2=167841&pathrev=167842

Revert 167841 "Revert 167826 "Rename text track methods to make ..."

This is a revert of the revert. r167826 was not a culprit of the test breakage in Linux bots.

> Revert 167826 "Rename text track methods to make way for audio &..."
> 
> This CL broke a lot of Media tests and Track tests in Linux bots.
> http://build.chromium.org/p/chromium.webkit/builders/WebKit%20Linux%2032
> 
> > Rename text track methods to make way for audio & video tracks.
> > 
> > - Rename various text track related methods to explicitly indicate that
> >   they apply to text tracks. This is to avoid confusion when audio &
> >   video track logic is landed.
> > 
> > - Renamed TextTrackList::clearOwner() to clearOwnerAndClients() and moved
> >   some the client clearing out of HTMLMediaElement and into TextTrackList.
> >   This doesn't introduce any change in behavior. It just restores some
> >   encapsulation.
> > 
> > BUG= 249427 
> > TEST=All LayoutTests still pass.
> > 
> > Review URL: https://codereview.chromium.org/180173002
> 
> TBR=acolwell@chromium.org
> 
> Review URL: https://codereview.chromium.org/177213015

TBR=haraken@chromium.org

Review URL: https://codereview.chromium.org/176943005
------------------------------------------------------------------------
Project Member

Comment 20 by bugdroid1@chromium.org, Feb 26 2014

The following revision refers to this bug:
    http://src.chromium.org/viewvc/blink?view=rev&rev=167842

------------------------------------------------------------------------
r167842 | haraken@chromium.org | 2014-02-26T01:32:27.165667Z

Changed paths:
   M http://src.chromium.org/viewvc/blink/trunk/Source/web/WebMediaPlayerClientImpl.cpp?r1=167842&r2=167841&pathrev=167842
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/html/HTMLMediaElement.cpp?r1=167842&r2=167841&pathrev=167842
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/html/track/TextTrackList.h?r1=167842&r2=167841&pathrev=167842
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/html/HTMLMediaElement.h?r1=167842&r2=167841&pathrev=167842
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/html/HTMLTrackElement.cpp?r1=167842&r2=167841&pathrev=167842
   M http://src.chromium.org/viewvc/blink/trunk/Source/platform/graphics/media/MediaPlayer.h?r1=167842&r2=167841&pathrev=167842
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/html/track/TextTrackList.cpp?r1=167842&r2=167841&pathrev=167842

Revert 167841 "Revert 167826 "Rename text track methods to make ..."

This is a revert of the revert. r167826 was not a culprit of the test breakage in Linux bots.

> Revert 167826 "Rename text track methods to make way for audio &..."
> 
> This CL broke a lot of Media tests and Track tests in Linux bots.
> http://build.chromium.org/p/chromium.webkit/builders/WebKit%20Linux%2032
> 
> > Rename text track methods to make way for audio & video tracks.
> > 
> > - Rename various text track related methods to explicitly indicate that
> >   they apply to text tracks. This is to avoid confusion when audio &
> >   video track logic is landed.
> > 
> > - Renamed TextTrackList::clearOwner() to clearOwnerAndClients() and moved
> >   some the client clearing out of HTMLMediaElement and into TextTrackList.
> >   This doesn't introduce any change in behavior. It just restores some
> >   encapsulation.
> > 
> > BUG= 249427 
> > TEST=All LayoutTests still pass.
> > 
> > Review URL: https://codereview.chromium.org/180173002
> 
> TBR=acolwell@chromium.org
> 
> Review URL: https://codereview.chromium.org/177213015

TBR=haraken@chromium.org

Review URL: https://codereview.chromium.org/176943005
------------------------------------------------------------------------
I won't be home for a couple weeks, but I should be able to look through the test failures and fix them in the 10th or so.
Project Member

Comment 22 by bugdroid1@chromium.org, Mar 1 2014

The following revision refers to this bug:
    http://src.chromium.org/viewvc/blink?view=rev&rev=168176

------------------------------------------------------------------------
r168176 | acolwell@chromium.org | 2014-03-01T01:06:13.256325Z

Changed paths:
   A http://src.chromium.org/viewvc/blink/trunk/LayoutTests/media/content/test-vp8-vorbis-webvtt.webm?r1=168176&r2=168175&pathrev=168176

Add media file with inband text tracks for testing.

BUG= 249427 
TBR=

Review URL: https://codereview.chromium.org/184113008
------------------------------------------------------------------------
Project Member

Comment 23 by bugdroid1@chromium.org, Mar 4 2014

The following revision refers to this bug:
    http://src.chromium.org/viewvc/blink?view=rev&rev=168357

------------------------------------------------------------------------
r168357 | acolwell@chromium.org | 2014-03-04T04:04:38.109345Z

Changed paths:
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/html/track/TextTrack.h?r1=168357&r2=168356&pathrev=168357
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/html/HTMLMediaElement.h?r1=168357&r2=168356&pathrev=168357
   D http://src.chromium.org/viewvc/blink/trunk/LayoutTests/media/track/track-remove-track-expected.txt?r1=168357&r2=168356&pathrev=168357
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/html/track/InbandTextTrack.cpp?r1=168357&r2=168356&pathrev=168357
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/html/track/TextTrackList.cpp?r1=168357&r2=168356&pathrev=168357
   M http://src.chromium.org/viewvc/blink/trunk/LayoutTests/media/track/track-remove-track.html?r1=168357&r2=168356&pathrev=168357
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/html/track/TextTrack.cpp?r1=168357&r2=168356&pathrev=168357
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/html/HTMLMediaElement.cpp?r1=168357&r2=168356&pathrev=168357
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/html/track/InbandTextTrack.h?r1=168357&r2=168356&pathrev=168357
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/html/track/TextTrackList.h?r1=168357&r2=168356&pathrev=168357

Prevent 'removetrack' events from firing when all inband text tracks are removed.

This patch fixes a bug where 'removetrack' events were being fired
when all the inband tracks were being removed. This behavior was trying
to implement the "forget the media element's media-resource-specific
tracks" algorithm, but it wasn't correct.

This patch includes the following changes:
- Renames removeAllInbandTracks() to forgetResourceSpecificTracks()
- Reorganized logic in prepareForLoad() to match the spec order. The
  shuffling should not have any user visible behaviour change.
- Added forgetResourceSpecificTracks() calls to other places that
  the spec indicates it should run.
- Added code to make sure forgetResourceSpecificTracks() doesn't
  trigger 'removetrack' events.
- Converted the existing media/track/track-remove-track.html test
  to W3C test & added a new test case that verifies that the event
  doesn't fire.

BUG= 249427 
TEST=Existing LayoutTests still pass. Added a new case to media/track/track-remove-track.html

Review URL: https://codereview.chromium.org/177243018
------------------------------------------------------------------------
Project Member

Comment 24 by bugdroid1@chromium.org, Mar 5 2014

The following revision refers to this bug:
    http://src.chromium.org/viewvc/blink?view=rev&rev=168534

------------------------------------------------------------------------
r168534 | acolwell@chromium.org | 2014-03-05T21:15:52.695991Z

Changed paths:
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/html/track/TextTrack.cpp?r1=168534&r2=168533&pathrev=168534
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/html/HTMLMediaElement.cpp?r1=168534&r2=168533&pathrev=168534
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/html/track/InbandTextTrack.h?r1=168534&r2=168533&pathrev=168534
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/html/track/TextTrackList.h?r1=168534&r2=168533&pathrev=168534
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/html/track/LoadableTextTrack.cpp?r1=168534&r2=168533&pathrev=168534
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/html/track/TextTrack.h?r1=168534&r2=168533&pathrev=168534
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/html/HTMLMediaElement.h?r1=168534&r2=168533&pathrev=168534
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/html/HTMLTrackElement.cpp?r1=168534&r2=168533&pathrev=168534
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/html/track/LoadableTextTrack.h?r1=168534&r2=168533&pathrev=168534
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/html/HTMLTrackElement.h?r1=168534&r2=168533&pathrev=168534
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/html/track/InbandTextTrack.cpp?r1=168534&r2=168533&pathrev=168534
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/html/track/TextTrackList.cpp?r1=168534&r2=168533&pathrev=168534

Remove TextTrackClient interface to simplify TextTrack code.

Removing the TextTrackClient interface because it was obscuring calls
made to the HTMLMediaElement and wasn't really providing much value.
Calls made to the HTMLMediaElement are much clearer now and it is
easier to follow what happens when tracks are removed from the
TextTrackList.

BUG= 249427 
TEST=All existing LayoutTests still pass
R=adamk@chromium.org

Review URL: https://codereview.chromium.org/177003025
------------------------------------------------------------------------
I have a patch that adds AudioTrack/VideoTrack/AudioTrackList/VideoTrackList almost ready, just waiting for some builds to finish. Unfortunately it's impossible to make layout tests without the patch on the media player side.
I have a patch too https://codereview.chromium.org/170233009/ that I was planning on rebasing now that these TextTrack changes have landed. I created some simply layout tests that verify event behavior to start with. This is pretty straightforward w/o the need for Chomium-side changes initially.

Comment 27 by kareng@google.com, Apr 7 2014

Labels: -M-35 MovedFrom-35 M-36
Moving all non essential bugs to the next Milestone.
Labels: -M-36 MovedFrom-36
This issue has already been moved once and is lower than Priority 1,therefore removing mstone.
Project Member

Comment 29 by bugdroid1@chromium.org, Jun 16 2014

The following revision refers to this bug:
  http://src.chromium.org/viewvc/blink?view=rev&rev=176224

------------------------------------------------------------------
r176224 | philipj@opera.com | 2014-06-16T17:04:37.453461Z

Changed paths:
   A http://src.chromium.org/viewvc/blink/trunk/LayoutTests/media/avtrack?r1=176224&r2=176223&pathrev=176224
   A http://src.chromium.org/viewvc/blink/trunk/LayoutTests/media/avtrack/video-track-selected.html?r1=176224&r2=176223&pathrev=176224
   A http://src.chromium.org/viewvc/blink/trunk/Source/core/html/track/VideoTrackList.h?r1=176224&r2=176223&pathrev=176224
   M http://src.chromium.org/viewvc/blink/trunk/Source/web/WebMediaPlayerClientImpl.h?r1=176224&r2=176223&pathrev=176224
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/html/track/TrackBase.cpp?r1=176224&r2=176223&pathrev=176224
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/html/HTMLMediaElement.idl?r1=176224&r2=176223&pathrev=176224
   A http://src.chromium.org/viewvc/blink/trunk/Source/core/html/track/VideoTrack.h?r1=176224&r2=176223&pathrev=176224
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/events/EventTargetFactory.in?r1=176224&r2=176223&pathrev=176224
   M http://src.chromium.org/viewvc/blink/trunk/public/platform/WebMediaPlayerClient.h?r1=176224&r2=176223&pathrev=176224
   M http://src.chromium.org/viewvc/blink/trunk/Source/platform/RuntimeEnabledFeatures.in?r1=176224&r2=176223&pathrev=176224
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/core.gypi?r1=176224&r2=176223&pathrev=176224
   A http://src.chromium.org/viewvc/blink/trunk/LayoutTests/media/avtrack/forget-on-load.html?r1=176224&r2=176223&pathrev=176224
   A http://src.chromium.org/viewvc/blink/trunk/LayoutTests/media/avtrack/addtrack.html?r1=176224&r2=176223&pathrev=176224
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/html/HTMLMediaElement.cpp?r1=176224&r2=176223&pathrev=176224
   A http://src.chromium.org/viewvc/blink/trunk/Source/core/html/track/VideoTrackList.idl?r1=176224&r2=176223&pathrev=176224
   M http://src.chromium.org/viewvc/blink/trunk/LayoutTests/webexposed/global-constructors-listing-expected.txt?r1=176224&r2=176223&pathrev=176224
   A http://src.chromium.org/viewvc/blink/trunk/Source/core/html/track/VideoTrack.idl?r1=176224&r2=176223&pathrev=176224
   A http://src.chromium.org/viewvc/blink/trunk/Source/core/html/track/VideoTrackList.cpp?r1=176224&r2=176223&pathrev=176224
   A http://src.chromium.org/viewvc/blink/trunk/Source/core/html/track/AudioTrackList.h?r1=176224&r2=176223&pathrev=176224
   M http://src.chromium.org/viewvc/blink/trunk/Source/web/WebMediaPlayerClientImpl.cpp?r1=176224&r2=176223&pathrev=176224
   A http://src.chromium.org/viewvc/blink/trunk/Source/core/html/track/VideoTrack.cpp?r1=176224&r2=176223&pathrev=176224
   A http://src.chromium.org/viewvc/blink/trunk/Source/core/html/track/AudioTrack.h?r1=176224&r2=176223&pathrev=176224
   A http://src.chromium.org/viewvc/blink/trunk/Source/core/html/track/TrackListBase.h?r1=176224&r2=176223&pathrev=176224
   M http://src.chromium.org/viewvc/blink/trunk/public/platform/WebMediaPlayer.h?r1=176224&r2=176223&pathrev=176224
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/html/track/TrackBase.h?r1=176224&r2=176223&pathrev=176224
   A http://src.chromium.org/viewvc/blink/trunk/Source/core/html/track/AudioTrackList.idl?r1=176224&r2=176223&pathrev=176224
   A http://src.chromium.org/viewvc/blink/trunk/LayoutTests/media/avtrack/getTrackById.html?r1=176224&r2=176223&pathrev=176224
   A http://src.chromium.org/viewvc/blink/trunk/Source/core/html/track/AudioTrack.idl?r1=176224&r2=176223&pathrev=176224
   M http://src.chromium.org/viewvc/blink/trunk/Source/core/html/HTMLMediaElement.h?r1=176224&r2=176223&pathrev=176224
   A http://src.chromium.org/viewvc/blink/trunk/Source/core/html/track/AudioTrackList.cpp?r1=176224&r2=176223&pathrev=176224
   M http://src.chromium.org/viewvc/blink/trunk/Source/bindings/v8/custom/V8TrackEventCustom.cpp?r1=176224&r2=176223&pathrev=176224
   A http://src.chromium.org/viewvc/blink/trunk/LayoutTests/media/avtrack/gc.html?r1=176224&r2=176223&pathrev=176224
   A http://src.chromium.org/viewvc/blink/trunk/Source/core/html/track/AudioTrack.cpp?r1=176224&r2=176223&pathrev=176224
   A http://src.chromium.org/viewvc/blink/trunk/LayoutTests/media/avtrack/audio-track-enabled.html?r1=176224&r2=176223&pathrev=176224

Implement AudioTrack, AudioTrackList, VideoTrack, and VideoTrackList

This is based on Aaron Colwell's patch:
https://codereview.chromium.org/170233009/#ps270001

BUG= 249427 
TEST=LayoutTests/media/avtracklists.html

Review URL: https://codereview.chromium.org/284513003
-----------------------------------------------------------------

Comment 30 by phil...@opera.com, Jun 17 2014

Aaron, is there anything else you wanted to fix in this bug, or can we close it?

Comment 31 by phil...@opera.com, Jun 17 2014

Aaron, is there anything else you wanted to fix in this bug, or can we close it?
Philip, I was planning on leaving it open until the Chromium-side media engine changes land. We'll take care of that. Thanks for your help on this.
This patch is not complete.
I can not enable or disable audio tracks in Video 
Blocking: -chromium:249428
Owner: wolenetz@chromium.org
Assigning to wolenetz@ since this will likely need to be addressed as part of the MSE tracks work.

Comment 36 by Deleted ...@, Jan 7 2015

Hi, we are working on a project which involves multi-language audio files.

We have an MP4 file with four encoded audio tracks. You can download this file at http://www.stuart-pinfold.co.uk/videotest/test.mp4 - play it in VLC, right-click and go to Audio > Audio Track, and you can change language.

Compare this page in Internet Explorer (version 10 or 11 only) with even the latest version of Chrome/Chromium:
http://www.stuart-pinfold.co.uk/videotest/

IN INTERNET EXPLORER:
- The four links to change language all work
- There is even a button in the HTML5 player itself to switch audio tracks

IN CHROME/CHROMIUM:
- Clicking immediately to play the video gives you English
- Clicking any of the languages results in silence and even clicking English does not re-activate the first audio track (English).
- There is no audio-selection tool in the video, even in the context menu.

Is this just a feature that Chrome/Chromium hasn't yet implemented (surprising, given IE10's release date in 2012) or does Chrome/Chromium interpret this video differently and thus our code needs to be changed?

Any help or advice appreciated!

Stuart

Comment 37 by phil...@opera.com, Jan 20 2015

Hi Stuart, I'm afraid Chromium hasn't shipped this feature yet, it's behind an experimental flag. Even if enabled, the necessary work in the backend hasn't been done to make it work yet. Following this bug is your best bet for finding out when it becomes available.
 * Track-related bugs:
  * crb/249428 (MSE desktop SourceBuffer.audio/video/textTracks)
  * (this bug) crb/249427 (desktop src=+MSE desktop HTMLME.audio/videoTracks)
  * crb/322115 (android src=+MSE textTracks)
  * new: crb/457379 (android src=+MSE audio/videoTracks)

Comment 39 by Deleted ...@, Jun 15 2015

any progress with this?
We have only made minimal recent progress on MSE side. More is in the works as MSE gets closer to showing interop-implementability to reach the next w3c spec phase (public rec).

Comment 41 by Deleted ...@, Jun 23 2015

Thanks for the replay. 
Do you have any suggestion in which chrome version we can expect this - 44,45,46 or later?

Comment 42 by sark...@gmail.com, Jan 18 2016

Has there been any progress on this? Both IE and Safari have implementations ready. An estimation on when we can expect this would be useful.
We're in the planning phase at the moment, potentially including work on fixing this. I'll update when a more clear estimation is available.
Project Member

Comment 44 by bugdroid1@chromium.org, Mar 1 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/95dc4ad373d2aef14c9b439397f1a4599b64bd6a

commit 95dc4ad373d2aef14c9b439397f1a4599b64bd6a
Author: servolk <servolk@chromium.org>
Date: Tue Mar 01 22:44:40 2016

Basic media tracks implementation for media stream parsers

This CL adds basic media::MediaTrack implementation, media::MediaTracks
collection that associates media tracks with audio/video configs, and
makes necessary changes in MSE stream parsers to pass around the
MediaTracks collection instead of individual audio/video configs.

BUG= 249427 ,  249428 

Review URL: https://codereview.chromium.org/1716503002

Cr-Commit-Position: refs/heads/master@{#378583}

[modify] https://crrev.com/95dc4ad373d2aef14c9b439397f1a4599b64bd6a/media/base/BUILD.gn
[add] https://crrev.com/95dc4ad373d2aef14c9b439397f1a4599b64bd6a/media/base/media_track.cc
[add] https://crrev.com/95dc4ad373d2aef14c9b439397f1a4599b64bd6a/media/base/media_track.h
[add] https://crrev.com/95dc4ad373d2aef14c9b439397f1a4599b64bd6a/media/base/media_tracks.cc
[add] https://crrev.com/95dc4ad373d2aef14c9b439397f1a4599b64bd6a/media/base/media_tracks.h
[modify] https://crrev.com/95dc4ad373d2aef14c9b439397f1a4599b64bd6a/media/base/stream_parser.h
[modify] https://crrev.com/95dc4ad373d2aef14c9b439397f1a4599b64bd6a/media/blink/websourcebuffer_impl.cc
[modify] https://crrev.com/95dc4ad373d2aef14c9b439397f1a4599b64bd6a/media/blink/websourcebuffer_impl.h
[modify] https://crrev.com/95dc4ad373d2aef14c9b439397f1a4599b64bd6a/media/filters/chunk_demuxer_unittest.cc
[modify] https://crrev.com/95dc4ad373d2aef14c9b439397f1a4599b64bd6a/media/filters/media_source_state.cc
[modify] https://crrev.com/95dc4ad373d2aef14c9b439397f1a4599b64bd6a/media/filters/media_source_state.h
[modify] https://crrev.com/95dc4ad373d2aef14c9b439397f1a4599b64bd6a/media/formats/common/stream_parser_test_base.cc
[modify] https://crrev.com/95dc4ad373d2aef14c9b439397f1a4599b64bd6a/media/formats/common/stream_parser_test_base.h
[modify] https://crrev.com/95dc4ad373d2aef14c9b439397f1a4599b64bd6a/media/formats/mp2t/mp2t_stream_parser.cc
[modify] https://crrev.com/95dc4ad373d2aef14c9b439397f1a4599b64bd6a/media/formats/mp2t/mp2t_stream_parser_unittest.cc
[modify] https://crrev.com/95dc4ad373d2aef14c9b439397f1a4599b64bd6a/media/formats/mp4/mp4_stream_parser.cc
[modify] https://crrev.com/95dc4ad373d2aef14c9b439397f1a4599b64bd6a/media/formats/mp4/mp4_stream_parser_unittest.cc
[modify] https://crrev.com/95dc4ad373d2aef14c9b439397f1a4599b64bd6a/media/formats/mpeg/mpeg_audio_stream_parser_base.cc
[modify] https://crrev.com/95dc4ad373d2aef14c9b439397f1a4599b64bd6a/media/formats/webm/webm_stream_parser.cc
[modify] https://crrev.com/95dc4ad373d2aef14c9b439397f1a4599b64bd6a/media/media.gyp
[modify] https://crrev.com/95dc4ad373d2aef14c9b439397f1a4599b64bd6a/media/test/pipeline_integration_test.cc

Project Member

Comment 45 by bugdroid1@chromium.org, Mar 2 2016

Labels: Merge-Merged-master1
The following revision refers to this bug:
  https://chrome-internal.googlesource.com/bling/chromium.git/+/95dc4ad373d2aef14c9b439397f1a4599b64bd6a

commit 95dc4ad373d2aef14c9b439397f1a4599b64bd6a
Author: servolk <servolk@chromium.org>
Date: Tue Mar 01 22:44:40 2016

Comment 46 by phil...@opera.com, Mar 10 2016

I just realized that there's no recent "Intent to Implement" for this. Bredan Long sent one, but it was so long ago that I expect people don't remember:
https://groups.google.com/a/chromium.org/d/msg/blink-dev/Jgg71B43w3Y/VDVaJltaIbkJ

Can someone send that to blink-dev? It doesn't need any LGTMs, but if anyone has feedback on the design that's a good place to discuss it.
Components: -Blink>Video Blink>Media>Video
Renaming Blink>Video to Blink>Media>Video for better characterization
Cc: servolk@chromium.org
Project Member

Comment 51 by bugdroid1@chromium.org, Mar 30 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/92333c0e5ca6b7768e5bcf6d41ff8fc733ebbfc1

commit 92333c0e5ca6b7768e5bcf6d41ff8fc733ebbfc1
Author: servolk <servolk@chromium.org>
Date: Wed Mar 30 17:13:46 2016

Pass MSE media track info from ChunkDemuxer to blink::SourceBuffer

Also moves TrackType enum definition from TrackBase to WebMediaPlayer

BUG= 249427 , 249428 

Review URL: https://codereview.chromium.org/1659653002

Cr-Commit-Position: refs/heads/master@{#384014}

[modify] https://crrev.com/92333c0e5ca6b7768e5bcf6d41ff8fc733ebbfc1/media/blink/webmediaplayer_impl.cc
[modify] https://crrev.com/92333c0e5ca6b7768e5bcf6d41ff8fc733ebbfc1/media/blink/websourcebuffer_impl.cc
[add] https://crrev.com/92333c0e5ca6b7768e5bcf6d41ff8fc733ebbfc1/third_party/WebKit/LayoutTests/http/tests/media/media-source/mediasource-avtracks.html
[modify] https://crrev.com/92333c0e5ca6b7768e5bcf6d41ff8fc733ebbfc1/third_party/WebKit/LayoutTests/webexposed/global-interface-listing-expected.txt
[modify] https://crrev.com/92333c0e5ca6b7768e5bcf6d41ff8fc733ebbfc1/third_party/WebKit/Source/core/html/track/AudioTrack.cpp
[modify] https://crrev.com/92333c0e5ca6b7768e5bcf6d41ff8fc733ebbfc1/third_party/WebKit/Source/core/html/track/AudioTrack.h
[modify] https://crrev.com/92333c0e5ca6b7768e5bcf6d41ff8fc733ebbfc1/third_party/WebKit/Source/core/html/track/AudioTrackList.h
[modify] https://crrev.com/92333c0e5ca6b7768e5bcf6d41ff8fc733ebbfc1/third_party/WebKit/Source/core/html/track/TextTrack.cpp
[modify] https://crrev.com/92333c0e5ca6b7768e5bcf6d41ff8fc733ebbfc1/third_party/WebKit/Source/core/html/track/TextTrack.h
[modify] https://crrev.com/92333c0e5ca6b7768e5bcf6d41ff8fc733ebbfc1/third_party/WebKit/Source/core/html/track/TextTrackList.h
[modify] https://crrev.com/92333c0e5ca6b7768e5bcf6d41ff8fc733ebbfc1/third_party/WebKit/Source/core/html/track/TrackBase.cpp
[modify] https://crrev.com/92333c0e5ca6b7768e5bcf6d41ff8fc733ebbfc1/third_party/WebKit/Source/core/html/track/TrackBase.h
[modify] https://crrev.com/92333c0e5ca6b7768e5bcf6d41ff8fc733ebbfc1/third_party/WebKit/Source/core/html/track/TrackEvent.cpp
[modify] https://crrev.com/92333c0e5ca6b7768e5bcf6d41ff8fc733ebbfc1/third_party/WebKit/Source/core/html/track/TrackEvent.h
[modify] https://crrev.com/92333c0e5ca6b7768e5bcf6d41ff8fc733ebbfc1/third_party/WebKit/Source/core/html/track/VideoTrack.cpp
[modify] https://crrev.com/92333c0e5ca6b7768e5bcf6d41ff8fc733ebbfc1/third_party/WebKit/Source/core/html/track/VideoTrack.h
[modify] https://crrev.com/92333c0e5ca6b7768e5bcf6d41ff8fc733ebbfc1/third_party/WebKit/Source/core/html/track/VideoTrackList.h
[modify] https://crrev.com/92333c0e5ca6b7768e5bcf6d41ff8fc733ebbfc1/third_party/WebKit/Source/modules/mediasource/SourceBuffer.cpp
[modify] https://crrev.com/92333c0e5ca6b7768e5bcf6d41ff8fc733ebbfc1/third_party/WebKit/Source/modules/mediasource/SourceBuffer.h
[modify] https://crrev.com/92333c0e5ca6b7768e5bcf6d41ff8fc733ebbfc1/third_party/WebKit/Source/modules/mediasource/SourceBuffer.idl
[modify] https://crrev.com/92333c0e5ca6b7768e5bcf6d41ff8fc733ebbfc1/third_party/WebKit/public/platform/WebMediaPlayer.h
[modify] https://crrev.com/92333c0e5ca6b7768e5bcf6d41ff8fc733ebbfc1/third_party/WebKit/public/platform/WebSourceBufferClient.h

Project Member

Comment 52 by bugdroid1@chromium.org, Mar 30 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/92333c0e5ca6b7768e5bcf6d41ff8fc733ebbfc1

commit 92333c0e5ca6b7768e5bcf6d41ff8fc733ebbfc1
Author: servolk <servolk@chromium.org>
Date: Wed Mar 30 17:13:46 2016

Pass MSE media track info from ChunkDemuxer to blink::SourceBuffer

Also moves TrackType enum definition from TrackBase to WebMediaPlayer

BUG= 249427 , 249428 

Review URL: https://codereview.chromium.org/1659653002

Cr-Commit-Position: refs/heads/master@{#384014}

[modify] https://crrev.com/92333c0e5ca6b7768e5bcf6d41ff8fc733ebbfc1/media/blink/webmediaplayer_impl.cc
[modify] https://crrev.com/92333c0e5ca6b7768e5bcf6d41ff8fc733ebbfc1/media/blink/websourcebuffer_impl.cc
[add] https://crrev.com/92333c0e5ca6b7768e5bcf6d41ff8fc733ebbfc1/third_party/WebKit/LayoutTests/http/tests/media/media-source/mediasource-avtracks.html
[modify] https://crrev.com/92333c0e5ca6b7768e5bcf6d41ff8fc733ebbfc1/third_party/WebKit/LayoutTests/webexposed/global-interface-listing-expected.txt
[modify] https://crrev.com/92333c0e5ca6b7768e5bcf6d41ff8fc733ebbfc1/third_party/WebKit/Source/core/html/track/AudioTrack.cpp
[modify] https://crrev.com/92333c0e5ca6b7768e5bcf6d41ff8fc733ebbfc1/third_party/WebKit/Source/core/html/track/AudioTrack.h
[modify] https://crrev.com/92333c0e5ca6b7768e5bcf6d41ff8fc733ebbfc1/third_party/WebKit/Source/core/html/track/AudioTrackList.h
[modify] https://crrev.com/92333c0e5ca6b7768e5bcf6d41ff8fc733ebbfc1/third_party/WebKit/Source/core/html/track/TextTrack.cpp
[modify] https://crrev.com/92333c0e5ca6b7768e5bcf6d41ff8fc733ebbfc1/third_party/WebKit/Source/core/html/track/TextTrack.h
[modify] https://crrev.com/92333c0e5ca6b7768e5bcf6d41ff8fc733ebbfc1/third_party/WebKit/Source/core/html/track/TextTrackList.h
[modify] https://crrev.com/92333c0e5ca6b7768e5bcf6d41ff8fc733ebbfc1/third_party/WebKit/Source/core/html/track/TrackBase.cpp
[modify] https://crrev.com/92333c0e5ca6b7768e5bcf6d41ff8fc733ebbfc1/third_party/WebKit/Source/core/html/track/TrackBase.h
[modify] https://crrev.com/92333c0e5ca6b7768e5bcf6d41ff8fc733ebbfc1/third_party/WebKit/Source/core/html/track/TrackEvent.cpp
[modify] https://crrev.com/92333c0e5ca6b7768e5bcf6d41ff8fc733ebbfc1/third_party/WebKit/Source/core/html/track/TrackEvent.h
[modify] https://crrev.com/92333c0e5ca6b7768e5bcf6d41ff8fc733ebbfc1/third_party/WebKit/Source/core/html/track/VideoTrack.cpp
[modify] https://crrev.com/92333c0e5ca6b7768e5bcf6d41ff8fc733ebbfc1/third_party/WebKit/Source/core/html/track/VideoTrack.h
[modify] https://crrev.com/92333c0e5ca6b7768e5bcf6d41ff8fc733ebbfc1/third_party/WebKit/Source/core/html/track/VideoTrackList.h
[modify] https://crrev.com/92333c0e5ca6b7768e5bcf6d41ff8fc733ebbfc1/third_party/WebKit/Source/modules/mediasource/SourceBuffer.cpp
[modify] https://crrev.com/92333c0e5ca6b7768e5bcf6d41ff8fc733ebbfc1/third_party/WebKit/Source/modules/mediasource/SourceBuffer.h
[modify] https://crrev.com/92333c0e5ca6b7768e5bcf6d41ff8fc733ebbfc1/third_party/WebKit/Source/modules/mediasource/SourceBuffer.idl
[modify] https://crrev.com/92333c0e5ca6b7768e5bcf6d41ff8fc733ebbfc1/third_party/WebKit/public/platform/WebMediaPlayer.h
[modify] https://crrev.com/92333c0e5ca6b7768e5bcf6d41ff8fc733ebbfc1/third_party/WebKit/public/platform/WebSourceBufferClient.h

Project Member

Comment 53 by bugdroid1@chromium.org, Apr 7 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/1b3dc5a8202a332891b6aed5d4d4521db0b26261

commit 1b3dc5a8202a332891b6aed5d4d4521db0b26261
Author: servolk <servolk@chromium.org>
Date: Thu Apr 07 18:31:05 2016

Add SourceBuffer implementations of Audio/VideoTracks

This CL provides SourceBuffer extensions described in MSE spec:
http://www.w3.org/TR/media-source/#audio-track-extensions
http://www.w3.org/TR/media-source/#video-track-extensions
The only difference between those extensions and the existing audio/
video track is that these have the .sourceBuffer property, which is
only going to be non-null for a/v tracks created by a SourceBuffer
and will return the parent SourceBuffer object.
I've looked at how other extension interfaces are implemented in blink,
e.g. NavigatorGeolocation extension property, and tried to do the same.

BUG= 249427 , 249428 

Review URL: https://codereview.chromium.org/1658033002

Cr-Commit-Position: refs/heads/master@{#385816}

[modify] https://crrev.com/1b3dc5a8202a332891b6aed5d4d4521db0b26261/third_party/WebKit/LayoutTests/http/tests/media/media-source/mediasource-avtracks.html
[modify] https://crrev.com/1b3dc5a8202a332891b6aed5d4d4521db0b26261/third_party/WebKit/LayoutTests/webexposed/global-interface-listing-expected.txt
[modify] https://crrev.com/1b3dc5a8202a332891b6aed5d4d4521db0b26261/third_party/WebKit/Source/core/core.gypi
[modify] https://crrev.com/1b3dc5a8202a332891b6aed5d4d4521db0b26261/third_party/WebKit/Source/core/html/track/TrackBase.cpp
[modify] https://crrev.com/1b3dc5a8202a332891b6aed5d4d4521db0b26261/third_party/WebKit/Source/core/html/track/TrackBase.h
[add] https://crrev.com/1b3dc5a8202a332891b6aed5d4d4521db0b26261/third_party/WebKit/Source/modules/mediasource/AudioTrackSourceBuffer.idl
[modify] https://crrev.com/1b3dc5a8202a332891b6aed5d4d4521db0b26261/third_party/WebKit/Source/modules/mediasource/SourceBuffer.cpp
[add] https://crrev.com/1b3dc5a8202a332891b6aed5d4d4521db0b26261/third_party/WebKit/Source/modules/mediasource/SourceBufferTrackBaseSupplement.cpp
[add] https://crrev.com/1b3dc5a8202a332891b6aed5d4d4521db0b26261/third_party/WebKit/Source/modules/mediasource/SourceBufferTrackBaseSupplement.h
[add] https://crrev.com/1b3dc5a8202a332891b6aed5d4d4521db0b26261/third_party/WebKit/Source/modules/mediasource/VideoTrackSourceBuffer.idl
[modify] https://crrev.com/1b3dc5a8202a332891b6aed5d4d4521db0b26261/third_party/WebKit/Source/modules/modules.gypi

Project Member

Comment 55 by bugdroid1@chromium.org, Apr 16 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/06e3a9ffca065ed73c1cd9e5d72f7ca2ee44f488

commit 06e3a9ffca065ed73c1cd9e5d72f7ca2ee44f488
Author: servolk <servolk@chromium.org>
Date: Sat Apr 16 02:20:10 2016

Allow muting/unmuting audio through media track API

This CL implements muting/unmuting audio via HTML5 media track APIs.
More specifically it adds basic plumbing to deliver media track state
changes from blink level to media::Renderer, by translating blink track
ids into DemuxerStream pointers in demuxer and then delivering
DemuxerStream state information to media renderers.  For now audio
stream is disabled by setting volume to 0, but this can be done more
efficiently in the future.

BUG= 249427 

Review URL: https://codereview.chromium.org/1812543003

Cr-Commit-Position: refs/heads/master@{#387789}

[modify] https://crrev.com/06e3a9ffca065ed73c1cd9e5d72f7ca2ee44f488/media/base/demuxer.h
[modify] https://crrev.com/06e3a9ffca065ed73c1cd9e5d72f7ca2ee44f488/media/base/media_tracks.cc
[modify] https://crrev.com/06e3a9ffca065ed73c1cd9e5d72f7ca2ee44f488/media/base/media_tracks.h
[modify] https://crrev.com/06e3a9ffca065ed73c1cd9e5d72f7ca2ee44f488/media/base/mock_filters.h
[modify] https://crrev.com/06e3a9ffca065ed73c1cd9e5d72f7ca2ee44f488/media/base/pipeline.h
[modify] https://crrev.com/06e3a9ffca065ed73c1cd9e5d72f7ca2ee44f488/media/base/pipeline_impl.cc
[modify] https://crrev.com/06e3a9ffca065ed73c1cd9e5d72f7ca2ee44f488/media/base/pipeline_impl.h
[modify] https://crrev.com/06e3a9ffca065ed73c1cd9e5d72f7ca2ee44f488/media/base/renderer.h
[modify] https://crrev.com/06e3a9ffca065ed73c1cd9e5d72f7ca2ee44f488/media/blink/webmediaplayer_impl.cc
[modify] https://crrev.com/06e3a9ffca065ed73c1cd9e5d72f7ca2ee44f488/media/blink/webmediaplayer_impl.h
[modify] https://crrev.com/06e3a9ffca065ed73c1cd9e5d72f7ca2ee44f488/media/blink/websourcebuffer_impl.cc
[modify] https://crrev.com/06e3a9ffca065ed73c1cd9e5d72f7ca2ee44f488/media/filters/chunk_demuxer.cc
[modify] https://crrev.com/06e3a9ffca065ed73c1cd9e5d72f7ca2ee44f488/media/filters/chunk_demuxer.h
[modify] https://crrev.com/06e3a9ffca065ed73c1cd9e5d72f7ca2ee44f488/media/filters/chunk_demuxer_unittest.cc
[modify] https://crrev.com/06e3a9ffca065ed73c1cd9e5d72f7ca2ee44f488/media/filters/ffmpeg_demuxer.cc
[modify] https://crrev.com/06e3a9ffca065ed73c1cd9e5d72f7ca2ee44f488/media/filters/ffmpeg_demuxer.h
[modify] https://crrev.com/06e3a9ffca065ed73c1cd9e5d72f7ca2ee44f488/media/filters/media_source_state.cc
[modify] https://crrev.com/06e3a9ffca065ed73c1cd9e5d72f7ca2ee44f488/media/filters/media_source_state.h
[modify] https://crrev.com/06e3a9ffca065ed73c1cd9e5d72f7ca2ee44f488/media/renderers/renderer_impl.cc
[modify] https://crrev.com/06e3a9ffca065ed73c1cd9e5d72f7ca2ee44f488/media/renderers/renderer_impl.h
[modify] https://crrev.com/06e3a9ffca065ed73c1cd9e5d72f7ca2ee44f488/media/test/pipeline_integration_test.cc
[modify] https://crrev.com/06e3a9ffca065ed73c1cd9e5d72f7ca2ee44f488/media/test/pipeline_integration_test_base.cc

Project Member

Comment 56 by bugdroid1@chromium.org, Apr 18 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/515ae317c22aafa3d6551ddf6871cc042050be6c

commit 515ae317c22aafa3d6551ddf6871cc042050be6c
Author: Per <perkj@chromium.org>
Date: Mon Apr 18 10:29:35 2016

Revert of Allow muting/unmuting audio through media track API (patchset #37 id:710001 of https://codereview.chromium.org/1812543003/ )

Reason for revert:
Tentative revert since it looks like it cause flake on

http/tests/media/media-source/mediasource-config-change-webm-av-audio-bitrate.html

https://build.chromium.org/p/chromium.webkit/builders/WebKit%20Mac10.11/builds/2433

https://build.chromium.org/p/chromium.webkit/builders/WebKit%20Win10?numbuilds=200

Original issue's description:
> Allow muting/unmuting audio through media track API
>
> This CL implements muting/unmuting audio via HTML5 media track APIs.
> More specifically it adds basic plumbing to deliver media track state
> changes from blink level to media::Renderer, by translating blink track
> ids into DemuxerStream pointers in demuxer and then delivering
> DemuxerStream state information to media renderers.  For now audio
> stream is disabled by setting volume to 0, but this can be done more
> efficiently in the future.
>
> BUG= 249427 
>
> Committed: https://crrev.com/06e3a9ffca065ed73c1cd9e5d72f7ca2ee44f488
> Cr-Commit-Position: refs/heads/master@{#387789}

TBR=wolenetz@chromium.org,servolk@chromium.org
# Not skipping CQ checks because original CL landed more than 1 days ago.
BUG= 249427 

Review URL: https://codereview.chromium.org/1894073002 .

Cr-Commit-Position: refs/heads/master@{#387883}

[modify] https://crrev.com/515ae317c22aafa3d6551ddf6871cc042050be6c/media/base/demuxer.h
[modify] https://crrev.com/515ae317c22aafa3d6551ddf6871cc042050be6c/media/base/media_tracks.cc
[modify] https://crrev.com/515ae317c22aafa3d6551ddf6871cc042050be6c/media/base/media_tracks.h
[modify] https://crrev.com/515ae317c22aafa3d6551ddf6871cc042050be6c/media/base/mock_filters.h
[modify] https://crrev.com/515ae317c22aafa3d6551ddf6871cc042050be6c/media/base/pipeline.h
[modify] https://crrev.com/515ae317c22aafa3d6551ddf6871cc042050be6c/media/base/pipeline_impl.cc
[modify] https://crrev.com/515ae317c22aafa3d6551ddf6871cc042050be6c/media/base/pipeline_impl.h
[modify] https://crrev.com/515ae317c22aafa3d6551ddf6871cc042050be6c/media/base/renderer.h
[modify] https://crrev.com/515ae317c22aafa3d6551ddf6871cc042050be6c/media/blink/webmediaplayer_impl.cc
[modify] https://crrev.com/515ae317c22aafa3d6551ddf6871cc042050be6c/media/blink/webmediaplayer_impl.h
[modify] https://crrev.com/515ae317c22aafa3d6551ddf6871cc042050be6c/media/blink/websourcebuffer_impl.cc
[modify] https://crrev.com/515ae317c22aafa3d6551ddf6871cc042050be6c/media/filters/chunk_demuxer.cc
[modify] https://crrev.com/515ae317c22aafa3d6551ddf6871cc042050be6c/media/filters/chunk_demuxer.h
[modify] https://crrev.com/515ae317c22aafa3d6551ddf6871cc042050be6c/media/filters/chunk_demuxer_unittest.cc
[modify] https://crrev.com/515ae317c22aafa3d6551ddf6871cc042050be6c/media/filters/ffmpeg_demuxer.cc
[modify] https://crrev.com/515ae317c22aafa3d6551ddf6871cc042050be6c/media/filters/ffmpeg_demuxer.h
[modify] https://crrev.com/515ae317c22aafa3d6551ddf6871cc042050be6c/media/filters/media_source_state.cc
[modify] https://crrev.com/515ae317c22aafa3d6551ddf6871cc042050be6c/media/filters/media_source_state.h
[modify] https://crrev.com/515ae317c22aafa3d6551ddf6871cc042050be6c/media/renderers/renderer_impl.cc
[modify] https://crrev.com/515ae317c22aafa3d6551ddf6871cc042050be6c/media/renderers/renderer_impl.h
[modify] https://crrev.com/515ae317c22aafa3d6551ddf6871cc042050be6c/media/test/pipeline_integration_test.cc
[modify] https://crrev.com/515ae317c22aafa3d6551ddf6871cc042050be6c/media/test/pipeline_integration_test_base.cc

i have lost some audio files i recorded on my voice recorder i need them for a case can i retrieve them in any way the phone i recorded them on is lost or stolen but i tght google may be able to retrieve them through my email linked to the voice recorder app.PLEASE HELP!!!!!!! thank you and god bless 
Project Member

Comment 58 by bugdroid1@chromium.org, Jun 16 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/fa5c37c65c5f038882af6ca8222566d0928930a0

commit fa5c37c65c5f038882af6ca8222566d0928930a0
Author: servolk <servolk@chromium.org>
Date: Thu Jun 16 17:56:47 2016

Generate and assign media track ids in demuxers.

Currently media track ids are generated on blink level, but this
complicates media track management. In particular when media tracks are
created on demuxer level we don't know their unique ids yet, so we
can't easily set up track id -> demuxer stream mappings. Assigning
track ids at the demuxer level makes that much easier.

BUG= 249427 , 341581 

Review-Url: https://codereview.chromium.org/2050043002
Cr-Commit-Position: refs/heads/master@{#400198}

[modify] https://crrev.com/fa5c37c65c5f038882af6ca8222566d0928930a0/content/renderer/media/webmediaplayer_ms_unittest.cc
[modify] https://crrev.com/fa5c37c65c5f038882af6ca8222566d0928930a0/media/base/media_track.h
[modify] https://crrev.com/fa5c37c65c5f038882af6ca8222566d0928930a0/media/base/media_tracks.cc
[modify] https://crrev.com/fa5c37c65c5f038882af6ca8222566d0928930a0/media/base/media_tracks.h
[modify] https://crrev.com/fa5c37c65c5f038882af6ca8222566d0928930a0/media/blink/webmediaplayer_impl.cc
[modify] https://crrev.com/fa5c37c65c5f038882af6ca8222566d0928930a0/media/blink/webmediaplayer_impl_unittest.cc
[modify] https://crrev.com/fa5c37c65c5f038882af6ca8222566d0928930a0/media/blink/websourcebuffer_impl.cc
[modify] https://crrev.com/fa5c37c65c5f038882af6ca8222566d0928930a0/media/filters/chunk_demuxer.cc
[modify] https://crrev.com/fa5c37c65c5f038882af6ca8222566d0928930a0/media/filters/chunk_demuxer.h
[modify] https://crrev.com/fa5c37c65c5f038882af6ca8222566d0928930a0/media/filters/chunk_demuxer_unittest.cc
[modify] https://crrev.com/fa5c37c65c5f038882af6ca8222566d0928930a0/media/filters/ffmpeg_demuxer.cc
[modify] https://crrev.com/fa5c37c65c5f038882af6ca8222566d0928930a0/media/filters/frame_processor_unittest.cc
[modify] https://crrev.com/fa5c37c65c5f038882af6ca8222566d0928930a0/media/filters/media_source_state.cc
[modify] https://crrev.com/fa5c37c65c5f038882af6ca8222566d0928930a0/media/filters/media_source_state.h
[modify] https://crrev.com/fa5c37c65c5f038882af6ca8222566d0928930a0/media/formats/mp4/mp4_stream_parser.cc
[modify] https://crrev.com/fa5c37c65c5f038882af6ca8222566d0928930a0/media/test/pipeline_integration_test.cc
[modify] https://crrev.com/fa5c37c65c5f038882af6ca8222566d0928930a0/media/test/pipeline_integration_test_base.cc
[modify] https://crrev.com/fa5c37c65c5f038882af6ca8222566d0928930a0/third_party/WebKit/Source/core/html/HTMLMediaElement.cpp
[modify] https://crrev.com/fa5c37c65c5f038882af6ca8222566d0928930a0/third_party/WebKit/Source/core/html/track/AudioTrack.cpp
[modify] https://crrev.com/fa5c37c65c5f038882af6ca8222566d0928930a0/third_party/WebKit/Source/core/html/track/TextTrackList.cpp
[modify] https://crrev.com/fa5c37c65c5f038882af6ca8222566d0928930a0/third_party/WebKit/Source/core/html/track/TrackBase.cpp
[modify] https://crrev.com/fa5c37c65c5f038882af6ca8222566d0928930a0/third_party/WebKit/Source/core/html/track/TrackBase.h
[modify] https://crrev.com/fa5c37c65c5f038882af6ca8222566d0928930a0/third_party/WebKit/Source/core/html/track/TrackListBase.h
[modify] https://crrev.com/fa5c37c65c5f038882af6ca8222566d0928930a0/third_party/WebKit/Source/core/html/track/VideoTrack.cpp
[modify] https://crrev.com/fa5c37c65c5f038882af6ca8222566d0928930a0/third_party/WebKit/Source/core/html/track/VideoTrackList.cpp
[modify] https://crrev.com/fa5c37c65c5f038882af6ca8222566d0928930a0/third_party/WebKit/Source/modules/mediasource/SourceBuffer.cpp
[modify] https://crrev.com/fa5c37c65c5f038882af6ca8222566d0928930a0/third_party/WebKit/public/platform/WebMediaPlayer.h
[modify] https://crrev.com/fa5c37c65c5f038882af6ca8222566d0928930a0/third_party/WebKit/public/platform/WebSourceBufferClient.h

Project Member

Comment 59 by bugdroid1@chromium.org, Jul 1 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/f25ceedfcb1e403c96eb80702cd29aa1dc370619

commit f25ceedfcb1e403c96eb80702cd29aa1dc370619
Author: servolk <servolk@chromium.org>
Date: Fri Jul 01 03:44:38 2016

Implement disabling and enabling media tracks

This CL implements disabling and enabling audio and video streams via
Javascript media tracks API. When a media track is disabled, the
corresponding DemuxerStream generates an EOS notification, thus ending
further decoding/rendering of that stream. When a media track is
enabled, the corresponding DemuxerStream is seeked to the current
playback position and we notify the renderers to resume reading from
the demuxer stream and rendering its contents.

BUG= 249427 

Review-Url: https://codereview.chromium.org/1935873002
Cr-Commit-Position: refs/heads/master@{#403403}

[modify] https://crrev.com/f25ceedfcb1e403c96eb80702cd29aa1dc370619/chromecast/media/cma/base/demuxer_stream_for_test.cc
[modify] https://crrev.com/f25ceedfcb1e403c96eb80702cd29aa1dc370619/chromecast/media/cma/base/demuxer_stream_for_test.h
[modify] https://crrev.com/f25ceedfcb1e403c96eb80702cd29aa1dc370619/media/base/android/media_url_demuxer.cc
[modify] https://crrev.com/f25ceedfcb1e403c96eb80702cd29aa1dc370619/media/base/android/media_url_demuxer.h
[modify] https://crrev.com/f25ceedfcb1e403c96eb80702cd29aa1dc370619/media/base/demuxer.h
[modify] https://crrev.com/f25ceedfcb1e403c96eb80702cd29aa1dc370619/media/base/demuxer_stream.h
[modify] https://crrev.com/f25ceedfcb1e403c96eb80702cd29aa1dc370619/media/base/fake_demuxer_stream.cc
[modify] https://crrev.com/f25ceedfcb1e403c96eb80702cd29aa1dc370619/media/base/fake_demuxer_stream.h
[modify] https://crrev.com/f25ceedfcb1e403c96eb80702cd29aa1dc370619/media/base/fake_text_track_stream.cc
[modify] https://crrev.com/f25ceedfcb1e403c96eb80702cd29aa1dc370619/media/base/fake_text_track_stream.h
[modify] https://crrev.com/f25ceedfcb1e403c96eb80702cd29aa1dc370619/media/base/mock_filters.h
[modify] https://crrev.com/f25ceedfcb1e403c96eb80702cd29aa1dc370619/media/base/pipeline.h
[modify] https://crrev.com/f25ceedfcb1e403c96eb80702cd29aa1dc370619/media/base/pipeline_impl.cc
[modify] https://crrev.com/f25ceedfcb1e403c96eb80702cd29aa1dc370619/media/base/pipeline_impl.h
[modify] https://crrev.com/f25ceedfcb1e403c96eb80702cd29aa1dc370619/media/blink/webmediaplayer_impl.cc
[modify] https://crrev.com/f25ceedfcb1e403c96eb80702cd29aa1dc370619/media/blink/webmediaplayer_impl.h
[modify] https://crrev.com/f25ceedfcb1e403c96eb80702cd29aa1dc370619/media/filters/chunk_demuxer.cc
[modify] https://crrev.com/f25ceedfcb1e403c96eb80702cd29aa1dc370619/media/filters/chunk_demuxer.h
[modify] https://crrev.com/f25ceedfcb1e403c96eb80702cd29aa1dc370619/media/filters/decrypting_demuxer_stream.cc
[modify] https://crrev.com/f25ceedfcb1e403c96eb80702cd29aa1dc370619/media/filters/decrypting_demuxer_stream.h
[modify] https://crrev.com/f25ceedfcb1e403c96eb80702cd29aa1dc370619/media/filters/ffmpeg_demuxer.cc
[modify] https://crrev.com/f25ceedfcb1e403c96eb80702cd29aa1dc370619/media/filters/ffmpeg_demuxer.h
[modify] https://crrev.com/f25ceedfcb1e403c96eb80702cd29aa1dc370619/media/mojo/services/mojo_demuxer_stream_adapter.cc
[modify] https://crrev.com/f25ceedfcb1e403c96eb80702cd29aa1dc370619/media/mojo/services/mojo_demuxer_stream_adapter.h
[modify] https://crrev.com/f25ceedfcb1e403c96eb80702cd29aa1dc370619/media/renderers/audio_renderer_impl.cc
[modify] https://crrev.com/f25ceedfcb1e403c96eb80702cd29aa1dc370619/media/renderers/renderer_impl.cc
[modify] https://crrev.com/f25ceedfcb1e403c96eb80702cd29aa1dc370619/media/renderers/renderer_impl.h
[modify] https://crrev.com/f25ceedfcb1e403c96eb80702cd29aa1dc370619/media/renderers/renderer_impl_unittest.cc
[modify] https://crrev.com/f25ceedfcb1e403c96eb80702cd29aa1dc370619/media/renderers/video_renderer_impl.cc
[modify] https://crrev.com/f25ceedfcb1e403c96eb80702cd29aa1dc370619/media/test/pipeline_integration_test.cc
[modify] https://crrev.com/f25ceedfcb1e403c96eb80702cd29aa1dc370619/media/test/pipeline_integration_test_base.cc
[modify] https://crrev.com/f25ceedfcb1e403c96eb80702cd29aa1dc370619/media/test/pipeline_integration_test_base.h

Comment 60 Deleted

Sorry for the noob'ish question, but does there exist a compiled build of Chrome/Chromium that supports HTMLMediaElement.videoTracks that we could test against? We're also wondering when we could expect to see (rough estimate) support for this in a) a canary build of Chrome, and b) a stable Chrome release?

We have a multi-videotrack desktop app that we're keen to port to the browser. We've proven the concept in Safari, but Chrome is the browser of choice for 99% of our customers.

Thanks for the efforts of all involved!
Please try, on Canary Chrome, with --enable-experimental-web-platform-features in the cmdline. Note that multi-track audio or multi-track video is not supported for MSE or normal src= in Chrome. But, for max 1 audio + max 1 video track, sourcing inband track attributes and populating the {audio,video}Tracks attribute on HTMLMediaElement is supported (with that flag) and certainly any feedback around that implementation would be helpful. Note also that MSE SourceBuffers.{audio,video}Tracks population is similarly supported (if that experimental flag is used).

Multi-track A/V for MSE is tracked in https://bugs.chromium.org/p/chromium/issues/detail?id=487288. I don't think we have a corresponding bug tracking the same for normal src= HTMLMediaElements. If that is what you want, please file one or let us know and we'll file one.

Comment 63 by fer...@gmail.com, Aug 3 2016

> I don't think we have a corresponding bug tracking the same for normal src= HTMLMediaElements. If that is what you want, please file one or let us know and we'll file one.

I'm not the commenter you were responding to, but I am eagerly awaiting this feature. I'd like a tracking bug that I can follow to keep up to date with the progress. Cheers!
Thanks for the response wolenetz.

So, I gave Canary Chrome a go (with --enable-experimental-web-platform-features flag enabled), and while videoTracks can now be queried, it only seems to ever have 1 track (as you mentioned).

Basically, what we are trying to do is play back a multi video track file in the browser, showing all tracks synchronised.

Here is a proof of concept that works in Safari -> https://jsfiddle.net/1v5wqwnt/7/
 
It adds video elements for the number of streams there are in the video source, points each video element to the same source, but changes the selected videotrack for each video element. The video elements then all belong to the same mediagroup, so playback is in sync. I'm not sure if this is the best solution (feedback welcome!), but it works (in Safari).

Aside, I also noticed video.controller is undefined in Canary Chrome.

I'm not sure if we should be using MSE or not, as we want to show all streams playing together, not switch between them in a single video element?
Labels: Needs-BlinkMediaTriage
@#64, video.controller is experimental, if at all, in Chrome currently (see thread ending in https://lists.w3.org/Archives/Public/public-html/2015Mar/0013.html).

Various workarounds with varying levels of success for polyfilling multi-element synchronization might be attempted. I think the most success I've seen was interestingly with a feedback-control-loop-style JS app which monitored the currentTime of the elements and adjusted their playbackRate attributes independently to approach synchronized advancement of their currentTimes. Note that, even if currentTime were perfectly syncrhonized, there's no guarantee in Chrome currently that the video frames render at precisely the right timestamp, due to cadence-based smoothing heuristics in Chrome's video stack intended to reduce visible jankiness in a single element playback.
One other method of synchronization might be to use WedAudio to synchronize audio clocks from multiple elements (with same caveat of lack of guaranteed video rendering precision, as above).

Regardless, thanks to servolk@, c#66 should be visible on a Canary soon, with --enable-experimental-web-platform-features, so you can try out at least some polyfills for mimicking HTMLMediaController synchronization for your mp4 files.
Clarification to c#67: c#66 is specific to MSE usage (not regular src=some_url_to_an.mp4). See https://www.w3.org/TR/media-source/
Re #c67 above: multi-track support is not ready yet. The CL mentioned in #c66 is just the first part which adds support for multiple tracks in MP4 parser. But we also need to make the necessary changes in demuxers to allow multiple tracks. I have a CL pending code review for FFmpegDemuxer (https://codereview.chromium.org/2284923003/) and a separate CL (which is work-in-progress, not quite ready to land yet) for supporting multiple tracks in MSE and ChunkDemuxer (https://codereview.chromium.org/2226443002/).
After those two CLs land you'll be able to select which audio/video tracks to play, but there's still going to be a limitation of 1 audio and 1 video track playing and switching tracks dynamically while playback is in progress is not going to work for now (needs some further work in media renderers).
wolenetz and servolk, thanks for the updates and appreciate your work on this! As soon as a the CL's land, i'll time-slice back onto this and see what I can get working.


Project Member

Comment 71 by bugdroid1@chromium.org, Sep 2 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/4fba1f0c2c06184687345d202f0afd08ae8dde2a

commit 4fba1f0c2c06184687345d202f0afd08ae8dde2a
Author: servolk <servolk@chromium.org>
Date: Fri Sep 02 02:04:50 2016

Implement support for multiple tracks in FFmpegDemuxer

Most of the changes are fairly straightforward. We need to adjust some
logic in FFmpegDemuxer::OnFindStreamInfoDone to allow handling multiple
tracks. Also need to tweak OnEnabledAudioTracksChanged and
OnSelectedVideoTrackChanged a little. And while we are at it let's get
rid of the deprecated ScopedVector and use standard vector instead.

BUG= 249427 

Review-Url: https://codereview.chromium.org/2284923003
Cr-Commit-Position: refs/heads/master@{#416163}

[modify] https://crrev.com/4fba1f0c2c06184687345d202f0afd08ae8dde2a/media/filters/ffmpeg_demuxer.cc
[modify] https://crrev.com/4fba1f0c2c06184687345d202f0afd08ae8dde2a/media/filters/ffmpeg_demuxer.h
[modify] https://crrev.com/4fba1f0c2c06184687345d202f0afd08ae8dde2a/media/filters/ffmpeg_demuxer_unittest.cc
[modify] https://crrev.com/4fba1f0c2c06184687345d202f0afd08ae8dde2a/media/test/data/README
[add] https://crrev.com/4fba1f0c2c06184687345d202f0afd08ae8dde2a/media/test/data/multitrack-3video-2audio.webm

Cc: wolenetz@chromium.org
Labels: -MovedFrom-36 -MovedFrom-35 -Merge-Merged-master1 -Needs-BlinkMediaTriage
Owner: servolk@chromium.org
Status: Fixed (was: Assigned)
Since this bug was tracking basic support for media track feature, I think we can consider this done. The remaining work for supporting multiple tracks is going to be tracked in crbug.com/487288

Sign in to add a comment