New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 737757 link

Starred by 1 user

Issue metadata

Status: Assigned
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug



Sign in to add a comment

Monitor and potentially deprecate support for multitrack SourceBuffer support of 'sequence' AppendMode

Project Member Reported by wolenetz@chromium.org, Jun 28 2017

Issue description

For context, see:
a) https://github.com/w3c/media-source/issues/186
b) https://chromium-review.googlesource.com/c/546963/

Associated tasks:
1) Get spec change landed
2) Get approval on intent to deprecate
3) Land removal of this support with graceful (compliant with spec change) behavior.

In parallel with 1-2, land a change that tracks with UMA, UKM, or RAPPOR the frequency per playback of such API usage, to help us decide when (3) is ok to do.

Consideration: Chromecast mpeg2ts MSE users might need this mode for some reason -- reach out to them, too.
 
servolk@, can you comment about any known requirement from Chromecast users for continued support for 'sequence' AppendMode when using mpeg2-ts?
TBH I don't know the answer to that question, I'll ask Ryan, who might know more about this, to take a look at this bug.
See bug 728477 for example of having to fix-up our frame processor to handle strange-but-currently-valid muxed sequence mode append sequences.
Summary: Monitor and potentially deprecate support for multitrack SourceBuffer support of 'sequence' AppendMode (was: Deprecate support for multitrack SourceBuffer support of 'sequence' AppendMode)
Project Member

Comment 5 by bugdroid1@chromium.org, Jul 13 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/f68e45d65cc00063ffc9cfb84e5b632bb8252104

commit f68e45d65cc00063ffc9cfb84e5b632bb8252104
Author: Matt Wolenetz <wolenetz@chromium.org>
Date: Thu Jul 13 23:00:40 2017

MSE: Report UseCounter and RAPPOR for two problematic MSE usages

This change adds UseCounter (for publicly visible % of Page Visits
statistics) and RAPPOR reporting to help us gauge the frequency of these
types of MSE API usages encountered in Chrome usage.  Such data may
assist subsequent work to clarify the spec, improve our implementation,
and potentially deprecate support.

"KeyframeTimeGreaterThanDependant" reporting: (Bug 739931)
If nonkeyframe's PTS precedes the PTS of the keyframe necessary to
decode it, this (in ISO-BMFF terminology) is not SAP Type 1 (though
other bytestreams might also encounter this GOP structure, too.) Our
buffering mechanism should gracefully handle this type of GOP, but the
MSE spec is unclear for how to precisely report resulting buffered
ranges and handle buffering/playback of overlaps of these types of
streams.

"MuxedSequenceMode" reporting: (Bug 737757)
If a multitrack SourceBuffer is used in 'sequence' AppendMode, the spec
leads to frequently surprising and undesirable results, usually due to
automatic timestampOffset updates based on one track after a
discontinuity are applied to all tracks.

At least bug 739931 may impact current PTS/DTS compliance work tracked
by  bug 718641 .

BUG=739931,737757, 718641 

Change-Id: I4fabb4ae0b389c5ce2eecb361d4b67c6d4874b04
Reviewed-on: https://chromium-review.googlesource.com/567558
Reviewed-by: Rick Byers <rbyers@chromium.org>
Reviewed-by: Chrome Cunningham <chcunningham@chromium.org>
Reviewed-by: Alexei Svitkine (slow) <asvitkine@chromium.org>
Commit-Queue: Matthew Wolenetz <wolenetz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#486512}
[modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/media/base/test_helpers.h
[modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/media/blink/websourcebuffer_impl.cc
[modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/media/blink/websourcebuffer_impl.h
[modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/media/filters/BUILD.gn
[modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/media/filters/chunk_demuxer.cc
[modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/media/filters/chunk_demuxer.h
[modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/media/filters/chunk_demuxer_unittest.cc
[modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/media/filters/frame_processor.cc
[modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/media/filters/frame_processor.h
[modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/media/filters/frame_processor_unittest.cc
[add] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/media/filters/source_buffer_parse_warnings.h
[modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/media/filters/source_buffer_state.cc
[modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/media/filters/source_buffer_state.h
[modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/media/filters/source_buffer_state_unittest.cc
[modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/media/test/mock_media_source.cc
[modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/media/test/mock_media_source.h
[modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/third_party/WebKit/Source/modules/mediasource/SourceBuffer.cpp
[modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/third_party/WebKit/Source/modules/mediasource/SourceBuffer.h
[modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/third_party/WebKit/public/platform/WebSourceBufferClient.h
[modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/third_party/WebKit/public/platform/web_feature.mojom
[modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/tools/metrics/histograms/enums.xml
[modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/tools/metrics/rappor/rappor.xml

So far, preliminary (Chrome M61 - dev channel) shows very miniscule % of occurrences of sequence muxed mode usage. However, this data is very preliminary, I'll update with public numbers once the related telemetry logging is in a more significant portion of Chrome usage.
Using the default 'segments' mode is supported, even for multitrack SourceBuffers.
After several versions of code which attempts to perform same task successfully at both Chromium and Firefox. Finally composed a version which achieves requirement at both Firefox and Chromium using `"sequence"` mode, https://github.com/guest271314/recordMediaFragments/blob/master/demos/recordMediaFragments.html.

When changing mode to `"segments"` Firefox renders expected result, at Chromium `MediaSource.duration` is not updated, resulting in only one second of media being played at `<video>` element.

How to adjust code to render same result using `"segments"` mode if the `"sequence"` mode is deprecated?


@#9 I'm uncertain especially since that demo link is broken now. Perhaps it's related to buffering logic. Try with Chrome 67 or higher, with cmdline parameter "--enable-features=MseBufferByPts".  This more compliant buffering mode may be part of what's needed for your solution.

Sign in to add a comment