New issue
Advanced search Search tips

Issue 739931 link

Starred by 2 users

Issue metadata

Status: Started
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug



Sign in to add a comment

Monitor MSE SAP Type 2 sequences for possible spec or implementation change

Project Member Reported by wolenetz@chromium.org, Jul 6 2017

Issue description

For context, see https://github.com/w3c/media-source/issues/187#issuecomment-313456041

Internally, see also go/msesaptype2 for context.

Associated tasks:
1) Obtain telemetry of how frequently this SAP type usage occurs in Chrome (including those cases where we perhaps overmark all keyframes as random access points),
1.5) Also, consider MEDIA_LOGGING a potential deprecation warning.
2) If telemetry confirms report from jyavenard@ mozilla that usage of this SAP Type is non-existent in MSE (or even just rare), then proceed with getting spec change landed to the ISO-BMFF bytestream format spec.

Regardless of deprecation of support, we'll likely need to at least gracefully proceed with handling these streams, though seek and overlap-append behavior could be impacted on elements using those streams.
 
With https://chromium-review.googlesource.com/c/563017/2 applied locally (and stream parser factory locally modified to not support any webm mime type), spot-checking various media sites for this occurrences of the new media log manually shows:

1) nest.com desktop site emits this new media log. However, they appear to use a new MediaSource object after every seek, so part of the concern of implications of non-SAP-Type-1 streams (seeking) seems mitigated by them already. Once PTS/DTS switch occurs, this should be re-checked to see if they're streams result in overlap-append issues that might produce gaps.

2) youtube.com desktop site (h264, various streams and resolutions) doesn't emit the new media log.

3) netflix.com desktop site -- pending EME-enabled local build..
Project Member

Comment 2 by bugdroid1@chromium.org, Jul 8 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/495493bad91c1fb4dd30462bb04b6f79c96ecc88

commit 495493bad91c1fb4dd30462bb04b6f79c96ecc88
Author: Matt Wolenetz <wolenetz@chromium.org>
Date: Sat Jul 08 01:33:04 2017

MSE: Detect and MEDIA_LOG one kind of problematic GOP structure

If a random access point doesn't have the earliest presentation time of
other frames that depend on it (eg, other frames in later decode time up
until the next random access point), the MSE spec was not designed to
support processing and buffering it well. With the change to managing
and reporting buffered ranges by PTS intervals instead of DTS intervals,
this could impact interop. This change detects this general case and
logs once per track to chrome://media-internals. Later changes might
include telemetry collection to assist removing or fixing support for
at least SAP Type 2 in the MSE ISOBMFF bytestream spec.

To verify the new log is emitted by the new test, this change also
upgrades FrameProcessorTest's |media_log_| to a StrictMock<MockMediaLog>
and includes new strict verification of logs emitted during
FrameProcessorTests.

See also related spec issue https://github.com/w3c/media-source/issues/187

BUG=739931, 718641 

Change-Id: I361177dee6a5c70edf17bdbde2f3ea643977e6ec
Reviewed-on: https://chromium-review.googlesource.com/563017
Commit-Queue: Chrome Cunningham <chcunningham@chromium.org>
Reviewed-by: Chrome Cunningham <chcunningham@chromium.org>
Cr-Commit-Position: refs/heads/master@{#485125}
[modify] https://crrev.com/495493bad91c1fb4dd30462bb04b6f79c96ecc88/media/base/test_helpers.h
[modify] https://crrev.com/495493bad91c1fb4dd30462bb04b6f79c96ecc88/media/filters/frame_processor.cc
[modify] https://crrev.com/495493bad91c1fb4dd30462bb04b6f79c96ecc88/media/filters/frame_processor_unittest.cc

@#1, spot-checking netflix.com, I didn't find any non-SAP-Type-1 streams.

Summary: Monitor MSE SAP Type 2 sequences for possible spec or implementation change (was: Deprecate MSE support for SAP Type 2 sequences)
Summary: Monitor MSE non-SAP-Type-1 sequences for possible spec or implementation change (was: Monitor MSE SAP Type 2 sequences for possible spec or implementation change)
Summary: Monitor MSE SAP Type 2 sequences for possible spec or implementation change (was: Monitor MSE non-SAP-Type-1 sequences for possible spec or implementation change)
Project Member

Comment 7 by bugdroid1@chromium.org, Jul 13 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/f68e45d65cc00063ffc9cfb84e5b632bb8252104

commit f68e45d65cc00063ffc9cfb84e5b632bb8252104
Author: Matt Wolenetz <wolenetz@chromium.org>
Date: Thu Jul 13 23:00:40 2017

MSE: Report UseCounter and RAPPOR for two problematic MSE usages

This change adds UseCounter (for publicly visible % of Page Visits
statistics) and RAPPOR reporting to help us gauge the frequency of these
types of MSE API usages encountered in Chrome usage.  Such data may
assist subsequent work to clarify the spec, improve our implementation,
and potentially deprecate support.

"KeyframeTimeGreaterThanDependant" reporting: (Bug 739931)
If nonkeyframe's PTS precedes the PTS of the keyframe necessary to
decode it, this (in ISO-BMFF terminology) is not SAP Type 1 (though
other bytestreams might also encounter this GOP structure, too.) Our
buffering mechanism should gracefully handle this type of GOP, but the
MSE spec is unclear for how to precisely report resulting buffered
ranges and handle buffering/playback of overlaps of these types of
streams.

"MuxedSequenceMode" reporting: (Bug 737757)
If a multitrack SourceBuffer is used in 'sequence' AppendMode, the spec
leads to frequently surprising and undesirable results, usually due to
automatic timestampOffset updates based on one track after a
discontinuity are applied to all tracks.

At least bug 739931 may impact current PTS/DTS compliance work tracked
by  bug 718641 .

BUG=739931,737757, 718641 

Change-Id: I4fabb4ae0b389c5ce2eecb361d4b67c6d4874b04
Reviewed-on: https://chromium-review.googlesource.com/567558
Reviewed-by: Rick Byers <rbyers@chromium.org>
Reviewed-by: Chrome Cunningham <chcunningham@chromium.org>
Reviewed-by: Alexei Svitkine (slow) <asvitkine@chromium.org>
Commit-Queue: Matthew Wolenetz <wolenetz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#486512}
[modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/media/base/test_helpers.h
[modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/media/blink/websourcebuffer_impl.cc
[modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/media/blink/websourcebuffer_impl.h
[modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/media/filters/BUILD.gn
[modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/media/filters/chunk_demuxer.cc
[modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/media/filters/chunk_demuxer.h
[modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/media/filters/chunk_demuxer_unittest.cc
[modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/media/filters/frame_processor.cc
[modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/media/filters/frame_processor.h
[modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/media/filters/frame_processor_unittest.cc
[add] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/media/filters/source_buffer_parse_warnings.h
[modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/media/filters/source_buffer_state.cc
[modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/media/filters/source_buffer_state.h
[modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/media/filters/source_buffer_state_unittest.cc
[modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/media/test/mock_media_source.cc
[modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/media/test/mock_media_source.h
[modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/third_party/WebKit/Source/modules/mediasource/SourceBuffer.cpp
[modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/third_party/WebKit/Source/modules/mediasource/SourceBuffer.h
[modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/third_party/WebKit/public/platform/WebSourceBufferClient.h
[modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/third_party/WebKit/public/platform/web_feature.mojom
[modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/tools/metrics/histograms/enums.xml
[modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/tools/metrics/rappor/rappor.xml

So far, preliminary (Chrome M61 - dev channel) shows a small % of occurrences of SAP-type-2 usage in MSE. However, this data is very preliminary, I'll update with public numbers once the related telemetry logging is in a more significant portion of Chrome usage.

Sign in to add a comment