Monitor MSE SAP Type 2 sequences for possible spec or implementation change |
||||
Issue descriptionFor context, see https://github.com/w3c/media-source/issues/187#issuecomment-313456041 Internally, see also go/msesaptype2 for context. Associated tasks: 1) Obtain telemetry of how frequently this SAP type usage occurs in Chrome (including those cases where we perhaps overmark all keyframes as random access points), 1.5) Also, consider MEDIA_LOGGING a potential deprecation warning. 2) If telemetry confirms report from jyavenard@ mozilla that usage of this SAP Type is non-existent in MSE (or even just rare), then proceed with getting spec change landed to the ISO-BMFF bytestream format spec. Regardless of deprecation of support, we'll likely need to at least gracefully proceed with handling these streams, though seek and overlap-append behavior could be impacted on elements using those streams.
,
Jul 8 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/495493bad91c1fb4dd30462bb04b6f79c96ecc88 commit 495493bad91c1fb4dd30462bb04b6f79c96ecc88 Author: Matt Wolenetz <wolenetz@chromium.org> Date: Sat Jul 08 01:33:04 2017 MSE: Detect and MEDIA_LOG one kind of problematic GOP structure If a random access point doesn't have the earliest presentation time of other frames that depend on it (eg, other frames in later decode time up until the next random access point), the MSE spec was not designed to support processing and buffering it well. With the change to managing and reporting buffered ranges by PTS intervals instead of DTS intervals, this could impact interop. This change detects this general case and logs once per track to chrome://media-internals. Later changes might include telemetry collection to assist removing or fixing support for at least SAP Type 2 in the MSE ISOBMFF bytestream spec. To verify the new log is emitted by the new test, this change also upgrades FrameProcessorTest's |media_log_| to a StrictMock<MockMediaLog> and includes new strict verification of logs emitted during FrameProcessorTests. See also related spec issue https://github.com/w3c/media-source/issues/187 BUG=739931, 718641 Change-Id: I361177dee6a5c70edf17bdbde2f3ea643977e6ec Reviewed-on: https://chromium-review.googlesource.com/563017 Commit-Queue: Chrome Cunningham <chcunningham@chromium.org> Reviewed-by: Chrome Cunningham <chcunningham@chromium.org> Cr-Commit-Position: refs/heads/master@{#485125} [modify] https://crrev.com/495493bad91c1fb4dd30462bb04b6f79c96ecc88/media/base/test_helpers.h [modify] https://crrev.com/495493bad91c1fb4dd30462bb04b6f79c96ecc88/media/filters/frame_processor.cc [modify] https://crrev.com/495493bad91c1fb4dd30462bb04b6f79c96ecc88/media/filters/frame_processor_unittest.cc
,
Jul 10 2017
@#1, spot-checking netflix.com, I didn't find any non-SAP-Type-1 streams.
,
Jul 11 2017
,
Jul 11 2017
,
Jul 11 2017
,
Jul 13 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/f68e45d65cc00063ffc9cfb84e5b632bb8252104 commit f68e45d65cc00063ffc9cfb84e5b632bb8252104 Author: Matt Wolenetz <wolenetz@chromium.org> Date: Thu Jul 13 23:00:40 2017 MSE: Report UseCounter and RAPPOR for two problematic MSE usages This change adds UseCounter (for publicly visible % of Page Visits statistics) and RAPPOR reporting to help us gauge the frequency of these types of MSE API usages encountered in Chrome usage. Such data may assist subsequent work to clarify the spec, improve our implementation, and potentially deprecate support. "KeyframeTimeGreaterThanDependant" reporting: (Bug 739931) If nonkeyframe's PTS precedes the PTS of the keyframe necessary to decode it, this (in ISO-BMFF terminology) is not SAP Type 1 (though other bytestreams might also encounter this GOP structure, too.) Our buffering mechanism should gracefully handle this type of GOP, but the MSE spec is unclear for how to precisely report resulting buffered ranges and handle buffering/playback of overlaps of these types of streams. "MuxedSequenceMode" reporting: (Bug 737757) If a multitrack SourceBuffer is used in 'sequence' AppendMode, the spec leads to frequently surprising and undesirable results, usually due to automatic timestampOffset updates based on one track after a discontinuity are applied to all tracks. At least bug 739931 may impact current PTS/DTS compliance work tracked by bug 718641 . BUG=739931,737757, 718641 Change-Id: I4fabb4ae0b389c5ce2eecb361d4b67c6d4874b04 Reviewed-on: https://chromium-review.googlesource.com/567558 Reviewed-by: Rick Byers <rbyers@chromium.org> Reviewed-by: Chrome Cunningham <chcunningham@chromium.org> Reviewed-by: Alexei Svitkine (slow) <asvitkine@chromium.org> Commit-Queue: Matthew Wolenetz <wolenetz@chromium.org> Cr-Commit-Position: refs/heads/master@{#486512} [modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/media/base/test_helpers.h [modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/media/blink/websourcebuffer_impl.cc [modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/media/blink/websourcebuffer_impl.h [modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/media/filters/BUILD.gn [modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/media/filters/chunk_demuxer.cc [modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/media/filters/chunk_demuxer.h [modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/media/filters/chunk_demuxer_unittest.cc [modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/media/filters/frame_processor.cc [modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/media/filters/frame_processor.h [modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/media/filters/frame_processor_unittest.cc [add] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/media/filters/source_buffer_parse_warnings.h [modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/media/filters/source_buffer_state.cc [modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/media/filters/source_buffer_state.h [modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/media/filters/source_buffer_state_unittest.cc [modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/media/test/mock_media_source.cc [modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/media/test/mock_media_source.h [modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/third_party/WebKit/Source/modules/mediasource/SourceBuffer.cpp [modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/third_party/WebKit/Source/modules/mediasource/SourceBuffer.h [modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/third_party/WebKit/public/platform/WebSourceBufferClient.h [modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/third_party/WebKit/public/platform/web_feature.mojom [modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/tools/metrics/histograms/enums.xml [modify] https://crrev.com/f68e45d65cc00063ffc9cfb84e5b632bb8252104/tools/metrics/rappor/rappor.xml
,
Jul 28 2017
So far, preliminary (Chrome M61 - dev channel) shows a small % of occurrences of SAP-type-2 usage in MSE. However, this data is very preliminary, I'll update with public numbers once the related telemetry logging is in a more significant portion of Chrome usage. |
||||
►
Sign in to add a comment |
||||
Comment 1 by wolenetz@chromium.org
, Jul 7 2017