Video playback via MSE drops last frame (consider flushing parser at MarkEndOfStream) |
|||||
Issue description
Chrome Version : 71.0.3578.98
OS Version: OS X 10.14.1
URLs (if applicable) :
Other browsers tested:
Add OK or FAIL after other browsers where you have tested this issue:
Safari: N/A
Firefox: Always drops last frame
IE/Edge: Plays correctly w/ MSE, wonky with file playback
What steps will reproduce the problem?
1. Open https://wasm-webm-repros.glitch.me/counter.html
2. See the left video count to 3, the right video count to 2
What is the expected result?
Both videos should count to 3.
What happens instead of that?
The right video drops the last frame.
Please provide any additional information below. Attach a screenshot if
possible.
Both videos use the same file as a source. The video is a webm with a frame rate of 2fps with 4 frames (showing the numbers from 0 to 3 sequentially, see attachment). The left video is using the file as a source via the `src` attribute. The right file is creating a MediaSource, using an ArrayBuffer from fetch() as a source.
,
Jan 3
chunningham@ plz triage
,
Jan 4
Nice clean demo. Looks like this webm file doesn't end cleanly. All the data for the frame's is there, but its probably missing a few bytes in the tail that effectively say the "list of frames has ended". This is causing the last frame to be held onto inside the demuxer. WebM files often don't have durations on their frames, so we hold back each frame until we get the next one (computing the duration as delta of timestamps). For the last frame in the cluster, we use the average of previous durations. In this case, we don't think the cluster ended so we're left holding onto the frame. code here https://cs.chromium.org/chromium/src/media/formats/webm/webm_cluster_parser.cc?rcl=088a341f7d0a9110b2cca2f8803b7bbe006f0dc4&l=835
,
Jan 4
I think this is more or less working as intended, but we might make a tweak to improve it. This demo does mark end of stream, so we could treat that the same as the end-of-cluster condition and flush out the held back buffer. We could plumb this easily from SourceBufferState -> WebmStreamParser -> ClusterParser. https://cs.chromium.org/chromium/src/media/filters/source_buffer_state.cc?rcl=088a341f7d0a9110b2cca2f8803b7bbe006f0dc4&l=456 Matt, any concerns?
,
Jan 4
Interesting. I'm build-sheriffing today so not much bandwidth for this at all. That said, I just filed https://github.com/w3c/media-source/issues/226 because of this issue. Currently, I *think* Chrome might allow resumption of the append of a media segment (e.g. webm cluster) that was interrupted by an endOfStream() call. If we run more than just the first step of the Reset Parser State algorithm, the behavior change might break apps (I'm uncertain if any depend on the existing behavior that might not reset the parser at endOfStream()).
,
Jan 5
Interesting! Thanks for taking a look! Could you please do me a favor and quickly verify if the webm is indeed missing some sort of “end of stream” marker? I generated[1] that webm with my library webm-wasm[2] (a WebAssembly port of libvpx and libwebm) and I was following the library’s docs on how to finalize a file. I just want to know if I need to bug the webm people to help me fix it :D [1]: https://glitch.com/edit/#!/wasm-webm-repros?path=generator.html:1:0 [2]: https://github.com/GoogleChromeLabs/webm-wasm
,
Jan 5
Sounds good. I'd be happy to run just the first step. For folks following along... https://www.w3.org/TR/media-source/#sourcebuffer-reset-parser-state > If the append state equals PARSING_MEDIA_SEGMENT and the input buffer contains some complete coded frames, then run the coded frame processing algorithm until all of these complete coded frames have been processed.
,
Jan 5
Re: comment 6, I've confirm from parser state that this is the case. To find exactly whats missing you'll want to follow this code https://cs.chromium.org/chromium/src/media/formats/webm/webm_cluster_parser.cc?rcl=e58bdd09150e967dbae8b80306a8099055f1d95e&l=101 which is waiting for the "list end" https://cs.chromium.org/chromium/src/media/formats/webm/webm_parser.cc?rcl=916408b7872b5784a60b93eb9617823064b3b678&l=982 which gets called in a few places https://cs.chromium.org/search/?q=f:webm_parser.cc+OnListEnd+&type=cs Sorry, I can't be more helpful. Its a tedious thing parse webm by hand ;)
,
Jan 5
Re comment 7, Matt signed off on us implementing the first step (enough wiggle room in spec as-is, might implement later steps pending outcome of his spec bug). This is good p3 material, so I'll just let it sit for now.
,
Jan 7
Re #8: Thanks very much for confirming. I’ll look how to fix that on my end then.
,
Jan 9
I have since then fixed the code of my generator and now agree with #4: “this is more or less working as intended”. For reference, here is the file with proper metadata. It plays correctly via MSE with no frames dropped.
,
Today
(8 hours ago)
In comment #9, pri is confirmed, work confirmed as relevant and intended, so moving to state "Available". |
|||||
►
Sign in to add a comment |
|||||
Comment 1 by swarnasree.mukkala@chromium.org
, Jan 3