New issue
Advanced search Search tips

Issue 918829 link

Starred by 1 user

Issue metadata

Status: Available
Owner: ----
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Linux , Android , Windows , Chrome , Mac , Fuchsia
Pri: 3
Type: Bug



Sign in to add a comment

Video playback via MSE drops last frame (consider flushing parser at MarkEndOfStream)

Project Member Reported by surma@chromium.org, Jan 3

Issue description

Chrome Version       : 71.0.3578.98
OS Version: OS X 10.14.1
URLs (if applicable) :
Other browsers tested:
  Add OK or FAIL after other browsers where you have tested this issue:
     Safari: N/A
    Firefox: Always drops last frame
    IE/Edge: Plays correctly w/ MSE, wonky with file playback

What steps will reproduce the problem?
1. Open https://wasm-webm-repros.glitch.me/counter.html
2. See the left video count to 3, the right video count to 2

What is the expected result?
Both videos should count to 3.

What happens instead of that?
The right video drops the last frame.

Please provide any additional information below. Attach a screenshot if
possible.

Both videos use the same file as a source. The video is a webm with a frame rate of 2fps with 4 frames (showing the numbers from 0 to 3 sequentially, see attachment). The left video is using the file as a source via the `src` attribute. The right file is creating a MediaSource, using an ArrayBuffer from fetch() as a source.


 
counter.html
1.0 KB View Download
counter.webm
3.1 KB View Download
Labels: Needs-Triage-M71
Cc: chcunningham@chromium.org
Components: Internals>Media>Source
Status: Untriaged (was: Unconfirmed)
chunningham@ plz triage
Nice clean demo. 

Looks like this webm file doesn't end cleanly. All the data for the frame's is there, but its probably missing a few bytes in the tail that effectively say the "list of frames has ended". This is causing the last frame to be held onto inside the demuxer. WebM files often don't have durations on their frames, so we hold back each frame until we get the next one (computing the duration as delta of timestamps). For the last frame in the cluster, we use the average of previous durations. In this case, we don't think the cluster ended so we're left holding onto the frame. 

code here
https://cs.chromium.org/chromium/src/media/formats/webm/webm_cluster_parser.cc?rcl=088a341f7d0a9110b2cca2f8803b7bbe006f0dc4&l=835
Cc: wolenetz@chromium.org
I think this is more or less working as intended, but we might make a tweak to improve it. This demo does mark end of stream, so we could treat that the same as the end-of-cluster condition and flush out the held back buffer. We could plumb this easily from SourceBufferState -> WebmStreamParser -> ClusterParser. 

https://cs.chromium.org/chromium/src/media/filters/source_buffer_state.cc?rcl=088a341f7d0a9110b2cca2f8803b7bbe006f0dc4&l=456

Matt, any concerns?


Labels: OS-Android OS-Chrome OS-Fuchsia OS-Linux OS-Windows
Summary: Video playback via MSE drops last frame (consider flushing parser at MarkEndOfStream) (was: Video playback via MSE drops last frame)
Interesting. I'm build-sheriffing today so not much bandwidth for this at all. 
That said, I just filed https://github.com/w3c/media-source/issues/226 because of this issue.

Currently, I *think* Chrome might allow resumption of the append of a media segment (e.g. webm cluster) that was interrupted by an endOfStream() call. If we run more than just the first step of the Reset Parser State algorithm, the behavior change might break apps (I'm uncertain if any depend on the existing behavior that might not reset the parser at endOfStream()).

Interesting! Thanks for taking a look!

Could you please do me a favor and quickly verify if the webm is indeed missing some sort of “end of stream” marker?

I generated[1] that webm with my library webm-wasm[2] (a WebAssembly port of libvpx and libwebm) and I was following the library’s docs on how to finalize a file. I just want to know if I need to bug the webm people to help me fix it :D

[1]: https://glitch.com/edit/#!/wasm-webm-repros?path=generator.html:1:0
[2]: https://github.com/GoogleChromeLabs/webm-wasm
Sounds good. I'd be happy to run just the first step. 


For folks following along...
https://www.w3.org/TR/media-source/#sourcebuffer-reset-parser-state

> If the append state equals PARSING_MEDIA_SEGMENT and the input buffer contains some complete coded frames, then run the coded frame processing algorithm until all of these complete coded frames have been processed.
Re: comment 6, I've confirm from parser state that this is the case. To find exactly whats missing you'll want to follow this code

https://cs.chromium.org/chromium/src/media/formats/webm/webm_cluster_parser.cc?rcl=e58bdd09150e967dbae8b80306a8099055f1d95e&l=101

which is waiting for the "list end"
https://cs.chromium.org/chromium/src/media/formats/webm/webm_parser.cc?rcl=916408b7872b5784a60b93eb9617823064b3b678&l=982

which gets called in a few places
https://cs.chromium.org/search/?q=f:webm_parser.cc+OnListEnd+&type=cs


Sorry, I can't be more helpful. Its a tedious thing parse webm by hand ;)
Re comment 7, Matt signed off on us implementing the first step (enough wiggle room in spec as-is, might implement later steps pending outcome of his spec bug). 

This is good p3 material, so I'll just let it sit for now. 
Re #8: Thanks very much for confirming. I’ll look how to fix that on my end then.
I have since then fixed the code of my generator and now agree with #4: “this is more or less working as intended”.

For reference, here is the file with proper metadata. It plays correctly via MSE with no frames dropped. 
klive.webm
3.1 KB View Download

Comment 12 by dbbrooks@chromium.org, Today (8 hours ago)

Status: Available (was: Untriaged)
In comment #9, pri is confirmed, work confirmed as relevant and intended, so moving to state "Available".

Sign in to add a comment