Frame-stepping in HTML5 video is glitchy
Reported by
ffeld...@imagestreamteam.com,
Aug 3 2016
|
||||||||
Issue descriptionUserAgent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.82 Safari/537.36 Example URL: Steps to reproduce the problem: 1. Load a video tag with h.264 video clip @ 30fps 2. On a button press, advance currentTime=currentTime+0.0334 3. Press the button repeatedly and see how not-nicely the frame advances. What is the expected behavior? The video should advance by 1/30th of a sec each time button is pressed. What went wrong? Currently, when you do advance by 1/30sec over and over, the frames glitches and goes backwards first. Sometimes it doesn't advance at all. It's not smooth by any means. This glitching didn't happen until v52, but even then it was slow. Did this work before? Yes Pre v52 Is it a problem with Flash or HTML5? HTML5 Does this work in other browsers? Yes Chrome version: 52.0.2743.82 Channel: stable OS Version: OS X 10.10.5 Flash Version: Shockwave Flash 22.0 r0
,
Aug 5 2016
This is not just an issue in Mac, but also Windows as well.
,
Aug 5 2016
I am not able to reproduce this on Mac using M52 stable or M54 canary, with either H.264 or VP9. Note that this is the same technique used by YouTube when you press ,/. to advance frames, so this is a generally well-tested path. While it is expected to be slow (it restarts decoding at the previous keyframe every time), the implementation was not changed in M52. Can you provide your actual test files that reproduce the changed behavior?
,
Aug 5 2016
These files show the behavior on my system right now. You can use left-arrow or right-arrow to step backwards/forwards or use the STEP buttons on the screen.
,
Aug 5 2016
Thanks, I am able to reproduce the behavior you describe with this test file on Mac. It does not occur with hardware decoding disabled, so we can be sure that the problem is in either the accelerated decoder or the compositing path. As a workaround, I would try re-encoding with no frame reordering (aka no B frames). Chrome's handling of seek timestamps is much more reliable for such media. I am not able to reproduce this behavior on Windows 10. ccameron@: Can you take a look to see if there is a compositing problem here?
,
Aug 5 2016
,
Aug 11 2016
May be related to use of AVSamplebufferDisplayLayer, which was in M52, but was disabled at the last minute. Taking a look.
,
Aug 24 2016
Issue 633581 has been merged into this issue.
,
Aug 24 2016
After instrumenting the code, it looks like this is a trivial bug around handling flushing in VTVDA. Between when GVD calls Reset() (to start the seek) and when VTVDA notifies that the reset is complete, VTVDA also emits the frames that remain in its reorder buffer. GVD should be dropping such frames, but I'll fix it in both places tomorrow and see if that resolves the issue.
,
Aug 24 2016
,
Aug 25 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/8cbe5d69e18a9c387cfc61adf826ae121cf89834 commit 8cbe5d69e18a9c387cfc61adf826ae121cf89834 Author: sandersd <sandersd@chromium.org> Date: Thu Aug 25 19:04:07 2016 VTVDA: Fix computation of |reorder_window|. Prior to this change, VTVDA would disable reordering when there was no VUI max_num_reorder_frames syntax element. This is incorrect; excluding an explicit list of profiles, it should be MaxDpbFrames. The incorrect behavior is masked during normal playback, because decoding is typically faster than rendering. This causes the reorder queue to fill up even though reordering is supposed to be turned off, and frames therefore get a chance to be ordered correctly. BUG= 634007 Review-Url: https://codereview.chromium.org/2272233002 Cr-Commit-Position: refs/heads/master@{#414499} [modify] https://crrev.com/8cbe5d69e18a9c387cfc61adf826ae121cf89834/media/gpu/vt_video_decode_accelerator_mac.cc [modify] https://crrev.com/8cbe5d69e18a9c387cfc61adf826ae121cf89834/media/video/h264_poc.cc
,
Aug 26 2016
This fix is included in M54, and is available now in the Canary channel. ffeldman@, can you confirm that the issue is fixed for you in Canary?
,
Aug 27 2016
I can confirm that the issue looks like it is fixed in M54, when comparing the same web application to M52 on Mac. I do not see the frame-glitch issue on Windows (in either M52 or M54) but in general the frame-stepping performance is slow. If we try the same web app in K-Meleon, for example, on the exact same machine we get much snappier performance. I didn't know if the performance of frame-step would somehow be improved by this fix too. Any ideas? Thanks!
,
Aug 29 2016
Thanks for testing! Seek performance in Chrome is almost entirely determined by keyframe density--the media pipeline will seek to the previous keyframe and play forward (preroll) to the requested media time for each currentTime modification. |
||||||||
►
Sign in to add a comment |
||||||||
Comment 1 by ellyjo...@chromium.org
, Aug 4 2016Status: Untriaged (was: Unconfirmed)