UserAgent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36

Example URL:

Steps to reproduce the problem:

I am loading fragmented mp4 video & audio in two different sourcebuffers and the playback is fine in general.  One weird thing I see is that during low bandwidth situations, sometimes the video element doesn't start playback even though MSE buffer length is greater than 1 sec until it gains about 3 seconds of buffer.  The playback starts automatically once we gain ~3sec of buffer.  Is this an intended limitation?

1. Load video element and throttle the network in Chrome to "Good 2G" profile - 450 kb bandwidth
2. Call play and start loading mp4 chunks into source buffers
3. Playback starts and MSE is perfect during initial 7 secs, where it plays even with less than 2 secs of buffer
4. After this initial period I could see the problem above consistently where the playback doesn't start until I see ~3 sec of buffer in HTMLMediaElement.buffered.

What is the expected behavior?
MSE should start playing as soon as we append a small amount of data, and not wait for 3 sec of buffer.

What went wrong?
Only in low bandwidth scenarios, and when rebuffering occurs frequently, MSE waits for 3 sec of buffer before resuming playback.

Did this work before? N/A 

Is it a problem with Flash or HTML5? HTML5

Does this work in other browsers? N/A 

Chrome version: 50.0.2661.102  Channel: stable
OS Version: OS X 10.11.3
Flash Version: Shockwave Flash 21.0 r0

Posing a related email thread here. Thank you Matt & Dale for the initial investigation and response!

On Tuesday, 3 May 2016 12:57 PM, Prabu Raja <praja@yahoo-inc.com> wrote:

Hi Dale,

The video frame duration is ~40 msec for most of the videos I test.

Thanks,
Prabu Raja

On Tuesday, 3 May 2016 12:40 PM, Dale Curtis <dalecurtis@chromium.org> wrote:

How long is each video frame in this case? Today we rely on the decoder having decoded 4 full frames, regardless of frame duration.

- dale

On Tue, May 3, 2016 at 11:33 AM, Prabu Raja <praja@yahoo-inc.com> wrote:
    Hi Matthew,

    Thanks much for your detailed response!

    // Are you seeing that you need multiple seconds within HTMLMediaElement.buffered prior to playback starting?

    Yes.  For example the video goes into the waiting state once HTMLMediaElement.buffered says 0.03 secs of buffer remaining.  And, as I add more frames I could see the remaining buffer increasing like this:

    chunk 1:

    audio: 69 frames
    video: 35 frames (starts with keyframe, and has another one key frame)
    HTMLMediaElement.buffered - 1.49 secs of buffer remaining after currentTime, video still not started playing

    chunk 2:

    audio: 32 frames
    video: 15 frames (starts with keyframe, and has another one key frame)
    HTMLMediaElement.buffered - 2.07 secs of buffer remaining after currentTime, video still not started playing

    chunk 3:

    audio: 85 frames
    video: 49 frames (starts with keyframe, and has another one key frame)
    HTMLMediaElement.buffered - 3.59 secs of buffer remaining after currentTime, video started playing

This doesn't happen all the time though.  Here's the scenario I reproduce this:

1. Load video element and throttle the network in Chrome to "Good 2G" profile - 450 kb bandwidth
2. Call play and start loading mp4 chunks into source buffers
3. Playback starts and MSE is perfect during initial 7 secs, where it plays even with less than 2 secs of buffer
4. After this initial period I could see the problem above consistently where the playback doesn't start until I see ~3 sec of buffer in HTMLMediaElement.buffered.

    I checked media-internals and not in low delay mode.  Do you think the smooth video playback optimization is causing this delay?  Sounds a bit weird to me as we have several key frames in the first two chunks and a good amount of buffer too (~2 secs).

    Also, could you please let me know how to mark a stream as live?

    I really appreciate your help on this, and can provide additional input if needed.  Thanks much!!

Thanks,
Prabu Raja

On Monday, 2 May 2016 4:16 PM, Matthew Wolenetz <wolenetz@chromium.org> wrote:

    +chcunningham@ and dalecurtis@ in case they have any further comment, too.

On Mon, May 2, 2016 at 4:12 PM, Matthew Wolenetz <wolenetz@chromium.org> wrote:

        Thank you for your question, Prabu.

        HTMLMediaElement.bufffered should report intersection of all activeSourceBuffers' buffered time ranges. Are you seeing that you need multiple seconds within HTMLMediaElement.buffered prior to playback starting? (This buffered status reflects the actual presentation interval buffered, excluding initial portions of a coded frame group appended that didn't begin with a keyframe.)

        A few things to note:

1. In recent Chrome (as part of http://youtube-eng.blogspot.ca/2015/11/smoother-in-chrome.html) we smooth video rendering playback to reduce jankiness, so we'll buffer a few decoded video frames before beginning their actual rendering unless the stream is marked as a live stream, in which case we'll try a lower-latency rendering path.
2. Orthogonally to above, if your videos have out-of-order decode (h.264 commonly does this), then multiple frames will need to be buffered and decoded sometimes before we can decode a frame with an earlier timestamp.

3. 1 & 2 can combine to increase the frame latency. Especially if you have low-framerate video, this could become even more apparent.

        To engage our experimental "low latency" video frame path (which doesn't attempt the video smoothing), see our logic for ISO-BMFF here or our logic for webm here. Also, it is only supported on some platforms, so to confirm you're using it, look in chrome://media-internals for the associated media player and see if you find a log entry like "Video rendering in low delay mode."

Matt

On Mon, May 2, 2016 at 1:36 PM, Prabu Raja <praja@yahoo-inc.com> wrote:

Hello,

            Good evening, and hope things are going well for you.  I have a quick question on MSE video playback in Chrome 50 and I thought you'd be the best person to reach out.

            I am loading fragmented mp4 video & audio in two different sourcebuffers and the playback is fine in general.  One weird thing I see is that during low bandwidth situations, sometimes the video element doesn't start playback even though MSE buffer length is greater than 1 sec until it gains about 3 seconds of buffer.  The playback starts automatically once we gain ~3sec of buffer.  Is this an intended limitation?

            I tried loading segments starting with a keyframe, and without that limitation (chrome 50) and I am able to reproduce this behavior in both the cases.  Do you think this is related to key frames in any ways?

            I'd appreciate any clues related to this.  Thanks in advance!

Thanks,
Prabu Raja