Yep. IIRC livestreaming in MSE is indicated by having duration = infinity, so should be easy to detect in WMPI.

Dan claims that data_source_->IsStreaming() should be more than enough.

If you mean WMPI::data_source_, then it only seems to be set at https://cs.chromium.org/chromium/src/media/blink/webmediaplayer_impl.cc?rcl=1481905173&l=392 and is only used for direct URL (non-MSE) playback. We should probably avoid disabling background video tracks for both direct URL and MSE live streaming playback.

avayvod@, OOC, why would this break them?

IIRC streaming videos often have very rare key frames. Maybe we should just rely on cutting off at some key frame distance?
Dan, Sergey, WDYT?

wolenetz@: What heuristic should we be using to determine if we can safely remove and add back a video track?

How can I check the keyframe of a live streamed video? Does media-internal expose this information?

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/102cdb66b3460080b67f5e27e2d7db9fc206c564

commit 102cdb66b3460080b67f5e27e2d7db9fc206c564
Author: avayvod <avayvod@chromium.org>
Date: Sat Jan 07 03:11:09 2017

[Media] Disable background video optimization for streaming videos

BUG= 675282 
TEST=manual

Review-Url: https://codereview.chromium.org/2610893004
Cr-Commit-Position: refs/heads/master@{#442147}

[modify] https://crrev.com/102cdb66b3460080b67f5e27e2d7db9fc206c564/media/blink/webmediaplayer_impl.cc
[modify] https://crrev.com/102cdb66b3460080b67f5e27e2d7db9fc206c564/media/blink/webmediaplayer_impl.h

Several things going on here. Some answers and questions:

* media-internals will show a debug event with message like "Video rendering in low delay mode." when the MSE or plain old direct src=URL playback considers the stream to be live. We have non-standardized heuristics currently for detecting this condition for various bytestreams (for MSE vNext, there are related bugs to get such standardized; for src=URL, not so sure.)

* keyframe info isn't exposed in media-internals. One way to check is to unfortunately build a debug Chrome, repro the issue while capturing debug logs from the parser (e.g. --v=3 --vmodule=frame_processor*=3,*=0) and observe the parser in action on appended media. Note that the output of the parser is then spliced into the timeline, perhaps overlapping previous parser outputs, but ostensibly live streams shouldn't have any overlaps or gaps.

* For MSE, there's an upper bound on GOP size (per platform, currently hardcoded buffered byte caps per each track), and GC/evictions to make room for more appends are done on a whole GOP basis (IOW, a non-keyframe isn't removed unless it's entire GOP is removed). This somewhat limits two things: 1) maximum interkeyframe frame count, and 2) cost of prerolling to currentTime upon track re-enablement.

 
@#9 Define safely please :).
@avayvod, is an *otherwise-working* live site *broken* when its track is disabled and later re-enabled, or does it work albeit with a decoder preroll delay? When the track is disabled, that doesn't necessarily mean that the app is no longer fetching buffers for it, does it?

Maybe I'm confused from the holidays :)

Also, note that #10 doesn't address MSE live streaming videos. data_source_ isn't used for MSE by WMPI; chunk_demuxer_ instead. There's a DemuxerStream::liveness() interface method that *all* demuxers report, and it's how the MSE heuristic reports LIVENESS_LIVE (or _RECORDED or _UNKNOWN) for each ChunkDemuxerStream. This heuristic commonly reports LIVE even when it isn't actually really LIVE, though, since it relies on metadata in the appended media, (unlike data_source_ which knows whether or not the source supports byte-range-requests or just is a streaming source).

Thanks Matt for your replies!

I'll look into using liveness() then.

We suspect that enabling video track for a video with large average keyframe distance will result in a bad experience - the video frames will stall and appear frozen for significant time upon bringing the tab to the foreground.

We also suspect that streaming videos often have few and infrequent keyframes so decided to disable the optimization for live videos.

Maybe the heuristic that disables the optimization for infrequent keyframes will simply cover the streaming videos and they will also work fine if the keyframes are present often enough.

Dale, Dan, any other reasons the streaming videos should be exempt from the background optimization?

@avayvod-- Could you please let us know if there are any manual repro steps to reproduce the issue, if there is , please provide us the expected result . So , that we can verify it from Chrome-TE end.

Thanks!

hdodda@ 
1. Run Chrome with --enable-features=BackgroundVideoTrackOptimization (can be enabled from chrome://flags).
2. Open a streaming video and chrome://media-internals view of the video player in a separate tab (the page should show 'streaming' as true in the player properties).
3. Put the playing video to the background.
4. Observe that there's no change in selected video tracks logged in chrome://media-internals for the player (for non-streaming video it should show an empty set, [], for backgrounded videos and [0] when they are foregrounded).

Note that doesn't work for MSE yet, only for videos played via a URL.
I'm not sure where to find a good example of such a video :/

> ... for videos played via a URL. I'm not sure where to find a good example of such a video :/

[For non-MSE:]
Grab a video that plays in chrome, copy it locally to a linux machine reachable from your test device, and use SimpleHTTPServer to serve the media, e.g.:
$ python -m SimpleHTTPServer 8000

WebMediaPlayerImpl::IsStreaming should report true for such a playback, since SimpleHTTPServer doesn't support range requests.

Note, it's not really a live stream, but it is a streaming data source since range requests aren't supported. Also note that proxies between your test device and the server may interfere with this setup's detection of a streaming server (I'm not sure, but there could be proxies that emulate range requests, though that sounds quite horrible to me...)

I don't think anything special needs to be done here besides checking if the data source is streaming or not; the keyframe heurstic should be sufficient for all other cases. There are plenty of YouTube videos reporting LIVE type that are recordings, so I don't think it's a good idea to use that for a flag.

Thanks Matt & Dave! Sounds like the bug is fixed then.

*Dale :)