As a quick note, --disable-background-timer-throttling has no effect. 

Could you grab a trace as described there: https://www.chromium.org/developers/how-tos/trace-event-profiling-tool/recording-tracing-runs?

From the scheduling perspective we're interested in toplevel, renderer.scheduler, disabled-by-default-renderer.scheduler and disabled-by-default.renderer.scheduler.debug categories. Media folks may be also interested in something else.

I've attached the trace, to provide a visual example of the latency increase over time I've also attached a short recording of the session during that trace.

The tab being throttled is "Rainway Client" or "play.rainway.ip"

Deleted: trace_TraceLog.json.zip 17.3 MB

trace_TraceLog.json.zip 17.3 MB Download

Deleted: 2017-11-28_14-25-54.webm 8.5 MB

	2017-11-28_14-25-54.webm 8.5 MB View Download

For the media folks so there is no confusion, the audio and video track are streamed separately. So the video has no actual audio track.  

-> Dale/Dan: I suspect the background video suspension/resumption is involved in this, and also that this is more general than MSE (probably impacts src=, too).

Ah, background video without audio disabled these days, so this is always going to fall out of sync. You will need to combine the audio/video into a single MediaSource element to avoid this.

That isn’t really possible if you want to maintain low latency. Sequence
doesn’t support audio on the same track and segments doesn’t really have
great “real-time” support. You have to skip across gaps of data otherwise
it will throw exceptions and by skipping you introduce latency each time.

We noted this on the MSE spec discussion and it seems a real-time mode is
being considered, however until one is we’re working within the implemented
version.

In a perfect world if audio is enabled on the page and the video box
contains the flags for latency

  if (movie->moov->mvex->mehd && movie->moov->mvex->mehd->fragment_duration) {
            trex->track->Header->duration = 0;
            Media_SetDuration(trex->track);
            movie->moov->mvex->mehd->fragment_duration = 0;
        }

it should continue playing without interruption. Firefox works fine in this
regard.

+wolenetz for commentary.

Regardless of the MSE issues, you'll likely need to find another workaround. E.g., there are plenty of extensions that trigger pause like behavior and on mobile pausing has long been common for backgrounding.

One workaround you can use is to use the page visibility API and always seek to head upon page shown if you notice you're behind. I'd recommend listening for play/pause events to trigger this workaround as well.

@#9: I'm unclear what's meant by "Sequence doesn't support audio on the same track". If what's meant is "Sequence mode with multiple tracks is soon to be deprecated in both MSE spec and Chrome", that's correct. Other interpretations are likely moot due to that.

A couple related ideas come to mind that may help you, until the spec (and implementations) settle on a "play through gaps / skip over gaps" API for MSE vNext:

1) Use multiple single-track SourceBuffers on a single MediaSource object. The sequence mode behavior is well-defined for each single-track SourceBuffer.
2) If you have good understanding of the buffered time ranges that would result from appending a media segment in segments mode, you can polyfill sequence mode more reliably by inspecting SourceBuffer.buffered and adjusting timestampOffset on the SourceBuffer and/or autogenerating "no-op" video segments to fill the video gaps (assuming audio is continuous).

Note that Chrome M63 and M64 contain a preliminary correction of buffering to buffer media by PTS intervals, rather than DTS intervals, which I'm stabilizing currently. Once that's shipped on-by-default, results of SourceBuffer.buffered should be more interoperable across browsers like Firefox and Chrome for bytestreams like ISOBMFF with codecs like h.264 where PTS != DTS.  To get a preliminary look at this new (more compliant) Chrome behavior, try latest dev/canary with --enable-features=MseBufferByPts. We're aware of some complications (including underspec'ed behavior around SAP-Type-2 streams) for which this new buffering model may regress app behavior. (See also  bug 718641  for the bulk of the compliance work that landed behind that flag in M63.)

Thanks Matt. Also keep in mind there are many other normal reasons why the video stream might fall behind. decoder underflow (cpu or network related) or system suspend/resume will both cause decoding to fall behind the live edge. You'll likely want some general workaround to always bring you to the live edge when falling behind.

Given all that, I'm going to close this as WontFix / Working as intended.

Thank you all for the suggestions. While we already have seeking code in place to "catch up" the video once it gets behind, the issue here is more so that the liveliness of the video isn't be respected in favor of what I assume is saving battery life. This could be understandable on mobile, but for desktop users multi-tasking between tabs is a given and people attempting to use a remote desktop session are going to tab over often. Its not an enjoyable experience to  see your stream skipping or lagging behind every time you return to the page.


@#12: I'm a bit confused how 1 would solve this issue compared to using multiple source buffers for both audio and video. can you expand? The second suggestion seems useful, unfortunately though our audio is not continuous, this is why we opted for sending them separately. 

I would argue its beneficial to implement logic that respects the liveness of a stream and if its MEHD fragment durations are set to 0, that it isn't interrupted when tabbing over, at least on desktop until MSE has a full live solution.

Firefox is able to be put into the background without issue and from all workarounds we've tried thus far, latency never recovers in Chrome.