New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 682338 link

Starred by 2 users

Issue metadata

Status: Available
Owner: ----
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 1
Type: Bug



Sign in to add a comment

mjpeg video in <img> doesn't pause when not visible

Project Member Reported by primiano@chromium.org, Jan 18 2017

Issue description

[PUBLIC BUG]

Follow up from internal discussion (Android Chrome beta burned 4GiB of data in background over night).
An internal user reported ~ 4 GB of background data usage. The possible cause is a tab left open in the background that was streaming a MJPEG video from a security camera. The user reports screen was probably off and chrome was definitely backgrounded.

Is this behavior WAI? Media folks, any thoughts?
 
In the case of classic media playback, we have the following behaviour:
- video without an audio track will never be played in the background
- video with an audio track will be automatically paused and can be resumed from the notification
- audio-only will run in the background seamlessly.

However, it's not because the users see a "video" that it is actually a <video> ;) It could as well be a XHR or WebRTC stream sending pixels to a <canvas>. If that's the case, whatever rule we follow is meaningless. I don't think Chrome blocks downloads when a page is in the background apart from slowing down timers which might slow down downloads as a consequence.

caywood@, do you know how the stream viewer is implemented?
Components: -Blink>Media Blink>Loader
mlamouri and I just checked, I can repro this issue as follows:
1) Open a MJPEG stream, e.g., http://webcam.st-malo.com/axis-cgi/mjpg/video.cgi?resolution=352x288
2) put chrome to backgroun and turn screen off
3) adb root + shell to the device, and check with tcpdump that data is still being transferred: tcpdump -n tcp port 80 (or 443)

together with mlamouri we checked that the mjpeg stream is treated as an <img> and not a <video>.
From the network viewpoint a mjpeg stream is a never-ending  multipart/x-mixed-replace;boundary= HTTP download, where every chunk is a JPEG frame.
There is NO client-side XHR involved

Note  <img src="http://...some-mjpeg-stream"> is quite popular on IP camera web interfaces. So, after all, this bug might not be that rare to trigger.
 

Comment 3 by caywood@google.com, Jan 19 2017

You got it- it's implemented as an <img> tag with the src being a never-ending multipart stream.



Cc: hajimehoshi@chromium.org
Owner: yhirano@chromium.org
Status: Assigned (was: Unconfirmed)
Tentatively assigning to me.
RendererSchedulerImpl::OnRendererBackgrounded
ChildThreadImpl::OnProcessBackgrounded
Cc: ojan@chromium.org
+ojan@. Do you have any thoughts?

Comment 8 by ojan@chromium.org, Feb 15 2017

Cc: skyos...@chromium.org altimin@chromium.org animohan@chromium.org jkarlin@chromium.org bengr@chromium.org
Components: Blink>Scheduling Programs>Alignment
Labels: -Pri-3 Pri-2
The current Chrome Android behavior is that after 5 minutes we pause timers in background tabs. We've had it on the backlog for a while now to pause everything (timers, loading tasks, WebSockets, etc). We've just been blocked on having an engineer to do the work. So, very happy if you'd be willing to take this on.

Ideally, if you're willing, we could do the work of pausing all work in background tabs on Android after 5 minutes, but if you'd like to focus on the specific problem of pausing downloading bytes off the network, that would be a fine first step.

You can pause all tasks by pausing the TaskRunners in the blink_scheduler. That won't catch everything yet, but it will over time as we get all the tasks routed through the scheduler. I don't know if that will implicitly pause loading bytes off the network or if the browser process also needs to be informed to stop downloading things.

Happy to help in any way I can to guide the work (e.g. review intent threads, review the design of what we're blocking, etc.).
Another option here is to pause the MJPEG stream the same way we pause out-of-view GIFs and requestAnimationFrame work.

Comment 10 by ojan@chromium.org, Feb 17 2017

Related is issue 452829.
Thank you!

I'd like to fix this one specific case because suspending all sounds somewhat controversial.

The simplest way is to cancel loading when the renderer is suspended (RendererSchedulerImpl::MainThreadOnly::renderer_suspended, I assume). Are you fine with the idea? Can we check if the renderer is suspended from platform/loader?

#11: One way to do that would be to connect the MJPEG downloading logic to the display logic so that the download stops if no frames are being displayed. I'm not sure how feasible this is though.

Another option would be to route some step of the download operation through a timer which will be throttled and ultimately paused.

Comment 13 by ojan@chromium.org, Feb 22 2017

Cc: chrishtr@chromium.org
Summary: mjpeg video in <img> doesn't pause when not visible (was: Android Chrome beta burned 4GiB of data in background over night)
At a behavior level, I think this should behave the same as animated gifs and muted, autoplayed video. Namely:
-Pause when offscreen (all platforms)
-Pause when in a background tab (all platforms)
-Pause when the screen is turned off or chrome is backgrounded on mobile

Do we have agreement on that?

Ideally we would actually reuse the animated gif logic for the pausing with some extra plumbing to the network stack to pause downloading bytes. +chrishtr since he's familiar with the animated gif logic.
Sounds like a plan to me.
> #13
Sounds reasonable. If "going to background / foreground" notification is arrived to each ImageResource or ImageResourceContent, I can utilize it.

I'd like to know how animated gifs are stopped. chrishtr@, could you provide a code search link?

Thanks,
Ojan, I think point #3 (screen turned off or background Chrome) should come for free with #2.
Animated GIFs throttling is implemented via a concept of "delayed paint invalidation",
because animated GIF painting is controlled by timers in Blink.
See PaintInvalidationDelayedFull and its callsites:

https://cs.chromium.org/chromium/src/third_party/WebKit/Source/platform/graphics/PaintInvalidationReason.h?sq=package:chromium&dr&l=44

How are mjpegs animated?
Blink draws a MJPEG frame as soon as it receives a MIME part. There is no timing concept in MJPEG AFAIK.
Owner: ----
Status: Available (was: Assigned)

Comment 20 by ojan@chromium.org, May 8 2018

Cc: -ojan@chromium.org
Labels: -Pri-2 Pri-1
Cc: rajendrant@chromium.org
It seems the issue is still valid - MJPEGs are still active in background tabs.

yhirano@, did you find out what would be necessary to implement mjpeg suspension?
I think we cannot suspend MJPEG. All we can do is to stop MJPEG animation.

Sign in to add a comment