New issue
Advanced search Search tips

Issue 639105 link

Starred by 14 users

Issue metadata

Status: Assigned
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Mac
Pri: 2
Type: Bug



Sign in to add a comment

canvas.captureStream() empty on background tab

Reported by maxstol...@gmail.com, Aug 18 2016

Issue description

UserAgent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36

Steps to reproduce the problem:
1. Populate a canvas element
2. captureStream that element
3. Send stream via WebRTC
4. Create receiver of stream and view in browser
4. Move to new tab from broadcaster browser
5. Observe empty stream on receiving end

What is the expected behavior?

What went wrong?
The stream goes blank when the broadcaster backgrounds the tab with the canvas element

Did this work before? N/A 

Chrome version: 52.0.2743.116  Channel: stable
OS Version: OS X 10.11.1
Flash Version: Shockwave Flash 22.0 r0
 
Components: Blink>Canvas Blink>WebRTC
It might be helpful to have a live webpage that demonstates the issue. Does the same webpage work in any other browser?
Labels: Needs-Feedback

Comment 3 by renau...@gmail.com, Aug 19 2016

I work with the OP of this issue and I threw together a demo. To be clear this doesn't show the problem exactly but replicates some of the undesirable behavior. The problem happens when you switch to a new tab thus hiding the source canvas. This is the only way we know how to replicate the problem that doesnt require an external WebRTC server.

Also, I haven't been able to get this to work with other browsers but I am not sure if that has to do with the fact that the WebRTC support is spotty or something else.

I have forked the https://github.com/webrtc/samples project and modified a single tab WebRTC peerconnection demo to stream a canvas using captureStream. When you set the canvas to display:none the stream's animation stops. When display:inline is set it resumes.

Steps to reproduce:
1) Go to https://mattrenaud.github.io/samples/src/content/peerconnection/pc1/
2) Start the stream by clicking "Start" then "Call".
3) Observe two clock animations (note that the colors may be a little different)
4) Click "Toggle" a few times waiting a few seconds between each click
5) Observe one clock hides and the other clock's animation stops.

Expected behavior:
The second clock's animation continues regardless of whether the canvas is visible or not.

Comment 4 by renau...@gmail.com, Aug 19 2016

I put together a different demo that I think demonstrates the problem much more directly and clearly. I again modified a sample from https://github.com/webrtc/samples. This time I used MediaRecorder to save the canvas.captureStream so that we can see the behavior when the tab is switched.

Steps to reproduce:
1) Go to https://mattrenaud.github.io/samples/src/content/getusermedia/record/
2) click "Start Recording".
3) Wait 3 seconds the switch tabs.
4) Wait for another 10 seconds and come back to the demo tab.
5) Wait 3 seconds and click "Stop Recording".
6) Click "Play"
7) Observe recorded video. Notice that the recording freezes during the time that the tab is "inactive".

Expected behavior:
Recording should be continuous and not freeze during tab switching

Also, I tried this new demo in Firefox and it works (mostly). The animation is choppy in Firefox because the requestAnimationFrame is called less frequently when the tab is inactive. But canvas.captureStream is getting updated.
It looks like this is a known issue:

"- Canvas in background tabs does not update properly."
https://groups.google.com/forum/#!topic/discuss-webrtc/fpbiC2HOgUY

Is there an existing issue in the tracker for this?
Cc: niklase@chromium.org
+niklase per IRC discussion.
Looks like emircan worked on a patch for this bug, but it wasn't accepted: https://codereview.chromium.org/1686823003/. I can't seem to find a live issue for this bug, though.


Cc: emir...@chromium.org
+cc emircan per IRC request

Comment 9 by junov@chromium.org, Aug 22 2016

Owner: emir...@chromium.org
Status: Assigned (was: Unconfirmed)
I see this issue is marked as 'Needs-Feedback'. Is anything else needed from us?
Thanks for the demos. The problem you have here is that your canvas animation loops are driven by requestAnimationFrame(). When tab is backgrounded requestAnimationFrame() may be reduced to a lower rate or stopped. Since there is nothing new drawn on the canvas, you don't get updated capture either. AFAICT, the idea behind these limitations is to limit background tabs from impacting the performance & battery life. Your workaround would be running your loop on something else that would be work while being backgrounded, i.e. <video> event listeners.
Hi there, thanks for the reply.

I work with the people who reported this bug and I think we did a poor job isolating the behavior we wanted to show in the demos we linked. Sorry about that. Sure enough requestAnimationFrame() doesn't run when the tab has been backgrounded, and that explains the still clock in the previous demo. 

I've since tweaked the clock demo to show that even if the canvas is being drawn to (in this case using setTimeout in place of requestAnimationFrame) the stream will not show the update.

https://kpetrovi.ch/samples/src/content/getusermedia/record/

If you examine console, you'll see the clock() function being called, albeit not as often when backgrounded. However the recording will not show any movement of the clock hands for the entire time the tab was backgrounded.

It was our hope that updates to the canvas would be shown in the capturedStream, even when backgrounded. Is the current behavior intended? What is the cause? Is there any way around it?
Labels: -Needs-Feedback
emircan: can you respond to the new information in #12? I believe we have got feedback now, so removing that label.
kylepetrovich@, sorry for my late response. I wanted to get some other opinions on this issue. Thanks a lot for the clear demos you provided.

Canvas does not paint/render new frames when backgrounded and as a result, the canvas capture does not contain any new items. AFAIK, there isn't a way to paint&capture canvas when tab is backgrounded right now in the way you expect on the demos using Chrome. The spec states that capture is tied to paint, and I am going to contact people bring back this issue from earlier.
https://github.com/whatwg/html/issues/924#issuecomment-202704417
Components: -Blink>Canvas -Blink>WebRTC Blink>MediaStream>CaptureFromElement
emircan: Thanks so much for taking up this bug. I'm running into it as well.  Any rough guesses on the timeline for it?

Thanks!

Comment 17 by chtho...@gmail.com, Jan 19 2017

Thank you for working on this bug. Any updates on it? 
Is there a work around for this bug? I tried disabling blur event, hacktimer.js or pop-out window without blur/tab-change.
There is a workaround on stackoverflow[1]

The basics is to use an homemade timing loop based on WebAudioAPI, which is not restricted to tab focus, and to record the webgl canvas from a 2DContext stream rather than directly from the webgl one.

[1] https://stackoverflow.com/questions/44156528/canvas-doesnt-repaint-when-tab-inactive-backgrounded-for-recording-webgl/44172801#44172801

Comment 20 Deleted

Note that this issue has become more problematic: apparently, recently apple seems to trigger the chrome background throttling when another window is overlaid over chrome -- even partially -- and not just when chrome itself is mimized or a tab in the background ; i can't seem to find any official information about this though.

Sign in to add a comment