canvas.captureStream() empty on background tab
Reported by
maxstol...@gmail.com,
Aug 18 2016
|
|||||||
Issue descriptionUserAgent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36 Steps to reproduce the problem: 1. Populate a canvas element 2. captureStream that element 3. Send stream via WebRTC 4. Create receiver of stream and view in browser 4. Move to new tab from broadcaster browser 5. Observe empty stream on receiving end What is the expected behavior? What went wrong? The stream goes blank when the broadcaster backgrounds the tab with the canvas element Did this work before? N/A Chrome version: 52.0.2743.116 Channel: stable OS Version: OS X 10.11.1 Flash Version: Shockwave Flash 22.0 r0
,
Aug 19 2016
,
Aug 19 2016
I work with the OP of this issue and I threw together a demo. To be clear this doesn't show the problem exactly but replicates some of the undesirable behavior. The problem happens when you switch to a new tab thus hiding the source canvas. This is the only way we know how to replicate the problem that doesnt require an external WebRTC server. Also, I haven't been able to get this to work with other browsers but I am not sure if that has to do with the fact that the WebRTC support is spotty or something else. I have forked the https://github.com/webrtc/samples project and modified a single tab WebRTC peerconnection demo to stream a canvas using captureStream. When you set the canvas to display:none the stream's animation stops. When display:inline is set it resumes. Steps to reproduce: 1) Go to https://mattrenaud.github.io/samples/src/content/peerconnection/pc1/ 2) Start the stream by clicking "Start" then "Call". 3) Observe two clock animations (note that the colors may be a little different) 4) Click "Toggle" a few times waiting a few seconds between each click 5) Observe one clock hides and the other clock's animation stops. Expected behavior: The second clock's animation continues regardless of whether the canvas is visible or not.
,
Aug 19 2016
I put together a different demo that I think demonstrates the problem much more directly and clearly. I again modified a sample from https://github.com/webrtc/samples. This time I used MediaRecorder to save the canvas.captureStream so that we can see the behavior when the tab is switched. Steps to reproduce: 1) Go to https://mattrenaud.github.io/samples/src/content/getusermedia/record/ 2) click "Start Recording". 3) Wait 3 seconds the switch tabs. 4) Wait for another 10 seconds and come back to the demo tab. 5) Wait 3 seconds and click "Stop Recording". 6) Click "Play" 7) Observe recorded video. Notice that the recording freezes during the time that the tab is "inactive". Expected behavior: Recording should be continuous and not freeze during tab switching Also, I tried this new demo in Firefox and it works (mostly). The animation is choppy in Firefox because the requestAnimationFrame is called less frequently when the tab is inactive. But canvas.captureStream is getting updated.
,
Aug 19 2016
It looks like this is a known issue: "- Canvas in background tabs does not update properly." https://groups.google.com/forum/#!topic/discuss-webrtc/fpbiC2HOgUY Is there an existing issue in the tracker for this?
,
Aug 19 2016
+niklase per IRC discussion.
,
Aug 21 2016
Looks like emircan worked on a patch for this bug, but it wasn't accepted: https://codereview.chromium.org/1686823003/. I can't seem to find a live issue for this bug, though.
,
Aug 22 2016
+cc emircan per IRC request
,
Aug 22 2016
,
Aug 26 2016
I see this issue is marked as 'Needs-Feedback'. Is anything else needed from us?
,
Aug 30 2016
Thanks for the demos. The problem you have here is that your canvas animation loops are driven by requestAnimationFrame(). When tab is backgrounded requestAnimationFrame() may be reduced to a lower rate or stopped. Since there is nothing new drawn on the canvas, you don't get updated capture either. AFAICT, the idea behind these limitations is to limit background tabs from impacting the performance & battery life. Your workaround would be running your loop on something else that would be work while being backgrounded, i.e. <video> event listeners.
,
Aug 30 2016
Hi there, thanks for the reply. I work with the people who reported this bug and I think we did a poor job isolating the behavior we wanted to show in the demos we linked. Sorry about that. Sure enough requestAnimationFrame() doesn't run when the tab has been backgrounded, and that explains the still clock in the previous demo. I've since tweaked the clock demo to show that even if the canvas is being drawn to (in this case using setTimeout in place of requestAnimationFrame) the stream will not show the update. https://kpetrovi.ch/samples/src/content/getusermedia/record/ If you examine console, you'll see the clock() function being called, albeit not as often when backgrounded. However the recording will not show any movement of the clock hands for the entire time the tab was backgrounded. It was our hope that updates to the canvas would be shown in the capturedStream, even when backgrounded. Is the current behavior intended? What is the cause? Is there any way around it?
,
Sep 9 2016
emircan: can you respond to the new information in #12? I believe we have got feedback now, so removing that label.
,
Sep 12 2016
kylepetrovich@, sorry for my late response. I wanted to get some other opinions on this issue. Thanks a lot for the clear demos you provided. Canvas does not paint/render new frames when backgrounded and as a result, the canvas capture does not contain any new items. AFAIK, there isn't a way to paint&capture canvas when tab is backgrounded right now in the way you expect on the demos using Chrome. The spec states that capture is tied to paint, and I am going to contact people bring back this issue from earlier. https://github.com/whatwg/html/issues/924#issuecomment-202704417
,
Oct 17 2016
,
Nov 1 2016
emircan: Thanks so much for taking up this bug. I'm running into it as well. Any rough guesses on the timeline for it? Thanks!
,
Jan 19 2017
Thank you for working on this bug. Any updates on it?
,
May 24 2017
Is there a work around for this bug? I tried disabling blur event, hacktimer.js or pop-out window without blur/tab-change.
,
Jun 27 2017
There is a workaround on stackoverflow[1] The basics is to use an homemade timing loop based on WebAudioAPI, which is not restricted to tab focus, and to record the webgl canvas from a 2DContext stream rather than directly from the webgl one. [1] https://stackoverflow.com/questions/44156528/canvas-doesnt-repaint-when-tab-inactive-backgrounded-for-recording-webgl/44172801#44172801
,
Mar 20 2018
Note that this issue has become more problematic: apparently, recently apple seems to trigger the chrome background throttling when another window is overlaid over chrome -- even partially -- and not just when chrome itself is mimized or a tab in the background ; i can't seem to find any official information about this though. |
|||||||
►
Sign in to add a comment |
|||||||
Comment 1 by thestig@chromium.org
, Aug 18 2016