New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.
Starred by 9 users

Issue metadata

Status: Fixed
Owner:
Last visit > 30 days ago
Closed: Jan 2011
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug

Blocking:
issue 66078
issue 70992

Restricted
  • Only users with EditIssue permission may comment.



Sign in to add a comment
link

Issue 64848: Implement Mozilla's requestAnimationFrame API

Reported by darin@chromium.org, Nov 30 2010 Project Member

Issue description

Implement Mozilla's requestAnimationFrame API

See discussion here:
http://lists.w3.org/Archives/Public/public-webapps/2010OctDec/0644.html

In reference to the API described here:
http://weblogs.mozillazine.org/roc/archives/2010/08/mozrequestanima.html

We should implement something like this in WebKit.
 

Comment 1 by nduca@chromium.org, Dec 1 2010

Comment 2 by gman@google.com, Dec 1 2010

So I guess just to list this here if anyone is looking. The proposal seems generally okay. The issues are

1) We need this to be an element level thing, not a window level thing. Mozilla has been convinced I believe.

The example is boingboing.net, it has around 10 flash areas at any given time. If all of them were instead canvases constantly animating the perf of the page would be every slow. Flash doesn't draw when offscreen, neither should we.

This means changing window.requestAnimationFrame to element.requestAnimationFrame. It also means removing the beforePaint event and instead required requestAnimationFrame take a callback

2) We need this to never fire if the element is offscreen.

Currently Moz wants them called from time to time while offscreen.

The argument for NOT calling when offscreen. If you have 4 tabs of boingboing.net (or several news sites) there would be 40 elements requesting animation frames. When each one fires it will need access to all of it's resources (images). This means even if each one is only called once every second or 2 there will be possibly be tons of memory trashing as each element's resources are swapped back in.

The argument against is that developers might write code the expects to get called back that never gets called back. An example might be if gmail had an animation before saving an email. you pick save, the animation starts, you hide the tab. The mail is never saved unless you expose the tab again.

I think that's it. It's possible we could just never call callbacks when offscreen in chrome and firefox can call them from time to time and developers will deal with it but it would be nice to get agreement here.

Comment 3 by nduca@chromium.org, Dec 1 2010

Dislike the term "beforePaint" but that's probably because I'm too close to the implementation.

Agreed that the beforePaint callback shouldn't occur when a tab is hidden.

While I understand the benefits of a per-element callback as well as a desired refresh interval  for a "clean architecture", I suspect that in practice they may not buy as much as we hope.

Since the "make me a new frame" callback" can dirty the DOM in any way, we still need to run our usual invalidate/setNeedsUpdate(rect)/setNeedsCommit logic to figure out whether we need to re-paint any layers, just update some layer transforms, schedule a draw with a new WebGL rendertarget, etc. That is to say, we can fire a "make me a new frame" callback as much as we want and simply check the dirty bits that we already maintain to decide whether to actually redraw the screen or not.

The only place where this does not work is for 2D and 3D APIs, since for those elements, "requesting a new frame" would actually issue new drawing commands. Can we fix this with a simple isVisible method on the canvas element?

Similarly, for the 11hz and 10hz problem, can we simply fire the callback at 60hz and do nothing if it fires and nothing is dirtied? Sure we called into JS needlessly, but other than that, nothing was done right?

I'm actually a little confused how we'd implement a "update this element at such and such hertz" feature... when the threaded compositor is enabled, "beginning a frame" is no longer a serial process. It may look something like:
1. compositor thread decide a new frame should happen. It posts a task to the WebKit thread with its current time for firing the beforePaint events.
2a. WK thread: The paint callbacks begin running, triggering layout and possibly webgl/canvas calls.
2b. Compositor thread: waits a fixed amount of time, eg 15ms, for that paint to complete. If it hasn't, it hasn't,   
3. If no timeout: we may have invalidations. Issue paint calls to webkit main and block until the remaining frame budget has been consumed.
4. Update the CSS and scroll animation controllers with the time for the start of the frame
5. Draw using the layer tree and latest animation values. Draw chekerboards where damage occured and a timeout occurred.
6. Make a note of whether the frame we draw was incomplete, due to pending updates OR due to webgl/canvas commands still pending. Plan to make another frame in that case.

Where do we track these "needs update" timers? Is it on the main thread? I think complexities arise either way that make me think the bookkeeping might not be worth the win...

Comment 4 by gman@google.com, Dec 1 2010

Just be clear, I believe moz has already agreed to complete get rid of "beforePaint". There will be no "event" anymore. There is only

element.requestAnimationFrame(callback);

That's it. Also there is no "at xxx hz".

As far as "isVisible" if the goal is that most areas will not hog the machine when offsceen then it would be best if it's as easy as possible. If it was me I'd personally add setIntervalIfVisible(callback, rate, opt_element) because then the only thing that would change for most animation is s/setInterval(/setIntervalIfVisible(/ but so far moz isn't into that idea. I get why, they want requestAnimationFrame to work similar to how CSS animation works so the 2 can stay in sync.

Anyway, the point is, it's less work for the developer to switch setInterval to requestAnimationFrame than it is to also get them to add checks for isVisible. It might seem simple but I think there is enough of a difference that it will be less likely to happen. Of course even getting them to use requestAnimationFrame will be hard. I'm hoping users will complain when pages hog the browser when not visible putting pressure on sites to stop hogging.

Also, isVisible can already be implemented in JavaScript, checking for an elements position relative to which part of the page is visible. Apparently this can all be checked from JavaScript. I just don't believe people will do it.

As for your question on how it works. I believe it works the same as CSS animations. I don't know how CSS animations work. Are they part of the compositor? I would assume they are part of the renderer. The renderer posts some thing that causes CSS animations to update and therefore mark something as dirty and the compositor runs again. Effectively requestAnimationFrame just does exposes that process to JavaScript.

Comment 5 by dar...@gmail.com, Dec 4 2010

http://poppit.pogo.com/hd/PoppitHD.html is a good example of a canvas based app, which desperately needs an API like this.  It burns CPU while in a background tab :-(

Comment 6 by downch...@gmail.com, Dec 13 2010

Where are the posts about dropping paint events, and adding element.*?
window.requestAnimationFrame seems simple enough.
Would element.* ignore requests if the element is set to visibility:hidden/display:none?


These seem to be the most up to date docs:
window.requestAnimationFrame(callback)
https://developer.mozilla.org/en/DOM/window.mozRequestAnimationFrame
http://lists.w3.org/Archives/Public/public-webapps/2010OctDec/0649.html

Poppit could use onblur to tone-down some of the work.

CSS view-mode has a minimized option, matchMedium allows for the script to check when a tab is not visible.

CSS view-mode:
http://dev.w3.org/2006/waf/widgets-vmmf/

 Issue #51500  mentions matchMedium landed in WebKit recently.
http://www.w3.org/TR/cssom-view/#media

Comment 7 by downch...@gmail.com, Dec 13 2010

Comment 8 by gman@chromium.org, Dec 13 2010

The posts about dropping the paint events are in the thread linked in the first post above.

onblur is not sufficient. There are plenty of apps that want to continue running when they don't have the focus. For example put CNN.com in one window, put gmail in another window. Tile the windows (both windows are visible). Make gmail have the focus. CNN.com would still prefer if their visible ads are still running.

window.requestAnimationFrame is not sufficient. See example of boingboing.net above (or pretty much any other large blog with lots of multi-media)for why window.requestAnimationFrame is not enough.

CSS view-mode and window.matchMedia events are also not sufficient for the same reasons.

Comment 9 by downch...@gmail.com, Dec 13 2010

My comments were about Poppit. Supporting HTML 5 ads is another matter.
I don't think view-mode is active in window.matchMedia in the current webkit.

I didn't see anything in the thread about element.requestAnimationFrame.

With HTML 5 ads, it's pretty likely that each will be served from an iframe, each having one window.requestAnimationFrame call to manage.

The RAF proposal, as I understand it, tries to keep things simple by allowing only one active callback per window, which is then removed when it's called. The state of the window (and the <iframe>) could be factored into when and how requestAnimationFrame callbacks are triggered.

Thrashing is really managed by the UA anyway; if there are 30 tabs open, it might as well not answer requestAnimationFrame so frequently. The current way things work, with setTimeout, is the worst-case scenario anyway.

Comment 10 by gman@google.com, Dec 13 2010

From the thread
http://lists.w3.org/Archives/Public/public-webapps/2010OctDec/0696.html
http://lists.w3.org/Archives/Public/public-webapps/2010OctDec/0698.html

Ads are rarely in iframes. Ads often want to hover things over the main page or break out of their element. Putting them in iframes doesn't allow this.

Comment 11 by downch...@gmail.com, Dec 13 2010

Thanks for posting those two. It's quite a thread. My gut tells me that window.requestAnimationFrame will be a lot more straightforward -- easier to program -- than element.requestAnimationFrame. As you've pointed out, there are a lot of options for integrating the element with CSS, on element.* hooks. I'd suggest focusing on an initial window.* binding because of that lack of scope.

The thread tailed off at the end; there was a complaint about the work needed to sync RAF with audio. It's an interesting use case, not because of the audio, but because of the sync issue.

I've not used the current moz bindings. I do a lot of work with Canvas, I'll set some time aside to work with them. I'll keep an eye on this thread and let you know if I have any feedback. At this point, I'm still a little cloudy on the element.* binding, but I'm sure I'll understand it given more time.

Sounds like its primary purpose is to check offsetTop against scrollTop, and other such measures, to ensure that a callback is only fired when the element comes into view. Is that about right?

Comment 12 by gman@google.com, Dec 13 2010

I don't see how element.requestAnimationFrame is hard. Browsers already optimize CSS animations so they don't repaint when off the screen. They already handle 3D CSS as well where I doubt offsetTop and scrollTop really have any meaning.

The browser is free to call requestAnimationFrame callbacks as often as it wants. If it wants to all every one of them every frame no matter if the element is on screen or not it is free to do so but putting it on element gives the browser a chance to optimize. Putting it on window removes the ability of the browser to optimize. It would be much better if pages were written today to be optimizable vs starting with something that is not optimizable and then asking everyone to change their pages some time in the future.

Comment 13 by jam...@chromium.org, Dec 16 2010

Status: Started
I'm working with the g-man on an implementation.

Comment 14 by lrb...@gmail.com, Dec 25 2010

Hi,

I've made it possible to use mozRequestAnimationFrame to control animations is jQuery: http://lrbabe.github.com/jquery-interval-bookmarklet/demo.html (only in Firefox4 of course).

I found out that this currently doesn't improve performance of animations, but quite the opposite. (other findings here: http://lrbabe.github.com/jquery-interval-bookmarklet/)
I'm pretty sure that letting the browser choose the best frame rate is the way to go, but there is either a problem with Mozilla's implementation or my javascript code.

Anyway, I would prefer something similar to the proposed setIntervalIfVisible() (or setFrameInterval) because it would be far easier for libraries to leverage this mechanism. It would be adopted as a faster pace and benefit our websites earlier.

This should also close the debate of window.requestAnimationFrame vs. element.requestAnimationFrame

Regards,
Louis-RĂ©mi

Comment 15 by jam...@chromium.org, Jan 5 2011

Preliminary WebKit patch available here: https://bugs.webkit.org/show_bug.cgi?id=51218, chromium side here: http://codereview.chromium.org/6007006.  Still needs a bit of work before landing - in particular I need to generate correct animation timestamps in RenderWidget, propagate them through to the animation controller, and add some throttling to avoid invoking the callbacks faster than we can actually produce frames.

Comment 16 by bugdroid1@chromium.org, Jan 6 2011

Project Member
The following revision refers to this bug:
    http://src.chromium.org/viewvc/chrome?view=rev&revision=70626

------------------------------------------------------------------------
r70626 | jamesr@chromium.org | Thu Jan 06 10:54:29 PST 2011

Changed paths:
 M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/renderer/render_widget_fullscreen_pepper.cc?r1=70626&r2=70625&pathrev=70626

Chromium support for webkitAnimationTime property

These are the chrome changes needed for https://bugs.webkit.org/show_bug.cgi?id=51952.

BUG= 64848 
TEST=layout tests

Review URL: http://codereview.chromium.org/5962018
------------------------------------------------------------------------

Comment 17 by bugdroid1@chromium.org, Jan 11 2011

Project Member
The following revision refers to this bug:
    http://src.chromium.org/viewvc/chrome?view=rev&revision=70974

------------------------------------------------------------------------
r70974 | jamesr@chromium.org | Mon Jan 10 16:44:24 PST 2011

Changed paths:
 M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/renderer/render_widget_fullscreen_pepper.cc?r1=70974&r2=70973&pathrev=70974

Revert 70626 - Chromium support for webkitAnimationTime property

These are the chrome changes needed for https://bugs.webkit.org/show_bug.cgi?id=51952.

BUG= 64848 
TEST=layout tests

Review URL: http://codereview.chromium.org/5962018

TBR=jamesr@chromium.org
Review URL: http://codereview.chromium.org/6161005
------------------------------------------------------------------------

Comment 18 by bugdroid1@chromium.org, Jan 20 2011

Project Member
The following revision refers to this bug:
    http://src.chromium.org/viewvc/chrome?view=rev&revision=71909

------------------------------------------------------------------------
r71909 | jamesr@chromium.org | Wed Jan 19 18:39:36 PST 2011

Changed paths:
 M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/renderer/render_widget_fullscreen_pepper.cc?r1=71909&r2=71908&pathrev=71909

Chromium support for window.webkitRequestAnimationFrame()

This is a very simple scheduler that targets 100fps.

BUG= 64848 
TEST=layout tests

Review URL: http://codereview.chromium.org/6136005
------------------------------------------------------------------------

Comment 19 by jam...@chromium.org, Jan 20 2011

WebKit side landed at http://trac.webkit.org/changeset/76194, need to wait for the next WebKit roll to land the rest of the chromium side.  Seems to work well for http://webglsamples.googlecode.com/ on my box.

Comment 20 by bugdroid1@chromium.org, Jan 21 2011

Project Member
The following revision refers to this bug:
    http://src.chromium.org/viewvc/chrome?view=rev&revision=72230

------------------------------------------------------------------------
r72230 | jamesr@chromium.org | Fri Jan 21 15:34:50 PST 2011

Changed paths:
 M http://src.chromium.org/viewvc/chrome/trunk/src/webkit/tools/test_shell/webwidget_host_gtk.cc?r1=72230&r2=72229&pathrev=72230
 M http://src.chromium.org/viewvc/chrome/trunk/src/build/features_override.gypi?r1=72230&r2=72229&pathrev=72230
 M http://src.chromium.org/viewvc/chrome/trunk/src/webkit/tools/test_shell/webwidget_host_win.cc?r1=72230&r2=72229&pathrev=72230
 M http://src.chromium.org/viewvc/chrome/trunk/src/webkit/tools/test_shell/test_webview_delegate.h?r1=72230&r2=72229&pathrev=72230
 M http://src.chromium.org/viewvc/chrome/trunk/src/webkit/tools/test_shell/test_shell.gypi?r1=72230&r2=72229&pathrev=72230
 M http://src.chromium.org/viewvc/chrome/trunk/src/webkit/tools/test_shell/test_webview_delegate.cc?r1=72230&r2=72229&pathrev=72230
 M http://src.chromium.org/viewvc/chrome/trunk/src/webkit/tools/test_shell/webwidget_host.h?r1=72230&r2=72229&pathrev=72230
 M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/renderer/render_widget.h?r1=72230&r2=72229&pathrev=72230
 M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/renderer/render_widget.cc?r1=72230&r2=72229&pathrev=72230
 A http://src.chromium.org/viewvc/chrome/trunk/src/webkit/tools/test_shell/webwidget_host.cc?r1=72230&r2=72229&pathrev=72230
 M http://src.chromium.org/viewvc/chrome/trunk/src/webkit/tools/test_shell/mac/webwidget_host.mm?r1=72230&r2=72229&pathrev=72230

Chromium support for window.webkitRequestAnimationFrame()

This is a very simple scheduler that targets 100fps.

BUG= 64848 
TEST=layout tests

Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=71909

Review URL: http://codereview.chromium.org/6136005
------------------------------------------------------------------------

Comment 21 by jam...@chromium.org, Jan 21 2011

Status: Fixed
I've manually tested this on the http://webglsamples.googlecode.com/ demos, and thanks to gman this seems to have the properties we want (specifically webgl in a background tab doesn't use any CPU or GPU resources).  Closing this as fixed, we can open more bugs about implementation issues as needed.

Comment 22 by bugdroid1@chromium.org, Oct 12 2012

Project Member
Blocking: -chromium:66078 -chromium:70992 chromium:66078 chromium:70992
Labels: Restrict-AddIssueComment-Commit
This issue has been closed for some time. No one will pay attention to new comments.
If you are seeing this bug or have new data, please click New Issue to start a new bug.

Comment 23 by bugdroid1@chromium.org, Mar 10 2013

Project Member
Labels: -Area-WebKit -Mstone-10 Cr-Content M-10

Comment 24 by bugdroid1@chromium.org, Mar 13 2013

Project Member
Labels: -Restrict-AddIssueComment-Commit Restrict-AddIssueComment-EditIssue

Comment 25 by bugdroid1@chromium.org, Apr 6 2013

Project Member
Labels: -Cr-Content Cr-Blink

Sign in to add a comment