New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 460919 link

Starred by 35 users

Issue metadata

Status: Assigned
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: All
Pri: 2
Type: Bug

Blocked on: View detail
issue 406158
issue 406161



Sign in to add a comment

Chrome suffers from excessive mouse 'input lag' in rAF <canvas> animations

Reported by jer...@duckware.com, Feb 23 2015

Issue description

UserAgent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.115 Safari/537.36

Steps to reproduce the problem:
Visit www.vsynctester.com and move the mouse cursor around (left to right; and back; repeat).  By default, the mouse 'input lag' detector test at vsynctester.com is turned ON (if not, click on the 'wheel'/settings and turn on the detector).  Chrome fails the input lag test, by having an 'input lag' of more than one frame.

What is the expected behavior?
There should be no more than one frame of delay in modern web browsers on modern/fast CPU/GPU systems, which should place the red dot in the vsynctester.com 'input lag' detector directly under the mouse cursor (The red dot is a prediction of where the mouse cursor will be one frame into the future, so on a browser with one frame of input delay, the two cancel out, and the red dot is placed under the mouse cursor, even when it moves fast).

Firefox actually passes this test.  So on a 60Hz display, Firefox has a 1/60 (16.66ms) input delay.

However, it appears that Chrome has a two frame input lag (in the vsynctester.com options, change the '1' to a '2' in the 'input lag' test checkbox line).  So on a 60Hz display, Chrome has a 2*1/60 (33.33ms) input delay.

What went wrong?
The delay between receiving mouse input, and acting upon that in a <canvas> requestAnimationFrame() animation, and then the user actually seeing the results on the display -- vs where the cursor now is on the display -- is too long.  The result in Chrome is a very noticeable 'input lag', where the mouse has moved on and is well ahead of the animation now displayed on the screen.

Did this work before? N/A 

Chrome version: 40.0.2214.115  Channel: stable
OS Version: 6.1 (Windows 7, Windows Server 2008 R2)
Flash Version: Shockwave Flash 16.0 r0

I went back and checked Chrome 31, and it appears that Chrome 31 has an even worse three frame input delay -- so my guess is that is not a regression.

Chrome Canary 43.0.2312.0 has an incredibly strange behavior regarding input lag.  When I launch Canary, as long as it is not maximized, and I do NOT resize the application window, Canary has a ONE frame input delay.  The moment I resize the Canary app window, the input delay switches to a TWO frame input delay.  This is likely related to Canary not being VSYNC synchronized until the Canary app window is resized (see issue 422000).

Of course, this 'input delay' is tightly related to VSYNC synchronization.  There is no reason that Chrome has to be 'dumb' and always resort to a 'safe' two frame delay.  Instead, Chrome can and should be (very smartly) adaptive (in realtime), and use a one frame delay when a one frame delay is possible, and switch to a two frame delay, when a one frame delay does not work.  The decision of which mode to use is being made constantly 'on the fly', with the rate of switches between the modes rate limited to smartly adapt to running conditions.  The result would be this:  For simple animations, a one frame delay.  For complex animations, a two frame delay, because one frame would cause jank.
 
Cc: briander...@chromium.org bajones@chromium.org kbr@chromium.org
Labels: -OS-Windows OS-All

Comment 2 by kbr@chromium.org, Feb 23 2015

Cc: sunn...@chromium.org
Labels: Cr-Internals-GPU-Scheduling

Comment 3 by nduca@chromium.org, Feb 23 2015

Labels: Hotlist-Scheduling
Honestly, I don't think it's worth the additional effort and complexity required to implement a switching system as described above, which sounds like another potential source of animation instability. 99% of applications (including most games) don't need that kind of tight latency.

If there is work to be done on chromium latency, it's on android, where touch lag can be as high as 300ms. At 60fps, that translates to latency of up to 20 frames, which truly is horrendous. Firefox for Android responds much, much faster (~3-4 frames in my estimation), so this is in no way a limitation of Android itself.

Comment 5 by mit...@mithis.com, Feb 27 2015

Cc: mit...@mithis.com

Comment 6 by jer...@duckware.com, Feb 27 2015

Whatever tests are running in the Canary 43.0.2315.0+ for VSYNC accuracy (1ms vs sub 1ms; see comment 138 in issue 422000) is also affecting input lag!!

It is very clear that when the prior method is used, there is around a 2-frame input delay.  However, for whatever reason, when Canary is experimenting with higher res vsync mode to get sub 1-ms timer accuracy, the input lag is increasing to 3 frames.

Comment 7 by jer...@duckware.com, Feb 27 2015

Also, I just tested Safari on OSX and found that Safari has a 1-frame input lag.

It would be nice to see Chrome match what Firefox and Safari can already do...
Actually, for any animation on a 60Hz display/system that could have run just fine at 120Hz (very common now as systems get faster) there is no reason why Chrome could not run those animations with a 1/2 frame input lag.  Consider it a challenge.  

Also, to reinforce the point made in comment #6, Canary has REGRRESSED from a 2-frame input lag to a 3-frame input lag, starting with 43.0.2315.0+.
Blockedon: chromium:406158 chromium:406161
See new related issue 464740 (Chrome is attempting to flatten inter-frame requestAnimationFrame() callback times; significant internal lag), which appears to be the trigger for an increase in the 'input lag' from 2-frames to 3-frames reported in #6.
Status: Available
The traces jerryj uploaded in issue 464740, indicate the Renderer and Browser do enter a high latency mode, which makes me more confident that the issues blocked on in #9 regarding latency recovery are the correct ones to block on.

With Brandon's fixes in issue 422000, we knew we were going to take a hit on latency sometimes because we are enabling vsync in cases where it wasn't enabled before.

We had to enable vsync to fix the smoothness issues, which were higher priority than the latency hit we've taken.

Tweaks to Brandon's patches will not fix the latency issue, so I think latency recovery is the best way forward.
Brandon's fix in issue 422000 accounts for a 1 frame to 2 frame 'input lag' change.  But the change from 2-frames to 3-frames is a regression, introduced between r317807 and r317814.
@briander, Chrome (Canary) will enter "high latency mode" all by itself on a hair trigger.  See issue 465105.  But Chrome 'release' is far worse and enters/exits high latency mode on a VERY periodic (like clockwork) basis for ALL requestAnimationFrame() animations (because 'release' is no longer vsync aligned and there is a Hz mismatch between the renderer and the back end buffer swaps) -- which begs the question as to how Chrome release went out the door in that shape.  It suggests no automated test exists that test if a known low latency animation is in fact running in low latency mode (so maybe that test should be written?).  The 2-frame to 3-frame input delay is now fully explained by issue 465105 (in summary, high latency mode in Chrome is broken, and once fixed, this delay will go away).

That only leaves the original 1-frame to 2-frame issue outstanding, which based upon the tone in comment #11, does not seem like chromium wants to address.  Too bad, since Safari has solved this issue today and has both smoothness, and low latency.
Apologies for making it sound like we don't want to fix this. We definitely do.

We are very close to being able to recover the latency lost due to Brandon's patch, there's just a couple blocking patches left before we can get there.

It sounds like there are multiple regressions though. Thanks for narrowing down the range for the second regression. I'll follow up separately in that bug.
briander/14, sorry! I read your #11 as an either/or.  So eventually being able to get smoothness and one frame of latency (or less) from Chrome is great.
Owner: briander...@chromium.org
Status: Assigned
brianderson@ I'm assigning this to you so that we don't lose track of it.
Labels: -Hotlist-Scheduling Cr-Blink-Scheduler
Labels: -Cr-Blink-Scheduler Cr-Blink-Scheduling

Comment 19 by pcpl...@gmail.com, Oct 15 2015

In October 2015, there is still INTENSE input lag and slow down because of Chrome's **** animations. I am using RDP and a weak GPU.

For EACH Chrome animations, the mouse is slowing down, espescially when I am hovering a ling there is a mini information text about the link on botto left or right appearing, and this is slowing down the machine.

I have disabled ALL GPU acceleration available with : --disable-gpu, disable GPU acc in the options, etc but Chrome still seems to accelerate those damn animations. Or even enable them.

Comment 20 by mit...@mithis.com, Feb 24 2016

Cc: -mit...@mithis.com

Comment 21 by mit...@mithis.com, Feb 24 2016

Cc: tansell@chromium.org
I believe the entire problem (under Windows) may be caused by Chrome swapping buffers during the vertical blanking period -- which under Windows with Aero off is the proper location.  BUT with Areo ON, that swap location is the absolute worst possible swap location because that swap location adds one frame of input lag.

Namely, swapping (with Aero ON) during vblank is NOT the end of current frame, but rather is already the beginning of the NEXT frame.  Meaning that the frame Chrome just sent to the OS has to wait an entire frame of time before being actually displayed on the screen.

Can anyone confirm/comment on this?

I just wrote some 'proof of concept' code to show how much better Chrome could be with very little effort.  The attached shows two snapshots (pasted together) taken from a video of a notebook display running the test, where the left window is a native win32 app, and the right window is Chrome -- with the mouse moving at a constant rate from left to right across the screen, and each app simply immediately displaying where it thinks the mouse is as a red box.  The difference between the location of the red box and the location of the hardware mouse cursor indicates the input lag.  Windows Aero is ON (the Windows OS is compositing).

The Win32 app was designed to have around 1/2 frame of input lag (and when moving the mouse, the red box appears to hug the cursor).  We know that Chrome (from vsynctester.com test) has around 2 frames of input lag (and when moving the mouse around, there is a noticeable lag in the red box).

If simply altering the location of the swap buffers (under Windows Aero) could reduce Chrome's input lag from two frames to one frame instantly -- is this something to look into sooner rather than later?
input-lag-comparison.jpg
19.5 KB View Download
Chrome actually tries to draw a while before vsync (compositor "deadline") but sometimes we miss a vsync either because we submitted a frame too late (more common) or because GPU rendering took a long time (less common). So swap buffers (dxgi swap chain present) for the next frame has to wait until the previous frame is consumed by DWM which happens after the next vsync. We don't really try to recover from this mode because we haven't figured out a way to reliably detect that we're in this mode.

You can capture a Chrome trace with the gpu category and see that a lot of swap buffers calls block.
ANGLE is vsyncing on Present DirectX calls -- and that is the problem.  It should not when Windows is compositing (DWM).

Search Chromium source code for ANGLE_VSYNC, which defaults to ANGLE_ENABLED.  It should be set to ANGLE_DISABLED.

This is causing Chrome buffers to be presented on vsync -- but by that point it is too late as DWM has already composited, and what Chrome presented then must wait another entire frame before reaching the display.

Will someone please fix this?

After an extensive bisect, the change that apparently caused Chrome to change from one frame of input lag to two frames of input lag has been found:

  https://bugs.chromium.org/p/angleproject/issues/detail?id=814

bajones, as this was your change, can you take a look?
Cc: -tansell@chromium.org
The solution is to eliminate the TWO VSYNC's on each frame that is currently happening (Windows only), by converting ANGLE_VSYNC from a compile-time option to a run-time option, and then treating 'ANGLE_VYSNC' at run-time as 'off' (in SwapChain9.cpp and SwapChain11.cpp) when either (1) Aero Glass is ON or (2) on "--disable-gpu-vsync" or "--disable-gpu-vsync=angle" (see  issue 492732  for need for fine grain control).

ANGLE_VSYNC needs to be turned 'off' on Aero Glass ON because Aero Glass is already VSYNCing.  To perform another VSYNC only adds another frame of lag.

ANGLE_VSYNC needs a command line flag to turn it off, because that is needed for testing max frame rates when Aero Glass is OFF (and then ANGLE_VSYNC is ON, syncing to VSYNC, throttling Present()'s).

I suspect the fix described above may also fix  issue 480361  (--disable-gpu-vsync does not work on Windows) -- because ANGLE_VSYNC is now always ON, throttling things.

Comments?

Components: Internals>GPU>ANGLE

Comment 30 by kbr@chromium.org, Feb 14 2017

Cc: jbau...@chromium.org geoffl...@chromium.org jmad...@chromium.org
As a FYI, geoffl...@chromium.org in  issue 480361  states that the solution is to leave ANGLE alone, and to instead change Chrome to call ANGLE with eglSwapInterval(0), instead of with an argument >0.

We have vsync enabled on purpose, as otherwise it's more likely that windows will skip drawing frames depending on what times they arrive. You can see this as an issue with the VSYNC synchronized indicator (at http://www.vsynctester.com/ )
jbauman, there is a mess of contradictory information.  There have been so many vsync related bugs in Chrome over the years, that one fix masked other vsync bugs -- is becomes very difficult to know where to even start.  In fact, almost no one was running with the extra vsync provided by ANGLE until just a couple of months ago (unless they resized the Chrome application Window) – due to yet another BUG in Chrome that was *just* fixed in the Chrome release on 2016/10/12.

I can go back to r302284, run it with --disable-gpu-vsync, go to vsynctester.com and set up a custom Hertz that matches the Hz of my display (it uses JavaScript timers to achieve a precise interval), and get perfect VSYNC indicator at vsynctester.com and get 1 frame of input delay – over an incredibly wide range of skew values (present’s at varied locations within the frame). That is exactly what is expected.  It works great.

If I can achieve in JavaScript what you say is problematic for Chrome in native code, we need to get to the bottom of that and find out why Chrome is unable to do that.  But that can not be tested anymore...

So I need your help.  There is currently no way to turn off Chrome asking ANGLE to vsync.  It is always there and ON now.  There needs to be a flag to turn OFF only ANGLE vsync.  Under that situation, there will only be one frame of input lag, and then Chrome can be tested against vsynctester.com.  Can this be done?

Did vsynctester.com work for you before 2016/10/12 in Chrome release?  If so, then you need to re-evaluate if and extra vsync in ANGLE even matters – because the extra ANGLE VSYNC was NOT there until just a few months ago.
 

How to test Chrome against vsynctester.com with ANGLE vsync on or off...

Until there is a flag to turn ANGLE vsync off, go back to a version of Chrome where a bug kept ANGLE vsync off (until the app window was resized).  So, download Chromium  snapshot build r409090 (chrome-win32.zip from https://commondatastorage.googleapis.com/chromium-browser-snapshots/index.html?prefix=Win_x64/409090/) and extract into a temp folder.

Close all running applications/windows and then run chrome.exe from the temp folder.  As long as the Chrome application is NOT resized in any way, ANGLE vsync should be off (one frame input lag).  But immediately after the Chrome application window is resized in any way, ANGLE vsync will be ON (two frame input lag).

To confirm and verify that you can reliable run Chrome without ANGLE vsync (a one frame input lag), play around with http://www.vsynctester.com/testing/mouse.html – and confirm that the distance from the hardware cursor to the red dot is half as much under ANGLE vsync OFF.

As a final confirmation, run the chrome.exe in the temp folder with "--disable-gpu-vsync" and open vsynctester.com.  It should have a very high frame rate (ANGLE vsync off), until you resize the Chrome application window and then it switches to matching your display Hz (ANGLE vsync on).

Finally, run Chrome with ANGLE vsync off, and run vsynctester.com.  Does it work (vsync properly) for you?  For me, vsynctester shows proper vsync on all systems that I have tested on except for one system (on that one old system from 2011, every second or two there is a vsync failure – even with ANGLE vsync on – so it does not help).

Only use a single Chrome window and a single tab.
kbr, jbauman must be busy (no response).  Can we start a discussion on this issue to resolve and explain the contradictions?

Chrome 24 to Chrome 37 used a wrong timebase/interval for rAF, but used ANGLE vsync to achieve vsync.  The end result was proper vsync.  With Chrome 37, ANGLE vsync was mistakenly turned off, exposing tons of jitter and the wrong timebase/interval.  People noticed right away.  See issue 422000.  Chrome was changed to use the correct timebase/interval.  The end result: proper vsync without ANGLE vsync -- and for the first time, one frame of input lag in Chrome!

Within weeks, ANGLE vsync was turned back on -- but due to a bug, ANGLE vsync was actually OFF for all Windows users (until the Chrome application window was resized by the end user).  That ANGLE vsync off bug went unnoticed from Oct 2014 until Aug 2016.  See  issue 632785 .

The conclusion is that the internal Chrome vsync (timebase/interval plus Areo Glass) works just fine as a vsync mechanism and that ANGLE vsync is not needed for virtually all rAF animations (before 422000 yes, but not after 422000).  And if ANGLE vsync is really needed, why was there no uproar whatsoever during the nearly two years that it was turned off?

At the very least, this proves that ANGLE vsync could be off for most (all?) Chrome users.  Only turn ANGLE vsync on in the very rare situations that it is needed (ANGLE vsync on should be adaptive, and not an 'always on' setting).  Always on means an input lag of two frames.

And we need a command line flag to turn it off for testing.
The problem with --disable-gpu-vsync is being fixed in https://codereview.chromium.org/2700433005/

sunnyps, Great.  However, that still leaves open the problem that ANGLE vsync can not be turned off individually (leave Chrome vsyncing normally and producing frames normally) -- just with no ANGLE vsync.
That's --disable-gpu-vsync=gpu (though I haven't actually tested that that works).
no, --disable-gpu-vsync=gpu does not work at all (before and after CL)
jbauman/38: No, --disable-gpu-vsync=<anything> results in ForceSwapIntervalZero(true) in pass_through_image_transport_surface.cc -- AND other actions throughout Chrome.

There is a need (for testing of THIS issue) to keep everything else in Chrome running as-is and ONLY turn ANGLE vynsc off, which I believe ForceSwapIntervalZero(true) in pass_through_image_transport_surface.cc is doing?

How can I trigger ForceSwapIntervalZero(true) and the command line (and trigger nothing else)?


 Issue 658601  has been merged into this issue.
Attached is the problem, shown visually.

Brian, is this a dead issue (will not fix), or something that is still 'in the works'?
chrome-dwm-composition.jpg
52.5 KB View Download
Any update on this issue?
I did a bunch of investigation surrounding this.

First of all, with DWM you cannot get less than two *total* frames of latency. In the diagram in #44 the earliest your frame can be displayed on screen is interval 2 i.e. your frame is on the screen from the beginning to the end of that interval (the frame that says "DWM composes"). jerryj@ I think you'd call that one frame of lag unless I misunderstood your terminology. However, we're doing much worse than this - we sometimes go up to 4 frames of latency.

The root cause of this bug is NOT because we call Present with SyncInterval = 1. #42 implies that DWM does not pick up the buffer on the next vsync. That is not the case. When you call Present, the D3D runtime enqueues a "present token" after rendering commands in the device context work queue. After rendering finishes on the GPU, this present token notifies DWM to composite using the newly rendered buffer. What SyncInterval = 1 tells DWM is to present the buffer for at least 1 vsync interval. So even if you rendered a new frame, DWM may not pick it up because it has to present a previously submitted frame for at least 1 vsync interval (which it might not have done because your last frame took too long to render on the GPU or there was a DWM hiccup).

What's interesting is that D3D allows an app to queue three frames of rendering by default (IDXGI1SetMaximumFrameLatency default is 3). Documented here: https://msdn.microsoft.com/en-us/library/windows/desktop/ff471334(v=vs.85).aspx

Note that this latency is different from swap chain BufferCount which is 2. That BufferCount is denotes how many physical buffers are available for rendering. After rendering, you have to wait for DWM to pick up the buffer on the next vsync. DWM generally uses the buffer for a very short time and returns it as soon it's copied into DWM's backbuffer (the entire desktop I guess). The wait on this buffer happens in the runtime. Your app is generally not blocked because of availability of buffers.

I've verified the above behavior in Chrome using GPUView. See attached GPUView screenshot with three frames of rendering enqueued in Chrome's device context. It also shows a DWM hiccup which causes DWM itself to run in a high latency mode.

Now, setting frame latency via IDXGIDevice1::SetMaximumFrameLatency to 1 makes every Present call block instead of blocking after enqueuing three frames. However, that is not enough since Chrome will start rendering the next frame (i.e. run the next raf) even if the previous frame hasn't finished presenting. One frame hiccup is enough to increase the latency from two to three total frames. Because of SyncInterval = 1, DWM will always draw the previous frame.

The correct way to avoid this is to use the new GetFrameLatencyWaitableObject (https://msdn.microsoft.com/en-us/library/windows/desktop/dn268309(v=vs.85).aspx) and wait on that before raf. IDXGISwapChain2::SetMaximumFrameLatency allows you to bound the maximum latency on the waitable object. If set to 1, the waitable object will block after one Present until it's consumed by DWM (two total frames of latency). If set to 2, it'll block if previous frame wasn't presented (three total frames of latency).

Note: IDXGIDevice1::SetMaximumFrameLatency is different from IDXGISwapChain2::SetMaximumFrameLatency. The former controls how many frames will cause Present to block on any swap chain. The latter controls how many frames will cause the waitable object on a specific swap chain to block, it does not control what Present does.

What we should do for now is to set IDXGIDevice1::SetMaximumFrameLatency to 2, so that Present blocks if the previous frame hasn't been consumed by DWM yet. It should reduce maximum latency by 1 without significant impact on throughput. This is similar to other double buffered systems.
Three frames of latency.png
149 KB View Download
I thought I'd clarify the behavior of SyncInterval = 0 as well.

What SyncInterval = 0 does is tell DWM to forget about any previous frames and instead present the current frame on next vsync. So when the device queue fills up with Presents after a hiccup, you drain it quickly assuming there's little GPU work to do. However, you're throwing away all the work done by Chrome and the GPU rendering for those skipped frames. So of course this recovers latency. However, the reason is not what's stated in #42.
sunnyps, it is long past due for someone to re-evaluate how Chrome avoids tearing to the screen (under Windows), and the impact those decisions have on latency.

Under Windows+DWM the OS handles 'no tearing'.  But then Chrome on top of that *also* performs 'no tearing' (ANGLE present), and all of that combined adds one extra frame of latency to everything Chrome sends to the screen.

Due to all of the bugs over many years, the 'input lag' indicator on vsynctester.com (move mouse around and notice where mouse hardware cursor aligns to the circle) would either show 1 frame or 2 frames of lag.  So one frame of input lag is possible in Chrome!  It now almost always shows 2 frames of lag.  (note: lag indicator shows the lag that is under the control of the browser).

I was previously told by a Chromium engineer that Chrome *must* use ANGLE Present(1) as that is the only way to get vsync to work properly under Windows.  However, that comment was only true because (at the time) Chrome's internal 'align to vsync' aligned to the wrong Hz value (issue 422000).  Of course in that 'bug' situation, the engineer was right.  ANGLE present was the only vsync method in Chrome that was working.  But now that Chrome is properly aligning to vsync (now using the correct Hz value), can Chrome present immediately and rely upon the OS (Windows DWM) to avoid tearing?  If so, that eliminates a frame of input lag.

That needs some serious testing.  It would really help that testing if the ANGLE present characteristic (now or on vsync) could be tweaked somehow (command line, flags, or ...).

Your thoughts on this?
FYI --disable-gpu-vsync doesn't unlock frame rate any more (after tonight's Canary) and only controls the behavior of swap chain present. Use --disable-frame-rate-limit if you want to unlock frame rate and disable vsync.

Soon --disable-gpu-vsync will enable swap chain tearing which is needed for variable refresh rate displays (GSYNC, FreeSync), and also has the side-effect of further reducing latency when fullscreen (see issue 807406).

Re: #46
It's correct that we'll see no tearing with DWM in windowed mode, but we can get tearing in fullscreen with Present(0) because DWM can promote fullscreen swap chains to "direct flip" which means the display scans out from the swap chain back buffers directly. Even in windowed mode frames DWM ignores all but the last frame with Present(0) which means that if two Present(0) calls happen (and GPU rendering completes for both) in between consecutive DWM compositions, then DWM will display the second one, and the first will simply be dropped.

https://www.youtube.com/watch?v=E3wTajGZOsA is a good explanation by D3D engineers for various swap chain scenarios.

Chrome uses a windowed flip swap chain so it typically has 3- frames of latency (best case is 2-) from when Present is called and additional latency between raf and Present which can be < 1 frame best case but up to 3 frames in worst case. So best case for total latency is 3 frames, and worst case is 6 frames.


--disable-gpu-vsync means Present(0) is called instead of Present(1) and no other changes to scheduling after tonight's Canary
Cc: fs...@chromium.org
Owner: sunn...@chromium.org
Components: Internals>Compositing

Sign in to add a comment