Note that the video attached makes it look like the canvas stutters as well which is not the case in "real" life.

Emircan, can you take a look? This seems way worse than expected.  

I got some output on Linux. It looks like 300x220 is the threshold where your canvas output becomes texture backed and we switch the capture strategy. On Linux it happens smoothly though. I will try to find a CrOS where I can reproduce this.

[1:1:0208/111344.067097:ERROR:canvas_capture_handler.cc(241)] ReadARGBPixelsSync300x212
[1:1:0208/111344.086869:ERROR:canvas_capture_handler.cc(275)] ReadARGBPixelsAsync300x220

It also repros on OSX.

What I have found from my studies on Mac:
- I tried bisect tool going back to 49 but it isn't a regression. This issue has been present, but not reproducible on Linux.
- There are 60 fps frames coming into canvas texture and passed to WebMediaPlayerMS for display. We aren't dropping frames in capture or display path.
- Reverting the readback strategy doesn't help. I forced it to use SkImage::readPixels instead of async calls but we observe the same stutter.
- Setting disable-accelerated-2d-canvas from chrome://flags fixes the problem though.
- I wrote some code checking the ball position in captured frames. It moves 2x2 as expected from JS code. 
- I dived into render pipeline to see if timestamp affects it or not. There I found some unexpected behavior, because <video> wasn't technically playing.
- I added autoplay on video: "<video id="video" autoplay="true"></video>" And it worked.
- It looks like an issue from damage rect. Running chrome with "--show-surface-damage-rects" also fixes the issue, and shows the flicker on <video>.

My understanding is that non-playing <video> will still be updated with a low frame rate. I am not sure what triggers this. 2d acceleration on canvas might be affecting the neighbor based on the effect of "disable-accelerated-2d-canvas". However, this might be confusing as it looks like <video> is playing. I am not sure if this is a bug after all. cc'ing some folks for their input.

On Chrome OS, adding "autoplay" to video doesn't seem to address the issue: Canvas still plays smoothly and video stutters.
You can try it at this custom URL: https://beaufortfrancois.github.io/sandbox/media/canvas-to-video.html?autoplay

On Mac though, adding "autoplay" to video fixes the issue.

Setting the disable-accelerated-2d-canvas switch though fixes the issue in Chrome for Mac and Chrome OS.

Thanks for the easy tests Francois. Can you try moving <video> around the page as well to see how that affects? Another way to check if it is render vs capture is video.webkitDecodedFrameCount. If that is increasing with a high rate although video is stuttering, it shows a problem in rendering.

It looks like a possible issue in play() or problem in setting up correct damage rects in the page. I dont have a proper CrOS setup to test now. cc'ing more folks who might know what CrOS specific issues there are.

I've just added video.webkitDecodedFrameCount to the demo at https://beaufortfrancois.github.io/sandbox/media/canvas-to-video.html. Hopefully this will help as well to debug.

As an aside, I noticed that on Linux with Version 65.0.3325.31 (Official Build) dev (64-bit), if I increase the canvas dimensions and scroll down so that none of the canvas is visible (and thus only the video element is displayed), the FPS of the video element is much lower than if the canvas is at least partially visible.

I had some more time to debug this today. I cannot reproduce the issue using link on #6 on Chromebook Pixel 2 using latest canary. fbeaufort@ can you double check on CrOS? What device are you using?

What I found is that, this page doesn't trigger play() automatically on the canvas capture content. Without autoplay[0], WebMediaPlayerMS triggers Load() but not Play(). Load() triggers MediaStreamVideoRendererSink to start sending frames and current frame is getting updated, but StartRendering() never runs. Therefore, the video updates in random intervals and seems like low fps capture. 
guidou@ can you take a look? Should we change logic in WebMediaPlayerMS to not update till play()? 

[0] https://beaufortfrancois.github.io/sandbox/media/canvas-to-video.html

Using latest canary (66.0.3353.0) with Chromebook Pixelbook, canvas on https://beaufortfrancois.github.io/sandbox/media/canvas-to-video.html?autoplay still stutters. However the threshold is much higher this time (eg. Canvas size: 738x150 (110700))

I believe this is not due to the canvas size this time as I can make it stutter with a lower threshold when window width is smaller. I think* it happens because the canvas element is moved below the video. Can you confirm my assumptions?

Re #11, your assumptions are correct. Canvas capture works as expected in all cases. It is just <video> element where capture is being played is not updated properly based on where it is on the page. The problem that I see in Mac and Linux is that the play sequence is not completed. load() is called but play() isn't. I am suspecting autoplay doesn't run based on page interactions or smt in your case. Which Chromebook version is it? 

Can you add a button that explicitly calls play() on produced <video> rather than autoplay? 

Test page has been updated to add media controls to video: https://beaufortfrancois.github.io/sandbox/media/canvas-to-video.html

Google Chrome	66.0.3355.0 (Official Build) canary (64-bit)
Revision	0
Platform	10437.0.0 (Official Build) canary-channel eve
Firmware Version	Google_Eve.9584.107.0

(gentle ping)

I suspect this is a duplicate of  bug 817346 .
I think WebMediaPlayerMS plays some frames upon Load() for some reason when it shouldn't and that playback often stutters. I'll continue digging into this to see if I can find the root cause.

Marking as duplicate of  bug 817346 .
Working on a fix.