New issue
Advanced search Search tips

Issue 824752 link

Starred by 1 user

Issue metadata

Status: Verified
Owner:
Closed: Apr 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug



Sign in to add a comment

Autoplay video playback from canvas.captureStream lags when canvas size is too big

Project Member Reported by fbeaufort@chromium.org, Mar 22 2018

Issue description

Google Chrome	67.0.3376.0 (Official Build) canary (64-bit)
Platform	10506.0.0 (Official Build) canary-channel eve

What steps will reproduce the problem?
(1) Go to https://beaufortfrancois.github.io/sandbox/media/canvas-to-video.html?autoplay
(2) Set canvas width to max

What is the expected result?
Video playback should still be as smooth.

What happens instead of that?
It is not. Video playback stutters even though it is marked as "autoplay".

Background: This bug is opened after https://bugs.chromium.org/p/chromium/issues/detail?id=817346#c13 has been marked as fixed.

Screen recording URL: https://photos.app.goo.gl/GIrZWev4oI1Y0Hr93
 
Cc: emir...@chromium.org

Comment 2 by guidou@chromium.org, Mar 22 2018

Do you get the same behavior if you start chrome with 
--autoplay-policy=no-user-gesture-required ?

What behavior do you get if you have autoplay disabled? (it shouldn't play in this case)

Comment 3 by guidou@chromium.org, Mar 22 2018

Labels: Needs-Feedback
I've tried with --autoplay-policy=no-user-gesture-required  and I get the same behaviour.

It doesn't play with autoplay disabled indeed.

Comment 5 by guidou@chromium.org, Mar 22 2018

Can you test playing a regular video file instead of the captured stream, both with the regular autoplay policy and with --autoplay-policy=no-user-gesture-required to see if there is any stuttering there?
Cc: -emir...@chromium.org fbeaufort@chromium.org
Owner: emir...@chromium.org
Status: Assigned (was: Untriaged)
I would normally suggest that this might be related to just performance, as capturing and converting 1000x150 canvas -with alpha channel- might be a lot of work for an older CrOS device. However, webkitDecodedFrameCount keeps increasing with the same rate even during the stutter. This suggests that there are still ~60 fps frames being captured and pushed to <video> which aren't displayed correctly. This looks like another compositing bug exposed to me.

I will try to get a hold of an older CrOs where I can repro the issue. From your earlier report, I found that you suggested this might be related to canvas positioning, https://bugs.chromium.org/p/chromium/issues/detail?id=810330#c11. I would suggest running chrome with "show-surface-damage-rects" to have a better understanding. It is kind of complicated to set the flag in CrOS, see https://www.chromium.org/developers/how-tos/run-chromium-with-flags. It is OK if you can't set that in your environment.
I've tried https://beaufortfrancois.github.io/sandbox/media/canvas-to-video.html?autoplay with --show-surface-damage-rects switch and I can't reproduce. I'm not sure what is happening ;(
Screenshot 2018-03-22 at 8.34.22 PM.png
161 KB View Download
Cc: emir...@chromium.org
Owner: dcasta...@chromium.org
Thanks fbeaufort@, if you can get a recording with the switch would help even more. I think this switch affects damage rects and refreshes the video element as it should.

dcastagna@ can you take a look and help with triage?
Here's a short recording: https://photos.app.goo.gl/Ue4OkQVRxC2rl7602
Let me know if that works for you.

Any news on this?
Owner: emir...@chromium.org
Emircan, the call VideoLayerImpl::SetNeedsRedraw is the one that grows the damaged area for the video. Do you mind checking that gets called every frame?
fbeaufort: can you try changing chrome://flags/#ui-disable-partial-swap?
Does that fix the issue?
Woohoo! Running chrome with --ui-disable-partial-swap fixes the issue. Is that expected?
Owner: dcasta...@chromium.org
dcastagna@, does that help narrow down the issue?
VideoLayerImpl::SetNeedsRedraw does not seems to be called, therefore we're not always redrawing the video region.

There seems to be a bug in WebMediaPlayerMSCompositor that incorrectly assumes that UpdateCurrentFrame is called before GetCurrentFrame.

crrev.com/c/996855 to fix that.
(gentle ping)
https://chromium-review.googlesource.com/c/chromium/src/+/1000149 is under review that should address this.
Project Member

Comment 18 by bugdroid1@chromium.org, Apr 19 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/6ffaac8d772617cc56ffdfecbba507cede2b5a5d

commit 6ffaac8d772617cc56ffdfecbba507cede2b5a5d
Author: Emircan Uysaler <emircan@chromium.org>
Date: Thu Apr 19 16:14:58 2018

Fix UpdateCurrentFrame() edge cases in WebMediaPlayerMSCompositor

This CL distinguishes two scenarios in updating the |current_frame_| in
WebMediaPlayerMSCompositor.

If we have VideoRendererAlgorithm running, then |current_frame_| will be updated
only with UpdateCurrentFrame() calls. There, we can keep a boolean to see if
|current_frame_| is displayed within two UpdateCurrentFrame() calls. If it is
displayed and there is no new frame, we should return false; true otherwise.
We should track PutCurrentFrame() instead of GetCurrentFrame() as the indication
of |current_frame_| being displayed as it is suggested in the comments.

If we don't have VideoRendererAlgorithm running, then |current_frame_| will be
set as soon as possible. If we get caught in a
UpdateCurrentFrame()-SetCurrentFrame()-GetCurrentFrame()-PutCurrentFrame()
cycle, as it is happening in the bug, UpdateCurrentFrame() will always return
false. This makes sense because by the time UpdateCurrentFrame() is called, we
do not know if a new frame will be available for the next GetCurrentFrame()
call. There, we should call VideoFrameProviderClientImpl::DidReceiveFrame()
right after SetCurrentFrame() to indicate GetCurrentFrame() will return new
data. DidReceiveFrame() ends up calling SetNeedsRedraw(), just like it would
if UpdateCurrentFrame() returned true.

Bug:  824752 
Change-Id: I8bf98a68f788deef7aab1ff997c55372aec8874b
Reviewed-on: https://chromium-review.googlesource.com/1000149
Reviewed-by: danakj <danakj@chromium.org>
Reviewed-by: Daniele Castagna <dcastagna@chromium.org>
Reviewed-by: Dale Curtis <dalecurtis@chromium.org>
Commit-Queue: Emircan Uysaler <emircan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#552033}
[modify] https://crrev.com/6ffaac8d772617cc56ffdfecbba507cede2b5a5d/content/renderer/media/stream/webmediaplayer_ms_compositor.cc
[modify] https://crrev.com/6ffaac8d772617cc56ffdfecbba507cede2b5a5d/content/renderer/media/stream/webmediaplayer_ms_compositor.h
[modify] https://crrev.com/6ffaac8d772617cc56ffdfecbba507cede2b5a5d/content/renderer/media/stream/webmediaplayer_ms_unittest.cc

Cc: -emir...@chromium.org dcasta...@chromium.org
Owner: emir...@chromium.org
Status: Fixed (was: Assigned)
fbeaufort@ can you verify the fix in your environment with the canary coming out today?
Yeah! It works perfectly fine in Chrome	68.0.3404.0. Thank you!
Status: Verified (was: Fixed)
[bulk-edit: disregard if N/A] Can the owner please set milestone to this bug if applicable?

Sign in to add a comment