Autoplay video playback from canvas.captureStream lags when canvas size is too big |
||||||||
Issue descriptionGoogle Chrome 67.0.3376.0 (Official Build) canary (64-bit) Platform 10506.0.0 (Official Build) canary-channel eve What steps will reproduce the problem? (1) Go to https://beaufortfrancois.github.io/sandbox/media/canvas-to-video.html?autoplay (2) Set canvas width to max What is the expected result? Video playback should still be as smooth. What happens instead of that? It is not. Video playback stutters even though it is marked as "autoplay". Background: This bug is opened after https://bugs.chromium.org/p/chromium/issues/detail?id=817346#c13 has been marked as fixed. Screen recording URL: https://photos.app.goo.gl/GIrZWev4oI1Y0Hr93
,
Mar 22 2018
Do you get the same behavior if you start chrome with --autoplay-policy=no-user-gesture-required ? What behavior do you get if you have autoplay disabled? (it shouldn't play in this case)
,
Mar 22 2018
,
Mar 22 2018
I've tried with --autoplay-policy=no-user-gesture-required and I get the same behaviour. It doesn't play with autoplay disabled indeed.
,
Mar 22 2018
Can you test playing a regular video file instead of the captured stream, both with the regular autoplay policy and with --autoplay-policy=no-user-gesture-required to see if there is any stuttering there?
,
Mar 22 2018
I would normally suggest that this might be related to just performance, as capturing and converting 1000x150 canvas -with alpha channel- might be a lot of work for an older CrOS device. However, webkitDecodedFrameCount keeps increasing with the same rate even during the stutter. This suggests that there are still ~60 fps frames being captured and pushed to <video> which aren't displayed correctly. This looks like another compositing bug exposed to me. I will try to get a hold of an older CrOs where I can repro the issue. From your earlier report, I found that you suggested this might be related to canvas positioning, https://bugs.chromium.org/p/chromium/issues/detail?id=810330#c11. I would suggest running chrome with "show-surface-damage-rects" to have a better understanding. It is kind of complicated to set the flag in CrOS, see https://www.chromium.org/developers/how-tos/run-chromium-with-flags. It is OK if you can't set that in your environment.
,
Mar 22 2018
I've tried https://beaufortfrancois.github.io/sandbox/media/canvas-to-video.html?autoplay with --show-surface-damage-rects switch and I can't reproduce. I'm not sure what is happening ;(
,
Mar 22 2018
Thanks fbeaufort@, if you can get a recording with the switch would help even more. I think this switch affects damage rects and refreshes the video element as it should. dcastagna@ can you take a look and help with triage?
,
Mar 22 2018
Here's a short recording: https://photos.app.goo.gl/Ue4OkQVRxC2rl7602 Let me know if that works for you.
,
Apr 2 2018
Any news on this?
,
Apr 2 2018
Emircan, the call VideoLayerImpl::SetNeedsRedraw is the one that grows the damaged area for the video. Do you mind checking that gets called every frame?
,
Apr 2 2018
fbeaufort: can you try changing chrome://flags/#ui-disable-partial-swap? Does that fix the issue?
,
Apr 3 2018
Woohoo! Running chrome with --ui-disable-partial-swap fixes the issue. Is that expected?
,
Apr 5 2018
dcastagna@, does that help narrow down the issue?
,
Apr 5 2018
VideoLayerImpl::SetNeedsRedraw does not seems to be called, therefore we're not always redrawing the video region. There seems to be a bug in WebMediaPlayerMSCompositor that incorrectly assumes that UpdateCurrentFrame is called before GetCurrentFrame. crrev.com/c/996855 to fix that.
,
Apr 17 2018
(gentle ping)
,
Apr 18 2018
https://chromium-review.googlesource.com/c/chromium/src/+/1000149 is under review that should address this.
,
Apr 19 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/6ffaac8d772617cc56ffdfecbba507cede2b5a5d commit 6ffaac8d772617cc56ffdfecbba507cede2b5a5d Author: Emircan Uysaler <emircan@chromium.org> Date: Thu Apr 19 16:14:58 2018 Fix UpdateCurrentFrame() edge cases in WebMediaPlayerMSCompositor This CL distinguishes two scenarios in updating the |current_frame_| in WebMediaPlayerMSCompositor. If we have VideoRendererAlgorithm running, then |current_frame_| will be updated only with UpdateCurrentFrame() calls. There, we can keep a boolean to see if |current_frame_| is displayed within two UpdateCurrentFrame() calls. If it is displayed and there is no new frame, we should return false; true otherwise. We should track PutCurrentFrame() instead of GetCurrentFrame() as the indication of |current_frame_| being displayed as it is suggested in the comments. If we don't have VideoRendererAlgorithm running, then |current_frame_| will be set as soon as possible. If we get caught in a UpdateCurrentFrame()-SetCurrentFrame()-GetCurrentFrame()-PutCurrentFrame() cycle, as it is happening in the bug, UpdateCurrentFrame() will always return false. This makes sense because by the time UpdateCurrentFrame() is called, we do not know if a new frame will be available for the next GetCurrentFrame() call. There, we should call VideoFrameProviderClientImpl::DidReceiveFrame() right after SetCurrentFrame() to indicate GetCurrentFrame() will return new data. DidReceiveFrame() ends up calling SetNeedsRedraw(), just like it would if UpdateCurrentFrame() returned true. Bug: 824752 Change-Id: I8bf98a68f788deef7aab1ff997c55372aec8874b Reviewed-on: https://chromium-review.googlesource.com/1000149 Reviewed-by: danakj <danakj@chromium.org> Reviewed-by: Daniele Castagna <dcastagna@chromium.org> Reviewed-by: Dale Curtis <dalecurtis@chromium.org> Commit-Queue: Emircan Uysaler <emircan@chromium.org> Cr-Commit-Position: refs/heads/master@{#552033} [modify] https://crrev.com/6ffaac8d772617cc56ffdfecbba507cede2b5a5d/content/renderer/media/stream/webmediaplayer_ms_compositor.cc [modify] https://crrev.com/6ffaac8d772617cc56ffdfecbba507cede2b5a5d/content/renderer/media/stream/webmediaplayer_ms_compositor.h [modify] https://crrev.com/6ffaac8d772617cc56ffdfecbba507cede2b5a5d/content/renderer/media/stream/webmediaplayer_ms_unittest.cc
,
Apr 19 2018
fbeaufort@ can you verify the fix in your environment with the canary coming out today?
,
Apr 25 2018
Yeah! It works perfectly fine in Chrome 68.0.3404.0. Thank you!
,
Apr 25 2018
,
Jun 7 2018
[bulk-edit: disregard if N/A] Can the owner please set milestone to this bug if applicable? |
||||||||
►
Sign in to add a comment |
||||||||
Comment 1 by niklase@chromium.org
, Mar 22 2018