Hangout screen is not working for first few attempts of screensharing on device Blaze |
||||||||||||||||||||||||||||||||||||
Issue descriptionChrome Version: 51.0.2695.1 / 8138.0.0 dev Device: Blaze What steps will reproduce the problem? 1. Join a two way successful hangout call between device Blaze and desktop(Linux) 2. From the device Blaze, select the screenshare option from the left side of the hangout window 3. A new window with the different screens shows up for the selection 4. Select any one of the options from the mini window and then click on 'Share' option at the bottom Expected result: Screen shared by device Blaze should be shown on desktop user screen Actual result: Screen shared by device Blaze is not shown on desktop user screen until few attempts are made to screenshare the same or different windows. Screensharing worked sometimes after sharing "Internal display" window. Notes: 1) This issue is reprocuded 80% of the times on device Blaze 2) This issue is not seen on M50 50.0.2661.57 / 7978.36.0 beta 3) This issue is seen on M49 stable 49.0.2623.111 / 7834.66.0 in one out of three attempts 4) Sometimes screensharing displays blank video for the first 1 or 2 screenshare windows and sometimes blank video is shown on desktop user screen as soon as hangout starts. It worked after muting and unmuting device video. 5) Please find attached webrtc-itnernals dump from desktop and device Blaze and generate_logs dump from device Blaze
,
Apr 4 2016
Can I have the WebRTC log for this? chrome://webrtc-logs/
,
Apr 7 2016
nikalse@: I did not collect chrome://webrtc-logs for hangout call mentioned in initial bug report. Retested the hangout scenario on device Blaze with M51 51.0.2695.1 / 8138.0.0 dev. Observed that screensharing worked in the second attempt. Please find attached webrtc-internal dump from desktop, device Blaze, generate_logs and chrome://webrtc-logs
,
Apr 13 2016
I tested myself and I think this can be a dupe of: https://bugs.chromium.org/p/chromium/issues/detail?id=594290#c1 I also observed very low frame on this machine. I looked at the ui log but nothing stands out.
,
Apr 13 2016
Using https://test.webrtc.org/manual/peer2peer or the extension of desktop capture example for screen capture, the capture seems fine.
,
Apr 14 2016
srcv@ Could you do a test on a couple of other chrome books. I would like to see whether this issue is with other ones or just special device blaze. Thanks,
,
Apr 14 2016
With Niklas' help, I can see the capture rate is very low when screen doesn't have much change in extension of desktop capture example.
,
Apr 14 2016
Just tried on another couple of chrome book devices. I had seen the same low capture ratio. Therefore this issue should cross the different chrome book types.
,
Apr 14 2016
Yes, my changes (https://codereview.chromium.org/1864813002 and https://codereview.chromium.org/1849003002/) would affect the desktop capture framerate on ChromeOS (and Window capture on all Aura platforms, IIRC). The behavior change is as follows: Before: Once a screen goes static, send frames at 30 FPS for approximately 25-30 seconds, then STOP sending frames until the next screen change. After (WebRTC-specific): Once a screen goes static, send copy of last frame at 1 FPS until the next screen change. These changes solve a number of problems (details in code change descriptions and code comments) and have drastic improvements on performance (particularly around CPU/GPU utilization/power). However, I don't see how my change would be causing this bug; especially since there are reports of it being observed on M49.
,
Apr 14 2016
miu, should mouse pointer movement trigger updates?
,
Apr 14 2016
I would assume that mouse pointer doesn't trigger updates because mouse is normally captured separately and then blended together, see webrtc::DesktopAndCursorComposer Also another problem with the new capturing mechanism is that webrtc may drop some frames silently, without telling anything to the capture source, and so the changes won't be encoded until the next frame. We had similar issue in remoting, worked around with https://codereview.chromium.org/1740443002
,
Apr 15 2016
OK, after some more investigations it seems like this and related bug reports boils down to 2 issues: - Youtube playout doesn't trigger capturing of new frames, not even 1 /sec from what I can see. This issue can be observed by looking at the local preview video. - On CrOS devices with HW encode there's a problem getting video streams started over Hangouts. The local preview is updated, but the other side gets no video. This happens to both screen share and camera video.
,
Apr 15 2016
From miu@ Any compositor update (i.e., if you see a change on your screen) will result in a new frame being captured. Even if the screen is static, you should always see at least one frame per second. So, if you only see one frame every 10+ seconds, that's a problem downstream.
,
Apr 19 2016
I did a screen capture When a youtube video is playing and scene is changing all the time. The log for AuraWindowCaptureMachine::OnCompositingEnded() is on the attached image. You can clear see there are several a few seconds gap from 16:35:33 to 16:35:46, from 16:35:47 to 16:36:01 and from 16:36:01 to 16:36:05
,
Apr 20 2016
sergeyu (regarding comment #11): They're talking about ChromeOS here, which uses the other desktop capturer (DesktopCaptureDeviceAura); and that doesn't interact with webrtc code in any way. For DesktopCaptureDeviceAura, mouse movement will NOT trigger updates (unless the mouse moves over something with a hover UI). gyzhou (regarding comment #13): My statement was in the context of my comment #9, and isn't relevant to this bug. To be clear, any time there is a change on-screen, capture should generate a new frame. If a Youtube video is playing and AuraWindowCaptureMachine::OnCompositingEnded() is not being called, some possibilities OTOH: 1. The compositor is not calling OnCompositingEnded() when it should. 2. There is some kind of HW-optimized overlay mode that is operating independent of the Chrome compositor. I don't know about the inner workings of video rendering on ChromeOS to provide any further guidance here. I'll e-mail the Video Stack team to see if they can provide further insights on this issue.
,
Apr 20 2016
,
Apr 20 2016
I think it's #2. +posciak for c#15.
,
Apr 20 2016
Actually, is this h264 or vp9 playout on YouTube?
,
Apr 20 2016
A little more information. Mouse clicking on the window that is capturing will lead to at least one frame of screen/window capture.
,
Apr 20 2016
dalecurtis@ How to figure out h264 or vp9?
,
Apr 20 2016
With Surfaces, the browser sets up the child surface, and the cc::Display draws everything independently of the browser compositor. So in that case the browser compositor won't do OnCompositingEnded because it didn't change anything in the current frame. To fix this, we'll need to hook up some notification that the cc::Display drew a frame back to the AuraWindowCaptureMachine. It might have to do this somehow through SurfaceFactory::WillDrawSurface.
,
Apr 20 2016
,
Apr 20 2016
Right click on the youtube video and enable stats for nerds. It will show the codec there.
,
Apr 20 2016
It is vp9 playout on YouTube.
,
Apr 20 2016
Hmm, should not be related to hardware decoding then -- this might be related to issue 603768 if you're expecting OnCompositorEnded to be called when the video is invisible/offscreen.
,
Apr 21 2016
Whether HW decoder is being used for WebRTC/Hangouts can be verified by checking Media.RTCVideoDecoderInitDecodeSuccess. Each HW-accelerated session should increment bucket 1.
,
Apr 21 2016
(It can also be disabled via disable-webrtc-hw-decoding in about:flags).
,
Apr 21 2016
After disable-webrtc-hw-decoding, the issue has no change.
,
Apr 21 2016
Issue 594290 has been merged into this issue.
,
Apr 23 2016
gyzhou pointed out some things in an e-mail that led me to look upstream of the "refresh logic" we've been talking about. It turns out the window/screen capturer may have been failing to run a callback in some transient-error cases, which caused it to think there was too much work in-flight and stop. This bug would have been around for several (or even a dozen) Chrome milestone versions, so it's strange we're just recently (i.e., as of M49) starting to see this. So, this is a highly-speculative fix: https://codereview.chromium.org/1913283004/ Once it lands, please test and let me know whether the issue is resolved. If all is well, we should merge this to M50 as well.
,
Apr 26 2016
gyzhou: This bug repro'ed on my desktop while I was testing, so I dug into this further. So, it seems the root cause of the problem is that AuraWindowCaptureMachine::OnCompositingEnded() is not being called reliably.
,
Apr 26 2016
miu@, I guess there are two bugs related to this issue. One is that AuraWindowCaptureMachine::OnCompositingEnded() is not being called reliably. The other is the one you tried to fix in comment 30.
,
Apr 26 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/47efd876604698bfebe78ef495d789148694f649 commit 47efd876604698bfebe78ef495d789148694f649 Author: miu <miu@chromium.org> Date: Tue Apr 26 20:12:47 2016 Aura Window Capture: Ensure capture callback is always run. The capture callback must always be run, or eventually the capture decision logic will think there are too many captures in-flight. This closes some "holes" in content::AuraWindowCaptureMachine where methods exit early without running the callback. This is a potential fix for crbug 600031. BUG= 600031 Review URL: https://codereview.chromium.org/1913283004 Cr-Commit-Position: refs/heads/master@{#389867} [modify] https://crrev.com/47efd876604698bfebe78ef495d789148694f649/content/browser/media/capture/aura_window_capture_machine.cc [modify] https://crrev.com/47efd876604698bfebe78ef495d789148694f649/content/browser/media/capture/aura_window_capture_machine.h [modify] https://crrev.com/47efd876604698bfebe78ef495d789148694f649/media/capture/content/thread_safe_capture_oracle.cc [modify] https://crrev.com/47efd876604698bfebe78ef495d789148694f649/media/capture/content/thread_safe_capture_oracle.h
,
Apr 27 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/1ca25389bc07e4cb43ccc46af8159b0291c7e5d5 commit 1ca25389bc07e4cb43ccc46af8159b0291c7e5d5 Author: miu <miu@chromium.org> Date: Wed Apr 27 19:57:29 2016 Short-term fix for Aura Desktop/Window capture. The compositor is not reliably calling the OnCompositingEnded() observer method, causing capture to "freeze." However, it's not clear that AuraWindowCaptureMachine should have been using it to trigger copy requests in the first place. This change switches over to using the only other reliable feedback mechanism available (as of this writing). That is, we now use the OnAnimationStep() observer method as a trigger to initiate copy requests. Long-term, we need a better observer callback interface from the compositor for this use case. The solution here will always capture frames at the maximum framerate, which means CPU/GPU is being wasted on redundant captures, and the quality/smoothness of animating content will suffer significantly. BUG= 492839 , 600031 Review-Url: https://codereview.chromium.org/1922143004 Cr-Commit-Position: refs/heads/master@{#390156} [modify] https://crrev.com/1ca25389bc07e4cb43ccc46af8159b0291c7e5d5/content/browser/media/capture/aura_window_capture_machine.cc [modify] https://crrev.com/1ca25389bc07e4cb43ccc46af8159b0291c7e5d5/content/browser/media/capture/aura_window_capture_machine.h
,
Apr 27 2016
Now that the 2nd change has landed, we believe this issue is Fixed. (George confirmed on a Blaze device.)
,
Apr 27 2016
Requesting merge for M51...
,
Apr 28 2016
Your change meets the bar and is auto-approved for M51 (branch: 2704)
,
Apr 29 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/a6a5c05b36174ab04dc9ed3ec125b799e4182653 commit a6a5c05b36174ab04dc9ed3ec125b799e4182653 Author: Yuri Wiitala <miu@chromium.org> Date: Fri Apr 29 23:28:08 2016 Aura Window Capture: Ensure capture callback is always run. The capture callback must always be run, or eventually the capture decision logic will think there are too many captures in-flight. This closes some "holes" in content::AuraWindowCaptureMachine where methods exit early without running the callback. This is a potential fix for crbug 600031. BUG= 600031 Review URL: https://codereview.chromium.org/1913283004 Cr-Commit-Position: refs/heads/master@{#389867} (cherry picked from commit 47efd876604698bfebe78ef495d789148694f649) Review URL: https://codereview.chromium.org/1941523002 . Cr-Commit-Position: refs/branch-heads/2704@{#321} Cr-Branched-From: 6e53600def8f60d8c632fadc70d7c1939ccea347-refs/heads/master@{#386251} [modify] https://crrev.com/a6a5c05b36174ab04dc9ed3ec125b799e4182653/content/browser/media/capture/aura_window_capture_machine.cc [modify] https://crrev.com/a6a5c05b36174ab04dc9ed3ec125b799e4182653/content/browser/media/capture/aura_window_capture_machine.h [modify] https://crrev.com/a6a5c05b36174ab04dc9ed3ec125b799e4182653/media/capture/content/thread_safe_capture_oracle.cc [modify] https://crrev.com/a6a5c05b36174ab04dc9ed3ec125b799e4182653/media/capture/content/thread_safe_capture_oracle.h
,
Apr 29 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/fdd26756b02ce6ae3b70ed99ce25155695f74b9e commit fdd26756b02ce6ae3b70ed99ce25155695f74b9e Author: Yuri Wiitala <miu@chromium.org> Date: Fri Apr 29 23:35:47 2016 Short-term fix for Aura Desktop/Window capture. The compositor is not reliably calling the OnCompositingEnded() observer method, causing capture to "freeze." However, it's not clear that AuraWindowCaptureMachine should have been using it to trigger copy requests in the first place. This change switches over to using the only other reliable feedback mechanism available (as of this writing). That is, we now use the OnAnimationStep() observer method as a trigger to initiate copy requests. Long-term, we need a better observer callback interface from the compositor for this use case. The solution here will always capture frames at the maximum framerate, which means CPU/GPU is being wasted on redundant captures, and the quality/smoothness of animating content will suffer significantly. BUG= 492839 , 600031 Review-Url: https://codereview.chromium.org/1922143004 Cr-Commit-Position: refs/heads/master@{#390156} (cherry picked from commit 1ca25389bc07e4cb43ccc46af8159b0291c7e5d5) Review URL: https://codereview.chromium.org/1942493002 . Cr-Commit-Position: refs/branch-heads/2704@{#322} Cr-Branched-From: 6e53600def8f60d8c632fadc70d7c1939ccea347-refs/heads/master@{#386251} [modify] https://crrev.com/fdd26756b02ce6ae3b70ed99ce25155695f74b9e/content/browser/media/capture/aura_window_capture_machine.cc [modify] https://crrev.com/fdd26756b02ce6ae3b70ed99ce25155695f74b9e/content/browser/media/capture/aura_window_capture_machine.h
,
May 13 2016
Tested hangouts on device Blaze with M51 51.0.2704.42 / 8172.28.0 beta and observed that screen sharing is not starting for first few attempts as mentioned in the initial bug report. niklase@ or gyzhou@ : Can you please confirm if this issue still needs to be resolved ? Note: 1. This issue is so far observed only on Blaze with latest M51 beta 2. Screensharing a youtube video during hangouts is working fine. No significant pauses were observed during youtube video screensharing.
,
May 16 2016
The initial description describes exactly the problem we still see on Blaze, but this bug ended up being used for a generic CrOS screen sharing problem, that now is fixed. Therefor I'm closing this one and have created https://bugs.chromium.org/p/chromium/issues/detail?id=612198 for the Blaze specific problem. |
||||||||||||||||||||||||||||||||||||
►
Sign in to add a comment |
||||||||||||||||||||||||||||||||||||
Comment 1 by tnakamura@chromium.org
, Apr 4 2016Labels: M-51
Owner: niklase@chromium.org
Status: Assigned (was: Untriaged)