New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 697780 link

Starred by 3 users

Issue metadata

Status: Fixed
Owner:
Closed: Mar 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Mac
Pri: 3
Type: Bug



Sign in to add a comment

chrome.desktopCapture broken on Mac: does not mirror context menus and shows two cursor

Reported by thembr...@gmail.com, Mar 2 2017

Issue description

UserAgent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3027.3 Safari/537.36

Steps to reproduce the problem:
1. Open hangouts.google.com
2. Select ... -> Share Screen
3. Right/option-click anywhere to bring up a context menu

What is the expected behavior?
Mirrored screen stream should show context menus

What went wrong?

I recorded this here:
 https://www.youtube.com/watch?v=Xu107fGJtts

Context menu not shown in screen mirrored via chrome.desktopCapture
and the mouse cursor shows up twice (easy to see e.g. on 0:15)

HTML select dropdowns also don't show up in the mirrored screen (they do sometimes though, while the context menu never does here). 

Did this work before? N/A 

Does this work in other browsers? N/A

Chrome version: 58.0.3027.3  Channel: canary
OS Version: OS X 10.12.3
Flash Version: 

Also reproduces on stable 56.0.2924.87 (Official Build) (64-bit)
Does not reproduce on Linux

 
Components: -Blink>GetUserMedia Blink>GetUserMedia>Desktop
Cc: niklase@chromium.org
Owner: braveyao@chromium.org
Status: Assigned (was: Unconfirmed)
Brave, can you take a look at the cursor issue?
Cc: sergeyu@chromium.org erikc...@chromium.org
erikchen@, could you please help to cast your opinion on this issue?

This issue looks like a OS limitation or bug to me.
This only happens to screen capture on Mac. Window/Tab capture works fine.
When you right click to bring up a context menu within Chrome window (bringing up context menu outside of Chrome window or sharing app window won't cause any problem), the handler registered by CGDisplayStreamCreate(), https://cs.chromium.org/chromium/src/third_party/webrtc/modules/desktop_capture/screen_capturer_mac.mm?type=cs&q=CgBlitPostLion&sq=package:chromium&l=941, won't be called by OSX any more.
Also in CgBlitPostion(), it will return at  https://cs.chromium.org/chromium/src/third_party/webrtc/modules/desktop_capture/screen_capturer_mac.mm?type=cs&q=CgBlitPostLion&sq=package:chromium&l=723, due to the empty copy_region.
Then the result is we'll get a static frame before the moment that context menu is brought up, until the context menu is gone.

It seems that the display stream by CFRunloop in OSX is blocked, waiting for the context menu input. What do you think? Is there any way to work around it?
Although it sounds like a slightly different problem, the solution is probably similar.  We need to modify the CFRunLoop to continue to run events for display stream while a context menu is showing.
Blockedon: 651203
Labels: -Pri-2 Pri-3
Thanks so much for the comments! 
Set this as blocked on issue651203 and will revisit this one when 651203's solved.
I'm not sure I would block this on 651203. Fixing that bug will allow the Chrome runloop to run during a nested CFRunloop. This bug will likely require a nested CFRunLoop to call out to another CFRunLoopSource. It's *possible* that a really clever solution would solve both problems, but the initial proposals in 651203 would not solve this use case.
Blockedon: -651203
Cc: -erikc...@chromium.org braveyao@chromium.org
Owner: erikc...@chromium.org
OK. Then could you please help to handle this too or help to triage to a more proper owner?
Cc: erikc...@chromium.org nicho...@chromium.org
Owner: ----
Status: Untriaged (was: Assigned)
I don't have the time to work on this myself, I can provide guidance if someone from the webrtc/chromoting team wants to look at it. 

Note that this issue arose [or was it already present?] because of my refactor of screen capture logic to use a newer API [CGDisplayStream] from a deprecated API. There appear to be other issues as well: https://codereview.webrtc.org/2496413002/#msg13. Once again, I can't tell if this is a new issue or an old one that is being assigned to me because I was the last person to touch the file.

If we decide that these issues are only present after my refactor, and the webrtc team isn't particularly concerned about using deprecated APIs, we can roll back my refactor.
Cc: -braveyao@chromium.org
Owner: braveyao@chromium.org
Status: Available (was: Untriaged)
Sorry erikchen@. I don't mean it your problem. Just thought you can make a better solution more quickly than me(having little experience on OSX so far).
I can take a look, and your help is definitely needed and appreciated.
Status: Assigned (was: Available)
Looking at disassembly for __NSSLMPopUpCarbonMenu3, which gets called from +[NSMenu popUpContextMenu:...], the nested run loop uses the NSEventTrackingRunLoopMode mode.

Looking at the screen capture logic, we grab a run loop source and manually add a source. 
https://cs.chromium.org/chromium/src/third_party/webrtc/modules/desktop_capture/screen_capturer_mac.mm?type=cs&q=cgdisplaystreamcreate+package:%5Echromium$&l=975

Try adding the line:
CFRunLoopAddSource(CFRunLoopGetCurrent(), source, NSEventTrackingRunLoopMode);

and see if that fixes the problem. :)
Interesting! It does work!
Seems like
  CFRunLoopAddSource(CFRunLoopGetCurrent(), source, kCFRunLoopDefaultMode) -- doing the capture without context menu
  CFRunLoopAddSource(CFRunLoopGetCurrent(), source, NSEventTrackingRunLoopMode) -- doing the capture with context menu
When both modes are set, screen capture works entirely.

Then I tried kCFRunLoopCommonModes and this mode can work equivalently. Is it OK to use this mode in Chromium?
The Apple doc, https://developer.apple.com/library/content/documentation/Cocoa/Conceptual/Multithreading/RunLoopManagement/RunLoopManagement.html, says: 
"For Cocoa applications, this set includes the default, modal, and event tracking modes by default. Core Foundation includes just the default mode initially. You can add custom modes to the set using the CFRunLoopAddCommonMode function."
I suppose Chromium is Cocoa app and there is no need to call CFRunLoopAddCommonMode(), right?


Oh yes, good point. kCFRunLoopCommonModes also includes the modal tracking mode, which we want as well.
Project Member

Comment 14 by bugdroid1@chromium.org, Mar 10 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/external/webrtc.git/+/95b27217f204e9985e2c070abddc4aff4c8d7d35

commit 95b27217f204e9985e2c070abddc4aff4c8d7d35
Author: braveyao <braveyao@webrtc.org>
Date: Fri Mar 10 17:46:49 2017

Mac: fix screen capture freezes when context menu popup in Chrome.

Previously we grab a run loop source and add a source with mode
kCFRunLoopDefaultMode. With this mode, it won't callback when context menu popup
(which needs the NSEventTrackingRunLoopMode), then screen capture can't get
refreshed frame with context menu until the context menu is gone.
The fix is to use kCFRunLoopComonModes, which includes default,modal and event
tracking modes by default.

BUG= chromium:697780 

Review-Url: https://codereview.webrtc.org/2732393003
Cr-Commit-Position: refs/heads/master@{#17171}

[modify] https://crrev.com/95b27217f204e9985e2c070abddc4aff4c8d7d35/webrtc/modules/desktop_capture/screen_capturer_mac.mm

Any chance to get this merged back to M57?
Status: Fixed (was: Assigned)
#15, no, 57 is already on stable and this is not a major problem.
Canary 59.0.3042.0 has included the patch.
Cc: braveyao@chromium.org
 Issue 708315  has been merged into this issue.
Labels: M-59

Sign in to add a comment