[GetUserMedia API]wrong cursor position during Mouse Cursor Dragging
Reported by
k.nobuha...@gmail.com,
Apr 3 2018
|
||||||||
Issue descriptionUserAgent: Mozilla/5.0 (X11; CrOS armv7l 10452.30.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.67 Safari/537.36 Platform: 10452.30.0 (Official Build) beta-channel veyron_minnie Steps to reproduce the problem: 1. share desktop and start capture desktop 2. drag mouse cursor(move window or drag on desktop) 3. stop share and stop capture What is the expected behavior? the mouse cursor moves according to dragged tracks What went wrong? the mouse cursor potion wrong(not move while it is dragging) Refer to he attached video file 1.CursorCaptureIssue_UseDesktopCapture.webm used app: Desktop Capture Sample 2.CursorCaptureIssue_UseScreencastify.webm use app: Screencastify-Screen Video Recoder Did this work before? N/A Chrome version: 66.0.3359.67 Channel: beta OS Version: 10452.30.0 Flash Version: 29.0.0.134 /opt/google/chrome/pepper/libpepflashplayer.so Use navigator.webkitGetUserMedia API happened only at the mouse dragging not happened while mouse cursor move not happened on MacOS not happened on WindowsOS
,
Apr 23 2018
,
Apr 26 2018
Yes, I can reproduce this problem locally. Will take a look soon.
,
May 18 2018
The problem is during dragging the window around, there is no MouseEvent [1] received. [1]: https://cs.chromium.org/chromium/src/content/browser/media/capture/cursor_renderer_aura.cc?dr&g=0&l=119 Some logs as enclosed, quoted some here: [30723:30723:0518/101805.566807:ERROR:cursor_renderer_aura.cc(120)] OnMouseEvent: event type: 4 [30723:30723:0518/101805.726686:ERROR:cursor_renderer_aura.cc(120)] OnMouseEvent: event type: 1 [30723:30723:0518/101816.146355:ERROR:cursor_renderer_aura.cc(120)] OnMouseEvent: event type: 10 [30723:30723:0518/101816.146536:ERROR:cursor_renderer_aura.cc(120)] OnMouseEvent: event type: 3 [30723:30723:0518/101817.528490:ERROR:cursor_renderer_aura.cc(120)] OnMouseEvent: event type: 4 In my test, I dragged the window around for 10s. As the logs above shows that within the 10s between event type 1(ET_MOUSE_PRESSED) and event type 3(ET_MOUSE_RELEASED), there is no other mouse event received(only a event type 10, "ET_MOUSE_CAPTURE_CHANGED", just before mouse release. What does this mean?). miu@, is this WAI or a known issue? Anything we can do?
,
May 19 2018
Sounds like this is something that none of the "cursor renderer" authors (myself included) have ever tested. :) From the description and #c4 that on ChromeOS Ash desktop, it seems that mouse events are not being received by CursorRendererAura while the window is being dragged around. To solve this problem, we'd probably want to add a listener for window position changes (or maybe making the mouse event listener a "pre-listener" would be simpler?).
,
May 21 2018
Is the event ET_MOUSE_DRAGGED (https://cs.chromium.org/chromium/src/ui/events/event_constants.h?dr&g=0&l=14) for such a scenario? Who may know more about this? Ash team?
,
May 30 2018
+owners from ui/events and ash/shelf. xiyuan@ and sadrul@, any insight?
,
May 30 2018
Sadrul is the expert here. The suspicion in #6 is probably correct. From code in [1], ET_MOUSE_DRAGGED is used instead of ET_MOUSE_MOVED when there is a mouse button down. [1]: https://cs.chromium.org/chromium/src/ui/events/event.cc?rcl=cad86e84f28c289d62399cf7588d7fd46ad98343&l=575
,
May 30 2018
It should be DRAGGED, yes. Although, when you are dragging a window, the CursorRendererAura (as a pre-target handler for the window) may still not see the DRAGGED events.
,
May 30 2018
What does the pre-target handler (and pre-listener in #5) mean here? Why does it suppose not being able to get the DRAGGED events?
,
Jun 18 2018
I'm looking into this, as I'm currently working on moving the mouse cursor rendering out of the browser process (into VIZ).
,
Jul 24
,
Jul 25
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/03ff689101b3bcca2b7c2dd2fccf6c6ee8b8bc77 commit 03ff689101b3bcca2b7c2dd2fccf6c6ee8b8bc77 Author: Yuri Wiitala <miu@chromium.org> Date: Wed Jul 25 21:45:44 2018 Add MouseCursorOverlayController. This controller is heavily based on content::CursorRendererXYZ, providing the logic for processing platform mouse events and using them to control the new viz::VideoCaptureOverlay. In a later CL, this will be integrated with the tab/desktop capture stack, to complete the migration discussed in crbug 810133. Bug: 810133 ,828274 Change-Id: I6502e1ef3cbc878e1c9b062b0abf6a0e6229f860 Reviewed-on: https://chromium-review.googlesource.com/1149264 Reviewed-by: Xiangjun Zhang <xjz@chromium.org> Reviewed-by: Antoine Labour <piman@chromium.org> Commit-Queue: Yuri Wiitala <miu@chromium.org> Cr-Commit-Position: refs/heads/master@{#578079} [modify] https://crrev.com/03ff689101b3bcca2b7c2dd2fccf6c6ee8b8bc77/content/browser/BUILD.gn [add] https://crrev.com/03ff689101b3bcca2b7c2dd2fccf6c6ee8b8bc77/content/browser/media/capture/mouse_cursor_overlay_controller.cc [add] https://crrev.com/03ff689101b3bcca2b7c2dd2fccf6c6ee8b8bc77/content/browser/media/capture/mouse_cursor_overlay_controller.h [add] https://crrev.com/03ff689101b3bcca2b7c2dd2fccf6c6ee8b8bc77/content/browser/media/capture/mouse_cursor_overlay_controller_aura.cc [add] https://crrev.com/03ff689101b3bcca2b7c2dd2fccf6c6ee8b8bc77/content/browser/media/capture/mouse_cursor_overlay_controller_browsertest.cc [add] https://crrev.com/03ff689101b3bcca2b7c2dd2fccf6c6ee8b8bc77/content/browser/media/capture/mouse_cursor_overlay_controller_mac.mm [modify] https://crrev.com/03ff689101b3bcca2b7c2dd2fccf6c6ee8b8bc77/content/test/BUILD.gn
,
Dec 18
Just revisited using current CrOS stable: The new cursor rendering code will: 1) Hide the cursor (after dragging for a few seconds); and 2) When the window dragging stops, the cursor re-appears at the correct location. Internally, the UI toolkit doesn't provide a convenient way to receive mouse events while dragging is taking place. There might be a way to install some kind of pre-target handler to get mouse events while dragging, but that would require some research. Setting status to P3 and Available so someone else could pick this up. Unfortunately, I won't be able to continue working on it. |
||||||||
►
Sign in to add a comment |
||||||||
Comment 1 by junov@chromium.org
, Apr 3 2018