SingleProcessMash: chrome automation API not detecting the virtual keyboard |
|||||
Issue description
Chrome Version: 11558.0.0 73.0.3667.0 dev-channel eve
OS: Chrome
What steps will reproduce the problem?
1) Build chrome-on-linux with
is_official_build = true
is_chrome_branded = true
This is so that we can get the ChromeVox extension running.
2) Run chrome with --enable-virtual-keyboard and remote debugging to make things easier
3) Enable ChromeVox
4) Open up the devtools console for ChromeVox and run
chrome.automation.getDesktop(root => console.log(root.find({ attributes: { role: 'keyboard' }})));
Expected Result:
Prints out an AutomationNode
Actual Result:
Prints null
For some reason the automation node doesn't seem to be able to find the virtual keyboard window under SingleProcessMash...The virtual keyboard seems to work okay so the problem might be on the automation API side?
,
Jan 13
,
Jan 14
P1 because it's blocking SingleProcessMash rollout.
,
Jan 14
+stevenjb. I'm still digging. I wonder if this is related to the use of views::RemoteViewProvider / ws::ServerRemoteViewHost. I see a remote window being created for the keyboard web contents, but the automation nodes don't appear in devtools. I can see the tree serializer in ax_ash_window_utils.cc walking over AshKeyboardUI and VirtualKeyboardContainer but the nodes don't get written. Perhaps some link in the tree is broken. I wonder if accessibility is working for the app list's answer card views. There's also this line in ChromeKeyboardWebContents::ChromeKeyboardWebContents() keyboard_window->SetProperty(ui::kAXRoleOverride, ax::mojom::Role::kKeyboard); I see that property showing up on the remote (browser) window under SingleProcessMash, but I wonder if the ash side needs to know about it. Window tree dumps attached.
,
Jan 15
There seems to be a lot more "button" nodes in the non-SPM version. With SPM, it looks like even the buttons in the VK are not being traversed, and not just the VK container.
,
Jan 17
(6 days ago)
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/5d72852411f01369cfa0a54066afbc319e3f9abd commit 5d72852411f01369cfa0a54066afbc319e3f9abd Author: James Cook <jamescook@chromium.org> Date: Thu Jan 17 03:16:33 2019 SingleProcessMash: Fix virtual keyboard not appearing in automation tree The virtual keyboard is an embedded remote view. The window service needs to use the embedded window tree to compute the transport ID for the VK window. This fixes the ash-to-chrome "fence hopping" code used by the accessibility chrome.automation system to serialize the desktop UI tree. Chrome OS only, see bug for more details. Bug: 921405 Change-Id: I16bd2b4a6050729529a735ad9830609076008667 Reviewed-on: https://chromium-review.googlesource.com/c/1415259 Commit-Queue: Scott Violet <sky@chromium.org> Reviewed-by: Scott Violet <sky@chromium.org> Cr-Commit-Position: refs/heads/master@{#623547} [modify] https://crrev.com/5d72852411f01369cfa0a54066afbc319e3f9abd/services/ws/window_service.cc
,
Jan 17
(5 days ago)
|
|||||
►
Sign in to add a comment |
|||||
Comment 1 by shend@chromium.org
, Jan 13