New issue
Advanced search Search tips

Issue 699617 link

Starred by 2 users

Issue metadata

Status: Archived
Owner:
Closed: May 2017
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 3
Type: Feature
Team-Accessibility

Blocking:
issue 593887
issue 709860



Sign in to add a comment

[Select-to-Speak] Should work with virtual keyboard

Project Member Reported by dmazz...@chromium.org, Mar 8 2017

Issue description

There are some technical issues preventing select-to-speak from working with the virtual keyboard.

 
Project Member

Comment 1 by bugdroid1@chromium.org, Mar 13 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/2404b8fb3068d18de9068c4e446344cc5aa3a2cb

commit 2404b8fb3068d18de9068c4e446344cc5aa3a2cb
Author: dmazzoni <dmazzoni@chromium.org>
Date: Mon Mar 13 17:10:12 2017

Select-to-speak event handler should be on root window, not ash Shell.

Otherwise we miss out on the virtual keyboard. This change alone won't fix
the virtual keyboard but it's a necessary piece of the fix. Tested manually
by debugging and seeing that we're getting mouse events over the virtual
keyboard now.

BUG= 699617 

Review-Url: https://codereview.chromium.org/2735413002
Cr-Commit-Position: refs/heads/master@{#456408}

[modify] https://crrev.com/2404b8fb3068d18de9068c4e446344cc5aa3a2cb/chrome/browser/chromeos/accessibility/select_to_speak_event_handler.cc

Project Member

Comment 2 by bugdroid1@chromium.org, Mar 13 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/05107990b575defc12ba29da8ae17a8660a29b67

commit 05107990b575defc12ba29da8ae17a8660a29b67
Author: dmazzoni <dmazzoni@chromium.org>
Date: Mon Mar 13 18:44:16 2017

Add a window property to associate RWHVA with its child AX tree ID

Working on making the Chrome OS virtual keyboard accessible to select-to-speak
and other accessibility services. Currently we link a views::WebView to its
inner web accessibility tree, but the VK just puts a WebContents directly in
an aura Window.

This change hooks them up by adding the child AX tree ID as a window property.

BUG= 699617 
CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.linux:linux_site_isolation

Review-Url: https://codereview.chromium.org/2742533003
Cr-Commit-Position: refs/heads/master@{#456445}

[modify] https://crrev.com/05107990b575defc12ba29da8ae17a8660a29b67/content/browser/frame_host/render_frame_host_impl.cc
[modify] https://crrev.com/05107990b575defc12ba29da8ae17a8660a29b67/content/browser/renderer_host/render_widget_host_view_aura.cc
[modify] https://crrev.com/05107990b575defc12ba29da8ae17a8660a29b67/content/browser/renderer_host/render_widget_host_view_aura.h
[modify] https://crrev.com/05107990b575defc12ba29da8ae17a8660a29b67/content/browser/renderer_host/render_widget_host_view_base.h
[modify] https://crrev.com/05107990b575defc12ba29da8ae17a8660a29b67/ui/accessibility/BUILD.gn
[modify] https://crrev.com/05107990b575defc12ba29da8ae17a8660a29b67/ui/accessibility/DEPS
[add] https://crrev.com/05107990b575defc12ba29da8ae17a8660a29b67/ui/accessibility/platform/aura_window_properties.cc
[add] https://crrev.com/05107990b575defc12ba29da8ae17a8660a29b67/ui/accessibility/platform/aura_window_properties.h
[modify] https://crrev.com/05107990b575defc12ba29da8ae17a8660a29b67/ui/views/accessibility/ax_window_obj_wrapper.cc

Project Member

Comment 3 by bugdroid1@chromium.org, Mar 13 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/4a204455d3793edd5b016c99225df4241c928237

commit 4a204455d3793edd5b016c99225df4241c928237
Author: qyearsley <qyearsley@chromium.org>
Date: Mon Mar 13 21:05:31 2017

Revert of Add a window property to associate RWHVA with its child AX tree ID (patchset #4 id:60001 of https://codereview.chromium.org/2742533003/ )

Reason for revert:
It looks like this causes the (static initializers) sizes step to fail:

https://luci-milo.appspot.com/buildbot/chromium/Linux%20x64/35021

Original issue's description:
> Add a window property to associate RWHVA with its child AX tree ID
>
> Working on making the Chrome OS virtual keyboard accessible to select-to-speak
> and other accessibility services. Currently we link a views::WebView to its
> inner web accessibility tree, but the VK just puts a WebContents directly in
> an aura Window.
>
> This change hooks them up by adding the child AX tree ID as a window property.
>
> BUG= 699617 
> CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.linux:linux_site_isolation
>
> Review-Url: https://codereview.chromium.org/2742533003
> Cr-Commit-Position: refs/heads/master@{#456445}
> Committed: https://chromium.googlesource.com/chromium/src/+/05107990b575defc12ba29da8ae17a8660a29b67

TBR=sky@chromium.org,dtseng@chromium.org,jam@chromium.org,dmazzoni@chromium.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG= 699617 

Review-Url: https://codereview.chromium.org/2751503002
Cr-Commit-Position: refs/heads/master@{#456479}

[modify] https://crrev.com/4a204455d3793edd5b016c99225df4241c928237/content/browser/frame_host/render_frame_host_impl.cc
[modify] https://crrev.com/4a204455d3793edd5b016c99225df4241c928237/content/browser/renderer_host/render_widget_host_view_aura.cc
[modify] https://crrev.com/4a204455d3793edd5b016c99225df4241c928237/content/browser/renderer_host/render_widget_host_view_aura.h
[modify] https://crrev.com/4a204455d3793edd5b016c99225df4241c928237/content/browser/renderer_host/render_widget_host_view_base.h
[modify] https://crrev.com/4a204455d3793edd5b016c99225df4241c928237/ui/accessibility/BUILD.gn
[modify] https://crrev.com/4a204455d3793edd5b016c99225df4241c928237/ui/accessibility/DEPS
[delete] https://crrev.com/1ba1ed8d70cc0f0db142b64bbefb997ef8efbe3c/ui/accessibility/platform/aura_window_properties.cc
[delete] https://crrev.com/1ba1ed8d70cc0f0db142b64bbefb997ef8efbe3c/ui/accessibility/platform/aura_window_properties.h
[modify] https://crrev.com/4a204455d3793edd5b016c99225df4241c928237/ui/views/accessibility/ax_window_obj_wrapper.cc

Project Member

Comment 4 by bugdroid1@chromium.org, Mar 23 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/4768bdb54f0b4c308d325c4ff0563a40744fc156

commit 4768bdb54f0b4c308d325c4ff0563a40744fc156
Author: dmazzoni <dmazzoni@chromium.org>
Date: Thu Mar 23 18:48:13 2017

Re-land: Add a window property to associate RWHVA with its child AX tree ID

Originally landed: https://codereview.chromium.org/2742533003/
Reverted: https://codereview.chromium.org/2751503002/

Only change: makes kNoAXTreeID into a constexpr to avoid a static initializer

BUG= 699617 
TBR=sky@chromium.org,jam@chromium.org
CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.linux:linux_site_isolation

Review-Url: https://codereview.chromium.org/2774683002
Cr-Commit-Position: refs/heads/master@{#459161}

[modify] https://crrev.com/4768bdb54f0b4c308d325c4ff0563a40744fc156/content/browser/frame_host/render_frame_host_impl.cc
[modify] https://crrev.com/4768bdb54f0b4c308d325c4ff0563a40744fc156/content/browser/renderer_host/render_widget_host_view_aura.cc
[modify] https://crrev.com/4768bdb54f0b4c308d325c4ff0563a40744fc156/content/browser/renderer_host/render_widget_host_view_aura.h
[modify] https://crrev.com/4768bdb54f0b4c308d325c4ff0563a40744fc156/content/browser/renderer_host/render_widget_host_view_base.h
[modify] https://crrev.com/4768bdb54f0b4c308d325c4ff0563a40744fc156/ui/accessibility/BUILD.gn
[modify] https://crrev.com/4768bdb54f0b4c308d325c4ff0563a40744fc156/ui/accessibility/DEPS
[modify] https://crrev.com/4768bdb54f0b4c308d325c4ff0563a40744fc156/ui/accessibility/ax_tree_id_registry.cc
[modify] https://crrev.com/4768bdb54f0b4c308d325c4ff0563a40744fc156/ui/accessibility/ax_tree_id_registry.h
[add] https://crrev.com/4768bdb54f0b4c308d325c4ff0563a40744fc156/ui/accessibility/platform/aura_window_properties.cc
[add] https://crrev.com/4768bdb54f0b4c308d325c4ff0563a40744fc156/ui/accessibility/platform/aura_window_properties.h
[modify] https://crrev.com/4768bdb54f0b4c308d325c4ff0563a40744fc156/ui/views/accessibility/ax_window_obj_wrapper.cc

Labels: NewComponent-Accessibility NewComponent-Accessibility-Features
Project Member

Comment 6 by bugdroid1@chromium.org, Mar 28 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/7a44adcc7ca0d47f73d8cbc286f540cbc65e5859

commit 7a44adcc7ca0d47f73d8cbc286f540cbc65e5859
Author: dmazzoni <dmazzoni@chromium.org>
Date: Tue Mar 28 19:55:00 2017

Generalize the HIT_TEST accessibility action so that it can send any event.

To determine what object is under a certain location on the screen we
have an accessibility action from the browser to renderer and then a
response event fired on the correct accessibility object. It gets more
complicated and bounces around a few times if there are out-of-process
iframes.

For historical reasons we fire a HOVER event on the hit object, but
this is somewhat arbitrary. Generalize this by letting the caller
specify what event to fire. That will be useful in improving the
select-to-speak feature on Chrome OS.

In this change, just generalize the HIT_TEST to fire any specified event
and add test coverage.

BUG= 699617 
CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.linux:closure_compilation;master.tryserver.chromium.linux:linux_site_isolation

Review-Url: https://codereview.chromium.org/2748463003
Cr-Commit-Position: refs/heads/master@{#460192}

[modify] https://crrev.com/7a44adcc7ca0d47f73d8cbc286f540cbc65e5859/chrome/browser/extensions/api/automation/automation_apitest.cc
[modify] https://crrev.com/7a44adcc7ca0d47f73d8cbc286f540cbc65e5859/chrome/browser/extensions/api/automation_internal/automation_internal_api.cc
[modify] https://crrev.com/7a44adcc7ca0d47f73d8cbc286f540cbc65e5859/chrome/common/extensions/api/automation.idl
[modify] https://crrev.com/7a44adcc7ca0d47f73d8cbc286f540cbc65e5859/chrome/common/extensions/api/automation_internal.idl
[modify] https://crrev.com/7a44adcc7ca0d47f73d8cbc286f540cbc65e5859/chrome/renderer/resources/extensions/automation/automation_node.js
[add] https://crrev.com/7a44adcc7ca0d47f73d8cbc286f540cbc65e5859/chrome/test/data/extensions/api_test/automation/sites/hit_test.html
[add] https://crrev.com/7a44adcc7ca0d47f73d8cbc286f540cbc65e5859/chrome/test/data/extensions/api_test/automation/tests/tabs/hit_test.html
[add] https://crrev.com/7a44adcc7ca0d47f73d8cbc286f540cbc65e5859/chrome/test/data/extensions/api_test/automation/tests/tabs/hit_test.js
[modify] https://crrev.com/7a44adcc7ca0d47f73d8cbc286f540cbc65e5859/content/browser/accessibility/browser_accessibility_manager.cc
[modify] https://crrev.com/7a44adcc7ca0d47f73d8cbc286f540cbc65e5859/content/browser/accessibility/browser_accessibility_manager.h
[modify] https://crrev.com/7a44adcc7ca0d47f73d8cbc286f540cbc65e5859/content/browser/accessibility/hit_testing_browsertest.cc
[modify] https://crrev.com/7a44adcc7ca0d47f73d8cbc286f540cbc65e5859/content/browser/frame_host/render_frame_host_impl.cc
[modify] https://crrev.com/7a44adcc7ca0d47f73d8cbc286f540cbc65e5859/content/browser/frame_host/render_frame_host_impl.h
[modify] https://crrev.com/7a44adcc7ca0d47f73d8cbc286f540cbc65e5859/content/common/accessibility_messages.h
[modify] https://crrev.com/7a44adcc7ca0d47f73d8cbc286f540cbc65e5859/content/renderer/accessibility/render_accessibility_impl.cc
[modify] https://crrev.com/7a44adcc7ca0d47f73d8cbc286f540cbc65e5859/content/renderer/accessibility/render_accessibility_impl.h
[modify] https://crrev.com/7a44adcc7ca0d47f73d8cbc286f540cbc65e5859/third_party/closure_compiler/externs/automation.js
[modify] https://crrev.com/7a44adcc7ca0d47f73d8cbc286f540cbc65e5859/ui/accessibility/ax_action_data.cc
[modify] https://crrev.com/7a44adcc7ca0d47f73d8cbc286f540cbc65e5859/ui/accessibility/ax_action_data.h
[modify] https://crrev.com/7a44adcc7ca0d47f73d8cbc286f540cbc65e5859/ui/accessibility/ax_enums.idl

Project Member

Comment 7 by bugdroid1@chromium.org, Apr 4 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/59016e60a48d96c7b7d9bf71cf61307ea1df0903

commit 59016e60a48d96c7b7d9bf71cf61307ea1df0903
Author: dmazzoni <dmazzoni@chromium.org>
Date: Tue Apr 04 04:00:30 2017

Move implementation of accessibility actions to views::View

We had two parallel, nearly identical implementations of accessible
actions for Views, one in NativeViewAccessibilityBase (used on Windows
and macOS), and one in AXViewObjWrapper (used on Chrome OS). Remove
the duplication and push all of those handlers to View.

While there were some tiny logic differences between the two that are
now merged, in effect this should be a pure refactoring change with no
user-visible impact.

BUG= 699617 

Review-Url: https://codereview.chromium.org/2795843002
Cr-Commit-Position: refs/heads/master@{#461624}

[modify] https://crrev.com/59016e60a48d96c7b7d9bf71cf61307ea1df0903/chrome/browser/ui/ash/accessibility/ax_tree_source_aura_unittest.cc
[modify] https://crrev.com/59016e60a48d96c7b7d9bf71cf61307ea1df0903/chrome/browser/ui/aura/accessibility/automation_manager_aura.cc
[modify] https://crrev.com/59016e60a48d96c7b7d9bf71cf61307ea1df0903/chrome/browser/ui/aura/accessibility/ax_tree_source_aura.cc
[modify] https://crrev.com/59016e60a48d96c7b7d9bf71cf61307ea1df0903/chrome/browser/ui/aura/accessibility/ax_tree_source_aura.h
[modify] https://crrev.com/59016e60a48d96c7b7d9bf71cf61307ea1df0903/content/browser/accessibility/browser_accessibility.cc
[modify] https://crrev.com/59016e60a48d96c7b7d9bf71cf61307ea1df0903/content/browser/accessibility/browser_accessibility.h
[modify] https://crrev.com/59016e60a48d96c7b7d9bf71cf61307ea1df0903/ui/accessibility/platform/ax_platform_node_delegate.h
[modify] https://crrev.com/59016e60a48d96c7b7d9bf71cf61307ea1df0903/ui/accessibility/platform/ax_platform_node_win.cc
[modify] https://crrev.com/59016e60a48d96c7b7d9bf71cf61307ea1df0903/ui/accessibility/platform/test_ax_node_wrapper.cc
[modify] https://crrev.com/59016e60a48d96c7b7d9bf71cf61307ea1df0903/ui/accessibility/platform/test_ax_node_wrapper.h
[modify] https://crrev.com/59016e60a48d96c7b7d9bf71cf61307ea1df0903/ui/views/accessibility/ax_aura_obj_wrapper.h
[modify] https://crrev.com/59016e60a48d96c7b7d9bf71cf61307ea1df0903/ui/views/accessibility/ax_view_obj_wrapper.cc
[modify] https://crrev.com/59016e60a48d96c7b7d9bf71cf61307ea1df0903/ui/views/accessibility/ax_view_obj_wrapper.h
[modify] https://crrev.com/59016e60a48d96c7b7d9bf71cf61307ea1df0903/ui/views/accessibility/native_view_accessibility_auralinux.cc
[modify] https://crrev.com/59016e60a48d96c7b7d9bf71cf61307ea1df0903/ui/views/accessibility/native_view_accessibility_base.cc
[modify] https://crrev.com/59016e60a48d96c7b7d9bf71cf61307ea1df0903/ui/views/accessibility/native_view_accessibility_base.h
[modify] https://crrev.com/59016e60a48d96c7b7d9bf71cf61307ea1df0903/ui/views/accessibility/native_view_accessibility_unittest.cc
[modify] https://crrev.com/59016e60a48d96c7b7d9bf71cf61307ea1df0903/ui/views/view.cc

Blocking: 709860
Project Member

Comment 9 by bugdroid1@chromium.org, Apr 20 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/b255728272ac622014b02d02f22d16d3f493f4ae

commit b255728272ac622014b02d02f22d16d3f493f4ae
Author: aboxhall <aboxhall@chromium.org>
Date: Thu Apr 20 01:03:35 2017

Revert of Finish implementation of automation API hit testing for the desktop. (patchset #2 id:20001 of https://codereview.chromium.org/2813083003/ )

Reason for revert:
DesktopHitTest is failing on MSan builder:
https://test-results.appspot.com/dashboards/flakiness_dashboard.html#tests=AutomationApiTest.DesktopHitTest

Original issue's description:
> Finish implementation of automation API hit testing for the desktop.
>
> This change finishes support for AX_ACTION_HIT_TEST in AXHostDelegate
> so that you can send a hit test request on the root of the whole automation
> tree and get a hit test on either views or web content.
>
> Switches Select-to-speak to use this new API.  A follow-up change will
> change the way select-to-speak works so that it sends the mouse event
> to the extension and the hit test comes from there, but this is a
> self-contained change that can land first.
>
> BUG= 699617 
>
> Review-Url: https://codereview.chromium.org/2813083003
> Cr-Commit-Position: refs/heads/master@{#465716}
> Committed: https://chromium.googlesource.com/chromium/src/+/28ac2d11e80db21b5b7a37968d558c7e95897ebf

TBR=dtseng@chromium.org,dmazzoni@chromium.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG= 699617 

Review-Url: https://codereview.chromium.org/2827103003
Cr-Commit-Position: refs/heads/master@{#465839}

[modify] https://crrev.com/b255728272ac622014b02d02f22d16d3f493f4ae/chrome/browser/chromeos/accessibility/select_to_speak_event_handler.cc
[modify] https://crrev.com/b255728272ac622014b02d02f22d16d3f493f4ae/chrome/browser/chromeos/accessibility/select_to_speak_event_handler.h
[modify] https://crrev.com/b255728272ac622014b02d02f22d16d3f493f4ae/chrome/browser/chromeos/accessibility/select_to_speak_event_handler_unittest.cc
[modify] https://crrev.com/b255728272ac622014b02d02f22d16d3f493f4ae/chrome/browser/extensions/api/automation/automation_apitest.cc
[modify] https://crrev.com/b255728272ac622014b02d02f22d16d3f493f4ae/chrome/browser/extensions/api/automation_internal/automation_internal_api.cc
[modify] https://crrev.com/b255728272ac622014b02d02f22d16d3f493f4ae/chrome/browser/ui/aura/accessibility/automation_manager_aura.cc
[modify] https://crrev.com/b255728272ac622014b02d02f22d16d3f493f4ae/chrome/browser/ui/aura/accessibility/automation_manager_aura.h
[delete] https://crrev.com/54d7e3050af7ebb57aa7cd99cef14037db03f3bf/chrome/test/data/extensions/api_test/automation/tests/desktop/hit_test.html
[delete] https://crrev.com/54d7e3050af7ebb57aa7cd99cef14037db03f3bf/chrome/test/data/extensions/api_test/automation/tests/desktop/hit_test.js

Project Member

Comment 10 by bugdroid1@chromium.org, Apr 20 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/6c4193276b6a8a8ccbd75c1e514317a815f90e85

commit 6c4193276b6a8a8ccbd75c1e514317a815f90e85
Author: dmazzoni <dmazzoni@chromium.org>
Date: Thu Apr 20 07:02:32 2017

Re-land: Finish implementation of automation API hit testing for the desktop

Original change: r465716 https://codereview.chromium.org/2813083003/
Reverted: r465839 https://codereview.chromium.org/2827103003/
The original change failed due to "test produced excessive output".
Fixed by running final test callback only once.

This change finishes support for AX_ACTION_HIT_TEST in AXHostDelegate
so that you can send a hit test request on the root of the whole automation
tree and get a hit test on either views or web content.

Switches Select-to-speak to use this new API.  A follow-up change will
change the way select-to-speak works so that it sends the mouse event
to the extension and the hit test comes from there, but this is a
self-contained change that can land first.

BUG= 699617 
TBR=dtseng@chromium.org

Review-Url: https://codereview.chromium.org/2831853002
Cr-Commit-Position: refs/heads/master@{#465930}

[modify] https://crrev.com/6c4193276b6a8a8ccbd75c1e514317a815f90e85/chrome/browser/chromeos/accessibility/select_to_speak_event_handler.cc
[modify] https://crrev.com/6c4193276b6a8a8ccbd75c1e514317a815f90e85/chrome/browser/chromeos/accessibility/select_to_speak_event_handler.h
[modify] https://crrev.com/6c4193276b6a8a8ccbd75c1e514317a815f90e85/chrome/browser/chromeos/accessibility/select_to_speak_event_handler_unittest.cc
[modify] https://crrev.com/6c4193276b6a8a8ccbd75c1e514317a815f90e85/chrome/browser/extensions/api/automation/automation_apitest.cc
[modify] https://crrev.com/6c4193276b6a8a8ccbd75c1e514317a815f90e85/chrome/browser/extensions/api/automation_internal/automation_internal_api.cc
[modify] https://crrev.com/6c4193276b6a8a8ccbd75c1e514317a815f90e85/chrome/browser/ui/aura/accessibility/automation_manager_aura.cc
[modify] https://crrev.com/6c4193276b6a8a8ccbd75c1e514317a815f90e85/chrome/browser/ui/aura/accessibility/automation_manager_aura.h
[add] https://crrev.com/6c4193276b6a8a8ccbd75c1e514317a815f90e85/chrome/test/data/extensions/api_test/automation/tests/desktop/hit_test.html
[add] https://crrev.com/6c4193276b6a8a8ccbd75c1e514317a815f90e85/chrome/test/data/extensions/api_test/automation/tests/desktop/hit_test.js

Labels: Type-Feature
Labels: -newcomponent-accessibility -newcomponent-accessibility-features
Project Member

Comment 13 by bugdroid1@chromium.org, Apr 28 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/621114575ac090c0137ac4c9c6eb3c6deebfe61c

commit 621114575ac090c0137ac4c9c6eb3c6deebfe61c
Author: dmazzoni <dmazzoni@chromium.org>
Date: Fri Apr 28 23:46:55 2017

Fix Chrome OS virtual keyboard accessibility

The user-visible change here is that Select-to-speak on Chrome OS now
works on the virtual keyboard. It also makes it visible to ChromeVox
so we can make that work better too.

Three fixes were required:

* We need to always set the window property on the window for a
  RenderWidgetHostViewAura, whether that WebContents already has
  accessibility enabled or not. That's solved by moving the
  trigger for the window property to RenderViewReady and
  WebContentsImpl::NotifySwappedFromRenderManager.

* AXAuraObjCache needs to listen for newly-created windows,
  changes to window bounds, and window property changes.
  Otherwise the virtual keyboard window is created but no
  accessibility events fire allowing automation clients to
  find it.

* The bounds for a AXWindowObjWrapper should be global bounds and not
  local. This wasn't caught before because widget bounds were correct,
  but the virtual keyboard is directly in an aura Window, with no Widget
  and no Views.

BUG= 699617 
CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.linux:closure_compilation;master.tryserver.chromium.linux:linux_site_isolation

Review-Url: https://codereview.chromium.org/2803823002
Cr-Commit-Position: refs/heads/master@{#468182}

[modify] https://crrev.com/621114575ac090c0137ac4c9c6eb3c6deebfe61c/chrome/browser/ui/aura/accessibility/automation_manager_aura.cc
[modify] https://crrev.com/621114575ac090c0137ac4c9c6eb3c6deebfe61c/chrome/browser/ui/aura/accessibility/automation_manager_aura.h
[add] https://crrev.com/621114575ac090c0137ac4c9c6eb3c6deebfe61c/chrome/browser/ui/aura/accessibility/automation_manager_aura_browsertest.cc
[modify] https://crrev.com/621114575ac090c0137ac4c9c6eb3c6deebfe61c/chrome/test/BUILD.gn
[modify] https://crrev.com/621114575ac090c0137ac4c9c6eb3c6deebfe61c/content/browser/frame_host/render_frame_host_impl.cc
[modify] https://crrev.com/621114575ac090c0137ac4c9c6eb3c6deebfe61c/content/browser/web_contents/web_contents_impl.cc
[modify] https://crrev.com/621114575ac090c0137ac4c9c6eb3c6deebfe61c/ui/views/accessibility/ax_aura_obj_cache.cc
[modify] https://crrev.com/621114575ac090c0137ac4c9c6eb3c6deebfe61c/ui/views/accessibility/ax_aura_obj_cache.h
[modify] https://crrev.com/621114575ac090c0137ac4c9c6eb3c6deebfe61c/ui/views/accessibility/ax_window_obj_wrapper.cc
[modify] https://crrev.com/621114575ac090c0137ac4c9c6eb3c6deebfe61c/ui/views/accessibility/ax_window_obj_wrapper.h

Status: Fixed (was: Assigned)
Project Member

Comment 15 by bugdroid1@chromium.org, May 17 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/ccb40f537fbbc17772e7255a7b575661ce4353cd

commit ccb40f537fbbc17772e7255a7b575661ce4353cd
Author: dmazzoni <dmazzoni@chromium.org>
Date: Wed May 17 05:25:30 2017

Refactor Select-to-speak so that mouse events are forwarded to the extension.

Previously the Select-to-speak event handler captured mouse events and
turned them into accessibility events that could be caught by
select-to-speak.

Instead, the event handler can mouse and keyboard events to the extension
background page, and the extension can use the automation API to do a hit test.

This is mostly a refactoring change, but it gives us some more flexibility
and makes the event handler simpler, possibly something that could be
general-purpose, while putting more of the logic into the extension code.

BUG= 699617 
CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.linux:closure_compilation

Review-Url: https://codereview.chromium.org/2814213002
Cr-Commit-Position: refs/heads/master@{#472339}

[modify] https://crrev.com/ccb40f537fbbc17772e7255a7b575661ce4353cd/chrome/browser/chromeos/BUILD.gn
[modify] https://crrev.com/ccb40f537fbbc17772e7255a7b575661ce4353cd/chrome/browser/chromeos/accessibility/select_to_speak_event_handler.cc
[modify] https://crrev.com/ccb40f537fbbc17772e7255a7b575661ce4353cd/chrome/browser/chromeos/accessibility/select_to_speak_event_handler.h
[modify] https://crrev.com/ccb40f537fbbc17772e7255a7b575661ce4353cd/chrome/browser/chromeos/accessibility/select_to_speak_event_handler_unittest.cc
[modify] https://crrev.com/ccb40f537fbbc17772e7255a7b575661ce4353cd/chrome/browser/resources/chromeos/select_to_speak/select_to_speak.js

Project Member

Comment 16 by bugdroid1@chromium.org, May 17 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/3baeba114a6669b0a2c02842cfa10bb293e5c57e

commit 3baeba114a6669b0a2c02842cfa10bb293e5c57e
Author: kolos <kolos@chromium.org>
Date: Wed May 17 09:33:31 2017

Revert of Refactor Select-to-speak so that mouse events are forwarded to the extension. (patchset #7 id:120001 of https://codereview.chromium.org/2814213002/ )

Reason for revert:
FindIt detected this CL as culprit for test failures.
https://findit-for-me.appspot.com/waterfall/build-failure?url=https://build.chromium.org/p/chromium.chromiumos/builders/Linux%20ChromiumOS%20Tests%20%28dbg%29%281%29/builds/26122

Original issue's description:
> Refactor Select-to-speak so that mouse events are forwarded to the extension.
>
> Previously the Select-to-speak event handler captured mouse events and
> turned them into accessibility events that could be caught by
> select-to-speak.
>
> Instead, the event handler can mouse and keyboard events to the extension
> background page, and the extension can use the automation API to do a hit test.
>
> This is mostly a refactoring change, but it gives us some more flexibility
> and makes the event handler simpler, possibly something that could be
> general-purpose, while putting more of the logic into the extension code.
>
> BUG= 699617 
> CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.linux:closure_compilation
>
> Review-Url: https://codereview.chromium.org/2814213002
> Cr-Commit-Position: refs/heads/master@{#472339}
> Committed: https://chromium.googlesource.com/chromium/src/+/ccb40f537fbbc17772e7255a7b575661ce4353cd

TBR=dtseng@chromium.org,dmazzoni@chromium.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG= 699617 

Review-Url: https://codereview.chromium.org/2883283003
Cr-Commit-Position: refs/heads/master@{#472397}

[modify] https://crrev.com/3baeba114a6669b0a2c02842cfa10bb293e5c57e/chrome/browser/chromeos/BUILD.gn
[modify] https://crrev.com/3baeba114a6669b0a2c02842cfa10bb293e5c57e/chrome/browser/chromeos/accessibility/select_to_speak_event_handler.cc
[modify] https://crrev.com/3baeba114a6669b0a2c02842cfa10bb293e5c57e/chrome/browser/chromeos/accessibility/select_to_speak_event_handler.h
[modify] https://crrev.com/3baeba114a6669b0a2c02842cfa10bb293e5c57e/chrome/browser/chromeos/accessibility/select_to_speak_event_handler_unittest.cc
[modify] https://crrev.com/3baeba114a6669b0a2c02842cfa10bb293e5c57e/chrome/browser/resources/chromeos/select_to_speak/select_to_speak.js

Project Member

Comment 17 by bugdroid1@chromium.org, May 23 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/8dadb18e68fa9688acfcd10757fade7ca980c0d7

commit 8dadb18e68fa9688acfcd10757fade7ca980c0d7
Author: dmazzoni <dmazzoni@chromium.org>
Date: Tue May 23 20:15:09 2017

Re-land: Refactor Select-to-speak so that mouse events are forwarded to the extension.

Originally landed: r472339 (https://codereview.chromium.org/2814213002)
Reverted: r472397 (https://codereview.chromium.org/2883283003)

Previously the Select-to-speak event handler captured mouse events and
turned them into accessibility events that could be caught by
select-to-speak.

Instead, the event handler can mouse and keyboard events to the extension
background page, and the extension can use the automation API to do a hit test.

This is mostly a refactoring change, but it gives us some more flexibility
and makes the event handler simpler, possibly something that could be
general-purpose, while putting more of the logic into the extension code.

BUG= 699617 
CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.linux:closure_compilation
TBR=dtseng

Review-Url: https://codereview.chromium.org/2901893002
Cr-Commit-Position: refs/heads/master@{#474026}

[modify] https://crrev.com/8dadb18e68fa9688acfcd10757fade7ca980c0d7/chrome/browser/chromeos/BUILD.gn
[modify] https://crrev.com/8dadb18e68fa9688acfcd10757fade7ca980c0d7/chrome/browser/chromeos/accessibility/select_to_speak_event_handler.cc
[modify] https://crrev.com/8dadb18e68fa9688acfcd10757fade7ca980c0d7/chrome/browser/chromeos/accessibility/select_to_speak_event_handler.h
[modify] https://crrev.com/8dadb18e68fa9688acfcd10757fade7ca980c0d7/chrome/browser/chromeos/accessibility/select_to_speak_event_handler_unittest.cc
[modify] https://crrev.com/8dadb18e68fa9688acfcd10757fade7ca980c0d7/chrome/browser/resources/chromeos/select_to_speak/select_to_speak.js

Labels: VerifyIn-61

Comment 19 by dchan@chromium.org, Jan 22 2018

Status: Archived (was: Fixed)

Sign in to add a comment