New issue
Advanced search Search tips

Issue 819773 link

Starred by 1 user
Status: Fixed
Owner:
dmazz...@chromium.org
Closed: Mar 2018
Cc:
a...@chromium.org
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug



Sign in to add a comment

WebContents::RequestAXTreeSnapshot does not traverse inner WebContentses

Project Member Reported by a...@chromium.org, Mar 7 2018 
 Bug 659753  is about speech not being able to say the contents of chrome://chrome-signin. That page is implemented using <webview>.

A way I'm planning on fixing that bug is getting the contents of the page using WebContents::RequestAXTreeSnapshot. The problem is that function's implementation:

  for (FrameTreeNode* frame_tree_node : frame_tree_.Nodes()) {
    frame_tree_node->current_frame_host()-> // AX stuff...
  }

The problem there is that the entire contents of what the user sees is that <webview> tag. For a <webview> tag, the webview itself is represented by a single FrameTreeNode in the parent WebContents. Yes, it has a stand-in RenderFrameHost returned by current_frame_host(), but that's not a real one and has no RenderFrame counterpart. All the real RenderFrames, with their contents, live in the inner WebContents.

What are the uses of WebContents::RequestAXTreeSnapshot? Should it be traversing the overall WebContents tree structure?

Comment 1 by a...@chromium.org, Mar 7 2018 

Finding the inner tree would be a simple recursion with WebContentsTreeNode::GetInnerWebContentsInFrame, but it's not immediately clear to me what changes would need to be made to the AXTreeCombiner.
Project Member

Comment 2 by bugdroid1@chromium.org, Mar 16 2018 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/1e620f311f9d2529ed23cdddee3bf7c6a5497ae9

commit 1e620f311f9d2529ed23cdddee3bf7c6a5497ae9
Author: Avi Drissman <avi@chromium.org>
Date: Fri Mar 16 13:57:29 2018

Improve RequestAXTreeSnapshot.

Allow it to traverse <webview> and allow the mode to
be specified.

BUG= 659753 ,  819773 

Change-Id: I45aaf11b6f91481a65ce40790e4e5d785c23ec1b
Reviewed-on: https://chromium-review.googlesource.com/956301
Reviewed-by: Daniel Cheng <dcheng@chromium.org>
Reviewed-by: Hidehiko Abe <hidehiko@chromium.org>
Reviewed-by: Dominic Mazzoni <dmazzoni@chromium.org>
Commit-Queue: Avi Drissman <avi@chromium.org>
Cr-Commit-Position: refs/heads/master@{#543691}
[modify] https://crrev.com/1e620f311f9d2529ed23cdddee3bf7c6a5497ae9/chrome/browser/chromeos/arc/voice_interaction/arc_voice_interaction_arc_home_service.cc
[modify] https://crrev.com/1e620f311f9d2529ed23cdddee3bf7c6a5497ae9/chrome/browser/chromeos/arc/voice_interaction/arc_voice_interaction_arc_home_service_browsertest.cc
[modify] https://crrev.com/1e620f311f9d2529ed23cdddee3bf7c6a5497ae9/content/browser/accessibility/snapshot_ax_tree_browsertest.cc
[modify] https://crrev.com/1e620f311f9d2529ed23cdddee3bf7c6a5497ae9/content/browser/frame_host/render_frame_host_impl.cc
[modify] https://crrev.com/1e620f311f9d2529ed23cdddee3bf7c6a5497ae9/content/browser/frame_host/render_frame_host_impl.h
[modify] https://crrev.com/1e620f311f9d2529ed23cdddee3bf7c6a5497ae9/content/browser/web_contents/web_contents_android.cc
[modify] https://crrev.com/1e620f311f9d2529ed23cdddee3bf7c6a5497ae9/content/browser/web_contents/web_contents_impl.cc
[modify] https://crrev.com/1e620f311f9d2529ed23cdddee3bf7c6a5497ae9/content/browser/web_contents/web_contents_impl.h
[modify] https://crrev.com/1e620f311f9d2529ed23cdddee3bf7c6a5497ae9/content/common/accessibility_messages.h
[modify] https://crrev.com/1e620f311f9d2529ed23cdddee3bf7c6a5497ae9/content/public/browser/web_contents.h
[modify] https://crrev.com/1e620f311f9d2529ed23cdddee3bf7c6a5497ae9/content/renderer/accessibility/aom_content_ax_tree.cc
[modify] https://crrev.com/1e620f311f9d2529ed23cdddee3bf7c6a5497ae9/content/renderer/accessibility/render_accessibility_impl.cc
[modify] https://crrev.com/1e620f311f9d2529ed23cdddee3bf7c6a5497ae9/content/renderer/accessibility/render_accessibility_impl.h
[modify] https://crrev.com/1e620f311f9d2529ed23cdddee3bf7c6a5497ae9/content/renderer/render_frame_impl.cc
[modify] https://crrev.com/1e620f311f9d2529ed23cdddee3bf7c6a5497ae9/content/renderer/render_frame_impl.h

Comment 3 by a...@chromium.org, Mar 16 2018

Status: Fixed (was: Assigned)
Project Member

Comment 4 by bugdroid1@chromium.org, Mar 17 2018 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/c32a0a57a3af05729cc265e6ac7a89032817e547

commit c32a0a57a3af05729cc265e6ac7a89032817e547
Author: Avi Drissman <avi@chromium.org>
Date: Sat Mar 17 02:35:02 2018

Use modern APIs to power speaking the whole page.

It's unclear how much the "Start Speaking" on the whole
page is used. It was implemented only three years ago and
has never worked correctly. It's not clear what it even
means to speak the entire web document for modern web
pages. However, this makes a decent attempt at doing so.

BUG= 659753 ,  584798 , 585164,  819773 
TEST=as in 659753

Change-Id: I252f38c7c7879173c3c4e0afd9dc3a42c81b8a64
Reviewed-on: https://chromium-review.googlesource.com/956029
Reviewed-by: Dominic Mazzoni <dmazzoni@chromium.org>
Reviewed-by: Robert Sesek <rsesek@chromium.org>
Commit-Queue: Avi Drissman <avi@chromium.org>
Cr-Commit-Position: refs/heads/master@{#543920}
[modify] https://crrev.com/c32a0a57a3af05729cc265e6ac7a89032817e547/content/DEPS
[modify] https://crrev.com/c32a0a57a3af05729cc265e6ac7a89032817e547/content/browser/renderer_host/render_widget_host_view_mac.h
[modify] https://crrev.com/c32a0a57a3af05729cc265e6ac7a89032817e547/content/browser/renderer_host/render_widget_host_view_mac.mm
[add] https://crrev.com/c32a0a57a3af05729cc265e6ac7a89032817e547/content/browser/renderer_host/render_widget_host_view_mac_browsertest.mm
[modify] https://crrev.com/c32a0a57a3af05729cc265e6ac7a89032817e547/content/common/view_messages.h
[modify] https://crrev.com/c32a0a57a3af05729cc265e6ac7a89032817e547/content/renderer/render_view_impl.cc
[modify] https://crrev.com/c32a0a57a3af05729cc265e6ac7a89032817e547/content/renderer/render_view_impl.h
[modify] https://crrev.com/c32a0a57a3af05729cc265e6ac7a89032817e547/content/test/BUILD.gn
►

Sign in to add a comment