The code path for starting the speech from the menu is different from Option+Esc. The former relies on either using |selected_text_| or getting it from WebContents.

From the menu we hit:
https://cs.chromium.org/chromium/src/content/browser/renderer_host/render_widget_host_view_mac.mm?rcl=0&l=3312

and end up at RWHV::SpeakSelection().

The same problem with OOPIFs.

avi@, is this possibly fixed by your work on  issue 390749 ?

No, 390749 was web speech recognition; this is platform speech synthesis.

Looking at this bug from accessibility angle, it might prevent some disabled users from being able to use Chrome, right?  Because of this, let me tentatively add the Proj-SiteIsolation-LaunchBlocking label.

RenderWidgetHostViewMac::SpeakSelection() has two paths.

One is if there is text selected; that uses RenderWidgetHostViewMac::GetTextSelection() which uses the text input manager. The other, the one that is broken both in the <webview> case as well as the OOPIF case, is sending the ViewMsg_GetRenderedText event, and feeding the result (returned in ViewMsg_GetRenderedTextCompleted) to the speaking code.

This was filed in 2016. Since then, have we added an easy way to get all the textual content of the page that we can reuse here?

Also, ViewMsg_GetRenderedText is implemented with RenderViewImpl::OnGetRenderedText(). If you take a look at that function, the sheer number of bugs ( bug 393444 ,  bug 584798 , bug 585164) filed against its implementation is ... notable.

A lot of our "all text" seems broken. If you go to http://csreis.github.io/tests/cross-site-iframe.html, do command-A to select all, then copy and paste, the "copy" command gets sent to the top frame and ignores the content of the OOPIF.

To fix this bug, we may need to build some new frameworks for getting text content across nested frames.

This is interesting. Looking at  bug 584798  apparently the current order of reading things is different between "Select All" and read v/s reading the whole page (which occurs when we do not have any text selected). How does "Select All" currently work for OOPIFs. Maybe that could give us some clues on how to approach this?

"Select All"... and what?

... "and copy" connects up to the focused frame's FrameInputHandler and calls it. But that sends the "copy" command to the focused frame, which can't provide the contents of the embedded oopif frame.

I don't know how selection works overall offhand, but it's not clear we correctly get "all the text" in the general case.

Well, Command+A seems to highlight the whole page but Command+C only contains the non-OOPIF part and ignores the text inside editable elements. So "Select+All" does not seem to be helping much.

Avi: would it be easier/possible to just somehow traverse the NSView hierarchy and get the text on the browser side? Esp. given that this bug is Mac only.

There is no NSView magic to help us here. NSViews are way too high-level to have the info we need to correctly thread text.

That is unfortunate. To coordinate and get text from all frames seems a lot like find-in-page.

Elly points out that there is speech-to-text in VoiceOver, which is an entirely different feature to this one. Elly, you had a different bug?

Re comment 6, I just played with this, and the speech that is used by accessibility is VoiceOver, and that seems to work with light testing. I don't see this as a blocker.

I have a reasonable approach for doing this. Let's see if it works.

The approach is going to be using WebContents::RequestAXTreeSnapshot() to grab a snapshot and string together anything that is a kStaticText. I have a preliminary version that works decently, and I'm working with Dominic to extend the API so that it only fetches the text from the renderers for me.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/1e620f311f9d2529ed23cdddee3bf7c6a5497ae9

commit 1e620f311f9d2529ed23cdddee3bf7c6a5497ae9
Author: Avi Drissman <avi@chromium.org>
Date: Fri Mar 16 13:57:29 2018

Improve RequestAXTreeSnapshot.

Allow it to traverse <webview> and allow the mode to
be specified.

BUG= 659753 ,  819773 

Change-Id: I45aaf11b6f91481a65ce40790e4e5d785c23ec1b
Reviewed-on: https://chromium-review.googlesource.com/956301
Reviewed-by: Daniel Cheng <dcheng@chromium.org>
Reviewed-by: Hidehiko Abe <hidehiko@chromium.org>
Reviewed-by: Dominic Mazzoni <dmazzoni@chromium.org>
Commit-Queue: Avi Drissman <avi@chromium.org>
Cr-Commit-Position: refs/heads/master@{#543691}
[modify] https://crrev.com/1e620f311f9d2529ed23cdddee3bf7c6a5497ae9/chrome/browser/chromeos/arc/voice_interaction/arc_voice_interaction_arc_home_service.cc
[modify] https://crrev.com/1e620f311f9d2529ed23cdddee3bf7c6a5497ae9/chrome/browser/chromeos/arc/voice_interaction/arc_voice_interaction_arc_home_service_browsertest.cc
[modify] https://crrev.com/1e620f311f9d2529ed23cdddee3bf7c6a5497ae9/content/browser/accessibility/snapshot_ax_tree_browsertest.cc
[modify] https://crrev.com/1e620f311f9d2529ed23cdddee3bf7c6a5497ae9/content/browser/frame_host/render_frame_host_impl.cc
[modify] https://crrev.com/1e620f311f9d2529ed23cdddee3bf7c6a5497ae9/content/browser/frame_host/render_frame_host_impl.h
[modify] https://crrev.com/1e620f311f9d2529ed23cdddee3bf7c6a5497ae9/content/browser/web_contents/web_contents_android.cc
[modify] https://crrev.com/1e620f311f9d2529ed23cdddee3bf7c6a5497ae9/content/browser/web_contents/web_contents_impl.cc
[modify] https://crrev.com/1e620f311f9d2529ed23cdddee3bf7c6a5497ae9/content/browser/web_contents/web_contents_impl.h
[modify] https://crrev.com/1e620f311f9d2529ed23cdddee3bf7c6a5497ae9/content/common/accessibility_messages.h
[modify] https://crrev.com/1e620f311f9d2529ed23cdddee3bf7c6a5497ae9/content/public/browser/web_contents.h
[modify] https://crrev.com/1e620f311f9d2529ed23cdddee3bf7c6a5497ae9/content/renderer/accessibility/aom_content_ax_tree.cc
[modify] https://crrev.com/1e620f311f9d2529ed23cdddee3bf7c6a5497ae9/content/renderer/accessibility/render_accessibility_impl.cc
[modify] https://crrev.com/1e620f311f9d2529ed23cdddee3bf7c6a5497ae9/content/renderer/accessibility/render_accessibility_impl.h
[modify] https://crrev.com/1e620f311f9d2529ed23cdddee3bf7c6a5497ae9/content/renderer/render_frame_impl.cc
[modify] https://crrev.com/1e620f311f9d2529ed23cdddee3bf7c6a5497ae9/content/renderer/render_frame_impl.h

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/c32a0a57a3af05729cc265e6ac7a89032817e547

commit c32a0a57a3af05729cc265e6ac7a89032817e547
Author: Avi Drissman <avi@chromium.org>
Date: Sat Mar 17 02:35:02 2018

Use modern APIs to power speaking the whole page.

It's unclear how much the "Start Speaking" on the whole
page is used. It was implemented only three years ago and
has never worked correctly. It's not clear what it even
means to speak the entire web document for modern web
pages. However, this makes a decent attempt at doing so.

BUG= 659753 ,  584798 , 585164,  819773 
TEST=as in 659753

Change-Id: I252f38c7c7879173c3c4e0afd9dc3a42c81b8a64
Reviewed-on: https://chromium-review.googlesource.com/956029
Reviewed-by: Dominic Mazzoni <dmazzoni@chromium.org>
Reviewed-by: Robert Sesek <rsesek@chromium.org>
Commit-Queue: Avi Drissman <avi@chromium.org>
Cr-Commit-Position: refs/heads/master@{#543920}
[modify] https://crrev.com/c32a0a57a3af05729cc265e6ac7a89032817e547/content/DEPS
[modify] https://crrev.com/c32a0a57a3af05729cc265e6ac7a89032817e547/content/browser/renderer_host/render_widget_host_view_mac.h
[modify] https://crrev.com/c32a0a57a3af05729cc265e6ac7a89032817e547/content/browser/renderer_host/render_widget_host_view_mac.mm
[add] https://crrev.com/c32a0a57a3af05729cc265e6ac7a89032817e547/content/browser/renderer_host/render_widget_host_view_mac_browsertest.mm
[modify] https://crrev.com/c32a0a57a3af05729cc265e6ac7a89032817e547/content/common/view_messages.h
[modify] https://crrev.com/c32a0a57a3af05729cc265e6ac7a89032817e547/content/renderer/render_view_impl.cc
[modify] https://crrev.com/c32a0a57a3af05729cc265e6ac7a89032817e547/content/renderer/render_view_impl.h
[modify] https://crrev.com/c32a0a57a3af05729cc265e6ac7a89032817e547/content/test/BUILD.gn