New issue
Advanced search Search tips

Issue 758301 link

Starred by 2 users

Issue metadata

Status: Fixed
Owner:
Closed: Dec 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 2
Type: Bug
STS
Team-Accessibility

Blocked on:
issue 770248
issue 773851
issue 792545



Sign in to add a comment

[Select to Speak] Issues with gmail

Project Member Reported by lpalmaro@chromium.org, Aug 23 2017

Issue description

OS: Chrome
Version: 61.0.3163.38

1) Enable select to speak (STS)
2) Navigate to gmail
3) try to use STS to read a hangouts chat, email draft, body of an email, etc. 

Notice that every time you select something, you first hear "basic html view", then a pause, then it typically will read more than what you've selected. In the case of selecting text from a chat, I actually can't get STS to read the chat content to me -- it continues to go to the background content in gmail instead. 

The biggest issues seem to be in email drafts and hangouts chats. In email messages I've received, I first hear "basic html view" then a pause, then it typically does read what I've selected. So, while the extra verbalization and pause is a bit or a productivity hit, at least I can hear that text. 


 
Owner: ----

Comment 2 by katydek@google.com, Sep 29 2017

Blockedon: 770248
It looks like the chat is rendered in an iframe, and we just found a bug where all iframe contents aren't read out at all ( crbug.com/770248 , added to blocking). As a hack for demos, etc you can turn on "Native accessibility API support" in chrome://accessibility to get iframes to work with select-to-speak.

Comment 3 by katydek@google.com, Sep 29 2017

Re Basic HTML view: Looks like gmail is creating a button to go to a "basic html view" which it will display visually to the user when the page is loading slowly. That button has 0 size but we seem to be assigning it a size anyway in the accessibility tree.

Specifically, in ax_tree.cc RelativeToTreeBounds we set the bounds to the container bounds if they are size 0. Maybe here we should check that the object should even have size -- are the bounds set to 0 in css / on purpose? Or maybe there's somewhere earlier in building the tree that added this node with non-zero size incorrectly?

I tried putting some logging at the very end of AXObject::GetRelativeBounds to see if that was setting the bounds incorrectly, but it seems it always sets bounds to 0x0.

String name = ComputedName();
if (name == "Basic HTML view") {
  LOG(ERROR) << ComputedName() << out_bounds_in_container.ToString();
}

So I think the problem may be because of RelativeToTreeBounds.

Comment 4 by katydek@google.com, Oct 2 2017

Regarding "Basic HTML View": 
Here's where it was made size (0,0) in gmail: https://critique.corp.google.com/#review/68387852. The gmail team wants users with screen readers to be able to tab-navigate to this element. However, they probably don't want Select-to-speak users to find the element because it should have no size.

In ax_tree.cc, the change to make the bounds equal to the container bounds came from https://bugs.chromium.org/p/chromium/issues/detail?id=756788. I'm thinking it would be better if nodes of 0 size had size (1, 1) bounds. It seems like a developer wouldn't explicitly set a thing to be size (0, 0) unless they didn't want it to be visible to users except for screen readers.

Comment 5 by katydek@google.com, Oct 2 2017

Cc: katydek@google.com dmazz...@chromium.org
Labels: STS

Comment 7 by katydek@google.com, Oct 18 2017

Owner: katydek@google.com

Comment 8 by katydek@google.com, Oct 18 2017

Blockedon: 773851

Comment 9 by katydek@google.com, Oct 31 2017

Cc: -katydek@google.com katie@chromium.org
Owner: katie@chromium.org

Comment 10 by katie@chromium.org, Nov 28 2017

Owner: lpalmaro@chromium.org
"Basic HTML view" and iframe problems fixed. 

Reassigning to Laura -- any other blocking or big issues with gmail and select-to-speak?
Owner: katydek@google.com
Status: Assigned (was: Available)
Just circling back here. This is a big improvement. The remaining items I noticed last week when testing were specifically around the email draft. Basically, when you try to use StS to read the contents of a new draft, StS reads what is behind the draft and not actually what's inside of it. This was the biggest remaining item I noticed in Gmail but will keep using it! 

Comment 12 by katie@chromium.org, Dec 12 2017

Blockedon: 792545

Comment 13 by katie@chromium.org, Dec 12 2017

Owner: katie@chromium.org

Comment 14 by katie@chromium.org, Dec 14 2017

Status: Fixed (was: Assigned)
Marking fixed because I think I've got the draft email issue fixed! Should be in a Canary near you shortly.
Components: UI>Accessibility>SelectToSpeak
Moving from just having STS label to also having the UI>Accessibility>SelectToSpeak component to make searching easier in the future. 

Sign in to add a comment