New issue
Advanced search Search tips

Issue 747122 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Oct 18
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 2
Type: Bug
STS



Sign in to add a comment

Need to support word highlighting in Select-to-speak on Android O

Project Member Reported by dmazz...@chromium.org, Jul 20 2017

Issue description

We need to implement addExtraDataToAccessibilityNodeInfo and specifically the EXTRA_DATA_TEXT_CHARACTER_LOCATION_KEY key.

This is tricky because we don't want to keep inline text box data in our accessibility tree at all times, we'll want to grab it as needed.

 
Project Member

Comment 1 by bugdroid1@chromium.org, Aug 7 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/0907dddfd38947587649d5acd6ceb4c28d4a2953

commit 0907dddfd38947587649d5acd6ceb4c28d4a2953
Author: Dominic Mazzoni <dmazzoni@chromium.org>
Date: Mon Aug 07 08:17:39 2017

Change SetAccessibilityFocus action to LoadInlineTextBoxes.

This more clearly reflects what the action actually does, and
extends it to support loading a set of nodes rather than a
single node. This will be needed by a future change that
implements new Android APIs to retrieve character bounds.

Covered by existing test: AndroidGranularityMovementBrowserTest

Bug:  747122 
Change-Id: I9a2e262e52d7a64a8114b2664251080d7f7e559f
Reviewed-on: https://chromium-review.googlesource.com/576369
Commit-Queue: Dominic Mazzoni <dmazzoni@chromium.org>
Reviewed-by: Devlin <rdevlin.cronin@chromium.org>
Reviewed-by: Nektarios Paisios <nektar@chromium.org>
Cr-Commit-Position: refs/heads/master@{#492284}
[modify] https://crrev.com/0907dddfd38947587649d5acd6ceb4c28d4a2953/chrome/browser/extensions/api/automation_internal/automation_internal_api.cc
[modify] https://crrev.com/0907dddfd38947587649d5acd6ceb4c28d4a2953/chrome/common/extensions/api/automation_internal.idl
[modify] https://crrev.com/0907dddfd38947587649d5acd6ceb4c28d4a2953/content/browser/accessibility/android_granularity_movement_browsertest.cc
[modify] https://crrev.com/0907dddfd38947587649d5acd6ceb4c28d4a2953/content/browser/accessibility/browser_accessibility_manager.cc
[modify] https://crrev.com/0907dddfd38947587649d5acd6ceb4c28d4a2953/content/browser/accessibility/browser_accessibility_manager.h
[modify] https://crrev.com/0907dddfd38947587649d5acd6ceb4c28d4a2953/content/browser/accessibility/web_contents_accessibility_android.cc
[modify] https://crrev.com/0907dddfd38947587649d5acd6ceb4c28d4a2953/content/renderer/accessibility/blink_ax_tree_source.cc
[modify] https://crrev.com/0907dddfd38947587649d5acd6ceb4c28d4a2953/content/renderer/accessibility/blink_ax_tree_source.h
[modify] https://crrev.com/0907dddfd38947587649d5acd6ceb4c28d4a2953/content/renderer/accessibility/render_accessibility_impl.cc
[modify] https://crrev.com/0907dddfd38947587649d5acd6ceb4c28d4a2953/content/renderer/accessibility/render_accessibility_impl.h
[modify] https://crrev.com/0907dddfd38947587649d5acd6ceb4c28d4a2953/ui/accessibility/ax_enums.idl
[modify] https://crrev.com/0907dddfd38947587649d5acd6ceb4c28d4a2953/ui/accessibility/ax_node_data.cc

Project Member

Comment 2 by bugdroid1@chromium.org, Aug 23 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/22fd1938bedd3bca1086bb100a1ca4f5d936ab2e

commit 22fd1938bedd3bca1086bb100a1ca4f5d936ab2e
Author: Dominic Mazzoni <dmazzoni@chromium.org>
Date: Wed Aug 23 18:05:10 2017

Support Android API to return accessibility character bounds.

Add support for an Android API to retrieve the bounding box of any
arbitrary character. This enables Android's built-in Select-to-speak
feature to highlight words as they're spoken.

Implementing the API itself is not too bad, just some coordinate
conversion. However, Chrome stores the underlying data needed to
retrieve accessible character bounding boxes in inline text box
nodes, which are not loaded by default on Android for performance
reasons.

So, if inline text boxes are missing when we get a request, we
lazily load them so that they're available for the subsequent
request. A future patch will improve this by allowing the
request to be fulfilled asynchronously, so we never have to
return incomplete results while waiting for the inline text box
data to load.

Bug:  747122 
Change-Id: I15ab4a18aba49e95869faec75d96f988b169bbfe
Reviewed-on: https://chromium-review.googlesource.com/580497
Commit-Queue: Dominic Mazzoni <dmazzoni@chromium.org>
Reviewed-by: David Trainor <dtrainor@chromium.org>
Reviewed-by: Paul Miller <paulmiller@chromium.org>
Cr-Commit-Position: refs/heads/master@{#496735}
[modify] https://crrev.com/22fd1938bedd3bca1086bb100a1ca4f5d936ab2e/content/browser/accessibility/browser_accessibility_android.cc
[modify] https://crrev.com/22fd1938bedd3bca1086bb100a1ca4f5d936ab2e/content/browser/accessibility/browser_accessibility_android.h
[modify] https://crrev.com/22fd1938bedd3bca1086bb100a1ca4f5d936ab2e/content/browser/accessibility/web_contents_accessibility_android.cc
[modify] https://crrev.com/22fd1938bedd3bca1086bb100a1ca4f5d936ab2e/content/browser/accessibility/web_contents_accessibility_android.h
[modify] https://crrev.com/22fd1938bedd3bca1086bb100a1ca4f5d936ab2e/content/public/android/BUILD.gn
[add] https://crrev.com/22fd1938bedd3bca1086bb100a1ca4f5d936ab2e/content/public/android/java/src/org/chromium/content/browser/accessibility/OWebContentsAccessibility.java
[modify] https://crrev.com/22fd1938bedd3bca1086bb100a1ca4f5d936ab2e/content/public/android/java/src/org/chromium/content/browser/accessibility/WebContentsAccessibility.java
[add] https://crrev.com/22fd1938bedd3bca1086bb100a1ca4f5d936ab2e/content/public/android/javatests/src/org/chromium/content/browser/accessibility/WebContentsAccessibilityTest.java

Comment 3 by lpalmaro@google.com, Nov 29 2017

Labels: STS
Status: Fixed (was: Started)

Sign in to add a comment