New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 842868 link

Starred by 3 users

Issue metadata

Status: Fixed
Owner:
Closed: Sep 3
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 1
Type: Bug
Team-Accessibility

Blocking:
issue 873039



Sign in to add a comment

Typing the wrong letter with touch/ChromeVox on the VK

Project Member Reported by lpalmaro@chromium.org, May 14 2018

Issue description

Chrome Version: 68.0.3429.0
OS: Chrome 

What steps will reproduce the problem?
(1) Enable ChromeVox (Ctrl Alt Z, or enable through accessibility settings)
(2) Enable the accessibility on-screen keyboard from accessibility settings (or use the vk on a convertible device or tablet)
(3)attempt to use your finger to navigate the vk using ChromeVox


What is the expected result?
You should be able to drag your finger around the keyboard and hear whatever is underneth it, and then lift to type that last letter you heard. 

What happens instead?
Instead, you drag, hear what you are touching, then lift but it types the wrong letter. It's typing the letter that is below that key you are pressing. For example, when you touch W and then lift, it types an S. When you press and lift G, it types a B. 

We need to make sure the right keys are being pressed, and that the focus highlight is on the right key. Right now, the highlight ring is also slightly off, too low.  



Please use labels and text to provide additional information.

If this is a regression (i.e., worked before), please consider using the
bisect tool (https://www.chromium.org/developers/bisect-builds-py) to help
us identify the root cause and more rapidly triage the issue.

For graphics-related bugs, please copy/paste the contents of the about:gpu
page at the end of this report.


 
Owner: tranbao...@chromium.org
Status: Assigned (was: Available)
Cc: aboxhall@chromium.org
Cc: gracec@chromium.org
Status: Started (was: Assigned)
Blocking: 873039
"Focus ring offset" and "wrong key being typed" are related. Basically, ChromeVox announces the key directly under the finger during touch exploration, but the focus ring is displayed at an offset from that key, then when the finger is lifted to activate, the key overlapped by the wrongly offset focus ring is activated, not the one directly under the user's finger. I guess it all boils down to the wrongly offset focus.

I've been investigating this for a while, tracing through various components and verifying the passing of bounds info between them. While I've ruled out quite some suspicious possible culprits, I've yet to identify the root cause. The issue seems obscure and subtle. Further investigation continues.

The following can be asserted up to now:
- The offset is always in the downward vertical direction. There's no offset in the horizontal direction.
- The offset amount is always about 25dp, i.e. about 50px on a screen with density scaling factor x2.
- The offset amount is unrelated to, i.e. not a function of, the heights of the following: 
  + ChromeVox's top bar
  + VK's top candidate row height
  + VK's total height
  + VK's individual key/row height
  although these are possible culprits "pushing" the focus ring down.
- The issue doesn't repro always esp. sometimes no repro if the keyboard is open before enabling ChromeVox.
- The issue can "fix itself" sometimes, e.g. after some screen rotation (i.e. physically rotating the device).
- The issue can repro on both docked and floating VK.

We still need to:
(1) identify the ultimate source of the vertical offset (what pushes it down? this will help unveil many things)
(2) look further into VK initialisation (any non-atomic sizing subject to race condition with the offset source?)
(3) check VK's interaction with ChromeVox, e.g. end-to-end on how and when bounds are communicated?
(4) inspect the accessibility tree further in greater detail, and check how it's related to the IME impl.

Cc: dvallet@chromium.org
It can now be confirmed that this is a regression caused by crrev.com/c/942603 six months ago, back in March 2018. The full timeline is as follows:

- Before crrev.com/c/940814 (1 Mar): Focus bounds in top screen area as if VK were docked to the top.

- crrev.com/c/940814 (1 Mar) --> before crrev.com/c/942603 (5 Mar): Focus bounds in bottom screen area and correctly aligned with VK keys.

- Since crrev.com/c/942603 (5 Mar): Focus bounds in bottom screen area but with mysterious vertical offset from VK keys.

The next step is to study crrev.com/c/942603 to identify the root cause (and probably also crrev.com/c/942603 for context), then consider either a rollback or a fix forward.

Project Member

Comment 8 by bugdroid1@chromium.org, Aug 29

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/7d715073832c03ad7eeca5bc939241287f5e627f

commit 7d715073832c03ad7eeca5bc939241287f5e627f
Author: David Tseng <dtseng@chromium.org>
Date: Wed Aug 29 19:21:49 2018

Ensure accessibility updates get sent when a window is transformed

When an aura Window receives a new transform, it possibly impacts its bounds. Previously, we were only observing OnWindowBoundsChanged, which does not get called for transform changes.

This change makes it so we also observe OnWindowTransformed and fire a location change to push the new location to all accessibility clients.

Bug:  842868 
Change-Id: I5251198da8b4af6cb96ad3197a6d37caa0a37faa
Reviewed-on: https://chromium-review.googlesource.com/1195645
Reviewed-by: Dominic Mazzoni <dmazzoni@chromium.org>
Commit-Queue: David Tseng <dtseng@chromium.org>
Cr-Commit-Position: refs/heads/master@{#587237}
[modify] https://crrev.com/7d715073832c03ad7eeca5bc939241287f5e627f/ui/views/accessibility/ax_window_obj_wrapper.cc
[modify] https://crrev.com/7d715073832c03ad7eeca5bc939241287f5e627f/ui/views/accessibility/ax_window_obj_wrapper.h

Status: Fixed (was: Started)
This has now been fixed by crrev.com/c/1195645 in ui/views/accessibility (see comment 8 above). Some analysis for posterity:

- VK-show/hide involves animation from a 30dp downward offset to normal location:
+ Full-width: 
https://cs.chromium.org/chromium/src/ui/keyboard/container_full_width_behavior.cc?rcl=813f8374b378fd8201f62bca6e75c19e20d92340&l=28-47
https://cs.chromium.org/chromium/src/ui/keyboard/container_full_width_behavior.h?rcl=d75ec76915bcaeb45dfccf5859480b502b660459&l=19-21
+ Floating: 
https://cs.chromium.org/chromium/src/ui/keyboard/container_floating_behavior.cc?rcl=813f8374b378fd8201f62bca6e75c19e20d92340&l=34-54
https://cs.chromium.org/chromium/src/ui/keyboard/container_floating_behavior.cc?rcl=813f8374b378fd8201f62bca6e75c19e20d92340&l=21-22

- Before crrev.com/c/942603: Snapshots of VK window's screen location into a11y tree upon bounds-change events.

- After crrev.com/c/942603, before crrev.com/c/1195645: Additional a11y snapshot upon window's visibility-change events fired at end of VK-hide animation and start of VK-show animation when VK is at 30dp-offset state. No re-snapshots when VK is back to its normal location.

- After crrev.com/c/1195645: Additional a11y snapshot at window's transform-change events. Re-snapshot at end of VK-show animation when transform zero is explicit set and VK is back to its normal location.

- No bounds-change events upon setting a bound-affecting transform, nor when the bounds are live-changing through an animation. A11y needs to listen to all of bounds-change, visibility-change, and transform-change events, so the a11y tree gets the most up-to-date info from the window, including screen location and visibility state.

Sign in to add a comment