Issue metadata
Sign in to add a comment
|
Typing the wrong letter with touch/ChromeVox on the VK |
||||||||||||||||||||||||
Issue descriptionChrome Version: 68.0.3429.0 OS: Chrome What steps will reproduce the problem? (1) Enable ChromeVox (Ctrl Alt Z, or enable through accessibility settings) (2) Enable the accessibility on-screen keyboard from accessibility settings (or use the vk on a convertible device or tablet) (3)attempt to use your finger to navigate the vk using ChromeVox What is the expected result? You should be able to drag your finger around the keyboard and hear whatever is underneth it, and then lift to type that last letter you heard. What happens instead? Instead, you drag, hear what you are touching, then lift but it types the wrong letter. It's typing the letter that is below that key you are pressing. For example, when you touch W and then lift, it types an S. When you press and lift G, it types a B. We need to make sure the right keys are being pressed, and that the focus highlight is on the right key. Right now, the highlight ring is also slightly off, too low. Please use labels and text to provide additional information. If this is a regression (i.e., worked before), please consider using the bisect tool (https://www.chromium.org/developers/bisect-builds-py) to help us identify the root cause and more rapidly triage the issue. For graphics-related bugs, please copy/paste the contents of the about:gpu page at the end of this report.
,
Jul 9
,
Jul 21
,
Jul 23
,
Aug 10
,
Aug 10
"Focus ring offset" and "wrong key being typed" are related. Basically, ChromeVox announces the key directly under the finger during touch exploration, but the focus ring is displayed at an offset from that key, then when the finger is lifted to activate, the key overlapped by the wrongly offset focus ring is activated, not the one directly under the user's finger. I guess it all boils down to the wrongly offset focus. I've been investigating this for a while, tracing through various components and verifying the passing of bounds info between them. While I've ruled out quite some suspicious possible culprits, I've yet to identify the root cause. The issue seems obscure and subtle. Further investigation continues. The following can be asserted up to now: - The offset is always in the downward vertical direction. There's no offset in the horizontal direction. - The offset amount is always about 25dp, i.e. about 50px on a screen with density scaling factor x2. - The offset amount is unrelated to, i.e. not a function of, the heights of the following: + ChromeVox's top bar + VK's top candidate row height + VK's total height + VK's individual key/row height although these are possible culprits "pushing" the focus ring down. - The issue doesn't repro always esp. sometimes no repro if the keyboard is open before enabling ChromeVox. - The issue can "fix itself" sometimes, e.g. after some screen rotation (i.e. physically rotating the device). - The issue can repro on both docked and floating VK. We still need to: (1) identify the ultimate source of the vertical offset (what pushes it down? this will help unveil many things) (2) look further into VK initialisation (any non-atomic sizing subject to race condition with the offset source?) (3) check VK's interaction with ChromeVox, e.g. end-to-end on how and when bounds are communicated? (4) inspect the accessibility tree further in greater detail, and check how it's related to the IME impl.
,
Aug 24
It can now be confirmed that this is a regression caused by crrev.com/c/942603 six months ago, back in March 2018. The full timeline is as follows: - Before crrev.com/c/940814 (1 Mar): Focus bounds in top screen area as if VK were docked to the top. - crrev.com/c/940814 (1 Mar) --> before crrev.com/c/942603 (5 Mar): Focus bounds in bottom screen area and correctly aligned with VK keys. - Since crrev.com/c/942603 (5 Mar): Focus bounds in bottom screen area but with mysterious vertical offset from VK keys. The next step is to study crrev.com/c/942603 to identify the root cause (and probably also crrev.com/c/942603 for context), then consider either a rollback or a fix forward.
,
Aug 29
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/7d715073832c03ad7eeca5bc939241287f5e627f commit 7d715073832c03ad7eeca5bc939241287f5e627f Author: David Tseng <dtseng@chromium.org> Date: Wed Aug 29 19:21:49 2018 Ensure accessibility updates get sent when a window is transformed When an aura Window receives a new transform, it possibly impacts its bounds. Previously, we were only observing OnWindowBoundsChanged, which does not get called for transform changes. This change makes it so we also observe OnWindowTransformed and fire a location change to push the new location to all accessibility clients. Bug: 842868 Change-Id: I5251198da8b4af6cb96ad3197a6d37caa0a37faa Reviewed-on: https://chromium-review.googlesource.com/1195645 Reviewed-by: Dominic Mazzoni <dmazzoni@chromium.org> Commit-Queue: David Tseng <dtseng@chromium.org> Cr-Commit-Position: refs/heads/master@{#587237} [modify] https://crrev.com/7d715073832c03ad7eeca5bc939241287f5e627f/ui/views/accessibility/ax_window_obj_wrapper.cc [modify] https://crrev.com/7d715073832c03ad7eeca5bc939241287f5e627f/ui/views/accessibility/ax_window_obj_wrapper.h
,
Sep 3
This has now been fixed by crrev.com/c/1195645 in ui/views/accessibility (see comment 8 above). Some analysis for posterity: - VK-show/hide involves animation from a 30dp downward offset to normal location: + Full-width: https://cs.chromium.org/chromium/src/ui/keyboard/container_full_width_behavior.cc?rcl=813f8374b378fd8201f62bca6e75c19e20d92340&l=28-47 https://cs.chromium.org/chromium/src/ui/keyboard/container_full_width_behavior.h?rcl=d75ec76915bcaeb45dfccf5859480b502b660459&l=19-21 + Floating: https://cs.chromium.org/chromium/src/ui/keyboard/container_floating_behavior.cc?rcl=813f8374b378fd8201f62bca6e75c19e20d92340&l=34-54 https://cs.chromium.org/chromium/src/ui/keyboard/container_floating_behavior.cc?rcl=813f8374b378fd8201f62bca6e75c19e20d92340&l=21-22 - Before crrev.com/c/942603: Snapshots of VK window's screen location into a11y tree upon bounds-change events. - After crrev.com/c/942603, before crrev.com/c/1195645: Additional a11y snapshot upon window's visibility-change events fired at end of VK-hide animation and start of VK-show animation when VK is at 30dp-offset state. No re-snapshots when VK is back to its normal location. - After crrev.com/c/1195645: Additional a11y snapshot at window's transform-change events. Re-snapshot at end of VK-show animation when transform zero is explicit set and VK is back to its normal location. - No bounds-change events upon setting a bound-affecting transform, nor when the bounds are live-changing through an animation. A11y needs to listen to all of bounds-change, visibility-change, and transform-change events, so the a11y tree gets the most up-to-date info from the window, including screen location and visibility state. |
|||||||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||||||
Comment 1 by rvera@chromium.org
, Jul 9Status: Assigned (was: Available)