New issue
Advanced search Search tips

Issue 778080 link

Starred by 1 user
Status: Fixed
Owner:
katie@chromium.org
Closed: Dec 2017
Cc:
dtseng@chromium.org
lpalmaro@chromium.org
katie@chromium.org
kathrelk...@chromium.org
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 3
Type: Bug
STS
Team-Accessibility

Blocking:
issue 775556



Sign in to add a comment

Select to Speak gets stuck on after clicking on a notification

Project Member Reported by kathrelk...@chromium.org, Oct 25 2017 
Chrome Version: 63.0.3239.11, 10032.7.0, Eve

Example repro:
(1) Create a notification (e.g. take a screenshot).
(2) Search click or search drag on that notification.

Instead of reading anything, clicking on the notification happens.  Then Select to Speak gets confused and continues to read whatever you click on (and not clicking!) until you hit Search again.

Comment 1 by yawano@chromium.org, Nov 27 2017

Blocking: 775556

Comment 2 by dmazz...@chromium.org, Nov 27 2017

Owner: katie@chromium.org
Status: Assigned (was: Untriaged)

Comment 3 by katie@chromium.org, Nov 27 2017

Cc: katie@chromium.org
Owner: kathrelk...@chromium.org
I'm not able to reproduce this at HEAD.

Comment 4 by lpalmaro@google.com, Nov 29 2017

Labels: STS

Comment 5 by katie@chromium.org, Dec 5 2017

Cc: kathrelk...@chromium.org
Owner: ----
I can repro it now. Maybe I had a weird build.

Comment 6 by katie@chromium.org, Dec 5 2017 

Looks like the screenshot notification is being marked as 'offscreen'.

Comment 7 by katie@chromium.org, Dec 6 2017

Owner: katie@chromium.org

Comment 8 by katie@chromium.org, Dec 6 2017 

We are notifying when the bounds of the window changes, but somehow AXTree isn't getting those updated bounds in RelativeToTreeBounds.

Here's the animation of some outer window, logged at AXWindowObjWrapper::OnWindowBoundsChanged:

[85500:85500:1205/172024.457216:ERROR:ax_window_obj_wrapper.cc(123)] 1354,390 5x328
[85500:85500:1205/172024.474775:ERROR:ax_window_obj_wrapper.cc(123)] 1260,390 99x328
[85500:85500:1205/172024.497555:ERROR:ax_window_obj_wrapper.cc(123)] 1156,390 203x328
[85500:85500:1205/172024.514665:ERROR:ax_window_obj_wrapper.cc(123)] 1096,390 263x328
[85500:85500:1205/172024.532314:ERROR:ax_window_obj_wrapper.cc(123)] 1049,390 310x328
[85500:85500:1205/172024.549236:ERROR:ax_window_obj_wrapper.cc(123)] 1019,390 340x328
[85500:85500:1205/172024.566967:ERROR:ax_window_obj_wrapper.cc(123)] 1002,390 357x328
[85500:85500:1205/172024.584694:ERROR:ax_window_obj_wrapper.cc(123)] 999,390 360x328


Meanwhile, after the window is fully done animating, AX tree is clipping its children because it thinks the size is equal to the FIRST animation frame only:
window size 0.000000,0.000000 5.000000x328.000000

So we need to make sure AXWindowObjWrapper is correctly piping changes to the AXNode.

AXTreeSerializer::SerializeChangedNodes is being called, leading to AXWindowObjWrapper::Serialize being called, and getting the bounds correctly...
[90297:90297:1205/174128.500745:ERROR:ax_window_obj_wrapper.cc(73)] 999,390 360x328

Comment 9 by katie@chromium.org, Dec 6 2017 

Note: this could be fixed by reverting https://chromium-review.googlesource.com/780362.

Comment 10 by katie@chromium.org, Dec 6 2017 

Looks like there are several layers of "window", and not all of them are getting updated in the AXTree when the animation happens Below is the relevant part of the AXTree as logged from AutomationInternalCustomBindings::OnAccessibilityEvent -- the same tree is found when AXTree::RelativeToTreeBounds is used.

Specifically, this Window is too narrow even after animation:
id=170 window (0, 0)-(5, 328) offset_container_id=171 role_description=Notification settings name=Screenshot taken
Show in folder

Relevant parts of the tree:

[1:1:1206/144601.428156:ERROR:automation_internal_custom_bindings.cc(1231)] AXTree tree_id=0 loaded=true loading_progress=1
id=1 desktop (0, 0)-(0, 0) chrome_channel= actions= child_ids=7,2
  id=7 window (0, 0)-(1366, 768) name=Display--2094967296 clips_children=true actions= child_ids=6
    id=6 window (0, 0)-(1366, 768) offset_container_id=7 name= clips_children=true actions= child_ids=8,5,9,10,11,12,13
        id=117 window (0, 0)-(1366, 768) offset_container_id=11 name= clips_children=true actions= child_ids=132
          id=172 window (999, 390)-(360, 328) offset_container_id=117 name= clips_children=true actions= child_ids=168
            id=168 window (0, 0)-(5, 328) offset_container_id=172 name= clips_children=true actions= child_ids=171
              id=171 window (0, 0)-(360, 328) offset_container_id=168 name= description= class_name=RootView clips_children=true actions= child_ids=170
                id=170 window (0, 0)-(5, 328) offset_container_id=171 role_description=Notification settings name=Screenshot taken
Show in folder
 description= class_name=ToastContentsView clips_children=true actions= child_ids=162
                  id=162 button FOCUSABLE (0, 0)-(360, 328) offset_container_id=170 role_description=Notification settings name=Screenshot taken
Show in folder
 description= class_name=View actions= child_ids=173,161,167,174,163
                    id=173 unknown (0, 0)-(360, 328) offset_container_id=162 description= class_name=View actions=
                    id=161 button FOCUSABLE (0, 0)-(360, 32) offset_container_id=162 action=press name= description= class_name=Button actions= child_ids=160,175,176
                      id=160 unknown (2, 2)-(149, 30) offset_container_id=161 description= class_name=View actions= child_ids=177,178,179,180,181
                        id=177 image (14, 0)-(22, 30) offset_container_id=160 name= description= class_name=ImageView actions=
                        id=178 staticText (38, 0)-(60, 30) offset_container_id=160 name=Screenshot description= class_name=Label actions=
                        id=179 staticText (100, 0)-(10, 30) offset_container_id=160 name= •  description= class_name=Label actions=
                        id=180 staticText (112, 0)-(23, 30) offset_container_id=160 name=now description= class_name=Label actions=
                        id=181 button FOCUSABLE (137, 0)-(10, 30) offset_container_id=160 name=Collapse notification description=Collapse notification class_name=ImageView actions=
                      id=175 unknown (153, 2)-(179, 30) offset_container_id=161 description= class_name=View actions=
                      id=176 unknown (334, 2)-(24, 24) offset_container_id=161 description= class_name=NotificationControlButtonsView actions= child_ids=165
                        id=165 button FOCUSABLE (0, 0)-(24, 24) offset_container_id=176 action=press name=Notification close description=Close class_name=ImageButton actions=
                    id=167 unknown (0, 32)-(360, 52) offset_container_id=162 description= class_name=View actions= child_ids=182,166
                      id=182 unknown (12, 0)-(300, 36) offset_container_id=167 description= class_name=View actions= child_ids=183,184
                        id=183 staticText (4, 2)-(292, 17) offset_container_id=182 name=Screenshot taken description= class_name=Label actions=
                        id=184 staticText (4, 19)-(292, 17) offset_container_id=182 name=Show in folder description= class_name=View actions=
                      id=166 unknown (312, 0)-(36, 36) offset_container_id=167 description= class_name=View actions=
                    id=174 unknown (0, 84)-(360, 196) offset_container_id=162 description= class_name=LargeImageContainerView actions= child_ids=185
                      id=185 unknown (12, 0)-(336, 184) offset_container_id=174 description= class_name=LargeImageView actions=
                    id=163 unknown (0, 280)-(360, 48) offset_container_id=162 description= class_name=View actions= child_ids=186
                      id=186 unknown (0, 0)-(360, 48) offset_container_id=163 description= class_name=View actions= child_ids=164
                        id=164 button FOCUSABLE (8, 8)-(141, 32) offset_container_id=186 action=press name=COPY TO CLIPBOARD description= class_name=NotificationButtonMD actions= child_ids=187,188
                          id=187 image (12, 16)-(0, 0) offset_container_id=164 name= description= class_name=ImageView actions=
                          id=188 staticText (12, 0)-(117, 32) offset_container_id=164 name=COPY TO CLIPBOARD description= class_name=Label actions=
Project Member

Comment 11 by bugdroid1@chromium.org, Dec 13 2017 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/c28658521df0eee696b38c677433a9086091325c

commit c28658521df0eee696b38c677433a9086091325c
Author: Katie Dektar <katydek@google.com>
Date: Wed Dec 13 02:13:28 2017

Fixes accessiblity updates in native views that scroll or animate.

Bug:  789920 , 778080 , 789282 

Change-Id: Ie4b9d18ffbff48602126f6a4024fdd39f49fe032
Reviewed-on: https://chromium-review.googlesource.com/815878
Commit-Queue: Katie D <katie@chromium.org>
Reviewed-by: Sadrul Chowdhury <sadrul@chromium.org>
Reviewed-by: Dominic Mazzoni <dmazzoni@chromium.org>
Cr-Commit-Position: refs/heads/master@{#523654}
[modify] https://crrev.com/c28658521df0eee696b38c677433a9086091325c/ui/views/accessibility/ax_window_obj_wrapper.cc
[modify] https://crrev.com/c28658521df0eee696b38c677433a9086091325c/ui/views/view.cc
[modify] https://crrev.com/c28658521df0eee696b38c677433a9086091325c/ui/views/view.h
[modify] https://crrev.com/c28658521df0eee696b38c677433a9086091325c/ui/views/view_unittest.cc

Comment 12 by katie@chromium.org, Dec 13 2017

Status: Fixed (was: Assigned)
Project Member

Comment 13 by bugdroid1@chromium.org, Dec 19 2017

Labels: merge-merged-3282
The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/4d06b722374fed663b8d5d604627160f5db2dc28

commit 4d06b722374fed663b8d5d604627160f5db2dc28
Author: Katie Dektar <katydek@google.com>
Date: Tue Dec 19 16:37:21 2017

Fixes accessiblity updates in native views that scroll or animate.

Bug:  789920 , 778080 , 789282 

Change-Id: Ie4b9d18ffbff48602126f6a4024fdd39f49fe032
Reviewed-on: https://chromium-review.googlesource.com/815878
Commit-Queue: Katie D <katie@chromium.org>
Reviewed-by: Sadrul Chowdhury <sadrul@chromium.org>
Reviewed-by: Dominic Mazzoni <dmazzoni@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#523654}(cherry picked from commit c28658521df0eee696b38c677433a9086091325c)
Reviewed-on: https://chromium-review.googlesource.com/832380
Cr-Commit-Position: refs/branch-heads/3282@{#295}
Cr-Branched-From: 5fdc0fab22ce7efd32532ee989b223fa12f8171e-refs/heads/master@{#520840}
[modify] https://crrev.com/4d06b722374fed663b8d5d604627160f5db2dc28/ui/views/accessibility/ax_window_obj_wrapper.cc
[modify] https://crrev.com/4d06b722374fed663b8d5d604627160f5db2dc28/ui/views/view.cc
[modify] https://crrev.com/4d06b722374fed663b8d5d604627160f5db2dc28/ui/views/view.h
[modify] https://crrev.com/4d06b722374fed663b8d5d604627160f5db2dc28/ui/views/view_unittest.cc

Comment 14 by leberly@chromium.org, Apr 10 2018

Components: UI>Accessibility>SelectToSpeak
Moving from just having STS label to also having the UI>Accessibility>SelectToSpeak component to make searching easier in the future.
►

Sign in to add a comment