New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 646690 link

Starred by 5 users

Issue metadata

Status: Fixed
Owner:
OOO until 4th Feb
Closed: Aug 20
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 2
Type: Bug
Team-Accessibility



Sign in to add a comment

hterm: support screen readers / text-to-speech with changed text (e.g. ChromeVox)

Reported by ni...@jaberspace.com, Sep 14 2016

Issue description

Mode: force_next
Version: 55.0.2858.0
Reproduction Steps: 
1. Enable CVox Next.
2. ctrl+alt+t
3. Shell
Expected: As I type shell, or any other command, I expect for the changed output on the screen to be read.
Actual: I hear silence.
 
Cc: dtseng@chromium.org
Labels: Phase3
Interesting, I'm on 55.0.2858.0 and can't repro this. I type shell but then  I hear spoken feedback starting from the top every time I type a letter. This doesn't seem like the ideal behavior either though, as I'd assume I would want to hear the letter I just typed, not all of the text from the top each time... What do you guys think? 
By changed content, I am referring to outputted text... as an example, when
changing directories or when performing commands that causes a change in
the output. I ran into this when playing around with crouton and getting
Linux working.

Comment 3 Deleted

Comment 4 Deleted

Comment 5 by chaok@google.com, Feb 12 2017

Cc: lpalmaro@chromium.org dmazz...@chromium.org
Who's the proper owner for crosh accessibility?
Thanks!

Comment 6 Deleted

Comment 7 Deleted

Comment 8 Deleted

Cc: vapier@chromium.org
Components: Platform>Apps>Default>Hterm
unfortunately i don't have enough screen reader experience to know what the expected behavior is here with hterm.  if you're using a terminal (gnome-terminal/konsole/whatever), how does it know what content to speak when you scroll a page ?

when it comes to HTML integration, i'm also not familiar with the tags/attributes needed to make sure this is exposed.

Inside of Gnome, open a terminal and start 'orca'. Try typing in some commands and observe.

Generally, all new text is read. Any additional text is typically queued up (does not interrupt).

I would try playing around with marking various parts of the buffer as aria-live. This indicates to the screen reader to echo any changed content. aria-relevant controls reading of "additions" or "removals". I would try additions first.

The prompt should be treated more like a text field (e.g. typing characters would echo the letter typed). Again, observe the behavior in Orca.

had to install orca and speech-dispatcher for it to start working, but i've got it going now.  looks like orca only rereads lines that are new/changed ... so if i scroll up three lines, it'll read those three new lines only.

the tricky thing about hterm/crosh is that there is no input/text field like html.  by nature of terminals, when you type keys, they show up wherever the cursor is and go to the screen buffer after round tripping ... there is no actual local buffer like a <input> element.  the line buffer you get at a shell prompt is an illusion performed by the shell (usually by way of readline).

when i use ChromeVox in Secure Shell, it is reading things i type/delete, but it seems to be inconsistent as to reading content that is displayed as a result of commands run.  sometimes it works, but usually it doesn't.  i'll try putting some aria labels on the elements and see what happens.
Owner: vapier@chromium.org
Status: Assigned (was: Untriaged)
 Issue 810661  has been merged into this issue.
Summary: hterm: support screen readers / text-to-speech with changed text (e.g. ChromeVox) (was: Add an option to automatically speak changed text in terminal)
Project Member

Comment 15 by bugdroid1@chromium.org, May 15 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/apps/libapps/+/cbc223d74723cd376a3ca28bd1fd2d9907017093

commit cbc223d74723cd376a3ca28bd1fd2d9907017093
Author: Raymes Khoury <raymes@chromium.org>
Date: Tue May 15 04:11:03 2018

hterm/nassh: Add basic support for announcing command output to AT in hterm.

This adds basic support for announcing command output for Assistive
Technology. A live region is created and <p> elements added to it to
indicate that the screen reader should announce the output. The live
region is separate from the <x-screen> element is used because not all
output will be rendered to the <x-screen>. It also allows the output to
be structured in a way that is more suitable for AT.

For large amounts of output, delays are inserted before rendering text
to the live region to avoid performance issues which cause the DOM to
hang.

nassh and crosh are updated to use this support only when a screen
reader is detected to be present. When that's not the case we don't
add nodes to the live region since there is an associated performance
cost.

Change-Id: I9ab09a4bc869038d22d907097e8d80d57d904710
Bug:  822490 ,  646690 
Reviewed-on: https://chromium-review.googlesource.com/1013822
Reviewed-by: David Tseng <dtseng@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Tested-by: Raymes Khoury <raymes@chromium.org>

[modify] https://crrev.com/cbc223d74723cd376a3ca28bd1fd2d9907017093/hterm/doc/hack.md
[modify] https://crrev.com/cbc223d74723cd376a3ca28bd1fd2d9907017093/nassh/manifest_v2.json
[modify] https://crrev.com/cbc223d74723cd376a3ca28bd1fd2d9907017093/hterm/js/hterm_terminal.js
[modify] https://crrev.com/cbc223d74723cd376a3ca28bd1fd2d9907017093/nassh/manifest_ext.json
[modify] https://crrev.com/cbc223d74723cd376a3ca28bd1fd2d9907017093/nassh/js/crosh.js
[modify] https://crrev.com/cbc223d74723cd376a3ca28bd1fd2d9907017093/hterm/concat/hterm.concat
[modify] https://crrev.com/cbc223d74723cd376a3ca28bd1fd2d9907017093/hterm/js/hterm.js
[modify] https://crrev.com/cbc223d74723cd376a3ca28bd1fd2d9907017093/hterm/js/hterm_scrollport.js
[add] https://crrev.com/cbc223d74723cd376a3ca28bd1fd2d9907017093/hterm/js/hterm_accessibility_reader.js
[modify] https://crrev.com/cbc223d74723cd376a3ca28bd1fd2d9907017093/nassh/js/nassh_main.js
[modify] https://crrev.com/cbc223d74723cd376a3ca28bd1fd2d9907017093/hterm/html/hterm_test.html
[modify] https://crrev.com/cbc223d74723cd376a3ca28bd1fd2d9907017093/nassh/manifest_v1.5.json
[add] https://crrev.com/cbc223d74723cd376a3ca28bd1fd2d9907017093/hterm/js/hterm_accessibility_reader_tests.js

 Issue 840874  has been merged into this issue.
Project Member

Comment 17 by bugdroid1@chromium.org, May 16 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/apps/libapps/+/082f211ca44bff3bb536b726d7ceb77c5b70fe5a

commit 082f211ca44bff3bb536b726d7ceb77c5b70fe5a
Author: Raymes Khoury <raymes@chromium.org>
Date: Wed May 16 06:35:24 2018

Revert "hterm/nassh: Add basic support for announcing command output to AT in hterm."

This reverts commit cbc223d74723cd376a3ca28bd1fd2d9907017093.

Reason for revert: It broke the crosh app, probably because 505FEAE9DD5B27637DCF72045ECA2D5D7F66D2FD wasn't whitelisted in this CL: https://chromium-review.googlesource.com/c/chromium/src/+/1018500/4/chrome/common/extensions/api/_permission_features.json

Will revert until that's fixed.

Original change's description:
> hterm/nassh: Add basic support for announcing command output to AT in hterm.
> 
> This adds basic support for announcing command output for Assistive
> Technology. A live region is created and <p> elements added to it to
> indicate that the screen reader should announce the output. The live
> region is separate from the <x-screen> element is used because not all
> output will be rendered to the <x-screen>. It also allows the output to
> be structured in a way that is more suitable for AT.
> 
> For large amounts of output, delays are inserted before rendering text
> to the live region to avoid performance issues which cause the DOM to
> hang.
> 
> nassh and crosh are updated to use this support only when a screen
> reader is detected to be present. When that's not the case we don't
> add nodes to the live region since there is an associated performance
> cost.
> 
> Change-Id: I9ab09a4bc869038d22d907097e8d80d57d904710
> Bug:  822490 ,  646690 
> Reviewed-on: https://chromium-review.googlesource.com/1013822
> Reviewed-by: David Tseng <dtseng@chromium.org>
> Reviewed-by: Mike Frysinger <vapier@chromium.org>
> Tested-by: Raymes Khoury <raymes@chromium.org>

TBR=raymes@chromium.org,vapier@chromium.org,dtseng@chromium.org

# Not skipping CQ checks because original CL landed > 1 day ago.

Bug:  822490 ,  646690 
Change-Id: If2fa1c9b0289e0b2de75e27ff21eed2405de9f28
Reviewed-on: https://chromium-review.googlesource.com/1060813
Reviewed-by: Raymes Khoury <raymes@chromium.org>
Tested-by: Raymes Khoury <raymes@chromium.org>

[modify] https://crrev.com/082f211ca44bff3bb536b726d7ceb77c5b70fe5a/hterm/doc/hack.md
[modify] https://crrev.com/082f211ca44bff3bb536b726d7ceb77c5b70fe5a/nassh/manifest_v2.json
[modify] https://crrev.com/082f211ca44bff3bb536b726d7ceb77c5b70fe5a/hterm/js/hterm_terminal.js
[modify] https://crrev.com/082f211ca44bff3bb536b726d7ceb77c5b70fe5a/nassh/manifest_ext.json
[modify] https://crrev.com/082f211ca44bff3bb536b726d7ceb77c5b70fe5a/nassh/js/crosh.js
[modify] https://crrev.com/082f211ca44bff3bb536b726d7ceb77c5b70fe5a/hterm/concat/hterm.concat
[modify] https://crrev.com/082f211ca44bff3bb536b726d7ceb77c5b70fe5a/hterm/js/hterm.js
[modify] https://crrev.com/082f211ca44bff3bb536b726d7ceb77c5b70fe5a/hterm/js/hterm_scrollport.js
[delete] https://crrev.com/cbc223d74723cd376a3ca28bd1fd2d9907017093/hterm/js/hterm_accessibility_reader.js
[modify] https://crrev.com/082f211ca44bff3bb536b726d7ceb77c5b70fe5a/nassh/js/nassh_main.js
[modify] https://crrev.com/082f211ca44bff3bb536b726d7ceb77c5b70fe5a/hterm/html/hterm_test.html
[modify] https://crrev.com/082f211ca44bff3bb536b726d7ceb77c5b70fe5a/nassh/manifest_v1.5.json
[delete] https://crrev.com/cbc223d74723cd376a3ca28bd1fd2d9907017093/hterm/js/hterm_accessibility_reader_tests.js

Project Member

Comment 18 by bugdroid1@chromium.org, May 17 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/apps/libapps/+/3e44bc9eb923050b154ba64a813fe876646aef13

commit 3e44bc9eb923050b154ba64a813fe876646aef13
Author: Raymes Khoury <raymes@chromium.org>
Date: Thu May 17 03:36:27 2018

hterm/nassh: Add basic support for announcing command output to AT in hterm. (Reland)

This adds basic support for announcing command output for Assistive
Technology. A live region is created and <p> elements added to it to
indicate that the screen reader should announce the output. The live
region is separate from the <x-screen> element is used because not all
output will be rendered to the <x-screen>. It also allows the output to
be structured in a way that is more suitable for AT.

For large amounts of output, delays are inserted before rendering text
to the live region to avoid performance issues which cause the DOM to
hang.

nassh and crosh are updated to use this support only when a screen
reader is detected to be present. When that's not the case we don't
add nodes to the live region since there is an associated performance
cost.

This is a reland of https://chromium-review.googlesource.com/c/1013822/
which broke because crosh didn't have accessibilityFeatures permission.
A runtime check has been added for now to avoid needing to wait for the
Chrome-side change to land.

Bug:  822490 ,  646690 
Change-Id: Ie588d5f962a6e61bbc994cf956f0622e40c690b4
Reviewed-on: https://chromium-review.googlesource.com/1063047
Tested-by: Raymes Khoury <raymes@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/3e44bc9eb923050b154ba64a813fe876646aef13/hterm/doc/hack.md
[modify] https://crrev.com/3e44bc9eb923050b154ba64a813fe876646aef13/nassh/manifest_v2.json
[modify] https://crrev.com/3e44bc9eb923050b154ba64a813fe876646aef13/hterm/js/hterm_terminal.js
[modify] https://crrev.com/3e44bc9eb923050b154ba64a813fe876646aef13/nassh/manifest_ext.json
[modify] https://crrev.com/3e44bc9eb923050b154ba64a813fe876646aef13/nassh/js/crosh.js
[modify] https://crrev.com/3e44bc9eb923050b154ba64a813fe876646aef13/hterm/concat/hterm.concat
[modify] https://crrev.com/3e44bc9eb923050b154ba64a813fe876646aef13/hterm/js/hterm.js
[modify] https://crrev.com/3e44bc9eb923050b154ba64a813fe876646aef13/hterm/js/hterm_scrollport.js
[add] https://crrev.com/3e44bc9eb923050b154ba64a813fe876646aef13/hterm/js/hterm_accessibility_reader.js
[modify] https://crrev.com/3e44bc9eb923050b154ba64a813fe876646aef13/nassh/js/nassh_main.js
[modify] https://crrev.com/3e44bc9eb923050b154ba64a813fe876646aef13/hterm/html/hterm_test.html
[modify] https://crrev.com/3e44bc9eb923050b154ba64a813fe876646aef13/nassh/manifest_v1.5.json
[add] https://crrev.com/3e44bc9eb923050b154ba64a813fe876646aef13/hterm/js/hterm_accessibility_reader_tests.js

Project Member

Comment 19 by bugdroid1@chromium.org, May 17 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/deecb0e4880276a9d7b9e0caea4ad16e847e03cd

commit deecb0e4880276a9d7b9e0caea4ad16e847e03cd
Author: Mike Frysinger <vapier@chromium.org>
Date: Thu May 17 04:11:22 2018

crosh_builtin: add more permissions

This syncs the set of permissions from Secure Shell that we want/use
in newer versions of hterm.

Bug:  822490 ,  646690 
Change-Id: I7abadb369b3178739808aa41fd0cae9ef3c0759e
Reviewed-on: https://chromium-review.googlesource.com/1061619
Reviewed-by: Raymes Khoury <raymes@chromium.org>
Reviewed-by: Achuith Bhandarkar <achuith@chromium.org>
Commit-Queue: Mike Frysinger <vapier@chromium.org>
Cr-Commit-Position: refs/heads/master@{#559417}
[modify] https://crrev.com/deecb0e4880276a9d7b9e0caea4ad16e847e03cd/chrome/browser/resources/chromeos/crosh_builtin/manifest.json

Project Member

Comment 20 by bugdroid1@chromium.org, May 29 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/apps/libapps/+/f1c61ba46a2bc07901b56333091f4763c0ac916d

commit f1c61ba46a2bc07901b56333091f4763c0ac916d
Author: Raymes Khoury <raymes@chromium.org>
Date: Tue May 29 00:08:18 2018

hterm: Improve accessibility command output

This improves accessibility command output in several ways:
1) Rather than using multiple elements for rendering the output to a
live region, instead we use a single <p> element which we manipulate the
aria-label attribute of. This is far more performant than the existing
solution. Because of this, it's no longer necessary to limit the amount
of output that we add per iteration which also simplifies logic.
2) As a result of the above change, it's necessary to introduce the
additional step of clearing the previous value of the aria-label prior
to setting its new value. This is necessary so that if the same string
is repeated multiple times, the screen reader will actually register it
as an attribute change. This adds an additional delay between iterations
however this can be optimized later if necessary.
3) Rather than inserting newlines between all strings passed to
announce() we instead only do this when there is actually a newline in
the terminal output. This avoids unwanted interruptions in text that
should appear together in the terminal.

Bug:  822490 ,  646690 
Change-Id: I024d13d24f126d31dae3fe0b2a0d8f818740354a
Reviewed-on: https://chromium-review.googlesource.com/1060716
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Tested-by: Raymes Khoury <raymes@chromium.org>

[modify] https://crrev.com/f1c61ba46a2bc07901b56333091f4763c0ac916d/hterm/js/hterm_terminal.js
[modify] https://crrev.com/f1c61ba46a2bc07901b56333091f4763c0ac916d/hterm/js/hterm_accessibility_reader.js
[modify] https://crrev.com/f1c61ba46a2bc07901b56333091f4763c0ac916d/hterm/js/hterm_accessibility_reader_tests.js

Project Member

Comment 21 by bugdroid1@chromium.org, Jun 6 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/apps/libapps/+/fa06b1df943fe777b8f0ab534ff7ad2cb77b104e

commit fa06b1df943fe777b8f0ab534ff7ad2cb77b104e
Author: Raymes Khoury <raymes@chromium.org>
Date: Wed Jun 06 06:44:26 2018

hterm: Preserve selection when scrolling collapsed selections

Currently we don't preserve collapsed selections when scrolling the
terminal (collapsed selections are those where the start/end of the
selection is the same point). Usually there would be no need, but it's
useful to do this for screen readers because they may use collapsed
selection to indicate the point they are navigated to. If this selection
isn't preserved, the screen reader will navigate to the parent element
when the row is removed.

This behavior isn't enabled by default because it can cause extra work
to be done during the ScrollPort.redraw_() function which is called
often. Instead it's only enabled when a screen reader is active.

Bug:  822490 ,  646690 
Change-Id: I74ca85a6cf87f7120ac04e2765a34f9e2a1a7da7
Reviewed-on: https://chromium-review.googlesource.com/1074573
Tested-by: Raymes Khoury <raymes@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/fa06b1df943fe777b8f0ab534ff7ad2cb77b104e/nassh/js/nassh_main.js
[modify] https://crrev.com/fa06b1df943fe777b8f0ab534ff7ad2cb77b104e/hterm/js/hterm_terminal.js
[modify] https://crrev.com/fa06b1df943fe777b8f0ab534ff7ad2cb77b104e/hterm/js/hterm_scrollport.js
[modify] https://crrev.com/fa06b1df943fe777b8f0ab534ff7ad2cb77b104e/nassh/js/crosh.js
[modify] https://crrev.com/fa06b1df943fe777b8f0ab534ff7ad2cb77b104e/hterm/js/hterm_scrollport_tests.js

Owner: raymes@chromium.org
Project Member

Comment 23 by bugdroid1@chromium.org, Jun 20 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/apps/libapps/+/6d9e2b1585c3f100315b677847c7f40429be6fec

commit 6d9e2b1585c3f100315b677847c7f40429be6fec
Author: Raymes Khoury <raymes@chromium.org>
Date: Wed Jun 20 06:51:49 2018

hterm: Don't print errors resulting from selecting page up/down buttons

Currently errors will be printed when page up/down are selected because
ScrollPort.Selection.sync only expects to be dealing with <x-row> nodes.
This adds a check to ignore these buttons.

Bug:  822490 ,  646690 
Change-Id: Ida699c5c475555f7616d1f748a459523c960564b
Reviewed-on: https://chromium-review.googlesource.com/1078127
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Tested-by: Raymes Khoury <raymes@chromium.org>

[modify] https://crrev.com/6d9e2b1585c3f100315b677847c7f40429be6fec/hterm/js/hterm_scrollport.js

Project Member

Comment 24 by bugdroid1@chromium.org, Jun 21 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/apps/libapps/+/4a04a0bf5f5accbc8d17b63b99379135ff7e89a0

commit 4a04a0bf5f5accbc8d17b63b99379135ff7e89a0
Author: Mike Frysinger <vapier@chromium.org>
Date: Thu Jun 21 23:05:56 2018

nassh: fix a11y crash in extension on non-CrOS systems

chrome.accessibilityFeatures.spokenFeedback is only available on CrOS,
so probe for that directly in addition to the other a11y objects.  This
fixes startup on macOS/Linux/Windows systems when using the extension.
The app works on those platforms because it's been whitelisted.

Bug:  822490 ,  646690 
Change-Id: Ie3752652bb688b70eec3704595da2256f7e74d8a
Reviewed-on: https://chromium-review.googlesource.com/1110707
Reviewed-by: Raymes Khoury <raymes@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/4a04a0bf5f5accbc8d17b63b99379135ff7e89a0/nassh/js/nassh_main.js
[modify] https://crrev.com/4a04a0bf5f5accbc8d17b63b99379135ff7e89a0/nassh/js/crosh.js

Project Member

Comment 25 by bugdroid1@chromium.org, Jun 25 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/apps/libapps/+/334625aa41dee698241c5dd745f58a672a8a8e0d

commit 334625aa41dee698241c5dd745f58a672a8a8e0d
Author: Raymes Khoury <raymes@chromium.org>
Date: Mon Jun 25 00:30:24 2018

hterm: Add skip-links to allow screen readers to scroll the terminal

This adds two buttons to the terminal, one at the top and one at the
bottom to scroll the terminal up/down respectively. These buttons are
not normally visible (being hidden off-screen). However when the buttons
are selected (e.g. by a screen reader) they will appear on-screen and
can be used to navigate through the history of the terminal.

The scroll up button appears immediately prior to the first on-screen
row of the terminal. Similarly the scroll down button appears
immediately after the last on-screen row. This means that when
navigating using a screen reader it will be the first thing that is hit
prior to navigating off the edge of the terminal.

Bug:  822490 ,  646690 
Change-Id: Ie71f3ef5051a1b05b63e01ead29ce52db8a31230
Reviewed-on: https://chromium-review.googlesource.com/1069953
Tested-by: Raymes Khoury <raymes@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/334625aa41dee698241c5dd745f58a672a8a8e0d/nassh/_locales/en/messages.json
[modify] https://crrev.com/334625aa41dee698241c5dd745f58a672a8a8e0d/hterm/js/hterm_terminal.js
[modify] https://crrev.com/334625aa41dee698241c5dd745f58a672a8a8e0d/hterm/js/hterm_scrollport.js
[modify] https://crrev.com/334625aa41dee698241c5dd745f58a672a8a8e0d/hterm/js/hterm_screen.js
[modify] https://crrev.com/334625aa41dee698241c5dd745f58a672a8a8e0d/hterm/js/hterm_scrollport_tests.js

Project Member

Comment 26 by bugdroid1@chromium.org, Jun 26 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/apps/libapps/+/177aec790ca76b51cce75a6b46d0a9a48f8d1610

commit 177aec790ca76b51cce75a6b46d0a9a48f8d1610
Author: Raymes Khoury <raymes@chromium.org>
Date: Tue Jun 26 00:59:44 2018

hterm: Announce the current screen content on Page up/down

This CL causes the current screen content to be announced as the
previous rows of content are paged through. It will interrupt existing
announcements when it is read out.

Bug:  822490 ,  646690 
Change-Id: I6ab9bf594b6ca8abc32666ccc2235dd82b5fb36f
Reviewed-on: https://chromium-review.googlesource.com/1078047
Tested-by: Raymes Khoury <raymes@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/177aec790ca76b51cce75a6b46d0a9a48f8d1610/hterm/js/hterm_scrollport.js
[modify] https://crrev.com/177aec790ca76b51cce75a6b46d0a9a48f8d1610/hterm/js/hterm_accessibility_reader.js
[modify] https://crrev.com/177aec790ca76b51cce75a6b46d0a9a48f8d1610/nassh/_locales/en/messages.json
[modify] https://crrev.com/177aec790ca76b51cce75a6b46d0a9a48f8d1610/hterm/js/hterm_terminal.js
[modify] https://crrev.com/177aec790ca76b51cce75a6b46d0a9a48f8d1610/hterm/js/hterm_accessibility_reader_tests.js
[modify] https://crrev.com/177aec790ca76b51cce75a6b46d0a9a48f8d1610/hterm/js/hterm_scrollport_tests.js

Project Member

Comment 27 by bugdroid1@chromium.org, Jun 26 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/apps/libapps/+/69366b80fb8abb3aebd89c37114db2350266ff4b

commit 69366b80fb8abb3aebd89c37114db2350266ff4b
Author: Raymes Khoury <raymes@chromium.org>
Date: Tue Jun 26 03:08:35 2018

hterm/nassh: Make it possible to enable/disable accessibility at runtime

Currently we will decide whether to turn on some accessibility features
in nassh and crosh at the startup of the terminal. This means that if
the screen reader is then subsequently enabled/disabled we won't respect
the user's setting and (for example) some accessibility features won't
work until the app is restarted. This change adds a listener to detect
when the setting changes and turn those features on or off accordingly.
Tests have also been added for these cases.

Bug:  822490 ,  646690 
Change-Id: I3e91ed78b8c2c45339c029b6013553647956be82
Reviewed-on: https://chromium-review.googlesource.com/1086675
Tested-by: Raymes Khoury <raymes@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/69366b80fb8abb3aebd89c37114db2350266ff4b/nassh/js/nassh_main.js
[modify] https://crrev.com/69366b80fb8abb3aebd89c37114db2350266ff4b/nassh/js/crosh.js
[modify] https://crrev.com/69366b80fb8abb3aebd89c37114db2350266ff4b/hterm/js/hterm_accessibility_reader.js
[modify] https://crrev.com/69366b80fb8abb3aebd89c37114db2350266ff4b/hterm/js/hterm_accessibility_reader_tests.js

Project Member

Comment 28 by bugdroid1@chromium.org, Jun 27 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/apps/libapps/+/4e9c92ef50ac0f6b4e0f383be75a83c899af7359

commit 4e9c92ef50ac0f6b4e0f383be75a83c899af7359
Author: Raymes Khoury <raymes@chromium.org>
Date: Wed Jun 27 03:26:21 2018

hterm: Fix an assertion failure in hterm.AccessibilityReader

Calling AccessibilityReader.newLine() twice before calling announce
would cause an exception to be thrown because we generally expect the
nextReadTimer_ to be started after something is announced. In the case
of newlines we don't want to do that. So we simply ignore newlines when
there is nothing already in the queue.

Bug:  822490 ,  646690 
Change-Id: Ie5004246b9d22a3b12a69467273745144ffc3375
Reviewed-on: https://chromium-review.googlesource.com/1116203
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Tested-by: Raymes Khoury <raymes@chromium.org>

[modify] https://crrev.com/4e9c92ef50ac0f6b4e0f383be75a83c899af7359/hterm/js/hterm_accessibility_reader.js
[modify] https://crrev.com/4e9c92ef50ac0f6b4e0f383be75a83c899af7359/hterm/js/hterm_accessibility_reader_tests.js

Project Member

Comment 29 by bugdroid1@chromium.org, Jun 28 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/apps/libapps/+/5013d96cd60a44de5144bbb523cb24648b1878ea

commit 5013d96cd60a44de5144bbb523cb24648b1878ea
Author: Raymes Khoury <raymes@chromium.org>
Date: Thu Jun 28 06:11:18 2018

hterm: Don't announce page up/down at the top/bottom of the screen

Currently paging up/down when at the top/bottom of the screen can cause
a page up/down notification to be announced by the screen reader. This
change detects that case and avoids the announcement.

Bug:  822490 ,  646690 
Change-Id: I5a6c2da6c77f84e5b1f636a9e21da2273bce9084
Reviewed-on: https://chromium-review.googlesource.com/1117730
Tested-by: Raymes Khoury <raymes@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/5013d96cd60a44de5144bbb523cb24648b1878ea/hterm/js/hterm_scrollport.js
[modify] https://crrev.com/5013d96cd60a44de5144bbb523cb24648b1878ea/hterm/js/hterm_scrollport_tests.js

Project Member

Comment 30 by bugdroid1@chromium.org, Jun 28 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/apps/libapps/+/7466ad240f52b4efaf076bd14842d3b045e846ad

commit 7466ad240f52b4efaf076bd14842d3b045e846ad
Author: Raymes Khoury <raymes@chromium.org>
Date: Thu Jun 28 06:47:59 2018

hterm: Fix announcement of current scrolled percentage

Currently the scroll percentage can be computed to NaN when the number
of rows onscreen is the same as the total number of rows. This fixes
the divide by zero issue.

Bug:  822490 ,  646690 
Change-Id: I65afb5cbf6271be5d6b6772bf2afe8f2711e1013
Reviewed-on: https://chromium-review.googlesource.com/1116212
Tested-by: Raymes Khoury <raymes@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/7466ad240f52b4efaf076bd14842d3b045e846ad/hterm/js/hterm_scrollport.js
[modify] https://crrev.com/7466ad240f52b4efaf076bd14842d3b045e846ad/hterm/js/hterm_scrollport_tests.js

Project Member

Comment 31 by bugdroid1@chromium.org, Jul 4

The following revision refers to this bug:
  https://chromium.googlesource.com/apps/libapps/+/c7a0638ecc27b57a9b7697f61407101f18bcdd6d

commit c7a0638ecc27b57a9b7697f61407101f18bcdd6d
Author: Raymes Khoury <raymes@chromium.org>
Date: Wed Jul 04 00:44:58 2018

hterm: Announce messages shown by showOverlay

This adds accessibiliy announcements for messages shown by
Terminal.showOverlay, for example the announcement for copying text.

A test is added. Relatedly, these announcements should be voiced even
when full accessibility support isn't enabled as there is no performance
cost to doing so. This removes the check which prevented those
announcements being voiced.

Bug:  822490 ,  646690 
Change-Id: I6999f2c811534f081c465d93b3ee7baf971633c4
Reviewed-on: https://chromium-review.googlesource.com/1117729
Tested-by: Raymes Khoury <raymes@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/c7a0638ecc27b57a9b7697f61407101f18bcdd6d/hterm/js/hterm_terminal_tests.js
[modify] https://crrev.com/c7a0638ecc27b57a9b7697f61407101f18bcdd6d/hterm/js/hterm_scrollport.js
[modify] https://crrev.com/c7a0638ecc27b57a9b7697f61407101f18bcdd6d/hterm/js/hterm_accessibility_reader.js
[modify] https://crrev.com/c7a0638ecc27b57a9b7697f61407101f18bcdd6d/hterm/js/hterm_terminal.js
[modify] https://crrev.com/c7a0638ecc27b57a9b7697f61407101f18bcdd6d/hterm/js/hterm_accessibility_reader_tests.js
[modify] https://crrev.com/c7a0638ecc27b57a9b7697f61407101f18bcdd6d/hterm/js/hterm_scrollport_tests.js

Project Member

Comment 32 by bugdroid1@chromium.org, Jul 10

The following revision refers to this bug:
  https://chromium.googlesource.com/apps/libapps/+/45eaf92a0390452fb448d96fff9acec2d575012a

commit 45eaf92a0390452fb448d96fff9acec2d575012a
Author: Raymes Khoury <raymes@chromium.org>
Date: Tue Jul 10 03:09:14 2018

hterm: Change <x-screen> to have a role="log"

This change is a compromise: in some ways the terminal behaves like a
text box but not in all ways. It is not editable in the same ways a text
box is editable and the content we want to be read out by a screen
reader does not always align with the edits (selection changes) that
happen in the terminal window. We change the role to a log so that the
screen reader doesn't treat it like a text box and announce all
selection changes. The announcements that we want spoken are generated
by the live region, which gives more control over what will be spoken.

Bug:  822490 ,  646690 
Change-Id: I5e0cff1415916c2efb7d44300084fe808e33b71d
Reviewed-on: https://chromium-review.googlesource.com/1121941
Tested-by: Raymes Khoury <raymes@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/45eaf92a0390452fb448d96fff9acec2d575012a/hterm/js/hterm_scrollport.js

Project Member

Comment 33 by bugdroid1@chromium.org, Jul 11

The following revision refers to this bug:
  https://chromium.googlesource.com/apps/libapps/+/7eff29f2b39006708b9a90e706eda7091f6379e0

commit 7eff29f2b39006708b9a90e706eda7091f6379e0
Author: Raymes Khoury <raymes@chromium.org>
Date: Wed Jul 11 01:22:07 2018

hterm: Simplify handling of duplicate live output

This changes how duplicate live output is handled. Previously we would
wait for a delay, clear the live region, wait for a delay again and then
update it with the new text to output. This is necessary so that the
cleared live region registers as an attribute change with the screen
reader and then the duplicate text is also registered as an attribute
change.

Instead of doing this we can simply check if the previous text in the
live region matches the current text we're about to display. If so we
slightly change the new text so that an attribute change will be
registered.

This approach is also added for the assertive live region so that it
will properly handle duplicates properly. Tests are.

Bug:  822490 ,  646690 
Change-Id: I1a75c55315abd14314960248ad0c695ccb9102c1
Reviewed-on: https://chromium-review.googlesource.com/1117726
Tested-by: Raymes Khoury <raymes@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/7eff29f2b39006708b9a90e706eda7091f6379e0/hterm/js/hterm_accessibility_reader.js
[modify] https://crrev.com/7eff29f2b39006708b9a90e706eda7091f6379e0/hterm/js/hterm_accessibility_reader_tests.js

 Issue 822502  has been merged into this issue.
Project Member

Comment 35 by bugdroid1@chromium.org, Jul 12

The following revision refers to this bug:
  https://chromium.googlesource.com/apps/libapps/+/b199d4d2c7e13f1972d39232b8da996f6ae58bb7

commit b199d4d2c7e13f1972d39232b8da996f6ae58bb7
Author: Raymes Khoury <raymes@chromium.org>
Date: Thu Jul 12 05:11:21 2018

hterm: Handle announcements of selection changes

We currently don't make any announcements for text that isn't contained
in the terminal output. For example if a character is deleted or the
user moves the cursor over previously edited text. This CL adds code to
announce these kinds of changes. This is done by listening for cursor
changes and heuristically trying to detect what kind of a change
happened based on the location of the cursor, changes to the row of
text that the cursor is on and whether a key event has been recently
received.

Note that a heuristic is the best that can be done here because we
can't distinguish input from output in terminal and so can't know which
changes to the cursor/terminal are caused by the user or the
application. Also, some announcements would be inappropriate in certain
terminal applications/contexts. A best-effort approach must be taken
where we handle the common cases.

Change-Id: Ic71df5bcc67583caecd163620e287eb1dfa29ce8
Bug:  822490 ,  646690 
Reviewed-on: https://chromium-review.googlesource.com/1121901
Tested-by: Raymes Khoury <raymes@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/b199d4d2c7e13f1972d39232b8da996f6ae58bb7/hterm/js/hterm_terminal.js
[modify] https://crrev.com/b199d4d2c7e13f1972d39232b8da996f6ae58bb7/hterm/js/hterm_accessibility_reader.js
[modify] https://crrev.com/b199d4d2c7e13f1972d39232b8da996f6ae58bb7/hterm/js/hterm_accessibility_reader_tests.js

Project Member

Comment 36 by bugdroid1@chromium.org, Jul 12

The following revision refers to this bug:
  https://chromium.googlesource.com/apps/libapps/+/9412816072aa0a196f0539190fcc66a266935315

commit 9412816072aa0a196f0539190fcc66a266935315
Author: Raymes Khoury <raymes@chromium.org>
Date: Thu Jul 12 05:17:30 2018

hterm: Add announcements of the Space character

Currently ChromeVox won't announce individual space characters when
they are added to a live region. They are only spoken when selection
changes occur. This adds support for speaking space characers.

This isn't an ideal solution (ideally we would rely on the screen reader
to announce the character). For now we only handle the U+0020 as a space
and not other space variations. This can be extended as needed.

Bug:  822490 ,  646690 
Change-Id: I3c14d1c7a8d2a573adde21dba603bad384208cea
Reviewed-on: https://chromium-review.googlesource.com/1125655
Tested-by: Raymes Khoury <raymes@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/9412816072aa0a196f0539190fcc66a266935315/nassh/_locales/en/messages.json
[modify] https://crrev.com/9412816072aa0a196f0539190fcc66a266935315/hterm/js/hterm_accessibility_reader.js
[modify] https://crrev.com/9412816072aa0a196f0539190fcc66a266935315/hterm/js/hterm_accessibility_reader_tests.js

Project Member

Comment 37 by bugdroid1@chromium.org, Jul 16

The following revision refers to this bug:
  https://chromium.googlesource.com/apps/libapps/+/13f32ec66d76e508cf39875a2845d5288b11970a

commit 13f32ec66d76e508cf39875a2845d5288b11970a
Author: Raymes Khoury <raymes@chromium.org>
Date: Mon Jul 16 23:36:29 2018

hterm: Remove invalid page-down-all-rows-visible test

This test is no longer valid because scrolling up and down when at the
top or bottom of the scrollport shouldn't do anything. This behavior is
already tested in page-up-down-dont-announce.

Bug:  822490 ,  646690 
Change-Id: Ib6886af6cdd4a649dda236f84b11d3dcbcccff2b
Reviewed-on: https://chromium-review.googlesource.com/1137969
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Tested-by: Raymes Khoury <raymes@chromium.org>

[modify] https://crrev.com/13f32ec66d76e508cf39875a2845d5288b11970a/hterm/js/hterm_scrollport_tests.js

Project Member

Comment 38 by bugdroid1@chromium.org, Jul 17

The following revision refers to this bug:
  https://chromium.googlesource.com/apps/libapps/+/15697f4f3af97e45a8055f5f31c94f1b94d38d73

commit 15697f4f3af97e45a8055f5f31c94f1b94d38d73
Author: Raymes Khoury <raymes@chromium.org>
Date: Tue Jul 17 04:19:41 2018

hterm: Update the selection when accessibility is enabled even if it's collapsed

This updates the html cursor (selection) to be in the same location
as the terminal cursor. We generally only do this if the selection
is collapsed to prevent clearing a selection that a user has set.
However when a screen reader is enabled, it is expected that
input should cause the selection to move to the cursor. So in this
case we always update selection, even if it's collapsed.

Bug:  822490 ,  646690 
Change-Id: I455b29e18dbdc0365baef0f14aad9400bd348c79
Reviewed-on: https://chromium-review.googlesource.com/1133604
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Tested-by: Raymes Khoury <raymes@chromium.org>

[modify] https://crrev.com/15697f4f3af97e45a8055f5f31c94f1b94d38d73/hterm/js/hterm_terminal.js
[modify] https://crrev.com/15697f4f3af97e45a8055f5f31c94f1b94d38d73/hterm/js/hterm_terminal_tests.js
[modify] https://crrev.com/15697f4f3af97e45a8055f5f31c94f1b94d38d73/hterm/js/hterm_accessibility_reader.js
[modify] https://crrev.com/15697f4f3af97e45a8055f5f31c94f1b94d38d73/hterm/js/hterm_accessibility_reader_tests.js

Project Member

Comment 39 by bugdroid1@chromium.org, Jul 17

The following revision refers to this bug:
  https://chromium.googlesource.com/apps/libapps/+/e790d62a598d79e5e20d4caea3dbea48075bab7e

commit e790d62a598d79e5e20d4caea3dbea48075bab7e
Author: Raymes Khoury <raymes@chromium.org>
Date: Tue Jul 17 04:37:08 2018

hterm: Mark offscreen rows that hold selection as hidden

When text in the terminal is selected and then scrolled offscreen, the
x-rows that are selected are not removed from the DOM. This is done so
that the selection is maintained even when scrolling. Instead they are
just moved to be offscreen. Because these rows are in the DOM, the screen
reader will still navigate to them. This CL marks those nodes as
visibility:hidden so that they don't show up for the screen reader.

Bug:  822490 ,  646690 
Change-Id: I6ef9581f751fc7107a0e40608e0ed6e886549da6
Reviewed-on: https://chromium-review.googlesource.com/1080365
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Tested-by: Raymes Khoury <raymes@chromium.org>

[modify] https://crrev.com/e790d62a598d79e5e20d4caea3dbea48075bab7e/hterm/js/hterm_scrollport.js
[modify] https://crrev.com/e790d62a598d79e5e20d4caea3dbea48075bab7e/hterm/js/hterm_scrollport_tests.js

Project Member

Comment 40 by bugdroid1@chromium.org, Jul 19

The following revision refers to this bug:
  https://chromium.googlesource.com/apps/libapps/+/747d167415b093bd7dcd88bcb1382ef4ec818f7d

commit 747d167415b093bd7dcd88bcb1382ef4ec818f7d
Author: Raymes Khoury <raymes@chromium.org>
Date: Thu Jul 19 05:17:27 2018

hterm: Only announce a space character if it is the character actually printed to the screen

Currently the heuristic for announcing space characters just examines
the last character behind the cursor. But when doing things like a bash
reverse search, the cursor will move relative to a space character
without a space character being entered and space will be announced.

This adds a requirement for a space character to be announced only if
it is printed to the screen.

Bug:  822490 ,  646690 
Change-Id: I6c2a6c7ebd5edcb12af4bc6d11b89b52eac52be6
Reviewed-on: https://chromium-review.googlesource.com/1139839
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Tested-by: Raymes Khoury <raymes@chromium.org>

[modify] https://crrev.com/747d167415b093bd7dcd88bcb1382ef4ec818f7d/hterm/js/hterm_accessibility_reader.js
[modify] https://crrev.com/747d167415b093bd7dcd88bcb1382ef4ec818f7d/hterm/js/hterm_accessibility_reader_tests.js

Status: Fixed (was: Assigned)
This feature is basically working now. Marking fixed.

Sign in to add a comment