New issue
Advanced search Search tips

Issue 747625 link

Starred by 0 users

Issue metadata

Status: Fixed
Owner:
Closed: Jul 2017
Components:
EstimatedDays: ----
NextAction: ----
OS: All
Pri: 3
Type: Feature



Sign in to add a comment

hterm: implement DOCS support (ESC %)

Project Member Reported by vapier@chromium.org, Jul 21 2017

Issue description

there's this todo in the code:
/**
 * 'ESC %' sequences, character set control.  Not currently implemented.
 *
 * To be implemented (currently ignored):
 *   ESC % @ - Set ISO 8859-1 character set.
 *   ESC % G - Set UTF-8 character set.
 *
 * All other ESC % sequences are echoed to the terminal.
 *
 * TODO(rginda): Implement.
 */

ECMA-35/ISO-2022 calls this Designate Other Coding System (DOCS)

mosh mentions this in their tech page:
  https://mosh.org/#techinfo
  Only Mosh will never get stuck in hieroglyphs when a nasty program writes to the terminal.

it also refers to Markus Kuhn's page:
  http://www.cl.cam.ac.uk/~mgk25/unicode.html#term

basically it advocates for terminals in UTF-8 mode (not ECMA-35/ISO-2022 mode) not supporting any graphics map at all.

iTerm2 has this discussion in its code:
  https://github.com/gnachman/iTerm2/blob/3.0.16/sources/VT100Terminal.m#L1518

which refers to these issues:
  https://gitlab.com/gnachman/iterm2/issues/1997
  https://gitlab.com/gnachman/iterm2/issues/2369
  https://gitlab.com/gnachman/iterm2/issues/2360

my take is that we should go the route of mosh & Markus's suggestions.  namely, if the remote side requests UTF-8 encoding, we don't support character maps.  it's hard to argue that people requesting UTF-8 encoding *and* support for character maps encoding translations should be supported.

this also makes the terminal much more robust against binary data switching to the graphics map by accident (ala  issue 735748 ).
 
Project Member

Comment 1 by bugdroid1@chromium.org, Jul 25 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/apps/libapps/+/eda9e69db337ac57b6de2ebbc9f215b10a79441a

commit eda9e69db337ac57b6de2ebbc9f215b10a79441a
Author: Mike Frysinger <vapier@chromium.org>
Date: Tue Jul 25 19:10:27 2017

hterm: add support for DOCS for transitioning to UTF-8

This lets people disable support for character maps at runtime.  This
makes the terminal more robust against (accidental) binary data changing
to graphics mode.

BUG= chromium:747625 

Change-Id: I703d42bdc44a1d0b2baa6fff58606ed833ddb1c5
Reviewed-on: https://chromium-review.googlesource.com/583588
Tested-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Brandon Gilmore <varz@google.com>

[modify] https://crrev.com/eda9e69db337ac57b6de2ebbc9f215b10a79441a/hterm/js/hterm_vt_tests.js
[modify] https://crrev.com/eda9e69db337ac57b6de2ebbc9f215b10a79441a/hterm/doc/ControlSequences.md
[modify] https://crrev.com/eda9e69db337ac57b6de2ebbc9f215b10a79441a/hterm/js/hterm_vt.js

Comment 2 by vapier@chromium.org, Jul 25 2017

Owner: vapier@chromium.org
Status: Fixed (was: Available)
this will be part of hterm-1.68+ & nassh-0.8.36.9+
Project Member

Comment 3 by bugdroid1@chromium.org, Aug 3 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/apps/libapps/+/a7768920c29a565c6ebdbaa804f395655d0e3f83

commit a7768920c29a565c6ebdbaa804f395655d0e3f83
Author: Mike Frysinger <vapier@chromium.org>
Date: Thu Aug 03 17:27:42 2017

hterm: add a pref for default terminal encoding

This allows for easy experimentation with terminal encoding and is a
step to setting the default encoding to utf8.  For now, the default is
unchanged (iso-2022/ecma-35) which means character map (graphics maps)
are available.  In the future, if all goes well, we'll default to utf8
and see if anyone notices, and if they do, they'll be able to back out
in the meantime.

BUG= chromium:747625 

Change-Id: Ia63542c9a97b18ec09abeb954dc2de75d682d92f
Reviewed-on: https://chromium-review.googlesource.com/591919
Reviewed-by: Brandon Gilmore <varz@google.com>
Tested-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/a7768920c29a565c6ebdbaa804f395655d0e3f83/hterm/js/hterm_preference_manager.js
[modify] https://crrev.com/a7768920c29a565c6ebdbaa804f395655d0e3f83/hterm/js/hterm_terminal.js

Project Member

Comment 4 by bugdroid1@chromium.org, Aug 17 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/apps/libapps/+/a1371e13de54fd7ec755024e51cd2351549e900d

commit a1371e13de54fd7ec755024e51cd2351549e900d
Author: Mike Frysinger <vapier@chromium.org>
Date: Thu Aug 17 22:41:19 2017

hterm: formalize encoding management a bit

Move the encoding state behind a helper function.  By itself, this
isn't terribly useful, but a following commit will leverage the new
function to update more internal runtime state.

BUG= chromium:747625 

Change-Id: Ic2b360bb53834492daa8ef4243fcc325dbbebfb9
Reviewed-on: https://chromium-review.googlesource.com/618546
Tested-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Brandon Gilmore <varz@google.com>

[modify] https://crrev.com/a1371e13de54fd7ec755024e51cd2351549e900d/hterm/js/hterm_terminal.js
[modify] https://crrev.com/a1371e13de54fd7ec755024e51cd2351549e900d/hterm/js/hterm_vt_tests.js
[modify] https://crrev.com/a1371e13de54fd7ec755024e51cd2351549e900d/hterm/js/hterm_vt.js

Project Member

Comment 5 by bugdroid1@chromium.org, Aug 17 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/apps/libapps/+/d9f524adcb0c72ce736cd773401dec93dcabfec7

commit d9f524adcb0c72ce736cd773401dec93dcabfec7
Author: Mike Frysinger <vapier@chromium.org>
Date: Thu Aug 17 22:42:22 2017

hterm: stop matching 8-bit control codes in utf8 mode

We currently search for 8-bit control codes in utf8 data streams.
This doesn't make sense as valid utf8 encoding won't match any of
the 8-bit escape sequences.  They're also not terribly useful as
every sequence has a standard 7-bit form that programs use.  To
speed up processing, and avoid breaking up utf8 codepoints, drop
the 8-bit matching when we're in utf8 mode.

BUG= chromium:747625 

Change-Id: Iae53c7bdb3b281c723c7b41d1e1284111386c9c4
Reviewed-on: https://chromium-review.googlesource.com/618547
Reviewed-by: Brandon Gilmore <varz@google.com>
Tested-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/d9f524adcb0c72ce736cd773401dec93dcabfec7/hterm/js/hterm_vt.js

Project Member

Comment 6 by bugdroid1@chromium.org, Nov 22 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/apps/libapps/+/73a684824f340336358d6853d5172d27fde11dfb

commit 73a684824f340336358d6853d5172d27fde11dfb
Author: Mike Frysinger <vapier@chromium.org>
Date: Wed Nov 22 05:43:46 2017

hterm: change default terminal encoding to utf8

Change the default terminal encoding to utf8.  Practically speaking,
this means graphics maps won't work by default.  Unicode aware progs
should be using UTF-8 anyways instead of changing the character map,
but it's not entirely clear how many still exist.  Lets find out!

BUG= chromium:747625 

Change-Id: I593de701721293bbb85731d3e6b4c82cae989a2b
Reviewed-on: https://chromium-review.googlesource.com/591920
Reviewed-by: Brandon Gilmore <varz@google.com>
Tested-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/73a684824f340336358d6853d5172d27fde11dfb/hterm/js/hterm_preference_manager.js

Project Member

Comment 7 by bugdroid1@chromium.org, Nov 27 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/apps/libapps/+/793bd8533e861dc1b9e618d335e38ac125185c36

commit 793bd8533e861dc1b9e618d335e38ac125185c36
Author: Mike Frysinger <vapier@chromium.org>
Date: Mon Nov 27 20:54:27 2017

hterm: fix tests after default encoding change

Some of these tests use graphic character maps, so make sure we set
the encoding back to iso-2022 regardless of the default (which we
have changed to utf-8).

BUG= chromium:747625 

Change-Id: I93812dcfe29c918819f3edbd6bcd5a1b25b608ec
Reviewed-on: https://chromium-review.googlesource.com/789437
Reviewed-by: Brandon Gilmore <varz@google.com>
Tested-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/793bd8533e861dc1b9e618d335e38ac125185c36/hterm/js/hterm_vt_canned_tests.js

Sign in to add a comment