New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.
Starred by 54 users

Issue metadata

Status: Assigned
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 2
Type: Bug

Blocked on:
issue 866189

Show other hotlists

Hotlists containing this issue:

Sign in to add a comment

Issue 826614: Input Method support / integration with Linux app

Reported by, Mar 28 2018 Project Member

Issue description

Chrome version: 67.0.3376.0 (Official Build) dev (64-bit)
OS: Chrome

Repro steps:
1. set up crostini, install VSCode
2. start VSCode
3. switch input language to Japanese
4. type 'a' in VSCode window

- start the normal text input flow (for Japanese) within the VSCode

- simply 'a' appears in the app, input language is ignored

Comment 1 by, Mar 28 2018

i guess the implication is that you're expecting the existing CrOS IME to be used ?  and, at least by default, not use an IME that the container has installed (e.g. anthy) ?  that's going to be tricky i think.

Comment 2 by, Mar 28 2018

yes, use of ChromeOS IME on linux apps in the container, and yes indeed it should be very tricky. However at least that works on Android apps in ARC++. nona@ and yukawa@ should know how ARC++ interacts with ChromeOS IMEs.

Installing linux IMEs into the container could be a choice, however an IME needs a bit more UI things which also complicates things on crostini apps, like current IME indicator (typically resides on a system tray on the linux desktop), global keyboard shortcut of IME switching, some additional windows like candidate windows.

An idea is to develop a "fake IME" in the container which implements something like gtkimmodule or fcitx and communicates with ChromeOS IME.

Comment 3 by, Mar 30 2018

FYI: For ARC++, we have a proxy of IME events in components/arc/ime. All IME events (for example, insertText) are sent to container via this proxy and the fake IME installed in container side handles the events.

Comment 4 by, Apr 10 2018

Status: Untriaged (was: Available)

Comment 5 by, Apr 23 2018

 Issue 829166  has been merged into this issue.

Comment 6 by, Apr 23 2018

Labels: -Pri-2 M-68 Pri-1
Status: Assigned (was: Untriaged)
+some crostini-ui folks

yukawa, nona -- we should meet up to discuss how to make IMEs work with Crostini.

Comment 7 by, Apr 23 2018

re #6: Sorry, correction; I've heard that yhanada@ and kinaba@ are responsible for IME support of ARC++ apps.

Comment 8 by, Apr 24 2018

+jkardatzke, reveman -- it sounds like there might be a way to do this over Wayland?

Comment 9 by, Apr 24 2018

Yes, we need to implement the upstream input method interface in chrome ( kinaba@ looked at this as part of implementing input methods for arc++ and would be the ideal person to work on crostini support.

Comment 10 by, Apr 24 2018

It's possible to go through wayland, and actually there is already an unstable input-method protocol definition, but I think we may want to extend the API if we want the full feature (related to ARC++ protocol).

I've been writing some design doc / notes internally, I'll share it with you all.

Comment 11 by, Apr 24 2018

For record, what I chatted with David a few weeks ago were:
* The structure/the conceptual design of Chrome IMF and the wayland protocol is basically similar. So the basic set of IME features should roughly map each other.
* There may be tough things in details. One thing is that Wayland uses UTF-8/byte-index and Chrome uses UTC-16/codepoint-index. Translating IME request like ("delete one char before the caret" may not simply work, because we don't know how many bytes to erase.)

What I actually don't know is the Crostini or wayland-client side.
Does the apps running inside is expected to implement the client-side IME protocol and adding the host-side implementation will make things work?
Or do we have to add some client IME service like fcitx or ibus to make the apps work?

Comment 12 by, Apr 24 2018

Yes, it's expected that toolkits like GTK and QT implement support for that protocol. Not sure what the story is for Xwayland. We might need some XIM ( bridge for that to work.

Comment 13 by, Apr 24 2018

Ignore my previous comment. mukai@ has been investigating this and created a proposal:

Comment 14 by, May 10 2018

Labels: Hotlist-Crostini-UI

Comment 15 by, May 23 2018

Labels: -Restrict-View-Google

Comment 16 by, Jun 29 2018

Started the implementation based on the document in #c13

Comment 17 by, Jul 18 2018

Project Member
The following revision refers to this bug:

commit ed95ce6013afb44be94d45167c080d12e597a75e
Author: Jun Mukai <>
Date: Wed Jul 18 00:30:35 2018

vm_tools: sommelier: add text_input_unstable_v1 protocol

This is one of CLs for the integration between crostini apps and
ChromeOS input methods. This should work with the text-input server
within Chrome (
and text-input clients within Linux apps. Plugins for such
Linux app are not yet publicly available -- will come in
other CLs.

TEST=manually -- run with server in Chrome and Linux app with special plugin
Change-Id: I510ba7ee688001db529f6ea90522ba0ca7c59932
Commit-Ready: Jun Mukai <>
Tested-by: Jun Mukai <>
Reviewed-by: David Reveman <>


Comment 18 by, Jul 20 2018

Project Member
The following revision refers to this bug:

commit a446ee7f1fc1e9560d6d5289454ae7bc9e0c723e
Author: Jun Mukai <>
Date: Fri Jul 20 22:55:15 2018

initial support of text-input-unstable protocol

This CL introduces the support of text-input-unstable protocol
defined in wayland-protocols repository.

This CL will be the first one for a series of CLs for supporting
this protocol and lacks some parts surrounding text.

TEST=manually, exo_unittests

Change-Id: I4f4e335ed26c51beea9f25479acc63e11f09f777
Commit-Queue: Jun Mukai <>
Reviewed-by: Antoine Labour <>
Reviewed-by: Mitsuru Oshima <>
Reviewed-by: Yuichiro Hanada <>
Cr-Commit-Position: refs/heads/master@{#577022}

Comment 19 by, Jul 20 2018

Blockedon: 866189

Comment 20 by, Jul 21 2018

Will it be possible to switch to another language in the linux app via the linux IME such as ibus? This will be helpful for languages that are not available in the chrome OS IME.

Currently, the shortcuts for switching via ibus do not appear to work - they are not passed through to the app. However, they can be manually switched via `ibus engine` command. Would be nice to be able to switch input methods within linux using the keyboard shortcuts as well.

Comment 21 by, Jul 23 2018

#20, that is not a goal of this effort, sorry. So far the goal is to make ChromeOS IMEs work with Linux apps.

Also I think the existing Linux IME frameworks like IBus are already working, at least at some level. If some features (like IME switching) isn't in IBus, I am afraid that's a bug of IBus, not ChromeOS.

Comment 22 by, Jul 23 2018

#21, thanks for the comment. I understand the distinction, I just wasn't sure how the ChromeOS IME interacts with the underlying Linux IME. I can get IBus to switch from the CLI, but what I haven't been able to figure out is how to switch via the keyboard shortcuts to the linux IME. It appears that the shortcuts are being "grabbed" by ChromeOS, though they don't appear to be reserved in the Ash accelerator table.

Comment 23 by, Jul 23 2018

My current plan is a separate Gtk ImModule or an XIM server. This means you'll set like "export GTK_IM_MODULE=cros", and then the app will connect to the chromeOS IME instead of entities like IBus. With this scheme, shortcuts like IME switching will be handled at ChromeOS level, and that would be fine.

However, always stealing IME shortcuts isn't a good idea, as you pointed out. And I don't think that "grabbing" isn't the current behavior -- the IME switching shortcuts will come to the linux environment, I think.

By the way, what is the keyboard shortcut exactly you are referring?  I guess it's Ctrl-space or Alt-space?

Comment 24 by, Jul 23 2018

Labels: -M-68 M-70
Maybe this isn't for M-68.

Comment 25 by, Jul 23 2018

You can set the shortcut to anything you like, I have tried a whole bunch of combinations to see if any of them got through; nothing I tried worked. The default for iBus is ctrl-space.

Comment 26 by, Jul 23 2018

I was asking since ctrl-space is also the IME switching shortcut key for ChromeOS, which could be intercepted by ChromeOS systems (although it shouldn't be). Things like alt-space wouldn't be used by ChromeOS IIRC, thus it should come to the Linux app as usual, so it's a mystery why it doesn't trigger IBus features -- let's track that on a separate issue. I filed issue 866681 for this; feel free to fill more details there. Thanks!

Comment 27 by, Jul 23 2018

Great, thanks for the help!

Comment 28 by, Oct 8

 Issue 835718  has been merged into this issue.

Comment 29 by, Nov 30

I found some behavior that related with this issue it would help you fix easier. In a terminal, I can type another language but I have to minimize Terminal then switch layout and back to Terminal to type again but in other applications (e.g., Vscode) are not support this behavior, Even I go out to Chrome OS to switch language and back to VScode my keyboard language is still English

Comment 30 by, Nov 30

I don't fully understand how input events work these days, but I believe there is a difference between X11 and Wayland applications. And that's quite possibly what you're seeing here

Comment 31 by, Nov 30

Sorry for the slow progress here. Basically, the Terminal is quite different from other Linux apps, from the text-input perspective. Normal Linux apps will communicate with ChromeOS window system through Wayland protocol, while Terminal isn't.

I've brought a wayland protocol support (zwp-text-input-v1) already (see, so the Linux environment should be able to see this. The remaining part is the Linux plugins (i.e. GtkImModule or XIM) implementing this protocol. is the initial CL for GtkImModule implementation, left work-in-progress for months but I've been busy on other things lately. I'll revisit soon.

Note that this GtkImModule is specific to Gtk3 and so not working with apps like VSCode unfortunately;  VSCode is on Gtk2, which lacks support of wayland directly. Same with Android Studio.

Comment 32 Deleted

Comment 33 by, Dec 1

Do you have any plan to make input method support Gtk2 or wait for Vscode update to gtk3? I'm very focused on this issue because of this in the last point for why I do not change to Chrome OS

Comment 34 by, Dec 1

Yes, I'll work on Gtk2 or XIM later, it was Gtk3 specific because it's simpler. Yet those supports need extra efforts and time, I can't promise when it's going to be done. Helps and contributions would be much appreciated.

Comment 35 by, Dec 1

Understand, I don't want a promise I only want to know your plan. I love to help, but I'm a web developer. My C++ skill of mine is zero. So sorry

Comment 36 by, Dec 1

Would this bug also cover support for on-screen keyboard input for Linux applications? As it stands, there appears to be no practical way to input text on tablet devices (or when devices are in tablet mode). Even forcing the on screen keyboard to open using accessibility settings doesn't rectify the issue as input from the software keyboard is not passed through.

Comment 37 by, Dec 1

#36: yes. The virtual keyboard (onscreen keyboard) of ChromeOS is also an input method, it works with the same protocol. Actually my demo on my local device works with the virtual keyboard.

However keep in mind that the virtual keyboard support won't be perfect. Yes it can input characters, though some neat features won't be there (for example, the text caret might be hidden by the virtual keyboard itself when it's on the bottom part of the screen -- the app won't scroll up automatically upon virtual keyboard appearance like mobile apps do). This is because the Linux apps do not care about tablet and virtual keyboard.

Comment 38 by, Dec 1

This is because the Linux apps do not care about tablet and virtual keyboard.

Ad far I can see, with the latest dev build 72.0.3609.3, when I recall chromeOs ime from the accessibility menu, do resize themself leaving all the necessary room for the on-screen keyboard. Such behaviour looks to be consistent across all applications, such as libreoffice, thunar, konsole... So, when your plugin is done, it should work in a neat way. I have had a look at it anyway and it looks that you are close to the point, apart from using C instead of C++. Good work, this will be a killer feature for tablet mode.

Comment 39 by, Dec 3

This P1 bug seems to have been stalled for a while.

mukai - do you have cycles to work on this for 73? If you don't maybe we should fine a new owner or update the priority.

Comment 40 by, Dec 3


Comment 41 by, Dec 4

Labels: -Pri-1 -M-70 M-74 Pri-2
I hope I could do something for M73, but I'm afraid that it is not likely to be done fully by M73. Since Crostini is still in beta, let me deprioritize this to P2 (tbuckley -- please reprerioritize if this is wrong).

benwells, if you or someone in your team is interested in working on this, I'm really happy. I can share any resources if so.

Comment 42 by, Dec 4

P2 sounds right to me, and 73 or 74 seems acceptable ... but please let us know if you think you won't get cycles to do that and we will try to figure something out.

Comment 43 by, Dec 4

Just want to say that I'll be happy to see this working for myself and all the new Pixel Slate users. I was bummed to breakout into Tablet mode and find the the virtual keyboard didn't popup when working in PyCharm. I'd hope to be able to do some code when travelling without the keyboard case.

Anyways - good work everyone! Just having Crostini is amazing and pushed to buy a Chrome device.

Comment 44 by, Dec 10

 Issue 913533  has been merged into this issue.

Comment 45 by, Dec 12

it seems it's hard for me to make time for this effort. tbuckley@, please find the owner.

Comment 46 by, Dec 12


Comment 47 by, Dec 12

We will take care of this in the LA team.

Comment 48 by, Jan 31

anything update? or I can use with some app in M73?

Comment 49 by bugdroid, Feb 7

Project Member

Sign in to add a comment