So this has two separate issues:
1. In ShouldWindowOverscroll(aura::Window* window), we have an exception for IME windows using the following logic:
return !ash::RootWindowController::ForWindow(window->GetRootWindow())
->GetContainer(ash::kShellWindowId_ImeWindowParentContainer)
->Contains(window);
It seems like we should be able to use a different mechanism to identify IME windows, maybe using Widget::InitParams.mus_properties, e.g. make the following logic always apply for mash (regardless of whether it is in process):
https://cs.chromium.org/chromium/src/chrome/browser/ui/ash/ash_util.cc?type=cs&q=SetupWidgetInitParamsForContainer&sq=package:chromium&g=0&l=38
Does that sound reasonable? Other suggestions?
2. ChromeKeyboardUI::GetInputMethod() is implemented as:
ash::Shell::GetRootWindowForNewWindows()->GetHost()->GetInputMethod();
It is not remotely clear to me what the correct InputMethod* instance should be.
Oh, and then there is the minor issue for Mash-MultiProcess where ChromeKeyboardUI is created from Ash through ChromeShellDelegate and passed to keyboard::KeyboardController which is owned by Ash. Fixing that is going to be a much larger problem.
+xiyuan
Re 1) If IME windows need special behaviors then adding an aura window property seems like one way to go. We have lots of ash-only window properties to support mash.
Re 2) Today in classic ash there is a single InputMethod* shared across the entire system:
https://cs.chromium.org/chromium/src/ui/aura/window_tree_host.cc?type=cs&sq=package:chromium&g=0&l=239
For mash I think we want to move to a model like desktop aura where there is one per toplevel window, and maybe one in the ash process. xiyuan would know more.
Re: making web stuff from ash, you might look at how the ChromeVox / accessibility panel is created. It's web content owned by chrome, but with special window handling in ash.
+shuchen for thoughts on preparing for new mojo-based IMF work.
As James mentioned, for classic ash, there is only one shared IME across the system.
For single process mash, we would still have that so that local aura::Window (from ash) still works. For non-local aura::Windows, they will have a InputMethodMus to wrap IME calls into mojo calls and handled by InputMethodBridge [1]. InputMethodBridge runs in the browser process and would create a real IME - InputMethodChromeOS. I suspect ChromeKeyboardUI could use that one. To get that, instead of checking with ash, we could go through ui::IMEBridge since we are technically still in the same process. And instances of InputMethodChromeOS would notify IMEBridge via SetInputContextHandler when it is active (i.e. its TextInputClient is focused). We probably need to fix call sites of ChromeKeyboardUI::GetInputMethod() that assume the returned IME instance never changes.
Not sure how things would work for multi process mash. It looks like we need almost a rewrite for the whole KeybaordController/KeyboardUI infra.
[1]: https://cs.chromium.org/chromium/src/chrome/browser/ui/views/ime_driver/input_method_bridge_chromeos.h
Looking more closely at ShouldWindowOverscroll and how it is called, I have come to the conclusion that:
a) The correct solution is largely tied to how we resolve IME for multi-process ash.
b) ChromeKeyboardUI really needs to be re-factored and split across ash and chrome. Again, the exact split depends on how we implement IME.
i.e. as xiyuan@ mentions, this may require a re-write / re-design, and that may actually be easier than trying to hack together short term solutions for single process mash.
shuchen@ / shend@, are there any design docs that you can point me to for the active work on IME or the virtual keyboard? I'd like to understand where we are going before trying to "fix" anything short term (and potentially just muck it up).
Please refer to go/mojo-imf.
In the future, the apps (which holds the window/input-field implementations) and the imes (which listens to key strokes and provide the on-screen keyboard UI) are connected through the input method manager (IMM mojo service) lives in Mash.
Currently the on-screen keyboard UI is implemented as web view so it is avoidable to bring browser process. But we're planning to implement it as native UI in the future.
The docs for chrome_keyboard_ui say:
// Subclass of KeyboardUI. It is used by KeyboardController to get
// access to the virtual keyboard window and setup Chrome extension functions.
Looking at the code, one of the responsibilities of that class is to forward events to extensions (extensions::events::VIRTUAL_KEYBOARD_PRIVATE_ON_BOUNDS_CHANGED, extensions::events::VIRTUAL_KEYBOARD_PRIVATE_ON_KEYBOARD_CLOSED), and to signal extensions::VirtualKeyboardAPI::OnKeyboardConfigChanged().
Much of the rest of ChromeKeyboardUI can probably be moved to Ash, and a mojo interface could be used to route extension events and handle other chrome specific tasks.
There is also some contents:: specific code also, which may also need to live in Chrome?
If there is nobody actively re-factoring the KeyboardUI class, I will put together some diagrams to document it, and propose ways to break it up between Chrome and Ash.
James/ Xiyuan - Are there existing examples where Ash owns a host window, and chrome owns the WebContents?
I am currently in the process of doing some cleanup to ChromeKeyboardUI which currently does a mix of WebContents stuff and aura::Window stuff. I believe that we can separate the two into Chrome and Ash, I'll just need a way to associate the Chrome Window hosting the WebContents with the Ash Window. Possibly there will also need to be a mojo API between the two, but I'm still sorting through that.
I only know that launcher answer card embed a WebContents in ash UI. This is broken for single process mash because ash is using aura LOCAL mode and the embedding code only works in MUS mode (i.e. there is a WindowTree). I re-opened issue 812434 to track the work to make it work again.
Currently, launcher answer card is still using the AnswerCardContentsRegistry hack.
You might want to chat with sky@.
I did something vaguely like this for ChromeVoxPanel / AccessibilityPanel. It's a widget with webcontents that is created by chrome, but the layout is managed by ash.
That said, ChromeVoxPanel isn't working in SingleProcessMash right now and I'm not sure why.
It will eventually blockme , but not until well after I get back from vacation on 9/5:
Currently I am splitting up the functionality of ChromeKeyboardUI, which is helping me wrap my head around the code, and should make the Chrome/Ash separation simpler, or at least more incremental.
Next I will need to flesh out keyboard.mojom so that the code that routes KeyboardController events to the extension API can use the mojo API instead (it currently uses KeyboardControllerObserver).
Ultimately we will need Ash and keyboard::KeyboardController (which will only exist in Ash) to control the keyboard window (showing, hiding, placement), and Chrome to own the WebContents which contains the web view and the extension hooks. That's the part where it seems like I may be blocked but I'm not quite sure how that's going to look.
One piece that might be tricky is window bounds; there is a kind of complex relationship between window bounds and the extension IME API that I'm still wrapping my head around.
Comment 1 by jamescook@chromium.org
, May 16 2018+shend for keyboard This is for out-of-process ash ("mash" part of go/mustash)