> Where only 1. and 4. are cancelable.

I'm starting to think the TPAC spec has too many assumptions baked in that IME stuff is not cancelable and I think this needs more empirical investigation.  I recently learned about Android native textbox feature InputFilter (https://developer.android.com/reference/android/text/InputFilter.html) which is a strong indication that Android IMEs have an implied contract to be able to tolerate receiving a different result than they specified for arbitrary composition update.  (That being said, InputFilters are rare in practice so it's not clear how well IMEs will tolerate the situation in reality -- thus empirical investigation is called for.)  And this constraint informs InputConnection API already in the sense that IME cannot specify cursor positions within a composed word (because the length of the word is expected to change arbitrarily underneath the IME).

I think we should take the same kind of approach as we did with passive event listeners and post-touch timeout events whereby the "cancelable" boolean of events depends dynamically on whether the particular IME event is preventDefaultable.

Because the entire universe is IME on Android and many valid JS use cases rely on preventDefault, I think we're never going to reach a satisfactory outcome for rich text editing on Android unless we start tackling the hard problem of how to make IME events cancelable.  This will be the only true fix for the IME-driven-but-JS-intercepted-spellcheck use case discussed on https://github.com/w3c/input-events/issues/31, for example, and we are likely to keep finding more.

Yes I agree that some IMEs might be cancelable, but at least some of them are not.

Our design for InputEvent is to simplify the IME model into 2 cancelable atomic actions:
  1. Delete selected / existing text at composition start.
  2. Insert final confirmed text at composition end.
And UA will take care of everything in the middle.

AFAIK this approach is good enough for most JS developers, where they don't actually care about what's happening during composition. The expected usage is:
  1. JS handles 'deleteByComposition' at the start
  2. Take a nap in the middle, and let UA do the composition
  3. UA finished composition, and should've cleared all marked text and recovered DOM
  4. JS handles 'insertFromComposition'

For version 1 we wanted to at least make the IME model interoperable, thus JS developers don't have to worry about special cases. Maybe in the future we could investigate how hard is it to make *some* IME cancelable, and how would it benefit developers.

Hmm, so kind of like a lock/unlock concept.  It's not clear to me it's viable for Android IME.

> 2 cancelable atomic actions:
  1. Delete selected / existing text at composition start.

There doesn't exist a way to tell IME we rejected the composition start on Android.  The IME will carry on with its sequence of IME commands.  So the only meaning preventDefaulting this could have would be to create UA-side state to ignore the result of future compositionUpdates.  That strictly worse and seems no easier to implement than allowing JS to preventDefault individual composition updates.

> And UA will take care of everything in the middle.

Composing text has an effect on layout of the surrounding text as it comes in character by character, though, so it can't be handled as a pure overlay.  Is your plan to use shadow DOM or something?

> AFAIK this approach is good enough for most JS developers, where they don't actually care about what's happening during composition.

A search engine wants to see letters come in one at a time to propose autocomplete suggestions, for example.  Presenting only completed words to JS would break this use case which works fine today.

> There doesn't exist a way to tell IME we rejected the composition start on Android.

We don't give any feedback to IME, and cancelling 'deleteByComposition' doesn't affect IME.
No matter whether or not 'deleteByComposition' is prevented, the next 'insertCompositionText' will always insert the entire composition string.

See explainer https://github.com/w3c/input-events/issues/42#issuecomment-254404854


> Composing text has an effect on layout of the surrounding text
Yes.
> Is your plan to use shadow DOM or something?
No.

Affecting layout/DOM temporally is OK, as long as UA can recover it to the original state after composition.


> A search engine wants to see letters come in one at a time to propose autocomplete suggestions

JS can still get composition string during the process, what they cannot do is to cancel it.

> We don't give any feedback to IME, and cancelling 'deleteByComposition' doesn't affect IME.
No matter whether or not 'deleteByComposition' is prevented, the next 'insertCompositionText' will always insert the entire composition string.

So, preventDefaulting deleteComposition means that the text will stay in place and simply be replaced atomically, instead of deleted then recreated?  I'm lacking context about what all these behaviors are supposed to achieve in the first place, a few questions:
A) What's the use case for the deletion?
B) What's the use case for preventing the deletion?
C) In case of deletion, what happens on the screen in the interval between the deletion and the first composition update, does the user see a flash of blank?  If not, how are you planning to avoid that?

> JS can still get composition string during the process, what they cannot do is to cancel it.

JS can also mutate the composition string during.  That will defeat your rewinding-based cancellation semantics.  What behavior is specified in this scenario?


My attitude currently is that this plan sounds unnecessarily complicated, and the more obvious approach of just simply allowing preventDefault of compositionUpdate (by which I mean atomic replacement of selected text combined with cursor and composition range change) should be pursued first unless there are specific reasons not to.  The simple approach seems more powerful and easier to design and implement, as far as I know so far.

The general idea for 'beforeinput' is: JS can prevent all DOM mutations (except during IME) by preventing all 'beforeinput'.

> So, preventDefaulting deleteComposition means that the text will stay in place and simply be replaced atomically

No. preventDefaulting 'deleteComposition' means the text won't get deleted by UA (and next 'insertCompositionText' will insert a separate copy of marked text on the forward direction of the original text). I think this is a natural behavior of preventing a *delete* action.

> A) What's the use case for the deletion?
It's telling JS that UA is going to deleting some text.

> B) What's the use case for preventing the deletion?
JS can implement it's own deletion, e.g. Strikethrough, and update it's own data model.

> C) In case of deletion, what happens on the screen in the interval between the deletion and the first composition update, does the user see a flash of blank?  If not, how are you planning to avoid that?

That's a good question, there could be a blank since the original text was removed.
A possible solution is https://github.com/w3c/input-events/issues/34#issuecomment-254642162, but it might make the spec too confusing.


> JS can also mutate the composition string during.  That will defeat your rewinding-based cancellation semantics.

There is no specific rewinding algorithm, UA just simply removed all marked text at the end of composition. The marked text would be plain-text if JS decided to prevent 'deleteComposition' as well.

> and the more obvious approach of just simply allowing preventDefault of compositionUpdate 

I believe the Editing TF has made some research about it, and the conclusion is IMEs are so complex and sensitive that we cannot make it cancelable and let JS mutate the composition string. If you are objecting this assumption I can try to dig out the original discussion.

> That's a good question, there could be a blank since the original text was removed.
A possible solution is https://github.com/w3c/input-events/issues/34#issuecomment-254642162, but it might make the spec too confusing.

Well, we should first reason about how the plan might be workable at all before we start worrying about confusingness.

It's a hard requirement that there must be no UX change whatsoever on Android in the case JS is not present or listening passively.  I don't follow what your plan is to delete the text in the DOM at start of composition but for that to have no effect on the visual display.

As I said, the only way I can think of for something like that to work is to hide the composing text into shadow DOM as opposed to literally deleting it.  And if you did that, then JS would lose the ability to act on composition updates one by one that it probably wants to do for certain use cases.

As for the proposal in https://github.com/w3c/input-events/issues/34#issuecomment-254642162, it doesn't really make sense to me either.  It doesn't sound like it actually winds up providing any new capability at all for JS to prevent IME from doing anything?

>> A) What's the use case for the deletion?
> It's telling JS that UA is going to deleting some text.

But the UA is *not* going to be deleting any text, that's just your own artificial construct.  What information is JS receiving that's not already in the compositionStart/Update/End events?

>> B) What's the use case for preventing the deletion?
> JS can implement it's own deletion, e.g. Strikethrough, and update it's own data model.

Again, JS can already today mutate the DOM with a strikethrough and update its internal data model in reaction to compositionStart/Update/End.

> I believe the Editing TF has made some research about it, and the conclusion is IMEs are so complex and sensitive that we cannot make it cancelable and let JS mutate the composition string. If you are objecting this assumption I can try to dig out the original discussion.

Yeah, I am, particularly as it may turn out to be the only path forward.  "complex and sensitive" isn't a specific technical argument, and there's reason to believe at least some polished modern IME can handle text changing out from under them (after all, powerful native apps like Microsoft Office are talking to the same IME APIs that we are so we should have the power to simulate their behavior).

And, on further thought, there's no real value added to providing preventDefault per se, and what we really need to do is to define a sane behavior when JS synchronously mutates the text in compositionUpdate handler.  JS can and is already doing that, and that's a generalization of cancellation (which is just one type of mutation -- removal -- of the text IME has already inserted from its point of view).  So the kind of stuff we need to be writing up in the spec might not involve any new events at all, but rather ordering and conflict resolution semantics for the events that we already have.

It seems proposal in #c1 doesn't work well.

How about introducing IME support level[1] concept and resume IME API work[2]?

Proposal:

Introduce "Partial IME support" and "Full IME support" level with (advanced) IME API[2].
 "Partial IME support" is current behavior; just only composition{start,update,end}
 "Full IME support" is composition{start,update,end} but no DOM update, JS owns DOM update and UI feedback, e.g. candidate window, status window etc.


[1] https://msdn.microsoft.com/en-us/library/cc194852.aspx
[2] https://www.w3.org/TR/ime-api/

#10: hmm, I don't think the IME API proposal is relevant to this bug.  That API is for allowing JS to generate a custom IME implementation, whereas this bug is about allowing JS to implement rich text editors that properly integrate with existing platform IME.

Personally, I don't have any interest in IME API because it doesn't seem like it can possibly be shippable on Android.  (Android actively prevents native apps from choosing their preferred IME implementation.  This is a user-controlled setting so that users are empowered to use their preferred IME on every application.  We would not make an exception for webapps.)

> Again, JS can already today mutate the DOM with a strikethrough and update its internal data model in reaction to compositionStart/Update/End.

Yes, JS has all the info they need, but what they can't do is preventing DOM mutation.
e.g. You can't prevent text from being delete by IME.

FYI currently JS editors relies on complex DOM diffing mechanism to figure what UA just did.
(Reference https://lists.w3.org/Archives/Public/public-editing-tf/2015Oct/0009.html)

> Yeah, I am, particularly as it may turn out to be the only path forward.

Here is the discussion for how to deal with IME:
https://lists.w3.org/Archives/Public/public-editing-tf/2015Oct/0002.html

The proposed solutions are:
  1. Make all composition events cancelable;
  2. Isolate IME from DOM ('sandbox' in the discussion)

IIUC you are suggesting solution 1, but I believe it was objected by:
  1. kojii@
    * 'IIRC some actions in IME are not really cancelable on some platforms, but I don't have all those details.'
    * https://lists.w3.org/Archives/Public/public-editing-tf/2015Oct/0030.html
  2. rniwa@apple.com
    * 'I don't think we can make each composition event cancelable per se.  Web browsers can only tell Web apps what IME told us to do.'
    * https://lists.w3.org/Archives/Public/public-editing-tf/2015Oct/0093.html
(I'm not sure if I quoted correctly as I wasn't here at the time, but kojii@ might have more context)

While solution 1 can take a large amount of work, solution 2 could be easier and meets 99% of what JS developers want.
(Reference https://lists.w3.org/Archives/Public/public-editing-tf/2015Oct/0014.html)
  1. Solution 2 can be implemented using shadow DOM, but it was objected later
  2. Solution 2 can also be implemented as doing IME in DOM, but revert to original state after
    * This is what I'm trying to do here

IIRC the final decision is to build an workable solution first, then investigate on how to cancel IME.

OK, further evolution in opinion for me.  I'm OK with a two-track short-term/long-term plan now.


yosin@, for the long term, I looked more carefully about IME API and I now agree with you completely about the idea of "Full IME support -- composition{start,update,end} but no DOM update, JS owns DOM update".  I think that's exactly the right long-term direction and it's well aligned with web platform extensibility manifesto.  JS could even redirect IME to canvas or over the network to a remote terminal.

What I had a bad reaction to in #12 is your mention that JS should own "UI feedback, e.g. candidate window, status window etc."  I feel strongly this should still be outside JS's responsibility.  On Android, this stuff is the responsibility of IME Service side, not the editor.

In my opinion the way "Full IME API" should work is like this: on user gesture, JS can attach "virtual textbox" to the IME.  Then it receives all IME calls more or less like Blink receives IME IPCs, and it responds to IME get calls on the textbox state (which have to include the content of the *entire* textbox on Android, not just the current composition).  This communication is entirely in plain text which is what IME understands.  Then JS can take care of maintaining a mapping of the plain text to rich text and displaying it on screen in whatever way it wants.

With such an API, we wouldn't be a position of having a default action and letting Javascript know about it after the fact (nor or being afraid of cancelling default action because that would be complex and interfere with normal textbox behavior).  We would just give Javascript full control to invent their own editor.


For short term, we still need to provide support for today's regular IME, but for that I'd like to keep a narrow focus on what existing users are doing and actively asking for -- for example, email apps want to intercept Enter composition commit in order to break out of inline, and search boxes want to autocomplete the query.  I don't think anymore that we should be overly ambitious about providing a total JS-customizable IME experience on top of ContentEditable, it's just too much of a mess and the "Full IME" idea is much more promising.  So I'd like you to explain what specific, concrete use case each of your proposed events solved and if I want to make sure it's the simplest and cleanest solution to those use cases.

> OK, further evolution in opinion for me.  I'm OK with a two-track short-term/long-term plan now.

Great!

> So I'd like you to explain what specific, concrete use case each of your proposed events solved and if I want to make sure it's the simplest and cleanest solution to those use cases.

Sure.

Example 1 - Track Changes Editor *without special handling for IME*
To implement an editor like http://www.loopindex.com/lite/demo/ using InputEvent, developers can simply
  1. Listen to all cancelable 'beforeinput' with inputType start with 'delete'
    * preventDefault() and strike-through the target range
  2. Listen to all cancelable 'beforeinput' with inputType start with 'insert'
    * preventDefault() and insert data with green background

Why it works for IME (Taking Android as an example):
Please take a look at the 'Example of Recomposition on Android' section in https://github.com/w3c/input-events/issues/34
  1. Preventing Step 2. 'deleteByComposition' means UA won't remove 'Wo'
  2. JS then can strike-through 'Wo'
  3. UA collapse selection forward and start composition
  4. UA re-insert composition text 'Wo' immediately (DOM would looks like '-Wo-_Wo_', where '-Wo-' is crossed text and _Wo_ is composition text)
  5. After some composition updates, JS can prevent 'insertFromComposition' and do custom text insertion

Then what's the purpose of 'insertCompositionText' and 'deleteCompositionText'?
  * They are just for consistency and the sake of default behaviors

Example 2 - Custom |maxlength| by handling all 'insert*' beforeinput
  * But sure you can do this with non-standard 'textInput' event.

Example 3 - Custom preserving style algorithm
  1. Mark range to be deleted in 'deleteByComposition' as hidden
  2. Later, compare with final 'insertFromComposition' and do partial replacing

Does that sounds reasonable, or do you want a talk / VC?

Also since everything is too complex, do you want a doc about my plan of implementation?

OK, thanks for the use cases, that's very useful information.

> Example 1 - Track Changes Editor *without special handling for IME*

The core problem to be solved here is avoid confusing the IME so that it doesn't go insane with the cursor and its autocompletion logic, so I'd like us to focus on that.  We need to understand how the various Android IMEs will react when they send "deleteSurroundingText(0, 1)" and the result is that the cursor goes back by 1 but the character stays.  To iterate quickly, I recommend that you write a hacky Android native app that subclasses BaseInputConnection/EditText and implements a native version of http://www.loopindex.com/lite/demo/.  Then, we can put in the web spec what we discover to work properly on every keyboard app.  You would learn a lot about Android IME by doing this.  And I believe Android IME is the richest and most demanding IME API, so if you can solve Android, the solution might work for Windows/Mac as well.

(Note that an totally alternate proposal for how JS might implement a Track Changes Editor is stop presenting the strikethroughed text to the IME view of textfield, even though the user sees it onscreen.  If the IME gets all the usual feedback that it deleted the text, it is guaranteed not to get confused, and we don't need to support preventDefault at all.  Then, figure out a way JS a way to inject DOM which is invisible to plainText() representation given to IME.  This would have a very different UX for selection -- the cursor would skip right over the strikethroughed text -- which may or may not be better.)


> Does that sounds reasonable, or do you want a talk / VC?

We should have a VC, but I am busy right now, so let's have it in 2 weeks before I leave on vacation.

Close as WontFix as we want to support only 'insertCompositionText' in MVP.

See  issue 689541