New issue
Advanced search Search tips
Note: Color blocks (like █ or █) mean that a user may not be available. Tooltip shows the reason.

Issue 826617 link

Starred by 4 users
Status: Available
Owner: ----
Cc:
omrilio@chromium.org
wuyingbing@chromium.org
█ blakeo@chromium.org
█ jzfeng@chromium.org
shend@chromium.org
█ dvallet@chromium.org
yhanada@chromium.org
pcovell@chromium.org
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 3
Type: Feature



Sign in to add a comment

Full screen Handwriting Implementation

Project Member Reported by jzfeng@chromium.org, Mar 28 2018 
The Chrome OS Virtual Keyboard provides two modes: a full width keyboard, or a floating keyboard. Under both modes,  user can switch to handwriting and write within a given writing area. This setting has two issues:
  1.  For full width keyboard, the area covers almost half of the screen, which blocks the web contents when user is writing.
  2.  On small screen devices, the area could be too small for writing.

This bug tracks the implementation of a full screen handwriting feature, which makes the writing area full screen and transparent (i.e., users can still see the web pages during writing). This feature is already available on Android for some languages, such as Chinese.

Design doc: https://docs.google.com/document/d/1LbZsSJZQvFPWB9MikF2T3sZvTs-5zY4KYu6qI9Ub4gs/edit?usp=sharing

Comment 1 by jzfeng@chromium.org, Mar 28 2018

Labels: M-72
Project Member

Comment 2 by bugdroid1@chromium.org, Mar 29 2018 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/4fb046cb69a15bfce7d93dfabef300eac4a89e99

commit 4fb046cb69a15bfce7d93dfabef300eac4a89e99
Author: Jianzhou Feng <jzfeng@google.com>
Date: Thu Mar 29 07:38:54 2018

Add enable-fullscreen-handwriting-virtual-keyboard flag definition.

This will be used to hide away ongoing fullscreen handwriting virtual keyboard work.

Bug: 826617
Change-Id: I3a67a98fd7de328e3d33aa9d7262ba4140b220ba
Reviewed-on: https://chromium-review.googlesource.com/983272
Reviewed-by: Sadrul Chowdhury <sadrul@chromium.org>
Commit-Queue: Jianzhou Feng <jzfeng@chromium.org>
Cr-Commit-Position: refs/heads/master@{#546761}
[modify] https://crrev.com/4fb046cb69a15bfce7d93dfabef300eac4a89e99/chrome/browser/about_flags.cc
[modify] https://crrev.com/4fb046cb69a15bfce7d93dfabef300eac4a89e99/chrome/browser/flag_descriptions.cc
[modify] https://crrev.com/4fb046cb69a15bfce7d93dfabef300eac4a89e99/chrome/browser/flag_descriptions.h
[modify] https://crrev.com/4fb046cb69a15bfce7d93dfabef300eac4a89e99/tools/metrics/histograms/enums.xml
[modify] https://crrev.com/4fb046cb69a15bfce7d93dfabef300eac4a89e99/ui/base/ui_base_features.cc
[modify] https://crrev.com/4fb046cb69a15bfce7d93dfabef300eac4a89e99/ui/base/ui_base_features.h
Project Member

Comment 3 by bugdroid1@chromium.org, Apr 18 2018 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/7c1b43d13f6c914c496dae48b1c392ee6e045794

commit 7c1b43d13f6c914c496dae48b1c392ee6e045794
Author: Jianzhou Feng <jzfeng@google.com>
Date: Wed Apr 18 06:10:14 2018

Prepare a transparent window and pass the fullscreen handwriting feature to IME

Make a transparent window, and use a zero overlap in InitInsets and UpdateInsetsForWindow
to avoid the window beneath not rendering anything.

Bug: 826617
Change-Id: I9b5e8140a6b9a9c481caf92ef3d244ffc14b64d8
Reviewed-on: https://chromium-review.googlesource.com/1013444
Commit-Queue: Jianzhou Feng <jzfeng@chromium.org>
Reviewed-by: Yuichiro Hanada <yhanada@chromium.org>
Cr-Commit-Position: refs/heads/master@{#551592}
[modify] https://crrev.com/7c1b43d13f6c914c496dae48b1c392ee6e045794/chrome/browser/extensions/api/virtual_keyboard_private/chrome_virtual_keyboard_delegate.cc
[modify] https://crrev.com/7c1b43d13f6c914c496dae48b1c392ee6e045794/chrome/browser/ui/ash/chrome_keyboard_ui.cc
[modify] https://crrev.com/7c1b43d13f6c914c496dae48b1c392ee6e045794/ui/keyboard/keyboard_util.cc
[modify] https://crrev.com/7c1b43d13f6c914c496dae48b1c392ee6e045794/ui/keyboard/keyboard_util.h
Project Member

Comment 4 by bugdroid1@chromium.org, Apr 24 2018 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/2d463f5527d6d5b4b82fbb563798462793c3b6f2

commit 2d463f5527d6d5b4b82fbb563798462793c3b6f2
Author: Jianzhou Feng <jzfeng@google.com>
Date: Tue Apr 24 04:31:56 2018

Add fullscreen container behavior.

Set the VK Aura Window to be transparent.
Otherwise WindowOcclusionTracker will set the Web Page Aura Window state
to be occluded and user can not see the committed input texts after VK
turns fullscreen. See conditions defined in VisibleWindowIsOpaque.

Bug: 826617
Change-Id: Idd8c525f205e608fe53fb59f85d302175b7a6246
Reviewed-on: https://chromium-review.googlesource.com/1023380
Commit-Queue: Jianzhou Feng <jzfeng@chromium.org>
Reviewed-by: Toni Barzic <tbarzic@chromium.org>
Reviewed-by: Yuichiro Hanada <yhanada@chromium.org>
Cr-Commit-Position: refs/heads/master@{#552995}
[modify] https://crrev.com/2d463f5527d6d5b4b82fbb563798462793c3b6f2/chrome/browser/extensions/api/virtual_keyboard_private/chrome_virtual_keyboard_delegate.cc
[modify] https://crrev.com/2d463f5527d6d5b4b82fbb563798462793c3b6f2/chrome/browser/ui/ash/chrome_keyboard_ui.cc
[modify] https://crrev.com/2d463f5527d6d5b4b82fbb563798462793c3b6f2/extensions/common/api/virtual_keyboard_private.json
[modify] https://crrev.com/2d463f5527d6d5b4b82fbb563798462793c3b6f2/ui/keyboard/BUILD.gn
[add] https://crrev.com/2d463f5527d6d5b4b82fbb563798462793c3b6f2/ui/keyboard/container_fullscreen_behavior.cc
[add] https://crrev.com/2d463f5527d6d5b4b82fbb563798462793c3b6f2/ui/keyboard/container_fullscreen_behavior.h
[modify] https://crrev.com/2d463f5527d6d5b4b82fbb563798462793c3b6f2/ui/keyboard/container_type.h
[modify] https://crrev.com/2d463f5527d6d5b4b82fbb563798462793c3b6f2/ui/keyboard/keyboard_controller.cc

Comment 5 by jzfeng@chromium.org, Apr 24 2018 

Note:
A quick suggestion from yuqing - candidates following cursor, to avoid finding candidates in long distance.

Comment 6 by shend@chromium.org, May 21 2018

Cc: jzfeng@chromium.org
Owner: shend@chromium.org
Taking this over from jzfeng@.
Project Member

Comment 7 by bugdroid1@chromium.org, May 23 2018 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/4563ced33ec2270253404a107b43652b0b1c4207

commit 4563ced33ec2270253404a107b43652b0b1c4207
Author: Darren Shen <shend@chromium.org>
Date: Wed May 23 01:20:26 2018

[VK] Add ContainerBehavior::GetOccludedBounds().

Currently, we have two container behaviors: full width and floating.
In the full width VK, the occluded bounds is the same as the bounds of
the keyboard window. In the floating VK, nothing is occluded.

Thus, with only these two container behaviors, we could get away with
having a single boolean function (BoundsObscureUsableRegion) that
indicates whether the occluded bounds is the bounds of the keyboard
window or the empty rectangle.

However, the new fullscreen VK has an occluded region, but it is not
the same as the keyboard window. Therefore, we need to change the
boolean function into a function that actually returns the occluded
bounds.

We also make some changes to clarify which bounds are being referred to.

Bug: 826617
Change-Id: I928294d81808cb11090f96b6fc81c6a123a2595e
Reviewed-on: https://chromium-review.googlesource.com/1067218
Commit-Queue: Darren Shen <shend@chromium.org>
Reviewed-by: calamity <calamity@chromium.org>
Reviewed-by: James Cook <jamescook@chromium.org>
Reviewed-by: Yuichiro Hanada <yhanada@chromium.org>
Cr-Commit-Position: refs/heads/master@{#560875}
[modify] https://crrev.com/4563ced33ec2270253404a107b43652b0b1c4207/ash/accessibility/touch_exploration_manager.cc
[modify] https://crrev.com/4563ced33ec2270253404a107b43652b0b1c4207/ash/wm/lock_action_handler_layout_manager_unittest.cc
[modify] https://crrev.com/4563ced33ec2270253404a107b43652b0b1c4207/ash/wm/lock_layout_manager_unittest.cc
[modify] https://crrev.com/4563ced33ec2270253404a107b43652b0b1c4207/ash/wm/system_modal_container_layout_manager.cc
[modify] https://crrev.com/4563ced33ec2270253404a107b43652b0b1c4207/ui/app_list/views/app_list_folder_view.cc
[modify] https://crrev.com/4563ced33ec2270253404a107b43652b0b1c4207/ui/keyboard/container_behavior.h
[modify] https://crrev.com/4563ced33ec2270253404a107b43652b0b1c4207/ui/keyboard/container_floating_behavior.cc
[modify] https://crrev.com/4563ced33ec2270253404a107b43652b0b1c4207/ui/keyboard/container_floating_behavior.h
[modify] https://crrev.com/4563ced33ec2270253404a107b43652b0b1c4207/ui/keyboard/container_full_width_behavior.cc
[modify] https://crrev.com/4563ced33ec2270253404a107b43652b0b1c4207/ui/keyboard/container_full_width_behavior.h
[modify] https://crrev.com/4563ced33ec2270253404a107b43652b0b1c4207/ui/keyboard/container_fullscreen_behavior.cc
[modify] https://crrev.com/4563ced33ec2270253404a107b43652b0b1c4207/ui/keyboard/container_fullscreen_behavior.h
[modify] https://crrev.com/4563ced33ec2270253404a107b43652b0b1c4207/ui/keyboard/keyboard_controller.cc
[modify] https://crrev.com/4563ced33ec2270253404a107b43652b0b1c4207/ui/keyboard/keyboard_controller.h
[modify] https://crrev.com/4563ced33ec2270253404a107b43652b0b1c4207/ui/keyboard/notification_manager.cc
[modify] https://crrev.com/4563ced33ec2270253404a107b43652b0b1c4207/ui/keyboard/notification_manager.h
Project Member

Comment 8 by bugdroid1@chromium.org, Jun 14 2018 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/2e9ed82ae038f43c191bde697c272297cdf867b1

commit 2e9ed82ae038f43c191bde697c272297cdf867b1
Author: Darren Shen <shend@chromium.org>
Date: Thu Jun 14 11:17:01 2018

Add SetOccludedBounds extension API.

The occluded bounds is stored in the fullscreen container and retreived to
ensure caret in work area. Also updated the places to notify the correct
occluded bounds for the new container mode.

Bug: 826617
Change-Id: Iafe3dcf291975b586dece39ee6e07bb3f7f49dba
Reviewed-on: https://chromium-review.googlesource.com/1036867
Commit-Queue: Darren Shen <shend@chromium.org>
Reviewed-by: Ben Wells <benwells@chromium.org>
Reviewed-by: Blake O'Hare <blakeo@chromium.org>
Reviewed-by: Yuichiro Hanada <yhanada@chromium.org>
Cr-Commit-Position: refs/heads/master@{#567222}
[modify] https://crrev.com/2e9ed82ae038f43c191bde697c272297cdf867b1/chrome/browser/extensions/api/virtual_keyboard_private/chrome_virtual_keyboard_delegate.cc
[modify] https://crrev.com/2e9ed82ae038f43c191bde697c272297cdf867b1/chrome/browser/extensions/api/virtual_keyboard_private/chrome_virtual_keyboard_delegate.h
[modify] https://crrev.com/2e9ed82ae038f43c191bde697c272297cdf867b1/chrome/browser/ui/ash/chrome_keyboard_ui.cc
[modify] https://crrev.com/2e9ed82ae038f43c191bde697c272297cdf867b1/chrome/test/data/chromeos/virtual_keyboard/default_extension/virtual_keyboard_test_base.js
[modify] https://crrev.com/2e9ed82ae038f43c191bde697c272297cdf867b1/extensions/browser/BUILD.gn
[modify] https://crrev.com/2e9ed82ae038f43c191bde697c272297cdf867b1/extensions/browser/api/virtual_keyboard_private/virtual_keyboard_delegate.h
[modify] https://crrev.com/2e9ed82ae038f43c191bde697c272297cdf867b1/extensions/browser/api/virtual_keyboard_private/virtual_keyboard_private_api.cc
[modify] https://crrev.com/2e9ed82ae038f43c191bde697c272297cdf867b1/extensions/browser/api/virtual_keyboard_private/virtual_keyboard_private_api.h
[add] https://crrev.com/2e9ed82ae038f43c191bde697c272297cdf867b1/extensions/browser/api/virtual_keyboard_private/virtual_keyboard_private_api_unittest.cc
[modify] https://crrev.com/2e9ed82ae038f43c191bde697c272297cdf867b1/extensions/browser/extension_function_histogram_value.h
[modify] https://crrev.com/2e9ed82ae038f43c191bde697c272297cdf867b1/extensions/common/api/virtual_keyboard_private.json
[modify] https://crrev.com/2e9ed82ae038f43c191bde697c272297cdf867b1/extensions/shell/browser/shell_virtual_keyboard_delegate.cc
[modify] https://crrev.com/2e9ed82ae038f43c191bde697c272297cdf867b1/extensions/shell/browser/shell_virtual_keyboard_delegate.h
[modify] https://crrev.com/2e9ed82ae038f43c191bde697c272297cdf867b1/tools/metrics/histograms/enums.xml
[modify] https://crrev.com/2e9ed82ae038f43c191bde697c272297cdf867b1/ui/keyboard/container_fullscreen_behavior.cc
[modify] https://crrev.com/2e9ed82ae038f43c191bde697c272297cdf867b1/ui/keyboard/container_fullscreen_behavior.h
[modify] https://crrev.com/2e9ed82ae038f43c191bde697c272297cdf867b1/ui/keyboard/keyboard_controller.cc
[modify] https://crrev.com/2e9ed82ae038f43c191bde697c272297cdf867b1/ui/keyboard/keyboard_controller.h
[modify] https://crrev.com/2e9ed82ae038f43c191bde697c272297cdf867b1/ui/keyboard/keyboard_controller_unittest.cc
[modify] https://crrev.com/2e9ed82ae038f43c191bde697c272297cdf867b1/ui/keyboard/notification_manager.cc
[modify] https://crrev.com/2e9ed82ae038f43c191bde697c272297cdf867b1/ui/keyboard/notification_manager.h
Project Member

Comment 9 by bugdroid1@chromium.org, Jun 21 2018 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/433b8f35731c0314df5d81a232297e3cdbdebd60

commit 433b8f35731c0314df5d81a232297e3cdbdebd60
Author: Darren Shen <shend@chromium.org>
Date: Thu Jun 21 23:47:44 2018

[VK] Fix virtualKeyboardPrivate.setOccludedBounds.

There was an extraneous line that messed up the validator.

Bug: 826617
Change-Id: Ie96b150093c2158187ab6751335cdec3bdbb2d4b
Reviewed-on: https://chromium-review.googlesource.com/1109664
Reviewed-by: Ben Wells <benwells@chromium.org>
Commit-Queue: Darren Shen <shend@chromium.org>
Cr-Commit-Position: refs/heads/master@{#569453}
[modify] https://crrev.com/433b8f35731c0314df5d81a232297e3cdbdebd60/extensions/common/api/virtual_keyboard_private.json

Comment 10 by shend@chromium.org, Sep 12

Cc: -yhanada@chromium.org shend@chromium.org
Owner: ----
Status: Available (was: Assigned)
Going to unassign myself since this feature needs more thinking.

Comment 11 by pcovell@chromium.org, Sep 20

Labels: -Pri-2 -M-72 Pri-3
►

Sign in to add a comment