New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 762352 link

Starred by 3 users

Issue metadata

Status: Fixed
Owner:
Closed: Jun 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Task

Blocking:
issue 764185



Sign in to add a comment

Heuristic language model

Project Member Reported by napper@chromium.org, Sep 6 2017

Issue description

Feature description:
We are implementing a heuristic that will model a user's languages in Chrome using a range of signals, including UI language, accept languages, detected languages from URL history, and the User Language Profile (ULP). The model will be used to improve accept-language header generation, translation triggering and target language, content recommendations, spell-checking, contextual search, and so on.

Eng owner:
chrome-language@google.com

Product owner:
yyushkina@chromium.org

Design doc:
https://docs.google.com/document/d/1iwVT7CAuFV7rZeqVLUfio8yDdonYOkxPSGLsq_AREFk

 
Blocking: 764185
Project Member

Comment 2 by bugdroid1@chromium.org, Sep 14 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/5b23b6627b8a5a9a07164cd24302e48b3215d0d0

commit 5b23b6627b8a5a9a07164cd24302e48b3215d0d0
Author: Michael Martis <martis@chromium.org>
Date: Thu Sep 14 02:44:27 2017

Added a language model interface and first implementation.

Currently not "connected" to any Chrome logic, so is a no-op.

Bug:  762352 
Change-Id: I854678c68d535c63dd3aa7b860e43d8bd7844029
Reviewed-on: https://chromium-review.googlesource.com/650071
Commit-Queue: Michael Martis <martis@chromium.org>
Reviewed-by: Jia Meng <jiameng@chromium.org>
Cr-Commit-Position: refs/heads/master@{#501849}
[modify] https://crrev.com/5b23b6627b8a5a9a07164cd24302e48b3215d0d0/components/language/core/browser/BUILD.gn
[add] https://crrev.com/5b23b6627b8a5a9a07164cd24302e48b3215d0d0/components/language/core/browser/baseline_language_model.cc
[add] https://crrev.com/5b23b6627b8a5a9a07164cd24302e48b3215d0d0/components/language/core/browser/baseline_language_model.h
[add] https://crrev.com/5b23b6627b8a5a9a07164cd24302e48b3215d0d0/components/language/core/browser/baseline_language_model_unittest.cc
[add] https://crrev.com/5b23b6627b8a5a9a07164cd24302e48b3215d0d0/components/language/core/browser/language_model.cc
[add] https://crrev.com/5b23b6627b8a5a9a07164cd24302e48b3215d0d0/components/language/core/browser/language_model.h
[modify] https://crrev.com/5b23b6627b8a5a9a07164cd24302e48b3215d0d0/components/language/core/browser/url_language_histogram.h

Components: Privacy
Project Member

Comment 4 by bugdroid1@chromium.org, Sep 20 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/f7bf39bae5c77ae3e18c1b5273c0efd6eca17b2b

commit f7bf39bae5c77ae3e18c1b5273c0efd6eca17b2b
Author: Michael Martis <martis@chromium.org>
Date: Wed Sep 20 03:51:26 2017

Added a (currently disconnected) factory for the user language model.

The factory will be used to attach a language model to a browsing session in
Content-based embedders. This patch also includes a flag to enable / disable
language model logic.

Bug:  762352 
Change-Id: If1ad07bda79fee70af978d4de632c8f13d2da3d9
Reviewed-on: https://chromium-review.googlesource.com/666360
Reviewed-by: David Vallet <dvallet@chromium.org>
Commit-Queue: Michael Martis <martis@chromium.org>
Cr-Commit-Position: refs/heads/master@{#503039}
[modify] https://crrev.com/f7bf39bae5c77ae3e18c1b5273c0efd6eca17b2b/chrome/browser/BUILD.gn
[add] https://crrev.com/f7bf39bae5c77ae3e18c1b5273c0efd6eca17b2b/chrome/browser/language/language_model_factory.cc
[add] https://crrev.com/f7bf39bae5c77ae3e18c1b5273c0efd6eca17b2b/chrome/browser/language/language_model_factory.h
[add] https://crrev.com/f7bf39bae5c77ae3e18c1b5273c0efd6eca17b2b/chrome/browser/language/language_model_factory_unittest.cc
[modify] https://crrev.com/f7bf39bae5c77ae3e18c1b5273c0efd6eca17b2b/chrome/test/BUILD.gn
[modify] https://crrev.com/f7bf39bae5c77ae3e18c1b5273c0efd6eca17b2b/components/language/core/browser/baseline_language_model.cc
[modify] https://crrev.com/f7bf39bae5c77ae3e18c1b5273c0efd6eca17b2b/components/language/core/browser/baseline_language_model.h

Project Member

Comment 5 by bugdroid1@chromium.org, Sep 20 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/29018888a07c2772554821502d3aa9e9d3aa2cb9

commit 29018888a07c2772554821502d3aa9e9d3aa2cb9
Author: Michael Martis <martis@chromium.org>
Date: Wed Sep 20 08:21:23 2017

Added a (disconnected) iOS factory for the user language model.

The factory will be used to attach a language model to a browsing session in
iOS embedders.

Bug:  762352 
Change-Id: If89eeef49f7d1268684719509ffcaf74d9dd31db
Reviewed-on: https://chromium-review.googlesource.com/672169
Commit-Queue: Michael Martis <martis@chromium.org>
Reviewed-by: David Vallet <dvallet@chromium.org>
Cr-Commit-Position: refs/heads/master@{#503085}
[modify] https://crrev.com/29018888a07c2772554821502d3aa9e9d3aa2cb9/ios/chrome/browser/language/BUILD.gn
[add] https://crrev.com/29018888a07c2772554821502d3aa9e9d3aa2cb9/ios/chrome/browser/language/language_model_factory.cc
[add] https://crrev.com/29018888a07c2772554821502d3aa9e9d3aa2cb9/ios/chrome/browser/language/language_model_factory.h
[add] https://crrev.com/29018888a07c2772554821502d3aa9e9d3aa2cb9/ios/chrome/browser/language/language_model_factory_unittest.cc

Project Member

Comment 6 by bugdroid1@chromium.org, Sep 21 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/63a99b490e1e463f2adbaa0ada07f1b0d25edcc4

commit 63a99b490e1e463f2adbaa0ada07f1b0d25edcc4
Author: Michael Martis <martis@chromium.org>
Date: Thu Sep 21 09:54:49 2017

Updated TranslateManager::GetTargetLanguage to optionally take a model.

The model argument is only populated in tests, so this patch shouldn't
affect browser behaviour.

Bug:  762352 
Change-Id: Iee3cbcf5a8401dd304eedefe6c8f9b2a235b0bb5
Reviewed-on: https://chromium-review.googlesource.com/672226
Commit-Queue: Michael Martis <martis@chromium.org>
Reviewed-by: Hajime Hoshi <hajimehoshi@chromium.org>
Cr-Commit-Position: refs/heads/master@{#503407}
[modify] https://crrev.com/63a99b490e1e463f2adbaa0ada07f1b0d25edcc4/components/translate/core/browser/BUILD.gn
[modify] https://crrev.com/63a99b490e1e463f2adbaa0ada07f1b0d25edcc4/components/translate/core/browser/translate_manager.cc
[modify] https://crrev.com/63a99b490e1e463f2adbaa0ada07f1b0d25edcc4/components/translate/core/browser/translate_manager.h
[modify] https://crrev.com/63a99b490e1e463f2adbaa0ada07f1b0d25edcc4/components/translate/core/browser/translate_manager_unittest.cc

Comment 7 by donnd@google.com, Oct 3 2017

Cc: donnd@chromium.org mahmoudi@chromium.org
Project Member

Comment 8 by bugdroid1@chromium.org, Oct 11 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/6a7ec6d0ea3c8d3105a86f67905d153aed18a62c

commit 6a7ec6d0ea3c8d3105a86f67905d153aed18a62c
Author: Michael Martis <martis@chromium.org>
Date: Wed Oct 11 01:38:07 2017

Added (disabled) plumbing to use the language model in translate.

Bug:  762352 
Change-Id: I8b7b56006d3d519abc02e34f0054cb48c6cc020e
Reviewed-on: https://chromium-review.googlesource.com/676923
Commit-Queue: Michael Martis <martis@chromium.org>
Reviewed-by: David Vallet <dvallet@chromium.org>
Reviewed-by: Hajime Hoshi <hajimehoshi@chromium.org>
Reviewed-by: Istiaque Ahmed <lazyboy@chromium.org>
Reviewed-by: Donn Denman <donnd@chromium.org>
Reviewed-by: Eugene But <eugenebut@chromium.org>
Cr-Commit-Position: refs/heads/master@{#507851}
[modify] https://crrev.com/6a7ec6d0ea3c8d3105a86f67905d153aed18a62c/chrome/browser/android/contextualsearch/contextual_search_delegate.cc
[modify] https://crrev.com/6a7ec6d0ea3c8d3105a86f67905d153aed18a62c/chrome/browser/extensions/BUILD.gn
[modify] https://crrev.com/6a7ec6d0ea3c8d3105a86f67905d153aed18a62c/chrome/browser/extensions/api/DEPS
[modify] https://crrev.com/6a7ec6d0ea3c8d3105a86f67905d153aed18a62c/chrome/browser/extensions/api/language_settings_private/language_settings_private_api.cc
[modify] https://crrev.com/6a7ec6d0ea3c8d3105a86f67905d153aed18a62c/chrome/browser/renderer_context_menu/render_view_context_menu.cc
[modify] https://crrev.com/6a7ec6d0ea3c8d3105a86f67905d153aed18a62c/chrome/browser/renderer_context_menu/render_view_context_menu.h
[modify] https://crrev.com/6a7ec6d0ea3c8d3105a86f67905d153aed18a62c/chrome/browser/translate/chrome_translate_client.cc
[modify] https://crrev.com/6a7ec6d0ea3c8d3105a86f67905d153aed18a62c/chrome/browser/translate/translate_service.cc
[modify] https://crrev.com/6a7ec6d0ea3c8d3105a86f67905d153aed18a62c/chrome/browser/translate/translate_service.h
[modify] https://crrev.com/6a7ec6d0ea3c8d3105a86f67905d153aed18a62c/chrome/browser/translate/translate_service_unittest.cc
[modify] https://crrev.com/6a7ec6d0ea3c8d3105a86f67905d153aed18a62c/components/translate/core/browser/translate_manager.cc
[modify] https://crrev.com/6a7ec6d0ea3c8d3105a86f67905d153aed18a62c/components/translate/core/browser/translate_manager.h
[modify] https://crrev.com/6a7ec6d0ea3c8d3105a86f67905d153aed18a62c/components/translate/core/browser/translate_manager_unittest.cc
[modify] https://crrev.com/6a7ec6d0ea3c8d3105a86f67905d153aed18a62c/components/translate/core/browser/translate_ui_delegate_unittest.cc
[modify] https://crrev.com/6a7ec6d0ea3c8d3105a86f67905d153aed18a62c/ios/chrome/browser/translate/chrome_ios_translate_client.mm
[modify] https://crrev.com/6a7ec6d0ea3c8d3105a86f67905d153aed18a62c/ios/web_view/internal/translate/web_view_translate_client.mm

Project Member

Comment 9 by bugdroid1@chromium.org, Oct 12 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/9d39e939e54ce0106cf0d36655faf7881062342e

commit 9d39e939e54ce0106cf0d36655faf7881062342e
Author: Michael Martis <martis@chromium.org>
Date: Thu Oct 12 08:39:35 2017

Enabled baseline model.

This CL flips the feature flag that causes translate code to
use the language model when detecting target language (rather
than heuristic logic).

In particular, this activates the "baseline" model, which
attempts to preserve legacy behavior. Future work will improve
the quality of model used.

Bug:  762352 
Change-Id: Ic278db200611a07532309c79ab53f56416270966
Reviewed-on: https://chromium-review.googlesource.com/711636
Reviewed-by: Hajime Hoshi <hajimehoshi@chromium.org>
Reviewed-by: Jia Meng <jiameng@chromium.org>
Commit-Queue: Michael Martis <martis@chromium.org>
Cr-Commit-Position: refs/heads/master@{#508285}
[modify] https://crrev.com/9d39e939e54ce0106cf0d36655faf7881062342e/chrome/browser/translate/translate_manager_render_view_host_unittest.cc
[modify] https://crrev.com/9d39e939e54ce0106cf0d36655faf7881062342e/components/language/core/browser/baseline_language_model.cc

Cc: napper@chromium.org
Owner: martis@chromium.org
Project Member

Comment 11 by bugdroid1@chromium.org, Oct 25 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/97d44d1d3fcef0841535a459a9e35d0798d4ca33

commit 97d44d1d3fcef0841535a459a9e35d0798d4ca33
Author: Michael Martis <martis@chromium.org>
Date: Wed Oct 25 03:34:09 2017

Added (disconnected) code and tests for the heuristic language model.

This model will eventually replace legacy Chrome behaviour (specificially, the
baseline language model) with more flexible logic. This new code will not have
any effect until we update our language model factories (for iOS, iOS WebView
and Content) to optionally produce a heuristic model based on the feature flag
included in this change.

Bug:  762352 
Change-Id: Ic6a942a2c1e5fa9c10785167b620aeccad72c7d5
Reviewed-on: https://chromium-review.googlesource.com/725063
Commit-Queue: Michael Martis <martis@chromium.org>
Reviewed-by: Jia Meng <jiameng@chromium.org>
Cr-Commit-Position: refs/heads/master@{#511354}
[modify] https://crrev.com/97d44d1d3fcef0841535a459a9e35d0798d4ca33/components/language/core/browser/BUILD.gn
[add] https://crrev.com/97d44d1d3fcef0841535a459a9e35d0798d4ca33/components/language/core/browser/heuristic_language_model.cc
[add] https://crrev.com/97d44d1d3fcef0841535a459a9e35d0798d4ca33/components/language/core/browser/heuristic_language_model.h
[add] https://crrev.com/97d44d1d3fcef0841535a459a9e35d0798d4ca33/components/language/core/browser/heuristic_language_model_unittest.cc

Project Member

Comment 12 by bugdroid1@chromium.org, Nov 3 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/7576f8c61fd093f85c729cb270ce16dbeb5434c7

commit 7576f8c61fd093f85c729cb270ce16dbeb5434c7
Author: Michael Martis <martis@chromium.org>
Date: Fri Nov 03 04:59:49 2017

Optionally produce the heuristic language model in Content.

 - This doesn't change any default behaviour, as the "use baseline" flag is
   enabled and the "use heuristic" flag is disabled by default.
 - We don't remove the "use baseline" flag as it is still read by the non-
   Content factories. CLs upcoming shortly will update these factories and
   remove the flag.
 - We add a ULP preference to pref_names (to match e.g. the accept language
   preference) which duplicates definitions in translate_prefs. The existing
   definitions will go away shortly ( crbug.com/777611 , where we back out of the
   translate experiment that uses them), so I think this is fine.

Bug:  762352 
Change-Id: I8a5a826bdb7529b2bab12fcef655814028d65dd8
Reviewed-on: https://chromium-review.googlesource.com/752701
Commit-Queue: Michael Martis <martis@chromium.org>
Reviewed-by: Jia Meng <jiameng@chromium.org>
Cr-Commit-Position: refs/heads/master@{#513692}
[modify] https://crrev.com/7576f8c61fd093f85c729cb270ce16dbeb5434c7/chrome/browser/language/language_model_factory.cc
[modify] https://crrev.com/7576f8c61fd093f85c729cb270ce16dbeb5434c7/chrome/browser/language/language_model_factory_unittest.cc
[modify] https://crrev.com/7576f8c61fd093f85c729cb270ce16dbeb5434c7/chrome/common/pref_names.cc
[modify] https://crrev.com/7576f8c61fd093f85c729cb270ce16dbeb5434c7/chrome/common/pref_names.h

Project Member

Comment 13 by bugdroid1@chromium.org, Nov 7 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/06a0168ad76d99c9cd637137f251cc66b7a0ed86

commit 06a0168ad76d99c9cd637137f251cc66b7a0ed86
Author: Michael Martis <martis@chromium.org>
Date: Tue Nov 07 01:57:35 2017

Optionally produce the heuristic language model in iOS.

- This doesn't change any default behaviour, as the "use baseline" flag is
  enabled and the "use heuristic" flag is disabled by default.
- We add a ULP preference to pref_names (to match e.g. the accept language
  preference) which duplicates definitions in translate_prefs. The existing
  definitions will go away shortly ( crbug.com/777611 , where we back out of the
  translate experiment that uses them), so I think this is fine.

Bug:  762352 
Change-Id: Id0a3b57089365c930cc139061d9a36780c4f8f42
Reviewed-on: https://chromium-review.googlesource.com/752965
Reviewed-by: Eugene But <eugenebut@chromium.org>
Reviewed-by: Jia Meng <jiameng@chromium.org>
Commit-Queue: Michael Martis <martis@chromium.org>
Cr-Commit-Position: refs/heads/master@{#514349}
[modify] https://crrev.com/06a0168ad76d99c9cd637137f251cc66b7a0ed86/components/language/core/browser/BUILD.gn
[modify] https://crrev.com/06a0168ad76d99c9cd637137f251cc66b7a0ed86/components/language/core/browser/baseline_language_model.cc
[modify] https://crrev.com/06a0168ad76d99c9cd637137f251cc66b7a0ed86/components/language/core/browser/baseline_language_model.h
[add] https://crrev.com/06a0168ad76d99c9cd637137f251cc66b7a0ed86/components/language/core/browser/pref_names.cc
[add] https://crrev.com/06a0168ad76d99c9cd637137f251cc66b7a0ed86/components/language/core/browser/pref_names.h
[modify] https://crrev.com/06a0168ad76d99c9cd637137f251cc66b7a0ed86/ios/chrome/browser/language/language_model_factory.cc
[modify] https://crrev.com/06a0168ad76d99c9cd637137f251cc66b7a0ed86/ios/chrome/browser/language/language_model_factory_unittest.cc
[modify] https://crrev.com/06a0168ad76d99c9cd637137f251cc66b7a0ed86/ios/web_view/internal/language/web_view_language_model_factory.cc

Project Member

Comment 14 by bugdroid1@chromium.org, Nov 9 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/459de498443ab25b40c2ded233f17ee06304d3b1

commit 459de498443ab25b40c2ded233f17ee06304d3b1
Author: Michael Martis <martis@chromium.org>
Date: Thu Nov 09 21:35:27 2017

Added a heuristic model experiment field trial config.

Bug:  762352 
Change-Id: I4120c6c5490fb9f92ca12d326170f7690000cb9d
Reviewed-on: https://chromium-review.googlesource.com/759708
Reviewed-by: Steven Holte <holte@chromium.org>
Commit-Queue: Michael Martis <martis@chromium.org>
Cr-Commit-Position: refs/heads/master@{#515298}
[modify] https://crrev.com/459de498443ab25b40c2ded233f17ee06304d3b1/testing/variations/fieldtrial_testing_config.json

Project Member

Comment 15 by bugdroid1@chromium.org, Jan 10 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/473b46b5b0cf5d924d3079fd069892d7b218b7dc

commit 473b46b5b0cf5d924d3079fd069892d7b218b7dc
Author: Michael Martis <martis@chromium.org>
Date: Wed Jan 10 02:02:32 2018

Added beta testing fieldtrial config for the heuristic language model.

This is required to roll out the heuristic language model to 50% on beta.

Bug:  762352 
Change-Id: I534e72a03f100d603b5d600eb2c5335dda67be60
Reviewed-on: https://chromium-review.googlesource.com/855798
Reviewed-by: Steven Holte <holte@chromium.org>
Commit-Queue: Michael Martis <martis@chromium.org>
Cr-Commit-Position: refs/heads/master@{#528207}
[modify] https://crrev.com/473b46b5b0cf5d924d3079fd069892d7b218b7dc/testing/variations/fieldtrial_testing_config.json

Project Member

Comment 16 by bugdroid1@chromium.org, Jan 31 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/452ac34e7cb628e35d0d2799eb2622ac33e49a63

commit 452ac34e7cb628e35d0d2799eb2622ac33e49a63
Author: Michael Martis <martis@chromium.org>
Date: Wed Jan 31 01:26:58 2018

Added stable testing fieldtrial config for the heuristic language model.

This is required to roll out the heuristic language model to 1% on stable.

Bug:  762352 
Change-Id: Iba29ae1e9335b38e853f46abd5e267f16705c786
Reviewed-on: https://chromium-review.googlesource.com/894743
Reviewed-by: Ilya Sherman <isherman@chromium.org>
Commit-Queue: Ilya Sherman <isherman@chromium.org>
Cr-Commit-Position: refs/heads/master@{#533144}
[modify] https://crrev.com/452ac34e7cb628e35d0d2799eb2622ac33e49a63/testing/variations/fieldtrial_testing_config.json

Owner: yyushkina@chromium.org
Status: Fixed (was: Assigned)

Sign in to add a comment