New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 597488 link

Starred by 49 users

Issue metadata

Status: Fixed
Owner:
Closed: Sep 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Feature

Blocked on:
issue 583549
issue 583564

Blocking:
issue 467061
issue 679294

Restricted
  • Only users with EditIssue permission may comment.



Sign in to add a comment

Remove encoding menu

Project Member Reported by jinsuk...@chromium.org, Mar 24 2016

Issue description

This is a part of the effort Project Eraser. Encoding-related UI will go away.

Quoted from email thread:

1) "Auto Detect" option in the hamburger menu.  It's a sticky global boolean that turns on a heavy text analyzer to guess the encoding better.  It's off by default because it regresses page load time by 10%-20%.  By selecting this, users see less gibberish but they make Chrome slower (and don't realize that).

2) Manual encoding selection in the hamburger menu.  This is a temporary setting that forces the current tab to the specified encoding, no matter what.  It will turn pages into gibberish if the user selects the wrong one.

3) "Default encoding" selector buried in chrome://settings.  This specifies which encoding is selected if "Auto Detect" is disabled and the web page doesn't specify its encoding.  It defaults to the UI language of the Chrome installation.
 
Status: Started (was: Assigned)

Comment 2 by tkent@chromium.org, Mar 24 2016

Cc: kenjibaheux@chromium.org
Labels: Hotlist-Japan
Labels: Needs-Feedback
It's not clear what is intended beyond the removal.

Surely, we are not thinking that we can just remove it and provide no other alternative than:
 1. give up on the being able to read the page
 2. switch browsers
Blockedon: 583564
Sorry if this bug took you by surprise. Please check out the result of the experiment (crbug.com/518968#c29, conducted on Android platform) to get the stat on the pages missing the encoding information.

Another stat shows that overall usage of the encoding menu is extremely low. https://docs.google.com/spreadsheets/d/1Em5Pk4PKeB3rBbnQoZh3MSQiRRZfdsLD0r45DuplUXU/edit#gid=2144179651 which led to putting up the menu for the candidate of the Project eraser(go/project-eraser). I can see that Japan/US is an exception in terms of the menu usage though.

The removal of the menu will certainly be linked to engineering efforts to mitigate potential impact, such as crbug.com/583564  crbug.com/583549 . The intent is to keep the core part of Chrome slim while taking care of exceptions with other methods like extension.


Comment 5 by js...@chromium.org, Mar 24 2016

Cc: js...@chromium.org
Components: UI>Internationalization
I'd rather be extremely careful. 

The encoding override is currently broken in that it affects not just the current page encoding but also any page after that in the same tab.  This is a long standing bug. When this is fixed, there's certainly a use case for the manual override. 

Nonetheless, there is a case where the encoding override is necessary (despite our best effort, we can get the encoding wrong. It's rarer and rarer). Although it'd be rare, it can be extremely frustrating when this happens. 

An example: we cannot associate 'com' domain with any encoding. So, it'll be the 'default' encoding (when auto-detection is OFF) or 'auto-detected' encoding (when auto-detection is ON).  I'm assuming that the page is NOT UTF-8. 

Even until recently (or even now), nytimes.com used(uses) windows-1252 WITHOUT charset declaration. ( I've asked them to fix it multiple times). My default encoding is set to Korean (windows-949/euc-kr). When I visit nytimes.com, I see gibberish here and there when they use non-ASCII characters. The same is true of a number of European pages. 

I can either override manually or turn on the encoding detector. With the encoding menu gone, I can't do that. 

Another use case of encoding menu is  to check the encoding of the current page (for developer or troubleshooting) quickly. 

As for the default encoding setting menu, I oppose its removal. UMA stat does not tell the whole story. Once it's set to the value a user wants, they don't have to visit it any more  (unless they're QA folks). So, just because it's extremely rarely used we can't simply dismiss it as not useful.  

Comment 6 by js...@chromium.org, Mar 24 2016

BTW, is there any reason for 'Restrict-View-Google'? 
Labels: -Needs-Feedback
Thanks for the quick reply.
I'm relieved to hear that there are follow-ups items.

Also, +1 to jshin's comments about being careful.

I gave a quick read at the stats in the other bug but it's unclear to me if the approach covers the whole issue. For instance, a lot of these pages where the encoding menu is being used are "old-school" desktop websites, they wouldn't rank very high on mobile search or even be a plausible hit for a mobile user story.


Overall, I agree that this menu is really awful.
A better solution is definitely welcomed :)
Labels: -Type-Bug Type-Feature
jshin/kenjibaheux@'s concern is certainly valid. Though not as articulated as yours, mine is not much different crbug.com/518968#c36.

Just for the sake of advocating the project Eraser (and at the risk of repeating what you're already aware of), I think the questions the project asks are 1) whether a feature in question is essential, appreciated by majority of users 2) whether there is a way to make up for it for those in need of the feature. The answers to the questions wrt encoding-related menu are 1-no and 2- yes from my point of view. I think it possible to write a Chrome extension that lets users to detect/set default encoding, though the technical detail may need more research. Better if we could help them figure out how to install it.

Anyway there's no need to do it in a rush. The menu will stay till we come up with a better solution.

> BTW, is there any reason for 'Restrict-View-Google'? 

It's just because I mentioned a project and a link to a doc internal to Google. 

Comment 9 by aelias@chromium.org, Mar 24 2016

There is another option "3. Install an extension to change the encoding".  Because the encoding menu is only used by a subset of power users, it's the perfect use case for an extension.

I believe it shouldn't be too difficult to write an extension that parses the page and converts every character to a readable encoding.  I expect such an extension to be written by a Japanese user shortly after we remove the feature, likewise in any other region that feels it's useful (I'm not sure there are really any other regions that really care, our stats only show a usage spike in JP).

> With the encoding menu gone, I can't do that. 

You should keep in mind that the majority of our users don't know about this menu and wouldn't understand what it means if they did.  If we show a gibberish encoding, it's a problem regardless and this menu is not a true solution.  It's exactly the kind of UX-complexity-adding half-solution that Eraser aims to remove.
I whipped up a proof-of-concept Chrome extension that forces the correct encoding on http://www.itojun.org/paper/keio-doctor97.html (a page using legacy ISO-2022-JP that none of our heuristics will fix).  I uploaded it to https://github.com/alexelias/encoding-changer/ and the meat is the 6 lines of header injection in https://github.com/alexelias/encoding-changer/blob/master/src/bg/background.js .

This proves it should be easy to write an extension that exactly reproduces the native Chrome menu (or is even more useful, for example via sticky domain-level encoding settings).
This is great. Thanks Alex!
There doesn't seem to be general agreement that this is the right thing to be doing.

Before we go about submitting code to remove the auto-detect option (
https://codereview.chromium.org/1894913002/), it would be good to have everyone on board with it.  Are jshin@ and kenjibaheux@'s comments only applicable to the manual encoding option?

Is everyone in agreement that the auto-detect option isn't helpful?  The enabled rates actually seem quite high in many countries, and I suspect they are the ones that would be most affected.
For the autodetect checkbox, the problem with it is that it can slow down page loads by 20%+ sometimes.  That's why it's not the default.  Given the user the option to enable/disable it is a bit absurd, it's picking your poison between sometimes see gibberish encoding, or slow down a lot of page loads usually to no benefit whatsoever (the autodetector usually does not change encoding outcome according to our data, even in the best countries).  So we want to solve the problem as much as possible by default with much faster autodetection heuristics like https://codereview.chromium.org/1721373002/ and 
https://codereview.chromium.org/1725283002/ that we simply provide for everyone by default, and then there's nothing to have an option for.

I realize it's hard to reconstruct the story from the discussions on various bugs, so Jinsuk will write a brief doc that explains the rationale for removal.
> I realize it's hard to reconstruct the story from the discussions on various bugs, so Jinsuk will write a brief doc that explains the rationale for removal.

Here it is.  Feel free to add comments to the doc if you have remaining questions: https://docs.google.com/document/d/1f9hqF_aLesnfaj9JKQLRCnwECY4fq3Q3udlT1saE5-g/edit?ts=57162529#
Cc: -kenjibaheux@chromium.org

Comment 16 by js...@chromium.org, Apr 29 2016

There are 3 items proposed for removal in the initial bug report. 

Of those three, IMHO, #3 (default encoding preference setting in Settings) should certainly stay. As I wrote before, how often it is used is NOT a good measure of its usefulness at all. Once it's set to the value a user wants, (s)he will never go there again. So, the usage frequency will be extremely low, but it can be critical for some users. 

#2 (encoding menu to show the current encoding and let users override). Given the following : 

1. overriding has been broken for a long while (it affects all the subsequent pages as well as the current page) 
2. some overriding is regarded as a security risk 
3. With recent changes to cut down the frequency of misdetection,

we might as well consider making an extension as suggested earlier while we keep a way to show what the current page encoding is. For the latter, we can just have 'encoding display' in the current location or we can move it to DOM inspector (for geekish users). 

#1 (autodetect): We need to replace the current detector with a much faster one and consider turning it on by default. 
OK, I think we have a consensus that we can immediately remove manual detection #2, and we were not currently intending to remove default setting #3 anyway.

This leaves a lack of consensus on the question of how to handle autodetect checkbox #1.  After #2 is removed, I don't think it's sane to leave that checkbox dangling in the top-level hotdog menu on its own -- it clearly wouldn't belong.  There are three options I can think of:
A. Bundle a new fast autodetector with Chrome to replace ICU's, then remove checkbox with zero regression.  jshin@ has raised the idea that we could open-source Google's internal encoding autodetector CED and bundle it with Chromium.  jshin@ reports that CED is more efficient and there should be no significant obstacle to open-sourcing it (it was already slated for open-sourcing).
B. Keep the autodetect checkbox, but bury it in chrome://settings alongside the default encoding selector #3.
C. Simply remove the autodetect checkbox and remove the ICU autodetect code, considering our UTF-8 and domain-based heuristics as having sufficiently mitigated the problem.

I still think C is a reasonable path, but we'll take a look at CED and understand how challenging it is to open-source and integrate into Chromium.  If it's mostly trivial, then OK.  One question I have about it in particular is how much binary size it will occupy in our APK.
Made an tentative patch to replace ICU with CED. The release apk size is increased by 90KB (55,290,354 -> 55,384,458). May increase/decrease a bit more but that would be the magnitude we can expect.

page_cycler benchmark on Android:
page_cycler.typical_25:cold_times.page_load_time 10.62±16.00% -> 11.31±15.79% (-6.49%)
page_cycler.typical_25 (2016-05-04 06:29:45)page_cycler.typical_25 9801.37±20.12% -> 10223.47±19.89% (-4.31%)

Benchmark on Linux reports about +1.7%, +1.9% improvment http://storage.googleapis.com/chromium-telemetry/html-results/results-2016-05-04_17-30-38 - the diffs are too small to mean anything.

I can see that CED performs better than ICU. 




Thank you, aelias@ for summary and Jinsuk for testing CED. 

> page_cycler.typical_25

Did you remove charset declaration from all the html pages in this set? Otherwise, the encoding detection wouldn't kick in (as you're well aware). 

And you compared the page load time with CED OFF and CED ON, didn't you?  If so, I don't see any significant difference between two (delta << sigma). Can you share the raw data (in spreadsheet)?  

Anyway, my preference is as following: 1) replace ICU detector with CED 2) move the UI to turn on or off the encoding detection to Settings next to the 'default encoding' (on desktop, 

BTW, one group of users who'd benefit most from the autodetection is (besides Japanese or Russian for which there are multiple legacy encodings) multilingual speakers (especially when legacy encodings for those languages 'conflict' with each other;  e.g. Finnish + Russian, Russian + Vietnamese, Arabic + French,  etc). 



> Did you remove charset declaration from all the html pages in this set? Otherwise, the encoding detection wouldn't kick in (as you're well aware). 

Thanks for the reminder. I simply modified TextResourceDecoder (had shouldDetectEncoding() return true) for ced to run, but realized that it was not enough since Chrome has auto detection option off by default (OTH it worked fine on Android because I ran it locally - perf bot didn't run for Android at this moment - with the option turned on by myself manually). That explains why Linux showed no difference. Running Linux perf bot again with the correct changes https://codereview.chromium.org/1956183002/ gives following result:


page_cycler.typical_25:cold_times.page_load_time  740.31 ± 10.55% -> 764.38 ± 11.78% (-3.25%)
page_cycler.typical_25:warm_times.page_load_time  609.88 ± 11.32% -> 667.36 ± 15.81% (-9.43%)

http://storage.googleapis.com/chromium-telemetry/html-results/results-2016-05-07_17-03-28

Android result is attached. I only have the averages for 25 sites returned by the benchmark, not the raw data for each page load.
results-android.html
733 KB View Download
Oops, the first one is the result with patch (force-enabled CED) and the latter is TOT (no auto-detection). So the result in https://bugs.chromium.org/p/chromium/issues/detail?id=597488#c20 indicates CED is faster than TOT, which doesn't mean much since the delta is too smaller than sigma as you pointed out.

Ran a new pair of tests for clarification.

http://storage.googleapis.com/chromium-telemetry/html-results/results-2016-05-08_20-37-54 (CED auto-detection vs. TOT)
page_cycler.typical_25:cold_times.page_load_time ms 705.70±9.49% vs. 760.31±11.90% (-7.74%)

http://storage.googleapis.com/chromium-telemetry/html-results/results-2016-05-08_21-20-58 (ICU auto-detection vs. TOT)
page_cycler.typical_25:cold_times.page_load_time 1085.13±9.31% 754.28±12.03% (30.49%)

It shows that ICU is much slower while CED introduces virtually no additional load.
Project Member

Comment 22 by bugdroid1@chromium.org, Jun 21 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/21d3369fc5ab4d70732f4d83388ad2e144aaebcb

commit 21d3369fc5ab4d70732f4d83388ad2e144aaebcb
Author: jinsukkim <jinsukkim@chromium.org>
Date: Tue Jun 21 06:16:03 2016

Initial repo setup for third party library CED

Initializes src/third_party/ced for CED, open-source library used to
detect texting encoding automatically.

This library is intended to replace ICU used in Blink for
automatic text encoding detection feature. The feature,
when enabled, helps web pages coming without encoding label
displayed as expected instead of showing gibberish.

The feature is disabled by default despite its usefulness,
since it slows down the page loading for those unlabeled ones
(about 2% of total pages) down to 20~30% in some cases. It is
left to users to turn it on via UI.

CED is used extensively in Google, and is a better
alternative in terms of accuracy, speed, size, etc.
Experiments show that CED introduces no significant load.
With CED in place, it is possible that automatic encoding
detection can be turned on by default without introducing
regression in page loading time. This will be particularly
beneficial on mobile platform like Android where UI has more
limitations. It will just work without individual users
having to turn it on.

BUG= 597488 

Review-Url: https://codereview.chromium.org/2047993002
Cr-Commit-Position: refs/heads/master@{#400906}

[modify] https://crrev.com/21d3369fc5ab4d70732f4d83388ad2e144aaebcb/.gitignore
[modify] https://crrev.com/21d3369fc5ab4d70732f4d83388ad2e144aaebcb/DEPS
[add] https://crrev.com/21d3369fc5ab4d70732f4d83388ad2e144aaebcb/third_party/ced/BUILD.gn
[add] https://crrev.com/21d3369fc5ab4d70732f4d83388ad2e144aaebcb/third_party/ced/LICENSE
[add] https://crrev.com/21d3369fc5ab4d70732f4d83388ad2e144aaebcb/third_party/ced/OWNERS
[add] https://crrev.com/21d3369fc5ab4d70732f4d83388ad2e144aaebcb/third_party/ced/README.chromium
[add] https://crrev.com/21d3369fc5ab4d70732f4d83388ad2e144aaebcb/third_party/ced/ced.gyp

Project Member

Comment 23 by bugdroid1@chromium.org, Jun 23 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/fdad2babe0c8940a094e0b77b1ac75bbb2fbc819

commit fdad2babe0c8940a094e0b77b1ac75bbb2fbc819
Author: jinsukkim <jinsukkim@chromium.org>
Date: Thu Jun 23 01:04:58 2016

[ced] Suppress a compiler warning on Windows

Added a new option suppressing MSVC compiler warning:

'<': signed/unsigned mismatch

for the places where a signed int is used for a loop variable
against unsigned int upper limit. This is necessary for Windows build.

BUG= 597488 

Review-Url: https://codereview.chromium.org/2092533003
Cr-Commit-Position: refs/heads/master@{#401492}

[modify] https://crrev.com/fdad2babe0c8940a094e0b77b1ac75bbb2fbc819/third_party/ced/BUILD.gn

Comment 24 by yosin@chromium.org, Jun 23 2016

Cc: yosin@chromium.org
Jinsukkim: Would  issue 233002  go away with your work? Feel free to mark as blocked on and close as wontfix when you launch.
The bug itself was already fixed - it does not call auto detector repeatedly any more once the encoding is determined. The auto detector uses the first chunk only as input.

But the statement "If the beginning of a file is ASCII-only and only a subsequent chunk has non-ASCII 'bytes', we can break those pages." is still valid. I've been bitten by this problem https://bugs.chromium.org/p/chromium/issues/detail?id=609053#c10  when attempting to switch the encoding from latin1 to UTF-8 for such a document using UTF-8 encoding detector. I think this better be tracked by a new bug entry and close the  issue 233002 .




Project Member

Comment 27 by bugdroid1@chromium.org, Jun 26 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/dc3932694326d603b9b40dec5f0f6fcf463eaa65

commit dc3932694326d603b9b40dec5f0f6fcf463eaa65
Author: jinsukkim <jinsukkim@chromium.org>
Date: Sun Jun 26 22:06:41 2016

Rolling DEPS for third_party/ced

This updates add a new utility method used to do string-to-enum
conversion for text encoding name.

BUG= 597488 
TEST=gclient sync, make

Review-Url: https://codereview.chromium.org/2095583002
Cr-Commit-Position: refs/heads/master@{#402088}

[modify] https://crrev.com/dc3932694326d603b9b40dec5f0f6fcf463eaa65/DEPS

Project Member

Comment 28 by bugdroid1@chromium.org, Jun 29 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/13510755ac11e6e1f58c34ef8c9bd4cf925c8d70

commit 13510755ac11e6e1f58c34ef8c9bd4cf925c8d70
Author: jinsukkim <jinsukkim@chromium.org>
Date: Wed Jun 29 00:45:26 2016

Replace ICU with CED for auto encoding detection

This is a drop-in replacement of ICU library performing automatic text
encoding detection with CED (Compact Encdoing Detection).

CED is used extensively in Google for every crawled web page,
email message, query string, etc., and recently open-sourced for
public use. (https://github.com/google/compact_enc_det)

Also it is a much better alternative to ICU in terms of speed.
ICU introduces significant regression in page loading (up to 30%):

= ICU auto-detection vs. TOT =
page_cycler.typical_25:cold_times.page_load_time 1085.13±9.31% 754.28±12.03% (30.49%)

http://storage.googleapis.com/chromium-telemetry/html-results/results-2016-05-08_21-20-58

while CED adds virtually no additional loading time (delta < sigma):

= CED auto-detection vs. TOT =
page_cycler.typical_25:cold_times.page_load_time ms 705.70±9.49% vs. 760.31±11.90% (-7.74%)

http://storage.googleapis.com/chromium-telemetry/html-results/results-2016-05-08_20-37-54

With CED, it is feasible to turn on auto encoding detection by default
so that web pages without encoding label can be taken care of. It will be
done in a follow-up CL.

BUG= 597488 

Review-Url: https://codereview.chromium.org/2081653007
Cr-Commit-Position: refs/heads/master@{#402622}

[modify] https://crrev.com/13510755ac11e6e1f58c34ef8c9bd4cf925c8d70/third_party/WebKit/Source/platform/BUILD.gn
[modify] https://crrev.com/13510755ac11e6e1f58c34ef8c9bd4cf925c8d70/third_party/WebKit/Source/platform/DEPS
[modify] https://crrev.com/13510755ac11e6e1f58c34ef8c9bd4cf925c8d70/third_party/WebKit/Source/platform/blink_platform.gyp
[modify] https://crrev.com/13510755ac11e6e1f58c34ef8c9bd4cf925c8d70/third_party/WebKit/Source/platform/text/TextEncodingDetector.cpp

Project Member

Comment 29 by bugdroid1@chromium.org, Jun 29 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/e6ea446fa4e5628f61f35e5fe88e40cbb0d0b924

commit e6ea446fa4e5628f61f35e5fe88e40cbb0d0b924
Author: shans <shans@chromium.org>
Date: Wed Jun 29 02:00:29 2016

Revert of Replace ICU with CED for auto encoding detection (patchset #4 id:120001 of https://codereview.chromium.org/2081653007/ )

Reason for revert:
This seems to have broken compile on Windows 8:

[46/8396] CXX obj\third_party\ced\src\compact_enc_det\ced.compact_enc_det_hint_code.obj
FAILED: obj/third_party/ced/src/compact_enc_det/ced.compact_enc_det_hint_code.obj
ninja -t msvc -e environment.x86 -- C:\b\build\slave\cache\cipd\goma/gomacc "C:\b\depot_tools\win_toolchain\vs_files\95ddda401ec5678f15eeed01d2bee08fcbc5ee97\VC\bin\amd64_x86\cl.exe" /nologo /showIncludes /FC @obj\third_party\ced\src\compact_enc_det\ced.compact_enc_det_hint_code.obj.rsp /c ..\..\third_party\ced\src\compact_enc_det\compact_enc_det_hint_code.cc /Foobj\third_party\ced\src\compact_enc_det\ced.compact_enc_det_hint_code.obj /Fdobj\third_party\ced\ced.cc.pdb
c:\b\build\slave\win8_gyp\build\src\third_party\ced\src\compact_enc_det\compact_enc_det_hint_code.cc(122): error C2220: warning treated as error - no 'object' file generated
c:\b\build\slave\win8_gyp\build\src\third_party\ced\src\compact_enc_det\compact_enc_det_hint_code.cc(122): warning C4018: '<': signed/unsigned mismatch
c:\b\build\slave\win8_gyp\build\src\third_party\ced\src\compact_enc_det\compact_enc_det_hint_code.cc(151): warning C4018: '<': signed/unsigned mismatch
c:\b\build\slave\win8_gyp\build\src\third_party\ced\src\compact_enc_det\compact_enc_det_hint_code.cc(169): warning C4018: '<': signed/unsigned mismatch

https://build.chromium.org/p/chromium.win/builders/Win8%20GYP/builds/89

Original issue's description:
> Replace ICU with CED for auto encoding detection
>
> This is a drop-in replacement of ICU library performing automatic text
> encoding detection with CED (Compact Encdoing Detection).
>
> CED is used extensively in Google for every crawled web page,
> email message, query string, etc., and recently open-sourced for
> public use. (https://github.com/google/compact_enc_det)
>
> Also it is a much better alternative to ICU in terms of speed.
> ICU introduces significant regression in page loading (up to 30%):
>
> = ICU auto-detection vs. TOT =
> page_cycler.typical_25:cold_times.page_load_time 1085.13±9.31% 754.28±12.03% (30.49%)
>
> http://storage.googleapis.com/chromium-telemetry/html-results/results-2016-05-08_21-20-58
>
> while CED adds virtually no additional loading time (delta < sigma):
>
> = CED auto-detection vs. TOT =
> page_cycler.typical_25:cold_times.page_load_time ms 705.70±9.49% vs. 760.31±11.90% (-7.74%)
>
> http://storage.googleapis.com/chromium-telemetry/html-results/results-2016-05-08_20-37-54
>
> With CED, it is feasible to turn on auto encoding detection by default
> so that web pages without encoding label can be taken care of. It will be
> done in a follow-up CL.
>
> BUG= 597488 
>
> Committed: https://crrev.com/13510755ac11e6e1f58c34ef8c9bd4cf925c8d70
> Cr-Commit-Position: refs/heads/master@{#402622}

TBR=tkent@chromium.org,jshin@chromium.org,jinsukkim@chromium.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG= 597488 

Review-Url: https://codereview.chromium.org/2109633003
Cr-Commit-Position: refs/heads/master@{#402652}

[modify] https://crrev.com/e6ea446fa4e5628f61f35e5fe88e40cbb0d0b924/third_party/WebKit/Source/platform/BUILD.gn
[modify] https://crrev.com/e6ea446fa4e5628f61f35e5fe88e40cbb0d0b924/third_party/WebKit/Source/platform/DEPS
[modify] https://crrev.com/e6ea446fa4e5628f61f35e5fe88e40cbb0d0b924/third_party/WebKit/Source/platform/blink_platform.gyp
[modify] https://crrev.com/e6ea446fa4e5628f61f35e5fe88e40cbb0d0b924/third_party/WebKit/Source/platform/text/TextEncodingDetector.cpp

Project Member

Comment 30 by bugdroid1@chromium.org, Jun 29 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/0f5f22c2b38dfbbd0d4fd49146ef636b6480f86b

commit 0f5f22c2b38dfbbd0d4fd49146ef636b6480f86b
Author: jinsukkim <jinsukkim@chromium.org>
Date: Wed Jun 29 05:09:17 2016

[ced] Suppress a compiler warning on Windows

Add a new option suppressing MSVC compiler warning:

'<': signed/unsigned mismatch  (/wd4018)

where a signed int is used for a loop variable against
unsigned int upper limit. This is a prerequisite to
hooking ced to Blink. The change to gn build file was
already made. This CL applies the same change to gyp.

BUG= 597488 

Review-Url: https://codereview.chromium.org/2103793005
Cr-Commit-Position: refs/heads/master@{#402728}

[modify] https://crrev.com/0f5f22c2b38dfbbd0d4fd49146ef636b6480f86b/third_party/ced/ced.gyp

Project Member

Comment 31 by bugdroid1@chromium.org, Jun 29 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/0c155762b0654045ed3031e5d1e2b447e40ca42e

commit 0c155762b0654045ed3031e5d1e2b447e40ca42e
Author: jinsukkim <jinsukkim@chromium.org>
Date: Wed Jun 29 16:13:47 2016

Reland "Replace ICU with CED for auto encoding detection"

The patch was reverted due to broken build on Windows 8 gyp.
gyp build was missing a compiler option suppressing a warning
message while gn build already handles it. crrev.com/2103793005
now fixed the issue.

This reverts commit e6ea446fa4e5628f61f35e5fe88e40cbb0d0b924.

BUG= 597488 

Review-Url: https://codereview.chromium.org/2105263002
Cr-Commit-Position: refs/heads/master@{#402832}

[modify] https://crrev.com/0c155762b0654045ed3031e5d1e2b447e40ca42e/third_party/WebKit/Source/platform/BUILD.gn
[modify] https://crrev.com/0c155762b0654045ed3031e5d1e2b447e40ca42e/third_party/WebKit/Source/platform/DEPS
[modify] https://crrev.com/0c155762b0654045ed3031e5d1e2b447e40ca42e/third_party/WebKit/Source/platform/blink_platform.gyp
[modify] https://crrev.com/0c155762b0654045ed3031e5d1e2b447e40ca42e/third_party/WebKit/Source/platform/text/TextEncodingDetector.cpp

Project Member

Comment 32 by bugdroid1@chromium.org, Jun 29 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/dbd9f219ef9b615d9e116964e46b1a390c9354a1

commit dbd9f219ef9b615d9e116964e46b1a390c9354a1
Author: petewil <petewil@chromium.org>
Date: Wed Jun 29 19:37:38 2016

Revert of Reland "Replace ICU with CED for auto encoding detection" (patchset #1 id:1 of https://codereview.chromium.org/2105263002/ )

Reason for revert:
Sheriff is reverting this patchset on suspicion of
adding a new static constructor (string) causing a failure of the "sizes" step
of the build.

Link to broken build:
https://build.chromium.org/p/chromium/builders/Linux%20x64/builds/21618

Relevant log lines:
FAILED linux-release-64/sizes/chrome-si/initializers: actual 9, expected 7,
better lower

We think this is ultimately coming from encodings.cc which is part of the third party ced package, and might be related to this DEPS roll.
revert_cq: 1
revert_reason_textarea: A revert of this CL (patchset #1 id:1) has been created in
https://codereview.chromium.org/2110853002/ by petewil@chromium.org.

The reason for reverting is: Sheriff is reverting this patchset on suspicion of
adding a new static constructor (string) causing a failure of the "sizes" step
of the build.

Link to broken build:
https://build.chromium.org/p/chromium/builders/Linux%20x64/builds/21618

Relevant log lines:
FAILED linux-release-64/sizes/chrome-si/initializers: actual 9, expected 7,
better lower

We think this is ultimately coming from encodings.cc which is part of the third party ced package, and might be related to this DEPS roll.

Original issue's description:
> Reland "Replace ICU with CED for auto encoding detection"
>
> The patch was reverted due to broken build on Windows 8 gyp.
> gyp build was missing a compiler option suppressing a warning
> message while gn build already handles it. crrev.com/2103793005
> now fixed the issue.
>
> This reverts commit e6ea446fa4e5628f61f35e5fe88e40cbb0d0b924.
>
> BUG= 597488 
>
> Committed: https://crrev.com/0c155762b0654045ed3031e5d1e2b447e40ca42e
> Cr-Commit-Position: refs/heads/master@{#402832}

TBR=tkent@chromium.org,jinsukkim@chromium.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG= 597488 

Review-Url: https://codereview.chromium.org/2110833005
Cr-Commit-Position: refs/heads/master@{#402901}

[modify] https://crrev.com/dbd9f219ef9b615d9e116964e46b1a390c9354a1/third_party/WebKit/Source/platform/BUILD.gn
[modify] https://crrev.com/dbd9f219ef9b615d9e116964e46b1a390c9354a1/third_party/WebKit/Source/platform/DEPS
[modify] https://crrev.com/dbd9f219ef9b615d9e116964e46b1a390c9354a1/third_party/WebKit/Source/platform/blink_platform.gyp
[modify] https://crrev.com/dbd9f219ef9b615d9e116964e46b1a390c9354a1/third_party/WebKit/Source/platform/text/TextEncodingDetector.cpp

Project Member

Comment 33 by bugdroid1@chromium.org, Jul 5 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/7d3f4cfeee19896159673226a001bc7776fb8344

commit 7d3f4cfeee19896159673226a001bc7776fb8344
Author: jinsukkim <jinsukkim@chromium.org>
Date: Tue Jul 05 04:58:37 2016

Rolling DEPS for third_party/ced

This update removes static initializers which should be avoided
in chrome build.

BUG= 597488 
TEST=gclient sync, make

Review-Url: https://codereview.chromium.org/2120503002
Cr-Commit-Position: refs/heads/master@{#403740}

[modify] https://crrev.com/7d3f4cfeee19896159673226a001bc7776fb8344/DEPS

Reland of revert from #32 was done here:
https://codereview.chromium.org/2138643002/

Comment 35 by js...@chromium.org, Jul 19 2016

Thank you for adding CED !

A couple of issues in CED. 

1. Encoding names are different from HTML5/WHATWG encoding specs. Blink has encoding aliases to make up for those differences, but we can standardize the encoding name in CED (at least when it's built with 'HTML5' defined, which we need to introduce to CED). 

2. 7-bit encoidng: Blink only supports ISO-2022-JP. The detection of other 7-bit encodings and other non-HTML5 encodings should be disabled in CED (again in 'HTML5' mode). 

I wanted to file these issues at github, but there's no issues tab. Do you plan to track bug for CED in monorail, instead?  

Thanks for the feedback. I think it is better to track upcoming issues on github as it has been open-sourced. Just created an 'issues' tab. 

Honestly I do not have enough knowledge on inner workings of CED. I'm planning to take care of other topics around build/configuration.
Just added the two issues you raised to https://github.com/google/compact_enc_det/issues
Project Member

Comment 38 by bugdroid1@chromium.org, Jul 22 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/f0829bf6d80a9109b399580fe48d8c3e1c66eeed

commit f0829bf6d80a9109b399580fe48d8c3e1c66eeed
Author: jinsukkim <jinsukkim@chromium.org>
Date: Fri Jul 22 03:11:25 2016

Android: turn on auto-detect encoding by default

Now that CED (Compact Encoding Detection) library replaced
ICU (crrev.com/2081653007), turning on auto-detect encoding
by default became feasible without regression on page
loading time. Also removed the encoding menu in site
setting (originally added in crrev.com/1248163002).

Will remove the menu on the other platforms in a follow-up
CL.

BUG= 597488 

Review-Url: https://codereview.chromium.org/1894913002
Cr-Commit-Position: refs/heads/master@{#407005}

[rename] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/android/java/res/drawable-hdpi/settings_translate.png
[rename] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/android/java/res/drawable-mdpi/settings_translate.png
[rename] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/android/java/res/drawable-xhdpi/settings_translate.png
[rename] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/android/java/res/drawable-xxhdpi/settings_translate.png
[rename] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/android/java/res/drawable-xxxhdpi/settings_translate.png
[delete] https://crrev.com/79f7b784a97cbb22f11064a05b621b0def87eab3/chrome/android/java/res/xml/language_preferences.xml
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/android/java/res/xml/site_settings_preferences.xml
[add] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/android/java/res/xml/translate_preferences.xml
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/android/java/src/org/chromium/chrome/browser/preferences/PrefServiceBridge.java
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/android/java/src/org/chromium/chrome/browser/preferences/website/SiteSettingsPreferences.java
[rename] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/android/java/src/org/chromium/chrome/browser/preferences/website/TranslatePreferences.java
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/android/java/strings/android_chrome_strings.grd
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/android/java_sources.gni
[add] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/android/javatests/src/org/chromium/chrome/browser/EncodingDetectionTest.java
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/android/javatests/src/org/chromium/chrome/browser/preferences/website/SiteSettingsPreferencesTest.java
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings.grd
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_am.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_ar.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_bg.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_bn.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_ca.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_cs.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_da.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_de.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_el.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_en-GB.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_es-419.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_es.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_et.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_fa.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_fi.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_fil.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_fr.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_gu.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_hi.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_hr.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_hu.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_id.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_it.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_ja.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_kn.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_ko.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_lt.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_lv.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_ml.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_mr.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_nb.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_nl.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_pl.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_pt-BR.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_pt-PT.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_ro.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_ru.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_sk.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_sl.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_sr.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_sv.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_sw.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_ta.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_te.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_th.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_tr.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_uk.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_vi.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_zh-CN.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_zh-TW.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/browser/android/preferences/pref_service_bridge.cc

Project Member

Comment 39 by bugdroid1@chromium.org, Jul 22 2016

Labels: merge-merged-2804
The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/f0829bf6d80a9109b399580fe48d8c3e1c66eeed

commit f0829bf6d80a9109b399580fe48d8c3e1c66eeed
Author: jinsukkim <jinsukkim@chromium.org>
Date: Fri Jul 22 03:11:25 2016

Android: turn on auto-detect encoding by default

Now that CED (Compact Encoding Detection) library replaced
ICU (crrev.com/2081653007), turning on auto-detect encoding
by default became feasible without regression on page
loading time. Also removed the encoding menu in site
setting (originally added in crrev.com/1248163002).

Will remove the menu on the other platforms in a follow-up
CL.

BUG= 597488 

Review-Url: https://codereview.chromium.org/1894913002
Cr-Commit-Position: refs/heads/master@{#407005}

[rename] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/android/java/res/drawable-hdpi/settings_translate.png
[rename] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/android/java/res/drawable-mdpi/settings_translate.png
[rename] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/android/java/res/drawable-xhdpi/settings_translate.png
[rename] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/android/java/res/drawable-xxhdpi/settings_translate.png
[rename] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/android/java/res/drawable-xxxhdpi/settings_translate.png
[delete] https://crrev.com/79f7b784a97cbb22f11064a05b621b0def87eab3/chrome/android/java/res/xml/language_preferences.xml
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/android/java/res/xml/site_settings_preferences.xml
[add] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/android/java/res/xml/translate_preferences.xml
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/android/java/src/org/chromium/chrome/browser/preferences/PrefServiceBridge.java
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/android/java/src/org/chromium/chrome/browser/preferences/website/SiteSettingsPreferences.java
[rename] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/android/java/src/org/chromium/chrome/browser/preferences/website/TranslatePreferences.java
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/android/java/strings/android_chrome_strings.grd
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/android/java_sources.gni
[add] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/android/javatests/src/org/chromium/chrome/browser/EncodingDetectionTest.java
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/android/javatests/src/org/chromium/chrome/browser/preferences/website/SiteSettingsPreferencesTest.java
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings.grd
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_am.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_ar.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_bg.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_bn.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_ca.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_cs.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_da.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_de.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_el.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_en-GB.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_es-419.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_es.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_et.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_fa.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_fi.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_fil.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_fr.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_gu.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_hi.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_hr.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_hu.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_id.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_it.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_ja.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_kn.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_ko.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_lt.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_lv.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_ml.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_mr.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_nb.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_nl.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_pl.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_pt-BR.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_pt-PT.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_ro.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_ru.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_sk.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_sl.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_sr.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_sv.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_sw.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_ta.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_te.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_th.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_tr.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_uk.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_vi.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_zh-CN.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/app/resources/locale_settings_zh-TW.xtb
[modify] https://crrev.com/f0829bf6d80a9109b399580fe48d8c3e1c66eeed/chrome/browser/android/preferences/pref_service_bridge.cc

Comment 40 by drott@chromium.org, Jul 26 2016

Thanks for this change junskkim@ - happy to see this. Do you think this change is what has improved https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg2tCpqwoM ?

Yes I think CED does that. It shows how much better CED is than ICU in terms of speed. That's a huge gain. Happy to see this improvement too. 

BTW the CL seems to have turned on autodetection on desktop as well by accident - or was it already on?
According to the discussion with folks@ui-review, I'll remove encoding menu + default encoding (in Settings -> Customized fonts -> Fonts and Encoding).
CL is up for review: https://crrev.com/2254273003

Comment 44 by js...@chromium.org, Aug 27 2016

re perf-improvement in comment 40: Using CED should not affect the performance unless that test in question uses *unlabelled* (no encoding declaration). Maybe, that test does use unlabelled html files. For the layout perf testing, we'd better use labelled documents, IMHO. 
 
Project Member

Comment 45 by bugdroid1@chromium.org, Aug 30 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/72125e2b16381f98e4bd44ae67240a736a648f5a

commit 72125e2b16381f98e4bd44ae67240a736a648f5a
Author: jinsukkim <jinsukkim@chromium.org>
Date: Tue Aug 30 23:51:48 2016

Remove text encoding UI

Text encoding UI is removed as a part of the project Eraser effort.
Manual encoding selection is not necessary any more as the new
encoding detector is turned on by default.

Request for manual encoding switching which may still exist
can be met by utilizing Chrome extension. See the bug for
detailed discussion.

Also removed the default encoding in Settings -> Customize fonts ->
Fonts and encoding.

Discussion at chrome-ui-review can be found here https://goo.gl/47cdcb

BUG= 597488 
CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.linux:closure_compilation

Review-Url: https://codereview.chromium.org/2254273003
Cr-Commit-Position: refs/heads/master@{#415497}

[modify] https://crrev.com/72125e2b16381f98e4bd44ae67240a736a648f5a/chrome/app/chrome_command_ids.h
[modify] https://crrev.com/72125e2b16381f98e4bd44ae67240a736a648f5a/chrome/app/generated_resources.grd
[modify] https://crrev.com/72125e2b16381f98e4bd44ae67240a736a648f5a/chrome/app/nibs/MainMenu.xib
[modify] https://crrev.com/72125e2b16381f98e4bd44ae67240a736a648f5a/chrome/app/resources/locale_settings.grd
[modify] https://crrev.com/72125e2b16381f98e4bd44ae67240a736a648f5a/chrome/app/settings_strings.grdp
[modify] https://crrev.com/72125e2b16381f98e4bd44ae67240a736a648f5a/chrome/browser/app_controller_mac.mm
[modify] https://crrev.com/72125e2b16381f98e4bd44ae67240a736a648f5a/chrome/browser/browser_encoding_browsertest.cc
[modify] https://crrev.com/72125e2b16381f98e4bd44ae67240a736a648f5a/chrome/browser/character_encoding.cc
[modify] https://crrev.com/72125e2b16381f98e4bd44ae67240a736a648f5a/chrome/browser/character_encoding.h
[add] https://crrev.com/72125e2b16381f98e4bd44ae67240a736a648f5a/chrome/browser/character_encoding_unittest.cc
[modify] https://crrev.com/72125e2b16381f98e4bd44ae67240a736a648f5a/chrome/browser/chrome_content_browser_client.cc
[modify] https://crrev.com/72125e2b16381f98e4bd44ae67240a736a648f5a/chrome/browser/prefs/browser_prefs.cc
[modify] https://crrev.com/72125e2b16381f98e4bd44ae67240a736a648f5a/chrome/browser/resources/options/font_settings.html
[modify] https://crrev.com/72125e2b16381f98e4bd44ae67240a736a648f5a/chrome/browser/resources/options/font_settings.js
[modify] https://crrev.com/72125e2b16381f98e4bd44ae67240a736a648f5a/chrome/browser/resources/settings/appearance_page/appearance_fonts_page.html
[modify] https://crrev.com/72125e2b16381f98e4bd44ae67240a736a648f5a/chrome/browser/resources/settings/appearance_page/appearance_fonts_page.js
[modify] https://crrev.com/72125e2b16381f98e4bd44ae67240a736a648f5a/chrome/browser/resources/settings/appearance_page/appearance_page.html
[modify] https://crrev.com/72125e2b16381f98e4bd44ae67240a736a648f5a/chrome/browser/ui/browser.cc
[modify] https://crrev.com/72125e2b16381f98e4bd44ae67240a736a648f5a/chrome/browser/ui/browser.h
[modify] https://crrev.com/72125e2b16381f98e4bd44ae67240a736a648f5a/chrome/browser/ui/browser_browsertest.cc
[modify] https://crrev.com/72125e2b16381f98e4bd44ae67240a736a648f5a/chrome/browser/ui/browser_command_controller.cc
[modify] https://crrev.com/72125e2b16381f98e4bd44ae67240a736a648f5a/chrome/browser/ui/cocoa/app_menu/app_menu_controller.mm
[modify] https://crrev.com/72125e2b16381f98e4bd44ae67240a736a648f5a/chrome/browser/ui/cocoa/browser_window_command_handler.mm
[delete] https://crrev.com/01725f5d867a32b82bdd3adf5088cfbab0d54f29/chrome/browser/ui/cocoa/encoding_menu_controller_delegate_mac.h
[delete] https://crrev.com/01725f5d867a32b82bdd3adf5088cfbab0d54f29/chrome/browser/ui/cocoa/encoding_menu_controller_delegate_mac.mm
[modify] https://crrev.com/72125e2b16381f98e4bd44ae67240a736a648f5a/chrome/browser/ui/prefs/prefs_tab_helper.cc
[modify] https://crrev.com/72125e2b16381f98e4bd44ae67240a736a648f5a/chrome/browser/ui/toolbar/app_menu_model.cc
[modify] https://crrev.com/72125e2b16381f98e4bd44ae67240a736a648f5a/chrome/browser/ui/toolbar/app_menu_model.h
[modify] https://crrev.com/72125e2b16381f98e4bd44ae67240a736a648f5a/chrome/browser/ui/toolbar/app_menu_model_unittest.cc
[delete] https://crrev.com/01725f5d867a32b82bdd3adf5088cfbab0d54f29/chrome/browser/ui/toolbar/encoding_menu_controller.cc
[delete] https://crrev.com/01725f5d867a32b82bdd3adf5088cfbab0d54f29/chrome/browser/ui/toolbar/encoding_menu_controller.h
[delete] https://crrev.com/01725f5d867a32b82bdd3adf5088cfbab0d54f29/chrome/browser/ui/toolbar/encoding_menu_controller_unittest.cc
[modify] https://crrev.com/72125e2b16381f98e4bd44ae67240a736a648f5a/chrome/browser/ui/views/frame/system_menu_model_builder.cc
[modify] https://crrev.com/72125e2b16381f98e4bd44ae67240a736a648f5a/chrome/browser/ui/views/frame/system_menu_model_builder.h
[modify] https://crrev.com/72125e2b16381f98e4bd44ae67240a736a648f5a/chrome/browser/ui/webui/options/font_settings_handler.cc
[modify] https://crrev.com/72125e2b16381f98e4bd44ae67240a736a648f5a/chrome/browser/ui/webui/options/font_settings_handler.h
[modify] https://crrev.com/72125e2b16381f98e4bd44ae67240a736a648f5a/chrome/browser/ui/webui/settings/font_handler.cc
[modify] https://crrev.com/72125e2b16381f98e4bd44ae67240a736a648f5a/chrome/browser/ui/webui/settings/md_settings_localized_strings_provider.cc
[modify] https://crrev.com/72125e2b16381f98e4bd44ae67240a736a648f5a/chrome/chrome_browser_ui.gypi
[modify] https://crrev.com/72125e2b16381f98e4bd44ae67240a736a648f5a/chrome/chrome_tests_unit.gypi
[modify] https://crrev.com/72125e2b16381f98e4bd44ae67240a736a648f5a/chrome/common/pref_names.cc
[modify] https://crrev.com/72125e2b16381f98e4bd44ae67240a736a648f5a/content/browser/web_contents/web_contents_impl.cc
[modify] https://crrev.com/72125e2b16381f98e4bd44ae67240a736a648f5a/content/browser/web_contents/web_contents_impl.h
[modify] https://crrev.com/72125e2b16381f98e4bd44ae67240a736a648f5a/content/common/view_messages.h
[modify] https://crrev.com/72125e2b16381f98e4bd44ae67240a736a648f5a/content/public/browser/web_contents.h
[modify] https://crrev.com/72125e2b16381f98e4bd44ae67240a736a648f5a/content/renderer/render_view_impl.cc
[modify] https://crrev.com/72125e2b16381f98e4bd44ae67240a736a648f5a/content/renderer/render_view_impl.h
[modify] https://crrev.com/72125e2b16381f98e4bd44ae67240a736a648f5a/tools/metrics/actions/actions.xml

Labels: -Restrict-View-Google -M-51 M-55
This will go out in 55 since we missed branch cut.

> > BTW, is there any reason for 'Restrict-View-Google'? 
> It's just because I mentioned a project and a link to a doc internal to Google. 

Eraser is publicly known, see  http://crbug.com/512852  , and a dead-end link to private doc is OK to have here and there.  I'll open the bug in case some users notice the change and want to either learn about the reasoning or object to it with concrete examples.
Project Member

Comment 47 by bugdroid1@chromium.org, Sep 1 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/a86b22ea40a9500c77b28372718f3bb6df8261a9

commit a86b22ea40a9500c77b28372718f3bb6df8261a9
Author: Jinsuk Kim <jinsukkim@chromium.org>
Date: Thu Sep 01 23:31:15 2016

Enable browser encoding autodetection test

Another attempt to enable the autodetection test that has been
disabled up to now. The new autodetection lib hopefully contributed
to the stability.

Removed Java-side AutodetectionTest since it is essentially a duplicated test
with only a subset of the test documents. With this removal,
WebContents interface getEncoding() can be removed too.

Notes on autodetection result:

gb18030 is detected as GBK. This is acceptable because gb18030
is a superset of GBK, and the test document doesn't contain chars
that appear only is gb18030.

ISO-8859-8-I -> windows-1255. This is also acceptable. See
https://goo.gl/bX0nxa for reference.

The test documents updated to reflect the change above are of non-UTF-8
which is not handled very well by Rietveld. Will land it manually.

BUG= 597488 
R=dtrainor@chromium.org, jshin@chromium.org, phajdan.jr@chromium.org, thestig@chromium.org

Review URL: https://codereview.chromium.org/2294173002 .

Cr-Commit-Position: refs/heads/master@{#416120}

[modify] https://crrev.com/a86b22ea40a9500c77b28372718f3bb6df8261a9/chrome/android/java_sources.gni
[delete] https://crrev.com/0afe4deba4f93d82430e90d789ab39e3c788c891/chrome/android/javatests/src/org/chromium/chrome/browser/EncodingDetectionTest.java
[modify] https://crrev.com/a86b22ea40a9500c77b28372718f3bb6df8261a9/chrome/browser/browser_encoding_browsertest.cc
[rename] https://crrev.com/a86b22ea40a9500c77b28372718f3bb6df8261a9/chrome/test/data/encoding_tests/auto_detect/GBK_with_no_encoding_specified.html
[rename] https://crrev.com/a86b22ea40a9500c77b28372718f3bb6df8261a9/chrome/test/data/encoding_tests/auto_detect/ISO-8859-8-I_with_no_encoding_specified.html
[rename] https://crrev.com/a86b22ea40a9500c77b28372718f3bb6df8261a9/chrome/test/data/encoding_tests/auto_detect/expected_results/expected_GBK_saved_from_no_encoding_specified.html
[rename] https://crrev.com/a86b22ea40a9500c77b28372718f3bb6df8261a9/chrome/test/data/encoding_tests/auto_detect/expected_results/expected_ISO-8859-8-I_saved_from_no_encoding_specified.html
[modify] https://crrev.com/a86b22ea40a9500c77b28372718f3bb6df8261a9/content/public/android/java/src/org/chromium/content/browser/webcontents/WebContentsImpl.java
[modify] https://crrev.com/a86b22ea40a9500c77b28372718f3bb6df8261a9/content/public/android/java/src/org/chromium/content_public/browser/WebContents.java

Project Member

Comment 48 by bugdroid1@chromium.org, Sep 7 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc

commit c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc
Author: jinsukkim <jinsukkim@chromium.org>
Date: Wed Sep 07 02:33:43 2016

Remove unused prefs/api/resource for setPageEncoding

Interface WebView::setPageEncoding was mainly used to override

text encoding manually via UI. Now that UI is gone (see https://crrev.com/2254273003),
the interface is not used any more by chrome.

Also removed other preferences/resources that became unnecessary.

BUG= 597488 

Review-Url: https://codereview.chromium.org/2293373002
Cr-Commit-Position: refs/heads/master@{#416827}

[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings.grd
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_am.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_ar.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_bg.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_bn.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_ca.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_cs.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_da.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_de.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_el.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_en-GB.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_es-419.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_es.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_et.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_fa.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_fake-bidi.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_fi.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_fil.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_fr.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_gu.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_he.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_hi.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_hr.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_hu.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_id.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_it.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_ja.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_kn.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_ko.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_lt.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_lv.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_ml.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_mr.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_nb.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_nl.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_pl.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_pt-BR.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_pt-PT.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_ro.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_ru.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_sk.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_sl.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_sr.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_sv.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_sw.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_ta.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_te.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_th.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_tr.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_uk.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_vi.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_zh-CN.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/app/resources/locale_settings_zh-TW.xtb
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/browser/browser_encoding_browsertest.cc
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/browser/chrome_content_browser_client.cc
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/browser/prefs/browser_prefs.cc
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/browser/prefs/chrome_pref_service_unittest.cc
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/browser/ui/prefs/prefs_tab_helper.cc
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/chrome/common/pref_names.cc
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/content/public/common/common_param_traits_macros.h
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/content/public/common/web_preferences.cc
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/content/public/common/web_preferences.h
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/content/renderer/render_view_impl.cc
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/third_party/WebKit/LayoutTests/TestExpectations
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/third_party/WebKit/LayoutTests/fast/encoding/meta-overrules-auto.html
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/third_party/WebKit/Source/core/frame/FrameHost.h
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/third_party/WebKit/Source/core/frame/Settings.in
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/third_party/WebKit/Source/core/loader/DocumentLoader.cpp
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/third_party/WebKit/Source/core/loader/FrameFetchContext.cpp
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/third_party/WebKit/Source/core/loader/FrameFetchContextTest.cpp
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/third_party/WebKit/Source/core/loader/TextResourceDecoderBuilder.cpp
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/third_party/WebKit/Source/web/WebSettingsImpl.cpp
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/third_party/WebKit/Source/web/WebSettingsImpl.h
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/third_party/WebKit/Source/web/WebViewImpl.cpp
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/third_party/WebKit/Source/web/WebViewImpl.h
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/third_party/WebKit/public/web/WebSettings.h
[modify] https://crrev.com/c07e0137ca0c4ccd4d1d2205b09c28b3f35fa6dc/third_party/WebKit/public/web/WebView.h

Project Member

Comment 49 by bugdroid1@chromium.org, Sep 7 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/eaef19fd17f39929bd714336bb39a35da4f501b7

commit eaef19fd17f39929bd714336bb39a35da4f501b7
Author: jinsukkim <jinsukkim@chromium.org>
Date: Wed Sep 07 05:12:12 2016

Disable autodetection for XML documents

Unlabelled XML should be treated as UTF-8-encoded documents.
This CL disables autodetection for XML, and puts the suppressed
layout tests back on.

BUG= 597488 

Review-Url: https://codereview.chromium.org/2320433002
Cr-Commit-Position: refs/heads/master@{#416848}

[modify] https://crrev.com/eaef19fd17f39929bd714336bb39a35da4f501b7/third_party/WebKit/LayoutTests/TestExpectations
[modify] https://crrev.com/eaef19fd17f39929bd714336bb39a35da4f501b7/third_party/WebKit/Source/core/loader/TextResourceDecoderBuilder.cpp

Status: Fixed (was: Started)
what exaclty Chrome extension can changee encoding?
No exact extension exists yet, just that it's simple to create one (proof of concept at https://github.com/alexelias/encoding-changer/blob/master/src/bg/background.js ).

Could you describe why exactly you want an extension (for example, a URL of a broken page)?
Blocking: 467061
Project Member

Comment 55 by bugdroid1@chromium.org, Oct 8 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/08b1d375eb4870c3e0f2bf4654f9b3faf29e6254

commit 08b1d375eb4870c3e0f2bf4654f9b3faf29e6254
Author: dschuyler <dschuyler@chromium.org>
Date: Sat Oct 08 02:45:44 2016

[MD settings] touch-up removal of encoding settings

This CL changes some comments and closure compiler values to touch-up
after the removal of the encodings menu.

BUG= 641899 ,  597488 
CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.linux:closure_compilation

Review-Url: https://codereview.chromium.org/2398233004
Cr-Commit-Position: refs/heads/master@{#424050}

[modify] https://crrev.com/08b1d375eb4870c3e0f2bf4654f9b3faf29e6254/chrome/browser/resources/settings/appearance_page/fonts_browser_proxy.js
[modify] https://crrev.com/08b1d375eb4870c3e0f2bf4654f9b3faf29e6254/chrome/browser/ui/webui/settings/font_handler.cc
[modify] https://crrev.com/08b1d375eb4870c3e0f2bf4654f9b3faf29e6254/chrome/browser/ui/webui/settings/font_handler.h

A website that needs this feature is www.quickbase.com, you'd need to make your own app to know why you need it. Anyone make an extension yet?
Encoding settings is NOT only used by power users. When East Asian users (mainly CJK) encounters mojibake (文字化け, or 乱码 in Chinese), they wouldn't just assume it couldn't be loaded in the right encoding and close the page. Even if they don't know about the encoding setting, they would search on the web about it. And then they would know. Creating an extension is too much for these non-power users. What's more, requiring an extension for a standard feature that all other browsers include support of is stupid.
There's many old webpages that didn't declare the encoding on the Japanese internet. And there's some webpages that incorrectly declare their encoding (like the error page here https://myaccount.ea.com/cp-ui/orderhistory/index?gameId=ebisu&locale=en_us). So the encoding settings is ABSOLUTELY NECESSARY for now. It may not be often used, but it's NECESSARY. Be aware that wrong encoding settings would render CJK texts entirely unreadable, while to languages that use the Latin alphabet, only the letters with umlaut/diacritic/whatever would be unreadable. I highly suspect that Chrome staff only thought about Latin languages, and ignored CJK. As a Chinese I urge the Chromium staff to revert this change.
This and forcing Material Design are two stupid changes in Chrome 55. If these changes are not reverted, I would be moving to Firefox.
Cc: kkaluri@chromium.org
 Issue 676790  has been merged into this issue.

Comment 59 by tyna...@gmail.com, Dec 27 2016

The majority of the time that I'm using the manual encoding override menu (on DESKTOP), it's because the website *does* specify in metadata what encoding it uses... But it has user-submitted content that does not use the same encoding, and is spat out on the page without any sort of conversion.

This is a case that no auto-detector can possibly handle, because you might have multiple encodings on a page. You can't handle *all* of them, you have to pick one.

And that pretty much means that removal of the menu makes Chrome dead on arrival for a large number of people who use websites that are programmed like this, such as forums and blog sites.

I agree that auto detection needed to be improved, especially speed-wise. But why would that necessitate the removal of the manual override menu? Even with a nice and speedy autodetector, there's *still* the possibility of failure.

And I do not believe that manually overriding the encoding is something which will, to quote the Project Eraser document linked in comment 14, "no longer provide enough value to justify (its) complexity, whether in the code or in the UI."

It's just a menu! And it was always a sort of hidden submenu, which didn't confuse users or even be seen by users unless they were looking for it and needed it! And forcing an encoding for a tab is *not* complex, so you can't say it doesn't provide less value than its complexity (since the value is immensely greater than the complexity when simply manually overriding the encoding).

I switched to Chrome because it seemed to me that it provided a good number of features I felt were important, even if nobody else ever even thought of them. Often they were hidden away in various parts of the settings, but the fact that Chrome still had them available kept me around.

Right now I have no comment about other items in Project Eraser, but the very name of it leaves a bad taste in my mouth. I'm going to start transitioning to Firefox. I have a lot of things synchronized on my Google account though, such as passwords and bookmarks, so I'm going to still be using Chrome mostly for now... But This is the writing on the wall. I'd be a moron to ignore it.

If I suddenly see the menu come back, I'll probably reverse my decision. But for now, I'll start setting up Firefox's sync more properly and slowly transition all my bookmarks. Who knew it'd take the removal of a freak'n menu?

Comment 60 by tyna...@gmail.com, Dec 27 2016

Seems I can't edit a comment, or maybe I'm just not seeing the button.

But an example of a page that needs a manual override is here:
http://www.bobandgeorge.com/archives/040417

Notice that the response headers indicate UTF-8, but the authors of the comic submitted the word 'Clichés' in the 'Western' encoding (according to Firefox. Not entirely sure which encoding that is, to be honest).

It's an old website for a webcomic that is no longer updated. The authors of the website will never care enough to change it. And without an option to manually override the encoding, Chrome will never again display that page correctly.

Comment 61 by homl...@gmail.com, Dec 28 2016

This and forcing Material Design are two stupid changes in Chrome 55.
好好的有一个方便网页制作人员的功能,怎么突然间就被去除了呢,试问哪一个不能手动更改网页编码的?!身为一个全球流行的浏览器,并且很多网页制作者用的最多的一款浏览器,竟然去掉了这个网页制作人员必须会用到的功能,真是让人无语,强烈建议改回来,哪怕是出一个官方扩展插件弥补一下也好,望采纳建议!!!

Comment 63 by ygh...@gmail.com, Dec 31 2016

Install IE....That's the only choice.
There is too many charset-racist in western and here is one of them!

Comment 65 by soyu...@gmail.com, Jan 2 2017

Building a great wall to block CJK users and charging them????
i hope can recover this feature, because it is very useful for many old Chinese language website, some of them use different encoding in different page or file export. if not this feature we alway get not normal file name. this feature is very important!!! 

Comment 67 Deleted

finally found a way to get around this : use pale moon!
Basically, any old big5 html page without encoding meta tags are no longer readable.
Now it comes a Chrome extension to change encoding from right-click menu.

https://chrome.google.com/webstore/detail/set-character-encoding/bpojelgakakmcfmjfilgdlmhefphglae

Seems to be a nice replacement for generic menu because you can remove it when not wanted.
Dear Google staff, please provide an official extension with similar functionality that was removed.

The usage of "overriding" encoding is EXTREMELY RARE, but it is CRUCIAL!
I'm already considering changing a browser for that functionality.

Because I spent 20 minutes googling how to override encoding in Google Chrome after I found this feature missing for some reason, and come here, and found that this functionality was removed! Spending 20 minutes to get this simple functionality is not productive at all!!

Comment 72 by dil...@gmail.com, Jan 10 2017

I agree with above comments that this is a very unfortunate, misguided change. I live and work in Japan, and manual encoding change is needed all the time for Japanese pages. It's not just power users. I've seen a dozen otherwise computer-inept people know how to use the encoding-change menu in IE. This is a bad, Western-centric change that definitely won't do anything to change the minds of all my coworkers that use IE.
Maybe this Chrome-extension helps to set encoding options:
https://chrome.google.com/webstore/detail/set-character-encoding/bpojelgakakmcfmjfilgdlmhefphglae
I agree with all of the above comments. This is a basic functionality feature. It is significant enough to cause users in several languages to abandon Chrome in favour of more user oriented browsers. Chrome, dont head where Netscape ended up.
Blocking: 679294
Auto-detect isn't working. Mail Chimp email is still displaying crazy characters. Either give us back the ability to override the default charset or suffer user attrition.
I found a simple solution while waiting for everyone long story. I am using Microsoft IE to open and use the encoder to solve it. 
As a Java Web developer, it is easy to come up with encoding problems. Any there can be various causes. Usually a simple change of encoding will do the trick. For the years I've been using Chrome, auto-detecting never worked correctly as expected. So instead of improving auto-detecting, you just think it works perfectly in any condition and remove the manual switch of encoding, which causes huge inconvenience to me. I really don't understand why you removed this feature. And now I have to use an additional script or extension to do the same. Won't it be really unnecessary? Hope it be back ASAP, thanks a lot :-)

Comment 79 by eugeb...@gmail.com, Jan 31 2017

Agree with comments above. Guys, please just get the manual setting back. Isn't that silly to install a 3rd party extension for a feature that have been native for all browsers for ages? And yep, autodetection doesn't work well with cyrillic characters as well. I'm disappointed and as many here I do hope you will get it back. Thanks :)
I certainly hope this 'Encoding option menu' back. Although not often, I do really encounter cases when Chrome cannot correctly render content (Chinese texts).

In some local online forums I already see people talking about switching to FF or some other similar browses, so please give this a consider, thanks.
Obviously, whoever made or supported this decision do not respect global Chrome users at all, or they are just too stupid. 
Is it because none of your decision makers speaks more than one language?
Shame!!

Comment 82 by dxq1...@gmail.com, Feb 9 2017

As a Web developer,Chrome this is forcing me to give up, do you want me to switch to other browsers?

Comment 83 by bum...@gmail.com, Feb 10 2017

If low frequency means not necessary, will you remove toilet completely? 
I see what you did there. A few weeks after Trump's election, before he's even inaugurated, you start to implement his 'America First' program by publishing a browser that makes the web less accessible for countries who don't follow the American Standard Code for Information Interchange. I'm switching to Opera. Norway First!

Comment 85 by odean@google.com, Feb 24 2017

Labels: Restrict-AddIssueComment-EditIssue
.... aaaand this is why we can't have nice things. If you would like to follow along for updates or express your concerns about this decision please feel free to star the bug. "Me too" comments clutter the working space and make it harder for engineers working on the code to communicate with each other. Political commentary should be reserved for more relevant forums. 
 Issue 701689  has been merged into this issue.
 Issue 703006  has been merged into this issue.

Sign in to add a comment