New issue
Advanced search Search tips

Issue 725654 link

Starred by 2 users

Issue metadata

Status: Fixed
Owner:
Closed: Jun 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: All
Pri: 1
Type: Bug

Blocked on:
issue 732893

Blocking:
issue 594639



Sign in to add a comment

<script type=module> treats files as ASCII instead of UTF-8

Reported by chrisv@movable-type.co.uk, May 23 2017

Issue description

Chrome Version       : 60.0.3100.0 (Official Build) dev (64-bit) Linux (V8 6.0.205)

URLs (if applicable) :
  http://www.movable-type.co.uk/dev/es-modules/classic-script-string.html
  http://www.movable-type.co.uk/dev/es-modules/classic-script-const.html
  http://www.movable-type.co.uk/dev/es-modules/module-script-string.html
  http://www.movable-type.co.uk/dev/es-modules/module-script-const.html

What steps will reproduce the problem?

visit URLs above & view console.log output; 'classic' versions work correctly, 'module' versions don't handle non-ASCII UTF-8 characters.

What is the expected result?

'script-string.html' URLs should show 'π' in the console; 'script-const.html' URLs should show '3.141592653589793' in the console.

What happens instead?

- 'classic' versions run correctly
- module-script-string.html shows 'Ï€' instead of 'π'
- module-script-const.html shows 'Uncaught SyntaxError: Missing initializer in const declaration'

Please provide any additional information below. Attach a screenshot if
possible.

Both 'classic' & 'module' versions of 'string' load .js file containing 'console.log('π');'.

Both 'classic' & 'module' versions of 'const' load .js file containing 'const π = Math.PI; console.log(π);'.

The 'module' versions use 'type="module"' to load the .js file.

It would appear that '<script type="module">' is treating file encoding as ASCII instead of UTF-8?

 
Labels: Needs-Triage-M60 Needs-Bisect
Cc: jmukthavaram@chromium.org
Components: Blink>JavaScript
Labels: -Needs-Bisect M-60 OS-Linux OS-Mac OS-Windows
Status: Untriaged (was: Unconfirmed)
Able to reproduce the issue on Windows 7, Mac 10.12.4 & Ubuntu 14.04 using chrome stable#58.0.3029.110, reported version#60.0.3100.0  & latest Canary#61.0.3114.0 as per the above steps & URL's.

Observed 'blank output' in Console log of dev tools for the below module related URL's are :

http://www.movable-type.co.uk/dev/es-modules/module-script-string.html
http://www.movable-type.co.uk/dev/es-modules/module-script-const.html

Observed 'π' for classic-script-string.html & '3.141592653589793' for classic-script-const.html file in Devtools-> Console.

Observed the same behavior from M30 (30.0.1549.0 ) builds to latest Canary, hence marking it as 'Untriaged' to get more inputs from dev for further investigation.
Please find the attached screencast for reference.

Thank you..!!


725654.mp4
749 KB View Download
Chrome versions prior to M60 show blank output in console, as they do not have ES modules implemented. Reported behaviour is for M60.


Screenshot version.png
89.3 KB View Download
Screenshot classic script string.png
33.4 KB View Download
Screenshot module script string.png
33.2 KB View Download
Screenshot classic script const.png
35.2 KB View Download
Screenshot module script const.png
36.4 KB View Download
Update: I have managed to resolve this by setting charset for Content-Type:application/javascript to 'utf-8' (using Apache 'AddDefaultCharset utf-8' and/or 'AddCharset utf-8 .js').

Why this is required for ES modules .js files and not for classic script .js files I don't know!

Comment 5 by danno@chromium.org, Jun 6 2017

Owner: adamk@chromium.org
Status: Assigned (was: Untriaged)

Comment 6 by adamk@chromium.org, Jun 6 2017

Blocking: 594639
Cc: hirosh...@chromium.org adamk@chromium.org kouhei@chromium.org
Components: Blink>HTML>Script
Labels: -Pri-3 -M-60 -Needs-Triage-M60 M-61 OS-Android OS-Chrome Pri-2
Owner: ----
Status: Available (was: Assigned)
Owner: kouhei@chromium.org
Status: Assigned (was: Available)
Status: Started (was: Assigned)
Blink bug.

We need to force UTF8 encoding per Step 8 in https://html.spec.whatwg.org/multipage/webappapis.html#fetch-a-module-script-tree

We somehow need to create ScriptResource w/ its TextDecoder set to force utf8 text encoding.
Owner: hirosh...@chromium.org
Status: Assigned (was: Started)
Components: -Blink>JavaScript Blink>Loader
Labels: -OS-Linux -OS-Android -OS-Windows -Pri-2 -OS-Chrome -OS-Mac OS-All Pri-1
Status: Started (was: Assigned)
Blockedon: 732893
Project Member

Comment 15 by bugdroid1@chromium.org, Jun 17 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/34552ccffd39365509f0bb21a446dd7fcc40e089

commit 34552ccffd39365509f0bb21a446dd7fcc40e089
Author: Hiroshige Hayashizaki <hiroshige@chromium.org>
Date: Sat Jun 17 02:03:31 2017

Make TextEncoding constructors explicit

This CL doesn't change the behavior.

Bug:  725654 
Change-Id: I94d477b78f11112fb800e2ee04d4593731722ae8
Reviewed-on: https://chromium-review.googlesource.com/535858
Reviewed-by: Kouhei Ueno <kouhei@chromium.org>
Reviewed-by: Kentaro Hara <haraken@chromium.org>
Commit-Queue: Hiroshige Hayashizaki <hiroshige@chromium.org>
Cr-Commit-Position: refs/heads/master@{#480269}
[modify] https://crrev.com/34552ccffd39365509f0bb21a446dd7fcc40e089/third_party/WebKit/Source/bindings/core/v8/ScriptStreamer.cpp
[modify] https://crrev.com/34552ccffd39365509f0bb21a446dd7fcc40e089/third_party/WebKit/Source/core/css/parser/CSSParserContext.cpp
[modify] https://crrev.com/34552ccffd39365509f0bb21a446dd7fcc40e089/third_party/WebKit/Source/core/html/parser/TextResourceDecoder.cpp
[modify] https://crrev.com/34552ccffd39365509f0bb21a446dd7fcc40e089/third_party/WebKit/Source/core/html/parser/TextResourceDecoderForFuzzing.h
[modify] https://crrev.com/34552ccffd39365509f0bb21a446dd7fcc40e089/third_party/WebKit/Source/core/inspector/InspectorPageAgent.cpp
[modify] https://crrev.com/34552ccffd39365509f0bb21a446dd7fcc40e089/third_party/WebKit/Source/core/loader/TextResourceDecoderBuilder.cpp
[modify] https://crrev.com/34552ccffd39365509f0bb21a446dd7fcc40e089/third_party/WebKit/Source/core/loader/resource/TextResource.cpp
[modify] https://crrev.com/34552ccffd39365509f0bb21a446dd7fcc40e089/third_party/WebKit/Source/core/workers/WorkerScriptLoader.cpp
[modify] https://crrev.com/34552ccffd39365509f0bb21a446dd7fcc40e089/third_party/WebKit/Source/core/xmlhttprequest/XMLHttpRequest.cpp
[modify] https://crrev.com/34552ccffd39365509f0bb21a446dd7fcc40e089/third_party/WebKit/Source/platform/wtf/text/TextCodecICU.cpp
[modify] https://crrev.com/34552ccffd39365509f0bb21a446dd7fcc40e089/third_party/WebKit/Source/platform/wtf/text/TextEncoding.h

Project Member

Comment 16 by bugdroid1@chromium.org, Jun 17 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/e34e28b4ec472c186b8ed2de2b6653c77d5c7a39

commit e34e28b4ec472c186b8ed2de2b6653c77d5c7a39
Author: Hiroshige Hayashizaki <hiroshige@chromium.org>
Date: Sat Jun 17 09:55:25 2017

Make Resource::Encoding() and SetCSSStyleSheet() use TextEncoding

This CL also makes Resource::SetEncoding() protected and
exposes TextResource::SetEncodingForTest() instead.

Bug:  725654 
Change-Id: I5ae53df9b134524601cb280969648263f639b8d3
Reviewed-on: https://chromium-review.googlesource.com/535859
Reviewed-by: Kouhei Ueno <kouhei@chromium.org>
Reviewed-by: Kinuko Yasuda <kinuko@chromium.org>
Commit-Queue: Hiroshige Hayashizaki <hiroshige@chromium.org>
Cr-Commit-Position: refs/heads/master@{#480294}
[modify] https://crrev.com/e34e28b4ec472c186b8ed2de2b6653c77d5c7a39/third_party/WebKit/Source/bindings/core/v8/ScriptStreamerTest.cpp
[modify] https://crrev.com/e34e28b4ec472c186b8ed2de2b6653c77d5c7a39/third_party/WebKit/Source/core/css/StyleRuleImport.cpp
[modify] https://crrev.com/e34e28b4ec472c186b8ed2de2b6653c77d5c7a39/third_party/WebKit/Source/core/css/StyleRuleImport.h
[modify] https://crrev.com/e34e28b4ec472c186b8ed2de2b6653c77d5c7a39/third_party/WebKit/Source/core/dom/ProcessingInstruction.cpp
[modify] https://crrev.com/e34e28b4ec472c186b8ed2de2b6653c77d5c7a39/third_party/WebKit/Source/core/dom/ProcessingInstruction.h
[modify] https://crrev.com/e34e28b4ec472c186b8ed2de2b6653c77d5c7a39/third_party/WebKit/Source/core/html/LinkStyle.cpp
[modify] https://crrev.com/e34e28b4ec472c186b8ed2de2b6653c77d5c7a39/third_party/WebKit/Source/core/html/LinkStyle.h
[modify] https://crrev.com/e34e28b4ec472c186b8ed2de2b6653c77d5c7a39/third_party/WebKit/Source/core/html/parser/CSSPreloadScanner.cpp
[modify] https://crrev.com/e34e28b4ec472c186b8ed2de2b6653c77d5c7a39/third_party/WebKit/Source/core/html/parser/CSSPreloadScanner.h
[modify] https://crrev.com/e34e28b4ec472c186b8ed2de2b6653c77d5c7a39/third_party/WebKit/Source/core/inspector/InspectorResourceContentLoader.cpp
[modify] https://crrev.com/e34e28b4ec472c186b8ed2de2b6653c77d5c7a39/third_party/WebKit/Source/core/loader/resource/StyleSheetResourceClient.h
[modify] https://crrev.com/e34e28b4ec472c186b8ed2de2b6653c77d5c7a39/third_party/WebKit/Source/core/loader/resource/TextResource.cpp
[modify] https://crrev.com/e34e28b4ec472c186b8ed2de2b6653c77d5c7a39/third_party/WebKit/Source/core/loader/resource/TextResource.h
[modify] https://crrev.com/e34e28b4ec472c186b8ed2de2b6653c77d5c7a39/third_party/WebKit/Source/platform/loader/fetch/Resource.cpp
[modify] https://crrev.com/e34e28b4ec472c186b8ed2de2b6653c77d5c7a39/third_party/WebKit/Source/platform/loader/fetch/Resource.h

Project Member

Comment 17 by bugdroid1@chromium.org, Jun 17 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/64ee99bc9764b104282c0bca7f29d8d024b741b6

commit 64ee99bc9764b104282c0bca7f29d8d024b741b6
Author: Hiroshige Hayashizaki <hiroshige@chromium.org>
Date: Sat Jun 17 12:20:19 2017

Make CSSParserContext::Create() and Charset() use TextEncoding

Bug:  725654 
Change-Id: Iafd2c9d47697e6d45f3c98857069cdfe82cfaed8
Reviewed-on: https://chromium-review.googlesource.com/535839
Reviewed-by: Kouhei Ueno <kouhei@chromium.org>
Commit-Queue: Hiroshige Hayashizaki <hiroshige@chromium.org>
Cr-Commit-Position: refs/heads/master@{#480300}
[modify] https://crrev.com/64ee99bc9764b104282c0bca7f29d8d024b741b6/third_party/WebKit/Source/core/css/CSSStyleSheet.cpp
[modify] https://crrev.com/64ee99bc9764b104282c0bca7f29d8d024b741b6/third_party/WebKit/Source/core/css/CSSStyleSheet.h
[modify] https://crrev.com/64ee99bc9764b104282c0bca7f29d8d024b741b6/third_party/WebKit/Source/core/css/CSSTestHelper.cpp
[modify] https://crrev.com/64ee99bc9764b104282c0bca7f29d8d024b741b6/third_party/WebKit/Source/core/css/StyleRuleImport.cpp
[modify] https://crrev.com/64ee99bc9764b104282c0bca7f29d8d024b741b6/third_party/WebKit/Source/core/css/StyleSheetContents.h
[modify] https://crrev.com/64ee99bc9764b104282c0bca7f29d8d024b741b6/third_party/WebKit/Source/core/css/parser/CSSParserContext.cpp
[modify] https://crrev.com/64ee99bc9764b104282c0bca7f29d8d024b741b6/third_party/WebKit/Source/core/css/parser/CSSParserContext.h
[modify] https://crrev.com/64ee99bc9764b104282c0bca7f29d8d024b741b6/third_party/WebKit/Source/core/dom/ProcessingInstruction.cpp
[modify] https://crrev.com/64ee99bc9764b104282c0bca7f29d8d024b741b6/third_party/WebKit/Source/core/dom/SelectorQuery.cpp
[modify] https://crrev.com/64ee99bc9764b104282c0bca7f29d8d024b741b6/third_party/WebKit/Source/core/dom/SelectorQueryTest.cpp
[modify] https://crrev.com/64ee99bc9764b104282c0bca7f29d8d024b741b6/third_party/WebKit/Source/core/dom/StyleEngine.cpp
[modify] https://crrev.com/64ee99bc9764b104282c0bca7f29d8d024b741b6/third_party/WebKit/Source/core/frame/FrameSerializer.cpp
[modify] https://crrev.com/64ee99bc9764b104282c0bca7f29d8d024b741b6/third_party/WebKit/Source/core/html/LinkStyle.cpp

Project Member

Comment 18 by bugdroid1@chromium.org, Jun 19 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/94f22629bdee3f65163cee9ba89d5d72cdfbe0fb

commit 94f22629bdee3f65163cee9ba89d5d72cdfbe0fb
Author: Hiroshige Hayashizaki <hiroshige@chromium.org>
Date: Mon Jun 19 01:50:26 2017

Use TextEncoding in FetchParameters::SetEncoding()

Bug:  725654 
Change-Id: I6065265b3e549453bfe7727c2ce0b5f04db1abdd
Reviewed-on: https://chromium-review.googlesource.com/535860
Reviewed-by: Kouhei Ueno <kouhei@chromium.org>
Reviewed-by: Kinuko Yasuda <kinuko@chromium.org>
Commit-Queue: Hiroshige Hayashizaki <hiroshige@chromium.org>
Cr-Commit-Position: refs/heads/master@{#480331}
[modify] https://crrev.com/94f22629bdee3f65163cee9ba89d5d72cdfbe0fb/third_party/WebKit/Source/core/css/StyleRuleImport.cpp
[modify] https://crrev.com/94f22629bdee3f65163cee9ba89d5d72cdfbe0fb/third_party/WebKit/Source/core/dom/ProcessingInstruction.cpp
[modify] https://crrev.com/94f22629bdee3f65163cee9ba89d5d72cdfbe0fb/third_party/WebKit/Source/core/dom/ScriptLoader.cpp
[modify] https://crrev.com/94f22629bdee3f65163cee9ba89d5d72cdfbe0fb/third_party/WebKit/Source/core/dom/ScriptLoader.h
[modify] https://crrev.com/94f22629bdee3f65163cee9ba89d5d72cdfbe0fb/third_party/WebKit/Source/core/html/LinkResource.cpp
[modify] https://crrev.com/94f22629bdee3f65163cee9ba89d5d72cdfbe0fb/third_party/WebKit/Source/core/html/LinkResource.h
[modify] https://crrev.com/94f22629bdee3f65163cee9ba89d5d72cdfbe0fb/third_party/WebKit/Source/core/html/LinkStyle.cpp
[modify] https://crrev.com/94f22629bdee3f65163cee9ba89d5d72cdfbe0fb/third_party/WebKit/Source/core/html/LinkStyle.h
[modify] https://crrev.com/94f22629bdee3f65163cee9ba89d5d72cdfbe0fb/third_party/WebKit/Source/core/html/parser/PreloadRequest.cpp
[modify] https://crrev.com/94f22629bdee3f65163cee9ba89d5d72cdfbe0fb/third_party/WebKit/Source/core/loader/LinkLoader.cpp
[modify] https://crrev.com/94f22629bdee3f65163cee9ba89d5d72cdfbe0fb/third_party/WebKit/Source/platform/loader/fetch/FetchParameters.h

Project Member

Comment 19 by bugdroid1@chromium.org, Jun 20 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/5ceed48c3549686d68ade225a1dc3becfb158427

commit 5ceed48c3549686d68ade225a1dc3becfb158427
Author: Hiroshige Hayashizaki <hiroshige@chromium.org>
Date: Tue Jun 20 22:07:02 2017

Make TextResourceDecoder::Create() take ContentType instead of mime type

The |mime_type| argument is used only to determine ContentType, and
set to a constant except when called from TextResourceDecoderBuilder.

This CL makes Create() to take ContentType directly as an argument
and moves DetermineContentType() to TextResourceDecoderBuilder.cpp.

Bug:  725654 , 732893
Change-Id: I582727705351a49d10ecffdfdd9c1578114b6321
Reviewed-on: https://chromium-review.googlesource.com/539957
Commit-Queue: Hiroshige Hayashizaki <hiroshige@chromium.org>
Reviewed-by: Kouhei Ueno <kouhei@chromium.org>
Reviewed-by: Takeshi Yoshino <tyoshino@chromium.org>
Reviewed-by: Kinuko Yasuda <kinuko@chromium.org>
Reviewed-by: Charlie Harrison <csharrison@chromium.org>
Cr-Commit-Position: refs/heads/master@{#480982}
[modify] https://crrev.com/5ceed48c3549686d68ade225a1dc3becfb158427/third_party/WebKit/Source/bindings/core/v8/ScriptStreamer.cpp
[modify] https://crrev.com/5ceed48c3549686d68ade225a1dc3becfb158427/third_party/WebKit/Source/core/fileapi/FileReaderLoader.cpp
[modify] https://crrev.com/5ceed48c3549686d68ade225a1dc3becfb158427/third_party/WebKit/Source/core/html/parser/TextResourceDecoder.cpp
[modify] https://crrev.com/5ceed48c3549686d68ade225a1dc3becfb158427/third_party/WebKit/Source/core/html/parser/TextResourceDecoder.h
[modify] https://crrev.com/5ceed48c3549686d68ade225a1dc3becfb158427/third_party/WebKit/Source/core/html/parser/TextResourceDecoderForFuzzing.h
[modify] https://crrev.com/5ceed48c3549686d68ade225a1dc3becfb158427/third_party/WebKit/Source/core/html/parser/TextResourceDecoderTest.cpp
[modify] https://crrev.com/5ceed48c3549686d68ade225a1dc3becfb158427/third_party/WebKit/Source/core/html/track/vtt/VTTParser.cpp
[modify] https://crrev.com/5ceed48c3549686d68ade225a1dc3becfb158427/third_party/WebKit/Source/core/inspector/InspectorPageAgent.cpp
[modify] https://crrev.com/5ceed48c3549686d68ade225a1dc3becfb158427/third_party/WebKit/Source/core/loader/TextResourceDecoderBuilder.cpp
[modify] https://crrev.com/5ceed48c3549686d68ade225a1dc3becfb158427/third_party/WebKit/Source/core/loader/resource/CSSStyleSheetResource.cpp
[modify] https://crrev.com/5ceed48c3549686d68ade225a1dc3becfb158427/third_party/WebKit/Source/core/loader/resource/DocumentResource.cpp
[modify] https://crrev.com/5ceed48c3549686d68ade225a1dc3becfb158427/third_party/WebKit/Source/core/loader/resource/ScriptResource.cpp
[modify] https://crrev.com/5ceed48c3549686d68ade225a1dc3becfb158427/third_party/WebKit/Source/core/loader/resource/StyleSheetResource.h
[modify] https://crrev.com/5ceed48c3549686d68ade225a1dc3becfb158427/third_party/WebKit/Source/core/loader/resource/TextResource.cpp
[modify] https://crrev.com/5ceed48c3549686d68ade225a1dc3becfb158427/third_party/WebKit/Source/core/loader/resource/TextResource.h
[modify] https://crrev.com/5ceed48c3549686d68ade225a1dc3becfb158427/third_party/WebKit/Source/core/loader/resource/XSLStyleSheetResource.cpp
[modify] https://crrev.com/5ceed48c3549686d68ade225a1dc3becfb158427/third_party/WebKit/Source/core/workers/WorkerScriptLoader.cpp
[modify] https://crrev.com/5ceed48c3549686d68ade225a1dc3becfb158427/third_party/WebKit/Source/core/xmlhttprequest/XMLHttpRequest.cpp
[modify] https://crrev.com/5ceed48c3549686d68ade225a1dc3becfb158427/third_party/WebKit/Source/platform/testing/FuzzedDataProvider.cpp
[modify] https://crrev.com/5ceed48c3549686d68ade225a1dc3becfb158427/third_party/WebKit/Source/platform/testing/FuzzedDataProvider.h

Project Member

Comment 20 by bugdroid1@chromium.org, Jun 21 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/b57b5b36e33de7d15c06615156e00a45c79a4c13

commit b57b5b36e33de7d15c06615156e00a45c79a4c13
Author: Hiroshige Hayashizaki <hiroshige@chromium.org>
Date: Wed Jun 21 01:06:24 2017

Refactor code around TextResourceDecoder::ShouldAutoDetect()

ShouldAutoDetect(), DetectTextEncoding() and SetEncoding() calls and
references to hint_* members are always coupled.
This CL moves these code to AutoDetectEncodingIfAllowed() to make
it clear that these are always coupled and executed only for
|kUseAllAutoDetection|.

Bug:  725654 , 732893
Change-Id: Ia5a295bb2f600a4566f28738d421266ebb900343
Reviewed-on: https://chromium-review.googlesource.com/540179
Commit-Queue: Hiroshige Hayashizaki <hiroshige@chromium.org>
Reviewed-by: Kouhei Ueno <kouhei@chromium.org>
Cr-Commit-Position: refs/heads/master@{#481055}
[modify] https://crrev.com/b57b5b36e33de7d15c06615156e00a45c79a4c13/third_party/WebKit/Source/core/html/parser/TextResourceDecoder.cpp
[modify] https://crrev.com/b57b5b36e33de7d15c06615156e00a45c79a4c13/third_party/WebKit/Source/core/html/parser/TextResourceDecoder.h

Project Member

Comment 21 by bugdroid1@chromium.org, Jun 21 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/ee353afe4b953401e505186f45e71e066592382a

commit ee353afe4b953401e505186f45e71e066592382a
Author: Hiroshige Hayashizaki <hiroshige@chromium.org>
Date: Wed Jun 21 06:21:40 2017

Introduce TextResourceDecoderOptions

This CL introduces TextResourceDecoderOptions and moves some members
and enums of TextResourceDecoder to TextResourceDecoderOptions,
so that the same TextResourceDecoderOptions result in the same
decoding result, including automatic encoding detection.

This is preparation for Issue 732893, where I'll stop reusing
Resource if TextResourceDecoderOptions is different.

Bug:  725654 , 732893
Change-Id: I9044b47f5d89f69a493e49036aa60cf32979f815
Reviewed-on: https://chromium-review.googlesource.com/540375
Commit-Queue: Hiroshige Hayashizaki <hiroshige@chromium.org>
Reviewed-by: Kouhei Ueno <kouhei@chromium.org>
Reviewed-by: Kinuko Yasuda <kinuko@chromium.org>
Reviewed-by: Takeshi Yoshino <tyoshino@chromium.org>
Cr-Commit-Position: refs/heads/master@{#481122}
[modify] https://crrev.com/ee353afe4b953401e505186f45e71e066592382a/third_party/WebKit/Source/bindings/core/v8/ScriptStreamer.cpp
[modify] https://crrev.com/ee353afe4b953401e505186f45e71e066592382a/third_party/WebKit/Source/core/fileapi/FileReaderLoader.cpp
[modify] https://crrev.com/ee353afe4b953401e505186f45e71e066592382a/third_party/WebKit/Source/core/html/parser/TextResourceDecoder.cpp
[modify] https://crrev.com/ee353afe4b953401e505186f45e71e066592382a/third_party/WebKit/Source/core/html/parser/TextResourceDecoder.h
[modify] https://crrev.com/ee353afe4b953401e505186f45e71e066592382a/third_party/WebKit/Source/core/html/parser/TextResourceDecoderForFuzzing.h
[modify] https://crrev.com/ee353afe4b953401e505186f45e71e066592382a/third_party/WebKit/Source/core/html/parser/TextResourceDecoderTest.cpp
[modify] https://crrev.com/ee353afe4b953401e505186f45e71e066592382a/third_party/WebKit/Source/core/html/track/vtt/VTTParser.cpp
[modify] https://crrev.com/ee353afe4b953401e505186f45e71e066592382a/third_party/WebKit/Source/core/inspector/InspectorPageAgent.cpp
[modify] https://crrev.com/ee353afe4b953401e505186f45e71e066592382a/third_party/WebKit/Source/core/loader/TextResourceDecoderBuilder.cpp
[modify] https://crrev.com/ee353afe4b953401e505186f45e71e066592382a/third_party/WebKit/Source/core/loader/TextResourceDecoderBuilder.h
[modify] https://crrev.com/ee353afe4b953401e505186f45e71e066592382a/third_party/WebKit/Source/core/loader/resource/CSSStyleSheetResource.cpp
[modify] https://crrev.com/ee353afe4b953401e505186f45e71e066592382a/third_party/WebKit/Source/core/loader/resource/DocumentResource.cpp
[modify] https://crrev.com/ee353afe4b953401e505186f45e71e066592382a/third_party/WebKit/Source/core/loader/resource/ScriptResource.cpp
[modify] https://crrev.com/ee353afe4b953401e505186f45e71e066592382a/third_party/WebKit/Source/core/loader/resource/StyleSheetResource.h
[modify] https://crrev.com/ee353afe4b953401e505186f45e71e066592382a/third_party/WebKit/Source/core/loader/resource/TextResource.cpp
[modify] https://crrev.com/ee353afe4b953401e505186f45e71e066592382a/third_party/WebKit/Source/core/loader/resource/TextResource.h
[modify] https://crrev.com/ee353afe4b953401e505186f45e71e066592382a/third_party/WebKit/Source/core/loader/resource/XSLStyleSheetResource.cpp
[modify] https://crrev.com/ee353afe4b953401e505186f45e71e066592382a/third_party/WebKit/Source/core/workers/WorkerScriptLoader.cpp
[modify] https://crrev.com/ee353afe4b953401e505186f45e71e066592382a/third_party/WebKit/Source/core/xmlhttprequest/XMLHttpRequest.cpp
[modify] https://crrev.com/ee353afe4b953401e505186f45e71e066592382a/third_party/WebKit/Source/modules/fetch/FetchDataLoader.cpp
[modify] https://crrev.com/ee353afe4b953401e505186f45e71e066592382a/third_party/WebKit/Source/platform/loader/BUILD.gn
[add] https://crrev.com/ee353afe4b953401e505186f45e71e066592382a/third_party/WebKit/Source/platform/loader/fetch/TextResourceDecoderOptions.cpp
[add] https://crrev.com/ee353afe4b953401e505186f45e71e066592382a/third_party/WebKit/Source/platform/loader/fetch/TextResourceDecoderOptions.h

Project Member

Comment 22 by bugdroid1@chromium.org, Jun 21 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/b17cd49a408ab1c8dfe642df07acab4f156f1514

commit b17cd49a408ab1c8dfe642df07acab4f156f1514
Author: Hiroshige Hayashizaki <hiroshige@chromium.org>
Date: Wed Jun 21 10:35:32 2017

Introduce NonTextResourceFactory

This is to simplify upcoming changes to charset handling.
NonTextResourceFactory::Create() and its overrides now don't have
charset argument and thus isn't needed to be modified by the
upcoming changes.

Bug:  725654 , 732893
Change-Id: I94104979bb65bdf65f98071343d9930d0a39fc1a
Reviewed-on: https://chromium-review.googlesource.com/540696
Commit-Queue: Hiroshige Hayashizaki <hiroshige@chromium.org>
Reviewed-by: Takeshi Yoshino <tyoshino@chromium.org>
Reviewed-by: Kouhei Ueno <kouhei@chromium.org>
Reviewed-by: Yutaka Hirano <yhirano@chromium.org>
Cr-Commit-Position: refs/heads/master@{#481166}
[modify] https://crrev.com/b17cd49a408ab1c8dfe642df07acab4f156f1514/third_party/WebKit/Source/core/loader/resource/FontResource.h
[modify] https://crrev.com/b17cd49a408ab1c8dfe642df07acab4f156f1514/third_party/WebKit/Source/core/loader/resource/ImageResource.cpp
[modify] https://crrev.com/b17cd49a408ab1c8dfe642df07acab4f156f1514/third_party/WebKit/Source/core/loader/resource/LinkFetchResource.h
[modify] https://crrev.com/b17cd49a408ab1c8dfe642df07acab4f156f1514/third_party/WebKit/Source/platform/loader/fetch/RawResource.h
[modify] https://crrev.com/b17cd49a408ab1c8dfe642df07acab4f156f1514/third_party/WebKit/Source/platform/loader/fetch/Resource.h
[modify] https://crrev.com/b17cd49a408ab1c8dfe642df07acab4f156f1514/third_party/WebKit/Source/platform/loader/testing/MockResource.cpp

Project Member

Comment 23 by bugdroid1@chromium.org, Jun 22 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/3987be28dc176cb9be540f2eb2202db2dca1b00a

commit 3987be28dc176cb9be540f2eb2202db2dca1b00a
Author: Hiroshige Hayashizaki <hiroshige@chromium.org>
Date: Thu Jun 22 00:14:57 2017

Use TextResourceDecoderOptions in FetchParameters and ResourceFactory

Previously, FetchParameters::Charset() was String, but to support UTF-8
enforcement in module scripts, the existing "UTF-8" requests (that can be
overridden by BOMs or HTTP headers) and enforced "UTF-8" requests must be
distinguished.

This CL uses TextResourceDecoderOptions (that can distinguish these two cases)
instead of String in ResourceFactory::Create() and FetchParameters.
The ContentType in TextResourceDecoderOptions in FetchParameters is
|kPlainTextContent| and is later overridden by ResourceFactory::ContentType()
in ResourceFetcher::RequestResource() before actual use.

As preparation for this CL, the preceding CLs replaced String that represents
an encoding with WTF::TextEncoding and introduced TextResourceDecoderOptions.

Bug:  725654 , 732893
Change-Id: I87e036059df08ce9f42779e2ce2fd694507e7571
Reviewed-on: https://chromium-review.googlesource.com/535983
Commit-Queue: Hiroshige Hayashizaki <hiroshige@chromium.org>
Reviewed-by: Takeshi Yoshino <tyoshino@chromium.org>
Reviewed-by: Kouhei Ueno <kouhei@chromium.org>
Reviewed-by: Kinuko Yasuda <kinuko@chromium.org>
Cr-Commit-Position: refs/heads/master@{#481368}
[modify] https://crrev.com/3987be28dc176cb9be540f2eb2202db2dca1b00a/third_party/WebKit/Source/bindings/core/v8/ScriptStreamerTest.cpp
[modify] https://crrev.com/3987be28dc176cb9be540f2eb2202db2dca1b00a/third_party/WebKit/Source/bindings/core/v8/V8ScriptRunnerTest.cpp
[modify] https://crrev.com/3987be28dc176cb9be540f2eb2202db2dca1b00a/third_party/WebKit/Source/core/html/parser/CSSPreloadScannerTest.cpp
[modify] https://crrev.com/3987be28dc176cb9be540f2eb2202db2dca1b00a/third_party/WebKit/Source/core/loader/resource/CSSStyleSheetResource.cpp
[modify] https://crrev.com/3987be28dc176cb9be540f2eb2202db2dca1b00a/third_party/WebKit/Source/core/loader/resource/CSSStyleSheetResource.h
[modify] https://crrev.com/3987be28dc176cb9be540f2eb2202db2dca1b00a/third_party/WebKit/Source/core/loader/resource/CSSStyleSheetResourceTest.cpp
[modify] https://crrev.com/3987be28dc176cb9be540f2eb2202db2dca1b00a/third_party/WebKit/Source/core/loader/resource/DocumentResource.cpp
[modify] https://crrev.com/3987be28dc176cb9be540f2eb2202db2dca1b00a/third_party/WebKit/Source/core/loader/resource/DocumentResource.h
[modify] https://crrev.com/3987be28dc176cb9be540f2eb2202db2dca1b00a/third_party/WebKit/Source/core/loader/resource/ScriptResource.cpp
[modify] https://crrev.com/3987be28dc176cb9be540f2eb2202db2dca1b00a/third_party/WebKit/Source/core/loader/resource/ScriptResource.h
[modify] https://crrev.com/3987be28dc176cb9be540f2eb2202db2dca1b00a/third_party/WebKit/Source/core/loader/resource/StyleSheetResource.h
[modify] https://crrev.com/3987be28dc176cb9be540f2eb2202db2dca1b00a/third_party/WebKit/Source/core/loader/resource/TextResource.cpp
[modify] https://crrev.com/3987be28dc176cb9be540f2eb2202db2dca1b00a/third_party/WebKit/Source/core/loader/resource/TextResource.h
[modify] https://crrev.com/3987be28dc176cb9be540f2eb2202db2dca1b00a/third_party/WebKit/Source/core/loader/resource/XSLStyleSheetResource.cpp
[modify] https://crrev.com/3987be28dc176cb9be540f2eb2202db2dca1b00a/third_party/WebKit/Source/core/loader/resource/XSLStyleSheetResource.h
[modify] https://crrev.com/3987be28dc176cb9be540f2eb2202db2dca1b00a/third_party/WebKit/Source/platform/loader/fetch/FetchParameters.cpp
[modify] https://crrev.com/3987be28dc176cb9be540f2eb2202db2dca1b00a/third_party/WebKit/Source/platform/loader/fetch/FetchParameters.h
[modify] https://crrev.com/3987be28dc176cb9be540f2eb2202db2dca1b00a/third_party/WebKit/Source/platform/loader/fetch/Resource.h
[modify] https://crrev.com/3987be28dc176cb9be540f2eb2202db2dca1b00a/third_party/WebKit/Source/platform/loader/fetch/ResourceFetcher.cpp
[modify] https://crrev.com/3987be28dc176cb9be540f2eb2202db2dca1b00a/third_party/WebKit/Source/platform/loader/fetch/ResourceFetcher.h
[modify] https://crrev.com/3987be28dc176cb9be540f2eb2202db2dca1b00a/third_party/WebKit/Source/platform/loader/fetch/TextResourceDecoderOptions.h

Project Member

Comment 24 by bugdroid1@chromium.org, Jun 22 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/b7bd2961b3386d09a426238a0a12a37bcb0b7e0a

commit b7bd2961b3386d09a426238a0a12a37bcb0b7e0a
Author: Hiroshige Hayashizaki <hiroshige@chromium.org>
Date: Thu Jun 22 00:54:21 2017

Do not override encoding when |kAlwaysUseUTF8ForText|

This is preparation for [1] to enforce UTF-8 encoding even if
UTF-16 BOMs or MIME types are supplied.
[1] https://chromium-review.googlesource.com/c/534456/

This CL doesn't affect the behavior because currently
|kAlwaysUseUTF8ForText| is used only by FetchDataLoader that doesn't
call SetEncoding().

Bug:  725654 
Change-Id: I997fa6152fd604300bcb3da6232c121b5bf332e5
Reviewed-on: https://chromium-review.googlesource.com/540698
Commit-Queue: Hiroshige Hayashizaki <hiroshige@chromium.org>
Reviewed-by: Kouhei Ueno <kouhei@chromium.org>
Cr-Commit-Position: refs/heads/master@{#481382}
[modify] https://crrev.com/b7bd2961b3386d09a426238a0a12a37bcb0b7e0a/third_party/WebKit/Source/core/html/parser/TextResourceDecoder.cpp

Project Member

Comment 25 by bugdroid1@chromium.org, Jun 22 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/1b0f374c61952f6b56762c29942c2a3b64e926cb

commit 1b0f374c61952f6b56762c29942c2a3b64e926cb
Author: Hiroshige Hayashizaki <hiroshige@chromium.org>
Date: Thu Jun 22 02:59:40 2017

Enforce UTF-8 decoding for module scripts

Bug:  725654 
Change-Id: I51495ca34d641d02bc2f73384900a5bd6b8543a3
Reviewed-on: https://chromium-review.googlesource.com/534456
Commit-Queue: Hiroshige Hayashizaki <hiroshige@chromium.org>
Reviewed-by: Kouhei Ueno <kouhei@chromium.org>
Cr-Commit-Position: refs/heads/master@{#481416}
[add] https://crrev.com/1b0f374c61952f6b56762c29942c2a3b64e926cb/third_party/WebKit/LayoutTests/external/wpt/html/semantics/scripting-1/the-script-element/module/charset-01.html
[add] https://crrev.com/1b0f374c61952f6b56762c29942c2a3b64e926cb/third_party/WebKit/LayoutTests/external/wpt/html/semantics/scripting-1/the-script-element/module/charset-02.html
[add] https://crrev.com/1b0f374c61952f6b56762c29942c2a3b64e926cb/third_party/WebKit/LayoutTests/external/wpt/html/semantics/scripting-1/the-script-element/module/charset-03.html
[add] https://crrev.com/1b0f374c61952f6b56762c29942c2a3b64e926cb/third_party/WebKit/LayoutTests/external/wpt/html/semantics/scripting-1/the-script-element/module/resources/import-non-utf8-with-charset-header.js
[add] https://crrev.com/1b0f374c61952f6b56762c29942c2a3b64e926cb/third_party/WebKit/LayoutTests/external/wpt/html/semantics/scripting-1/the-script-element/module/resources/import-non-utf8.js
[add] https://crrev.com/1b0f374c61952f6b56762c29942c2a3b64e926cb/third_party/WebKit/LayoutTests/external/wpt/html/semantics/scripting-1/the-script-element/module/resources/import-utf8-with-charset-header.js
[add] https://crrev.com/1b0f374c61952f6b56762c29942c2a3b64e926cb/third_party/WebKit/LayoutTests/external/wpt/html/semantics/scripting-1/the-script-element/module/resources/import-utf8.js
[add] https://crrev.com/1b0f374c61952f6b56762c29942c2a3b64e926cb/third_party/WebKit/LayoutTests/external/wpt/html/semantics/scripting-1/the-script-element/resources/bom-utf-16be.js
[add] https://crrev.com/1b0f374c61952f6b56762c29942c2a3b64e926cb/third_party/WebKit/LayoutTests/external/wpt/html/semantics/scripting-1/the-script-element/resources/bom-utf-16le.js
[add] https://crrev.com/1b0f374c61952f6b56762c29942c2a3b64e926cb/third_party/WebKit/LayoutTests/external/wpt/html/semantics/scripting-1/the-script-element/resources/bom-utf-8.js
[modify] https://crrev.com/1b0f374c61952f6b56762c29942c2a3b64e926cb/third_party/WebKit/Source/core/loader/modulescript/ModuleScriptLoader.cpp

Status: Fixed (was: Started)
Fixed.

Issue 732893 remains open, but I think we can unblock 594639 now because Issue 732893 has been existed for a long time (for classic scripts), and fixing it will take more time.

Sign in to add a comment