Fetch API: BOM is lost when Request and Response's |body| is retrieved as form data |
||
Issue description
This causes "request.formData() with input: test=" and "response.formData() with input: test=" to fail in LayoutTests/external/wpt/url/urlencoded-parser.html:
FAIL request.formData() with input: test= assert_array_equals: property 0, expected "test" but got "test"
FAIL response.formData() with input: test= assert_array_equals: property 0, expected "test" but got "test"
The strings above are misleading, as test's actually creating Request and Response objects like this:
let init = new Request("about:blank", { body: "\uFEFFtest=\uFEFF", method: "LADIDA", headers: {"Content-Type": "application/x-www-form-urlencoded;charset=windows-1252"} }).formData()
"\uFEFF" is the UTF-16 BOM.
When |body| is being extracted (https://fetch.spec.whatwg.org/#concept-bodyinit-extract), we parse a USVString ("\uFEFFtest=\uFEFF") and correctly encode it into UTF-8 ("\xef\xbb\xbftest=\xef\xbb\xbf").
However, if we later analyze that request or response by calling formData(), json() or text(), we can see that the initial "\uFEFF" is lost and we're left with "test=\uFEFF". https://url.spec.whatwg.org/#urlencoded-parsing says we should just decode the UTF-8 string without BOM, we shouldn't be stripping the BOM from the beginning.
The BOM is being stripped because TextResourceDecoder::Decode() unconditionally decodes content without the BOM if it finds one.
,
Dec 20 2017
Thanks for pointing out that json() and text() are working as expected. I've filed https://github.com/whatwg/fetch/issues/650 to see if there's anything that should be changed in the spec.
,
Dec 21 2017
It doesn't look like the spec's going to change according to the discussion there, so I guess I should look into changing the Blink implementation.
,
Jun 12 2018
|
||
►
Sign in to add a comment |
||
Comment 1 by hirosh...@chromium.org
, Dec 19 2017Components: -Blink>HTML>Parser
Summary: Fetch API: BOM is lost when Request and Response's |body| is retrieved as form data (was: Fetch API: BOM is lost when Request and Response's |body| is retrieved as form data, json or text)