Project: chromium Issues People Development process History Sign in
New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.
Issue 39206 Issue with Chunked Transfer Encoding in HTTP Response
Starred by 8 users Reported by abhib...@gmail.com, Mar 24 2010 Back to list
Status: WontFix
Owner:
Closed: Mar 2010
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: All
Pri: ----
Type: Bug

Restricted
  • Only users with Commit permission may comment.



Sign in to add a comment
Chrome Version (from the about:version page): 5.0.307.11 (Official Build
39572) beta
Is this the most recent version: yes
OS + version: Debian Lenny 2.6.26-2-686
CPU architecture (32-bit / 64-bit): 32 bit
Window manager: LXDE
URLs (if relevant):
Behavior in Linux Firefox: Works
Behavior in Windows Chrome (if you have access to it):

What steps will reproduce the problem?
1. I'm sending a response from my server with chunked transfer encoding
2.
3.

What is the expected result?
Firefox keeps receiving the chunked transfer and XMLHttpRequest readyState
3 also displays the responseText for the chunked transfer.

What happens instead?
Chrome closes the connection after receiving the first packet.
XMLHttpRequest doesn't show anything for the chunked transfer on readyState 3.


Please provide any additional information below. Attach a screenshot
and backtrace if possible.

Attaching two wireshark dumps

 
firefox
922 KB Download
chrome
14.7 KB Download
Comment 1 by evan@chromium.org, Mar 24 2010
Labels: -Area-Undefined Area-Internals Internals-Network
Comment 3 by evan@chromium.org, Mar 24 2010
 Issue 39203  has been merged into this issue.
Comment 4 by eroman@chromium.org, Mar 24 2010
I doubt this is a chunked encoding issue,
but that rather, it is a WebKit charset sniffing problem.

Your XHR response (/?stocks) has:
    Content-Type: text/html

This means that when webkit is receiving the bytes, it will block in order to search for the charset, so it can 
decode the bytes properly.

This explains why in Chrome (or if you ran it in Safari for that matter), you are seeing an empty responseText 
on the initial ready state changes for 3.

Firefox gives you the text earlier.

Try sending your response as:
Content-Type: text/plain

And it won't run into this problem.

(Also, you should still be getting the full response on the ready state change = 4).
Comment 5 by abhib...@gmail.com, Mar 25 2010
Ok I will try changing the encoding and tell you what happens. 
Comment 6 by abhib...@gmail.com, Mar 25 2010
Hi eroman

I tried with Content Type: text/plain

I still face the same issue. Chrome resets the connection while Firefox continues to 
receive the Chunked response. 

I have attached the Wireshark Dumps. 

You suppose there is something wrong with the protocol strings being sent by my 
server?
firefoxwplaintxt
1.4 MB Download
chromewplaintxt
14.7 KB Download
Comment 7 by abhib...@gmail.com, Mar 25 2010
I tried out the same response with the latest Safari on Windows and it worked 
correctly. 

Safari kept processing the chunked response when I set the Content-Type: text/plain. 
However with Content-Type: text/html it didn't work.
Comment 8 by abhib...@gmail.com, Mar 25 2010
I tried out the response with a latest WebKit build and GtkLauncher. Same behaviour. 
It receives one or two chunks then disconnects.
Comment 9 by abhib...@gmail.com, Mar 25 2010
I pointed Chrome directly at the URL. 

This is what I got for an error:
Error 321 (net::ERR_INVALID_CHUNKED_ENCODING): Unknown error.

On Firefox and Safari it just loads.
Do you have a public URL that we could test against to reproduce this error?
Comment 11 by abhib...@gmail.com, Mar 25 2010
Hi willchan

I dont have a public url. I have provided wireshark dumps for the problem I am
facing. i suspect something could be wrong with my server's protocol string output.
Will someone be able to verify this?

In the meantime I shall see if I can get a public url.
Status: WontFix
I took at look at your wireshark dump, and indeed the chunked response is not quite right.
Here is a snippet from it:

9
ABB:105
8
PNG:58
9
VIN:140


The problem here is the chunks are not separated by CRLF (or alternately, the chunk size is off by 2).
The general format for chunked response is:

    <chunk_length> CRLF
    <data of length chunk_length> CRLF

If we look at the first line, the "9" defines the payload as:

'A', 'B', 'B', ':', '1', '0', '5', '\r', \n'

Therefore there is not chunk delimeter (\r\n) left before reaching the next chunk (8).

The solution is to add an extra CRLF after the chunk payload (or alternately if the inner CRLF is not supposed to be part of the payload, 
decrement the chunk length by 2).

Chrome is strict about requiring the chunk delimeters to be present, since this is what the spec says. I think we should keep this strict 
behavior unless a compatibility problem is found with a mainstream website.
Comment 13 by abhib...@gmail.com, Mar 29 2010
hi eroman 

Thanks for pointing out my mistake. Everything works now. :)

Right now I was baffled why this error was happening because I did fix this day 
before yesterday except I missed out the \r\n size calculation on one line. For 
Firefox and Safari it didn't make a difference whereas Chrome kept breaking.

So now I would like to know why were other browsers accepting the chunked output? Is 
it possible that at times network errors might cause a chunk to have extra data or 
data missing from it? Why would Firefox and Safari not be as strict as Chrome is?

Also what potential security hazards could happen in Firefox and Safari as a result 
of not being strict with chunk sizes?
Labels: -OS-Linux OS-All
> So now I would like to know why were other browsers accepting the chunked output?

In general implementations end up being more permissive by accident, although at times it is intentional for 
compatibility reasons.

Writing parsing code is tedious, and implementations often differ because of shortcuts taken in the 
implementation, and not explicitly disallowing malformed inputs. As an example, in Firefox's implementation 
of chunked decoding it uses sscanf("%x") to parse the chunk length. Consequently it is more permissive than 
what the RFC stipulates as the number format (sscanf allows things like a leading '+'/'-', and whitespace 
padding).

As a website author I suggest matching the spec as closely as possible, since this gives the site the best 
chance of working across all clients, future and current (there are lots of mobile devices and proxy servers 
which have their own set of quirks).

> Also what potential security hazards could happen in Firefox and Safari as a result
> of not being strict with chunk sizes?

I don't see a security concern in this instance. In general though it is safer to fail than to interpret what seems 
like malformed data.
Project Member Comment 15 by bugdroid1@chromium.org, Oct 12 2012
Labels: Restrict-AddIssueComment-Commit
This issue has been closed for some time. No one will pay attention to new comments.
If you are seeing this bug or have new data, please click New Issue to start a new bug.
Project Member Comment 16 by bugdroid1@chromium.org, Mar 10 2013
Labels: -Area-Internals -Internals-Network Cr-Internals Cr-Internals-Network
Sign in to add a comment