When using chunked encoding, each chunk size is limited to 2GB
Reported by
romain.f...@gmail.com,
Jul 22 2016
|
|||||
Issue descriptionUserAgent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.75 Safari/537.36 Example URL: Steps to reproduce the problem: 1. run the attached ./test_server.py 2. open http://127.0.0.1:8080/ 3. the download will fail What is the expected behavior? the download should work What went wrong? rfc7230 does not specify any limit on the chunk-size, it just defines: chunk-size = 1*HEXDIG in http_chunked_decoder.cc HttpChunkedDecoder::ParseChunkSize uses HexStringToInt to parse the chunk-size. This restricts the chunk size to 2^31-1 Using an uint64_t and HexStringToUInt64 would be better Firefox seems to be using a uint and be limited to 2^32-1 Did this work before? No Chrome version: 49.0.2623.75 Channel: stable OS Version: Flash Version: Shockwave Flash 22.0 r0
,
Jul 22 2016
Thanks for the high quality bug report, much appreciated! You're absolutely correct about there being no specced max chunk size. I'll take this on myself. Not high priority, but it's a simple enough thing to fix, and a silly reason to fail requests.
,
Jul 25 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/8ea249f8b70f154f9995ed538fc853fe9cf46503 commit 8ea249f8b70f154f9995ed538fc853fe9cf46503 Author: mmenke <mmenke@chromium.org> Date: Mon Jul 25 19:49:16 2016 HttpChunkedDecoder: Support chunks longer than 2^31-1 bytes. We were using HexStringToInt to parse chunk size, which returns a 32-bit int. This CL switches to using HexStringToInt64, which uses 64-bit ints, so we can now support chunks up to 2^63-1 bytes. That should be enough for anybody. [Cue dramatic music] BUG= 630680 Review-Url: https://codereview.chromium.org/2170133004 Cr-Commit-Position: refs/heads/master@{#407549} [modify] https://crrev.com/8ea249f8b70f154f9995ed538fc853fe9cf46503/net/http/http_chunked_decoder.cc [modify] https://crrev.com/8ea249f8b70f154f9995ed538fc853fe9cf46503/net/http/http_chunked_decoder.h [modify] https://crrev.com/8ea249f8b70f154f9995ed538fc853fe9cf46503/net/http/http_chunked_decoder_unittest.cc
,
Jul 25 2016
,
Jul 25 2016
Thanks for the fix, but wouldn't it be better to use uint64_t instead ?
,
Jul 25 2016
I kept it signed so that the existing logic just worked - mixing ints and uints when not needed is just asking for trouble, and most place in net/, we use ints. I don't think there's much gain from being able to handle chunks from 8 to 16 exabytes in size, anyways - doubling the size just doesn't get us much. There are reasonable use cases for chunk sizes from 2 GB to 8 exabytes, however.
,
Jul 26 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/2321e8a45603ca595296b67e4d064c070f2b3591 commit 2321e8a45603ca595296b67e4d064c070f2b3591 Author: mpearson <mpearson@chromium.org> Date: Tue Jul 26 00:33:06 2016 Revert of HttpChunkedDecoder: Support chunks longer than 2^31-1 bytes. (patchset #4 id:60001 of https://codereview.chromium.org/2170133004/ ) Reason for revert: Speculative revert for bug 631246. Have no better ideas. Apologies if this wasn't the cause. Original issue's description: > HttpChunkedDecoder: Support chunks longer than 2^31-1 bytes. > > We were using HexStringToInt to parse chunk size, which returns a > 32-bit int. This CL switches to using HexStringToInt64, which uses > 64-bit ints, so we can now support chunks up to 2^63-1 bytes. > > That should be enough for anybody. [Cue dramatic music] > > BUG= 630680 > > Committed: https://crrev.com/8ea249f8b70f154f9995ed538fc853fe9cf46503 > Cr-Commit-Position: refs/heads/master@{#407549} TBR=eroman@chromium.org,mmenke@chromium.org # Skipping CQ checks because original CL landed less than 1 days ago. NOPRESUBMIT=true NOTREECHECKS=true NOTRY=true BUG= 630680 Review-Url: https://codereview.chromium.org/2183433003 Cr-Commit-Position: refs/heads/master@{#407651} [modify] https://crrev.com/2321e8a45603ca595296b67e4d064c070f2b3591/net/http/http_chunked_decoder.cc [modify] https://crrev.com/2321e8a45603ca595296b67e4d064c070f2b3591/net/http/http_chunked_decoder.h [modify] https://crrev.com/2321e8a45603ca595296b67e4d064c070f2b3591/net/http/http_chunked_decoder_unittest.cc
,
Jul 26 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/3fca2b4fdb391fa3c14386af8b72fc2092657029 commit 3fca2b4fdb391fa3c14386af8b72fc2092657029 Author: mmenke <mmenke@chromium.org> Date: Tue Jul 26 04:01:41 2016 Reland of HttpChunkedDecoder: Support chunks longer than 2^31-1 bytes. (patchset #1 id:1 of https://codereview.chromium.org/2183433003/ ) Reason for revert: The CL was blamed for compile failures it didn't cause, re-landing. Original issue's description: > Revert of HttpChunkedDecoder: Support chunks longer than 2^31-1 bytes. (patchset #4 id:60001 of https://codereview.chromium.org/2170133004/ ) > > Reason for revert: > Speculative revert for bug 631246. Have no better ideas. > > Apologies if this wasn't the cause. > > Original issue's description: > > HttpChunkedDecoder: Support chunks longer than 2^31-1 bytes. > > > > We were using HexStringToInt to parse chunk size, which returns a > > 32-bit int. This CL switches to using HexStringToInt64, which uses > > 64-bit ints, so we can now support chunks up to 2^63-1 bytes. > > > > That should be enough for anybody. [Cue dramatic music] > > > > BUG= 630680 > > > > Committed: https://crrev.com/8ea249f8b70f154f9995ed538fc853fe9cf46503 > > Cr-Commit-Position: refs/heads/master@{#407549} > > TBR=eroman@chromium.org,mmenke@chromium.org > # Skipping CQ checks because original CL landed less than 1 days ago. > NOPRESUBMIT=true > NOTREECHECKS=true > NOTRY=true > BUG= 630680 > > Committed: https://crrev.com/2321e8a45603ca595296b67e4d064c070f2b3591 > Cr-Commit-Position: refs/heads/master@{#407651} TBR=eroman@chromium.org,mpearson@chromium.org # Skipping CQ checks because original CL landed less than 1 days ago. NOPRESUBMIT=true NOTREECHECKS=true NOTRY=true BUG= 630680 Review-Url: https://codereview.chromium.org/2180063002 Cr-Commit-Position: refs/heads/master@{#407700} [modify] https://crrev.com/3fca2b4fdb391fa3c14386af8b72fc2092657029/net/http/http_chunked_decoder.cc [modify] https://crrev.com/3fca2b4fdb391fa3c14386af8b72fc2092657029/net/http/http_chunked_decoder.h [modify] https://crrev.com/3fca2b4fdb391fa3c14386af8b72fc2092657029/net/http/http_chunked_decoder_unittest.cc
,
Jul 26 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/3fca2b4fdb391fa3c14386af8b72fc2092657029 commit 3fca2b4fdb391fa3c14386af8b72fc2092657029 Author: mmenke <mmenke@chromium.org> Date: Tue Jul 26 04:01:41 2016 Reland of HttpChunkedDecoder: Support chunks longer than 2^31-1 bytes. (patchset #1 id:1 of https://codereview.chromium.org/2183433003/ ) Reason for revert: The CL was blamed for compile failures it didn't cause, re-landing. Original issue's description: > Revert of HttpChunkedDecoder: Support chunks longer than 2^31-1 bytes. (patchset #4 id:60001 of https://codereview.chromium.org/2170133004/ ) > > Reason for revert: > Speculative revert for bug 631246. Have no better ideas. > > Apologies if this wasn't the cause. > > Original issue's description: > > HttpChunkedDecoder: Support chunks longer than 2^31-1 bytes. > > > > We were using HexStringToInt to parse chunk size, which returns a > > 32-bit int. This CL switches to using HexStringToInt64, which uses > > 64-bit ints, so we can now support chunks up to 2^63-1 bytes. > > > > That should be enough for anybody. [Cue dramatic music] > > > > BUG= 630680 > > > > Committed: https://crrev.com/8ea249f8b70f154f9995ed538fc853fe9cf46503 > > Cr-Commit-Position: refs/heads/master@{#407549} > > TBR=eroman@chromium.org,mmenke@chromium.org > # Skipping CQ checks because original CL landed less than 1 days ago. > NOPRESUBMIT=true > NOTREECHECKS=true > NOTRY=true > BUG= 630680 > > Committed: https://crrev.com/2321e8a45603ca595296b67e4d064c070f2b3591 > Cr-Commit-Position: refs/heads/master@{#407651} TBR=eroman@chromium.org,mpearson@chromium.org # Skipping CQ checks because original CL landed less than 1 days ago. NOPRESUBMIT=true NOTREECHECKS=true NOTRY=true BUG= 630680 Review-Url: https://codereview.chromium.org/2180063002 Cr-Commit-Position: refs/heads/master@{#407700} [modify] https://crrev.com/3fca2b4fdb391fa3c14386af8b72fc2092657029/net/http/http_chunked_decoder.cc [modify] https://crrev.com/3fca2b4fdb391fa3c14386af8b72fc2092657029/net/http/http_chunked_decoder.h [modify] https://crrev.com/3fca2b4fdb391fa3c14386af8b72fc2092657029/net/http/http_chunked_decoder_unittest.cc
,
Jul 6
,
Jul 6
|
|||||
►
Sign in to add a comment |
|||||
Comment 1 by mef@chromium.org
, Jul 22 2016Labels: -OS-Linux -Type-Bug -Pri-2 OS-All Pri-3 Type-Feature
Status: Untriaged (was: Unconfirmed)