The current code always writes XHR with responseType=blob into files. This is to avoid extra copies and OOM with large blobs. See https://docs.google.com/document/d/1pWFNc6paT_kohDbYGd3qYtPWVw82SPWv1LR0VxZ9Q5I/edit# and the original bug 269055 . We want to avoid the network service knowing about blobs or needing logic to write the response to disk. A solution for implementing the XHR blob scenario is to have the renderer transfer the data pipe to the blob service, which then copies it to disk as currently occurs. Some background: https://docs.google.com/a/chromium.org/document/d/1fjZGvZe7lyh3bH_mIS2VP6NYgY_tUEahwh17ALbG7JM/edit?usp=sharing Assigning to Daniel per chat (but feel free to redirect!).
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/12e1a4a7d02ee7c37886591ce6fa12d66636b9c3 commit 12e1a4a7d02ee7c37886591ce6fa12d66636b9c3 Author: John Abd-El-Malek <jam@chromium.org> Date: Fri Aug 11 05:52:36 2017 Disable XHRs with responseType=blob saving the response to disk when the network service is in use. This code path isn't hooked up yet. Until then, store the blob in memory. This way we can make tests that aren't related to this bug pass. BUG= 754493 Change-Id: Iffe0d5c64e09afd2fd86ad18012cb8291d2d1533 Reviewed-on: https://chromium-review.googlesource.com/611406 Commit-Queue: John Abd-El-Malek <jam@chromium.org> Reviewed-by: Daniel Murphy <dmurph@chromium.org> Reviewed-by: Kenneth Russell <kbr@chromium.org> Cr-Commit-Position: refs/heads/master@{#493675} [modify] https://crrev.com/12e1a4a7d02ee7c37886591ce6fa12d66636b9c3/content/child/runtime_features.cc [modify] https://crrev.com/12e1a4a7d02ee7c37886591ce6fa12d66636b9c3/third_party/WebKit/Source/core/xmlhttprequest/XMLHttpRequest.cpp [modify] https://crrev.com/12e1a4a7d02ee7c37886591ce6fa12d66636b9c3/third_party/WebKit/Source/platform/RuntimeEnabledFeatures.json5 [modify] https://crrev.com/12e1a4a7d02ee7c37886591ce6fa12d66636b9c3/third_party/WebKit/Source/platform/exported/WebRuntimeFeatures.cpp [modify] https://crrev.com/12e1a4a7d02ee7c37886591ce6fa12d66636b9c3/third_party/WebKit/public/platform/WebRuntimeFeatures.h
Apologies, applied the wrong component in bulk.
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/bfa8838b96797309bb38692b11ff0ebb445ec873 commit bfa8838b96797309bb38692b11ff0ebb445ec873 Author: Marijn Kruisselbrink <mek@chromium.org> Date: Mon Feb 26 19:29:10 2018 [Blobs] Add API to create a Blob from streaming data. This API can be used to replace download_to_file functionality in the network stack (currently only used by XHR) by letting blink instead pass the data pipe that is the result of fetching some data to the blob system. Bug: 754493 , 712693 , 791702 Change-Id: I75c5289a6e7e7823e32838d9ada76d74028d8e40 Reviewed-on: https://chromium-review.googlesource.com/916731 Commit-Queue: Marijn Kruisselbrink <mek@chromium.org> Reviewed-by: Kinuko Yasuda <kinuko@chromium.org> Reviewed-by: Daniel Murphy <dmurph@chromium.org> Cr-Commit-Position: refs/heads/master@{#539233} [modify] https://crrev.com/bfa8838b96797309bb38692b11ff0ebb445ec873/storage/browser/BUILD.gn [add] https://crrev.com/bfa8838b96797309bb38692b11ff0ebb445ec873/storage/browser/blob/blob_builder_from_stream.cc [add] https://crrev.com/bfa8838b96797309bb38692b11ff0ebb445ec873/storage/browser/blob/blob_builder_from_stream.h [add] https://crrev.com/bfa8838b96797309bb38692b11ff0ebb445ec873/storage/browser/blob/blob_builder_from_stream_unittest.cc [modify] https://crrev.com/bfa8838b96797309bb38692b11ff0ebb445ec873/storage/browser/blob/blob_data_item.cc [modify] https://crrev.com/bfa8838b96797309bb38692b11ff0ebb445ec873/storage/browser/blob/blob_data_item.h [modify] https://crrev.com/bfa8838b96797309bb38692b11ff0ebb445ec873/storage/browser/blob/blob_memory_controller.cc [modify] https://crrev.com/bfa8838b96797309bb38692b11ff0ebb445ec873/storage/browser/blob/blob_memory_controller.h [modify] https://crrev.com/bfa8838b96797309bb38692b11ff0ebb445ec873/storage/browser/blob/blob_registry_impl.cc [modify] https://crrev.com/bfa8838b96797309bb38692b11ff0ebb445ec873/storage/browser/blob/blob_registry_impl.h [modify] https://crrev.com/bfa8838b96797309bb38692b11ff0ebb445ec873/storage/browser/blob/blob_registry_impl_unittest.cc [modify] https://crrev.com/bfa8838b96797309bb38692b11ff0ebb445ec873/storage/browser/blob/blob_storage_context.cc [modify] https://crrev.com/bfa8838b96797309bb38692b11ff0ebb445ec873/storage/browser/blob/blob_storage_context.h [modify] https://crrev.com/bfa8838b96797309bb38692b11ff0ebb445ec873/storage/browser/blob/scoped_file.cc [modify] https://crrev.com/bfa8838b96797309bb38692b11ff0ebb445ec873/storage/browser/blob/scoped_file.h [modify] https://crrev.com/bfa8838b96797309bb38692b11ff0ebb445ec873/storage/browser/blob/shareable_file_reference.h [modify] https://crrev.com/bfa8838b96797309bb38692b11ff0ebb445ec873/third_party/WebKit/Source/core/fileapi/PublicURLManager.cpp [modify] https://crrev.com/bfa8838b96797309bb38692b11ff0ebb445ec873/third_party/WebKit/Source/core/xmlhttprequest/XMLHttpRequest.h [modify] https://crrev.com/bfa8838b96797309bb38692b11ff0ebb445ec873/third_party/WebKit/Source/platform/blob/BlobData.h [modify] https://crrev.com/bfa8838b96797309bb38692b11ff0ebb445ec873/third_party/WebKit/Source/platform/blob/testing/FakeBlobRegistry.cpp [modify] https://crrev.com/bfa8838b96797309bb38692b11ff0ebb445ec873/third_party/WebKit/Source/platform/blob/testing/FakeBlobRegistry.h [modify] https://crrev.com/bfa8838b96797309bb38692b11ff0ebb445ec873/third_party/WebKit/common/BUILD.gn [modify] https://crrev.com/bfa8838b96797309bb38692b11ff0ebb445ec873/third_party/WebKit/public/mojom/blob/blob_registry.mojom [add] https://crrev.com/bfa8838b96797309bb38692b11ff0ebb445ec873/third_party/WebKit/public/mojom/blob/data_element.mojom
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/34ae3947b6d2ae5c3eee7d99a8301436c408a53f commit 34ae3947b6d2ae5c3eee7d99a8301436c408a53f Author: Marijn Kruisselbrink <mek@chromium.org> Date: Wed Feb 28 00:14:22 2018 [Blobs] Handle length_hint when creating a blob from stream. Pre-allocate enough space for length_hint bytes when starting to create a blob from a stream. Also don't allocate more space until we're sure the data pipe is going to send more data. Bug: 754493 Change-Id: I6a7c888f6fd0687ec976aa8c54b469e8454c7f02 Reviewed-on: https://chromium-review.googlesource.com/938285 Commit-Queue: Marijn Kruisselbrink <mek@chromium.org> Reviewed-by: Daniel Murphy <dmurph@chromium.org> Cr-Commit-Position: refs/heads/master@{#539595} [modify] https://crrev.com/34ae3947b6d2ae5c3eee7d99a8301436c408a53f/storage/browser/blob/blob_builder_from_stream.cc [modify] https://crrev.com/34ae3947b6d2ae5c3eee7d99a8301436c408a53f/storage/browser/blob/blob_builder_from_stream.h [modify] https://crrev.com/34ae3947b6d2ae5c3eee7d99a8301436c408a53f/storage/browser/blob/blob_builder_from_stream_unittest.cc
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/70f80afddade04adeceebc0120db3ba442db1db0 commit 70f80afddade04adeceebc0120db3ba442db1db0 Author: Marijn Kruisselbrink <mek@chromium.org> Date: Wed Feb 28 03:31:13 2018 [Blobs] Grow files when creating a blob from stream. Rather than creating a new file, extend existing files until they reach their max size when streaming to a blob on disk. Bug: 754493 Change-Id: Icce67682011c9d0e31fbdc1ef0f180bc46b74b9e Reviewed-on: https://chromium-review.googlesource.com/938671 Reviewed-by: Daniel Murphy <dmurph@chromium.org> Commit-Queue: Marijn Kruisselbrink <mek@chromium.org> Cr-Commit-Position: refs/heads/master@{#539690} [modify] https://crrev.com/70f80afddade04adeceebc0120db3ba442db1db0/storage/browser/blob/blob_builder_from_stream.cc [modify] https://crrev.com/70f80afddade04adeceebc0120db3ba442db1db0/storage/browser/blob/blob_builder_from_stream.h [modify] https://crrev.com/70f80afddade04adeceebc0120db3ba442db1db0/storage/browser/blob/blob_builder_from_stream_unittest.cc [modify] https://crrev.com/70f80afddade04adeceebc0120db3ba442db1db0/storage/browser/blob/blob_data_item.cc [modify] https://crrev.com/70f80afddade04adeceebc0120db3ba442db1db0/storage/browser/blob/blob_data_item.h [modify] https://crrev.com/70f80afddade04adeceebc0120db3ba442db1db0/storage/browser/blob/blob_memory_controller.cc [modify] https://crrev.com/70f80afddade04adeceebc0120db3ba442db1db0/storage/browser/blob/blob_memory_controller.h
I started writing a design doc for this at https://docs.google.com/document/d/1V_rFqFeYc_XpzEc_hGF9Wlat5mH62lREJ1I1dCJ4lX0/edit?usp=sharing
Issue 232257 has been merged into this issue.
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/090b1a4fea8bd71d2bd5e3503d3397985ad58022 commit 090b1a4fea8bd71d2bd5e3503d3397985ad58022 Author: Marijn Kruisselbrink <mek@chromium.org> Date: Fri Mar 30 19:54:12 2018 [XHR] Replace usage of download_to_file with new download_to_blob feature. download_to_file isn't really implementable with the network service (since there is no central browser-side place where all fetches pass through), and because we'd like blobs downloaded by XHR to be managed by the blob system anyway, this replaces download_to_file with a new download_to_blob feature. With this, blink::ResourceLoader redirects the datapipe it receives from a URLLoader to the blob system, and passes the resulting blob back to XHR. Design doc: https://docs.google.com/document/d/1V_rFqFeYc_XpzEc_hGF9Wlat5mH62lREJ1I1dCJ4lX0/edit#heading=h.2ndyupa99wqr Bug: 754493 , 712693 , 791702 Change-Id: I258150937c424b72824dadbc2832764ed54c1ca5 Reviewed-on: https://chromium-review.googlesource.com/955957 Reviewed-by: Lei Zhang <thestig@chromium.org> Reviewed-by: Yutaka Hirano <yhirano@chromium.org> Reviewed-by: Kinuko Yasuda <kinuko@chromium.org> Commit-Queue: Marijn Kruisselbrink <mek@chromium.org> Cr-Commit-Position: refs/heads/master@{#547248} [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/chrome/renderer/extensions/extension_localization_peer.cc [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/chrome/renderer/extensions/extension_localization_peer.h [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/chrome/renderer/extensions/extension_localization_peer_unittest.cc [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/chrome/renderer/security_filter_peer.cc [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/chrome/renderer/security_filter_peer.h [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/content/public/renderer/request_peer.h [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/content/renderer/loader/resource_dispatcher.cc [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/content/renderer/loader/resource_dispatcher.h [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/content/renderer/loader/resource_dispatcher_unittest.cc [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/content/renderer/loader/sync_load_context.cc [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/content/renderer/loader/sync_load_context.h [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/content/renderer/loader/sync_load_response.h [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/content/renderer/loader/test_request_peer.cc [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/content/renderer/loader/test_request_peer.h [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/content/renderer/loader/url_loader_client_impl.cc [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/content/renderer/loader/url_loader_client_impl.h [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/content/renderer/loader/url_loader_client_impl_unittest.cc [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/content/renderer/loader/url_response_body_consumer_unittest.cc [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/content/renderer/loader/web_url_loader_impl.cc [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/content/renderer/loader/web_url_loader_impl.h [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/content/renderer/loader/web_url_loader_impl_unittest.cc [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/third_party/WebKit/Source/core/inspector/InspectorNetworkAgent.cpp [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/third_party/WebKit/Source/core/inspector/InspectorNetworkAgent.h [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/third_party/WebKit/Source/core/inspector/NetworkResourcesData.cpp [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/third_party/WebKit/Source/core/inspector/NetworkResourcesData.h [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/third_party/WebKit/Source/core/loader/DocumentThreadableLoader.cpp [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/third_party/WebKit/Source/core/loader/DocumentThreadableLoader.h [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/third_party/WebKit/Source/core/loader/FrameFetchContext.cpp [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/third_party/WebKit/Source/core/loader/FrameFetchContext.h [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/third_party/WebKit/Source/core/loader/ThreadableLoaderClient.h [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/third_party/WebKit/Source/core/loader/WorkerThreadableLoader.cpp [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/third_party/WebKit/Source/core/loader/WorkerThreadableLoader.h [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/third_party/WebKit/Source/core/probe/CoreProbes.json5 [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/third_party/WebKit/Source/core/probe/CoreProbes.pidl [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/third_party/WebKit/Source/core/xmlhttprequest/XMLHttpRequest.cpp [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/third_party/WebKit/Source/core/xmlhttprequest/XMLHttpRequest.h [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/third_party/WebKit/Source/platform/exported/WebURLRequest.cpp [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/third_party/WebKit/Source/platform/loader/fetch/FetchContext.cpp [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/third_party/WebKit/Source/platform/loader/fetch/FetchContext.h [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/third_party/WebKit/Source/platform/loader/fetch/RawResource.cpp [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/third_party/WebKit/Source/platform/loader/fetch/RawResource.h [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/third_party/WebKit/Source/platform/loader/fetch/Resource.h [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/third_party/WebKit/Source/platform/loader/fetch/ResourceFetcher.cpp [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/third_party/WebKit/Source/platform/loader/fetch/ResourceLoader.cpp [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/third_party/WebKit/Source/platform/loader/fetch/ResourceLoader.h [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/third_party/WebKit/Source/platform/loader/fetch/ResourceRequest.cpp [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/third_party/WebKit/Source/platform/loader/fetch/ResourceRequest.h [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/third_party/WebKit/Source/platform/testing/weburl_loader_mock.cc [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/third_party/WebKit/Source/platform/testing/weburl_loader_mock.h [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/third_party/WebKit/public/platform/WebURLLoader.h [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/third_party/WebKit/public/platform/WebURLLoaderClient.h [modify] https://crrev.com/090b1a4fea8bd71d2bd5e3503d3397985ad58022/third_party/WebKit/public/platform/WebURLRequest.h
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/b31800d9f07d054a3b68ae7b917899baea017bc6 commit b31800d9f07d054a3b68ae7b917899baea017bc6 Author: Marijn Kruisselbrink <mek@chromium.org> Date: Mon Apr 02 18:19:21 2018 [Blobs] Add support for progress events to BlobBuilderFromStream. And use that in the XHR code. Bug: 754493 , 712693 , 791702 Change-Id: Ic0904558d43131cc2240805eaa794359f6a3f1d7 Reviewed-on: https://chromium-review.googlesource.com/957600 Commit-Queue: Marijn Kruisselbrink <mek@chromium.org> Reviewed-by: Tom Sepez <tsepez@chromium.org> Reviewed-by: Yutaka Hirano <yhirano@chromium.org> Reviewed-by: Daniel Murphy <dmurph@chromium.org> Cr-Commit-Position: refs/heads/master@{#547471} [modify] https://crrev.com/b31800d9f07d054a3b68ae7b917899baea017bc6/content/renderer/loader/sync_load_context.cc [modify] https://crrev.com/b31800d9f07d054a3b68ae7b917899baea017bc6/storage/browser/BUILD.gn [modify] https://crrev.com/b31800d9f07d054a3b68ae7b917899baea017bc6/storage/browser/blob/blob_builder_from_stream.cc [modify] https://crrev.com/b31800d9f07d054a3b68ae7b917899baea017bc6/storage/browser/blob/blob_builder_from_stream.h [modify] https://crrev.com/b31800d9f07d054a3b68ae7b917899baea017bc6/storage/browser/blob/blob_builder_from_stream_unittest.cc [modify] https://crrev.com/b31800d9f07d054a3b68ae7b917899baea017bc6/storage/browser/blob/blob_registry_impl.cc [modify] https://crrev.com/b31800d9f07d054a3b68ae7b917899baea017bc6/storage/browser/blob/blob_registry_impl.h [modify] https://crrev.com/b31800d9f07d054a3b68ae7b917899baea017bc6/storage/browser/blob/blob_registry_impl_unittest.cc [add] https://crrev.com/b31800d9f07d054a3b68ae7b917899baea017bc6/storage/browser/test/fake_progress_client.cc [add] https://crrev.com/b31800d9f07d054a3b68ae7b917899baea017bc6/storage/browser/test/fake_progress_client.h [modify] https://crrev.com/b31800d9f07d054a3b68ae7b917899baea017bc6/third_party/WebKit/Source/platform/blob/testing/FakeBlobRegistry.cpp [modify] https://crrev.com/b31800d9f07d054a3b68ae7b917899baea017bc6/third_party/WebKit/Source/platform/blob/testing/FakeBlobRegistry.h [modify] https://crrev.com/b31800d9f07d054a3b68ae7b917899baea017bc6/third_party/WebKit/Source/platform/loader/fetch/ResourceLoader.cpp [modify] https://crrev.com/b31800d9f07d054a3b68ae7b917899baea017bc6/third_party/WebKit/Source/platform/loader/fetch/ResourceLoader.h [modify] https://crrev.com/b31800d9f07d054a3b68ae7b917899baea017bc6/third_party/WebKit/public/mojom/blob/blob_registry.mojom
Comment 1 by bugdroid1@chromium.org
, Aug 11 2017