New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 706286 link

Starred by 2 users

Issue metadata

Status: Fixed
Owner:
Closed: Oct 31
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Feature

Blocking:
issue 724323



Sign in to add a comment

Stop using channel-associated URLLoader and URLLoaderFactory

Project Member Reported by yhirano@chromium.org, Mar 29 2017

Issue description

Currently we are using channel-associated URLLoaderFactory and URLLoader to keep the ordering with other IPC messages. Stop doing that.

 

Comment 1 by jam@chromium.org, May 24 2017

Blocking: 724323
Project Member

Comment 2 by bugdroid1@chromium.org, May 29 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/f510d0c432ea0330c394e4a1349d6e7ee31c116e

commit f510d0c432ea0330c394e4a1349d6e7ee31c116e
Author: Yutaka Hirano <yhirano@chromium.org>
Date: Mon May 29 15:30:31 2017

Use Independent URLLoader

This CL removes "associated" for URLLoader in
URLLoaderFactory::CreateLoaderAndStart.

The associated keyword was needed in order to keep the ordering guarantee
which the current ChromeIPC provides, but it is harmful from the network
servicification POV.

As all tests pass without the keyword (when kLoadingWithMojo is enabled),
let's stop using the associated interface.

Bug:  706286 ,  724323 
Change-Id: I771ee5ed35ca882cfa77c5e7ce2e862261c3f62d
Reviewed-on: https://chromium-review.googlesource.com/513643
Commit-Queue: Yutaka Hirano <yhirano@chromium.org>
Reviewed-by: Daniel Cheng <dcheng@chromium.org>
Reviewed-by: Kinuko Yasuda <kinuko@chromium.org>
Reviewed-by: John Abd-El-Malek <jam@chromium.org>
Cr-Commit-Position: refs/heads/master@{#475356}
[modify] https://crrev.com/f510d0c432ea0330c394e4a1349d6e7ee31c116e/content/browser/appcache/appcache_url_loader_factory.cc
[modify] https://crrev.com/f510d0c432ea0330c394e4a1349d6e7ee31c116e/content/browser/appcache/appcache_url_loader_factory.h
[modify] https://crrev.com/f510d0c432ea0330c394e4a1349d6e7ee31c116e/content/browser/loader/mojo_async_resource_handler.cc
[modify] https://crrev.com/f510d0c432ea0330c394e4a1349d6e7ee31c116e/content/browser/loader/mojo_async_resource_handler.h
[modify] https://crrev.com/f510d0c432ea0330c394e4a1349d6e7ee31c116e/content/browser/loader/mojo_async_resource_handler_unittest.cc
[modify] https://crrev.com/f510d0c432ea0330c394e4a1349d6e7ee31c116e/content/browser/loader/navigation_url_loader_network_service.cc
[modify] https://crrev.com/f510d0c432ea0330c394e4a1349d6e7ee31c116e/content/browser/loader/navigation_url_loader_network_service.h
[modify] https://crrev.com/f510d0c432ea0330c394e4a1349d6e7ee31c116e/content/browser/loader/resource_dispatcher_host_impl.cc
[modify] https://crrev.com/f510d0c432ea0330c394e4a1349d6e7ee31c116e/content/browser/loader/resource_dispatcher_host_impl.h
[modify] https://crrev.com/f510d0c432ea0330c394e4a1349d6e7ee31c116e/content/browser/loader/resource_message_filter.cc
[modify] https://crrev.com/f510d0c432ea0330c394e4a1349d6e7ee31c116e/content/browser/loader/resource_message_filter.h
[modify] https://crrev.com/f510d0c432ea0330c394e4a1349d6e7ee31c116e/content/browser/loader/resource_request_info_impl.cc
[modify] https://crrev.com/f510d0c432ea0330c394e4a1349d6e7ee31c116e/content/browser/loader/resource_request_info_impl.h
[modify] https://crrev.com/f510d0c432ea0330c394e4a1349d6e7ee31c116e/content/browser/loader/url_loader_factory_impl.cc
[modify] https://crrev.com/f510d0c432ea0330c394e4a1349d6e7ee31c116e/content/browser/loader/url_loader_factory_impl.h
[modify] https://crrev.com/f510d0c432ea0330c394e4a1349d6e7ee31c116e/content/browser/loader/url_loader_factory_impl_unittest.cc
[modify] https://crrev.com/f510d0c432ea0330c394e4a1349d6e7ee31c116e/content/browser/service_worker/service_worker_fetch_dispatcher.cc
[modify] https://crrev.com/f510d0c432ea0330c394e4a1349d6e7ee31c116e/content/browser/site_per_process_browsertest.cc
[modify] https://crrev.com/f510d0c432ea0330c394e4a1349d6e7ee31c116e/content/browser/webui/web_ui_url_loader_factory.cc
[modify] https://crrev.com/f510d0c432ea0330c394e4a1349d6e7ee31c116e/content/child/resource_dispatcher.cc
[modify] https://crrev.com/f510d0c432ea0330c394e4a1349d6e7ee31c116e/content/child/resource_dispatcher.h
[modify] https://crrev.com/f510d0c432ea0330c394e4a1349d6e7ee31c116e/content/child/url_loader_client_impl_unittest.cc
[modify] https://crrev.com/f510d0c432ea0330c394e4a1349d6e7ee31c116e/content/common/url_loader_factory.mojom
[modify] https://crrev.com/f510d0c432ea0330c394e4a1349d6e7ee31c116e/content/network/network_service_url_loader_factory_impl.cc
[modify] https://crrev.com/f510d0c432ea0330c394e4a1349d6e7ee31c116e/content/network/network_service_url_loader_factory_impl.h
[modify] https://crrev.com/f510d0c432ea0330c394e4a1349d6e7ee31c116e/content/network/url_loader_impl.cc
[modify] https://crrev.com/f510d0c432ea0330c394e4a1349d6e7ee31c116e/content/network/url_loader_impl.h
[modify] https://crrev.com/f510d0c432ea0330c394e4a1349d6e7ee31c116e/content/network/url_loader_unittest.cc

Comment 3 by jam@chromium.org, May 30 2017

Cc: rdsmith@chromium.org yzshen@chromium.org mmenke@chromium.org
Randy wrote the following comment on the above issue: "I haven't looked at this change in any detail, just watched the comments go by, but I wanted to call out that if the associated keyword was removed purely based on test results, I suspect that was a mistake.  If there is a bug, whether or not the lack of "associated" causes problems is going to be dependent on racy behavior between old IPC and mojo.  So I would have recommended not removing the "associated" keyword without a careful evaluation of what races might occur and a good argument that they are no longer present.  FWIW."

We chatted about this briefly in today's network service sync up. Randy and Matt gave the example of setting a cookie in JS and then making a request. While these interfaces (URLLoaderFactory and RenderFrameMessageFilter) are not associated, I think it's a mojo implementation detail that they won't race. To improve this, we might either want to combine the two interfaces. They'll both be implemented in the network service in the future as well.

Randy suggested that it would be good to audit IPCs and look for ones related to networking/cookies/cache. Yutaka: can you please do this? Since this landed right after branch point, we have 6 weeks to ensure there are no races before next branch :)

Comment 4 by yzshen@chromium.org, May 30 2017

John and I chatted about this issue and here is the summary: (Please correct me if I am wrong, John)

Looking at the code, it seems fine to change URLLoaderFactory.CreateLoaderAndStart() to return a non-associated interface. It is the order between "starting URL request" and "setting cookies" that matters. Starting URL request is a message of URLLoaderFactory, which is still associated with the RenderFrameMessageFilter interface in the MojoLoading code path. (Both are channel-associated interfaces.)

In the network service case, we don't use IPC channel between the network process and renderer process. In that case, we could consider using another Factory interface to vend associated URLLoaderFactory and CookieStore interfaces. For example,

interface NetworkFactory {
  GetURLLoaderFactory(associated URLLoaderFactory& factory);
  GetCookieStore(associated CookieStore& cookie_store);
};


Comment 5 by kinuko@chromium.org, May 31 2017

Yes the summary sounds right to me too. Once a request starts any reordering or throttling could happen already in the current code, while if request to starting a loader hits the service earlier than other requests like setting a cookie things could get wrong.
When the network service is fully up, I expect the cookie interface to be part of the larger network service interface (associated at a minimum, but it may just be wrapped in).  And similarly for other IPCs, which will have been converted to mojo.

But my basic point still stands: A lack of failing tests doesn't really say much about whether or not there aren't any races, so changes that specifically may open up racy behavior should IMO be validated from a theoretical perspective as well.  I offered cookies as a example, but it was to elucidate the problem, not because I thought it was the only possible problem.  I'd recommend *some* amount of due diligence to validate the lack of races beyond trusting tests in removing the associated keyword.

Having said all that, I was indeed confusing the interface for a URLLoader with the interface to start a request/create a URLLoader.  The URLLoader interface itself seems like there are noticeably fewer race possibilities, just because the process of fetching a URL is so naturally variable because of server response.  Glancing at the URLLoader interface, it seems very simple, but I will call out that the "FollowRedirect()" method may allow for cookie races if you get a notification of a redirect, set a cookie, and then follow the redirect.  It's a corner case, but it's the type of thing I'd recommend looking for in this kind of situation.  The corner cases are the bug that will be the absolute hardest to find when they do come up.

URLLoaderFactory actually have three methods. ChangePriority, FollowRedirect, and Cancel. I've ran tests a few times, and I saw tests failures only with Cancel (at that time the failure was consistent).

Redirect handling is not notified to scripts, at least synchronously.
Project Member

Comment 8 by bugdroid1@chromium.org, Jun 2 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/c9ad960e2f22c27c7a984ec5bdb14c612737c5ad

commit c9ad960e2f22c27c7a984ec5bdb14c612737c5ad
Author: Yutaka Hirano <yhirano@chromium.org>
Date: Fri Jun 02 10:41:23 2017

Revert "Use Independent URLLoader"

This reverts commit f510d0c432ea0330c394e4a1349d6e7ee31c116e.

Reason for revert: Performance regression: See  crbug.com/728467 

Original change's description:
> Use Independent URLLoader
> 
> This CL removes "associated" for URLLoader in
> URLLoaderFactory::CreateLoaderAndStart.
> 
> The associated keyword was needed in order to keep the ordering guarantee
> which the current ChromeIPC provides, but it is harmful from the network
> servicification POV.
> 
> As all tests pass without the keyword (when kLoadingWithMojo is enabled),
> let's stop using the associated interface.
> 
> Bug:  706286 ,  724323 
> Change-Id: I771ee5ed35ca882cfa77c5e7ce2e862261c3f62d
> Reviewed-on: https://chromium-review.googlesource.com/513643
> Commit-Queue: Yutaka Hirano <yhirano@chromium.org>
> Reviewed-by: Daniel Cheng <dcheng@chromium.org>
> Reviewed-by: Kinuko Yasuda <kinuko@chromium.org>
> Reviewed-by: John Abd-El-Malek <jam@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#475356}

TBR=dcheng@chromium.org,kinuko@chromium.org,jam@chromium.org,yhirano@chromium.org
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug:  706286 ,  724323 ,  728467 

Change-Id: If0a91fe994b2b784020b69499a5206a41596075c
Reviewed-on: https://chromium-review.googlesource.com/522184
Reviewed-by: Yutaka Hirano <yhirano@chromium.org>
Commit-Queue: Yutaka Hirano <yhirano@chromium.org>
Cr-Commit-Position: refs/heads/master@{#476614}
[modify] https://crrev.com/c9ad960e2f22c27c7a984ec5bdb14c612737c5ad/content/browser/appcache/appcache_url_loader_factory.cc
[modify] https://crrev.com/c9ad960e2f22c27c7a984ec5bdb14c612737c5ad/content/browser/appcache/appcache_url_loader_factory.h
[modify] https://crrev.com/c9ad960e2f22c27c7a984ec5bdb14c612737c5ad/content/browser/loader/mojo_async_resource_handler.cc
[modify] https://crrev.com/c9ad960e2f22c27c7a984ec5bdb14c612737c5ad/content/browser/loader/mojo_async_resource_handler.h
[modify] https://crrev.com/c9ad960e2f22c27c7a984ec5bdb14c612737c5ad/content/browser/loader/mojo_async_resource_handler_unittest.cc
[modify] https://crrev.com/c9ad960e2f22c27c7a984ec5bdb14c612737c5ad/content/browser/loader/navigation_url_loader_network_service.cc
[modify] https://crrev.com/c9ad960e2f22c27c7a984ec5bdb14c612737c5ad/content/browser/loader/navigation_url_loader_network_service.h
[modify] https://crrev.com/c9ad960e2f22c27c7a984ec5bdb14c612737c5ad/content/browser/loader/resource_dispatcher_host_impl.cc
[modify] https://crrev.com/c9ad960e2f22c27c7a984ec5bdb14c612737c5ad/content/browser/loader/resource_dispatcher_host_impl.h
[modify] https://crrev.com/c9ad960e2f22c27c7a984ec5bdb14c612737c5ad/content/browser/loader/resource_message_filter.cc
[modify] https://crrev.com/c9ad960e2f22c27c7a984ec5bdb14c612737c5ad/content/browser/loader/resource_message_filter.h
[modify] https://crrev.com/c9ad960e2f22c27c7a984ec5bdb14c612737c5ad/content/browser/loader/resource_request_info_impl.cc
[modify] https://crrev.com/c9ad960e2f22c27c7a984ec5bdb14c612737c5ad/content/browser/loader/resource_request_info_impl.h
[modify] https://crrev.com/c9ad960e2f22c27c7a984ec5bdb14c612737c5ad/content/browser/loader/url_loader_factory_impl.cc
[modify] https://crrev.com/c9ad960e2f22c27c7a984ec5bdb14c612737c5ad/content/browser/loader/url_loader_factory_impl.h
[modify] https://crrev.com/c9ad960e2f22c27c7a984ec5bdb14c612737c5ad/content/browser/loader/url_loader_factory_impl_unittest.cc
[modify] https://crrev.com/c9ad960e2f22c27c7a984ec5bdb14c612737c5ad/content/browser/service_worker/service_worker_fetch_dispatcher.cc
[modify] https://crrev.com/c9ad960e2f22c27c7a984ec5bdb14c612737c5ad/content/browser/site_per_process_browsertest.cc
[modify] https://crrev.com/c9ad960e2f22c27c7a984ec5bdb14c612737c5ad/content/browser/webui/web_ui_url_loader_factory.cc
[modify] https://crrev.com/c9ad960e2f22c27c7a984ec5bdb14c612737c5ad/content/child/throttling_url_loader.h
[modify] https://crrev.com/c9ad960e2f22c27c7a984ec5bdb14c612737c5ad/content/child/throttling_url_loader_unittest.cc
[modify] https://crrev.com/c9ad960e2f22c27c7a984ec5bdb14c612737c5ad/content/child/url_loader_client_impl_unittest.cc
[modify] https://crrev.com/c9ad960e2f22c27c7a984ec5bdb14c612737c5ad/content/common/url_loader_factory.mojom
[modify] https://crrev.com/c9ad960e2f22c27c7a984ec5bdb14c612737c5ad/content/network/network_service_url_loader_factory_impl.cc
[modify] https://crrev.com/c9ad960e2f22c27c7a984ec5bdb14c612737c5ad/content/network/network_service_url_loader_factory_impl.h
[modify] https://crrev.com/c9ad960e2f22c27c7a984ec5bdb14c612737c5ad/content/network/url_loader_impl.cc
[modify] https://crrev.com/c9ad960e2f22c27c7a984ec5bdb14c612737c5ad/content/network/url_loader_impl.h
[modify] https://crrev.com/c9ad960e2f22c27c7a984ec5bdb14c612737c5ad/content/network/url_loader_unittest.cc

Project Member

Comment 9 by bugdroid1@chromium.org, Jul 25 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/4f87e237dabe3d4025c52a43885a469a7e872cd2

commit 4f87e237dabe3d4025c52a43885a469a7e872cd2
Author: Yutaka Hirano <yhirano@chromium.org>
Date: Tue Jul 25 08:13:52 2017

Use Independent URLLoader

This is a reland of
https://chromium.googlesource.com/chromium/src/+/f510d0c432ea0330c394e4a1349d6e7ee31c116e.

This CL removes "associated" for URLLoader in
URLLoaderFactory::CreateLoaderAndStart.

The associated keyword was needed in order to keep the ordering guarantee
which the current ChromeIPC provides, but it is harmful from the network
servicification POV.

As all tests pass without the keyword (when kLoadingWithMojo is enabled),
let's stop using the associated interface.

Bug:  706286 ,  724323 
Change-Id: I2662ea948850e5b10b4278740beb1286fbeb6003
Reviewed-on: https://chromium-review.googlesource.com/578700
Reviewed-by: Eric Roman <eroman@chromium.org>
Reviewed-by: Daniel Cheng <dcheng@chromium.org>
Reviewed-by: Kinuko Yasuda <kinuko@chromium.org>
Commit-Queue: Yutaka Hirano <yhirano@chromium.org>
Cr-Commit-Position: refs/heads/master@{#489254}
[modify] https://crrev.com/4f87e237dabe3d4025c52a43885a469a7e872cd2/chrome/browser/net/network_context_configuration_browsertest.cc
[modify] https://crrev.com/4f87e237dabe3d4025c52a43885a469a7e872cd2/content/browser/appcache/appcache_subresource_url_factory.cc
[modify] https://crrev.com/4f87e237dabe3d4025c52a43885a469a7e872cd2/content/browser/appcache/appcache_subresource_url_factory.h
[modify] https://crrev.com/4f87e237dabe3d4025c52a43885a469a7e872cd2/content/browser/appcache/appcache_url_loader_job.cc
[modify] https://crrev.com/4f87e237dabe3d4025c52a43885a469a7e872cd2/content/browser/appcache/appcache_url_loader_job.h
[modify] https://crrev.com/4f87e237dabe3d4025c52a43885a469a7e872cd2/content/browser/blob_storage/blob_url_loader_factory.cc
[modify] https://crrev.com/4f87e237dabe3d4025c52a43885a469a7e872cd2/content/browser/blob_storage/blob_url_loader_factory.h
[modify] https://crrev.com/4f87e237dabe3d4025c52a43885a469a7e872cd2/content/browser/blob_storage/blob_url_unittest.cc
[modify] https://crrev.com/4f87e237dabe3d4025c52a43885a469a7e872cd2/content/browser/loader/mojo_async_resource_handler.cc
[modify] https://crrev.com/4f87e237dabe3d4025c52a43885a469a7e872cd2/content/browser/loader/mojo_async_resource_handler.h
[modify] https://crrev.com/4f87e237dabe3d4025c52a43885a469a7e872cd2/content/browser/loader/mojo_async_resource_handler_unittest.cc
[modify] https://crrev.com/4f87e237dabe3d4025c52a43885a469a7e872cd2/content/browser/loader/resource_dispatcher_host_impl.cc
[modify] https://crrev.com/4f87e237dabe3d4025c52a43885a469a7e872cd2/content/browser/loader/resource_dispatcher_host_impl.h
[modify] https://crrev.com/4f87e237dabe3d4025c52a43885a469a7e872cd2/content/browser/loader/resource_message_filter.cc
[modify] https://crrev.com/4f87e237dabe3d4025c52a43885a469a7e872cd2/content/browser/loader/resource_message_filter.h
[modify] https://crrev.com/4f87e237dabe3d4025c52a43885a469a7e872cd2/content/browser/loader/resource_request_info_impl.cc
[modify] https://crrev.com/4f87e237dabe3d4025c52a43885a469a7e872cd2/content/browser/loader/resource_request_info_impl.h
[modify] https://crrev.com/4f87e237dabe3d4025c52a43885a469a7e872cd2/content/browser/loader/url_loader_factory_impl.cc
[modify] https://crrev.com/4f87e237dabe3d4025c52a43885a469a7e872cd2/content/browser/loader/url_loader_factory_impl.h
[modify] https://crrev.com/4f87e237dabe3d4025c52a43885a469a7e872cd2/content/browser/loader/url_loader_factory_impl_unittest.cc
[modify] https://crrev.com/4f87e237dabe3d4025c52a43885a469a7e872cd2/content/browser/service_worker/service_worker_fetch_dispatcher.cc
[modify] https://crrev.com/4f87e237dabe3d4025c52a43885a469a7e872cd2/content/browser/service_worker/service_worker_provider_host.cc
[modify] https://crrev.com/4f87e237dabe3d4025c52a43885a469a7e872cd2/content/browser/service_worker/service_worker_url_loader_job.cc
[modify] https://crrev.com/4f87e237dabe3d4025c52a43885a469a7e872cd2/content/browser/site_per_process_browsertest.cc
[modify] https://crrev.com/4f87e237dabe3d4025c52a43885a469a7e872cd2/content/browser/webui/web_ui_url_loader_factory.cc
[modify] https://crrev.com/4f87e237dabe3d4025c52a43885a469a7e872cd2/content/child/url_loader_client_impl_unittest.cc
[modify] https://crrev.com/4f87e237dabe3d4025c52a43885a469a7e872cd2/content/common/throttling_url_loader.cc
[modify] https://crrev.com/4f87e237dabe3d4025c52a43885a469a7e872cd2/content/common/throttling_url_loader.h
[modify] https://crrev.com/4f87e237dabe3d4025c52a43885a469a7e872cd2/content/common/throttling_url_loader_unittest.cc
[modify] https://crrev.com/4f87e237dabe3d4025c52a43885a469a7e872cd2/content/network/network_service_url_loader_factory_impl.cc
[modify] https://crrev.com/4f87e237dabe3d4025c52a43885a469a7e872cd2/content/network/network_service_url_loader_factory_impl.h
[modify] https://crrev.com/4f87e237dabe3d4025c52a43885a469a7e872cd2/content/network/url_loader_impl.cc
[modify] https://crrev.com/4f87e237dabe3d4025c52a43885a469a7e872cd2/content/network/url_loader_impl.h
[modify] https://crrev.com/4f87e237dabe3d4025c52a43885a469a7e872cd2/content/network/url_loader_unittest.cc
[modify] https://crrev.com/4f87e237dabe3d4025c52a43885a469a7e872cd2/content/public/common/url_loader_factory.mojom
[modify] https://crrev.com/4f87e237dabe3d4025c52a43885a469a7e872cd2/content/renderer/renderer_blink_platform_impl.cc

Labels: -Pri-3 Pri-1
Owner: yhirano@chromium.org
Changing owner, raising priority to reflect the situation better

(Status: watching some spotty regressions, for now we believe they're not that significant that may require reverting)
Have we done anything to prevent "cookie races"?
Oh, right, turned out setting cookies was blocking, IIRC
And, no, it's not.

Sorry, misread the commit message as removing the associated pointer from the loader factory, not the loader.
Components: Internals>Network>Service
Components: -Internals>Network>Service Internals>Services>Network
Apologies, applied the wrong component in bulk.
Cc: -rdsmith@chromium.org

Comment 17 by dxie@chromium.org, May 22 2018

Components: -Internals>Services>Network
Status: Fixed (was: Assigned)

Sign in to add a comment