blob: urls triggering net::URLRequestContext::AssertNoURLRequests crashes |
|||||||||||||||
Issue descriptionI looked through a bunch of recent minidumps for net::URLRequestContext::AssertNoURLRequests. The majority were for blob:uuid/... urls, one for blob://see_user_data/, and a few there were unrelated. Crash URL (note, this is all leaked URLRequests, so you need to check the minidump for the url_buf value in the AssertNoURLRequests stack frame to be sure which cause applies to a given crash report): https://crash.corp.google.com/browse?q=product.name%20CONTAINS%20%27Chrome%27%20AND%20custom_data.ChromeCrashProto.ptype%3D%27browser%27%20AND%20%20custom_data.ChromeCrashProto.magic_signature_1.name%3D%27net%3A%3AURLRequestContext%3A%3AAssertNoURLRequests%27&ignore_case=false&enable_rewrite=true&omit_field_name=&omit_field_value=&omit_field_opt=%3D I saw blob: urls in win, mac, and linux minidumps at least. Assigning based on git blame, please reassign if there's a better person to handle this. Issue 701851 also looks relevant, insofar as stopping using URLFetcher should remove the issue.
,
Jun 4 2017
Users experienced this crash on the following builds: Mac Dev 60.0.3112.10 - 0.29 CPM, 1 reports, 1 clients (signature net::URLRequestContext::AssertNoURLRequests) If this update was incorrect, please add "Fracas-Wrong" label to prevent future updates. - Go/Fracas
,
Jun 13 2017
Users experienced this crash on the following builds: Mac Canary 61.0.3128.0 - 0.25 CPM, 1 reports, 1 clients (signature net::URLRequestContext::AssertNoURLRequests) Linux Beta 60.0.3112.24 - 0.64 CPM, 4 reports, 2 clients (signature net::URLRequestContext::AssertNoURLRequests) If this update was incorrect, please add "Fracas-Wrong" label to prevent future updates. - Go/Fracas
,
Oct 19 2017
Users experienced this crash on the following builds: Ios Dev 63.0.3231.0 - 10.08 CPM, 1 reports, 1 clients (signature net::URLRequestContext::AssertNoURLRequests) If this update was incorrect, please add "Fracas-Wrong" label to prevent future updates. - Go/Fracas
,
Oct 24 2017
Users experienced this crash on the following builds: Ios Beta 62.0.3202.52 - 7.21 CPM, 1 reports, 1 clients (signature net::URLRequestContext::AssertNoURLRequests) If this update was incorrect, please add "Fracas-Wrong" label to prevent future updates. - Go/Fracas
,
Jan 8 2018
We're still seeing these, though now the URLs the requests are for tend to look like "blob:uuid/<GUID>". AssertNoURLRequests is now the #6 crasher on Windows Dev, but I can't confirm if it's also because of these blob URLs. On stable, 3 of 4 randomly sampled crashers were because of outstanding blob requests, but none of the 400 Windows dev crashers in AssertNoURLRequests have stack traces, so the recent jump may well be due to something else.
,
Jan 8 2018
Can you explain what exactly the bug here is? What is this assert asserting, and what kind of situations could be triggering it?
,
Jan 8 2018
General changes in this area, in M64 the javascript FileReader API no longer uses blob URLs, so that would explain a reduction of URL requests of the form blob:<origin>/<GUID>. It doesn't sound like that shape of blob URL ever triggered this in the first place, though? I'm also not sure I understand what you're trying to say about hat changed for these URLs? In the original report it already seems to say most of the URLs are "blob:uuid/<GUID>" style URLs?
,
Jan 8 2018
The issue is that URLRequests are still alive when the URLRequestContext is destroyed (And presumably when ProfileIOData is destroyed as well). The network stack has never supported URLRequests outliving the URLRequestContext. When this happens, the network stack forces a crash, and records the URL of a URLRequest that lived too long. Sample crash: https://crash.corp.google.com/browse?q=product.name%3D%27Chrome%27%20AND%20custom_data.ChromeCrashProto.ptype%3D%27browser%27%20AND%20custom_data.ChromeCrashProto.magic_signature_1.name%3D%27net%3A%3AURLRequestContext%3A%3AAssertNoURLRequests%27%20AND%20custom_data.ChromeCrashProto.magic_signature_1.component%3D%27src%2Fnet%2Furl_request%27%20AND%20NOT%20upload_info.discard_minidump&sql_dialect=googlesql&ignore_case=false&enable_rewrite=true&omit_field_name=&omit_field_value=&omit_field_opt=%3D&stbtiq=&reportid=9bc897ed5c4c5fa1&index=3#0 We've run into this issue before in objects that directly consume URLRequests (i.e., not requests that go through the renderer's standard request path, but instead things that live in the browser process, and interact directly with their own URLRequests). In Chrome 63, most of the requests where this happens are for blob URLs. This was also the case when this bug was reported, back in Chrome 60 (?). I believe there's some path of make a URLRequest direct through blob infrastructure in Chrome, and I suspect that some consumer of this path is the culprit for these crashes. This sort of crash is a top crasher in Chrome 65, however, I have not been able to confirm if requests for blob URLs are still the main culprit in Chrome 65, since no Windows Chrome 65 Canary crashes here have memory dumps. The recent jump may well be because of Mojo loading / network service / PlzNavigate changes, which all significantly refactor consumers of URLRequests.
,
Jan 8 2018
I intend to dig deeper into the recent regression, to figure out what's going on there, but to the extent of my knowledge, regardless of whether it's related to the recent regression or not, some consumer of blob URLRequests is outliving ProfileIOData without cancelling its URLRequests, and that consumer (Or those consumers) should be identified and fixed.
,
Jan 8 2018
Ah, I see. The two main paths I can think of where Blob's are read from te browser process, but through blob URLs are indeed the extensions::BlobReader code-path you mentioned earlier, but also IndexedDB, whenever a blob needs to be written to disk (i.e. roughly the code at https://cs.chromium.org/chromium/src/content/browser/indexed_db/indexed_db_backing_store.cc?l=1593). I don't think either of that changed much in M65 (although the IDB case might happen more often since M64 I think). And of course it's somewhere on my todo list to get rid of all those cases...
,
Jan 8 2018
And with the network service, these things may just go away, since we have to completely rewrite everything anyways (And Mojo requests the network service handle out-of-order teardown itself), so, despite what I wrote above, it may not be worth looking into, unless I discover blob URLs are the cause of the recent regression.
,
Jan 8 2018
(mostly as note-to-self, blob://see-user-data/ URLs are created by BlobProtocolHandler::CreateBlobRequest, and seem to be caused by CacheStorage, ServiceWorker and FileSystem API related usage. But yeah, all of this is getting rewritten as part of content modularisation/servicication anyway).
,
Jan 9 2018
assigning to mek@ to get off the triage plate since; please reassign/update status as appropriate
,
Jan 18 2018
Users experienced this crash on the following builds: Win Dev 65.0.3322.3 - 0.20 CPM, 17 reports, 17 clients (signature net::URLRequestContext::AssertNoURLRequests) If this update was incorrect, please add "Fracas-Wrong" label to prevent future updates. - Go/Fracas
,
Jan 23 2018
Users experienced this crash on the following builds: Win Dev 65.0.3322.3 - 0.11 CPM, 41 reports, 37 clients (signature net::URLRequestContext::AssertNoURLRequests) Ios Beta 64.0.3282.97 - 23.24 CPM, 1 reports, 1 clients (signature net::URLRequestContext::AssertNoURLRequests) If this update was incorrect, please add "Fracas-Wrong" label to prevent future updates. - Go/Fracas
,
Feb 2 2018
Users experienced this crash on the following builds: Win Canary 66.0.3335.0 - 0.25 CPM, 4 reports, 4 clients (signature net::URLRequestContext::AssertNoURLRequests) Ios Beta 64.0.3282.97 - 4.43 CPM, 1 reports, 1 clients (signature net::URLRequestContext::AssertNoURLRequests) If this update was incorrect, please add "Fracas-Wrong" label to prevent future updates. - Go/Fracas
,
Feb 24 2018
This crash has high impact on Chrome's stability. Signature: net::URLRequestContext::AssertNoURLRequests. Channel: canary. Platform: win. Labeling issue 728854 with ReleaseBlock-Dev. If this update was incorrect, please add "Fracas-Wrong" label to prevent future updates. - Go/Fracas
,
Feb 26 2018
mmenke@: Should we file another bug for the recent spike of this signature on Canary 66.0.3352.0 (Win and Mac)? I'm suspecting the previous investigation that pointed towards blobs: urls may have nothing to do with this new spike.
,
Feb 26 2018
This issue is marked as a release blocker with no milestone associated. Please add an appropriate milestone. All release blocking issues should have milestones associated to it, so that the issue can tracked and the fixes can be pushed promptly. Thanks for your time! To disable nags, add the Disable-Nags label. For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Feb 26 2018
That would be issue 800391, I assume.
,
Feb 26 2018
Actually, it's not - looks like another new spike. :( I'll file a new bug.
,
Mar 2 2018
This issue is marked as a release blocker with no milestone associated. Please add an appropriate milestone. All release blocking issues should have milestones associated to it, so that the issue can tracked and the fixes can be pushed promptly. Thanks for your time! To disable nags, add the Disable-Nags label. For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Mar 12 2018
Unmarking as releaseblock-dev, since if I understand things correctly the related spikes are actually for different (i.e. non blob:) urls that are already tracked in different bugs.
,
Mar 15 2018
Users experienced this crash on the following builds: Ios Beta 66.0.3359.12 - 8.50 CPM, 1 reports, 1 clients (signature net::URLRequestContext::AssertNoURLRequests) If this update was incorrect, please add "Fracas-Wrong" label to prevent future updates. - Go/Fracas
,
Mar 18 2018
Users experienced this crash on the following builds: Mac Canary 67.0.3373.0 - 0.41 CPM, 1 reports, 1 clients (signature net::URLRequestContext::AssertNoURLRequests) Ios Beta 66.0.3359.12 - 12.80 CPM, 2 reports, 2 clients (signature net::URLRequestContext::AssertNoURLRequests) If this update was incorrect, please add "Fracas-Wrong" label to prevent future updates. - Go/Fracas
,
Apr 3 2018
could AssertNoURLRequests() be changed not to crash and just log at least with the release build? in my case, the AssertNoURLRequests() crash happens during the app shutdown procedure according to the crash backtrace (btw, we have no on-demand repro steps). CHECK() ==> immediate crash DCHECK() ==> debug check? and no crash with the release build? 153 // We're leaking URLRequests :( Dump the URL of the first one and record how 154 // many we leaked so we have an idea of how bad it is. ... 162 CHECK(false) << "Leaked " << num_requests << " URLRequest(s). First URL: " 163 << request->url().spec().c_str() << ".";
,
Apr 3 2018
No, it can't - when you destroy the URLRequest that outlived the URLRequestContext, it will try to call into the destroyed URLRequestContext, which will crash. AssertNoURLRequests prevents the UAF crash with a more predictable and understandable crash stack.
,
May 31 2018
This is still second most frequent crash signature of all net:: crashes across all platforms, see https://crash.corp.google.com/browse?q=product_name+LIKE+%27Chrome%25%27+AND+expanded_custom_data.ChromeCrashProto.ptype%3D%27browser%27+AND+expanded_custom_data.ChromeCrashProto.magic_signature_1.name+LIKE+%27net%3A%3A%25%27#-property-selector,magicsignature:50,-magicsignature2:50,-stablesignature:50,-clientid,-magicsignaturesorted:50 See related issue 617560. |
|||||||||||||||
►
Sign in to add a comment |
|||||||||||||||
Comment 1 by mattm@chromium.org
, Jun 1 2017