Recent browser updates causing ERR_ABORTED AJAX requests
Reported by
ryan.heath@software2.eu,
Sep 7 2017
|
|||||||||||||||||||||||||||
Issue descriptionUserAgent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36 Steps to reproduce the problem: 1. Initiate a custom protocol launch using JS and window.location = 'custom://blah' 2. Intitate two different threads of AJAX calls, one polling a URL every x seconds, the other following a request workflow to load data into the app 3. Both sets of AJAX requests get ERR_ABORTED 4. Repeating this without the AJAX server polling (second channel) eliminates the issue 5. Launching the protocol with a hidden iframe (location.href='custom://blah') eliminates the issue 6. Closing the browser and deleting the Local State file eliminates the issue for the NEXT LAUNCH ONLY, but then the issue returns unless you repeat. This issue was seen for a while after the 60.0.3112.101 update but was resolved permenantly by killing all chrome processes post-update. It has since returned and is reproducable following 61.0.3163.79 update What is the expected behavior? This method we are using to trigger these protocol requests has been working without issue for years. The protocol launches and the AJAX requests continue, this is only an issue we are seeing after Chrome updates. What went wrong? Howdy, Our web app is experiencing some very strange behaviour following some recent Chrome Updates. We first saw the issue following the 60.0.3112.101 update and posted the update in the attached PDF which basically summised that the issue was related to the update process itself and was resolved by a hard reset of all chrome processes but as we could no longer replicate it, we were unable to investigate further. We have since started seeing the issue again following the 61.0.3163.79 update and at present we are still able to replicate it. Now we have dug a little deeper it seems we can resolve the issue by closing Chrome, deleting the local state file (C:\Users\<username>\AppData\Local\Google\Chrome\User Data\Local State) and then re-opening chrome and browsing back to our app. This resolves the issue until the next time you restart chrome, at which point it returns, unless you repeat and delete the local state file again. What are we doing? When a user logs into our app, we automatically initiate a custom protocol URI request to send a message to a client app installed locally on the device, this then reports device information back to the server and the app periodically polls the server for the device information. The protocol request is triggered simply using a window.location='ourprotocol://blah' (but obviously wrapped in a load of success/failure handling JS). What problem our we seeing? The problem we are seeing post-update seems to be either: 1) Some form of conflict between the AJAX requests that poll for other information and those that poll for device info status (which happen at the same time) - we wondered whether the browser was seeing the protocol request as a "navigate away" and cancelling all subsequent requests OR 2) Some form of security issue with the requests being cancelled because they are not user-actioned. However, neither of these really make sense when you consider that refreshing the local state file resolves the problem, which suggests this isn't either an app issue or a result of an actual change implemented in the newer version. Symptoms: Basically there are two requests happening at around the same time (application/list and get-device-info), both are showing as cancelled (in the same manner) as shown in the attached net-stat-log.txt. This results in failure for our app to load. One thing to note is that the protocol request IS permitted and DOES complete, meaning the client is triggered and device info sent, so if you refresh the page, our app has no need to do the second set of AJAX calls (only the application/list requests) which execute without issue. Investigations so far: It is intersting to note that we if switch the protocol launch method to launch in a hidden iframe by setting its location.href then we don't see the issue! I include this as a possible clue but at this stage, changing the product is not a viable solution as we have 400+ customer installations world-wide that we can't update without change control approval and remote access. (Please no comments on this, we are actively moving to a better deployment methodology so we can be more flexible!) We have grabbed a copy of the Local State file from a machine experiencing the issue from when it was working and when it stopped, both of which are attached. We're not really sure what we should be looking for in here but it might offer someone an idea. We are continuing our investigations but wanted to put this out to the wider community in the hope of getting some assistance as it makes no sense currently why a local state file could have such a heavy impact on browser behaviour Did this work before? Yes 60.0.3112.101 (after a restart post-update - see above) Chrome version: 61.0.3163.79 Channel: stable OS Version: 10.0 Flash Version: Also referenced in https://groups.google.com/a/chromium.org/forum/#!topic/chromium-dev/D14A4KCJuN4
,
Sep 7 2017
cc'ing tyoshino@ and yhirano@ for further triage of the issue.
,
Sep 8 2017
Adding Triage help label, so that dev team can take a look into it.
,
Sep 8 2017
Based on the timing I suspect the involvement of PlzNavigate.
,
Sep 8 2017
,
Sep 8 2017
,
Sep 8 2017
Does this reproduce if you disable this experiment: chrome://flags/#browser-side-navigation ?
,
Sep 8 2017
I can confirm, when the flag is enabled the issue occurs, when it is disabled it doesn't. Good shout! What does this mean? Cheers
,
Sep 8 2017
Just to re-iterate @jam, when the issue occurs, it's not an all-out fail - we can temporarily resolve the issue (most of the time) by closing Chrome, deleting the local state file (C:\Users\<username>\AppData\Local\Google\Chrome\User Data\Local State) and then re-opening chrome and browsing back to our app. This resolves the issue until the next time you restart chrome, at which point it returns, unless you repeat and delete the local state file again.
,
Sep 8 2017
Another request. Would it be possible to attach a minimal repro html file? It will help us ensure we are investigating the issue you are seeing instead of us guessing.
,
Sep 8 2017
Not easily, or quickly. We can try. In the meantime: 1) Go to demo.software2.com 2) Login with chrome / Password-123 3) Select I've already used AppsAnywhere on this device (there shouldn't be any need to actually install our client) 4) Click Logout 5) Log in again with same credentials 6) This time, it won't ask that initial question and you should see the issue as the ajax requests try to complete
,
Sep 8 2017
Confirmed possible to replicate without needing our client using the steps above, as long as you deny the external protocol request
,
Sep 8 2017
Camille: please triage
,
Sep 8 2017
@jam Was that sufficient to demonstrate or would you still like a min version in possible?
,
Sep 8 2017
I just went by comment 8. A minimal repro (i.e. html file) would help ensure it doesnt regress again.
,
Sep 11 2017
I've tried reproducing on M61 on Linux without success. When I go to the website demo.software2.com and login, I cannot select "I've already used AppsAnywhere on this device". Logging out, and logging in again doesn't seem to cause issue with the Ajax requests.
,
Sep 11 2017
Sorry, our demo site does not launch protocols on Linux as we do not have a Linux client. We have created a min file as requested to demonstrate the issue, tested on Windows and Mac OS, the AJAX calls succeed with browser-side-navigation disabled but fail with it enabled. We quickly checked this min file on Linux Mint 18 with Chrome 61 and interestingly could not replicate the issue as seen on Windows and Mac OS
,
Sep 11 2017
@jam: can you have a look at this? I don't have easy access to a Windows machine and it doesn't reproduce on Linux.
,
Sep 11 2017
Thanks for the minimal repro, that's very helpful! When the frame commits, it sets window.location to an external protocol and makes XHR requests. Since the window.location creates a navigation which fails, RenderFrameHostImpl::FailedNavigation calls FrameTreeNode::ResetNavigationRequest. The latter, seeing that it's a renderer initiated, sends a FrameMsg_Stop IPC which cancels all pending loads (e.g. XHRs). Without plznavigate, a provisional load failure doesn't stop loads from the previous commit. @clamy: this behavior was added in https://crrev.com/2567242af04760d30f47a96e08bb0314a4c0b6b7, I'm curiou why?
,
Sep 12 2017
To add some more notes to the above: this only happens with navigations that abort (i.e. the external protocol in this case) with net::ERR_ABORTED. For navigation errors that do show an error page, this doesn't repro.
,
Sep 12 2017
@jam: because otherwise the render frame still believes its frame is loading even though it's not. If it's a subframe, the parent frame won't be notified of its load stop, and so it won't execute its load event. However, this issue of not cancelling all loaders on errors is already known: in fact we have https://chromium-review.googlesource.com/c/chromium/src/+/565261 that addresses it. The CL didn't land because the reviewers wanted a test to ensure it doesn't regress. Crafting one was hard, and while doing that we realized that there was another issue in which even if we didn't stop the load we would still not get the OnLoad event because of some logic in Blink.
,
Sep 12 2017
Am I right to assume then that the deletion of the Local State file just momentarily affects whether the new plznavigate feature is enabled and was therefore a false flag relating to the issue? You speak of failed/aborted navigation but I do note that even when this is working, the protocol request is launched and accepted, it still shows as (cancelled) with the state "stalled" in dev tools. Presumably this is expected? Trying to follow along as best I can but is there agreement as to whether this new behavior is "correct" or not? If it is to remain as it is now then we have to seriously re-think our strategy so I appreciate it is early days but clarification would be appreciated.
,
Sep 12 2017
Yes if you launch a request with an external protocol then from the point of view of the browser it is aborted: it is not an HTML navigation that completes inside the browser. The fact that this results in other requests being cancelled in the frame is a bug we're trying to fix.
,
Sep 12 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/dcc7bf4424c988bb00d2ed4c84ccb9e5a6ee2625 commit dcc7bf4424c988bb00d2ed4c84ccb9e5a6ee2625 Author: John Abd-El-Malek <jam@chromium.org> Date: Tue Sep 12 22:30:23 2017 PlzNavigate: don't stop all loaders when renderer-initiated nav fails When a renderer-initiated navigation is aborted by the browser process in PlzNavigate, we currently send a FrameMsg_Stop IPC to the renderer process, in order for the renderer process to know that there is no longer a pending navigation being processed by the browser. This is wrong, as it also cancels any ongoing load in the current page. Instead, this CL introduces a new IPC that will only cancel the dummy provisional DocumentLoader that was created when the navigation was sent to the browser. Continuation of patch: * https://codereview.chromium.org/2768813002/ by clamy@. * https://chromium-review.googlesource.com/c/chromium/src/+/565261 by @ahemery. This CL adds a test for: https://crbug.com/762945 . This bug happens when a navigation is aborted but keeps the current document alive. All the pending loads for the current document were canceled when it should not. Bug:576261, 762945 Cq-Include-Trybots: master.tryserver.chromium.linux:linux_site_isolation Change-Id: I210c5f9132945d81d0c5d7d62fff974597648f20 Reviewed-on: https://chromium-review.googlesource.com/663140 Reviewed-by: Daniel Cheng <dcheng@chromium.org> Reviewed-by: John Abd-El-Malek <jam@chromium.org> Reviewed-by: Camille Lamy <clamy@chromium.org> Commit-Queue: John Abd-El-Malek <jam@chromium.org> Cr-Commit-Position: refs/heads/master@{#501427} [modify] https://crrev.com/dcc7bf4424c988bb00d2ed4c84ccb9e5a6ee2625/content/browser/frame_host/frame_tree_node.cc [modify] https://crrev.com/dcc7bf4424c988bb00d2ed4c84ccb9e5a6ee2625/content/browser/frame_host/navigation_request.cc [modify] https://crrev.com/dcc7bf4424c988bb00d2ed4c84ccb9e5a6ee2625/content/browser/frame_host/navigation_request.h [modify] https://crrev.com/dcc7bf4424c988bb00d2ed4c84ccb9e5a6ee2625/content/browser/frame_host/navigator_impl.cc [modify] https://crrev.com/dcc7bf4424c988bb00d2ed4c84ccb9e5a6ee2625/content/browser/frame_host/render_frame_host_impl_browsertest.cc [modify] https://crrev.com/dcc7bf4424c988bb00d2ed4c84ccb9e5a6ee2625/content/browser/frame_host/render_frame_host_manager.cc [modify] https://crrev.com/dcc7bf4424c988bb00d2ed4c84ccb9e5a6ee2625/content/common/frame_messages.h [modify] https://crrev.com/dcc7bf4424c988bb00d2ed4c84ccb9e5a6ee2625/content/renderer/render_frame_impl.cc [modify] https://crrev.com/dcc7bf4424c988bb00d2ed4c84ccb9e5a6ee2625/content/renderer/render_frame_impl.h [modify] https://crrev.com/dcc7bf4424c988bb00d2ed4c84ccb9e5a6ee2625/third_party/WebKit/Source/core/frame/WebLocalFrameImpl.cpp [modify] https://crrev.com/dcc7bf4424c988bb00d2ed4c84ccb9e5a6ee2625/third_party/WebKit/Source/core/frame/WebLocalFrameImpl.h [modify] https://crrev.com/dcc7bf4424c988bb00d2ed4c84ccb9e5a6ee2625/third_party/WebKit/Source/core/loader/FrameLoader.cpp [modify] https://crrev.com/dcc7bf4424c988bb00d2ed4c84ccb9e5a6ee2625/third_party/WebKit/Source/core/loader/FrameLoader.h [modify] https://crrev.com/dcc7bf4424c988bb00d2ed4c84ccb9e5a6ee2625/third_party/WebKit/public/web/WebLocalFrame.h
,
Sep 13 2017
@ryan.heath@software2.eu: can you quantify how many users are affected by this bug?
,
Sep 13 2017
Our last estimate of our user base was 1.5 million. I can't say for sure how many are chrome users but I know that more than a handful of our customers use Chrome as their default browser on all managed machines which based on averages is prob about 100k but the whole point of our product is to support users bringing in their own device so a very rough assumption would be to apply your current estimated market share (50-60%?) to our user base.
,
Sep 13 2017
Requesting merge to 61. This only affects PlzNavigate code path, so can be turned off by finch. We've had it on canary for a day with no reported bugs. The patch is bigger than it looks because of a variable rename and plumbing to reach WebKit code. We think it's pretty low risk. Thanks
,
Sep 13 2017
This bug requires manual review: Request affecting a post-stable build Please contact the milestone owner if you have questions. Owners: amineer@(Android), cmasso@(iOS), ketakid@(ChromeOS), govind@(Desktop) For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Sep 13 2017
Approving merge to M61 branch 3163 based on comment #28 and per offline chat with jam@. Please merge ASAP. Also request a merge to M62. Thank you.
,
Sep 13 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/819fd99740781b4c2e5815f7d84ef34c4488b943 commit 819fd99740781b4c2e5815f7d84ef34c4488b943 Author: Matt Menke <mmenke@chromium.org> Date: Wed Sep 13 16:36:15 2017 Disable a content_browsertest when using the network service. RenderFrameHostImplBrowserTest.AbortedRendererInitiatedNavigationDoNotCancelPendingXHR is now failing, possibly due to https://chromium-review.googlesource.com/663140 TBR: jam@chromium.org NOTRY: true Bug: 576261, 762945 Change-Id: Ib9a8893e105444be13653438c6bd364511dcfad7 Reviewed-on: https://chromium-review.googlesource.com/665070 Reviewed-by: Matt Menke <mmenke@chromium.org> Commit-Queue: Matt Menke <mmenke@chromium.org> Cr-Commit-Position: refs/heads/master@{#501659} [modify] https://crrev.com/819fd99740781b4c2e5815f7d84ef34c4488b943/testing/buildbot/filters/mojo.fyi.network_content_browsertests.filter
,
Sep 13 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/12a50651b2ffdbcb210a2c2f471fa0a6c5b6b452 commit 12a50651b2ffdbcb210a2c2f471fa0a6c5b6b452 Author: John Abd-El-Malek <jam@chromium.org> Date: Wed Sep 13 17:21:59 2017 PlzNavigate: don't stop all loaders when renderer-initiated nav fails When a renderer-initiated navigation is aborted by the browser process in PlzNavigate, we currently send a FrameMsg_Stop IPC to the renderer process, in order for the renderer process to know that there is no longer a pending navigation being processed by the browser. This is wrong, as it also cancels any ongoing load in the current page. Instead, this CL introduces a new IPC that will only cancel the dummy provisional DocumentLoader that was created when the navigation was sent to the browser. Continuation of patch: * https://codereview.chromium.org/2768813002/ by clamy@. * https://chromium-review.googlesource.com/c/chromium/src/+/565261 by @ahemery. This CL adds a test for: https://crbug.com/762945 . This bug happens when a navigation is aborted but keeps the current document alive. All the pending loads for the current document were canceled when it should not. Bug:576261, 762945 Cq-Include-Trybots: master.tryserver.chromium.linux:linux_site_isolation Change-Id: I210c5f9132945d81d0c5d7d62fff974597648f20 Reviewed-on: https://chromium-review.googlesource.com/663140 Reviewed-by: Daniel Cheng <dcheng@chromium.org> Reviewed-by: John Abd-El-Malek <jam@chromium.org> Reviewed-by: Camille Lamy <clamy@chromium.org> Commit-Queue: John Abd-El-Malek <jam@chromium.org> Cr-Original-Commit-Position: refs/heads/master@{#501427} Reviewed-on: https://chromium-review.googlesource.com/664600 Cr-Commit-Position: refs/branch-heads/3163@{#1185} Cr-Branched-From: ff259bab28b35d242e10186cd63af7ed404fae0d-refs/heads/master@{#488528} [modify] https://crrev.com/12a50651b2ffdbcb210a2c2f471fa0a6c5b6b452/content/browser/frame_host/frame_tree_node.cc [modify] https://crrev.com/12a50651b2ffdbcb210a2c2f471fa0a6c5b6b452/content/browser/frame_host/navigation_request.cc [modify] https://crrev.com/12a50651b2ffdbcb210a2c2f471fa0a6c5b6b452/content/browser/frame_host/navigation_request.h [modify] https://crrev.com/12a50651b2ffdbcb210a2c2f471fa0a6c5b6b452/content/browser/frame_host/navigator_impl.cc [modify] https://crrev.com/12a50651b2ffdbcb210a2c2f471fa0a6c5b6b452/content/browser/frame_host/render_frame_host_manager.cc [modify] https://crrev.com/12a50651b2ffdbcb210a2c2f471fa0a6c5b6b452/content/common/frame_messages.h [modify] https://crrev.com/12a50651b2ffdbcb210a2c2f471fa0a6c5b6b452/content/renderer/render_frame_impl.cc [modify] https://crrev.com/12a50651b2ffdbcb210a2c2f471fa0a6c5b6b452/content/renderer/render_frame_impl.h [modify] https://crrev.com/12a50651b2ffdbcb210a2c2f471fa0a6c5b6b452/third_party/WebKit/Source/core/loader/FrameLoader.cpp [modify] https://crrev.com/12a50651b2ffdbcb210a2c2f471fa0a6c5b6b452/third_party/WebKit/Source/core/loader/FrameLoader.h [modify] https://crrev.com/12a50651b2ffdbcb210a2c2f471fa0a6c5b6b452/third_party/WebKit/Source/web/WebLocalFrameImpl.cpp [modify] https://crrev.com/12a50651b2ffdbcb210a2c2f471fa0a6c5b6b452/third_party/WebKit/Source/web/WebLocalFrameImpl.h [modify] https://crrev.com/12a50651b2ffdbcb210a2c2f471fa0a6c5b6b452/third_party/WebKit/public/web/WebLocalFrame.h
,
Sep 13 2017
,
Sep 14 2017
This bug requires manual review: M62 has already been promoted to the beta branch, so this requires manual review Please contact the milestone owner if you have questions. Owners: amineer@(Android), cmasso@(iOS), bhthompson@(ChromeOS), abdulsyed@(Desktop) For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Sep 15 2017
Approving merge to M62. Branch:3202
,
Sep 18 2017
Am I to take from C#35 that this will be in an upcoming release? Would there be a date assigned if so?
,
Sep 18 2017
@ryan.heath: yes we've merged the fix to a new version of 61 (61.0.3163.91) that was released a few days ago.
,
Sep 18 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/f5001334356121487e1023f9ead9e072601153b1 commit f5001334356121487e1023f9ead9e072601153b1 Author: John Abd-El-Malek <jam@chromium.org> Date: Mon Sep 18 11:08:21 2017 PlzNavigate: don't stop all loaders when renderer-initiated nav fails When a renderer-initiated navigation is aborted by the browser process in PlzNavigate, we currently send a FrameMsg_Stop IPC to the renderer process, in order for the renderer process to know that there is no longer a pending navigation being processed by the browser. This is wrong, as it also cancels any ongoing load in the current page. Instead, this CL introduces a new IPC that will only cancel the dummy provisional DocumentLoader that was created when the navigation was sent to the browser. Continuation of patch: * https://codereview.chromium.org/2768813002/ by clamy@. * https://chromium-review.googlesource.com/c/chromium/src/+/565261 by @ahemery. This CL adds a test for: https://crbug.com/762945 . This bug happens when a navigation is aborted but keeps the current document alive. All the pending loads for the current document were canceled when it should not. Bug:576261, 762945 Cq-Include-Trybots: master.tryserver.chromium.linux:linux_site_isolation Change-Id: I210c5f9132945d81d0c5d7d62fff974597648f20 Reviewed-on: https://chromium-review.googlesource.com/663140 Reviewed-by: Daniel Cheng <dcheng@chromium.org> Reviewed-by: John Abd-El-Malek <jam@chromium.org> Reviewed-by: Camille Lamy <clamy@chromium.org> Commit-Queue: John Abd-El-Malek <jam@chromium.org> Cr-Original-Commit-Position: refs/heads/master@{#501427}(cherry picked from commit dcc7bf4424c988bb00d2ed4c84ccb9e5a6ee2625) Reviewed-on: https://chromium-review.googlesource.com/670845 Cr-Commit-Position: refs/branch-heads/3202@{#279} Cr-Branched-From: fa6a5d87adff761bc16afc5498c3f5944c1daa68-refs/heads/master@{#499098} [modify] https://crrev.com/f5001334356121487e1023f9ead9e072601153b1/content/browser/frame_host/frame_tree_node.cc [modify] https://crrev.com/f5001334356121487e1023f9ead9e072601153b1/content/browser/frame_host/navigation_request.cc [modify] https://crrev.com/f5001334356121487e1023f9ead9e072601153b1/content/browser/frame_host/navigation_request.h [modify] https://crrev.com/f5001334356121487e1023f9ead9e072601153b1/content/browser/frame_host/navigator_impl.cc [modify] https://crrev.com/f5001334356121487e1023f9ead9e072601153b1/content/browser/frame_host/render_frame_host_impl_browsertest.cc [modify] https://crrev.com/f5001334356121487e1023f9ead9e072601153b1/content/browser/frame_host/render_frame_host_manager.cc [modify] https://crrev.com/f5001334356121487e1023f9ead9e072601153b1/content/common/frame_messages.h [modify] https://crrev.com/f5001334356121487e1023f9ead9e072601153b1/content/renderer/render_frame_impl.cc [modify] https://crrev.com/f5001334356121487e1023f9ead9e072601153b1/content/renderer/render_frame_impl.h [modify] https://crrev.com/f5001334356121487e1023f9ead9e072601153b1/third_party/WebKit/Source/core/frame/WebLocalFrameImpl.cpp [modify] https://crrev.com/f5001334356121487e1023f9ead9e072601153b1/third_party/WebKit/Source/core/frame/WebLocalFrameImpl.h [modify] https://crrev.com/f5001334356121487e1023f9ead9e072601153b1/third_party/WebKit/Source/core/loader/FrameLoader.cpp [modify] https://crrev.com/f5001334356121487e1023f9ead9e072601153b1/third_party/WebKit/Source/core/loader/FrameLoader.h [modify] https://crrev.com/f5001334356121487e1023f9ead9e072601153b1/third_party/WebKit/public/web/WebLocalFrame.h
,
Sep 26 2017
,
Sep 26 2017
Thank you to everyone involved in resolving this issue, this was a critical one for us and your responsiveness and professionalism in getting this dealt with in such a short time is very much appreciated.
,
Sep 26 2017
@ryan.heath: you're welcome. Please have folks in your company use Chrome Beta for testing to prevent such problems in the future.
,
Sep 26 2017
That's absolutely the lesson we have learned from this whole experience and will definitely be doing so!
,
Sep 26 2017
Great, thanks. |
|||||||||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||||||||
Comment 1 by atwilson@chromium.org
, Sep 7 2017