Issue metadata
Sign in to add a comment
|
POST XHR requests getting cancelled
Reported by
unreal...@gmail.com,
Jul 24
|
||||||||||||||||||||||||
Issue descriptionUserAgent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36 Example URL: Steps to reproduce the problem: The problem is intermittent ( a.k.a. I haven't found a good way to reproduce it ), but it happens from time to time and persists Chrome restarts ). I'm working on WordPress.com's Calypso user interface and one of the functions we have implemented is site imports via file upload. The flow looks like this: 1. User clicks a button 2. A file selection dialog is shown 3. We submit the form with the file and additional POST data via an iframe proxy What is the expected behavior? The request successfully finishing and the file being uploaded to the backend. What went wrong? The request gets cancelled virtually immediately, without even being sent to the server. Did this work before? N/A Chrome version: 67.0.3396.99 Channel: stable OS Version: OS X 10.13.5 Flash Version: I mentioned "virtually immediately" above, since the request is not quite immediately cancelled in terms of what I'm able to capture via debugging the XMLHttpRequest object that gets created. What happens is that immediately after `send`-ing the request I get exactly one `onprogress` event fired for this call and then the request magically gets status Cancelled in Chrome's Dev tools. No other events are fired for the request, including `onerror`, `onload`. Also no trace of the request in Chrome's log ( CHROME_LOG_FILE=~/Downloads/browsers/chromelog.log ./Google\ Chrome --enable-logging --v=20 --blink-platform-log-channels <testsite_url> ). No trace of the request in Chrome's net-internals Events tab ( chrome://net-internals/#events ). As if it didn't even get registered in the network stack. I don't see any other requests running at the time. The remote server supports HTTP/2 and other requests to similar endpoints go over that. The offending request doesn't get a protocol assigned. The issue occurs from time to time, but it persists after several restarts of Chrome and switching command line parameters. One thing that fixes the issue most of the times is toggling Strict Site Isolation on or off and restarting Chrome. Not sure what this resets internally. I haven't cleared cache or cookies just in case if I'm able to track down the issue that's causing the request to fail. I'm currently reproducing it via a local instance of Calypso ( running on `http://calypso.localhost:3000` ), but I've seen this happen even in production builds and hosts ( i.e. https://wordpress.com context ). I'm happy to help debug the issue and gather more data/debug build if there is a way to enable more logging. I'd prefer providing trace/log data privately if possible, as it may contain sensitive information about URLs accessed and such.
,
Jul 25
,
Jul 31
Could you explain the "iframe proxy" part some more? If the iframe gets detached for some reason then an XHR launched from that iframe would be cancelled. Is it possible to reduce the page down to the absolute minimum necessary to reproduce the issue, and then share it with us?
,
Jul 31
> Is it possible to reduce the page down to the absolute minimum necessary to reproduce the issue, and then share it with us? I'm still working on that, but it might take a while, since the issue is not always present and it takes a while to reappear. It's quite a big React app and there are a lot of moving parts. I'm trying to also figure out if there is a way to enable extremely verbose logging in Chrome to figure out possible sources of errors when the file upload happens, so I can make sure I'm looking in the right places. Any pointers here are greatly appreciated! > Could you explain the "iframe proxy" part some more? If the iframe gets detached for some reason then an XHR launched from that iframe would be cancelled. Calypso is a React app that is served from a NodeJS server. To communicate with the API we send the XHRs to the API via the iframe proxy I mentioned. The API and rest-proxy iframe live on `https://public-api.wordpress.com`, while Calypso may live on several different hosts: * https://wordpress.com (production) * https://wpcalypso.wordpress.com (staging/test environment) * http://calypso.localhost:3000 (local development) The way that "sending" happens is: * XHR request data from Calypso is serialized and sent to the iframe via `window.postMessage` * JS in the iframe side then converts the data to a proper XMLHttpRequest * On any event of the actual XMLHttpRequest, the iframe fires another `postMessage` to the parent (Calypso) to notify of the progress * When the XMLHttpRequest finishes, the data/error gets sent back again via `postMessage` to the parent (Calypso) When the failed state occurs, only the file upload request I make is failing. Every other request through that proxy is passing through both ways to and from the iframe. I did some debugging during failures and I can see the request being made. The request receives body of `FormData` object, which has two items, a JSON data as a string and `File` object that contains the file that should be uploaded. When trying to `console.log` debug a bit around the `File` object I see it reported as `size: 0`, even if trying with different sized files. I'm having a slight suspicion that the file upload might be causing the issue with that request, but I'm not sure how to confirm that. I don't see any errors anywhere, just the request being cancelled. Going to research that as a reproduction case too. Right now I've managed to get Chrome stuck in this state and the previous fix of toggling Strict Site Isolation on and off doesn't help, even after multiple full-restarts (CMD + Q) of the browser. Any suggestions how to approach further debugging and what I might look out for in Chrome settings/data?
,
Jul 31
Thank you for providing more feedback. Adding the requester to the cc list. For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Jul 31
Just to confirm, are you seeing this problem even if you turn off site isolation via chrome://flags/#site-isolation-trial-opt-out?
,
Jul 31
> Just to confirm, are you seeing this problem even if you turn off site isolation via chrome://flags/#site-isolation-trial-opt-out? I'm testing with `Strict site isolation` ( #enable-site-per-process ) enable/disable (had it enabled at the moment it was released). I see the opt-out wouldn't matter in this case: "Opt-out has no effect if site isolation is force-enabled via #enable-site-per-process" The issue happens in both cases - enabled or disabled.
,
Jul 31
Forgot to add to the above - a while ago toggling Site Isolation on and off was fixing the issue. Probably it was resetting something that was causing the bug. I had it confirmed that toggling site isolation was "fixing" it with several colleagues who were hitting the same issue. Another things I forgot to add in Comment 4 ( https://bugs.chromium.org/p/chromium/issues/detail?id=866805#c4 ): The bug appeared on multiple machines, all Macs. We haven't tested with Windows or Linux versions of Chrome. The issue manifests in an interesting way. Since I mentioned that we have several "origins" for Calypso: * https://wordpress.com (production) * https://wpcalypso.wordpress.com (staging/test environment) * http://calypso.localhost:3000 (local development) Sometimes the issue appears on one, two or all of the origins, while working on the others. So for example if it's broken on calypso.localhost, it might work on wpcalypso.wordpress.com and wordpress.com. Other times it was working on calypso.localhost, but not on wpcalypso.wordpress.com. This was all tested and confirmed in the same session ( parallel open tabs in the same window ) with both Site Isolation enabled and disabled.
,
Aug 1
,
Aug 2
Just a sanity check on how the flags work. If #enable-site-per-process is on, then you will be running with strict site isolation regardless of other flags. If #enable-site-per-process is OFF (the default), Chrome will fall back to a field trial config to determine whether or not site isolation should be enabled, and since M67, almost all desktop users are configured to use site isolation. So to test Chrome with site isolation off, it is necessary to both have #enable-site-per-process as off and also setting chrome://flags/#site-isolation-trial-opt-out to "Opt out". You might have already been testing like this, just wanted to confirm.
,
Aug 3
The "stickiness" of the issue is interesting. It may imply that there's a field trial involved. It's a long shot, but could you copy and paste the "Variations" field from chrome://version/ here?
,
Aug 3
> So to test Chrome with site isolation off, it is necessary to both have #enable-site-per-process as off and also setting chrome://flags/#site-isolation-trial-opt-out to "Opt out". You might have already been testing like this, just wanted to confirm. Oops :) I misunderstood how this worked then! I retested with having "Strict site isolation" set to Disabled and changed only the Opt-out option: Strict Site Isolation: Disabled Site isolation trial opt-out: Default => Failure to upload file - behavior as described previously. Strict Site Isolation: Disabled Site isolation trial opt-out: Opt-out => I was able to upload the file! Reverting the flags to the previous state (Disabled/Default) lead it to broken state again, so it didn't persist the "fix" as it was doing before. Previously toggling Site Isolation on/off managed to fix it for a while before it appeared again, now it doesn't. > The "stickiness" of the issue is interesting. It may imply that there's a field trial involved. It's a long shot, but could you copy and paste the "Variations" field from chrome://version/ here? Since I wasn't sure if it mattered to capture what the differences are (after figuring out that I haven't tested properly previously, as mentioned above), I did the test 4 times, by toggling the Site Isolation trial opt-out between Opt-out and Default: Site Isolation trial opt-out: default (1st try - failure to upload) c134752e-552119 5f419cc9-ca7d8d80 59aeb88e-3f4a17df ebeb14fc-3f4a17df 34a6bf44-ca7d8d80 31101bd6-71a713bf 3095aa95-3f4a17df 7c1bc906-f55a7974 47e5d3db-3d47f4f4 4dc30737-b8a5ea08 589b4e9f-3f4a17df f9884634-659882c0 121ae2bc-ca7d8d80 ceff87ec-3f4a17df 44827ee5-f23d1dea d0ecf1da-1c2cc847 8f1e27f-ca7d8d80 937cad47-17943b1e 9773d3bd-f23d1dea 9e5c75f1-1039a221 da92f51d-3d47f4f4 f79cb77b-3d47f4f4 4ea303a6-2e46ed91 bcc34a89-3f4a17df 2b33233e-cc2bd7de 2c1d398c-ca7d8d80 58a025e3-36e97b2c 2a32876a-ca7d8d80 ff29b1bd-37ef7e17 da460ac8-3f4a17df 4bc337ce-69465896 1354da85-ca7d8d80 17507c76-ca7d8d80 494d8760-52325d43 f47ae82a-86f22ee5 3ac60855-486e2a9c f296190c-6bdfffe7 4442aae2-7158671e ed1d377-e1cc0f14 12e17bc5-e1cc0f14 75f0f0a0-e1cc0f14 e2b18481-6e3b1976 e7e71889-4ad60575 f5fff3a2-fe8847a 94e68624-803f8fc4 Site Isolation trial opt-out: Opt-out (2nd try - successful upload) c134752e-552119 5f419cc9-ca7d8d80 59aeb88e-3f4a17df ebeb14fc-3f4a17df 34a6bf44-ca7d8d80 3095aa95-3f4a17df 7c1bc906-f55a7974 47e5d3db-3d47f4f4 4dc30737-b8a5ea08 589b4e9f-3f4a17df f9884634-659882c0 121ae2bc-ca7d8d80 ceff87ec-3f4a17df 44827ee5-f23d1dea d0ecf1da-1c2cc847 8f1e27f-ca7d8d80 937cad47-17943b1e 9773d3bd-f23d1dea 9e5c75f1-1039a221 da92f51d-3d47f4f4 f79cb77b-3d47f4f4 4ea303a6-2e46ed91 bcc34a89-3f4a17df 2b33233e-cc2bd7de 2c1d398c-ca7d8d80 58a025e3-36e97b2c 2a32876a-ca7d8d80 da460ac8-3f4a17df 4bc337ce-69465896 1354da85-ca7d8d80 17507c76-ca7d8d80 494d8760-52325d43 f47ae82a-86f22ee5 3ac60855-486e2a9c f296190c-6bdfffe7 4442aae2-7158671e ed1d377-e1cc0f14 12e17bc5-e1cc0f14 75f0f0a0-e1cc0f14 e2b18481-6e3b1976 e7e71889-4ad60575 f5fff3a2-fe8847a 94e68624-803f8fc4 Site Isolation trial opt-out: Default (3rd try - failure to upload) c134752e-552119 5f419cc9-ca7d8d80 59aeb88e-3f4a17df ebeb14fc-3f4a17df 34a6bf44-ca7d8d80 3095aa95-3f4a17df 7c1bc906-f55a7974 47e5d3db-3d47f4f4 4dc30737-b8a5ea08 589b4e9f-3f4a17df f9884634-659882c0 121ae2bc-ca7d8d80 ceff87ec-3f4a17df 44827ee5-f23d1dea d0ecf1da-1c2cc847 8f1e27f-ca7d8d80 937cad47-17943b1e 9773d3bd-f23d1dea 9e5c75f1-1039a221 da92f51d-3d47f4f4 f79cb77b-3d47f4f4 4ea303a6-2e46ed91 bcc34a89-3f4a17df 2b33233e-cc2bd7de 2c1d398c-ca7d8d80 58a025e3-36e97b2c 2a32876a-ca7d8d80 ff29b1bd-37ef7e17 da460ac8-3f4a17df 4bc337ce-69465896 1354da85-ca7d8d80 17507c76-ca7d8d80 494d8760-52325d43 f47ae82a-86f22ee5 3ac60855-486e2a9c f296190c-6bdfffe7 4442aae2-7158671e ed1d377-e1cc0f14 12e17bc5-e1cc0f14 75f0f0a0-e1cc0f14 e2b18481-6e3b1976 e7e71889-4ad60575 f5fff3a2-fe8847a 94e68624-803f8fc4 Site Isolation trial opt-out: Opt-out (4th try - successful upload) c134752e-552119 5f419cc9-ca7d8d80 59aeb88e-3f4a17df ebeb14fc-3f4a17df 34a6bf44-ca7d8d80 3095aa95-3f4a17df 7c1bc906-f55a7974 47e5d3db-3d47f4f4 4dc30737-b8a5ea08 589b4e9f-3f4a17df f9884634-659882c0 121ae2bc-ca7d8d80 ceff87ec-3f4a17df 44827ee5-f23d1dea d0ecf1da-94996243 8f1e27f-ca7d8d80 937cad47-17943b1e 9773d3bd-f23d1dea 9e5c75f1-1039a221 da92f51d-3d47f4f4 f79cb77b-3d47f4f4 4ea303a6-2e46ed91 bcc34a89-3f4a17df 2b33233e-cc2bd7de 2c1d398c-ca7d8d80 58a025e3-36e97b2c 2a32876a-ca7d8d80 da460ac8-3f4a17df 4bc337ce-69465896 1354da85-ca7d8d80 17507c76-ca7d8d80 494d8760-52325d43 f47ae82a-86f22ee5 3ac60855-486e2a9c f296190c-6bdfffe7 4442aae2-7158671e ed1d377-e1cc0f14 12e17bc5-e1cc0f14 75f0f0a0-e1cc0f14 e2b18481-6e3b1976 e7e71889-4ad60575 f5fff3a2-fe8847a 94e68624-803f8fc4
,
Aug 3
Decoded hashes: Site Isolation trial opt-out: default (1st try - failure to upload) AccountConsistencyVariations-DicePrepareMigration_Launched ArrayPrototypeValues-Default AutofillCreditCardBankNameDisplay-Enabled AutofillResetFullServerCardsOnAuthError-Enabled AutofillRestrictUnownedFieldsToFormlessCheckout-Default AutofillSingleClick-EnabledDefault ChromeChannelStable-Enabled ClientSideDetectionModel-Model0 DataCompressionProxyHoldback-Disabled DisableAscriberIfDataSaverDisabled-CDUA_Disabled GCMTokenInvalidAfterDays-Enabled GaiaPasswordReuse-SofterWarningWarnOnLowReputation HTTPBadPhase3-Default LocalScreenCasting-Enabled LowPriorityIframes2-Control MacV2Sandbox-MacV2Sandbox_R3 ModernMediaControls-Default NetAdaptiveProxyConnectionTimeout-Enabled_v7 NetDelayableH2AndQuicRequests-Control OmniboxBundledExperimentV1-ChromeDesktop_ZeroSuggestRedirectToChrome_Enabled8_PostPeriod2 OmniboxWarmUpSearchProviderOnFocus-Disabled PasswordGeneration-Disabled QUIC-FlagDisabled RendererSideResourceScheduler-Enabled SafeBrowsingAdSamplerTrigger-AdSampler_NormalRate2 ScrollAnchorSerialization-Default SharedArrayBuffer-EnableSAB SimplifyHttpsIndicator-Default SitePerProcess-Enabled_97_SitePerProcess_20180607 SpdyProxiesRequestsDelayable-Enabled TLS13Variant-EnabledV4 TranslateRankerModel-Default TranslateUserEvents-Default UKM-Enabled_20180314 UMA-Dynamic-Uniformity-Trial-Group6 UMA-Population-Restrict-normal UMA-Uniformity-Trial-1-Percent-group_02 UMA-Uniformity-Trial-10-Percent-group_05 UMA-Uniformity-Trial-100-Percent-group_01 UMA-Uniformity-Trial-100-Percent-V2-group_01 UMA-Uniformity-Trial-20-Percent-group_01 UMA-Uniformity-Trial-5-Percent-group_09 UMA-Uniformity-Trial-50-Percent-default UseMojoAudioOutputStreamFactory-DontUseMojoAudioOutputStreamFactory VideoCaptureService-Enabled2 Site Isolation trial opt-out: Opt-out (2nd try - successful upload) Same as #1 except the following lines are *removed*: AutofillSingleClick-EnabledDefault SitePerProcess-Enabled_97_SitePerProcess_20180607 Site Isolation trial opt-out: Default (3rd try - failure to upload) Same as #1 except the following line is *removed*: AutofillSingleClick-EnabledDefault Site Isolation trial opt-out: Opt-out (4th try - successful upload) Same as #2 except MacV2Sandbox-MacV2Sandbox_R3 has been replaced with MacV2Sandbox-Control_R3 I'm confident the change to the AutofillSingleClick trial is unrelated. The MacV2Sandbox change is unlikely to be related. So, it looks like brokenness is strongly correlated with site-per-process being enabled. However, IIUC, some other installations of the same version on the same OS still work even when site-per-process is enabled, so it's not the only factor involved. It's probably unrelated, but if possible could you compare chrome://site-engagement/ for the relevant hosts on a browser that has the issue and a browser that doesn't?
,
Aug 3
,
Aug 9
Would site-engagement tell us much, if the issue appears intermittently in the same browser? I'd imagine the site engagement for the hosts won't change much? I had a go at this too. I had the issue in Chromium this morning - there seemed a clear link with site-isolation-trial-opt-out, because the upload broke with it at default, worked with it opted out, and broke again with it at default (#enable-site-per-process was disabled). When I tried again ten minutes later, the issue had disappeared, no matter which flag setting I used. In case useful, attached is the variations I get in different success/failure cases in Chrome. The host I tested with currently has a site-engagement value of 53.67 - it's 6th on the list.
,
Aug 9
> It's probably unrelated, but if possible could you compare chrome://site-engagement/ for the relevant hosts on a browser that has the issue and a browser that doesn't? Unfortunately I couldn't find a browser that doesn't fail right now :( All the browsers I've tested with fail reliably with the Site Isolation flags set above. For me the site engagement is 26.01, while another test laptop I have is 6. I'm still trying to get a small reproduction scenario, as it's a bit hard to figure out what to cut out and make it work ( usually just fails :) ), but I've managed to get a reproducible scenario that requires a clean build of Calypso, so anyone should be able to reproduce it. To reproduce: 1. Requirements: git, npm, nodejs (preferably 10.5.0) 2. Clone https://github.com/Automattic/wp-calypso in a local folder 3. Go into the cloned working copy 4. npm install (yarn install might be faster here) 5. npm start (takes a while) 6. Go to http;//calypso.localhost:3000/start 7. Go through signup and when prompted for domain address type anything and choose the `.wordpress.com` subdomain variant 8. Choose Free as a plan 9. Sign up either with a known mail or throw-away mail (you have to confirm your e-mail in order to test one of the fail paths, the other doesn't require it). 10. Finish signup and you should land inside a check-list of sort. 11. On the sidebar choose `Add` under `Blog posts` or `Pages` 12. Open Dev tools 13. Type a few lines of text so you can scroll down 14. Scroll down to the bottom (bad UI, sorry :( ) 15. Drag/drop a file over the content area. You should see a BLUE drop zone light up. Drop the file there. 15.1 When testing this I found out that the orange drop zone works, but it's using the same code, so I'm going to investigate further there. That doesn't help us figure out the original issue. 15.2 Further testing (while writing the test instructions) shows that while the request succeeds for Featured image (orange drop zone), the server receives an empty file. See notes/remarks below. 16. See the request gets cancelled. This is the simplest reproduction scenario I could think of for the moment. Steps 1-10 should be done only once if you don't run in incognito mode and clear out cookies. Otherwise you have to either remember account/password pair to log-in again. As for the second path - you have to verify your e-mail. 1. Go to `Import` on the sidebar (you have to exit the post editor via Close in the upper left) 2. Choose WordPress 3. Drag any file over the drop zone and drop it 4. The request gets cancelled Notes/remarks: As pointed in 15.2 above, while testing this I discovered that in the post editor, almost the same code behaves differently. While one of the requests gets cancelled, the other one succeeds, but the server says the file sent to it is empty. This leads me to believe that there is an issue with `File` objects that get passed between Calypso and the Proxy inside the hidden iframe when Strict Site Isolation is enabled. Pending further investigation of the code differences to figure out why one of the requests manages to get sent to the server, but with an empty file, while the other is outright cancelled. We just tested this with running Calypso's local copy over httpS (https://calypso.localhost:3001) - the same issue. So there's something at play that only affects the `calypso.localhost` case at the moment. Works fine under `wpcalypso.wordpress.com` and `wordpress.com`
,
Aug 9
Thank you for providing more feedback. Adding the requester to the cc list. For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Aug 9
Do you keep a reference to the File in the original page after posting it to the iframe? Premature garbage collection of some sort of reference could possibly be an issue. Added Blink>Storage>FileAPI to get some storage-related eyeballs on this.
,
Aug 9
This sounds a lot like bug 631877, which you would be much more likely to trigger with a cross origin iframe and site isolation enabled. What is happening is a combination of two things: - certain type of File objects (i.e. those that represent actual files on disk) are sometimes treated specially, and one particular place where that happens is in the network upload code. - this special treatment doesn't work when transferring the File object to a different process. One possible workaround that will probably work for you is to wrap the File object you get from the input form in a new File or Blob object before sending it to the cross origin iframe. That should be enough to thwart the (broken) special handling of files, and should then let the cross origin iframe upload the file correctly.
,
Aug 10
> One possible workaround that will probably work for you is to wrap the File object you get from the input form in a new File or Blob object before sending it to the cross origin iframe. That should be enough to thwart the (broken) special handling of files, and should then let the cross origin iframe upload the file correctly.
I can confirm that this worked!
```
const newFile = new File( [ file.slice( 0, file.size ) ], file.name, { type: file.type } );
```
Before the upload action happens and the file gets successfully uploaded.
> - certain type of File objects (i.e. those that represent actual files on disk) are sometimes treated specially, and one particular place where that happens is in the network upload code.
What are the conditions for file types and "sometimes", since this is happening only in the `calypso.localhost` context, but not when the same code is ran in `wpcalypso.wordpress.com` or `wordpress.com` context.
,
Aug 13
|
|||||||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||||||
Comment 1 by mmenke@chromium.org
, Jul 24