[iOS] New URLRequests are going through cellular interface even when client is connected to wifi |
||||||||||||||||||
Issue descriptionWhen an iPhone (or iPad with a SIM) connects to wifi network, Cronet continues to use cellular connection. This behavior might be expected. However, if the client wants to switch to WiFi, there is no way to accomplish that. Even when the 'old' requests are canceled and new requests are created, the traffic goes through the cellular interface. This behavior is only observed for QUIC but not H2. Theory: QUIC continues reusing connections from the pool after device transitions to wifi. That can happen if iOS doesn't kill connected sockets when wifi interface becomes active. To handle this, the QUIC implementation should monitor network changes and mark sessions as "go away" when the default ip address changes. However, that mechanism kicks in only if "close_session_on_ip_change" QUIC parameter is "true"(1). However, it is "false" by default. 1) https://cs.chromium.org/chromium/src/net/quic/chromium/quic_stream_factory.cc?l=850
,
May 24 2018
To confirm the theory, it would be nice to collect a NetLog to take a look at the QUIC session liveness upon ip address change. If the session still sticks around after the ip address change and is able to make slow progress, then iOS didn't kill the socket and QUIC was stuck in the old network in this case.
,
May 24 2018
Re #1: Instead of close all the sessions on the old network, we could mark sessions as going away, let them finish on the old interface. One thing to note is that "close_sessions_on_ip_change" relies on the NetworkChangeNotifier to deliver the ip address change message. On Android, there's a theory that IP address change notification from NCN may be delivered spuriously even when you are on the same network, the platform decides to kill the socket on default network change therefore. I am not sure how the NCN on iOS works.
,
May 24 2018
,
May 24 2018
Re #2 I can provide the NetLog files under different scenarios. "NetLog - false close_sessions_on_ip_change.json" is the use case with reachability switching from wwan to wifi under default setting, i.e. close_sessions_on_ip_change is false. NetLog with close_sessions_on_ip_change as true will be attached later.
,
May 24 2018
The NCN on iOS used SCNetworkReachability(1) to monitor the network status. We should be able to reproduce the issue locally. jni@, please let me know if the logs are not easily available. I can try to collect them locally in that case. 1) https://developer.apple.com/documentation/systemconfiguration/scnetworkreachability
,
May 24 2018
Is it possible for us to have something like "max_time_on_non_default_network_seconds" to keep a maximum time bound to stay on the old connection on network interface changes. This can prevent repeatedly closing and opening connections on transient network conditions as recommended by apple here: https://forums.developer.apple.com/thread/39632
,
May 24 2018
"max_time_on_non_default_network_seconds" is a config option used in connection migration to control data usage on non default network, especially for cellular. Since QUIC can use 0-RTT, open up a new connection shouldn't introduce too much overhead. Ideally we may want to have connection migration on iOS to solve the problem: seamlessly migrate requests between networks without killing the connections. However, currently NetworkHandle is not supported in iOS, which blocks us to bring the feature to iOS.
,
May 25 2018
zhongyi@ kapishnikov@ NetLogs with different settings are as attached ones.
,
May 25 2018
Did you notice any behavior difference between this two? Looking at the NetLogs, especially the one with param on, there's no QUIC_SESSION closed with IP_ADDRESS_CHANGE. I suspect the NCN on iOS is not working as expected and we didn't receive any IPAddressChange. Unfortunately, current cronet didn't have any net log entry for IPAddressChange. It would be nice to add a net log entry to confirm the behavior of NCN on iOS. https://cs.chromium.org/chromium/src/net/quic/chromium/quic_stream_factory.cc?dr&sq=package:chromium&rcl=0a0244661b2798de0b4232bd1bcff6b297f82236&g=0&l=1261
,
May 25 2018
Re #10 We do see behavior differences between these two: 1. With close_sessions_on_ip_change off, requests which use QUIC seem still going through the old connections after network interface change. We identify this by checking the ip address and also the reusedConnection flag in the metrics: https://cs.chromium.org/chromium/src/components/cronet/ios/cronet_metrics.h?sq=package:chromium&dr&g=0&l=62 2. With close_sessions_on_ip_change on, outstanding requests upon network interface change failed due to NETWORK_CHANGED error (see attached screenshot). While in the other case they will not fail. For the second point, is there any way to detect whether these request failed just due to no connection in the middle of switch, or it is due to QUIC_SESSION close?
,
May 25 2018
Could you double check the NetLog with flag on provided in comment #9 is the right one? I didn't saw any request fail due to NETWORK_CHANGED, neither did I see any QUIC session closed due to IP_ADDRESS_CHANGE. My guess is that the NetLog_ close_sessions_on_ip_change.json.zip may not be the right log.
,
May 25 2018
Re #12 zhongyi@ Maybe some events were missing in the previous NetLog. Would you help check the NetLog as attached file? Thanks!
,
May 25 2018
I checked the new NetLog, it still doesn't seem to match your observed behavior. But your observed behavior matches my understanding of the code. Talked to Andrei offline, he will work on collecting a correct NetLog to verify the fix. At the same time, we will work on change the close session on ip address to mark session away so that old requests can finish on the old network, leaving the decision for app layer to decide if the request should be cancelled.
,
May 25 2018
Re #14: Thanks zhongyi@ kapishnikov@. Questions: For 'seamlessly migrate requests between networks without killing the connections', what is the difference from 'mark session away so that old requests can finish on the old network'? And will similar functionality as NetworkHandle be provided for iOS in the future? Thanks.
,
May 25 2018
Attaching the logs. Two request were started one after another serially. The device was connected to the cellular network before the first requests was started. While the first request was in progress, the device connected to WiFi.
Experimental options:
@"{\"QUIC\":{\"connection_options\":\"AKD4\", \"close_sessions_on_ip_change\":true}}"];
,
May 25 2018
Attaching another logs. Same scenario but the flag is turned off. Device connects to WiFi while the first request is in progress.
Experimental options:
@"{\"QUIC\":{\"connection_options\":\"AKD4\", \"close_sessions_on_ip_change\":false}}"
,
May 25 2018
Thanks Andrei, this is looking great! I have checked the - NetLog in comment #16 with flag on. There're 2 QUIC_SESSIONs, id:9 was the old session and is closed with QUIC_IP_ADDRESS_CHANGED on ip address change, the second request creates a new QUIC_SESSION(id:17). - NetLog in comment #17 with flag off. There's only one QUIC_SESSION(on cellular), the second request is still served in the old QUIC_SESSION. So turning on the flag will be the right temporary fix, though existing request will fail with NETWORK_CHANGED.
,
May 25 2018
Re #15: "mark session away so that old requests can finish on the old network" will let outstanding requests finish on the old network. New requests will creates new QuicSession, which may introduce an extra RTT as your CHLO in the second session maybe rejected by the server due to "RREJ: SOURCE_ADDRESS_TOKEN_DIFFERENT_IP_ADDRESS_FAILURE". "seamlessly migrate requests between networks without killing the connections" is what we called connection migration not supported on iOS currently. Connection Migration keeps the session alive all the time, and migrating sockets inside the session on network change or connectivity issue. All requests goes to the same session and the session manages to send packets on the right interface.
,
May 30 2018
Summarizing our plans after QUIC meeting: 1. iOS cronet embedders can use "close_sessions_on_ip_change" for the time being to mitigate the issue. Old requests do fail in this case. 2. QUIC dev will implement "go away session on ip change" to match ios platform default behavior, and that will be first experimented on iOS. * If no issue encountered, we may proceed to default goaway session on iOS, and experiment with Android cronet embedders. It may be a good idea to talk to iOS folks to know more details about network change on iOS.
,
Jun 1 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/63574b7fab7e9c51b583ac899c1e91e74ded9fd0 commit 63574b7fab7e9c51b583ac899c1e91e74ded9fd0 Author: Zhongyi Shi <zhongyi@chromium.org> Date: Fri Jun 01 20:22:25 2018 Mark sessions as goaway when IP address changes Mark sessions as goaway when IP address changes is reported to QuicStreamFactory if "goaway_sessions_on_ip_changes" is set. Bug: 846079 Cq-Include-Trybots: master.tryserver.chromium.android:android_cronet_tester;master.tryserver.chromium.mac:ios-simulator-cronet Change-Id: Ifb7971560268b1059debcdb7d5fab6ad9fa85ded Reviewed-on: https://chromium-review.googlesource.com/1080175 Commit-Queue: Zhongyi Shi <zhongyi@chromium.org> Reviewed-by: Andrei Kapishnikov <kapishnikov@chromium.org> Reviewed-by: Ryan Hamilton <rch@chromium.org> Cr-Commit-Position: refs/heads/master@{#563770} [modify] https://crrev.com/63574b7fab7e9c51b583ac899c1e91e74ded9fd0/components/cronet/url_request_context_config.cc [modify] https://crrev.com/63574b7fab7e9c51b583ac899c1e91e74ded9fd0/components/cronet/url_request_context_config_unittest.cc [modify] https://crrev.com/63574b7fab7e9c51b583ac899c1e91e74ded9fd0/components/network_session_configurator/browser/network_session_configurator.cc [modify] https://crrev.com/63574b7fab7e9c51b583ac899c1e91e74ded9fd0/components/network_session_configurator/browser/network_session_configurator_unittest.cc [modify] https://crrev.com/63574b7fab7e9c51b583ac899c1e91e74ded9fd0/net/http/http_network_session.cc [modify] https://crrev.com/63574b7fab7e9c51b583ac899c1e91e74ded9fd0/net/http/http_network_session.h [modify] https://crrev.com/63574b7fab7e9c51b583ac899c1e91e74ded9fd0/net/http/http_proxy_client_socket_wrapper_unittest.cc [modify] https://crrev.com/63574b7fab7e9c51b583ac899c1e91e74ded9fd0/net/quic/chromium/quic_stream_factory.cc [modify] https://crrev.com/63574b7fab7e9c51b583ac899c1e91e74ded9fd0/net/quic/chromium/quic_stream_factory.h [modify] https://crrev.com/63574b7fab7e9c51b583ac899c1e91e74ded9fd0/net/quic/chromium/quic_stream_factory_fuzzer.cc [modify] https://crrev.com/63574b7fab7e9c51b583ac899c1e91e74ded9fd0/net/quic/chromium/quic_stream_factory_test.cc
,
Jun 5 2018
The change referred by Comment 21 was landed in 69.0.3448.0.
,
Jun 15 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/aa518c2f5078396aac57a4a2c2579898115c0053 commit aa518c2f5078396aac57a4a2c2579898115c0053 Author: Zhongyi Shi <zhongyi@chromium.org> Date: Fri Jun 15 04:37:36 2018 Default turning on goaway sessions on ip address change for iOS. Bug: 846079 Cq-Include-Trybots: master.tryserver.chromium.android:android_cronet_tester;master.tryserver.chromium.mac:ios-simulator-cronet Change-Id: I384116a58acb044453a42dbedc1c9f7d23290640 Reviewed-on: https://chromium-review.googlesource.com/1093311 Reviewed-by: Andrei Kapishnikov <kapishnikov@chromium.org> Commit-Queue: Zhongyi Shi <zhongyi@chromium.org> Cr-Commit-Position: refs/heads/master@{#567549} [modify] https://crrev.com/aa518c2f5078396aac57a4a2c2579898115c0053/components/cronet/url_request_context_config.cc [modify] https://crrev.com/aa518c2f5078396aac57a4a2c2579898115c0053/components/cronet/url_request_context_config_unittest.cc
,
Jul 10
The fixed is turned on by default for iOS in cronet versions 69.0.3462.0 and above. jni@snapchat.com: could you help verify the fix in Snapchat and let me know if there's any issue? Thanks!
,
Jul 10
Re #24: Sure. I will do a local test and come back to this thread later. Thanks for the fix.
,
Jul 19
zhongyi@chromium.org I verified that it is fixed. Thanks you for taking care of this.
,
Jul 19
Thanks for the feedback, just to confirm, by saying fixed, did you manage to see the request filed on the old network is able to be completed while new requests will start on the new network?
,
Jul 20
zhongyi@chromium.org Yes I confirm that the new request are using the new interface and the old ones succeeds. Thanks for the fix!
,
Jul 20
Sweet! Thanks for the feedback! Feel free to let us know if you have any other issue. If you are filing chromium bugs directly, please attach the Internals>Network>QUIC label and we should be on in fairly soon. |
||||||||||||||||||
►
Sign in to add a comment |
||||||||||||||||||
Comment 1 by kapishnikov@chromium.org
, May 23 2018