DCHECK failure in the destructor of SpdySession when shutdown CronetEngine
Reported by
jiangyic...@gmail.com,
Nov 30 2017
|
|||
Issue description
Example URL:
Steps to reproduce the problem:
1. Continually starting engine sending requests and shutdown engine
2. Change network type from WiFi to 4G at the same time
3. Occasionally observed a crash for the dcheck
What is the expected behavior?
What went wrong?
The stack of the crash is (in brev):
net::SpdySession::DcheckDraining()
net::SpdySession::~SpdySession()
net::SpdySession::RemoveUnavailableSession()
net::SpdySessionPool::~SpdySessionPool()
net::HttpNetworkSession::~HttpNetworkSession()
....
I read some code, and I think the root cause is as follows:
1. In SpdySessionPool::OnIPAddressChanged, the sessions will be marked as GOAWAY instead of DRAINING because of the Marcos: #if defined(OS_ANDROID) || defined(OS_WIN) || defined(OS_IOS)
2. Next, the engine is stopped, and the SpdySessionPool is destructed
3. Although CloseAllSessions() will be called, previous sessions will not be closed because available_sessions_ is empty
4. RemoveUnavailableSession will destroy all managed SpdySessions
5. The SpdySessions marked as GOAWAY will cause the crash.
I append some log to analysis about SpdySessionPool::OnIPAddressChanged:
// For OSs that terminate TCP connections upon relevant network changes,
// attempt to preserve active streams by marking all sessions as going
// away, rather than explicitly closing them. Streams may still fail due
// to a generated TCP reset.
#if defined(OS_ANDROID) || defined(OS_WIN) || defined(OS_IOS)
(*it)->MakeUnavailable();
(*it)->StartGoingAway(kLastStreamId, ERR_NETWORK_CHANGED);
(*it)->MaybeFinishGoingAway();
#else
(*it)->CloseSessionOnError(ERR_NETWORK_CHANGED,
"Closing current sessions.");
DCHECK((*it)->IsDraining());
#endif // defined(OS_ANDROID) || defined(OS_WIN) || defined(OS_IOS)
DCHECK(!IsSessionAvailable(*it));
}
If there is a request alive when network changing, then:
The comments point that active streams is preserved. SpdySession::StartGoingAway will clean all streams whose Id > (0x7FFFFFFF + 1), which means no active stream will be cleaned. So DoDrainSession() will not be called.
If the request is failed (by application cancel in my case), then the engine is shutdown. The preserved active stream will cause this crash. I think this might be a bug.
Besides, is there anyone can help to explain why the active streams is preserved?
If the connection is changed, what are these GOINGAWAY streams used for?
Did this work before? N/A
Chrome version: <Copy from: 'about:version'> Channel: n/a
OS Version:
Flash Version:
see https://groups.google.com/a/chromium.org/forum/#!mydiscussions/net-dev/-KaRDlLYJ3Q
,
Dec 6 2017
,
Dec 6 2017
It is worth noting that receiving a graceful GOAWAY (with high last_seen_stream_id and NO_ERROR error code) from the server also puts a SpdySession in STATE_GOING_AWAY, and if it is destroyed while in this state, the same DCHECK is triggered.
The culprit is CloseAllSessions():
void SpdySessionPool::CloseAllSessions() {
while (!available_sessions_.empty()) {
CloseCurrentSessionsHelper(ERR_ABORTED, "Closing all sessions.",
false /* idle_only */);
}
}
If |available_sessions_| is empty (and indeed, a STATE_GOING_AWAY session is not available), then CloseCurrentSessionsHelper() will not be called.
CloseAllSessions() is called from both HttpNetworkSession destructor and SpdySessionPool destructor, so it should take care of draining all the sessions before they are destroyed.
,
Dec 11 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/3ba8c339f6d4629543f328685e373d3bd2842525 commit 3ba8c339f6d4629543f328685e373d3bd2842525 Author: Bence Béky <bnc@chromium.org> Date: Mon Dec 11 20:19:24 2017 Avoid DCHECK on SpdySessionPool destruction. It is possible that a DCHECK in SpdySession::DcheckDraining() is triggered when SpdySessionPool destructor call SpdySessionPool::RemoveUnavailableSession() which destroys the SpdySession, if the SpdySession is in STATE_GOING_AWAY. One way for this is that on certain platforms, SpdySessionPool::OnIPAddressChanged() may leave a SpdySession with an active stream in STATE_GOING_AWAY. Another possible way is receiving a "graceful GOAWAY" from the server, which also puts a SpdySession with active streams in STATE_GOING_AWAY. Patch set 1 introduces two unit tests, one for each of the above scenarios, that trigger this DCHECK. See trybot outputs for failing tests. Patch set 2 fixes the bug by ensuring in SpdySessionPool::CloseAllSessions() that every owned SpdySession is draining, even if there are no currently active sessions. This method is not only called by SpdySessionPool destructor, but also by HttpNetworkSession destructor (also adding a TODO for that). Also include build/build_config.h for OS_ANDROID, OS_WIN, OS_IOS macros. Bug: 789791 Change-Id: Idc3e2350b2ac6354be5c5e6b32ea8e0227f3e2d6 Reviewed-on: https://chromium-review.googlesource.com/810889 Reviewed-by: Helen Li <xunjieli@chromium.org> Commit-Queue: Bence Béky <bnc@chromium.org> Cr-Commit-Position: refs/heads/master@{#523186} [modify] https://crrev.com/3ba8c339f6d4629543f328685e373d3bd2842525/net/http/http_network_session.cc [modify] https://crrev.com/3ba8c339f6d4629543f328685e373d3bd2842525/net/spdy/chromium/spdy_session_pool.cc [modify] https://crrev.com/3ba8c339f6d4629543f328685e373d3bd2842525/net/spdy/chromium/spdy_session_pool.h [modify] https://crrev.com/3ba8c339f6d4629543f328685e373d3bd2842525/net/spdy/chromium/spdy_session_pool_unittest.cc
,
Dec 11 2017
|
|||
►
Sign in to add a comment |
|||
Comment 1 by ckrasic@chromium.org
, Nov 30 2017Components: -Internals>Network Internals>Network>HTTP2