New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 699303 link

Starred by 0 users

Issue metadata

Status: Verified
Owner:
Closed: Mar 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 2
Type: Bug



Sign in to add a comment

shill assertion (crash) in EthernetService: "Check failed: ethernet_."

Project Member Reported by briannorris@chromium.org, Mar 7 2017

Issue description

Forked from b/35648179

---

Seen during suspend/resume stress testing. e.g.:

test_that --board=kevin $IP power_SuspendStress.4hours

with USB ethernet attached.

Using a self-built 9325.0.2017_03_06_1016 from a few days ago.

---

/var/spool/crash fills up with a bunch of crash dumps. Examining stack trace shows it's a CHECK() failure -> LOG(LOG_FATAL):

#0  __libc_do_syscall () at ../sysdeps/unix/sysv/linux/arm/libc-do-syscall.S:47
#1  0xf1815cd4 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54
#2  0xf1816c36 in __GI_abort () at abort.c:89
#3  0xf1bb2d3e in base::debug::BreakDebugger() () from /build/kevin/usr/lib/libbase-core-395517.so
#4  0xf1bc8f3a in logging::LogMessage::~LogMessage() () from /build/kevin/usr/lib/libbase-core-395517.so
#5  0xb75a3cb6 in ?? ()


From net.log (a sample):

2017-03-07T10:54:14.361621-08:00 INFO shill[14238]: [INFO:service.cc(318)] Suppressed autoconnect to service 382 (no carrier)
2017-03-07T10:54:14.444894-08:00 INFO shill[14238]: [INFO:chromeos_power_manager_proxy.cc(274)] SuspendImminent
2017-03-07T10:54:14.444924-08:00 INFO shill[14238]: [INFO:power_manager.cc(112)] OnSuspendImminent(93847746)
2017-03-07T10:54:14.444965-08:00 INFO shill[14238]: [INFO:result_aggregator.cc(53)] Error type org.chromium.flimflam.Error.Success: Success (no error) reported
2017-03-07T10:54:14.445477-08:00 INFO shill[14238]: message repeated 34 times: [ [INFO:result_aggregator.cc(53)] Error type org.chromium.flimflam.Error.Success: Success (no error) reported]
2017-03-07T10:54:14.445505-08:00 INFO shill[14238]: [INFO:wifi.cc(1835)] OnBeforeSuspend: not connected
2017-03-07T10:54:14.445971-08:00 INFO shill[14238]: [INFO:result_aggregator.cc(53)] Error type org.chromium.flimflam.Error.Success: Success (no error) reported
2017-03-07T10:54:14.445995-08:00 INFO shill[14238]: [INFO:result_aggregator.cc(53)] Error type org.chromium.flimflam.Error.Success: Success (no error) reported
2017-03-07T10:54:14.446014-08:00 INFO shill[14238]: [INFO:manager.cc(1821)] Finished suspend actions. Result: org.chromium.flimflam.Error.Success: Success (no error)
2017-03-07T10:54:14.446155-08:00 INFO shill[14238]: [INFO:chromeos_power_manager_proxy.cc(247)] ReportSuspendReadinessInternal(93847555, 93847746, dark=false)
2017-03-07T10:54:14.704926-08:00 INFO shill[14238]: [INFO:manager.cc(2036)] Auto-connect suppressed -- system is suspending.
2017-03-07T10:54:15.560007-08:00 INFO ModemManager[2037]: <info>  Couldn't check support for device '/sys/devices/platform/usb@fe800000/fe800000.dwc3/xhci-hcd.2.auto/usb5/5-1': not supported by any plugin
2017-03-07T10:54:19.251121-08:00 ERR shill[14238]: [ERROR:device.cc(1274)] IP flag write failed: 1 to /proc/sys/net/ipv6/conf/eth0/disable_ipv6
2017-03-07T10:54:19.263547-08:00 INFO shill[14238]: [INFO:manager.cc(2036)] Auto-connect suppressed -- system is suspending.
2017-03-07T10:54:19.263884-08:00 WARNING shill[14238]: [WARNING:device.cc(606)] No default route for global IPv6 address 2620:0:1000:1305:200:ff:fe00:79e
2017-03-07T10:54:19.266541-08:00 INFO shill[14238]: [INFO:manager.cc(1516)] Device eth0 updated: enabled
2017-03-07T10:54:19.266836-08:00 INFO ModemManager[2037]: <info>  (net/eth0): released by device '/sys/devices/platform/usb@fe800000/fe800000.dwc3/xhci-hcd.2.auto/usb5/5-1'
2017-03-07T10:54:19.410700-08:00 ERR shill[14238]: [ERROR:device.cc(1274)] IP flag write failed: 1 to /proc/sys/net/ipv6/conf/eth0/disable_ipv6
2017-03-07T10:54:19.410923-08:00 ERR shill[14238]: [ERROR:device.cc(1274)] IP flag write failed: 1 to /proc/sys/net/ipv6/conf/eth0/disable_ipv6
2017-03-07T10:54:19.411429-08:00 INFO shill[14238]: [INFO:device.cc(245)] Device destructed: eth0 index 196
2017-03-07T10:54:19.411918-08:00 CRIT shill[14238]: [FATAL:ethernet_service.cc(128)] Check failed: ethernet_. #012
2017-03-07T10:54:19.411995-08:00 ERR shill[14247]: [FATAL:ethernet_service.cc(128)] Check failed: ethernet_.

 
Owner: benchan@chromium.org
Status: Started (was: Untriaged)
I believe the issue is that Manager still holds a refptr to EthernetService after Ethernet is destroyed. The EtherentService refptr is likely held by Manager::last_default_physical_service_ as the refptr is removed from Manager::services_ during DeregisterService.
Project Member

Comment 2 by bugdroid1@chromium.org, Mar 14 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/aosp/platform/system/connectivity/shill/+/63caa3edd038688361a118cc31f14f9d0c90dda8

commit 63caa3edd038688361a118cc31f14f9d0c90dda8
Author: Ben Chan <benchan@chromium.org>
Date: Tue Mar 14 05:39:59 2017

shill: remove manager references to service being deregistered

When Manager deregisters a Service object, Manager may still hold a
reference to the Service object via
|Manager::last_default_physical_service_|, which defers the destruction
of the Service object until the scheduled SortServicesTask is executed
at a later point. By then, the Device object associated with the Service
object is already destroyed, which violates the assumption that the
Device object typically outlives the Service object.

As a Service object doesn't need to live beyond the time it gets
deregistered, this CL makes sure that Manager doesn't hold any more
reference to a Service object being deregistered.

BUG= chromium:699303 
TEST=Run unit tests.
TEST=Manually unplug a USB Ethernet adapter and observe from the shill
log that the EthernetService object is destroyed before the
SortServicesTask is scheduled.

Change-Id: I74d9fd11ba178e28c9deb091d35c75af4a412ee9
Reviewed-on: https://chromium-review.googlesource.com/452791
Commit-Ready: Ben Chan <benchan@chromium.org>
Tested-by: Ben Chan <benchan@chromium.org>
Reviewed-by: Kevin Cernekee <cernekee@chromium.org>

[modify] https://crrev.com/63caa3edd038688361a118cc31f14f9d0c90dda8/manager.cc

Status: Fixed (was: Started)
Status: Verified (was: Fixed)
Reran the original repro steps on a self-built image + the above shill change. No shill crashes, and the test passes fine. I'd consider that verified. Thanks!

Sign in to add a comment