New issue
Advanced search Search tips
Note: Color blocks (like █ or █) mean that a user may not be available. Tooltip shows the reason.

Issue 832881 link

Starred by 1 user
Status: Duplicate
Merged: issue 830531
Owner:
briannorris@chromium.org
Closed: Apr 2018
Cc:
derat@chromium.org
█ kirtika@chromium.org
diand...@chromium.org
grundler@chromium.org
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 2
Type: Bug



Sign in to add a comment

powerd: millions of powerd_setuid_helper "crashes" (via set_wifi_transmit_power failure)

Project Member Reported by briannorris@chromium.org, Apr 13 2018 
Similar to  bug 781838 

We have over a million reports of this:

(gdb) bt
#0  __libc_do_syscall () at ../sysdeps/unix/sysv/linux/arm/libc-do-syscall.S:47
#1  0xecf725b2 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54
#2  0xecf7337a in __GI_abort () at abort.c:89
#3  0xed1408c6 in base::debug::BreakDebugger() () from /build/kevin/usr/lib/libbase-core-395517.so
#4  0xed1570f8 in logging::LogMessage::~LogMessage() () from /build/kevin/usr/lib/libbase-core-395517.so
#5  0xed157254 in logging::ErrnoLogMessage::~ErrnoLogMessage() () from /build/kevin/usr/lib/libbase-core-395517.so
#6  0x0d247138 in RunCommand (command=0xd248681 "ectool", arg=0x0)
    at ../../../../../../../../../../mnt/host/source/src/platform2/power_manager/powerd/powerd_setuid_helper.cc:51
#7  0x0d2478a2 in main (argc=<optimized out>, argv=0xff90cf24)
    at ../../../../../../../../../../mnt/host/source/src/platform2/power_manager/powerd/powerd_setuid_helper.cc:109


https://crash.corp.google.com/browse?q=product.name%3D%27ChromeOS%27%20AND%20EXISTS%20(SELECT%201%20FROM%20UNNEST(productdata)%20WHERE%20Key%3D%27exec_name%27%20AND%20Value%3D%27set_wifi_transmit_power%27)%20AND%20expanded_custom_data.ChromeCrashProto.magic_signature_1.name%3D%27base%3A%3Adebug%3A%3ABreakDebugger%27&stbtiq=&reportid=&index=0

I've traces several local instances of this down to the set_wifi_transmit_power tool again, except this time the failures get blamed on the caller (our setuid_helper). This is because set_wifi_transmit_power is returning errors for well-known "failure" cases. e.g. if powerd starts before the driver finishes loading; or if the Wifi driver crashes, then future tablet mode transitions won't have any driver to touch.

It's trivial to see this by, e.g., rmmod mwifiex_pcie; restart powerd or similar.

I've started on
(a) removing the failure exit code from set_wifi_transmit_power
(b) helping direct the logging for this into /var/log/power_manager/powerd.LATEST -- currently, this stuff gets lost in the ether, so we *only* get an indication of failures here by seeing the CHECK() assertions dump out crash reports

Comment 1 by derat@chromium.org, Apr 13 2018 

If that stack trace is representative, this doesn't have anything to do with set_wifi_transmit_power. Rather, it's a dupe of  issue 830531 .

As an aside, powerd runs set_wifi_transmit_power asynchronously. As far as I'm aware, it doesn't care about (or even know) its exit code. If set_wifi_transmit_power exits with a nonzero code (rather than SIGABRT, which I think I've removed for "expected" failures), that shouldn't trigger a crash report. If it logs something, it might show up in powerd.out.

Comment 2 by briannorris@chromium.org, Apr 13 2018

Mergedinto: 830531
Status: Duplicate (was: Started)
Ugh, of course you're right. I did notice the ectool problem, and worked around it by symlinking /usr/local/sbin/ectool to /usr/sbin/ectool. But then I failed to go back and check for additional aborts.

I also didn't notice the sync/async behavior, where 'set_wifi_transmit_power' is indeed run asynchronously (and therefore error codes are essentially ignored).

https://chromium.googlesource.com/chromiumos/platform2/+/8d6618eab7298d82cba758bc4a7d12045b4ef06e/power_manager/powerd/daemon.cc#770
https://chromium.googlesource.com/chromiumos/platform2/+/8d6618eab7298d82cba758bc4a7d12045b4ef06e/power_manager/powerd/daemon.cc#461

And lastly, I didn't notice that stderr and stdout were redirected to powerd.out, so we *do* actually get proper indications of what went wrong (if you know where to look). Once again, this was actually documented:

https://chromium.googlesource.com/chromiumos/platform2/+/master/power_manager/docs/logging.md

So I guess I don't need to do anything more here, other than account for  bug 830531 .

Sigh. Thanks.
►

Sign in to add a comment