New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 619171 link

Starred by 2 users

Issue metadata

Status: Verified
Owner:
Last visit > 30 days ago
Closed: Aug 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 1
Type: Bug

Blocking:
issue 589753



Sign in to add a comment

logging_UserCrash_SERVER_JOB due to minidump_stackwalk command not found

Project Member Reported by jen...@chromium.org, Jun 10 2016

Issue description

Seen this error on:https://chromegw.corp.google.com/i/chromeos/builders/lumpy-chrome-pfq/builds/8743

The error log shows: 

06/10 12:06:54.488 DEBUG|        base_utils:0185| Running 'minidump_stackwalk "/usr/local/autotest/results/66371233-chromeos-test/chromeos2-row3-rack10-host5/logging_UserCrash/sysinfo/iteration.1/var/spool/crash/crasher_nobreakpad.20160610.120644.24871.dmp" "/usr/local/autotest/server/../../../lib/debug" > "/usr/local/autotest/results/66371233-chromeos-test/chromeos2-row3-rack10-host5/logging_UserCrash/sysinfo/iteration.1/var/spool/crash/crasher_nobreakpad.20160610.120644.24871.dmp.txt"'
06/10 12:06:54.508 INFO | site_crashcollect:0089| Failed to generate stack trace locally for dump /usr/local/autotest/results/66371233-chromeos-test/chromeos2-row3-rack10-host5/logging_UserCrash/sysinfo/iteration.1/var/spool/crash/crasher_nobreakpad.20160610.120644.24871.dmp (rc=127):
CmdError('minidump_stackwalk "/usr/local/autotest/results/66371233-chromeos-test/chromeos2-row3-rack10-host5/logging_UserCrash/sysinfo/iteration.1/var/spool/crash/crasher_nobreakpad.20160610.120644.24871.dmp" "/usr/local/autotest/server/../../../lib/debug" > "/usr/local/autotest/results/66371233-chromeos-test/chromeos2-row3-rack10-host5/logging_UserCrash/sysinfo/iteration.1/var/spool/crash/crasher_nobreakpad.20160610.120644.24871.dmp.txt"', * Command: 
    minidump_stackwalk "/usr/local/autotest/results/66371233-chromeos-test/ch
    romeos2-row3-rack10-host5/logging_UserCrash/sysinfo/iteration.1/var/spool/
    crash/crasher_nobreakpad.20160610.120644.24871.dmp"
    "/usr/local/autotest/server/../../../lib/debug" >
    "/usr/local/autotest/results/66371233-chromeos-test/chromeos2-row3-rack10-
    host5/logging_UserCrash/sysinfo/iteration.1/var/spool/crash/crasher_nobrea
    kpad.20160610.120644.24871.dmp.txt"
Exit status: 127
Duration: 0.00318384170532

stderr:
/bin/bash: minidump_stackwalk: command not found, 'Command returned non-zero exit status')

 

Comment 1 by jen...@chromium.org, Jun 13 2016

Labels: Hotlist-CrOS-Gardener
Cc: ihf@chromium.org
Labels: -Pri-3 Pri-1
Prioritizing due to the 300+ failures since April in issue 589753.
Note ihf's comment that the minidump_stackwalk error is also printed in a successful run: https://bugs.chromium.org/p/chromium/issues/detail?id=589753#c343
Owner: victorhsieh@chromium.org
Status: Started (was: Untriaged)
I'm looking at the latest error[1].  The autotest client refers the executable by /usr/bin/minidump_stackwalk, and the server refers by simply minidump_stackwalk.  I wonder if /usr/bin is not in PATH, or the executable is not installed to /usr/bin.  But how can I check?

[1] https://pantheon.corp.google.com/storage/browser/chromeos-autotest-results/72546702-chromeos-test/chromeos4-row2-rack10-host9/debug/
Failures in minidump_stackwalk have been going on since roughly
the dawn of time (or at least, since the dinosaurs...)  Generally,
this error is a nuisance.  It's never been known to be the cause
of an actual failure.

How do we know that that particular error is causing failures?
Do we actually know that?


I looked at a test, and the result is passed.  Since the test actually generate a crash on the client side, and probably not intentionally, the dump is copied back to the server, I start to wonder if the issue is automatically captured just because of that.
More positive signal to my theory.  The only crash dump that the server tried to call with minidump_stackwalk is crasher_nobreakpad.20160809.063718.24009.*.  Looked at the client side[2], logging_UserCrash generated many crashes as expected.  But from the log, the client always `rm -rf /var/spool/crash` before running a test, but not after.  That leaves the last crash dump in the directory, and rsync fetched them back to the server.  The "auto bug opener" probably then filed those issues since the server says the client has crash dump.

I'll try to make the client clean up more clearly.
CL is here: https://chromium-review.googlesource.com/#/c/367540/

Who would be able to help review the change?  I isolate the change to logging_UserCrash just to avoid touch the other logging_*Crash tests.  Otherwise, I could rename cleanup to after_run_once in the parent test class, crash_test.  But I don't know if it's ok.
Project Member

Comment 9 by bugdroid1@chromium.org, Aug 23 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/dcaa00f750ece133b603a6606848ff54729e8189

commit dcaa00f750ece133b603a6606848ff54729e8189
Author: Victor Hsieh <victorhsieh@chromium.org>
Date: Tue Aug 09 23:06:22 2016

Stop collecting intentional crash by logging_*

TEST=test_that logging_UserCrash  # No longer see INFO New Crash Dump
BUG= chromium:619171 

Change-Id: I8758a0fc32c277d6255da7a3b5df5406e8d1ffbe
Reviewed-on: https://chromium-review.googlesource.com/367540
Commit-Ready: Victor Hsieh <victorhsieh@chromium.org>
Tested-by: Victor Hsieh <victorhsieh@chromium.org>
Reviewed-by: Achuith Bhandarkar <achuith@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>

[modify] https://crrev.com/dcaa00f750ece133b603a6606848ff54729e8189/client/cros/crash_test.py

Status: Fixed (was: Started)
Labels: VerifyIn-54
Labels: VerifyIn-55

Comment 13 by dchan@chromium.org, Oct 10 2016

Labels: -VerifyIn-55

Comment 14 by dchan@google.com, Nov 19 2016

Labels: VerifyIn-56

Comment 15 by dchan@google.com, Jan 21 2017

Labels: VerifyIn-57

Comment 16 by dchan@google.com, Mar 4 2017

Labels: VerifyIn-58

Comment 17 by dchan@google.com, Apr 17 2017

Labels: VerifyIn-59

Comment 18 by dchan@google.com, May 30 2017

Labels: VerifyIn-60
Labels: VerifyIn-61
Status: Verified (was: Fixed)
Closing. Please reopen it if its not fixed. Thanks!

Sign in to add a comment