New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 610000 link

Starred by 2 users

Issue metadata

Status: Verified
Owner:
Last visit > 30 days ago
Closed: Jun 2016
Cc:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 1
Type: Bug



Sign in to add a comment

logging_CrashSender HWTest is failing on multiple paladins

Project Member Reported by rspangler@chromium.org, May 6 2016

Issue description

CQ failed on multiple paladins HWTest with two types of failure
[bvt-inline] - logging_CrashSender: retry_count: 2, FAIL: Simple minidump send failed
[bvt-cq] - logging_UserCrash: FAIL: Did not find version 8288.0.0-rc2 in log output

Sample build for first failure:
https://uberchromegw.corp.google.com/i/chromeos/builders/veyron_mighty-paladin/builds/1904

Logfile output:

05/06 14:29:50.108 DEBUG|        base_utils:0176| Running 'logger "autotest finished iteration /usr/local/autotest/results/default/logging_CrashSender/sysinfo/iteration.1"'
05/06 14:29:50.121 WARNI|              test:0606| Autotest caught exception when running test:
Traceback (most recent call last):
  File "/usr/local/autotest/common_lib/test.py", line 600, in _exec
    _call_test_function(self.execute, *p_args, **p_dargs)
  File "/usr/local/autotest/common_lib/test.py", line 804, in _call_test_function
    return func(*args, **dargs)
  File "/usr/local/autotest/common_lib/test.py", line 461, in execute
    dargs)
  File "/usr/local/autotest/common_lib/test.py", line 347, in _call_run_once_with_retry
    postprocess_profiled_run, args, dargs)
  File "/usr/local/autotest/common_lib/test.py", line 376, in _call_run_once
    self.run_once(*args, **dargs)
  File "/usr/local/autotest/tests/logging_CrashSender/logging_CrashSender.py", line 320, in run_once
    'sender_error_type']);
  File "/usr/local/autotest/cros/crash_test.py", line 696, in run_crash_tests
    getattr(self, '_test_' + test_name)()
  File "/usr/local/autotest/tests/logging_CrashSender/logging_CrashSender.py", line 72, in _test_sender_simple_minidump
    self._check_simple_minidump_send(None)
  File "/usr/local/autotest/tests/logging_CrashSender/logging_CrashSender.py", line 53, in _check_simple_minidump_send
    raise error.TestFail('Simple minidump send failed')
TestFail: Simple minidump send failed

Sample build for second failure:
https://uberchromegw.corp.google.com/i/chromeos/builders/peppy-paladin/builds/11767

Log output:
05/06 14:29:59.343 DEBUG|        base_utils:0176| Running 'logger "autotest finished iteration /usr/local/autotest/results/default/logging_UserCrash/sysinfo/iteration.1"'
05/06 14:29:59.352 WARNI|              test:0606| Autotest caught exception when running test:
Traceback (most recent call last):
  File "/usr/local/autotest/common_lib/test.py", line 600, in _exec
    _call_test_function(self.execute, *p_args, **p_dargs)
  File "/usr/local/autotest/common_lib/test.py", line 804, in _call_test_function
    return func(*args, **dargs)
  File "/usr/local/autotest/common_lib/test.py", line 461, in execute
    dargs)
  File "/usr/local/autotest/common_lib/test.py", line 347, in _call_run_once_with_retry
    postprocess_profiled_run, args, dargs)
  File "/usr/local/autotest/common_lib/test.py", line 376, in _call_run_once
    self.run_once(*args, **dargs)
  File "/usr/local/autotest/tests/logging_UserCrash/logging_UserCrash.py", line 710, in run_once
    initialize_crash_reporter=True)
  File "/usr/local/autotest/cros/crash_test.py", line 696, in run_crash_tests
    getattr(self, '_test_' + test_name)()
  File "/usr/local/autotest/tests/logging_UserCrash/logging_UserCrash.py", line 445, in _test_chronos_crasher
    self._check_crashing_process('chronos')
  File "/usr/local/autotest/tests/logging_UserCrash/logging_UserCrash.py", line 430, in _check_crashing_process
    'minidump')
  File "/usr/local/autotest/tests/logging_UserCrash/logging_UserCrash.py", line 319, in _check_generated_report_sending
    version_match.group(1))
TestFail: Did not find version 8288.0.0-rc2 in log output
 
Cc: kcwu@chromium.org jen...@chromium.org mruthven@chromium.org
Seems like a flaky issue, veyron-mighty-paladin passed in the next build.
https://uberchromegw.corp.google.com/i/chromeos/builders/veyron_mighty-paladin/builds/1905
Labels: -Pri-3 Pri-1
Some error happens on several builds today.
https://uberchromegw.corp.google.com/i/chromeos/builders/daisy_skate-paladin/builds/6543
Cc: vapier@chromium.org
cc'd Mike. He seems to know about this logging_CrashSender test.
It looks like both of the errors may be caused by an issue recognizing the version string in the result output

https://chromium.googlesource.com/chromiumos/third_party/autotest/+/stabilize-link-2913.278/client/site_tests/logging_CrashSender/logging_CrashSender.py#48

https://chromium.googlesource.com/chromiumos/third_party/autotest/+/0.12.362.B/client/site_tests/logging_UserCrash/logging_UserCrash.py#304

From the batch of cls possibly to blame in the initial failure, this one messes around with the version. I am not sure if this is the correct version.
https://chromium-review.googlesource.com/#/c/342574/1/build_library/base_image_util.sh
Owner: andreyu@google.com

Comment 7 by andreyu@google.com, May 12 2016

Status: Fixed (was: Untriaged)

Comment 8 by andreyu@google.com, May 12 2016

Status: Assigned (was: Fixed)

Comment 9 by andreyu@google.com, Jun 15 2016

Status: Fixed (was: Assigned)
Project Member

Comment 10 by bugdroid1@chromium.org, Jun 25 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform2/+/aebaf73804bf4cb8006198e9bc4559d3a4c5c2df

commit aebaf73804bf4cb8006198e9bc4559d3a4c5c2df
Author: Andrey Ulanov <andreyu@google.com>
Date: Mon May 09 23:52:40 2016

crash: use BUILD_ID in /etc/os-release to find version number

To determine build version crash_sender looks for (1) GOOGLE_CRASH_VERSION_ID
or (2) VERSION_ID in /etc/os-release or (3) CHROMEOS_RELEASE_VERSION in
/etc/lsb-release (in that order). It expects that version to be full
build identifier.
CL:342574 adds VERSION_ID and a few other fields in os-release file. It sets
VERSION_ID to chrome milestone number, and uses BUILD_ID for the whole version
string.

This change makes crash_sender prefer BUILD_ID instead of VERSION_ID. It also
rewrites the corresponding piece of code to make it more compact and readable.

BUG= chromium:610000 
BUG=b:27841076
TEST=logging_CrashSender

Change-Id: I108c64e9f7486a04a46c418aa599f0e894482b30
Reviewed-on: https://chromium-review.googlesource.com/343549
Commit-Ready: Andrey Ulanov <andreyu@google.com>
Tested-by: Andrey Ulanov <andreyu@google.com>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/aebaf73804bf4cb8006198e9bc4559d3a4c5c2df/crash-reporter/crash_sender

Labels: VerifyIn-53
Labels: VerifyIn-54
Labels: VerifyIn-55

Comment 14 by dchan@google.com, Nov 19 2016

Labels: VerifyIn-56

Comment 15 by dchan@google.com, Jan 21 2017

Labels: VerifyIn-57

Comment 16 by dchan@google.com, Mar 4 2017

Labels: VerifyIn-58

Comment 17 by dchan@google.com, Apr 17 2017

Labels: VerifyIn-59

Comment 18 by dchan@google.com, May 30 2017

Labels: VerifyIn-60

Sign in to add a comment