logging_CrashSender HWTest is failing on multiple paladins |
||||||||||||||||
Issue descriptionCQ failed on multiple paladins HWTest with two types of failure [bvt-inline] - logging_CrashSender: retry_count: 2, FAIL: Simple minidump send failed [bvt-cq] - logging_UserCrash: FAIL: Did not find version 8288.0.0-rc2 in log output Sample build for first failure: https://uberchromegw.corp.google.com/i/chromeos/builders/veyron_mighty-paladin/builds/1904 Logfile output: 05/06 14:29:50.108 DEBUG| base_utils:0176| Running 'logger "autotest finished iteration /usr/local/autotest/results/default/logging_CrashSender/sysinfo/iteration.1"' 05/06 14:29:50.121 WARNI| test:0606| Autotest caught exception when running test: Traceback (most recent call last): File "/usr/local/autotest/common_lib/test.py", line 600, in _exec _call_test_function(self.execute, *p_args, **p_dargs) File "/usr/local/autotest/common_lib/test.py", line 804, in _call_test_function return func(*args, **dargs) File "/usr/local/autotest/common_lib/test.py", line 461, in execute dargs) File "/usr/local/autotest/common_lib/test.py", line 347, in _call_run_once_with_retry postprocess_profiled_run, args, dargs) File "/usr/local/autotest/common_lib/test.py", line 376, in _call_run_once self.run_once(*args, **dargs) File "/usr/local/autotest/tests/logging_CrashSender/logging_CrashSender.py", line 320, in run_once 'sender_error_type']); File "/usr/local/autotest/cros/crash_test.py", line 696, in run_crash_tests getattr(self, '_test_' + test_name)() File "/usr/local/autotest/tests/logging_CrashSender/logging_CrashSender.py", line 72, in _test_sender_simple_minidump self._check_simple_minidump_send(None) File "/usr/local/autotest/tests/logging_CrashSender/logging_CrashSender.py", line 53, in _check_simple_minidump_send raise error.TestFail('Simple minidump send failed') TestFail: Simple minidump send failed Sample build for second failure: https://uberchromegw.corp.google.com/i/chromeos/builders/peppy-paladin/builds/11767 Log output: 05/06 14:29:59.343 DEBUG| base_utils:0176| Running 'logger "autotest finished iteration /usr/local/autotest/results/default/logging_UserCrash/sysinfo/iteration.1"' 05/06 14:29:59.352 WARNI| test:0606| Autotest caught exception when running test: Traceback (most recent call last): File "/usr/local/autotest/common_lib/test.py", line 600, in _exec _call_test_function(self.execute, *p_args, **p_dargs) File "/usr/local/autotest/common_lib/test.py", line 804, in _call_test_function return func(*args, **dargs) File "/usr/local/autotest/common_lib/test.py", line 461, in execute dargs) File "/usr/local/autotest/common_lib/test.py", line 347, in _call_run_once_with_retry postprocess_profiled_run, args, dargs) File "/usr/local/autotest/common_lib/test.py", line 376, in _call_run_once self.run_once(*args, **dargs) File "/usr/local/autotest/tests/logging_UserCrash/logging_UserCrash.py", line 710, in run_once initialize_crash_reporter=True) File "/usr/local/autotest/cros/crash_test.py", line 696, in run_crash_tests getattr(self, '_test_' + test_name)() File "/usr/local/autotest/tests/logging_UserCrash/logging_UserCrash.py", line 445, in _test_chronos_crasher self._check_crashing_process('chronos') File "/usr/local/autotest/tests/logging_UserCrash/logging_UserCrash.py", line 430, in _check_crashing_process 'minidump') File "/usr/local/autotest/tests/logging_UserCrash/logging_UserCrash.py", line 319, in _check_generated_report_sending version_match.group(1)) TestFail: Did not find version 8288.0.0-rc2 in log output
,
May 6 2016
Seems like a flaky issue, veyron-mighty-paladin passed in the next build. https://uberchromegw.corp.google.com/i/chromeos/builders/veyron_mighty-paladin/builds/1905
,
May 9 2016
Some error happens on several builds today. https://uberchromegw.corp.google.com/i/chromeos/builders/daisy_skate-paladin/builds/6543
,
May 9 2016
cc'd Mike. He seems to know about this logging_CrashSender test.
,
May 9 2016
It looks like both of the errors may be caused by an issue recognizing the version string in the result output https://chromium.googlesource.com/chromiumos/third_party/autotest/+/stabilize-link-2913.278/client/site_tests/logging_CrashSender/logging_CrashSender.py#48 https://chromium.googlesource.com/chromiumos/third_party/autotest/+/0.12.362.B/client/site_tests/logging_UserCrash/logging_UserCrash.py#304 From the batch of cls possibly to blame in the initial failure, this one messes around with the version. I am not sure if this is the correct version. https://chromium-review.googlesource.com/#/c/342574/1/build_library/base_image_util.sh
,
May 9 2016
,
May 12 2016
,
May 12 2016
,
Jun 15 2016
,
Jun 25 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/platform2/+/aebaf73804bf4cb8006198e9bc4559d3a4c5c2df commit aebaf73804bf4cb8006198e9bc4559d3a4c5c2df Author: Andrey Ulanov <andreyu@google.com> Date: Mon May 09 23:52:40 2016 crash: use BUILD_ID in /etc/os-release to find version number To determine build version crash_sender looks for (1) GOOGLE_CRASH_VERSION_ID or (2) VERSION_ID in /etc/os-release or (3) CHROMEOS_RELEASE_VERSION in /etc/lsb-release (in that order). It expects that version to be full build identifier. CL:342574 adds VERSION_ID and a few other fields in os-release file. It sets VERSION_ID to chrome milestone number, and uses BUILD_ID for the whole version string. This change makes crash_sender prefer BUILD_ID instead of VERSION_ID. It also rewrites the corresponding piece of code to make it more compact and readable. BUG= chromium:610000 BUG=b:27841076 TEST=logging_CrashSender Change-Id: I108c64e9f7486a04a46c418aa599f0e894482b30 Reviewed-on: https://chromium-review.googlesource.com/343549 Commit-Ready: Andrey Ulanov <andreyu@google.com> Tested-by: Andrey Ulanov <andreyu@google.com> Reviewed-by: Mike Frysinger <vapier@chromium.org> [modify] https://crrev.com/aebaf73804bf4cb8006198e9bc4559d3a4c5c2df/crash-reporter/crash_sender
,
Jul 1 2016
,
Aug 29 2016
,
Oct 7 2016
,
Nov 19 2016
,
Jan 21 2017
,
Mar 4 2017
,
Apr 17 2017
,
May 30 2017
,
May 31 2017
tests passed: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=logging_CrashSender&builds=R60-9591.0.0 |
||||||||||||||||
►
Sign in to add a comment |
||||||||||||||||
Comment 1 by rspangler@chromium.org
, May 6 2016