ValueError: invalid literal for int() with base 10: '' |
|||||
Issue description
Seeing this error sporadically in various autotest logs. It adds confusion and spam. Seems like an easy fix.
Dan, related to anything you've touched lately?
09/21 09:40:39.691 INFO | crashcollect:0252| chromeos4-row6-rack12-host3 already up, collecting crash info
09/21 09:40:39.692 INFO | crashcollect:0103| Collecting /var/tmp/messages.autotest_start...
09/21 09:40:39.699 DEBUG| ssh_host:0296| Running (ssh) 'ls -ld /var/tmp/messages.autotest_start | cut -d" " -f5' from 'collect_messages|collect_log_file|_get_file_stats|run|wrapper|run_very_slowly'
09/21 09:40:39.868 ERROR| utils:0280| [stderr] ls: cannot access '/var/tmp/messages.autotest_start': No such file or directory
09/21 09:40:39.871 ERROR| crashcollect:0122| Non-critical failure: collection of /var/tmp/messages.autotest_start failed: invalid literal for int() with base 10: ''
Traceback (most recent call last):
File "/usr/local/autotest/server/crashcollect.py", line 109, in collect_log_file
file_stats = _get_file_stats(host, log_path)
File "/usr/local/autotest/server/crashcollect.py", line 160, in _get_file_stats
file_size = int(host.run(cmd).stdout)
ValueError: invalid literal for int() with base 10: ''
https://storage.cloud.google.com/chromeos-autotest-results/143816328-chromeos-test/chromeos4-row6-rack12-host3/debug/autoserv.DEBUG?_ga=2.190583723.-1235484419.1474328066
,
Sep 22 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/73980516cf0c21e848981b7fc308f0087a3a6a17 commit 73980516cf0c21e848981b7fc308f0087a3a6a17 Author: Dan Shi <dshi@google.com> Date: Fri Sep 22 09:02:03 2017 [autotest] Handle ValueError when getting file size For some special file, the command returns empty string for the file size. Default that to zero. BUG= chromium:767602 TEST=None Change-Id: Ibbb3ab26249df78a1354f1b043f87c4ff1784ac7 Reviewed-on: https://chromium-review.googlesource.com/677140 Commit-Ready: Dan Shi <dshi@google.com> Tested-by: Dan Shi <dshi@google.com> Reviewed-by: Allen Li <ayatane@chromium.org> [modify] https://crrev.com/73980516cf0c21e848981b7fc308f0087a3a6a17/server/crashcollect.py
,
Oct 2 2017
Fixing the spam from #2 seems to have just passed the buck... If the file doesn't exist, the "ls " in _get_file_stats() generates error "No such file or directory", and returns ''. We aren't checking for an error here, but instead just trying to cast '' to int. This is what was generating the "ValueError" exception previously. With CL:677140, this exception is now being handled, with a message and _get_file_stats() happily returns _FileStats(0, 1.0). Thus, collect_log_file() can now continue (previously it would except out on the unhandled ValueError), and it tries to collect the non-existent file. This in turn excepts out a bit later, like this: [from https://storage.cloud.google.com/chromeos-autotest-results/146469517-chromeos-test/chromeos4-row9-rack10-host17/debug/autoserv.DEBUG?_ga=2.124780748.-734044362.1501703718] 10/02 10:59:43.774 INFO | crashcollect:0103| Collecting /sys/kernel/debug/dri/0/i915_error_state... 10/02 10:59:43.795 DEBUG| ssh_host:0296| Running (ssh) 'ls -ld /sys/kernel/debug/dri/0/i915_error_state | cut -d" " -f5' from 'get_crashinfo|collect_log_file|_get_file_stats|run|wrapper|run_very_slowly' 10/02 10:59:44.214 ERROR| utils:0280| [stderr] ls: cannot access '/sys/kernel/debug/dri/0/i915_error_state': No such file or directory 10/02 10:59:44.217 WARNI| crashcollect:0163| Getting size of file '/sys/kernel/debug/dri/0/i915_error_state' on host <remote host: chromeos4-row9-rack10-host17> failed: invalid literal for int() with base 10: '' 10/02 10:59:44.234 DEBUG| ssh_host:0296| Running (ssh) 'mktemp -d' from 'collect_log_file|_collect_log_file_with_tmpdir|__enter__|run|wrapper|run_very_slowly' 10/02 10:59:44.724 DEBUG| ssh_host:0296| Running (ssh) 'cp -rp /sys/kernel/debug/dri/0/i915_error_state /tmp/tmp.GCUhLnjnM7' from 'get_crashinfo|collect_log_file|_collect_log_file_with_tmpdir|run|wrapper|run_very_slowly' 10/02 10:59:45.145 ERROR| utils:0280| [stderr] cp: cannot stat '/sys/kernel/debug/dri/0/i915_error_state': No such file or directory 10/02 10:59:45.163 DEBUG| ssh_host:0296| Running (ssh) 'rm -rf /tmp/tmp.GCUhLnjnM7' from 'collect_log_file|_collect_log_file_with_tmpdir|__exit__|run|wrapper|run_very_slowly' 10/02 10:59:45.564 ERROR| crashcollect:0122| Non-critical failure: collection of /sys/kernel/debug/dri/0/i915_error_state failed: command execution error * Command: /usr/bin/ssh -a -x -o ControlPath=/tmp/_autotmp_W2Umcfssh- master/socket -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o BatchMode=yes -o ConnectTimeout=30 -o ServerAliveInterval=900 -o ServerAliveCountMax=3 -o ConnectionAttempts=4 -o Protocol=2 -l root -p 22 chromeos4-row9-rack10-host17 "export LIBC_FATAL_STDERR_=1; if type \"logger\" > /dev/null 2>&1; then logger -tag \"autotest\" \"server[stack::_collect_log_file_with_tmpdir|run|wrapper] -> ssh_run(cp -rp /sys/kernel/debug/dri/0/i915_error_state /tmp/tmp.GCUhLnjnM7)\";fi; cp -rp /sys/kernel/debug/dri/0/i915_error_state /tmp/tmp.GCUhLnjnM7" Exit status: 1 Duration: 0.344256877899 stderr: cp: cannot stat '/sys/kernel/debug/dri/0/i915_error_state': No such file or directory Traceback (most recent call last): File "/usr/local/autotest/server/crashcollect.py", line 117, in collect_log_file _collect_log_file_with_tmpdir(host, log_path, dest_path) File "/usr/local/autotest/server/crashcollect.py", line 145, in _collect_log_file_with_tmpdir host.run('cp -rp %s %s' % (pipes.quote(log_path), pipes.quote(tmpdir))) File "/usr/local/autotest/server/hosts/ssh_host.py", line 316, in run return self.run_very_slowly(*args, **kwargs) File "/usr/local/autotest/site-packages/chromite/lib/metrics.py", line 482, in wrapper return fn(*args, **kwargs) File "/usr/local/autotest/server/hosts/ssh_host.py", line 307, in run_very_slowly ssh_failure_retry_ok) File "/usr/local/autotest/server/hosts/ssh_host.py", line 258, in _run raise error.AutoservRunError("command execution error", result) AutoservRunError: command execution error * Command: /usr/bin/ssh -a -x -o ControlPath=/tmp/_autotmp_W2Umcfssh- master/socket -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o BatchMode=yes -o ConnectTimeout=30 -o ServerAliveInterval=900 -o ServerAliveCountMax=3 -o ConnectionAttempts=4 -o Protocol=2 -l root -p 22 chromeos4-row9-rack10-host17 "export LIBC_FATAL_STDERR_=1; if type \"logger\" > /dev/null 2>&1; then logger -tag \"autotest\" \"server[stack::_collect_log_file_with_tmpdir|run|wrapper] -> ssh_run(cp -rp /sys/kernel/debug/dri/0/i915_error_state /tmp/tmp.GCUhLnjnM7)\";fi; cp -rp /sys/kernel/debug/dri/0/i915_error_state /tmp/tmp.GCUhLnjnM7" Exit status: 1 Duration: 0.344256877899 stderr: cp: cannot stat '/sys/kernel/debug/dri/0/i915_error_state': No such file or directory
,
Oct 2 2017
That's kind of expected. The crash collection has a list of files to collect. Instead of checking if file exists or not, it forces to do a cp. That saves 1 ssh call to check if file exists, but it does leave some ugly non-critical exception in the log.
,
Oct 2 2017
I think I can move the ValueError handling to collect_log_file, so it won't log a long stack trace.
,
Oct 10 2017
,
Oct 10 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/c6b709e18e41446aae12a3ab55d322e23ef995d6 commit c6b709e18e41446aae12a3ab55d322e23ef995d6 Author: Dan Shi <dshi@google.com> Date: Tue Oct 10 19:19:09 2017 [autotest] Don't log detailed stack trace for non-existing file. BUG= chromium:767602 TEST=repair a dut Change-Id: Iaa474c162cc688065db86fafc593981798b7cec4 Reviewed-on: https://chromium-review.googlesource.com/706135 Commit-Ready: Dan Shi <dshi@google.com> Tested-by: Dan Shi <dshi@google.com> Reviewed-by: Daniel Kurtz <djkurtz@chromium.org> Reviewed-by: Mike Frysinger <vapier@chromium.org> [modify] https://crrev.com/c6b709e18e41446aae12a3ab55d322e23ef995d6/server/crashcollect.py
,
Jan 22 2018
,
Jan 23 2018
|
|||||
►
Sign in to add a comment |
|||||
Comment 1 by dshi@chromium.org
, Sep 21 2017