New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 767602 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Last visit > 30 days ago
Closed: Oct 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 2
Type: Bug



Sign in to add a comment

ValueError: invalid literal for int() with base 10: ''

Project Member Reported by akes...@chromium.org, Sep 21 2017

Issue description

Seeing this error sporadically in various autotest logs. It adds confusion and spam. Seems like an easy fix.

Dan, related to anything you've touched lately?

09/21 09:40:39.691 INFO |      crashcollect:0252| chromeos4-row6-rack12-host3 already up, collecting crash info
09/21 09:40:39.692 INFO |      crashcollect:0103| Collecting /var/tmp/messages.autotest_start...
09/21 09:40:39.699 DEBUG|          ssh_host:0296| Running (ssh) 'ls -ld /var/tmp/messages.autotest_start | cut -d" " -f5' from 'collect_messages|collect_log_file|_get_file_stats|run|wrapper|run_very_slowly'
09/21 09:40:39.868 ERROR|             utils:0280| [stderr] ls: cannot access '/var/tmp/messages.autotest_start': No such file or directory
09/21 09:40:39.871 ERROR|      crashcollect:0122| Non-critical failure: collection of /var/tmp/messages.autotest_start failed: invalid literal for int() with base 10: ''
Traceback (most recent call last):
  File "/usr/local/autotest/server/crashcollect.py", line 109, in collect_log_file
    file_stats = _get_file_stats(host, log_path)
  File "/usr/local/autotest/server/crashcollect.py", line 160, in _get_file_stats
    file_size = int(host.run(cmd).stdout)
ValueError: invalid literal for int() with base 10: ''


https://storage.cloud.google.com/chromeos-autotest-results/143816328-chromeos-test/chromeos4-row6-rack12-host3/debug/autoserv.DEBUG?_ga=2.190583723.-1235484419.1474328066
 

Comment 1 by dshi@chromium.org, Sep 21 2017

I didn't change/add that part of the code. Handling the error should just work:
https://chromium-review.googlesource.com/#/c/chromiumos/third_party/autotest/+/677140/
Project Member

Comment 2 by bugdroid1@chromium.org, Sep 22 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/73980516cf0c21e848981b7fc308f0087a3a6a17

commit 73980516cf0c21e848981b7fc308f0087a3a6a17
Author: Dan Shi <dshi@google.com>
Date: Fri Sep 22 09:02:03 2017

[autotest] Handle ValueError when getting file size

For some special file, the command returns empty string for the file size.
Default that to zero.

BUG= chromium:767602 
TEST=None

Change-Id: Ibbb3ab26249df78a1354f1b043f87c4ff1784ac7
Reviewed-on: https://chromium-review.googlesource.com/677140
Commit-Ready: Dan Shi <dshi@google.com>
Tested-by: Dan Shi <dshi@google.com>
Reviewed-by: Allen Li <ayatane@chromium.org>

[modify] https://crrev.com/73980516cf0c21e848981b7fc308f0087a3a6a17/server/crashcollect.py

Cc: -dshi@chromium.org ayatane@chromium.org
Labels: OS-Chrome
Owner: dshi@chromium.org
Status: Assigned (was: Untriaged)
Fixing the spam from #2 seems to have just passed the buck...

If the file doesn't exist, the "ls " in _get_file_stats() generates error "No such file or directory", and returns ''.  We aren't checking for an error here, but instead just trying to cast '' to int.  This is what was generating the "ValueError" exception previously.

With CL:677140, this exception is now being handled, with a message and _get_file_stats() happily returns _FileStats(0, 1.0).
Thus, collect_log_file() can now continue (previously it would except out on the unhandled ValueError), and it tries to collect the non-existent file.
This in turn excepts out a bit later, like this:

[from https://storage.cloud.google.com/chromeos-autotest-results/146469517-chromeos-test/chromeos4-row9-rack10-host17/debug/autoserv.DEBUG?_ga=2.124780748.-734044362.1501703718]

10/02 10:59:43.774 INFO |      crashcollect:0103| Collecting /sys/kernel/debug/dri/0/i915_error_state...
10/02 10:59:43.795 DEBUG|          ssh_host:0296| Running (ssh) 'ls -ld /sys/kernel/debug/dri/0/i915_error_state | cut -d" " -f5' from 'get_crashinfo|collect_log_file|_get_file_stats|run|wrapper|run_very_slowly'
10/02 10:59:44.214 ERROR|             utils:0280| [stderr] ls: cannot access '/sys/kernel/debug/dri/0/i915_error_state': No such file or directory
10/02 10:59:44.217 WARNI|      crashcollect:0163| Getting size of file '/sys/kernel/debug/dri/0/i915_error_state' on host <remote host: chromeos4-row9-rack10-host17> failed: invalid literal for int() with base 10: ''
10/02 10:59:44.234 DEBUG|          ssh_host:0296| Running (ssh) 'mktemp -d' from 'collect_log_file|_collect_log_file_with_tmpdir|__enter__|run|wrapper|run_very_slowly'
10/02 10:59:44.724 DEBUG|          ssh_host:0296| Running (ssh) 'cp -rp /sys/kernel/debug/dri/0/i915_error_state /tmp/tmp.GCUhLnjnM7' from 'get_crashinfo|collect_log_file|_collect_log_file_with_tmpdir|run|wrapper|run_very_slowly'
10/02 10:59:45.145 ERROR|             utils:0280| [stderr] cp: cannot stat '/sys/kernel/debug/dri/0/i915_error_state': No such file or directory
10/02 10:59:45.163 DEBUG|          ssh_host:0296| Running (ssh) 'rm -rf /tmp/tmp.GCUhLnjnM7' from 'collect_log_file|_collect_log_file_with_tmpdir|__exit__|run|wrapper|run_very_slowly'
10/02 10:59:45.564 ERROR|      crashcollect:0122| Non-critical failure: collection of /sys/kernel/debug/dri/0/i915_error_state failed: command execution error
* Command: 
    /usr/bin/ssh -a -x    -o ControlPath=/tmp/_autotmp_W2Umcfssh-
    master/socket -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null
    -o BatchMode=yes -o ConnectTimeout=30 -o ServerAliveInterval=900 -o
    ServerAliveCountMax=3 -o ConnectionAttempts=4 -o Protocol=2 -l root -p 22
    chromeos4-row9-rack10-host17 "export LIBC_FATAL_STDERR_=1; if type
    \"logger\" > /dev/null 2>&1; then logger -tag \"autotest\"
    \"server[stack::_collect_log_file_with_tmpdir|run|wrapper] -> ssh_run(cp
    -rp /sys/kernel/debug/dri/0/i915_error_state /tmp/tmp.GCUhLnjnM7)\";fi; cp
    -rp /sys/kernel/debug/dri/0/i915_error_state /tmp/tmp.GCUhLnjnM7"
Exit status: 1
Duration: 0.344256877899

stderr:
cp: cannot stat '/sys/kernel/debug/dri/0/i915_error_state': No such file or directory
Traceback (most recent call last):
  File "/usr/local/autotest/server/crashcollect.py", line 117, in collect_log_file
    _collect_log_file_with_tmpdir(host, log_path, dest_path)
  File "/usr/local/autotest/server/crashcollect.py", line 145, in _collect_log_file_with_tmpdir
    host.run('cp -rp %s %s' % (pipes.quote(log_path), pipes.quote(tmpdir)))
  File "/usr/local/autotest/server/hosts/ssh_host.py", line 316, in run
    return self.run_very_slowly(*args, **kwargs)
  File "/usr/local/autotest/site-packages/chromite/lib/metrics.py", line 482, in wrapper
    return fn(*args, **kwargs)
  File "/usr/local/autotest/server/hosts/ssh_host.py", line 307, in run_very_slowly
    ssh_failure_retry_ok)
  File "/usr/local/autotest/server/hosts/ssh_host.py", line 258, in _run
    raise error.AutoservRunError("command execution error", result)
AutoservRunError: command execution error
* Command: 
    /usr/bin/ssh -a -x    -o ControlPath=/tmp/_autotmp_W2Umcfssh-
    master/socket -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null
    -o BatchMode=yes -o ConnectTimeout=30 -o ServerAliveInterval=900 -o
    ServerAliveCountMax=3 -o ConnectionAttempts=4 -o Protocol=2 -l root -p 22
    chromeos4-row9-rack10-host17 "export LIBC_FATAL_STDERR_=1; if type
    \"logger\" > /dev/null 2>&1; then logger -tag \"autotest\"
    \"server[stack::_collect_log_file_with_tmpdir|run|wrapper] -> ssh_run(cp
    -rp /sys/kernel/debug/dri/0/i915_error_state /tmp/tmp.GCUhLnjnM7)\";fi; cp
    -rp /sys/kernel/debug/dri/0/i915_error_state /tmp/tmp.GCUhLnjnM7"
Exit status: 1
Duration: 0.344256877899

stderr:
cp: cannot stat '/sys/kernel/debug/dri/0/i915_error_state': No such file or directory

Comment 4 by dshi@chromium.org, Oct 2 2017

That's kind of expected. The crash collection has a list of files to collect. Instead of checking if file exists or not, it forces to do a cp. That saves 1 ssh call to check if file exists, but it does leave some ugly non-critical exception in the log.

Comment 5 by dshi@chromium.org, Oct 2 2017

I think I can move the ValueError handling to collect_log_file, so it won't log a long stack trace.

Comment 6 by dshi@chromium.org, Oct 10 2017

Status: Fixed (was: Assigned)
Project Member

Comment 7 by bugdroid1@chromium.org, Oct 10 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/c6b709e18e41446aae12a3ab55d322e23ef995d6

commit c6b709e18e41446aae12a3ab55d322e23ef995d6
Author: Dan Shi <dshi@google.com>
Date: Tue Oct 10 19:19:09 2017

[autotest] Don't log detailed stack trace for non-existing file.

BUG= chromium:767602 
TEST=repair a dut

Change-Id: Iaa474c162cc688065db86fafc593981798b7cec4
Reviewed-on: https://chromium-review.googlesource.com/706135
Commit-Ready: Dan Shi <dshi@google.com>
Tested-by: Dan Shi <dshi@google.com>
Reviewed-by: Daniel Kurtz <djkurtz@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/c6b709e18e41446aae12a3ab55d322e23ef995d6/server/crashcollect.py

Comment 8 by dchan@chromium.org, Jan 22 2018

Status: Archived (was: Fixed)

Comment 9 by dchan@chromium.org, Jan 23 2018

Status: Fixed (was: Archived)

Sign in to add a comment