New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 657017 link

Starred by 2 users

Issue metadata

Status: Duplicate
Merged: issue 638728
Owner: ----
Closed: Oct 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 1
Type: Bug



Sign in to add a comment

R55 builds are all failing provision_AutoUpdate.double

Project Member Reported by bhthompson@google.com, Oct 18 2016

Issue description

Since the build after the last dev release, 8872.8.0, we have been failing provision_AutoUpdate.double.

An example failure can be found at https://pantheon.corp.google.com/storage/browser/chromeos-autotest-results/81325485-chromeos-test/chromeos4-row8-rack8-host15
...
10/18 01:01:10.927 DEBUG|      abstract_ssh:0348| Using Rsync.
10/18 01:01:10.928 DEBUG|        base_utils:0185| Running 'rsync -L  --timeout=1800 --rsh='/usr/bin/ssh -a -x    -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o BatchMode=yes -o ConnectTimeout=30 -o ServerAliveInterval=900 -o ServerAliveCountMax=3 -o ConnectionAttempts=4 -o Protocol=2 -l root -p 22' -az --no-o --no-g root@chromeos4-row8-rack8-host15:"/tmp/autoserv-T56brl/sysinfo.pickle" "/tmp/tmp_vwJ4I"'
10/18 01:01:12.332 WARNI|              test:0606| Autotest caught exception when running test:
Traceback (most recent call last):
  File "/usr/local/autotest/client/common_lib/test.py", line 600, in _exec
    _call_test_function(self.execute, *p_args, **p_dargs)
  File "/usr/local/autotest/client/common_lib/test.py", line 810, in _call_test_function
    raise error.UnhandledTestFail(e)
UnhandledTestFail: Unhandled DevServerException: CrOS auto-update failed for host 100.115.199.56: has reached AU retry limit.
Traceback (most recent call last):
  File "/usr/local/autotest/client/common_lib/test.py", line 804, in _call_test_function
    return func(*args, **dargs)
  File "/usr/local/autotest/client/common_lib/test.py", line 461, in execute
    dargs)
  File "/usr/local/autotest/client/common_lib/test.py", line 347, in _call_run_once_with_retry
    postprocess_profiled_run, args, dargs)
  File "/usr/local/autotest/client/common_lib/test.py", line 376, in _call_run_once
    self.run_once(*args, **dargs)
  File "/usr/local/autotest/server/site_tests/provision_AutoUpdate/provision_AutoUpdate.py", line 147, in run_once
    force_full_update=force)
  File "/usr/local/autotest/server/afe_utils.py", line 237, in machine_install_and_update_labels
    *args, **dargs)
  File "/usr/local/autotest/server/hosts/cros_host.py", line 738, in machine_install_by_devserver
    full_update=force_full_update)
  File "/usr/local/autotest/client/common_lib/cros/dev_server.py", line 1829, in auto_update
    error_msg % (host_name, 'has reached AU retry limit.'))
DevServerException: CrOS auto-update failed for host 100.115.199.56: has reached AU retry limit.

10/18 01:01:12.333 DEBUG|   logging_manager:0627| Logging subprocess finished
...

Deltas are https://crosland.corp.google.com/log/8872.7.0..8872.8.0 and https://chromium.googlesource.com/chromium/src/+log/55.0.2883.9..55.0.2883.11?pretty=fuller&n=10000 though I don't see anything immediately obvious related to updates. 

Could this be related to https://bugs.chromium.org/p/chromium/issues/detail?id=651974 ?
 
Cc: jrbarnette@chromium.org
It is also interesting the bvt-inline did not fail the build due to this, and that bvt-cq was started anyway.
Labels: OS-Chrome

Comment 3 by xixuan@chromium.org, Oct 18 2016

Could this be related to https://bugs.chromium.org/p/chromium/issues/detail?id=651974 ?

--- No

From the logs in sysinfo/, looks like the two times' provision are suddenly cut off by a deadline or sth, and no further logs about whether 'rsync' succeeds or fails. 
Cc: dhadd...@chromium.org
Labels: M-55
Failing on 8743.74.0 as well
Components: Infra>Client>ChromeOS
Labels: bvttriage
Mergedinto: 638728
Status: Duplicate (was: Untriaged)

Sign in to add a comment