New issue
Advanced search Search tips

Issue 749740 link

Starred by 1 user

Issue metadata

Status: Verified
Owner: ----
Closed: Jul 2017
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Bug



Sign in to add a comment

Individual device set up failing on swarming shard

Project Member Reported by petermayo@chromium.org, Jul 27 2017

Issue description

services_unittests failing on chromium.linux/Android Tests (dbg)

Builders failed on: 
- Android Tests (dbg): 
  https://build.chromium.org/p/chromium.linux/builders/Android%20Tests%20%28dbg%29



E   33.881s individual_device_set_up(03d1c361006afe9c)  Shard died: individual_device_set_up(03d1c361006afe9c)
Traceback (most recent call last):
  File "/b/swarming/w/ir/build/android/pylib/local/device/local_device_environment.py", line 59, in wrapper
    return f(dev, *args, **kwargs)
  File "/b/swarming/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 52, in traced_function
    return func(*args, **kwargs)
  File "/b/swarming/w/ir/build/android/pylib/local/device/local_device_gtest_run.py", line 319, in individual_device_set_up
    step()
  File "/b/swarming/w/ir/build/android/pylib/local/device/local_device_gtest_run.py", line 293, in push_test_data
    delete_device_stale=True)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/android/decorators.py", line 57, in timeout_retry_wrapper
    retry_if_func=retry_if_func)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/utils/timeout_retry.py", line 159, in Run
    error_log_func=error_log_func)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 186, in JoinAll
    self._JoinAll(watcher, timeout)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 158, in _JoinAll
    thread.ReraiseIfException()
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 81, in run
    self._ret = self._func(*self._args, **self._kwargs)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/utils/timeout_retry.py", line 152, in <lambda>
    child_thread = reraiser_thread.ReraiserThread(lambda: func(*args, **kwargs),
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl
    return f(*args, **kwargs)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/android/device_utils.py", line 1343, in PushChangedFiles
    self._GetChangedAndStaleFiles(h, d, delete_device_stale))
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/android/device_utils.py", line 1411, in _GetChangedAndStaleFiles
    calculate_device_checksums))
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 228, in RunAsync
    return thread_group.GetAllReturnValues(watcher=watcher)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 200, in GetAllReturnValues
    self.JoinAll(watcher)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 186, in JoinAll
    self._JoinAll(watcher, timeout)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 158, in _JoinAll
    thread.ReraiseIfException()
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 81, in run
    self._ret = self._func(*self._args, **self._kwargs)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/android/device_utils.py", line 1403, in calculate_device_checksums
    sums = md5sum.CalculateDeviceMd5Sums(specific_device_paths, self)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/android/md5sum.py", line 92, in CalculateDeviceMd5Sums
    out = device.RunShellCommand(md5sum_script, shell=True, check_return=True)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/android/decorators.py", line 51, in timeout_retry_wrapper
    return impl()
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl
    return f(*args, **kwargs)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/android/device_utils.py", line 1039, in RunShellCommand
    output = handle_large_output(cmd, large_output)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/android/device_utils.py", line 1008, in handle_large_output
    return handle_large_command(cmd)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/android/device_utils.py", line 990, in handle_large_command
    return handle_check_return(cmd)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/android/device_utils.py", line 981, in handle_check_return
    return run(cmd)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/android/device_utils.py", line 977, in run
    return self.adb.Shell(cmd)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 489, in Shell
    output = self._RunDeviceAdbCmd(args, timeout, retries, check_error=False)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 286, in _RunDeviceAdbCmd
    check_error=check_error)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/android/decorators.py", line 51, in timeout_retry_wrapper
    return impl()
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl
    return f(*args, **kwargs)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 263, in _RunAdbCmd
    raise device_errors.DeviceUnreachableError(device_serial)
DeviceUnreachableError: 03d1c361006afe9c
E   33.881s individual_device_set_up(03d1c361006afe9c)  Device 03d1c361006afe9c blacklisted: individual_device_set_up
C   33.882s Main  Cannot upload logcat file: file doesn't exist.
E   33.883s Main  Error occurred.
Traceback (most recent call last):
  File "/b/swarming/w/ir/build/android/pylib/local/device/local_device_environment.py", line 59, in wrapper
    return f(dev, *args, **kwargs)
  File "/b/swarming/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 52, in traced_function
    return func(*args, **kwargs)
  File "/b/swarming/w/ir/build/android/pylib/local/device/local_device_gtest_run.py", line 319, in individual_device_set_up
    step()
  File "/b/swarming/w/ir/build/android/pylib/local/device/local_device_gtest_run.py", line 293, in push_test_data
    delete_device_stale=True)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/android/decorators.py", line 57, in timeout_retry_wrapper
    retry_if_func=retry_if_func)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/utils/timeout_retry.py", line 159, in Run
    error_log_func=error_log_func)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 186, in JoinAll
    self._JoinAll(watcher, timeout)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 158, in _JoinAll
    thread.ReraiseIfException()
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 81, in run
    self._ret = self._func(*self._args, **self._kwargs)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/utils/timeout_retry.py", line 152, in <lambda>
    child_thread = reraiser_thread.ReraiserThread(lambda: func(*args, **kwargs),
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl
    return f(*args, **kwargs)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/android/device_utils.py", line 1343, in PushChangedFiles
    self._GetChangedAndStaleFiles(h, d, delete_device_stale))
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/android/device_utils.py", line 1411, in _GetChangedAndStaleFiles
    calculate_device_checksums))
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 228, in RunAsync
    return thread_group.GetAllReturnValues(watcher=watcher)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 200, in GetAllReturnValues
    self.JoinAll(watcher)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 186, in JoinAll
    self._JoinAll(watcher, timeout)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 158, in _JoinAll
    thread.ReraiseIfException()
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 81, in run
    self._ret = self._func(*self._args, **self._kwargs)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/android/device_utils.py", line 1403, in calculate_device_checksums
    sums = md5sum.CalculateDeviceMd5Sums(specific_device_paths, self)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/android/md5sum.py", line 92, in CalculateDeviceMd5Sums
    out = device.RunShellCommand(md5sum_script, shell=True, check_return=True)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/android/decorators.py", line 51, in timeout_retry_wrapper
    return impl()
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl
    return f(*args, **kwargs)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/android/device_utils.py", line 1039, in RunShellCommand
    output = handle_large_output(cmd, large_output)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/android/device_utils.py", line 1008, in handle_large_output
    return handle_large_command(cmd)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/android/device_utils.py", line 990, in handle_large_command
    return handle_check_return(cmd)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/android/device_utils.py", line 981, in handle_check_return
    return run(cmd)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/android/device_utils.py", line 977, in run
    return self.adb.Shell(cmd)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 489, in Shell
    output = self._RunDeviceAdbCmd(args, timeout, retries, check_error=False)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 286, in _RunDeviceAdbCmd
    check_error=check_error)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/android/decorators.py", line 51, in timeout_retry_wrapper
    return impl()
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl
    return f(*args, **kwargs)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 263, in _RunAdbCmd
    raise device_errors.DeviceUnreachableError(device_serial)
DeviceUnreachableError: 03d1c361006afe9c
E   33.881s individual_device_set_up(03d1c361006afe9c)  Device 03d1c361006afe9c blacklisted: individual_device_set_up

 
Status: Verified (was: Available)
bot that failed this now quarantined

Sign in to add a comment