New issue
Advanced search Search tips

Issue 680981 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Jan 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: ----
Type: ----



Sign in to add a comment

chrome_public_test_apk on Android failing on chromium.android/Lollipop Phone Tester

Project Member Reported by clamy@chromium.org, Jan 13 2017

Issue description

chrome_public_test_apk on Android keeps failing on chromium.android/Lollipop Phone Tester due to exceeding the allocated time for a pending shard. Note: it seems this is a swarming build, but with only one shard. Does it make sense?

This also fails on Android Dbg as well, for the same reasons.

Type: infra-failure

Builders failed on: 
- Lollipop Phone Tester: 
  https://build.chromium.org/p/chromium.android/builders/Lollipop%20Phone%20Tester



 
Owner: jbudorick@chromium.org
Status: Started (was: Available)
We've got some outdated dimensions set for this suite on this bot: 960s for the timeout + at least 4 devices. (By comparison, we have this suite set for 1800s + at least 6 devices on the K bots & 1200s on the M bots.) When it gets picked up by a swarming slave with less than 6 (maybe less than 7?) phones, it can't run within that 16-minute window.

Re sharding: we don't currently do sharding at the swarming level. Our android swarming slaves each run with multiple devices attached (typically, but not always, 7), and sharding across those is handled within a task. (This is all going to change over the course of this year, though.)
re Android Tests (dbg): one or more devices are failing at setup time with:

E    0.157s prepare_device(0384884f0ae5848c)  Shard failed: prepare_device(0384884f0ae5848c)
Traceback (most recent call last):
  File "/b/swarm_slave/w/ir6QnwmC/build/android/pylib/local/device/local_device_environment.py", line 55, in wrapper
    return f(dev, *args, **kwargs)
  File "/b/swarm_slave/w/ir6QnwmC/build/android/pylib/local/device/local_device_environment.py", line 124, in prepare_device
    d.WaitUntilFullyBooted(timeout=10, retries=0)
  File "/b/swarm_slave/w/ir6QnwmC/third_party/catapult/devil/devil/android/decorators.py", line 57, in timeout_retry_wrapper
    retry_if_func=retry_if_func)
  File "/b/swarm_slave/w/ir6QnwmC/third_party/catapult/devil/devil/utils/timeout_retry.py", line 159, in Run
    error_log_func=error_log_func)
  File "/b/swarm_slave/w/ir6QnwmC/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 186, in JoinAll
    self._JoinAll(watcher, timeout)
  File "/b/swarm_slave/w/ir6QnwmC/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 158, in _JoinAll
    thread.ReraiseIfException()
  File "/b/swarm_slave/w/ir6QnwmC/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 81, in run
    self._ret = self._func(*self._args, **self._kwargs)
  File "/b/swarm_slave/w/ir6QnwmC/third_party/catapult/devil/devil/utils/timeout_retry.py", line 152, in <lambda>
    child_thread = reraiser_thread.ReraiserThread(lambda: func(*args, **kwargs),
  File "/b/swarm_slave/w/ir6QnwmC/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl
    return f(*args, **kwargs)
  File "/b/swarm_slave/w/ir6QnwmC/third_party/catapult/devil/devil/android/device_utils.py", line 597, in WaitUntilFullyBooted
    self.adb.WaitForDevice()
  File "/b/swarm_slave/w/ir6QnwmC/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 803, in WaitForDevice
    self._RunDeviceAdbCmd(['wait-for-device'], timeout, retries)
  File "/b/swarm_slave/w/ir6QnwmC/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 282, in _RunDeviceAdbCmd
    check_error=check_error)
  File "/b/swarm_slave/w/ir6QnwmC/third_party/catapult/devil/devil/android/decorators.py", line 51, in timeout_retry_wrapper
    return impl()
  File "/b/swarm_slave/w/ir6QnwmC/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl
    return f(*args, **kwargs)
  File "/b/swarm_slave/w/ir6QnwmC/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 258, in _RunAdbCmd
    args, output, status, device_serial)
AdbCommandFailedError: (device: 0384884f0ae5848c) adb wait-for-device: failed with exit status 1 and output:
- error: could not install *smartsocket* listener: Address already in use
- ADB server didn't ACK
- * failed to start daemon *
- error: cannot connect to daemon
- * daemon not running. starting it now on port 5037 *

That may be related to https://codereview.chromium.org/2628953003/ (specifically, the lack of retries).
Project Member

Comment 3 by bugdroid1@chromium.org, Jan 13 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/4c94218865f2cbdabb6ca5991c22b97abab55f16

commit 4c94218865f2cbdabb6ca5991c22b97abab55f16
Author: jbudorick <jbudorick@chromium.org>
Date: Fri Jan 13 18:43:20 2017

[Android] Raise timeout and required devices for chrome_public_test_apk on L Phone.

BUG= 680981 
TBR=bpastene@chromium.org

Review-Url: https://codereview.chromium.org/2631713002
Cr-Commit-Position: refs/heads/master@{#443623}

[modify] https://crrev.com/4c94218865f2cbdabb6ca5991c22b97abab55f16/testing/buildbot/chromium.android.json

Project Member

Comment 4 by bugdroid1@chromium.org, Jan 13 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/3bef24fc1daaa77d2306e5f3f5c9422abe8b58d1

commit 3bef24fc1daaa77d2306e5f3f5c9422abe8b58d1
Author: jbudorick <jbudorick@chromium.org>
Date: Fri Jan 13 19:44:12 2017

[Android] Enable retries in initial device check during test setup.

Follow up to https://codereview.chromium.org/2628953003/ after hitting
a few possibly transient exceptions on Android Tests (dbg).

BUG=664308, 680981 
TBR=bpastene@chromium.org

Review-Url: https://codereview.chromium.org/2635503002
Cr-Commit-Position: refs/heads/master@{#443641}

[modify] https://crrev.com/3bef24fc1daaa77d2306e5f3f5c9422abe8b58d1/build/android/pylib/local/device/local_device_environment.py

Status: Fixed (was: Started)
Should be fixed starting with build #9124. I'll reopen if this recurs.
Labels: -undefined

Sign in to add a comment