New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 776089 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Last visit > 30 days ago
Closed: Oct 2017
Cc:
Components:
EstimatedDays: ----
NextAction: 2017-10-19
OS: Android
Pri: ----
Type: ----



Sign in to add a comment

blink_perf.layout failing on chromium.perf/Android Nexus6 Perf due to problems with stopping AndroidBrowserBackend

Project Member Reported by charliea@google.com, Oct 18 2017

Issue description

blink_perf.layout failing on chromium.perf/Android Nexus6 Perf

Builders failed on: 
- Android Nexus6 Perf: 
  https://build.chromium.org/p/chromium.perf/builders/Android%20Nexus6%20Perf

It's not clear to me what's causing the failure - the common entry point to all of these failures seems to be AndroidBrowserBackend.StopBrowser() in Telemetry. Given that this doesn't seem to happen at a consistent story, I don't have much hope for stopping this until we can do more investigation.

nednguyen@, have you seen issues like this before - do failures in AndroidBrowserBackend.StopBrowser() indicate some other problem?

Failure excerpt (https://chromium-swarm.appspot.com/task?id=393cf2e50d23a410&refresh=10&show_raw=1): 

Exception from TearDownState:

Traceback (most recent call last):
  Run at /b/swarming/w/ir/third_party/catapult/telemetry/telemetry/internal/story_runner.py:251
    state.TearDownState()
  traced_function at /b/swarming/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py:52
    return func(*args, **kwargs)
  TearDownState at /b/swarming/w/ir/third_party/catapult/telemetry/telemetry/page/shared_page_state.py:329
    self._StopBrowser()
  traced_function at /b/swarming/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py:52
    return func(*args, **kwargs)
  _StopBrowser at /b/swarming/w/ir/third_party/catapult/telemetry/telemetry/page/shared_page_state.py:335
    self._browser.Close()
  Close at /b/swarming/w/ir/third_party/catapult/telemetry/telemetry/internal/browser/browser.py:278
    self._browser_backend.Close()
  traced_function at /b/swarming/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py:52
    return func(*args, **kwargs)
  Close at /b/swarming/w/ir/third_party/catapult/telemetry/telemetry/internal/backends/chrome/android_browser_backend.py:261
    self._StopBrowser()
  traced_function at /b/swarming/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py:52
    return func(*args, **kwargs)
  _StopBrowser at /b/swarming/w/ir/third_party/catapult/telemetry/telemetry/internal/backends/chrome/android_browser_backend.py:78
    self.platform_backend.StopApplication(self._backend_settings.package)
  StopApplication at /b/swarming/w/ir/third_party/catapult/telemetry/telemetry/internal/platform/android_platform_backend.py:371
    self._device.ForceStop(application)
  timeout_retry_wrapper at /b/swarming/w/ir/third_party/catapult/devil/devil/android/decorators.py:57
    retry_if_func=retry_if_func)
  Run at /b/swarming/w/ir/third_party/catapult/devil/devil/utils/timeout_retry.py:157
    error_log_func=error_log_func)
  JoinAll at /b/swarming/w/ir/third_party/catapult/devil/devil/utils/reraiser_thread.py:186
    self._JoinAll(watcher, timeout)
  _JoinAll at /b/swarming/w/ir/third_party/catapult/devil/devil/utils/reraiser_thread.py:158
    thread.ReraiseIfException()
  run at /b/swarming/w/ir/third_party/catapult/devil/devil/utils/reraiser_thread.py:81
    self._ret = self._func(*self._args, **self._kwargs)
  <lambda> at /b/swarming/w/ir/third_party/catapult/devil/devil/utils/timeout_retry.py:150
    child_thread = reraiser_thread.ReraiserThread(lambda: func(*args, **kwargs),
  impl at /b/swarming/w/ir/third_party/catapult/devil/devil/android/decorators.py:47
    return f(*args, **kwargs)
  ForceStop at /b/swarming/w/ir/third_party/catapult/devil/devil/android/device_utils.py:1265
    if self.GetPids(package):
  timeout_retry_wrapper at /b/swarming/w/ir/third_party/catapult/devil/devil/android/decorators.py:51
    return impl()
  impl at /b/swarming/w/ir/third_party/catapult/devil/devil/android/decorators.py:47
    return f(*args, **kwargs)
  GetPids at /b/swarming/w/ir/third_party/catapult/devil/devil/android/device_utils.py:2275
    '%s | grep -F %s' % (ps_cmd, cmd_helper.SingleQuote(process_name)))
  _RunPipedShellCommand at /b/swarming/w/ir/third_party/catapult/devil/devil/android/device_utils.py:1063
    output = self.RunShellCommand(script, **kwargs)
  timeout_retry_wrapper at /b/swarming/w/ir/third_party/catapult/devil/devil/android/decorators.py:51
    return impl()
  impl at /b/swarming/w/ir/third_party/catapult/devil/devil/android/decorators.py:47
    return f(*args, **kwargs)
  RunShellCommand at /b/swarming/w/ir/third_party/catapult/devil/devil/android/device_utils.py:1041
    output = handle_large_output(cmd, large_output)
  handle_large_output at /b/swarming/w/ir/third_party/catapult/devil/devil/android/device_utils.py:1010
    return handle_large_command(cmd)
  handle_large_command at /b/swarming/w/ir/third_party/catapult/devil/devil/android/device_utils.py:992
    return handle_check_return(cmd)
  handle_check_return at /b/swarming/w/ir/third_party/catapult/devil/devil/android/device_utils.py:983
    return run(cmd)
  run at /b/swarming/w/ir/third_party/catapult/devil/devil/android/device_utils.py:979
    return self.adb.Shell(cmd)
  Shell at /b/swarming/w/ir/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py:505
    output = self._RunDeviceAdbCmd(args, timeout, retries, check_error=False)
  _RunDeviceAdbCmd at /b/swarming/w/ir/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py:301
    check_error=check_error)
  timeout_retry_wrapper at /b/swarming/w/ir/third_party/catapult/devil/devil/android/decorators.py:51
    return impl()
  impl at /b/swarming/w/ir/third_party/catapult/devil/devil/android/decorators.py:47
    return f(*args, **kwargs)
  _RunAdbCmd at /b/swarming/w/ir/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py:278
    raise device_errors.DeviceUnreachableError(device_serial)
DeviceUnreachableError: ZX1G523MP9

 
The NextAction date has arrived: 2017-10-19
Cc: -nednguyen@chromium.org
Components: Speed>Benchmarks
Owner: e...@chromium.org
Status: Assigned (was: Available)
Cc: nedngu...@google.com
Components: -Speed>Benchmarks Infra>Labs
Owner: vhang@chromium.org
Err, this is more likely a flaky device error. Lab: can you help taking a look at the device 'build16-b1--device5'?

Comment 4 by pschm...@google.com, Oct 19 2017

Owner: pschmidt@chromium.org
Status: Started (was: Assigned)
4 of the 7 N6's attached to build16-b1 have swollen batteries.  I'm swapping those.

I also noticed that there is one on build15-b1.  I'll switch that one out too.
Thanks Peter!
On closer inspection there are 3 N6's on build15-b1 that have swollen batteries.  Will fix those in a bit.

The complement of devices on build16-b1 are:

List of devices attached
LP5A230134	device
LP5V230012	device
ZX1G22K86J	device
ZY222X2RF7	device
ZX1G523MP9	device
NP5B2A0027	device
NP5A2B0084	device

Status: Fixed (was: Started)
Bad N6's on build15-b1 have been replaced.

Updated device list is:

List of devices attached
ZX1G22DJ7J	device
ZX1G22KFD9	device
ZX1G522Z3D	device
ZX1G22KGFM	device
ZX1G22KGWK	device
LP5A2L0063	device
ZX1G22Q2S7	device


Was I hacjed

Sign in to add a comment