New issue
Advanced search Search tips

Issue 765799 link

Starred by 1 user

Issue metadata

Status: Available
Owner: ----
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: ----



Sign in to add a comment

blink_perf.canvas failing on chromium.perf/Android Nexus6 Perf

Project Member Reported by charliea@chromium.org, Sep 15 2017

Issue description

blink_perf.canvas failing on chromium.perf/Android Nexus6 Perf

Builders failed on: 
- Android Nexus6 Perf: 
  https://build.chromium.org/p/chromium.perf/builders/Android%20Nexus6%20Perf

It seems that this benchmark is failing in a way that's causing the device to go offline, or at least become unresponsive. The stack trace is as follows (pulled from https://chromium-swarm.appspot.com/task?id=38924bae35a19d10&refresh=10&show_raw=1): 

Traceback (most recent call last):
  RunBenchmark at /b/swarming/w/ir/third_party/catapult/telemetry/telemetry/internal/story_runner.py:332
    expectations=expectations, metadata=benchmark.GetMetadata())
  Run at /b/swarming/w/ir/third_party/catapult/telemetry/telemetry/internal/story_runner.py:212
    state.TearDownState()
  traced_function at /b/swarming/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py:52
    return func(*args, **kwargs)
  TearDownState at /b/swarming/w/ir/third_party/catapult/telemetry/telemetry/page/shared_page_state.py:336
    self._StopBrowser()
  traced_function at /b/swarming/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py:52
    return func(*args, **kwargs)
  _StopBrowser at /b/swarming/w/ir/third_party/catapult/telemetry/telemetry/page/shared_page_state.py:342
    self._browser.Close()
  Close at /b/swarming/w/ir/third_party/catapult/telemetry/telemetry/internal/browser/browser.py:278
    self._browser_backend.Close()
  traced_function at /b/swarming/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py:52
    return func(*args, **kwargs)
  Close at /b/swarming/w/ir/third_party/catapult/telemetry/telemetry/internal/backends/chrome/android_browser_backend.py:261
    self._StopBrowser()
  traced_function at /b/swarming/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py:52
    return func(*args, **kwargs)
  _StopBrowser at /b/swarming/w/ir/third_party/catapult/telemetry/telemetry/internal/backends/chrome/android_browser_backend.py:78
    self.platform_backend.StopApplication(self._backend_settings.package)
  StopApplication at /b/swarming/w/ir/third_party/catapult/telemetry/telemetry/internal/platform/android_platform_backend.py:370
    self._device.ForceStop(application)
  timeout_retry_wrapper at /b/swarming/w/ir/third_party/catapult/devil/devil/android/decorators.py:57
    retry_if_func=retry_if_func)
  Run at /b/swarming/w/ir/third_party/catapult/devil/devil/utils/timeout_retry.py:159
    error_log_func=error_log_func)
  JoinAll at /b/swarming/w/ir/third_party/catapult/devil/devil/utils/reraiser_thread.py:186
    self._JoinAll(watcher, timeout)
  _JoinAll at /b/swarming/w/ir/third_party/catapult/devil/devil/utils/reraiser_thread.py:158
    thread.ReraiseIfException()
  run at /b/swarming/w/ir/third_party/catapult/devil/devil/utils/reraiser_thread.py:81
    self._ret = self._func(*self._args, **self._kwargs)
  <lambda> at /b/swarming/w/ir/third_party/catapult/devil/devil/utils/timeout_retry.py:152
    child_thread = reraiser_thread.ReraiserThread(lambda: func(*args, **kwargs),
  impl at /b/swarming/w/ir/third_party/catapult/devil/devil/android/decorators.py:47
    return f(*args, **kwargs)
  ForceStop at /b/swarming/w/ir/third_party/catapult/devil/devil/android/device_utils.py:1265
    if self.GetPids(package):
  timeout_retry_wrapper at /b/swarming/w/ir/third_party/catapult/devil/devil/android/decorators.py:51
    return impl()
  impl at /b/swarming/w/ir/third_party/catapult/devil/devil/android/decorators.py:47
    return f(*args, **kwargs)
  GetPids at /b/swarming/w/ir/third_party/catapult/devil/devil/android/device_utils.py:2272
    '%s | grep -F %s' % (ps_cmd, cmd_helper.SingleQuote(process_name)))
  _RunPipedShellCommand at /b/swarming/w/ir/third_party/catapult/devil/devil/android/device_utils.py:1063
    output = self.RunShellCommand(script, **kwargs)
  timeout_retry_wrapper at /b/swarming/w/ir/third_party/catapult/devil/devil/android/decorators.py:51
    return impl()
  impl at /b/swarming/w/ir/third_party/catapult/devil/devil/android/decorators.py:47
    return f(*args, **kwargs)
  RunShellCommand at /b/swarming/w/ir/third_party/catapult/devil/devil/android/device_utils.py:1041
    output = handle_large_output(cmd, large_output)
  handle_large_output at /b/swarming/w/ir/third_party/catapult/devil/devil/android/device_utils.py:1010
    return handle_large_command(cmd)
  handle_large_command at /b/swarming/w/ir/third_party/catapult/devil/devil/android/device_utils.py:992
    return handle_check_return(cmd)
  handle_check_return at /b/swarming/w/ir/third_party/catapult/devil/devil/android/device_utils.py:983
    return run(cmd)
  run at /b/swarming/w/ir/third_party/catapult/devil/devil/android/device_utils.py:979
    return self.adb.Shell(cmd)
  Shell at /b/swarming/w/ir/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py:505
    output = self._RunDeviceAdbCmd(args, timeout, retries, check_error=False)
  _RunDeviceAdbCmd at /b/swarming/w/ir/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py:301
    check_error=check_error)
  timeout_retry_wrapper at /b/swarming/w/ir/third_party/catapult/devil/devil/android/decorators.py:51
    return impl()
  impl at /b/swarming/w/ir/third_party/catapult/devil/devil/android/decorators.py:47
    return f(*args, **kwargs)
  _RunAdbCmd at /b/swarming/w/ir/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py:278
    raise device_errors.DeviceUnreachableError(device_serial)
DeviceUnreachableError: ZX1G22K86J

Going to disable this story on Nexus 6, as it's been flaky for quite a while now.
 
It seems like the story went from not flaky to slightly flaky to very flaky, though, so I had to use a really wide revision range (5-7 days) and high repeat count (20). We'll see what happens.
Project Member

Comment 3 by 42576172...@developer.gserviceaccount.com, Sep 15 2017


=== BISECT JOB RESULTS ===
NO Test failure found

Bisect Details
  Configuration: android_nexus6_perf_bisect
  Benchmark    : blink_perf.canvas
  Metric       : draw-static-canvas-2d-to-hw-accelerated-canvas-2d/draw-static-canvas-2d-to-hw-accelerated-canvas-2d.html

Revision             Exit Code      N
chromium@499416      0 +- N/A       20      good
chromium@500411      0 +- N/A       20      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=draw.static.canvas.2d.to.hw.accelerated.canvas.2d.html blink_perf.canvas

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8968354610664962464


For feedback, file a bug with component Speed>Bisection
Project Member

Comment 4 by bugdroid1@chromium.org, Sep 15 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/014ec031e5868bc8bca34dead9a736828c310c68

commit 014ec031e5868bc8bca34dead9a736828c310c68
Author: Charlie Andrews <charliea@chromium.org>
Date: Fri Sep 15 21:58:27 2017

Disable blink_perf.canvas's draw-static-canvas-2d... story on N6

The story is repeatedly failing and seems to be hanging the device.

TBR=nednguyen@google.com

Bug: 765799
Change-Id: Ib90a3fa10b949f1511bf9ff4646c87670911a396
Reviewed-on: https://chromium-review.googlesource.com/669398
Reviewed-by: Charlie Andrews <charliea@chromium.org>
Commit-Queue: Charlie Andrews <charliea@chromium.org>
Cr-Commit-Position: refs/heads/master@{#502394}
[modify] https://crrev.com/014ec031e5868bc8bca34dead9a736828c310c68/tools/perf/benchmarks/blink_perf.py

Comment 5 by charliea@google.com, Oct 18 2017

Cc: -junov@chromium.org
Owner: junov@chromium.org
Components: Blink>Canvas
Owner: ----
Labels: Pri-2
Setting defect without priority to Pri-2.

Sign in to add a comment