New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 655653 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Last visit > 30 days ago
Closed: Nov 2016
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Bug-Regression



Sign in to add a comment

system_health.memory_mobile failure on chromium.perf on Anroid Nexus 5X (1)

Project Member Reported by eyaich@chromium.org, Oct 13 2016

Issue description

Failing builder: 
https://uberchromegw.corp.google.com/i/chromium.perf/builders/Android%20Nexus5X%20Perf%20%281%29?numbuilds=200

Failing log: 
https://uberchromegw.corp.google.com/i/chromium.perf/builders/Android%20Nexus5X%20Perf%20%281%29/builds/3442/steps/system_health.memory_mobile/logs/stdio


Stack trace: 

INFO:py_utils.cloud_storage:Done locking /b/rr/tmpplEsBT/w/src/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb
INFO:devil.utils.cmd_helper:[host]> /b/rr/tmpplEsBT/w/src/third_party/catapult/devil/bin/deps/linux2/x86_64/forwarder_host --adb=/b/rr/tmpplEsBT/w/src/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb --serial-id=00b239635198b466 --unmap 39019
INFO:root:Try printing formatted exception: None None None

Traceback (most recent call last):
  RunBenchmark at /b/rr/tmpplEsBT/w/src/third_party/catapult/telemetry/telemetry/internal/story_runner.py:341
    benchmark.ShouldTearDownStateAfterEachStorySetRun())
  Run at /b/rr/tmpplEsBT/w/src/third_party/catapult/telemetry/telemetry/internal/story_runner.py:252
    state.TearDownState()
  traced_function at /b/rr/tmpplEsBT/w/src/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py:52
    return func(*args, **kwargs)
  TearDownState at /b/rr/tmpplEsBT/w/src/third_party/catapult/telemetry/telemetry/page/shared_page_state.py:322
    self._StopBrowser()
  traced_function at /b/rr/tmpplEsBT/w/src/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py:52
    return func(*args, **kwargs)
  _StopBrowser at /b/rr/tmpplEsBT/w/src/third_party/catapult/telemetry/telemetry/page/shared_page_state.py:328
    self._browser.Close()
  Close at /b/rr/tmpplEsBT/w/src/third_party/catapult/telemetry/telemetry/internal/browser/browser.py:265
    self._browser_backend.Close()
  traced_function at /b/rr/tmpplEsBT/w/src/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py:52
    return func(*args, **kwargs)
  Close at /b/rr/tmpplEsBT/w/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome/android_browser_backend.py:227
    self.platform_backend.StopForwardingHost(self._port)
  StopForwardingHost at /b/rr/tmpplEsBT/w/src/third_party/catapult/telemetry/telemetry/internal/platform/android_platform_backend.py:487
    if line[0] == self._device and line[1] == 'tcp:%s' % host_port:
IndexError: list index out of range

Locals:
  host_port : 43628
  line      : ['00b239635198b466']


If the test is disabled, please downgrade to Pri-2.

 

Comment 1 by eyaich@chromium.org, Oct 13 2016

similar error to what is happening on Android Nexus 5X Perf (2) for blink_style.top_25

Link to builder: 
https://uberchromegw.corp.google.com/i/chromium.perf/builders/Android%20Nexus5X%20Perf%20%282%29/builds/1830

Link to failing log for blink_style.top_25: https://uberchromegw.corp.google.com/i/chromium.perf/builders/Android%20Nexus5X%20Perf%20%282%29/builds/1830/steps/blink_style.top_25/logs/stdio

Same stack trace: 

INFO:py_utils.cloud_storage:Done locking /b/rr/tmpdC48Gu/w/src/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb
INFO:devil.utils.cmd_helper:[host]> /b/rr/tmpdC48Gu/w/src/third_party/catapult/devil/bin/deps/linux2/x86_64/forwarder_host --adb=/b/rr/tmpdC48Gu/w/src/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb --serial-id=00b5216f896322e1 --unmap 38298
INFO:root:Try printing formatted exception: None None None

Traceback (most recent call last):
  RunBenchmark at /b/rr/tmpdC48Gu/w/src/third_party/catapult/telemetry/telemetry/internal/story_runner.py:341
    benchmark.ShouldTearDownStateAfterEachStorySetRun())
  Run at /b/rr/tmpdC48Gu/w/src/third_party/catapult/telemetry/telemetry/internal/story_runner.py:252
    state.TearDownState()
  traced_function at /b/rr/tmpdC48Gu/w/src/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py:52
    return func(*args, **kwargs)
  TearDownState at /b/rr/tmpdC48Gu/w/src/third_party/catapult/telemetry/telemetry/page/shared_page_state.py:322
    self._StopBrowser()
  traced_function at /b/rr/tmpdC48Gu/w/src/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py:52
    return func(*args, **kwargs)
  _StopBrowser at /b/rr/tmpdC48Gu/w/src/third_party/catapult/telemetry/telemetry/page/shared_page_state.py:328
    self._browser.Close()
  Close at /b/rr/tmpdC48Gu/w/src/third_party/catapult/telemetry/telemetry/internal/browser/browser.py:265
    self._browser_backend.Close()
  traced_function at /b/rr/tmpdC48Gu/w/src/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py:52
    return func(*args, **kwargs)
  Close at /b/rr/tmpdC48Gu/w/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome/android_browser_backend.py:227
    self.platform_backend.StopForwardingHost(self._port)
  StopForwardingHost at /b/rr/tmpdC48Gu/w/src/third_party/catapult/telemetry/telemetry/internal/platform/android_platform_backend.py:487
    if line[0] == self._device and line[1] == 'tcp:%s' % host_port:
IndexError: list index out of range

Locals:
  host_port : 58332
  line      : ['00b5216f896322e1']

Owner: jbudorick@chromium.org
I suspect that this is due to John's work in https://codereview.chromium.org/2400283004/

Comment 5 by eyaich@chromium.org, Oct 13 2016

Failing also on Android One Perf (2) for 

1) blink_perf.shadow_dom.reference -- index out of range
2)  oilpan_gc_times.key_silk_cases.reference -- index out of range
3)  oilpan_gc_times.sync_scroll.key_mobile_sites_smooth -- index out of range

See most recent failure of these tests on recent build page: 

https://uberchromegw.corp.google.com/i/chromium.perf/builders/Android%20One%20Perf%20%282%29/builds/4452

Comment 6 by eyaich@chromium.org, Oct 13 2016

Also failing on Android Nexus5X WebView Perf (1) with same exception


Failing builder page:
https://uberchromegw.corp.google.com/i/chromium.perf/builders/Android%20Nexus5X%20WebView%20Perf%20%281%29/builds/2138

Comment 7 by eyaich@chromium.org, Oct 14 2016

Cc: jbudorick@chromium.org
Owner: rnep...@chromium.org
Project Member

Comment 8 by bugdroid1@chromium.org, Oct 14 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/3859ae3ca6c3e20ea450bdf022454f430ac1c56a

commit 3859ae3ca6c3e20ea450bdf022454f430ac1c56a
Author: catapult-deps-roller <catapult-deps-roller@chromium.org>
Date: Fri Oct 14 19:22:13 2016

Roll src/third_party/catapult/ fcf833066..baf2eab47 (2 commits).

https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/fcf8330661f6..baf2eab473e1

$ git log fcf833066..baf2eab47 --date=short --no-merges --format='%ad %ae %s'
2016-10-14 rnephew [Android] Check list length before printing forwarder information.
2016-10-14 alexandermont Update power metrics based on new design document.

BUG= 655653 

CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel
TBR=catapult-sheriff@chromium.org

Review-Url: https://codereview.chromium.org/2416283002
Cr-Commit-Position: refs/heads/master@{#425424}

[modify] https://crrev.com/3859ae3ca6c3e20ea450bdf022454f430ac1c56a/DEPS

Index error when unmapping the forwarder no longer happens with my fix. Still need to look at the new logging message and decide if there is more work to do there, but it looks reasonable.

It before was dying when there were lines that were only device IDs instead of a device ID port tuple.

John, do you think its safe to just ignore lines in that format?

WARNING:root:ADB forwarder list output format unexpected. Expected format "<device id> tcp:<host port> but got:
['00bc9aad38a6c29']

See CL above for more context. 

I think it's ok to ignore that, but I'd be interested in the entire contents of the ForwardList output in that case.
Project Member

Comment 11 by bugdroid1@chromium.org, Oct 22 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/99557feac28b32cd281951050b9b7d2397534e31

commit 99557feac28b32cd281951050b9b7d2397534e31
Author: catapult-deps-roller <catapult-deps-roller@chromium.org>
Date: Sat Oct 22 01:09:34 2016

Roll src/third_party/catapult/ fe41618f8..b7d8fe883 (8 commits).

https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/fe41618f8d39..b7d8fe883c16

$ git log fe41618f8..b7d8fe883 --date=short --no-merges --format='%ad %ae %s'
2016-10-21 bpastene devil: Add some logging to device_monitor.
2016-10-21 jbudorick Revert of [devil] Roll to ADB 1.0.36. (patchset #1 id:1 of https://codereview.chromium.org/2436373002/ )
2016-10-21 aiolos Add solidSelectedDots option to chart_series config.
2016-10-21 benjhayden Set text-overflow:ellipsis on Name column cells in histogram-set-table.
2016-10-21 benjhayden Fix and test metrics side panel.
2016-10-21 rnephew [Android][Telemetry] Show output of adb.ForwardList when error detected.
2016-10-21 rnephew [Devil] Fix more forwarder logging.
2016-10-21 jbudorick [devil] Roll to ADB 1.0.36.

BUG=649481, 623989 , 655653 , 655625 , 623989 

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, see:
http://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium#TOC-Failures-due-to-DEPS-rolls

CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel
TBR=catapult-sheriff@chromium.org

Review-Url: https://chromiumcodereview.appspot.com/2445473002
Cr-Commit-Position: refs/heads/master@{#426940}

[modify] https://crrev.com/99557feac28b32cd281951050b9b7d2397534e31/DEPS

Status: Fixed (was: Assigned)
out of range problem no longer happens.

Sign in to add a comment