New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 639538 link

Starred by 4 users

Issue metadata

Status: WontFix
Owner:
Closed: Sep 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 1
Type: Bug



Sign in to add a comment

telemetry_perf_unittests failing intermittently on linux_android_rel_ng

Project Member Reported by kbr@chromium.org, Aug 20 2016

Issue description

telemetry_perf_unittests is failing intermittently on the linux_android_rel_ng tryserver. This is causing legitimate CLs to fail the CQ and be retried. It may also be causing large numbers of retries on this configuration.

https://build.chromium.org/p/tryserver.chromium.android/builders/linux_android_rel_ng?numbuilds=200

Example failure from my CL tryjob:
https://build.chromium.org/p/tryserver.chromium.android/builders/linux_android_rel_ng/builds/126244

It looks like a test is failing and the retries aren't working, and that possibly, the test is hanging during the last retry.

This seems like a different issue than those being tracked in the most recently filed bugs about telemetry_perf_unittests, namely Issue 637104 and  Issue 637904  .


------------------------------------------------

Retrying failed tests (attempt #2 of 3)...

(INFO) 2016-08-19 16:59:13,559 pid=9714  cmd_helper._ValidateAndLogCommand:153  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
(INFO) 2016-08-19 16:59:13,567 pid=9714  cmd_helper._ValidateAndLogCommand:153  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
(INFO) 2016-08-19 16:59:13,577 pid=9714  cmd_helper._ValidateAndLogCommand:153  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
(INFO) 2016-08-19 16:59:13,597 pid=9714  cmd_helper._ValidateAndLogCommand:153  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
(INFO) 2016-08-19 16:59:13,609 pid=9714  cmd_helper._ValidateAndLogCommand:153  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
(INFO) 2016-08-19 16:59:13,618 pid=9714  cmd_helper._ValidateAndLogCommand:153  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
(INFO) 2016-08-19 16:59:13,628 pid=9714  cmd_helper._ValidateAndLogCommand:153  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
(INFO) 2016-08-19 16:59:13,637 pid=9714  cmd_helper._ValidateAndLogCommand:153  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
56 KB/s (3387 bytes in 0.059s)
[1/1] benchmarks.system_health_smoke_test.SystemHealthBenchmarkSmokeTest.system_health.memory_mobile.load:news:cnn failed unexpectedly 104.9018s:
  [ RUN      ] load:news:cnn
  Traceback (most recent call last):
    File "/b/swarm_slave/w/irLIKLu4/third_party/catapult/telemetry/telemetry/internal/story_runner.py", line 86, in _RunStoryAndProcessErrorIfNeeded
      state.RunStory(results)
    File "/b/swarm_slave/w/irLIKLu4/third_party/catapult/telemetry/telemetry/page/shared_page_state.py", line 295, in RunStory
      self._current_page.Run(self)
    File "/b/swarm_slave/w/irLIKLu4/third_party/catapult/telemetry/telemetry/page/__init__.py", line 99, in Run
      self.RunPageInteractions(action_runner)
    File "/b/swarm_slave/w/irLIKLu4/tools/perf/page_sets/system_health/system_health_story.py", line 82, in RunPageInteractions
      self._Measure(action_runner)
    File "/b/swarm_slave/w/irLIKLu4/tools/perf/page_sets/system_health/system_health_story.py", line 51, in _Measure
      action_runner.MeasureMemory(deterministic_mode=True)
    File "/b/swarm_slave/w/irLIKLu4/third_party/catapult/telemetry/telemetry/internal/actions/action_runner.py", line 138, in MeasureMemory
      assert dump_id, 'Unable to obtain memory dump'
  AssertionError: Unable to obtain memory dump
  
  [  FAILED  ] load:news:cnn (96887 ms)
  [  PASSED  ] 0 tests.
  [  FAILED  ] 1 test, listed below:
  [  FAILED  ]  load:news:cnn
  
  1 FAILED TEST
  
  
  View result at file:///b/swarm_slave/w/irLIKLu4/third_party/catapult/telemetry/telemetry/testing/results.html
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( pm list packages );echo %$?'
  Chose browser: PossibleAndroidBrowser(browser_type=android-chromium)
  Installing /b/swarm_slave/w/irLIKLu4/out/Release/apks/ChromePublic.apk on device if needed.
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/aapt dump xmltree /b/swarm_slave/w/irLIKLu4/out/Release/apks/ChromePublic.apk AndroidManifest.xml
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/md5sum_host /b/swarm_slave/w/irLIKLu4/out/Release/apks/ChromePublic.apk
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( p=org.chromium.chrome;if [[ "$(ps)" = *$p* ]]; then am force-stop $p; fi );echo %$?'
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( pm list packages );echo %$?'
  Chose browser: PossibleAndroidBrowser(browser_type=android-chromium)
  Installing /b/swarm_slave/w/irLIKLu4/out/Release/apks/ChromePublic.apk on device if needed.
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/aapt dump xmltree /b/swarm_slave/w/irLIKLu4/out/Release/apks/ChromePublic.apk AndroidManifest.xml
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/md5sum_host /b/swarm_slave/w/irLIKLu4/out/Release/apks/ChromePublic.apk
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( p=org.chromium.chrome;if [[ "$(ps)" = *$p* ]]; then am force-stop $p; fi );echo %$?'
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( pm list packages );echo %$?'
  Chose browser: PossibleAndroidBrowser(browser_type=android-chromium)
  Installing /b/swarm_slave/w/irLIKLu4/out/Release/apks/ChromePublic.apk on device if needed.
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/aapt dump xmltree /b/swarm_slave/w/irLIKLu4/out/Release/apks/ChromePublic.apk AndroidManifest.xml
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/md5sum_host /b/swarm_slave/w/irLIKLu4/out/Release/apks/ChromePublic.apk
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( p=org.chromium.chrome;if [[ "$(ps)" = *$p* ]]; then am force-stop $p; fi );echo %$?'
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( pm list packages );echo %$?'
  Chose browser: PossibleAndroidBrowser(browser_type=android-chromium)
  Installing /b/swarm_slave/w/irLIKLu4/out/Release/apks/ChromePublic.apk on device if needed.
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/aapt dump xmltree /b/swarm_slave/w/irLIKLu4/out/Release/apks/ChromePublic.apk AndroidManifest.xml
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/md5sum_host /b/swarm_slave/w/irLIKLu4/out/Release/apks/ChromePublic.apk
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( p=org.chromium.chrome;if [[ "$(ps)" = *$p* ]]; then am force-stop $p; fi );echo %$?'
  Test certificate authority installed on target platform.
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( /data/local/tmp/profilers/memtrack_helper -d );echo %$?'
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( echo -n '"'"'{"trace_config": {"excluded_categories": ["*"], "included_categories": ["disabled-by-default-memory-infra"], "record_mode": "record-as-much-as-possible"}}'"'"' > /data/local/chrome-trace-config.json );echo %$?'
  Starting Web-Page-Replay: ['/usr/bin/python', '/b/swarm_slave/w/irLIKLu4/third_party/catapult/telemetry/third_party/web-page-replay/replay.py', '--host=127.0.0.1', '--port=0', '--ssl_port=0', '--no-dns_forwarding', '--use_closest_match', '--log_level=warning', '--should_generate_certs', '--https_root_ca_cert_path=/b/swarm_slave/w/itPLji67/tmpdgPD3x/testca.pem', u'/b/swarm_slave/w/irLIKLu4/tools/perf/page_sets/data/system_health_mobile_006.wpr']
  WPR ports: {}
  WPR ports: {'http': 54349, 'https': 34706}
  Forwarding using commands: [[u'--adb=/b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb', '--serial-id=03d12eac006adbc4', '--map', '0', '54349'], [u'--adb=/b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb', '--serial-id=03d12eac006adbc4', '--map', '0', '34706']]
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/forwarder_host --adb=/b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb --serial-id=03d12eac006adbc4 --map 0 54349
  Forwarding device port: 40710 to host port: 54349.
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/forwarder_host --adb=/b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb --serial-id=03d12eac006adbc4 --map 0 34706
  Forwarding device port: 58716 to host port: 34706.
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( p=org.chromium.chrome;if [[ "$(ps)" = *$p* ]]; then am force-stop $p; fi );echo %$?'
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( ls /root );echo %$?'
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( test -e /data/data/org.chromium.chrome/ );echo %$?'
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( TZ=utc ls -a -l /data/data/org.chromium.chrome/ );echo %$?'
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( rm -r /data/data/org.chromium.chrome/app_chrome /data/data/org.chromium.chrome/app_tabs /data/data/org.chromium.chrome/app_textures /data/data/org.chromium.chrome/cache /data/data/org.chromium.chrome/files /data/data/org.chromium.chrome/shared_prefs );echo %$?'
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 root
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 wait-for-device
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( test -d /storage/emulated/legacy );echo %$?'
  condition 'sd_card_ready' met (0.1s)
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( pm path android );echo %$?'
  condition 'pm_ready' met (0.7s)
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( getprop sys.boot_completed );echo %$?'
  condition 'boot_completed' met (0.7s)
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( su -c ls /root && ! ls /root );echo %$?'
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( stop mpdecision );echo %$?'
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( cd /sys/devices/system/cpu && for CPU in cpu0 cpu1 cpu2 cpu3; do echo 1 > "$CPU/online"; echo -n "%~%$?%~%"; done );echo %$?'
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( cd /sys/devices/system/cpu && for CPU in cpu0 cpu1 cpu2 cpu3; do cat "$CPU/online"; echo -n "%~%$?%~%"; done );echo %$?'
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( cd /sys/devices/system/cpu && for CPU in cpu0 cpu1 cpu2 cpu3; do test -e "$CPU/cpufreq/scaling_governor" && echo performance > "$CPU/cpufreq/scaling_governor"; echo -n "%~%$?%~%"; done );echo %$?'
  Successfully set cpufreq/scaling_governor to 'performance' on: cpu0 cpu1 cpu2 cpu3
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( cd /sys/devices/system/cpu && for CPU in cpu0 cpu1 cpu2 cpu3; do test -e "$CPU/cpufreq/scaling_max_freq" && echo 1190400 > "$CPU/cpufreq/scaling_max_freq"; echo -n "%~%$?%~%"; done );echo %$?'
  Successfully set cpufreq/scaling_max_freq to '1190400' on: cpu0 cpu1 cpu2 cpu3
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( echo -n 200000000 > /sys/class/kgsl/kgsl-3d0/max_gpuclk );echo %$?'
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( ndc resolver flushdefaultif );echo %$?'
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( logcat -c );echo %$?'
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( ( dumpsys window windows )>/data/local/tmp/temp_file-441d86811af57 );echo %$?'
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 pull /data/local/tmp/temp_file-441d86811af57 /b/swarm_slave/w/itPLji67/tmpO7y23C/tmp_ReadFileWithPull
  Trying to dismiss Error dialog for org.chromium.chrome
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( input keyevent 22 );echo %$?'
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell 'rm -f /data/local/tmp/temp_file-441d86811af57'
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( input keyevent 22 );echo %$?'
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( input keyevent 66 );echo %$?'
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( ( dumpsys window windows )>/data/local/tmp/temp_file-791461bab18ab );echo %$?'
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 pull /data/local/tmp/temp_file-791461bab18ab /b/swarm_slave/w/itPLji67/tmp36voKP/tmp_ReadFileWithPull
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell 'rm -f /data/local/tmp/temp_file-791461bab18ab'
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( ( dumpsys window windows )>/data/local/tmp/temp_file-a4869664fe9f8 );echo %$?'
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 pull /data/local/tmp/temp_file-a4869664fe9f8 /b/swarm_slave/w/itPLji67/tmpFVPg1E/tmp_ReadFileWithPull
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell 'rm -f /data/local/tmp/temp_file-a4869664fe9f8'
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( test -e /data/local/chrome-command-line );echo %$?'
  Android app/browser startup args: chrome --enable-memory-benchmarking --force-fieldtrial-params=DisallowFetchForDocWrittenScriptsInMainFrame.DocumentWriteEvaluatorGroup:disallowFetchForDocWrittenScriptsInMainFrameOnSlowConnections/false/disallowFetchForDocWrittenScriptsInMainFrame/false,UpdateMenuItem.Enabled:enable_update_menu_item/true,SafeBrowsingAndroid.Enabled:enabled/true/types_to_check/0%2C1%2C3%2C6%2C7%2C8%2C9%2C10%2C11%2C13%2C14%2C15,QUIC.Enabled:delay_tcp_race/true,ProgressBarAnimationAndroid.SmoothAnimation:progress-bar-animation/smooth,NetworkQualityEstimator.Enabled:Ethernet%2EDefaultMedianRTTMsec/106/Slow2G%2EThresholdMedianHttpRTTMsec/2000/4G%2EDefaultMedianKbps/1756/4G%2EDefaultMedianRTTMsec/159/None%2EDefaultMedianRTTMsec/272/correlation_logging_probability/0%2E001/2G%2EDefaultMedianRTTMsec/1337/3G%2EDefaultMedianKbps/658/3G%2EDefaultMedianRTTMsec/297/Bluetooth%2EDefaultMedianRTTMsec/135/Unknown%2EDefaultMedianKbps/1916/2G%2EDefaultMedianKbps/81/Unknown%2EDefaultMedianRTTMsec/121/None%2EDefaultMedianKbps/567/Bluetooth%2EDefaultMedianKbps/449/WiFi%2EDefaultMedianRTTMsec/168/Ethernet%2EDefaultMedianKbps/3263/WiFi%2EDefaultMedianKbps/2736/HalfLifeSeconds/60,MinidumpDirectoryObserver.ObserverExperimentGroup:Enabled/true,ParseHTMLOnMainThread.Enabled:parseHTMLOnMainThreadSyncTokenize/false/parseHTMLOnMainThreadCoalesceChunks/false,DataCompressionProxyLoFi.Enabled:rtt_msec/2000/hysteresis_period_seconds/120,ReportCertificateErrors.ShowAndPossiblySend:sendingThreshold/1%2E0,UMA_EnableCellularLogUpload.Enabled_wo_sampling:Sample_Probability/100/Uma_Quota/204800/Enabled/true/Optimize/true/Uma_Ratio/0%2E05,ContextualSearch.Expected:enable_translation/true/recent_scroll_duration_ms/300/enable_quick_answers/true/suppression_taps/10,LowPriorityIFrames.Enabled:lowPriorityIframes/true,VarationsServiceControl.Interval_30min:fetch_period_min/30,AutofillProfileOrderByFrecency.EnabledLimitTo3:limit/3,DataReductionProxyServerExperiments.Prefetch:exp/prefetch --enable-features=DocumentWriteEvaluator,ExpectCTReporting,ImportantSitesInCBD,NTPSnippets,DownloadsUi,MainFrameBeforeActivation,PassiveDocumentEventListeners,PhysicalWeb,ParseHTMLOnMainThread,NonValidatingReloadOnRefreshContent,NTPOfflinePages,PointerEvent,NetworkTimeServiceQuerying,ImeThread,ReadItLaterInMenu,NewAudioRenderingMixingStrategy,use-new-media-cache,NonValidatingReloadOnRefreshContentV2,WebRTC-EnableWebRtcEcdsa,PreconnectMore,token-binding --disable-features=AllBookmarks --force-fieldtrials=DisallowFetchForDocWrittenScriptsInMainFrame/DocumentWriteEvaluatorGroup/ExpectCTReporting/ExpectCTReportingEnabled/SdchPersistence/Enabled/ImportantSitesInCBD/DialogEnabled/UpdateMenuItem/Enabled/LimitSyncPassphrasePrompt/Enabled/MojoChannel/Enabled/NTPSnippets/Enabled/AutofillFieldMetadata/Enabled/SyncHttpContentCompression/Enabled/SafeBrowsingAndroid/Enabled/DownloadsUi/Enabled/QUIC/Enabled/SchedulerExpensiveTaskBlocking/Enabled/MainFrameBeforeActivation/Enabled/SafeBrowsingReportPhishingErrorLink/Enabled/ChromotingQUIC/Enabled/InvalidationsGCMUpstream/Enabled/TabSyncByRecency/Enabled/StrictSecureCookies/Enabled/ProgressBarAnimationAndroid/SmoothAnimation/AsyncDns/AsyncDnsA/PassiveDocumentEventListeners/Enabled/NetworkQualityEstimator/Enabled/PageRevisitInstrumentation/Enabled/PasswordBranding/SmartLockBrandingSavePromptOnly/ReaderModeUIFeedback/Show/AutodetectEncoding/Enabled/PhysicalWeb/Enabled/MinidumpDirectoryObserver/ObserverExperimentGroup/WebRTC-LocalIPPermissionCheck/Enabled/ReaderModeUI/AdaBoost/ParseHTMLOnMainThread/Enabled/PasswordManagerSettingsMigration/Enable/Precache/Enabled/NonValidatingReloadOnRefreshContent/Enabled/WebFontsInterventionV2/Disabled/OfflinePages/EnabledAsBookmarks/PointerEvent/Enabled/DataCompressionProxyLoFi/Enabled/DataReductionProxyFREPromo/Enabled/OfferUploadCreditCards/Enabled/EnableMediaThreadForMediaPlayback/Enabled/AllBookmarks/Disabled/NetworkTimeQueries/NetworkTimeQueriesEnabled/ImeThread/Enabled/ReadItLater/Enabled/ReportCertificateErrors/ShowAndPossiblySend/UnifiedMediaPipelineTrial/Enabled/NTPPopularSites/Enabled/V8CacheStrategiesForCacheStorage/default/UMA_EnableCellularLogUpload/Enabled_wo_sampling/NewAudioRenderingMixingStrategy/Enabled/use-new-media-cache/Enabled/NonValidatingReloadOnRefreshContentV2/Enabled/DataCompressionProxyPromoVisibility/Enabled/ContextualSearch/Expected/LowPriorityIFrames/Enabled/AndroidSessionNotifications/Disabled/AutofillClassifier/Enabled/WebRTC-PeerConnectionDTLS1.2/Enabled/DataReductionProxyUseQuic/Enabled/WebRTC-EnableWebRtcEcdsa/Enabled/VarationsServiceControl/Interval_30min/PreconnectMore/Enabled/ModerateBindingOnBackgroundTabCreation/Enabled/DataCompressionProxyDevRollout/Control/AutofillProfileOrderByFrecency/EnabledLimitTo3/TokenBinding/TokenBinding/DataReductionProxyServerExperiments/Prefetch --enable-net-benchmarking --metrics-recording-only --no-default-browser-check --no-first-run --enable-gpu-benchmarking --disable-background-networking --host-resolver-rules='MAP * 127.0.0.1,EXCLUDE localhost' --testing-fixed-http-port=40710 --testing-fixed-https-port=58716 --disable-component-extensions-with-background-pages --disable-default-apps --enable-remote-debugging --disable-fre --disable-external-intent-requests
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( test -d /data/local/chrome-command-line );echo %$?'
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 push /b/swarm_slave/w/itPLji67/tmpB9kCRT /data/local/chrome-command-line
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( am start -W -d about:blank -n org.chromium.chrome/com.google.android.apps.chrome.Main );echo %$?'
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 forward --no-rebind tcp:41926 localabstract:chrome_devtools_remote
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( rm -f /data/local/chrome-command-line );echo %$?'
  OS: android K
  Model: Nexus 5
  GPU device 0: VENDOR = 0x0 (Qualcomm), DEVICE = 0x0 (Adreno (TM) 330)
  GPU Attributes:
    adapter_luid        : 0.0
    amd_switchable      : False
    basic_info_state    : 1
    can_lose_context    : False
    context_info_state  : 1
    direct_rendering    : True
    driver_date         : 
    driver_vendor       : 
    driver_version      : 53.0
    encrypted_only      : False
    gl_extensions       : GL_AMD_compressed_ATC_texture GL_AMD_performance_monitor GL_AMD_program_binary_Z400 GL_EXT_debug_label GL_EXT_debug_marker GL_EXT_discard_framebuffer GL_EXT_robustness GL_EXT_texture_format_BGRA8888 GL_EXT_texture_type_2_10_10_10_REV GL_NV_fence GL_OES_compressed_ETC1_RGB8_texture GL_OES_depth_texture GL_OES_depth24 GL_OES_EGL_image GL_OES_EGL_image_external GL_OES_element_index_uint GL_OES_fbo_render_mipmap GL_OES_fragment_precision_high GL_OES_get_program_binary GL_OES_packed_depth_stencil GL_OES_depth_texture_cube_map GL_OES_rgb8_rgba8 GL_OES_standard_derivatives GL_OES_texture_3D GL_OES_texture_float GL_OES_texture_half_float GL_OES_texture_half_float_linear GL_OES_texture_npot GL_OES_vertex_half_float GL_OES_vertex_type_10_10_10_2 GL_OES_vertex_array_object GL_QCOM_alpha_test GL_QCOM_binning_control GL_QCOM_driver_control GL_QCOM_perfmon_global_mode GL_QCOM_extended_get GL_QCOM_extended_get2 GL_QCOM_tiled_rendering GL_QCOM_writeonly_rendering GL_EXT_sRGB GL_EXT_texture_filter_anisotropic GL_EXT_color_buffer_float GL_EXT_color_buffer_half_float GL_EXT_disjoint_timer_query 
    gl_renderer         : Adreno (TM) 330
    gl_reset_notification_strategy: 33362
    gl_vendor           : Qualcomm
    gl_version          : OpenGL ES 3.0 V@53.0 AU@  (CL@)
    gl_ws_extensions    : 
    gl_ws_vendor        : 
    gl_ws_version       : 
    in_process_gpu      : False
    initialization_time : 0.124642
    jpeg_decode_accelerator_supported: False
    lenovo_dcute        : False
    max_msaa_samples    : 4
    max_resolution_height: 2160
    max_resolution_width: 3840
    min_resolution_height: 0
    min_resolution_width: 0
    optimus             : False
    pixel_shader_version: 3.00
    process_crash_count : 0
    profile             : 10
    sandboxed           : False
    software_rendering  : False
    vertex_shader_version: 3.00
    video_decode_accelerator_flags: 7
  Feature Status:
    2d_canvas           : enabled
    flash_3d            : enabled
    flash_stage3d       : enabled
    flash_stage3d_baseline: enabled
    gpu_compositing     : enabled
    multiple_raster_threads: disabled_off
    native_gpu_memory_buffers: disabled_software
    rasterization       : enabled
    video_decode        : enabled
    video_encode        : unavailable_software
    vpx_decode          : enabled
    webgl               : enabled
  Driver Bug Workarounds:
    clear_uniforms_before_first_program_use
    disable_chromium_framebuffer_multisample
    disable_depth_texture
    disable_framebuffer_cmaa
    disable_program_cache
    force_cube_map_positive_x_allocation
    ignore_egl_sync_failures
    max_copy_texture_chromium_size_1048576
    max_texture_size_limit_4096
    scalarize_vec_and_mat_constructor_args
    unbind_egl_context_to_flush_driver_caches
    use_virtualized_gl_contexts
    wake_up_gpu_before_drawing
  PageCacheTemperature: any
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( sync );echo %$?'
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( echo -n 3 > /proc/sys/vm/drop_caches );echo %$?'
  *************** BROWSER STANDARD OUTPUT ***************
  Cannot get standard output on Android
  *********** END OF BROWSER STANDARD OUTPUT ************
  ********************* BROWSER LOG *********************
  No log file
  ***************** END OF BROWSER LOG ******************
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( /system/bin/screencap -p /data/local/tmp/temp_file-6580bb2124a5b.png );echo %$?'
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 pull /data/local/tmp/temp_file-6580bb2124a5b.png /b/swarm_slave/w/itPLji67/tmpiEP82d.png
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell 'rm -f /data/local/tmp/temp_file-6580bb2124a5b.png'
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( rm -f /data/local/chrome-trace-config.json );echo %$?'
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( ps | grep -F memtrack_helper; echo "PIPESTATUS: ${PIPESTATUS[@]}" );echo %$?'
  KillAll('memtrack_helper', ...) attempting to kill the following:
    28734 /data/local/tmp/profilers/memtrack_helper
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( kill -9 28734 );echo %$?'
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( ps | grep -F memtrack_helper; echo "PIPESTATUS: ${PIPESTATUS[@]}" );echo %$?'
  condition 'all_pids_killed' met (0.2s)
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( ps | grep -F org.chromium.chrome; echo "PIPESTATUS: ${PIPESTATUS[@]}" );echo %$?'
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( ls /root );echo %$?'
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( cd /sys/devices/system/cpu && for CPU in cpu0 cpu1 cpu2 cpu3; do cat "$CPU/online"; echo -n "%~%$?%~%"; done );echo %$?'
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( cd /sys/devices/system/cpu && for CPU in cpu0 cpu1 cpu2 cpu3; do test -e "$CPU/cpufreq/scaling_max_freq" && echo 2265600 > "$CPU/cpufreq/scaling_max_freq"; echo -n "%~%$?%~%"; done );echo %$?'
  Successfully set cpufreq/scaling_max_freq to '2265600' on: cpu0 cpu1 cpu2 cpu3
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( echo -n 450000000 > /sys/class/kgsl/kgsl-3d0/max_gpuclk );echo %$?'
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( cd /sys/devices/system/cpu && for CPU in cpu0 cpu1 cpu2 cpu3; do test -e "$CPU/cpufreq/scaling_governor" && echo ondemand > "$CPU/cpufreq/scaling_governor"; echo -n "%~%$?%~%"; done );echo %$?'
  Successfully set cpufreq/scaling_governor to 'ondemand' on: cpu0 cpu1 cpu2 cpu3
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( start mpdecision );echo %$?'
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 shell '( p=org.chromium.chrome;if [[ "$(ps)" = *$p* ]]; then am force-stop $p; fi );echo %$?'
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 forward --list
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 03d12eac006adbc4 forward --remove tcp:41926
  Undo forwarding using command: [u'--adb=/b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb', '--serial-id=03d12eac006adbc4', '--unmap', '40710']
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/forwarder_host --adb=/b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb --serial-id=03d12eac006adbc4 --unmap 40710
  Undo forwarding using command: [u'--adb=/b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb', '--serial-id=03d12eac006adbc4', '--unmap', '58716']
  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/forwarder_host --adb=/b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb --serial-id=03d12eac006adbc4 --unmap 58716
  Try printing formatted exception: None None None
  
  Traceback (most recent call last):
    RunBenchmark at /b/swarm_slave/w/irLIKLu4/third_party/catapult/telemetry/telemetry/internal/story_runner.py:318
      benchmark.ShouldTearDownStateAfterEachStorySetRun())
    Run at /b/swarm_slave/w/irLIKLu4/third_party/catapult/telemetry/telemetry/internal/story_runner.py:227
      _RunStoryAndProcessErrorIfNeeded(story, results, state, test)
    _RunStoryAndProcessErrorIfNeeded at /b/swarm_slave/w/irLIKLu4/third_party/catapult/telemetry/telemetry/internal/story_runner.py:86
      state.RunStory(results)
    RunStory at /b/swarm_slave/w/irLIKLu4/third_party/catapult/telemetry/telemetry/page/shared_page_state.py:295
      self._current_page.Run(self)
    Run at /b/swarm_slave/w/irLIKLu4/third_party/catapult/telemetry/telemetry/page/__init__.py:99
      self.RunPageInteractions(action_runner)
    RunPageInteractions at /b/swarm_slave/w/irLIKLu4/tools/perf/page_sets/system_health/system_health_story.py:82
      self._Measure(action_runner)
    _Measure at /b/swarm_slave/w/irLIKLu4/tools/perf/page_sets/system_health/system_health_story.py:51
      action_runner.MeasureMemory(deterministic_mode=True)
    MeasureMemory at /b/swarm_slave/w/irLIKLu4/third_party/catapult/telemetry/telemetry/internal/actions/action_runner.py:138
      assert dump_id, 'Unable to obtain memory dump'
  AssertionError: Unable to obtain memory dump
  
  Locals:
    deterministic_mode : True
    dump_id            : None
    platform           : <telemetry.core.android_platform.AndroidPlatform object at 0x7fe7f65bb310>
  
  Traceback (most recent call last):
    File "/b/swarm_slave/w/irLIKLu4/tools/perf/benchmarks/system_health_smoke_test.py", line 74, in RunTest
      msg='Failed: %s' % benchmark_class)
  AssertionError: Failed: <class 'benchmarks.system_health.MobileMemorySystemHealth'>

Retrying failed tests (attempt #3 of 3)...

(INFO) 2016-08-19 17:00:58,549 pid=9714  cmd_helper._ValidateAndLogCommand:153  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
(INFO) 2016-08-19 17:00:58,558 pid=9714  cmd_helper._ValidateAndLogCommand:153  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
(INFO) 2016-08-19 17:00:58,568 pid=9714  cmd_helper._ValidateAndLogCommand:153  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
(INFO) 2016-08-19 17:00:58,579 pid=9714  cmd_helper._ValidateAndLogCommand:153  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
(INFO) 2016-08-19 17:00:58,589 pid=9714  cmd_helper._ValidateAndLogCommand:153  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
(INFO) 2016-08-19 17:00:58,598 pid=9714  cmd_helper._ValidateAndLogCommand:153  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
(INFO) 2016-08-19 17:00:58,608 pid=9714  cmd_helper._ValidateAndLogCommand:153  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
(INFO) 2016-08-19 17:00:58,617 pid=9714  cmd_helper._ValidateAndLogCommand:153  [host]> /b/swarm_slave/w/irLIKLu4/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
55 KB/s (3387 bytes in 0.059s)
+--------------------------------------------------------------------------------+
| End of shard 0  Pending: 1588.4s  Duration: 960.0s  Bot: build83-b4  Exit: -15 |
+--------------------------------------------------------------------------------+
Total duration: 960.0s

 

Comment 1 by kbr@chromium.org, Aug 20 2016

I'm not sure the problems on this bot are specific to Telemetry tests. The retry job failed content_browsertests and unit_tests:

https://build.chromium.org/p/tryserver.chromium.android/builders/linux_android_rel_ng/builds/126305

Comment 2 by kbr@chromium.org, Aug 20 2016

Cc: jbudorick@chromium.org
Cc: primiano@chromium.org petrcermak@chromium.org
Owner: perezju@chromium.org
+memory-infra folks
Petr, any idea why we could be missing memory dumps?

As a short term solution, we could also switch that assert to a warning.
It's difficult to tell without logging. From the output of the failing bot:

  *************** BROWSER STANDARD OUTPUT ***************
  Cannot get standard output on Android
  *********** END OF BROWSER STANDARD OUTPUT ************
  ********************* BROWSER LOG *********************
  No log file
  ***************** END OF BROWSER LOG ******************

I thought that logging was enabled (https://cs.chromium.org/chromium/src/tools/perf/benchmarks/system_health_smoke_test.py?l=112), but it turns out that it wasn't. Here's a fix: https://codereview.chromium.org/2267663003
Project Member

Comment 6 by bugdroid1@chromium.org, Aug 22 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/3d44ebe6c88dbce694e2cdce41c5bd12246ce1e4

commit 3d44ebe6c88dbce694e2cdce41c5bd12246ce1e4
Author: petrcermak <petrcermak@chromium.org>
Date: Mon Aug 22 13:41:12 2016

[system-health] Actually enable browser logging in smoke test

It turns out that the original patch
(https://codereview.chromium.org/2133093002) didn't actually enable
browser logging in the System Health smoke tests.

BUG= 623058 , 639538 

Review-Url: https://codereview.chromium.org/2267663003
Cr-Commit-Position: refs/heads/master@{#413437}

[modify] https://crrev.com/3d44ebe6c88dbce694e2cdce41c5bd12246ce1e4/tools/perf/benchmarks/system_health_smoke_test.py

Status: Assigned (was: Untriaged)

Comment 8 by rch@google.com, Aug 29 2016

FWIW, this was failing for me repeatedly over the weekend on

  https://codereview.chromium.org/2287073002/

This is a tiny CL in net which should have no impact on anything perf related (and only changes behavior on Mac)

Comment 9 by rch@chromium.org, Aug 29 2016

Cc: rch@chromium.org

Comment 11 by rch@chromium.org, Aug 29 2016

Ah, I see. Good point! Thanks. (Not sure how I failed to find that bug via crbug search)
Status: WontFix (was: Assigned)
There haven't been any more missing memory dumps in the past 200 builds, so closing now. Feel free to re-open if the issue arises again.

Sign in to add a comment