List perf tests are failing for Android bots |
|||||
Issue description
John, can you take a look at this? I also found that this make the list perf test smoke failed after 2-3 hundred seconds in telemetry_perf_unittest, bumping the test's total cycle time.
Log:
(WARNING) 2016-08-29 17:19:28,418 android_browser_finder.UpdateExecutableIfNeeded:147 Installing /b/rr/tmpuF8QpA/w/src/out/Release/apks/ChromePublic.apk on device if needed.
(WARNING) 2016-08-29 17:19:36,547 android_browser_finder.UpdateExecutableIfNeeded:147 Installing /b/rr/tmpuF8QpA/w/src/third_party/catapult/catapult_base/catapult_base/bin/reference_build/android_l_armeabi-v7a/Chrome.apk on device if needed.
(CRITICAL) 2016-08-29 17:20:13,154 timeout_retry.Run:172 (TimeoutThread-1-for-MainThread) Exception on Install(ZX1G22K86J, /b/rr/tmpuF8QpA/w/src/third_party/catapult/catapult_base/catapult_base/bin/reference_build/android_l_armeabi-v7a/Chrome.apk, retries=3, timeout=120), attempt 1 of 4: AdbCommandFailedError(u'(device: ZX1G22K86J) adb install /b/rr/tmpuF8QpA/w/src/third_party/catapult/catapult_base/catapult_base/bin/reference_build/android_l_armeabi-v7a/Chrome.apk: failed and output:\n- 2523 KB/s (87069729 bytes in 33.697s)\n- \tpkg: /data/local/tmp/Chrome.apk\n- Failure [INSTALL_FAILED_NO_MATCHING_ABIS]\n',)
(CRITICAL) 2016-08-29 17:20:47,858 timeout_retry.Run:172 (TimeoutThread-2-for-MainThread) Exception on Install(ZX1G22K86J, /b/rr/tmpuF8QpA/w/src/third_party/catapult/catapult_base/catapult_base/bin/reference_build/android_l_armeabi-v7a/Chrome.apk, retries=3, timeout=120), attempt 2 of 4: AdbCommandFailedError(u'(device: ZX1G22K86J) adb install /b/rr/tmpuF8QpA/w/src/third_party/catapult/catapult_base/catapult_base/bin/reference_build/android_l_armeabi-v7a/Chrome.apk: failed and output:\n- 2683 KB/s (87069729 bytes in 31.690s)\n- \tpkg: /data/local/tmp/Chrome.apk\n- Failure [INSTALL_FAILED_NO_MATCHING_ABIS]\n',)
(CRITICAL) 2016-08-29 17:21:25,866 timeout_retry.Run:172 (TimeoutThread-3-for-MainThread) Exception on Install(ZX1G22K86J, /b/rr/tmpuF8QpA/w/src/third_party/catapult/catapult_base/catapult_base/bin/reference_build/android_l_armeabi-v7a/Chrome.apk, retries=3, timeout=120), attempt 3 of 4: AdbCommandFailedError(u'(device: ZX1G22K86J) adb install /b/rr/tmpuF8QpA/w/src/third_party/catapult/catapult_base/catapult_base/bin/reference_build/android_l_armeabi-v7a/Chrome.apk: failed and output:\n- 2421 KB/s (87069729 bytes in 35.114s)\n- \tpkg: /data/local/tmp/Chrome.apk\n- Failure [INSTALL_FAILED_NO_MATCHING_ABIS]\n',)
Traceback (most recent call last):
<module> at /b/rr/tmpuF8QpA/w/src/tools/perf/run_benchmark:22
sys.exit(main())
main at /b/rr/tmpuF8QpA/w/src/tools/perf/run_benchmark:19
return benchmark_runner.main(config, [trybot_command.Trybot])
main at /b/rr/tmpuF8QpA/w/src/third_party/catapult/telemetry/telemetry/benchmark_runner.py:445
return command_instance.Run(options)
Run at /b/rr/tmpuF8QpA/w/src/third_party/catapult/telemetry/telemetry/benchmark_runner.py:160
possible_reference_browser = browser_finder.FindBrowser(args)
Cacher at /b/rr/tmpuF8QpA/w/src/third_party/catapult/telemetry/telemetry/decorators.py:38
cacher.__cache[key] = obj(*args, **kwargs)
FindBrowser at /b/rr/tmpuF8QpA/w/src/third_party/catapult/telemetry/telemetry/internal/browser/browser_finder.py:124
chosen_browser.UpdateExecutableIfNeeded()
Cacher at /b/rr/tmpuF8QpA/w/src/third_party/catapult/telemetry/telemetry/decorators.py:38
cacher.__cache[key] = obj(*args, **kwargs)
UpdateExecutableIfNeeded at /b/rr/tmpuF8QpA/w/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome/android_browser_finder.py:148
self.platform.InstallApplication(self._local_apk)
InstallApplication at /b/rr/tmpuF8QpA/w/src/third_party/catapult/telemetry/telemetry/core/platform.py:209
return self._platform_backend.InstallApplication(application)
InstallApplication at /b/rr/tmpuF8QpA/w/src/third_party/catapult/telemetry/telemetry/internal/platform/android_platform_backend.py:367
self._device.Install(application)
timeout_retry_wrapper at /b/rr/tmpuF8QpA/w/src/third_party/catapult/devil/devil/android/decorators.py:57
retry_if_func=retry_if_func)
Run at /b/rr/tmpuF8QpA/w/src/third_party/catapult/devil/devil/utils/timeout_retry.py:157
error_log_func=error_log_func)
JoinAll at /b/rr/tmpuF8QpA/w/src/third_party/catapult/devil/devil/utils/reraiser_thread.py:186
self._JoinAll(watcher, timeout)
_JoinAll at /b/rr/tmpuF8QpA/w/src/third_party/catapult/devil/devil/utils/reraiser_thread.py:158
thread.ReraiseIfException()
run at /b/rr/tmpuF8QpA/w/src/third_party/catapult/devil/devil/utils/reraiser_thread.py:81
self._ret = self._func(*self._args, **self._kwargs)
<lambda> at /b/rr/tmpuF8QpA/w/src/third_party/catapult/devil/devil/utils/timeout_retry.py:150
child_thread = reraiser_thread.ReraiserThread(lambda: func(*args, **kwargs),
impl at /b/rr/tmpuF8QpA/w/src/third_party/catapult/devil/devil/android/decorators.py:47
return f(*args, **kwargs)
Install at /b/rr/tmpuF8QpA/w/src/third_party/catapult/devil/devil/android/device_utils.py:643
reinstall=reinstall, permissions=permissions)
_InstallInternal at /b/rr/tmpuF8QpA/w/src/third_party/catapult/devil/devil/android/device_utils.py:729
base_apk.path, reinstall=reinstall, allow_downgrade=allow_downgrade)
Install at /b/rr/tmpuF8QpA/w/src/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py:682
cmd, output, device_serial=self._device_serial)
AdbCommandFailedError: (device: ZX1G22K86J) adb install /b/rr/tmpuF8QpA/w/src/third_party/catapult/catapult_base/catapult_base/bin/reference_build/android_l_armeabi-v7a/Chrome.apk: failed and output:
- 2206 KB/s (87069729 bytes in 38.534s)
- pkg: /data/local/tmp/Chrome.apk
- Failure [INSTALL_FAILED_NO_MATCHING_ABIS]
Locals:
allow_downgrade : False
apk_path : u'/b/rr/tmpuF8QpA/w/src/third_party/catapult/catapult_base/catapult_base/bin/reference_build/android_l_armeabi-v7a/Chrome.apk'
cmd : ['install', u'/b/rr/tmpuF8QpA/w/src/third_party/catapult/catapult_base/catapult_base/bin/reference_build/android_l_armeabi-v7a/Chrome.apk']
forward_lock : False
output : '2206 KB/s (87069729 bytes in 38.534s)\n\tpkg: /data/local/tmp/Chrome.apk\r\nFailure [INSTALL_FAILED_NO_MATCHING_ABIS]\r\n'
reinstall : False
retries : 2
sd_card : False
timeout : 120
step returned non-zero exit code: 1
,
Aug 30 2016
This appears to be limited to the S5? If that's the case, this isn't P0.
,
Aug 30 2016
and +aiolos because this is appears to be due to the reference build update. It first appeared on the S5 w/ the catapult roll that brought in that change: S5 (1): https://build.chromium.org/p/chromium.perf/builders/Android%20Galaxy%20S5%20Perf%20%281%29/builds/3667 S5 (2): https://build.chromium.org/p/chromium.perf/builders/Android%20Galaxy%20S5%20Perf%20%282%29/builds/3301 S5 (3): https://build.chromium.org/p/chromium.perf/builders/Android%20Galaxy%20S5%20Perf%20%283%29/builds/3082
,
Aug 30 2016
This is also happening on Nexus 6: https://build.chromium.org/p/chromium.perf/builders/Android%20Nexus6%20Perf%20%281%29/builds/3904
,
Aug 30 2016
Thanks for your diagnose, I revert aiolos's CL: https://codereview.chromium.org/2288853004/
,
Aug 30 2016
although the reference build *shouldn't* affect this afaict. :/
,
Aug 30 2016
Based on the stack, the script fails at the step of listing available benchmarks for reference browser:
"
Run at /b/rr/tmpuF8QpA/w/src/third_party/catapult/telemetry/telemetry/benchmark_runner.py:160
possible_reference_browser = browser_finder.FindBrowser(args)
"
At this step, browser_finder will try to install the reference APK on the target device. I am guessing that it's failing because the updated reference APK doesn't match the architecture of the device.
,
Aug 30 2016
Emily: another reason why the model of generating the tests statically is better.
,
Aug 30 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/f1b0eb09a58f749c42838aaa59fa705195f7d2ae commit f1b0eb09a58f749c42838aaa59fa705195f7d2ae Author: catapult-deps-roller <catapult-deps-roller@chromium.org> Date: Tue Aug 30 06:01:54 2016 Roll src/third_party/catapult/ 0672ab2c7..14f70933a (10 commits). https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/0672ab2c7c2f..14f70933ab02 $ git log 0672ab2c7..14f70933a --date=short --no-merges --format='%ad %ae %s' 2016-08-29 nednguyen Revert of Update android stable chrome reference builds. (patchset #1 id:1 of https://codereview.chromium.org/2283943002/ ) 2016-08-29 sullivan Disable notifications on deleted tests from mapreduce. 2016-08-29 oysteine Fixed a few Closure errors 2016-08-29 eakuefner [TBMv2] Add test for adding value from dict with sample diagnostic 2016-08-29 nednguyen Improve list_telemetry_unittests script to also output test's total time 2016-08-29 afakhry Disable WebviewInExtensionTest.testWebviewInExtension on all platforms 2016-08-29 nednguyen Fix csv_pivot_table_output_formatter_unittest's flakiness 2016-08-29 benjhayden Fix tracing_metric_test.diagnostics. 2016-08-29 benjhayden Migrate hazardMetric from ScalarNumeric to Histogram 2016-08-29 benjhayden Migrate tracing_metric from ScalarNumeric to Histograms BUG= 642202 ,641934,508325, 642145 TBR=catapult-sheriff@chromium.org Review-Url: https://codereview.chromium.org/2293773002 Cr-Commit-Position: refs/heads/master@{#415194} [modify] https://crrev.com/f1b0eb09a58f749c42838aaa59fa705195f7d2ae/DEPS
,
Aug 30 2016
Looks like the revert fix it: https://build.chromium.org/p/chromium.perf/builders/Android%20Nexus6%20Perf%20%281%29/builds/3921 @Kari: please watch out for the architecture of chrome apk in the next ref build update CL. Not sure if there is a way for us to sanity check the supported architecture for a given apk without relying on a test device.
,
Sep 1 2016
Issue 642311 has been merged into this issue. |
|||||
►
Sign in to add a comment |
|||||
Comment 1 by jbudorick@chromium.org
, Aug 30 2016