New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 831388 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Apr 2018
Cc:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 3
Type: Bug



Sign in to add a comment

Android test runner pushing unnecessary files to device

Project Member Reported by agrieve@chromium.org, Apr 10 2018

Issue description

Noticed this when looking at test logs to see if unnecessary work was being done:

For webview_instrumentation_test_apk, the following unneeded files are pushed:
/sdcard/chromium_tests_root/bin/stack_aw_net_test_support_apk
/sdcard/chromium_tests_root/third_party/android_platform/development/scripts/stack
/sdcard/chromium_tests_root/third_party/llvm-build/Release+Asserts/bin/llvm-symbolizer
/sdcard/chromium_tests_root/exe.unstripped/md5sum_bin
/sdcard/chromium_tests_root/md5sum_bin_host
/sdcard/chromium_tests_root/clang_x64/md5sum_bin
/sdcard/chromium_tests_root/icudtl.dat
/sdcard/chromium_tests_root/bin/stack_net_test_support_apk

What's even worse, is that with --enable-device-cache (default for incremental builds), they are pushed every time (caching is broken).

Note that md5sum_bin_host and icudtl.dat together are 27.5mb
 
Cc: bpastene@chromium.org
additional candidates for https://codesearch.chromium.org/chromium/src/build/android/pylib/utils/device_dependencies.py?rcl=0c1ebfdd04f18de027a8a00d413d24096df44715&l=11, perhaps.

we don't currently have a great way to distinguish in gn between runtime dependencies that need to be on the host vs those that need to be on the device; something like that would be nice here in the long term. (bpastene was asking about that recently for cros-related things.)
Project Member

Comment 2 by bugdroid1@chromium.org, Apr 11 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/catapult/+/b73d92a7b028416996568cfa329a77eede85663b

commit b73d92a7b028416996568cfa329a77eede85663b
Author: Andrew Grieve <agrieve@chromium.org>
Date: Wed Apr 11 14:25:03 2018

device_utils.PushChangedFiles: Fix caching logic when pushing files

For directories it was working fine, but for files the cache entries
were getting a trailing / appended to them, causing subsequent lookups
to always fail.

Bug:  chromium:831388 
Change-Id: I99fd179df6368b08a3d17742e27050839308a6ae
Reviewed-on: https://chromium-review.googlesource.com/1005568
Commit-Queue: agrieve <agrieve@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>

[modify] https://crrev.com/b73d92a7b028416996568cfa329a77eede85663b/devil/devil/android/device_utils.py
[modify] https://crrev.com/b73d92a7b028416996568cfa329a77eede85663b/devil/devil/android/device_utils_test.py

Project Member

Comment 3 by bugdroid1@chromium.org, Apr 12 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/267d66ce578fbf8d490cf8eb6e088c8cd720c5ee

commit 267d66ce578fbf8d490cf8eb6e088c8cd720c5ee
Author: catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Date: Thu Apr 12 05:59:38 2018

Roll src/third_party/catapult/ a22719853..cc64c0d81 (4 commits)

https://chromium.googlesource.com/catapult.git/+log/a22719853354..cc64c0d81aab

$ git log a22719853..cc64c0d81 --date=short --no-merges --format='%ad %ae %s'
2018-04-10 simonhatch Dashboard - Fix stats being generated in some cases
2018-04-11 eakuefner [TBMv2] Add HAD_FAILURES diagnostic and use it to avoid merging
2018-04-11 agrieve device_utils.PushChangedFiles: Fix caching logic when pushing files
2018-04-09 ulan Reduce noise of memory metric.

Created with:
  roll-dep src/third_party/catapult
BUG= chromium:831388 , chromium:826384 


The AutoRoll server is located here: https://catapult-roll.skia.org

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.


TBR=sullivan@chromium.org

Change-Id: I4216dad219b5d0fdaf0af49063866dae9fe9bba8
Reviewed-on: https://chromium-review.googlesource.com/1007325
Reviewed-by: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Commit-Queue: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#550036}
[modify] https://crrev.com/267d66ce578fbf8d490cf8eb6e088c8cd720c5ee/DEPS

Project Member

Comment 4 by bugdroid1@chromium.org, Apr 19 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/c143cf3c8e4fadbc616e696f0430efb0e7f8e158

commit c143cf3c8e4fadbc616e696f0430efb0e7f8e158
Author: Andrew Grieve <agrieve@chromium.org>
Date: Thu Apr 19 19:39:17 2018

Android: Exclude more files from being pushed to device

Bug:  831388 
Change-Id: If885e2205c1f120e0f718522bdfc7de03e125eab
Reviewed-on: https://chromium-review.googlesource.com/1006180
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: agrieve <agrieve@chromium.org>
Cr-Commit-Position: refs/heads/master@{#552119}
[modify] https://crrev.com/c143cf3c8e4fadbc616e696f0430efb0e7f8e158/build/android/pylib/utils/device_dependencies.py

Status: Fixed (was: Started)
Looks like the bots were happy with #4, so this is now fixed!

Sign in to add a comment