Add Resource Shrinking to our Android build rules |
||||||||||||||||
Issue descriptionWe currently don't have many unused resources, but we accomplish this by manually whitelist resources from support libraries / play services that we need. This manual whitelisting is a maintenance burden, so it would be better to have the build just remove unused resources for us. Both Gradle a Bazel have implementations of Resource Shrinking. Bazel's seems easier to re-use though: https://github.com/bazelbuild/bazel/blob/master/src/tools/android/java/com/google/devtools/build/android/BUILD See "ResourceShrinkerAction" In order to use it, we'll need to add it to third_party. Likely we'll want to put it under //third_party/bazel, since there are other useful things in bazel that we'll want over time (e.g. manifest merging, turbine) Concrete steps for an MVP: a) Add a GN arg to android_apk() for remove_unused_resources = true b) Set it for chrome_public_apk when is_java_debug=false c) Add a prebuilt jar for AndroidResourceMergingAction_deploy to //third_party/bazel (see #10) d) Replace android_apk's existing resource logic with logic that uses the resource merger e) Add a prebuilt jar for ResourceShrinkerAction_deploy (see #12) f) Run ResourceShrinkerAction to create our .ap_ file Steps for Super-Awesome v2: a) Run a first-pass ProGuard (see #7), which is used only by ResourceUsageAnalyzer b) Write a custom Main.java for ResourceUsageAnalyzer.java that just outputs the shrunk resources (rather than creating an .ap_) c) Produce an updated set of R.class files based on the shrank resources d) Continue on with normal apk building steps. Whether or not we want to go the path of running proguard twice is somewhat up for debate, as it currently takes 90 seconds for a single run (with -optimizationpasses=1)
,
Nov 24 2016
,
Feb 9 2017
,
Feb 13 2017
,
Feb 13 2017
,
Feb 14 2017
Confirmed with author of the action that this should be ready to use. Specific link to entry point: https://github.com/bazelbuild/bazel/blob/master/src/tools/android/java/com/google/devtools/build/android/ResourceShrinkerAction.java#L56
,
Feb 17 2017
Alright, having second thoughts again... I'm wondering if we could do even better (and not end up with null entries for unused strings) by:
1. Proguard with shrinking & optimizations, but without allowing inlining of resource IDs:
-dontobfuscate
-keepclassmembernames class **.R.* {
*;
}
This might also require non-final fields to prevent inlining.
2. Feed this into the tool to strip the resources
3. Swap out R.class files from the proguarded .jar with newly generated ones based on the stripped resources (now with final fields)
4. Run proguard on it again, this time with the normal flags.
We'd likely want to add a flag to allow skipping of running aapt as well here: resourceProcessor.processResources(
https://github.com/bazelbuild/bazel/blob/master/src/tools/android/java/com/google/devtools/build/android/ResourceShrinkerAction.java#L238
,
Feb 23 2017
,
Feb 23 2017
Spent some time doing a one-off of this to see what the savings are. What I did: 1. bazel build //.../ResourceShrinkerAction_deploy.jar 2. Use ApkTool on Monochrome.apk to create a merged res/ directory 3. Created the resource zip via: zip -r res res 4. Ran the resource shrinker: java -jar ../ResourceShrinkerAction_deploy.jar --aapt ../../third_party/android_tools/sdk/build-tools/24.0.2/aapt --annotationJar lib.java/android.interface.jar --androidJar ../../third_party/android_tools/sdk/platforms/android-24/android.jar --shrunkJar gen/clank/java/monochrome_apk/monochrome_apk.proguard.jar --resources apks/Monochrome/res.zip --rTxt ./gen/clank/java/monochrome_apk__process_resources_R.txt --primaryManifest ./gen/clank/java/monochrome_apk/AndroidManifest.xml --resourcePackages com.google.android.apps.chrome.internal,org.chromium.chrome,org.chromium.third_party.android.media,org.chromium.android_webview,org.chromium.components.web_contents_delegate_android,android.support.v7.mediarouter,android.support.design,org.chromium.content,android.support.v7.recyclerview,com.google.vr.cardboard,android.support.v7.gridlayout,org.chromium.ui,org.chromium.third_party.android,com.google.android.gms,android.support.v7.appcompat,org.chromium.components.autofill,com.google.android.webview --shrunkResources apks/minres.zip --shrunkResourceApk apks/minres.ap_ --rTxtOutput apks/minres.R.txt --log apks/minres.log resource packages was taken from ./gen/clank/java/monochrome_apk.build_config The result: $ ls -l apks/minres.ap_ ./gen/clank/java/monochrome_apk/monochrome_apk.apk_intermediates.ap_ -rw-r--r-- 1 agrieve eng 6334015 Dec 31 1969 apks/minres.ap_ -rw-r--r-- 1 agrieve eng 6618088 Feb 23 03:42 ./gen/clank/java/monochrome_apk/monochrome_apk.apk_intermediates.ap_ $ unzip -lv ./gen/clank/java/monochrome_apk/monochrome_apk.apk_intermediates.ap_ | grep arsc 3767524 Stored 3767524 0% 1980-12-31 19:00 1827801d resources.arsc $ unzip -lv apks/minres.ap_ | grep arsc 3624648 Stored 3624648 0% 1980-12-31 19:00 e9a03a82 resources.arsc I've attached the shrinker log, which lists which resources were removed (some of these are dead code and we should just delete!)
,
Feb 23 2017
We currently don't merge resources, and instead pass multiple directories to aapt, which does the merging when creating the .ap_ file. That's why I used ApkTool above. Rather than using apktool to do the resource merging, Bazel has: https://github.com/bazelbuild/bazel/blob/master/src/tools/android/java/com/google/devtools/build/android/AndroidResourceMergingAction.java The gradle version is not stand-alone at all :/ https://github.com/yeungeek/android-platform-tools-base/blob/694a9e2f0bf51b5707f0359520ccb853ac3027b4/sdk-common/src/main/java/com/android/ide/common/res2/MergedResourceWriter.java
,
May 9 2017
,
Jun 7 2017
The logic for determining used resources is: https://github.com/bazelbuild/bazel/blob/master/third_party/java/aosp_gradle_core/java/com/android/build/gradle/tasks/ResourceUsageAnalyzer.java It also contains most of the stripping logic.
,
Jun 7 2017
,
Jun 7 2017
,
Sep 4 2017
,
Sep 14 2017
,
Nov 7 2017
,
Nov 7 2017
,
Dec 15 2017
,
Feb 6 2018
,
Feb 8 2018
I worked with the AOSP version of ResourceUsageAnalyzer/Model and with aapt2 to allow chrome to use resource stripping. This required some changes to both (go/chrome-resource-stripping). A preliminary version of adding resource stripping the chrome build system, resulted in the following savings: Arround 95kB saved by removing unused resources: mheikal@mheikal0:~/code $ ll OldProguard.apk OptimizedOldProguard.apk -rw-r--r-- 1 mheikal eng 88613470 Feb 8 15:10 OldProguard.apk -rw-r--r-- 1 mheikal eng 88518593 Feb 8 15:16 OptimizedOldProguard.apk Using a newer version of proguard with Conditional keeps improves the resources savings by around 1.5k (on top of the savings in java): mheikal@mheikal0:~/code $ ll NewProguardKeep.apk OptimizedNewProguardKeep.apk -rw-r--r-- 1 mheikal eng 88604642 Feb 8 14:21 NewProguardKeep.apk -rw-r--r-- 1 mheikal eng 88508426 Feb 8 15:16 OptimizedNewProguardKeep.apk Compressing gaps in resources.arsc, ie. reissuing ids so that removed resources don't form holes in the file, provides a further 12k of savings. However, there was no simple way of doing this without recompiling the code again because of changing IDs. The new Proguard conditional keep rules only remove an extra 5 resources from the apk: mheikal@mheikal0:~/code $ diff OldProguardConfig.cfg NewProguardKeepConfig.cfg 2711c2711 < id/snackbar_action# --- > id/snackbar_action#remove 2716c2716 < id/snackbar_text# --- > id/snackbar_text#remove 5260c5260 < styleable/ForegroundLinearLayout# --- > styleable/ForegroundLinearLayout#remove 5294c5294 < styleable/RecycleListView# --- > styleable/RecycleListView#remove 5304c5304 < styleable/SnackbarLayout# --- > styleable/SnackbarLayout#remove
,
Feb 8 2018
CLs for this change: Aapt2: https://googleplex-android-review.git.corp.google.com/c/platform/frameworks/base/+/3454951 ResourceUsageAnalyzer/Model: https://googleplex-android-review.git.corp.google.com/c/platform/tools/base/+/3577670 Chrome build changes: https://chromium-review.googlesource.com/#/c/chromium/src/+/909724 |
||||||||||||||||
►
Sign in to add a comment |
||||||||||||||||
Comment 1 by agrieve@chromium.org
, Aug 10 2016