Investigate RAM reduction by ordering dex classes |
||
Issue descriptionMotivations: * .odex files are ~25mb on ART devices, and so can consume as much as 25mb of RAM. * bug 833312 showed an improvement of 2mb of RAM when we changed the dex ordering from hash-based to sequential. If the 2mb reduction was from better dex locality, then there is potential for even better locality. We'd want to test this out on ART with AOT: * For Android L & M, all odex is AOT upon install. * For N+, odex files are "quickened" upon install * On N, you can't force it to aot easily (so just avoid testing on N * For O+ the compile-dex command of the generated wrapper scripts can force it on. Here are three ideas for test orderings: 1) Use the main-dex class list to put all renderer/webview/gpu classes together. 2) Figure out via heap dump which classes are loaded during start-up, and put those together. 3) Run chrome for a bit, then put all used classes together (or unused ones - should have the same result). For any approach, the ordering will be imperfect because many classes cannot be renamed due to -keep rules. A few because they are listed in the AndroidManifest.xml, but many more because they have JNI in them. This is especially true for renderer classes, since the majority of classes needed by renderers are needed only because of JNI. For 1), renderers don't use much java, so it might be better to measure the savings on webview-using applications For 2), not sure which tool to use to dump classes, but pretty sure some exist. There are some tools for measuring & visualizing native code locality here: https://cs.chromium.org/chromium/src/tools/android/memdump/memdump.cc https://cs.chromium.org/chromium/src/tools/android/native_lib_memory/ These tools could likely be adapted a bit to track odex page cache residency.
,
Aug 22
,
Aug 22
Related notes: Many dex-related tools can now be built for the host: repo init -u persistent-https://googleplex-android.git.corp.google.com/platform/manifest -b master-art-host time repo sync -c -j128 . build/envsetup.sh lunch full-eng export SOONG_ALLOW_MISSING_DEPENDENCIES=true time m -j16 out/host/linux-x86/bin/{dexlayout,profman,etc} The android dex profile can be converted to text. On device, do the following to get text profile. profman --dump-classes-and-methods --profile-file=/data/misc/profiles/cur/0/com.google.android.apps.chrome/primary.prof --apk=/data/app/com.google.android.apps.chrome-*/base.apk --dex-location=base.apk > profile.txt Then on host, create one with the right version. Touch an empty primary.prof, then run the following: out/host/linux-x86/bin/profman --dex-location=base.apk --apk=/chromium/work/src/out-android/Official/apks/Monochrome.apk --create-profile-from=/chromium/dexlayout/profile.txt --reference-profile-file=/chromium/dexlayout/primary.prof The output is in primary.prof, which is an in/out parameter. This profile can be used by dexlayout as follows (note the dexlayout code must be modified to set options.update_checksum in order to get a valid dex file): ./out/host/linux-x86/bin/dexlayout -u -w /chromium/dexlayout/ -p /chromium/dexlayout/primary.prof ../work/src/out-android/Official/apks/Monochrome.apk > /chromium/dexlayout/stdout --> This produces /chromium/dexlayout/Monochrome.apk that's actually a dex file.
,
Sep 10
,
Sep 20
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/775dab783973a7d9f6a1b51093c39f30ed7e5871 commit 775dab783973a7d9f6a1b51093c39f30ed7e5871 Author: Matthew Cary <mattcary@chromium.org> Date: Thu Sep 20 06:19:03 2018 Android Build: add //third_party/android_build_tools/art. Adds ART build tools used for android builds, in particular dexlayout and associated profile management. Bug: 875276 Change-Id: I323e460dfca032a5b28a634d149d1f50c8563f3b Reviewed-on: https://chromium-review.googlesource.com/1218584 Reviewed-by: Grace Kloba <klobag@chromium.org> Reviewed-by: Robert Sesek <rsesek@chromium.org> Reviewed-by: agrieve <agrieve@chromium.org> Commit-Queue: Matthew Cary <mattcary@chromium.org> Cr-Commit-Position: refs/heads/master@{#592696} [modify] https://crrev.com/775dab783973a7d9f6a1b51093c39f30ed7e5871/DEPS [modify] https://crrev.com/775dab783973a7d9f6a1b51093c39f30ed7e5871/third_party/android_build_tools/README.chromium [add] https://crrev.com/775dab783973a7d9f6a1b51093c39f30ed7e5871/third_party/android_build_tools/art/NOTICE [add] https://crrev.com/775dab783973a7d9f6a1b51093c39f30ed7e5871/third_party/android_build_tools/art/OWNERS [add] https://crrev.com/775dab783973a7d9f6a1b51093c39f30ed7e5871/third_party/android_build_tools/art/README.chromium [add] https://crrev.com/775dab783973a7d9f6a1b51093c39f30ed7e5871/third_party/android_build_tools/art/cipd.yaml
,
Sep 27
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/61ee6366ac709b5f56a22c8a5a226ebeacdd62d4 commit 61ee6366ac709b5f56a22c8a5a226ebeacdd62d4 Author: Matthew Cary <mattcary@chromium.org> Date: Thu Sep 27 10:45:30 2018 Build: dexlayout build rules This adds dexlayout to the build system. The dexlayout binary is from a host build of ART (the Android RunTime) and will be populated by .DEPS. Bug: 875276 Change-Id: I1931bb2effe893b9e7962777b5e776e6e2490e8b Reviewed-on: https://chromium-review.googlesource.com/1199583 Reviewed-by: agrieve <agrieve@chromium.org> Commit-Queue: Matthew Cary <mattcary@chromium.org> Cr-Commit-Position: refs/heads/master@{#594658} [modify] https://crrev.com/61ee6366ac709b5f56a22c8a5a226ebeacdd62d4/build/android/gyp/dex.py [modify] https://crrev.com/61ee6366ac709b5f56a22c8a5a226ebeacdd62d4/build/config/android/internal_rules.gni [modify] https://crrev.com/61ee6366ac709b5f56a22c8a5a226ebeacdd62d4/build/config/android/rules.gni
,
Oct 2
The following revision refers to this bug: https://chromium.googlesource.com/catapult/+/59297c6f73a6485334542123ebb48c648cef676b commit 59297c6f73a6485334542123ebb48c648cef676b Author: Matthew Cary <mattcary@chromium.org> Date: Tue Oct 02 08:03:52 2018 Tracing: add java_base* executable memory metrics. This adds java_base_clean_resident and java_base_pss metrics to telemetry benchmarks, which measure clank java code memory footprint. They are parallel to the native_library* memory footprint metrics. Bug: chromium:875276 Change-Id: I1ad5a33d7ae2db2de6f67ee3acb6a236366d24b0 Reviewed-on: https://chromium-review.googlesource.com/1201585 Reviewed-by: Ethan Kuefner <eakuefner@chromium.org> Commit-Queue: Matthew Cary <mattcary@chromium.org> [modify] https://crrev.com/59297c6f73a6485334542123ebb48c648cef676b/tracing/tracing/model/vm_region_test.html [modify] https://crrev.com/59297c6f73a6485334542123ebb48c648cef676b/tracing/tracing/metrics/system_health/memory_metric_test.html [modify] https://crrev.com/59297c6f73a6485334542123ebb48c648cef676b/tracing/tracing/metrics/system_health/memory_metric.html [modify] https://crrev.com/59297c6f73a6485334542123ebb48c648cef676b/tracing/tracing/model/vm_region.html
,
Oct 2
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/a058ca00d98fae360d24b2d490afb13a8bd4f5c1 commit a058ca00d98fae360d24b2d490afb13a8bd4f5c1 Author: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com> Date: Tue Oct 02 10:18:03 2018 Roll src/third_party/catapult 95ed233ed1a3..59297c6f73a6 (1 commits) https://chromium.googlesource.com/catapult.git/+log/95ed233ed1a3..59297c6f73a6 git log 95ed233ed1a3..59297c6f73a6 --date=short --no-merges --format='%ad %ae %s' 2018-10-02 mattcary@chromium.org Tracing: add java_base* executable memory metrics. Created with: gclient setdep -r src/third_party/catapult@59297c6f73a6 The AutoRoll server is located here: https://autoroll.skia.org/r/catapult-autoroll Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, please contact the current sheriff, who should be CC'd on the roll, and stop the roller if necessary. CQ_INCLUDE_TRYBOTS=luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel BUG=chromium:875276 TBR=sullivan@chromium.org Change-Id: I0f47af7fffdfbc9c497d5cdce9c61dc82782c527 Reviewed-on: https://chromium-review.googlesource.com/1256077 Reviewed-by: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com> Commit-Queue: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com> Cr-Commit-Position: refs/heads/master@{#595781} [modify] https://crrev.com/a058ca00d98fae360d24b2d490afb13a8bd4f5c1/DEPS
,
Oct 4
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/ad43dc4037b42cab9c5469d093523476158585d1 commit ad43dc4037b42cab9c5469d093523476158585d1 Author: Matthew Cary <mattcary@chromium.org> Date: Thu Oct 04 14:29:52 2018 Android build: dexlayout build fixes. * Unnecessary debug art libraries are removed from dependancies. * Command-line flag dash vs underscore usage diverged from what the python script uses. Bug: 875276 Change-Id: Icb308a4ce62bdac4d6341952f6f088ad7aa201da Reviewed-on: https://chromium-review.googlesource.com/c/1261555 Reviewed-by: agrieve <agrieve@chromium.org> Commit-Queue: Matthew Cary <mattcary@chromium.org> Cr-Commit-Position: refs/heads/master@{#596658} [modify] https://crrev.com/ad43dc4037b42cab9c5469d093523476158585d1/build/config/android/internal_rules.gni
,
Oct 4
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/255c1a2054bf0b7679f7be46c222518dc7e5b574 commit 255c1a2054bf0b7679f7be46c222518dc7e5b574 Author: Mohamed Heikal <mheikal@chromium.org> Date: Thu Oct 04 20:41:03 2018 [Android] Android profile obfuscator/deobfuscator This cl creates a script that can parse an obfuscated android text profile and using the proguard mapping file as well as the apk it can deobfuscate the profile. It can also take an unobfuscated profile and then using a proguard mapping file + apk, it can obfuscate the profile accordingly. Bug: 875276 Change-Id: I065b5360288af80f0b7c1ae027f8ff88caba805c Reviewed-on: https://chromium-review.googlesource.com/c/1237179 Commit-Queue: Mohamed Heikal <mheikal@chromium.org> Reviewed-by: Matthew Cary <mattcary@chromium.org> Reviewed-by: agrieve <agrieve@chromium.org> Cr-Commit-Position: refs/heads/master@{#596838} [modify] https://crrev.com/255c1a2054bf0b7679f7be46c222518dc7e5b574/build/android/PRESUBMIT.py [add] https://crrev.com/255c1a2054bf0b7679f7be46c222518dc7e5b574/build/android/convert_dex_profile.py [add] https://crrev.com/255c1a2054bf0b7679f7be46c222518dc7e5b574/build/android/convert_dex_profile_tests.py
,
Dec 12
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/9e10a67a866eab3bda6b25c9aec9705d3c898874 commit 9e10a67a866eab3bda6b25c9aec9705d3c898874 Author: Matthew Cary <mattcary@chromium.org> Date: Wed Dec 12 12:31:58 2018 [Android] Dexlayout with obfuscated profiles Extends the dexlayout build path with profile obfuscation. This hooks build/android/gyp/dex.py into the recent convert_dex_profile.py, and makes the necessary changes to GN to pass the proguard mapping to the dex pass. Bug: 875276 Change-Id: Id595b17fdba69c7e2e2dcd252c467954a558c56f Reviewed-on: https://chromium-review.googlesource.com/c/1366315 Commit-Queue: Matthew Cary <mattcary@chromium.org> Reviewed-by: agrieve <agrieve@chromium.org> Cr-Commit-Position: refs/heads/master@{#615863} [modify] https://crrev.com/9e10a67a866eab3bda6b25c9aec9705d3c898874/build/android/convert_dex_profile.py [modify] https://crrev.com/9e10a67a866eab3bda6b25c9aec9705d3c898874/build/android/convert_dex_profile_tests.py [modify] https://crrev.com/9e10a67a866eab3bda6b25c9aec9705d3c898874/build/android/gyp/dex.py [modify] https://crrev.com/9e10a67a866eab3bda6b25c9aec9705d3c898874/build/android/gyp/dex.pydeps [modify] https://crrev.com/9e10a67a866eab3bda6b25c9aec9705d3c898874/build/config/android/internal_rules.gni
,
Dec 14
The following revision refers to this bug: https://chrome-internal.googlesource.com/clank/internal/apps/+/9d8d570ffecd16b9cba12af7aa056222bf5a7f16 commit 9d8d570ffecd16b9cba12af7aa056222bf5a7f16 Author: Matthew Cary <mattcary@google.com> Date: Fri Dec 14 15:00:40 2018 |
||
►
Sign in to add a comment |
||
Comment 1 by pasko@chromium.org
, Aug 17