New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 681933 link

Starred by 3 users

Issue metadata

Status: Duplicate
Merged: issue 895194
Owner:
Closed: Oct 15
Cc:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 3
Type: Bug



Sign in to add a comment

Experiment with zipping packed relocations (100kb savings)

Project Member Reported by agrieve@chromium.org, Jan 17 2017

Issue description

libchrome.so on Android usee packed relocations to shrink the size of .rel.dyn from (as of now): 2,709,016 bytes down to 341,528

However, gzipping the packed relocation further shrinks them to just 22,936 (320kb savings)

One of the design goals of packed relocations is that unpacking is done in a streaming way. Gzip is compatible with this.

The biggest unknown to me is whether WebView / Monochrome will be able to take advantage of zipping these, as it would re-introduce dependence on our custom linker. One alternative to a custom linker is adding an entry to .init_array that performs the relocations (bundle the relocation code), but this is much more complex and might be incompatible with relocation sharing.


Packed relocations references:
https://android.googlesource.com/platform/bionic/+/refs/heads/master/tools/relocation_packer/
https://android.googlesource.com/platform/bionic/+/refs/heads/master/tools/relocation_packer/src/delta_encoder.h

Support for packed relocations was added directly to Android's linker in M:
https://android-review.googlesource.com/#/c/135550/

How I calculated savings:
Show size of .rel.dyn before packing:
readelf -S out/Release/libchrome.so

Show size of .rel.dyn after packing:
unzip out/Release/apks/Chrome.apk lib/armeabi-v7a/libchrome.so
readelf -S lib/armeabi-v7a/libchrome.so

Show size of it gzipped:
objdump -h lib/armeabi-v7a/libchrome.so | grep .rel.dyn | awk '{print "dd if='lib/armeabi-v7a/libchrome.so' of='dyn.rel' bs=1 count=$[0x" $3 "] skip=$[0x" $6 "]"}' | bash
ls -l dyn.rel


Assigning to torne@ to comment on whether this is an issue for webview.
 

Comment 1 by torne@chromium.org, Jan 18 2017

Owner: agrieve@chromium.org
WebView cannot use the custom linker. Monochrome therefore can't have any custom binary file format changes that aren't supported by the system linker.

Bundling the relocation code won't work either; relocation sharing is handled by the system linker before constructors are called, there's no chance to intervene.

Adding this to the system linker in future seems like the only option :(
Okay, that's what I was afraid of :/

So we could apply this for pre-monochrome via custom linker, but not for webview nor monochrome unless we add it to android's linker.

Cc: digit@chromium.org
digit - noticed your name in the crazy linker code... Any desire to work on this?
Labels: Performance-Browser
Labels: -apk-size binary-size
Labels: -binary-size Performance-Size
Updated number now that we've switched to clang. For libmonochrome.so, 

dyn.rel: 353496
dyn.rel.gz: 20412 (94% compression)


And for interest's sake, using zopfli gives:
dyn.rel.zopfli: 18179 (95% compression)
Another data point using lz4 compression:

$ ./lz4 -12 -B4 -BD dyn.rel dyn.rel.lz4
using blocks of size 64 KB
Compressed 373136 bytes into 28569 bytes ==> 7.66%

This is probably a better format to use since it decompresses ~10x faster.

Comment 10 by digit@google.com, Sep 18 2017

True, but this adds an lz4 decompressor to the linker. I hope this is less than 300 KiB, otherwise we have a problem. There is no such issue when using zlib which is part of the platform :)

Comment 11 by digit@google.com, Sep 18 2017

Actually, it looks like an lz4 decompressor can be very small, see the "LZ4 decompressor" section at:

http://create.stephan-brumme.com/smallz4/#numbers

Using non-streaming lz4hc (lower decode ram requirement):
 libchrome.so = 28764 (92% compression) (64k chunks)
 libchrome.so = 29874 (92% compression) (32k chunks)
 libchrome.so = 31065 (91.5% compression) (16k chunks)

Using lz4hc without Sleb128 encoding:
 libchrome.so = 38122 (89.5% compression) (64k chunks)

My guess is that the vast majority of ints fit into a single byte, which is why sleb128 is still beneficial even with compression.
Summary: Experiment with zipping packed relocations (100kb savings) (was: Experiment with zipping packed relocations (320kb savings))
The proposal for RELR relocations is here:
https://groups.google.com/d/msg/generic-abi/bX460iggiKg/Pi9aSwwABgAJ

For Chrome, this would shrink our .rel.dyn 181025 -> ~80kb, and improve start-up time (55ms to apply relocations vs 15ms on my Moto G first gen).

This shrinks the size of arm64 drastically more (by MBs), but not super relevant because... 

We would not be able to use this for Monochrome. Webview is always loaded using the system linker.
Cc: p...@chromium.org
Note: Biggest blocker here is getting it implemented in lld.
Status: Assigned (was: Available)
Mergedinto: 895194
Status: Duplicate (was: Assigned)

Sign in to add a comment