New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 760838 link

Starred by 3 users

Issue metadata

Status: Fixed
Owner:
Closed: Oct 2017
Cc:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 3
Type: Bug

Blocked on:
issue 775171

Blocking:
issue 742655



Sign in to add a comment

Implement Android relocation packing in lld

Project Member Reported by p...@chromium.org, Aug 31 2017

Issue description

lld's output is currently incompatible with the Android relocation packer as implemented because it expects the .rel.dyn section to be laid out early enough in the output file that it can easily resize it. This happens to be the 
case for gold, but it is not the case for lld because .rel.dyn is laid out between .rodata and .text:

libchrome.so  :
section                  size       addr
.ARM.exidx            2472608        340
.rodata               6134668    2472960
.ARM.extab              14184    8607628
.note.gnu.build-id         36    8621812
.dynsym                  6576    8621848
.gnu.version              822    8628424
.gnu.version_r             96    8629248
.hash                    3296    8629344
.dynstr                  4082    8632640
.rel.dyn              2838288    8636724
.rel.plt                 2840   11475012
.text                35492856   11481088
.plt                     5700   46973952
.data                   41676   46981120
.data.rel.ro          1915920   47026176
.init_array                40   48942096
.fini_array                 8   48942136
.dynamic                  240   48942144
.got                    52716   48942384
.got.plt                 1432   48995100
.bss                  1593728   49000448
.comment                  177          0
Total                50581989

and there may be relative relocations between the two sections that are not explicit in the output.

It may be possible to fix this problem in the relocation packer, but a far better solution would be to implement relocation packing in lld itself.
 
Project Member

Comment 1 by bugdroid1@chromium.org, Aug 31 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/1e3ea3b50cf7e9ad4145bf9f388ad0ac4fec147c

commit 1e3ea3b50cf7e9ad4145bf9f388ad0ac4fec147c
Author: Peter Collingbourne <pcc@chromium.org>
Date: Thu Aug 31 03:53:26 2017

Android: Disable relocation packing under lld for now.

lld's output is currently incompatible with the relocation packer
tool. We will probably want to implement relocation packing in lld
itself eventually.

Bug:  742655 ,  760838 
Change-Id: I91d12e3d29c31d7509e1e34f980f6ceab2746b4d
Reviewed-on: https://chromium-review.googlesource.com/644555
Commit-Queue: Peter Collingbourne <pcc@chromium.org>
Reviewed-by: Nico Weber <thakis@chromium.org>
Cr-Commit-Position: refs/heads/master@{#498748}
[modify] https://crrev.com/1e3ea3b50cf7e9ad4145bf9f388ad0ac4fec147c/build/config/android/rules.gni

Comment 2 by thakis@chromium.org, Aug 31 2017

Cool! What's the holdup for landing those patches? Not quite the right approach?

Comment 3 by ruiu@google.com, Aug 31 2017

If you have a pointer to a document for the relocation packer, can you share it with me? It looks like it compacts relocations by using different encoding scheme (it seems delta-encoding and leb128?), but I'm not sure if my understanding is correct.

Comment 4 by p...@chromium.org, Sep 1 2017

Cc: dimitry@chromium.org
I'm not aware of any specific docs other than the source code of the relocation packer, but maybe Dimitry has one.

https://android.googlesource.com/platform/bionic/+/refs/heads/master/tools/relocation_packer/src/
Cc: rahulchaudhry@chromium.org srivatsanr@google.com
we have been discussing reducing size of relocations for PIE executables with Android and the plan is to replace relocation_packer by the mechanims described in go/smallpie.  Please take a look. Our plan is to implement this in Gold and LLD.
I think the plan (please someone correct me if I'm wrong) is to use a hybrid of the approach described on go/smallpie and the leb128-based compression of relocation packer.

Sample implementation here:
https://android-review.googlesource.com/c/platform/bionic/+/487338

Comment 7 by p...@chromium.org, Oct 20 2017

So that document appears to describe a new relocation packing format. While I think that it's great that we're working on a new format, wouldn't we still need some level of support for the existing format if we want to be able to build .so's that can be shipped to older devices?
Cc: torne@chromium.org
For Chrome and ChromeModern, we use crazylinker to load the library, and so would just update that to the new format.

For Monochrome though, we use the system linker, so probably need to support the old format, unless we can shim in the new format by using a static initializer to apply the relocations rather than the system linker. I'm not sure if that will be incompatible with how webview implements relro sharing though... Torne, do you know?

Comment 9 by torne@chromium.org, Oct 20 2017

It's not compatible. The system linker handles relro sharing immediately after applying relocations; the static initializers haven't run yet. Monochrome also  relies on this when running as chrome, not just webview.

Comment 10 by p...@chromium.org, Oct 20 2017

Owner: p...@chromium.org
Status: Started (was: Untriaged)
Okay, so it sounds like we're going to need support for the old relocation format then. I'll start looking at that.

Comment 11 by p...@chromium.org, Oct 21 2017

Here's my implementation so far: https://reviews.llvm.org/D39152

It does a better job at compressing relative relocations than Android's own relocation packer. TL;DR: If I link Chromium for Android targeting ARM32 I get a .rel.dyn of size 174693 bytes, as compared to 371832 bytes with gold and the Android packer.
That's awesome!! Great work! This will also make it much easier to add smallpie relocation packing to lld when it's ready.
Cc: llozano@chromium.org
Cc: enh@google.com srhines@google.com
Project Member

Comment 15 by bugdroid1@chromium.org, Oct 27 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/9be1753f99a21934b64d5f9828ab755cb54b5184

commit 9be1753f99a21934b64d5f9828ab755cb54b5184
Author: Andrew Grieve <agrieve@chromium.org>
Date: Fri Oct 27 17:48:28 2017

Android: Use lld's relocation packing when use_lld=true

There is an outstanding patch to lld that adds support for android relocation
packing. This change works with this patch (https://reviews.llvm.org/D39152).

This also removes the gn args:
 * chrome_public_apk_use_chromium_linker
 * chrome_public_apk_load_library_from_apk
 * chrome_public_apk_use_relocation_packer

It doesn't look like they are used anywhere, and I don't think these
are knobs that we should encourage anyone to touch.

Bug:  760838 
Change-Id: Ie20ddd06e275942293c8e99bc2c3e4ffabe9f8f8
Reviewed-on: https://chromium-review.googlesource.com/737249
Reviewed-by: John Budorick <jbudorick@chromium.org>
Reviewed-by: Bo <boliu@chromium.org>
Commit-Queue: agrieve <agrieve@chromium.org>
Cr-Commit-Position: refs/heads/master@{#512228}
[modify] https://crrev.com/9be1753f99a21934b64d5f9828ab755cb54b5184/android_webview/BUILD.gn
[modify] https://crrev.com/9be1753f99a21934b64d5f9828ab755cb54b5184/build/android/gyp/create_test_runner_script.py
[modify] https://crrev.com/9be1753f99a21934b64d5f9828ab755cb54b5184/build/android/gyp/write_build_config.py
[modify] https://crrev.com/9be1753f99a21934b64d5f9828ab755cb54b5184/build/android/pylib/instrumentation/instrumentation_test_instance.py
[modify] https://crrev.com/9be1753f99a21934b64d5f9828ab755cb54b5184/build/android/pylib/symbols/stack_symbolizer.py
[modify] https://crrev.com/9be1753f99a21934b64d5f9828ab755cb54b5184/build/android/test_runner.py
[modify] https://crrev.com/9be1753f99a21934b64d5f9828ab755cb54b5184/build/android/tombstones.py
[modify] https://crrev.com/9be1753f99a21934b64d5f9828ab755cb54b5184/build/config/android/BUILD.gn
[modify] https://crrev.com/9be1753f99a21934b64d5f9828ab755cb54b5184/build/config/android/internal_rules.gni
[modify] https://crrev.com/9be1753f99a21934b64d5f9828ab755cb54b5184/build/config/android/rules.gni
[modify] https://crrev.com/9be1753f99a21934b64d5f9828ab755cb54b5184/chrome/android/BUILD.gn
[modify] https://crrev.com/9be1753f99a21934b64d5f9828ab755cb54b5184/chrome/android/chrome_public_apk_tmpl.gni

Comment 16 by p...@chromium.org, Oct 27 2017

Status: Fixed (was: Started)
My patch landed in LLVM in r316775.
Blockedon: 775171

Sign in to add a comment