New issue
Advanced search Search tips

Issue 870331 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Nov 29
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug



Sign in to add a comment

Clean up llvm bundle + minor refactoring

Project Member Reported by mmoroz@chromium.org, Aug 2

Issue description

There seem to be some libraries which no one needs, e.g. compiler-rt for Android on i686. Would be nice to remove those and use less globs with wildcards when collecting files for the bundle.
 
Project Member

Comment 1 by bugdroid1@chromium.org, Aug 3

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/dcc289d12efe6c37ce3920f9441dfe533598d144

commit dcc289d12efe6c37ce3920f9441dfe533598d144
Author: Max Moroz <mmoroz@chromium.org>
Date: Fri Aug 03 04:43:29 2018

Do not bundle compiler-rt libraries (ASan,UBSan,Profile) for Android i686.

Bug:  870331 
Change-Id: Icbf8bc191e4078bd3ee8fb7fe15e51caae59e372
Reviewed-on: https://chromium-review.googlesource.com/1160861
Reviewed-by: Nico Weber <thakis@chromium.org>
Commit-Queue: Max Moroz <mmoroz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#580453}
[modify] https://crrev.com/dcc289d12efe6c37ce3920f9441dfe533598d144/tools/clang/scripts/update.py

Project Member

Comment 2 by bugdroid1@chromium.org, Sep 26

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/c79bf2ea4cf65431dccb57cb2a44528c284645a1

commit c79bf2ea4cf65431dccb57cb2a44528c284645a1
Author: Max Moroz <mmoroz@chromium.org>
Date: Wed Sep 26 16:53:42 2018

Clang scripts: package only "fuzzer_no_main" runtime.

This requires that fuzz targets use "-fsanitize=fuzzer-no-link"
instead of "-fsanitize=fuzzer". All Chromium fuzz targets should
do this already; projects rolling in this change need to change
from the latter to the former.

Removes 1.7MB library on Linux, 1.5MB on Mac, 7.7MB on Windows

Bug:  870331 
Change-Id: I124207477d0e8338f0502a794200edcd5102bd45
Reviewed-on: https://chromium-review.googlesource.com/1244802
Commit-Queue: Max Moroz <mmoroz@chromium.org>
Reviewed-by: Nico Weber <thakis@chromium.org>
Reviewed-by: Jonathan Metzman <metzman@chromium.org>
Cr-Commit-Position: refs/heads/master@{#594350}
[modify] https://crrev.com/c79bf2ea4cf65431dccb57cb2a44528c284645a1/tools/clang/scripts/package.py

Project Member

Comment 3 by bugdroid1@chromium.org, Nov 2

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/d7b595bcb39c11385b6b1c6f97ad10571e389236

commit d7b595bcb39c11385b6b1c6f97ad10571e389236
Author: Max Moroz <mmoroz@chromium.org>
Date: Fri Nov 02 13:54:58 2018

Clang package: bundle "ubsan_standalone*" isntead of "ubsan*" libraries.

Apparently there also is ubsan_minimal runtime which we do not use:
https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html#minimal-runtime.

It's fairy small though:

20K	libclang_rt.ubsan_minimal-i386.a
20K	libclang_rt.ubsan_minimal-i386.so
24K	libclang_rt.ubsan_minimal-x86_64.a
4.0K	libclang_rt.ubsan_minimal-x86_64.a.syms
20K	libclang_rt.ubsan_minimal-x86_64.so


Bug:  870331 
Change-Id: I280fa27fce61095bffe5e91fed92245c75d051db
Reviewed-on: https://chromium-review.googlesource.com/c/1313756
Reviewed-by: Nico Weber <thakis@chromium.org>
Commit-Queue: Max Moroz <mmoroz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#604908}
[modify] https://crrev.com/d7b595bcb39c11385b6b1c6f97ad10571e389236/tools/clang/scripts/package.py

Project Member

Comment 4 by bugdroid1@chromium.org, Nov 12

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/500b4b174e5a872bbc9c437e73b29f22f1443fcc

commit 500b4b174e5a872bbc9c437e73b29f22f1443fcc
Author: Max Moroz <mmoroz@chromium.org>
Date: Mon Nov 12 16:53:28 2018

Clean up LLVM bundle package: don't use wildcards and exclude unneeded libs.

Hopefully this won't break too many things. Libraries removed:

- Linux: removed ~5.5MB from ~24.5MB of libs (excluding Android libs):
1.5M	libclang_rt.asan-i386.so
4.0K	libclang_rt.asan-preinit-i386.a
1.5M	libclang_rt.asan-x86_64.so
4.0K	libclang_rt.asan-preinit-x86_64.a
36K	libclang_rt.hwasan_cxx-x86_64.a
4.0K	libclang_rt.hwasan_cxx-x86_64.a.syms
1.4M	libclang_rt.hwasan-x86_64.a
4.0K	libclang_rt.hwasan-x86_64.a.syms
388K	libclang_rt.hwasan-x86_64.so
352K	libclang_rt.ubsan_standalone-i386.so
332K	libclang_rt.ubsan_standalone-x86_64.so

- Darwin: removed ~3MB from ~9MB of libs
1.5M	libclang_rt.fuzzer_no_main_ios.a
1.6M	libclang_rt.fuzzer_no_main_iossim.a

- Windows: (insignificant) removed ~50KB from ~28MB of libs
48094	clang_rt.asan-preinit-i386.lib
892	clang_rt.asan-preinit-x86_64.lib

Bug:  870331 
Change-Id: I0ccf1310b2ab5bde6d514a31b2c5ba7eb2844401
Reviewed-on: https://chromium-review.googlesource.com/c/1315361
Reviewed-by: Nico Weber <thakis@chromium.org>
Reviewed-by: Reid Kleckner <rnk@chromium.org>
Reviewed-by: Jonathan Metzman <metzman@chromium.org>
Commit-Queue: Max Moroz <mmoroz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#607265}
[modify] https://crrev.com/500b4b174e5a872bbc9c437e73b29f22f1443fcc/tools/clang/scripts/package.py

Project Member

Comment 5 by bugdroid1@chromium.org, Nov 16

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/29a3a80a284ce7fb6fefbbeb6171945006b52e93

commit 29a3a80a284ce7fb6fefbbeb6171945006b52e93
Author: Hans Wennborg <hans@chromium.org>
Date: Fri Nov 16 14:56:41 2018

Roll clang 346388-1:346388-3

This picks up package.py changes #607265 and #608413. It does not change
the version of clang.

It also includes a crazy workaround for msys binaries (in our case
find.exe and mv.exe, used by lit tests) crashing during some unknown
chrome infra problem (see last bug).

Bug:  870331 ,  905289 
Change-Id: Ic1d9fa64d6fcd4b590139c9343bed5bbe4d3faa3
Reviewed-on: https://chromium-review.googlesource.com/c/1337614
Commit-Queue: Hans Wennborg <hans@chromium.org>
Reviewed-by: Max Moroz <mmoroz@chromium.org>
Reviewed-by: Reid Kleckner <rnk@chromium.org>
Reviewed-by: Nico Weber <thakis@chromium.org>
Cr-Commit-Position: refs/heads/master@{#608776}
[modify] https://crrev.com/29a3a80a284ce7fb6fefbbeb6171945006b52e93/tools/clang/scripts/package.py
[modify] https://crrev.com/29a3a80a284ce7fb6fefbbeb6171945006b52e93/tools/clang/scripts/update.py

Project Member

Comment 6 by bugdroid1@chromium.org, Nov 16

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/83630c9e56e1b089c4981608b41f6d2cd83206b5

commit 83630c9e56e1b089c4981608b41f6d2cd83206b5
Author: Reid Kleckner <rnk@chromium.org>
Date: Fri Nov 16 23:49:20 2018

Revert "Roll clang 346388-1:346388-3"

This reverts commit 29a3a80a284ce7fb6fefbbeb6171945006b52e93.

Reason for revert:
We still need to package asan for i686-android: https://crbug.com/906246

Original change's description:
> Roll clang 346388-1:346388-3
> 
> This picks up package.py changes #607265 and #608413. It does not change
> the version of clang.
> 
> It also includes a crazy workaround for msys binaries (in our case
> find.exe and mv.exe, used by lit tests) crashing during some unknown
> chrome infra problem (see last bug).
> 
> Bug:  870331 ,  905289 
> Change-Id: Ic1d9fa64d6fcd4b590139c9343bed5bbe4d3faa3
> Reviewed-on: https://chromium-review.googlesource.com/c/1337614
> Commit-Queue: Hans Wennborg <hans@chromium.org>
> Reviewed-by: Max Moroz <mmoroz@chromium.org>
> Reviewed-by: Reid Kleckner <rnk@chromium.org>
> Reviewed-by: Nico Weber <thakis@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#608776}

TBR=thakis@chromium.org,hans@chromium.org,rnk@chromium.org,mmoroz@chromium.org

Change-Id: I225d64bd4e166c0150d63cfa7ab0bad7e0dd92f7
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug:  870331 ,  905289 , 906246
Reviewed-on: https://chromium-review.googlesource.com/c/1340836
Reviewed-by: Reid Kleckner <rnk@chromium.org>
Commit-Queue: Reid Kleckner <rnk@chromium.org>
Cr-Commit-Position: refs/heads/master@{#609028}
[modify] https://crrev.com/83630c9e56e1b089c4981608b41f6d2cd83206b5/tools/clang/scripts/package.py
[modify] https://crrev.com/83630c9e56e1b089c4981608b41f6d2cd83206b5/tools/clang/scripts/update.py

Project Member

Comment 7 by bugdroid1@chromium.org, Nov 20

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/3e3b34828c23ab8d66267c9d9fccf79fd7324693

commit 3e3b34828c23ab8d66267c9d9fccf79fd7324693
Author: Max Moroz <mmoroz@chromium.org>
Date: Tue Nov 20 01:22:07 2018

Clang scripts: reland workaround for msys binaries by hans@ from https://crrev.com/c/1337614.

Bug:  870331 ,  905289 
Change-Id: I4c5decd8b299d64d2b6c302bcd0049f8f9581f48
Reviewed-on: https://chromium-review.googlesource.com/c/1343258
Commit-Queue: Max Moroz <mmoroz@chromium.org>
Commit-Queue: Nico Weber <thakis@chromium.org>
Reviewed-by: Nico Weber <thakis@chromium.org>
Cr-Commit-Position: refs/heads/master@{#609544}
[modify] https://crrev.com/3e3b34828c23ab8d66267c9d9fccf79fd7324693/tools/clang/scripts/update.py

Project Member

Comment 8 by bugdroid1@chromium.org, Nov 21

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/c03fc8e7bfcf61e51114d9daaf9628887a8f170c

commit c03fc8e7bfcf61e51114d9daaf9628887a8f170c
Author: Max Moroz <mmoroz@chromium.org>
Date: Wed Nov 21 16:21:16 2018

Roll clang 346388-1:346388-4.

Ran `tools/clang/scripts/upload_revision.py 346388 4`.

Bug:  870331 
Change-Id: Ie2ebb3aa52bc757f4a0680cdbf8f060a030ce9a6
Reviewed-on: https://chromium-review.googlesource.com/c/1344209
Reviewed-by: Hans Wennborg <hans@chromium.org>
Reviewed-by: Nico Weber <thakis@chromium.org>
Commit-Queue: Hans Wennborg <hans@chromium.org>
Commit-Queue: Nico Weber <thakis@chromium.org>
Cr-Commit-Position: refs/heads/master@{#610096}
[modify] https://crrev.com/c03fc8e7bfcf61e51114d9daaf9628887a8f170c/tools/clang/scripts/update.py

Keeping this issue open for a little while in case anything important got deleted.
Project Member

Comment 10 by bugdroid1@chromium.org, Nov 21

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/8e4125b5c6cc12bfd1ebd19e1b3b84edadcf3f55

commit 8e4125b5c6cc12bfd1ebd19e1b3b84edadcf3f55
Author: Gayane Petrosyan <gayane@chromium.org>
Date: Wed Nov 21 19:00:49 2018

Revert "Roll clang 346388-1:346388-4."

This reverts commit c03fc8e7bfcf61e51114d9daaf9628887a8f170c.

Reason for revert: <INSERT REASONING HERE>

Original change's description:
> Roll clang 346388-1:346388-4.
> 
> Ran `tools/clang/scripts/upload_revision.py 346388 4`.
> 
> Bug:  870331 
> Change-Id: Ie2ebb3aa52bc757f4a0680cdbf8f060a030ce9a6
> Reviewed-on: https://chromium-review.googlesource.com/c/1344209
> Reviewed-by: Hans Wennborg <hans@chromium.org>
> Reviewed-by: Nico Weber <thakis@chromium.org>
> Commit-Queue: Hans Wennborg <hans@chromium.org>
> Commit-Queue: Nico Weber <thakis@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#610096}

TBR=thakis@chromium.org,hans@chromium.org,rnk@chromium.org,mmoroz@chromium.org

Change-Id: Ica508f221abd3f53dee66eb3f1937773c8947d27
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug:  870331 
Reviewed-on: https://chromium-review.googlesource.com/c/1347104
Reviewed-by: Gayane Petrosyan <gayane@chromium.org>
Commit-Queue: Gayane Petrosyan <gayane@chromium.org>
Cr-Commit-Position: refs/heads/master@{#610111}
[modify] https://crrev.com/8e4125b5c6cc12bfd1ebd19e1b3b84edadcf3f55/tools/clang/scripts/update.py

Project Member

Comment 11 by bugdroid1@chromium.org, Nov 22

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/040c80644f3669c95194f225a73164c45d27b872

commit 040c80644f3669c95194f225a73164c45d27b872
Author: Max Moroz <mmoroz@chromium.org>
Date: Thu Nov 22 00:32:00 2018

Clang scripts: package ubsan_standalone runtime for Android on ARM and AArch64.

Bug: 906129,  870331 
Change-Id: I19b7ceb187cfd1e6a28a808005606311bc4c96cc
Reviewed-on: https://chromium-review.googlesource.com/c/1347691
Commit-Queue: Max Moroz <mmoroz@chromium.org>
Reviewed-by: Nico Weber <thakis@chromium.org>
Cr-Commit-Position: refs/heads/master@{#610271}
[modify] https://crrev.com/040c80644f3669c95194f225a73164c45d27b872/tools/clang/scripts/package.py

Project Member

Comment 12 by bugdroid1@chromium.org, Nov 27

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/cd145051149fc2f01335591e00e64ad9398f8569

commit cd145051149fc2f01335591e00e64ad9398f8569
Author: Max Moroz <mmoroz@chromium.org>
Date: Tue Nov 27 18:39:05 2018

Roll clang 346388-1:346388-5.

Ran `tools/clang/scripts/upload_revision.py 346388 5`.

Bug:  870331 
Change-Id: Id09c926a34194487757e641283f63fb63a614436
Reviewed-on: https://chromium-review.googlesource.com/c/1351029
Commit-Queue: Hans Wennborg <hans@chromium.org>
Reviewed-by: Hans Wennborg <hans@chromium.org>
Cr-Commit-Position: refs/heads/master@{#611214}
[modify] https://crrev.com/cd145051149fc2f01335591e00e64ad9398f8569/tools/clang/scripts/update.py

Status: Fixed (was: Started)
Looks like this can be closed now. The final result is not too significant compared to the total size of the bundle, but at least I got rid of some redundant libraries and cleaned up the packaging code a bit.

Linux$ du -hs *
254M	346388-1
249M	346388-5
77M	Linux_x64%2Fclang-346388-1.tar
75M	Linux_x64%2Fclang-346388-5.tar

Darwin$ du -hs *
144M	346388-1
141M	346388-5
45M	Mac%2Fclang-346388-1.tar
44M	Mac%2Fclang-346388-5.tar

Windows$ du -hs *
208M	346388-1
208M	346388-5
68M	Win%2Fclang-346388-1.tar
68M	Win%2Fclang-346388-5.tar

(the files removed on Windows were just tens of KBs).
The package.py cleanup is really nice, thanks for pushing this through.
Cc: r...@chromium.org
Thanks for all the help Hans, Nico and Reid!
+1, thanks much for doing this :-)

Sign in to add a comment