New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 917404 link

Starred by 3 users

Issue metadata

Status: Fixed
Owner:
Closed: Jan 10
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Linux , Mac
Pri: 3
Type: Bug

Blocking:
issue 903751
issue 920687



Sign in to add a comment

Build clang with CLANG_PLUGIN_SUPPORT=OFF

Project Member Reported by thakis@chromium.org, Dec 21

Issue description

We currently build clang with CLANG_PLUGIN_SUPPORT=ON (the default). This causes all kinds of symbols to be exported, and disables dead code stripping. We could set that to OFF.

We do use plugins, but on Windows we already compile them into the clang binary itself, since clang's plugin interface doesn't work on Windows.


This would also allow us to build without pic on linux, which may or may not help a bit with perf.

I tried disabling plugin support and got these numbers:

Before:
$ readelf --dyn-syms third_party/llvm-build/Release+Asserts/bin/clang | wc -l
42252

After:
$ readelf --dyn-syms third_party/llvm-build/Release+Asserts/bin/clang | wc -l
279

Builds of base_unittests in a vanilla debug linux build take ~56.3s before, 54.1s after.
Builds of stroke_opacity_custom.cc in a linux->win debug build take 8.5s before, 8.4s after.

But also, before:
$ ls -hl third_party/llvm-build/Release+Asserts/lib/*.so
-rwxr-xr-x 1 thakis primarygroup 1.6M Nov 26 10:43 third_party/llvm-build/Release+Asserts/lib/libBlinkGCPlugin.so
-rwxr-xr-x 1 thakis primarygroup 554K Nov 26 10:43 third_party/llvm-build/Release+Asserts/lib/libFindBadConstructs.so
$ ls -hl third_party/llvm-build/Release+Asserts/bin/clang
-rwxr-xr-x 1 thakis primarygroup 99M Nov 26 10:43 third_party/llvm-build/Release+Asserts/bin/clang

After:
$ ls -hl third_party/llvm-build/Release+Asserts/bin/clang
lrwxrwxrwx 1 thakis primarygroup 7 Dec 20 14:14 third_party/llvm-build/Release+Asserts/bin/clang -> clang-8
$ ls -hl third_party/llvm-build/Release+Asserts/bin/clang-8
-rwxr-xr-x 1 thakis primarygroup 103M Dec 20 14:14 third_party/llvm-build/Release+Asserts/bin/clang-8




It makes our non-Win and Win codepaths more similar, makes the gn files a bit simpler, reduces the number of dynsyms by a ton, is perf-positive-to-neutral, and slightly size negative (which surprised me).



Hans and Reid and me are all in favor of doing this, so let's.
 
> slightly size negative (which surprised me).

That would surprise me too, so I patched in your change to take a look. Looks like the sizes are:

$ tar tvzf clang-349417-1.tgz | grep bin/clang
-rwxr-xr-x chrome-bot/chrome-bot 104989048 2018-12-18 04:03 bin/clang
lrwxrwxrwx chrome-bot/chrome-bot         0 2018-12-18 04:03 bin/clang++ -> clang
lrwxrwxrwx chrome-bot/chrome-bot         0 2018-12-18 04:03 bin/clang-cl -> clang

$ tar tvzf clang-349417-2.tgz | grep bin/clang
lrwxrwxrwx pcc/primarygroup        0 2018-12-21 12:21 bin/clang-cl -> clang
lrwxrwxrwx pcc/primarygroup        0 2018-12-21 12:21 bin/clang++ -> clang
-rwxr-xr-x pcc/primarygroup 98156600 2018-12-21 12:21 bin/clang

Are you sure that you were looking at the right binaries?
Yes, reasonably sure.

Note how the "before" dir has the .so files an the "after" file has the -8 symlink, which suggests that the former is downloaded by the package, and the latter is built locally.

To get the numbers, I looked at the downloaded clang (I was at 347874-5, before yesterday's roll). Doing that again, that's 104,312,552 bytes, and 104312552 >> 20 is 99, which matches the "before" number in comment 0. It also roughly matches your "before" number.

I then ran package.py with patch set 1 from https://chromium-review.googlesource.com/c/chromium/src/+/1387395 and got the "After:" numbers. I do not know why I'd get different numbers than you for this step, but the clang-8 symlink and the absence of .so files suggests to me that I didn't hold it completely wrong.

Weird.
Cc: p...@chromium.org
Ah, the copy of clang-8 in my third_party/llvm-build directory is approximately the same size as yours:

$ ls -lh third_party/llvm-build/Release+Asserts/bin/clang-8
-rwxr-xr-x 1 pcc primarygroup 105M Dec 21 12:17 third_party/llvm-build/Release+Asserts/bin/clang-8

But if I strip it I get the same size as in my tarball:

$ strip -o foo third_party/llvm-build/Release+Asserts/bin/clang-8
$ ls -l foo
-rwxr-xr-x 1 pcc primarygroup 98156600 Dec 21 13:01 foo

And we are indeed stripping files before adding them to the tarball:
https://cs.chromium.org/chromium/src/tools/clang/scripts/package.py?l=410
So that explains it.
Hm, I did add a sys.exit(1) to package.py in https://chromium-review.googlesource.com/c/chromium/src/+/1387395/1/tools/clang/scripts/package.py but it was after the strip step. Not sure why the strip step apparently didn't run over here then.

(But looking at this I see that with CLANG_PLUGIN_SUPPORT=OFF we can stop passing -x to strip on mac; I'll update my CL.)
Mac:

http://commondatastorage.googleapis.com/chromium-browser-clang/index.html?path=Mac/

clang-349417-1.tgz 44.33MB
clang-349417-2.tgz 41.81MB

2.5MB / 5.6% smaller compressed size.

Unpacked:
bin/clang 93.6MB -> 88.5MB
libBlinkGCPlugin.dylib was 868kB, libFindBadConstructs.dylib was 302kB

6.27MB / 6.7% uncompressed size.



Linux:

http://commondatastorage.googleapis.com/chromium-browser-clang/index.html?path=Linux_x64/
clang-349417-1.tgz 75.97MB
clang-349417-2.tgz 73.66MB

2.31 MB / 3% smaller compressed size.

Unpacked:
bin/clang 105MB -> 99.1MB

libBlinkGCPlugin.so was 1.7MB, libFindBadConstructs.so was 567kB

3.67MB / 3.5% uncompressed size.




(why is the linux package so much bigger? because it's much more data: 258.6MB unpacked, vs 141.1MB, 117.5MB more. Why? We bundle lld on Linux but not mac, that's 71.4MB, this explains all but 46.1MB. llvm-14 are 18.4MB, leaving 27.7MB. The lib/ folder is 18.8MB larger (runtimes for android), and the rest is probably slightly larger binaries, maybe due to the GOT/PLT indirection. But the linux binary shrinking less kind of disproves that gues.)
Err, linux should've read:

(105 - 99.1 + 1.7 + 0.567) / 105

8.167MB / 7.7%


So uncompressed shrank similarly, and compressed the percentage is smaller because clang is a smaller percentage of the whole package.
Project Member

Comment 7 by bugdroid1@chromium.org, Jan 3

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/25a9c5ec90f0bc3df2fdc827d5b36b83bc8520de

commit 25a9c5ec90f0bc3df2fdc827d5b36b83bc8520de
Author: Nico Weber <thakis@chromium.org>
Date: Thu Jan 03 06:09:34 2019

Build clang without plugin support.

See bug for details.

Bug:  917404 , 256342 
Change-Id: Ic5fd959f23e77b9194af4faecceeefffe0b924ae
Reviewed-on: https://chromium-review.googlesource.com/c/1387395
Commit-Queue: Nico Weber <thakis@chromium.org>
Reviewed-by: Reid Kleckner <rnk@chromium.org>
Cr-Commit-Position: refs/heads/master@{#619595}
[modify] https://crrev.com/25a9c5ec90f0bc3df2fdc827d5b36b83bc8520de/build/config/clang/BUILD.gn
[modify] https://crrev.com/25a9c5ec90f0bc3df2fdc827d5b36b83bc8520de/third_party/blink/renderer/BUILD.gn
[modify] https://crrev.com/25a9c5ec90f0bc3df2fdc827d5b36b83bc8520de/tools/clang/blink_gc_plugin/CMakeLists.txt
[modify] https://crrev.com/25a9c5ec90f0bc3df2fdc827d5b36b83bc8520de/tools/clang/blink_gc_plugin/tests/test.py
[modify] https://crrev.com/25a9c5ec90f0bc3df2fdc827d5b36b83bc8520de/tools/clang/plugins/CMakeLists.txt
[modify] https://crrev.com/25a9c5ec90f0bc3df2fdc827d5b36b83bc8520de/tools/clang/plugins/tests/test.py
[modify] https://crrev.com/25a9c5ec90f0bc3df2fdc827d5b36b83bc8520de/tools/clang/pylib/clang/plugin_testing.py
[modify] https://crrev.com/25a9c5ec90f0bc3df2fdc827d5b36b83bc8520de/tools/clang/scripts/package.py
[modify] https://crrev.com/25a9c5ec90f0bc3df2fdc827d5b36b83bc8520de/tools/clang/scripts/update.py

Cc: j...@opera.com most...@vewd.com
Status: Fixed (was: Started)
Blocking: 920687

Sign in to add a comment