Two observations from looking at Release/obj/chrome/chrome_initial.ninja:

1. Possible link optimization difference?
GYP: ldflags=-Wl,-O1, ... -Wl,-O1, ... -Wl,-O2 
GN:  ldflags=-Wl,-O1

2. Libraries vs. .o files?
My ninja fu is weak, but the GN file contains >8000 .o files in the link and the GYP file contains ~700 .a files with a handful of .o files

llozano@, can you take this up or reassign?

Clearly there are some differences in SimpleChrome at least that need to be investigated. Hopefully they will have the same root cause as the chroot difference.

Either way this extends way beyond my area of expertise.

There is a major change between GYP and GN in that many targets that are specified as static_library()s in GYP were specified as source_sets() in GN. As a result, you get lists of .o's instead of lists of .a's.

It was generally believed that this made things link faster, at least for when a given binary actually depends on most everything in the targets, but it relies on the linker to strip out dead code. We have also been gradually changing many of these back to static_library()s since it turns out that we may have optimized for chrome's linking speed but made the rest of the build slower as many binaries seem to not actually need all of the files in many targets (see bug 623661 for more context).

So, make sure you're tracking tip-of-tree chromium when doing these investigations, or you might find that you're duplicating some of the work we've done recently.

However, I wouldn't generally think the above would explain the differences in chrome itself, since, as noted, the chrome binary itself should generally need everything in the source sets.

You should make sure you're eliminating dead code in the linker either way, though.

This was all done with ToT as of yesterday morning.

I modified src/build/config/compiler/BUILD.gn to use -Wl,-O2 but that did not appear to make any difference in file size.

I am attaching the chrome_inital.ninja files for samus (src/out_samus/Release/obj/chrome/chrome_initial.ninja) for GYP and GN.

Here are the ldflags differences:

GN:
ldflags = -pie -Wl,--fatal-warnings -fPIC -Wl,-z,noexecstack -Wl,-z,now -Wl,-z,relro -Wl,-z,defs -fuse-ld=gold -B../../third_party/binutils/Linux_x64/Release/bin -Wl,--threads -Wl,--thread-count=4 -Wl,--icf=safe -pthread -m64 -Wl,-O1 -Wl,--gc-sections -Wl,--no-as-needed -lpthread -Wl,--as-needed --sysroot=../../../.cros_cache/chrome-sdk/tarballs/samus+8584.0.0+sysroot_chromeos-base_chromeos-chrome.tar.xz -L/usr/local/google/home/stevenjb/Work/chrome/.cros_cache/chrome-sdk/tarballs/samus+8584.0.0+sysroot_chromeos-base_chromeos-chrome.tar.xz/lib64 -Wl,-rpath-link=/usr/local/google/home/stevenjb/Work/chrome/.cros_cache/chrome-sdk/tarballs/samus+8584.0.0+sysroot_chromeos-base_chromeos-chrome.tar.xz/lib64 -L/usr/local/google/home/stevenjb/Work/chrome/.cros_cache/chrome-sdk/tarballs/samus+8584.0.0+sysroot_chromeos-base_chromeos-chrome.tar.xz/usr/lib64 -Wl,-rpath-link=/usr/local/google/home/stevenjb/Work/chrome/.cros_cache/chrome-sdk/tarballs/samus+8584.0.0+sysroot_chromeos-base_chromeos-chrome.tar.xz/usr/lib64 -L/usr/local/google/home/stevenjb/Work/chrome/.cros_cache/chrome-sdk/tarballs/samus+8584.0.0+sysroot_chromeos-base_chromeos-chrome.tar.xz/usr/local/lib64 -Wl,-rpath-link=/usr/local/google/home/stevenjb/Work/chrome/.cros_cache/chrome-sdk/tarballs/samus+8584.0.0+sysroot_chromeos-base_chromeos-chrome.tar.xz/usr/local/lib64 -L/usr/local/google/home/stevenjb/Work/chrome/.cros_cache/chrome-sdk/tarballs/samus+8584.0.0+sysroot_chromeos-base_chromeos-chrome.tar.xz/lib -Wl,-rpath-link=/usr/local/google/home/stevenjb/Work/chrome/.cros_cache/chrome-sdk/tarballs/samus+8584.0.0+sysroot_chromeos-base_chromeos-chrome.tar.xz/lib -L/usr/local/google/home/stevenjb/Work/chrome/.cros_cache/chrome-sdk/tarballs/samus+8584.0.0+sysroot_chromeos-base_chromeos-chrome.tar.xz/usr/lib -Wl,-rpath-link=/usr/local/google/home/stevenjb/Work/chrome/.cros_cache/chrome-sdk/tarballs/samus+8584.0.0+sysroot_chromeos-base_chromeos-chrome.tar.xz/usr/lib -L/usr/local/google/home/stevenjb/Work/chrome/.cros_cache/chrome-sdk/tarballs/samus+8584.0.0+sysroot_chromeos-base_chromeos-chrome.tar.xz/usr/local/lib -Wl,-rpath-link=/usr/local/google/home/stevenjb/Work/chrome/.cros_cache/chrome-sdk/tarballs/samus+8584.0.0+sysroot_chromeos-base_chromeos-chrome.tar.xz/usr/local/lib -Wl,-rpath-link=../Release -Wl,--disable-new-dtags -L/usr/local/google/home/stevenjb/Work/chrome/.cros_cache/chrome-sdk/tarballs/samus+8584.0.0+sysroot_chromeos-base_chromeos-chrome.tar.xz/usr/lib64

GYP:
ldflags = -Wl,-O1 -Wl,-O2 -Wl,--as-needed -Wl,-z,now -Wl,-z,relro $
    -Wl,-z,defs -pthread -Wl,-z,noexecstack -fPIC -fuse-ld=gold $
    -Wl,--disable-new-dtags -pie $
    -Wl,-uIsHeapProfilerRunning,-uProfilerStart $
    -Wl,-u_Z21InitialMallocHook_NewPKvj,-u_Z22InitialMallocHook_MMapPKvS0_jiiix,-u_Z22InitialMallocHook_SbrkPKvi $
    -Wl,-u_Z21InitialMallocHook_NewPKvm,-u_Z22InitialMallocHook_MMapPKvS0_miiil,-u_Z22InitialMallocHook_SbrkPKvl $
    -Wl,-u_ZN15HeapLeakChecker12IgnoreObjectEPKv,-u_ZN15HeapLeakChecker14UnIgnoreObjectEPKv $
    -L/usr/local/google/home/stevenjb/Work/chrome/.cros_cache/chrome-sdk/tarballs/samus+8562.0.0+sysroot_chromeos-base_chromeos-chrome.tar.xz/usr/lib64 $
    -L/usr/local/google/home/stevenjb/Work/chrome/.cros_cache/chrome-sdk/tarballs/samus+8562.0.0+sysroot_chromeos-base_chromeos-chrome.tar.xz/usr/lib64 $
    -L/usr/local/google/home/stevenjb/Work/chrome/.cros_cache/chrome-sdk/tarballs/samus+8562.0.0+sysroot_chromeos-base_chromeos-chrome.tar.xz/usr/lib64 $
    -L/usr/local/google/home/stevenjb/Work/chrome/.cros_cache/chrome-sdk/tarballs/samus+8562.0.0+sysroot_chromeos-base_chromeos-chrome.tar.xz/usr/lib64 $
    -L/usr/local/google/home/stevenjb/Work/chrome/.cros_cache/chrome-sdk/tarballs/samus+8562.0.0+sysroot_chromeos-base_chromeos-chrome.tar.xz/usr/lib64 $
    -L/usr/local/google/home/stevenjb/Work/chrome/.cros_cache/chrome-sdk/tarballs/samus+8562.0.0+sysroot_chromeos-base_chromeos-chrome.tar.xz/usr/lib64 $
    -L/usr/local/google/home/stevenjb/Work/chrome/.cros_cache/chrome-sdk/tarballs/samus+8562.0.0+sysroot_chromeos-base_chromeos-chrome.tar.xz/usr/lib64 $
    -L/usr/local/google/home/stevenjb/Work/chrome/.cros_cache/chrome-sdk/tarballs/samus+8562.0.0+sysroot_chromeos-base_chromeos-chrome.tar.xz/usr/lib64 $
    -L/usr/local/google/home/stevenjb/Work/chrome/.cros_cache/chrome-sdk/tarballs/samus+8562.0.0+sysroot_chromeos-base_chromeos-chrome.tar.xz/usr/lib64 $
    -m64 $
    --sysroot=/usr/local/google/home/stevenjb/Work/chrome/.cros_cache/chrome-sdk/tarballs/samus+8562.0.0+sysroot_chromeos-base_chromeos-chrome.tar.xz $
    -L/usr/local/google/home/stevenjb/Work/chrome/.cros_cache/chrome-sdk/tarballs/samus+8562.0.0+sysroot_chromeos-base_chromeos-chrome.tar.xz/lib64 $
    -Wl,-rpath-link=/usr/local/google/home/stevenjb/Work/chrome/.cros_cache/chrome-sdk/tarballs/samus+8562.0.0+sysroot_chromeos-base_chromeos-chrome.tar.xz/lib64 $
    -L/usr/local/google/home/stevenjb/Work/chrome/.cros_cache/chrome-sdk/tarballs/samus+8562.0.0+sysroot_chromeos-base_chromeos-chrome.tar.xz/usr/lib64 $
    -Wl,-rpath-link=/usr/local/google/home/stevenjb/Work/chrome/.cros_cache/chrome-sdk/tarballs/samus+8562.0.0+sysroot_chromeos-base_chromeos-chrome.tar.xz/usr/lib64 $
    -L/usr/local/google/home/stevenjb/Work/chrome/.cros_cache/chrome-sdk/tarballs/samus+8562.0.0+sysroot_chromeos-base_chromeos-chrome.tar.xz/usr/local/lib64 $
    -Wl,-rpath-link=/usr/local/google/home/stevenjb/Work/chrome/.cros_cache/chrome-sdk/tarballs/samus+8562.0.0+sysroot_chromeos-base_chromeos-chrome.tar.xz/usr/local/lib64 $
    -L/usr/local/google/home/stevenjb/Work/chrome/.cros_cache/chrome-sdk/tarballs/samus+8562.0.0+sysroot_chromeos-base_chromeos-chrome.tar.xz/lib $
    -Wl,-rpath-link=/usr/local/google/home/stevenjb/Work/chrome/.cros_cache/chrome-sdk/tarballs/samus+8562.0.0+sysroot_chromeos-base_chromeos-chrome.tar.xz/lib $
    -L/usr/local/google/home/stevenjb/Work/chrome/.cros_cache/chrome-sdk/tarballs/samus+8562.0.0+sysroot_chromeos-base_chromeos-chrome.tar.xz/usr/lib $
    -Wl,-rpath-link=/usr/local/google/home/stevenjb/Work/chrome/.cros_cache/chrome-sdk/tarballs/samus+8562.0.0+sysroot_chromeos-base_chromeos-chrome.tar.xz/usr/lib $
    -L/usr/local/google/home/stevenjb/Work/chrome/.cros_cache/chrome-sdk/tarballs/samus+8562.0.0+sysroot_chromeos-base_chromeos-chrome.tar.xz/usr/local/lib $
    -Wl,-rpath-link=/usr/local/google/home/stevenjb/Work/chrome/.cros_cache/chrome-sdk/tarballs/samus+8562.0.0+sysroot_chromeos-base_chromeos-chrome.tar.xz/usr/local/lib $
    -Wl,--icf=all -Wl,-O1 -Wl,--as-needed -Wl,--gc-sections $
    -Wl,--no-as-needed -lpthread -Wl,--as-needed

Deleted: chrome_initial.ninja.gn 516 KB

chrome_initial.ninja.gn 516 KB Download

Deleted: chrome_initial.ninja.gyp 144 KB

chrome_initial.ninja.gyp 144 KB Download

I ran ldd on GN and GYP as follows. Attached are the results.

~/Work/chrome/src (master) $ ldd out_samus/ReleaseGYP/chrome >& ldd_gyp.txt
~/Work/chrome/src (master) $ ldd out_samus/Release/chrome >& ldd_gn.txt

There appear to be significantly more entries in GYP (70) vs GN (53) so that does not immediately appear to be the issue.

Deleted: ldd_gyp.txt 5.5 KB

ldd_gyp.txt 5.5 KB View Download

Deleted: ldd_gn.txt 4.1 KB

ldd_gn.txt 4.1 KB View Download

llozano@ suggested checking the --icf value and sure enough, 'safe' vs 'all' accounted for the majority of the difference on x86 only. Changing safe -> all brings x86 GN within 1% if GYP, same as ARM.

I will put up a CL to fix this.

So, there is a CL to "fix" this which will bring GN in line with GYP:

https://codereview.chromium.org/2153223002/

However, the change to use --icf=safe instead of --icf=all was at least somewhat intentional;
--icf=all may potentially be increasing the crash rate on Chrome, see  issue 576197 .
(The change was only made in GN, and it is not clear how much the file size increase was discussed).

+abodenha@, +zelidrag@ - We may need to make a judgement call:
* We can continue to build Chrome for Chrome OS with --icf=all
* We can build Chrome for Chrome OS with --icf=some which increases the stripped binary size by 5% (137 MB -> 145 MB) but may slightly decrease the crash rate (presumably by a very small amount, otherwise we would have noticed the issue sooner).

also notice that, if we decide to disable, we only need to disable icf=all for Intel processors (ARM is not affected).

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/b2eb9de87515796cf9b7c17fd31a34d92427eb8b

commit b2eb9de87515796cf9b7c17fd31a34d92427eb8b
Author: stevenjb <stevenjb@chromium.org>
Date: Sat Jul 16 00:17:27 2016

GN: Use icf=all instead of icf=safe

BUG= 628441 

Review-Url: https://codereview.chromium.org/2153223002
Cr-Commit-Position: refs/heads/master@{#405909}

[modify] https://crrev.com/b2eb9de87515796cf9b7c17fd31a34d92427eb8b/build/config/compiler/BUILD.gn

I talked to the developer working on the fix for this problem. He says there is already a fix that is going through review and should be finalized this week.
So, lets wait for the fix. I don't think there is a need to disable icf for such short period of time.

Agreed, especially since it appears to have been enabled on linux/chromeos
since June 2015:
https://chromium.googlesource.com/chromium/src/+/eca59926d104516f65793063c6a5c04855a8a3fd

Did this get resolved? If not, does it really still need to be Pri-0 ?

Lowering the priority.

The current status is:
* Currently we are building chrome for Chrome OS only with icf=all which brings the binary size within 1% of binaries built with GYP.
* We suspect icf=all *might* be causing some crashes, but not sufficiently to have noticed it when it was set in GYP.
* per comment #11, there should be fix soon that will address any such potential crashes.

So, there is really nothing more to do here, I'm going to go ahead and close this.

llozano@, if you want to re-open this to track testing the fix described in comment #11 (I'm not sure how though), lower the priority and remove the laebl.

bulk verified