copied from the original crbug. please ignore mac related parts.
----
All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=621802

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgwvS-tgoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICggtX7vwoM


Bot(s) for this bug's original alert(s):

Google Chrome Mac
Google Chrome Win

Snippets from https://bugs.chromium.org/p/chromium/issues/detail?id=621802#c4
----
Looking Linux graph to find a suspicious CL:
https://chromeperf.appspot.com/report?sid=a514f920dd8efd47095d638a55303444b0c85ad57d7b9308c58a37e4dcdf810e&start_rev=400719&end_rev=400784

I can not find any regression in Linux within the range of Windows regression.
https://chromium.googlesource.com/chromium/src/+log/c548a8dc422953c7e06a65e6abe2e68ef063e5dd%5E..9eddf5d3cbb03e87dc091dc0b1c40917b80ec071?pretty=fuller
Considering these changes, it looks flipping GN build may cause this.

Oops, wrong rev in the summary.

Other useful comments from the original bug.
----
Comment 9 by brucedawson
On my over-the-weekend gyp/gn build comparisons I see that on official builds chrome.dll increased by 0.8 MiB and chrome_child.dll shrunk by 0.3 MiB when switching from gyp to gn.

That's from 35.5 to 36.3 for chrome.dll and 44.2 to 43.9 MiB for chrome_child.dll.

On static_library builds both DLLs shrunk significantly when going from gyp to gn. On shared_library builds chrome.dll shrunk hugely when going from gyp to gn.

The only difference in my test builds was gyp versus gn. My build settings for the official build were:

gyp:
GYP_DEFINES=buildtype=Official
gn:
gn_args=target_cpu="x86";is_debug=false;is_component_build=false;is_official_build=true

----
Comment 10 by brucedawson
At 4cc4f198b1, local build with settings shown in comment #9:

gyp official-build sizes:
37,176,832 chrome.dll
46,396,416 chrome_child.dll
   270,848 chrome_elf.dll
   450,048 chrome_watcher.dll
 84,294,144 bytes

gn official-build sizes:
38,073,856 chrome.dll
45,983,232 chrome_child.dll
   356,352 chrome_elf.dll
   579,584 chrome_watcher.dll
 84,993,024 bytes

The relative increase in chrome_elf.dll is particularly significant, and may allow for faster iteration on testing possible solutions since it will build relatively quickly.

We're not going to ship a GN-built Chrome on Windows in M53, so clearing the M-53 label.

I compared official 32-bit gyp/gn builds of chrome_elf.dll. I used dia2dump -p to dump all of the symbols and then printed the ones that occur in the gn but not gyp builds. The result is below, although it's not immediately obvious what is triggering the pulling in of these functions - different build settings?

Continuing to investigate. Total chrome_elf.dll regression is 88,016 bytes. Here's the list of gn only functions:

_aesni_cbc_encrypt(_aesni_cbc_encrypt)
_gcm_ghash_4bit_x86(_gcm_ghash_4bit_x86)
_OPENSSL_ia32cap_P(_OPENSSL_ia32cap_P)
_bn_mul_add_words(_bn_mul_add_words)
_bn_sqr_comba4(_bn_sqr_comba4)
_asm_AES_set_decrypt_key(_asm_AES_set_decrypt_key)
_gcm_gmult_4bit_x86(_gcm_gmult_4bit_x86)
_bn_mul_comba8(_bn_mul_comba8)
_gcm_ghash_clmul(_gcm_ghash_clmul)
_aesni_decrypt(_aesni_decrypt)
_sha256_block_data_order(_sha256_block_data_order)
_bn_sqr_words(_bn_sqr_words)
_vpaes_set_encrypt_key(_vpaes_set_encrypt_key)
_aesni_ccm64_decrypt_blocks(_aesni_ccm64_decrypt_blocks)
_asm_RC4(_asm_RC4)
_gcm_ghash_4bit_mmx(_gcm_ghash_4bit_mmx)
_bn_sqr_comba8(_bn_sqr_comba8)
_bn_sub_part_words(_bn_sub_part_words)
_sha512_block_data_order(_sha512_block_data_order)
_gcm_gmult_4bit_mmx(_gcm_gmult_4bit_mmx)
_aesni_set_decrypt_key(_aesni_set_decrypt_key)
_aesni_set_encrypt_key(_aesni_set_encrypt_key)
_asm_AES_decrypt(_asm_AES_decrypt)
_gcm_init_clmul(_gcm_init_clmul)
_aesni_encrypt(_aesni_encrypt)
_gcm_gmult_clmul(_gcm_gmult_clmul)
_vpaes_set_decrypt_key(_vpaes_set_decrypt_key)
_vpaes_decrypt(_vpaes_decrypt)
_bn_add_words(_bn_add_words)
_md5_block_asm_data_order(_md5_block_asm_data_order)
_vpaes_cbc_encrypt(_vpaes_cbc_encrypt)
_ChaCha20_ctr32(_ChaCha20_ctr32)
_bn_mul_mont(_bn_mul_mont)
_bn_div_words(_bn_div_words)
_sha1_block_data_order(_sha1_block_data_order)
_aesni_ctr32_encrypt_blocks(_aesni_ctr32_encrypt_blocks)
_aesni_xts_encrypt(_aesni_xts_encrypt)
_asm_AES_encrypt(_asm_AES_encrypt)
_aesni_xts_decrypt(_aesni_xts_decrypt)
_asm_AES_set_encrypt_key(_asm_AES_set_encrypt_key)
?keeper_@CrashKeysWin@breakpad@@0PAV12@A(private: static class breakpad::CrashKeysWin * breakpad::CrashKeysWin::keeper_)
_vpaes_encrypt(_vpaes_encrypt)
_bn_mul_comba4(_bn_mul_comba4)
_ChaCha20_ssse3(_ChaCha20_ssse3)
_bn_mul_words(_bn_mul_words)
_asm_AES_cbc_encrypt(_asm_AES_cbc_encrypt)
__aullrem(__aullrem)
_aesni_ccm64_encrypt_blocks(_aesni_ccm64_encrypt_blocks)
_asm_RC4_set_key(_asm_RC4_set_key)
_bn_sub_words(_bn_sub_words)
_aesni_ecb_encrypt(_aesni_ecb_encrypt)

 Issue 624132  has been merged into this issue.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/0ab90cb1edf8f08f1fa5d2551aa21cac225163cf

commit 0ab90cb1edf8f08f1fa5d2551aa21cac225163cf
Author: brucedawson <brucedawson@chromium.org>
Date: Fri Jul 01 17:55:48 2016

Shrink chrome_elf.dll by switching source_sets to static_library

Investigation of why official chrome_elf.dll builds were 86,016 bytes
larger on gn than gyp showed that encryption functions such as
aesni_cbc_encrypt were being pulled in. Deleting the implementations
of these functions from the .asm file caused no errors which proved that
the functions weren't being referenced, they just weren't being
discarded. Changing the yasm_assemble template to use static_library
fixed that and made the file sizes identical.

Comparison of the dia2dump -p output after that change showed there were
still two extra symbols in the gn builds:
  class breakpad::CrashKeysWin * breakpad::CrashKeysWin::keeper_
  __aullrem(__aullrem)

keeper_ is a global variable - those are harder to discard, even with
/Gw - but making its source file part of a static_library fixed it.

link /verbose showed that __aullrem is pulled in by an object file that
is then discarded but I decided that following that trail wasn't worth
it. The file sizes are identical and dumpbin /headers shows that the
.text sections virtual size is just 96 bytes larger in the gn case.

Additional changes of source set to static_library may address the
remaining diffs, but they are not large enough to matter.

TL;DR The linker tries to discard unreferenced symbols but it is not
perfect at doing it, even with /Gw and /Gy. Assembly functions may be
difficult also. Use source_set with care.

BUG= 624274 

Review-Url: https://codereview.chromium.org/2114783002
Cr-Commit-Position: refs/heads/master@{#403480}

[modify] https://crrev.com/0ab90cb1edf8f08f1fa5d2551aa21cac225163cf/build/linux/unbundle/yasm.gn
[modify] https://crrev.com/0ab90cb1edf8f08f1fa5d2551aa21cac225163cf/components/crash/content/app/BUILD.gn
[modify] https://crrev.com/0ab90cb1edf8f08f1fa5d2551aa21cac225163cf/third_party/yasm/yasm_assemble.gni

The change in crrev.com/2114783002 fixed chrome_elf.dll but made little difference to the other DLLs.

I'm investigating chrome.dll sizes right now and I looked at the changes in the sizes of all of the different sections. Here are the increases (official build) when going from gyp to gn:

.data		774144
.gfids		0
.rdata		167936
.reloc		16384
.rodata		8192
.rsrc		180224
.text		458752
.tls		0
CPADinfo	4096
_RDATA		0
-----------------------
Total		1609728

The .text (code) increase is significant, but the .data increase is actually larger. And, note that the total increase is 1.6 MB, while the file-size regression is only about 1 MB. That's because the table above is measuring the size that these sections will take in memory, not on disk. In particular, the on-disk size of the .data section only increased by 10,752 bytes (from 261,120 to 271,872 bytes), but the in-memory size increased by 772,756 bytes (from 620,628 to 1,393,384 bytes). Dumping the largest global variables shows where the growth is coming from:

131072 ff_cos_65536
131072 ff_sin_65536
 65536 ff_sin_32768
 65536 ff_cos_65536_fixed
 65536 ff_cos_32768
 53248 av_crc_table
 32768 ff_sin_16384
 32768 ff_cos_32768_fixed

~256 KiB for ff_cos_*, ~256 KiB for ff_sin_*, ~128 KiB for ff_cos_*_fixed, so that's 640 KiB of .data segment growth without including av_crc_table.

Presumably this is from third_party\ffmpeg's fft_template.c, rdft.c, and crc.c getting linked in to the gn build but not the gyp build.

Presumably there is some source_set which, when changed to a static_library, will correct this behavior, but I have not yet found it.

Reassigning to brucedawson@chromium.org, who appears to be driving this.

I decided to compare non-official non-branded release builds of chrome.dll and chrome_child.dll, in order to let me iterate faster on finding where the size discrepancies are coming from. I wrote a script (attached) to run dumpbin on the binaries and dump the section sizes (in memory and on disk) using decimal MB. The results are:

python c:\bin\pe_summarize.py out\gn_release\chrome.dll
      name:   mem size  ,  disk size
     .text: 30.333747 MB
    .rdata:  6.081878 MB
     .data:  1.250824 MB,  0.274944 MB
      .tls:  0.000025 MB
   .rodata:  0.005840 MB
  CPADinfo:  0.000036 MB
    .gfids:  0.001016 MB
    _RDATA:  0.000304 MB
     .rsrc:  0.313352 MB
    .reloc:  1.268972 MB

python c:\bin\pe_summarize.py out\release\chrome.dll
      name:   mem size  ,  disk size
     .text: 37.922862 MB
    .rdata:  5.911258 MB
     .data:  0.635028 MB,  0.265216 MB
      .tls:  0.000025 MB
    .gfids:  0.001008 MB
    _RDATA:  0.004384 MB
     .rsrc:  0.132504 MB
    .reloc:  1.388296 MB

python c:\bin\pe_summarize.py out\gn_release\chrome_child.dll
      name:   mem size  ,  disk size
     .text: 32.075918 MB
    .rdata:  9.712010 MB
     .data:  1.382008 MB,  0.480768 MB
      .tls:  0.000025 MB
  CPADinfo:  0.000036 MB
   .rodata:  0.005840 MB
    .gfids:  0.001016 MB
    _RDATA:  0.000304 MB
     .rsrc:  0.052344 MB
    .reloc:  1.274728 MB

python c:\bin\pe_summarize.py out\release\chrome_child.dll
      name:   mem size  ,  disk size
     .text: 40.095884 MB
    .rdata:  9.692196 MB
     .data:  1.378904 MB,  0.480768 MB
      .tls:  0.000025 MB
   .rodata:  0.005840 MB
    .gfids:  0.001028 MB
    _RDATA:  0.007200 MB
     .rsrc:  0.001640 MB
    .reloc:  1.471292 MB

So... in this particular build configuration (GYP_DEFINES=<>, args.gn=is_component_build = false, is_debug = false, target_cpu="x86"):

The .text sections (code) are significantly *smaller* in gn builds
The .rdata+.rodata sections (read-only data) are slightly bigger in gn builds
The .data (read/write data sections) have a significantly larger in-memory size in chrome.dll, as mentioned in comment #12
The .rsrc sections are bigger in gn builds
The .reloc sections are smaller in gn builds, which is just a symptom of the .text segments being smaller
The DLLs are smaller in gn builds, due to the .text and .reloc sections being much smaller.

The fact that the size regression does not repro in non-official builds means that the investigation is made more difficult since each test requires waiting for a full LTCG link cycle.

I'm going to investigate the .rsrc differences since those look easier, and non-trivial.

Deleted: pe_summarize.py 1.5 KB

pe_summarize.py 1.5 KB View Download

Extra resource size is entirely from:

  obj/chrome/installer/util/strings/installer_util_strings.res

It turns out that adding a 181,056 byte file to your .rsrc section makes it bigger by about that amount.

With that removed the chrome.dll .rsrc section is slightly smaller (because it is intentionally omitting the ETW resources).

icu's smallintformatter.cpp has a 16 KiB array (4096 entries, each a 32-bit int) that holds the number of digits in the numbers from 0 to 4095. This appears to be linked into chrome.dll and chrome_child.dll, for an easy savings of 24 KiB merely by changing the type from int32_t to int8_t. This is not gn specific.

Comparing gyp/gn official builds of chrome.exe shows (with tiny/matching sections omitted):

gyp:
Size of out\Release\chrome.exe is 1.044992 MB
  name:   mem size  ,  disk size
 .text:  0.638265 MB
.rdata:  0.296022 MB
 .data:  0.028488 MB,  0.004096 MB
_RDATA:  0.000288 MB
 .rsrc:  0.074792 MB

gn:
Size of out\gn_Release_official\chrome.exe is 1.258496 MB
  name:   mem size  ,  disk size
 .text:  0.820636 MB
.rdata:  0.229980 MB
 .data:  0.057700 MB,  0.005120 MB
.pdata:  0.033480 MB
_RDATA:  0.032816 MB
 .rsrc:  0.125496 MB

chrome.exe's .rsrc section is ~50 KB larger in gn than in gyp builds. That is due to Cursor resources, which come from:

  out\gn_Release\obj\ui\resources\ui_unscaled_resources_grd\ui_unscaled_resources.res

which is 49,848 bytes.

gn's .text section is noticeably larger.
gn's .rdata section is smaller but it's _RDATA section is larger - just different placement of data?
gn has a 33,480 .pdata section that is missing from gyp builds - this is used for exception handling.
gn's .data section (read/write data) is ~29 KB larger in memory, although only 1 KiB larger on disk.

Experimenting on chrome.exe is much easier because it links much faster on official builds. Link times are approximately 32 s (gyp) and 60 s (gn).

Note to self: don't compare 64-bit builds to 32-bit builds. The .pdata section should have been a dead giveaway. Trying again with 32-bit builds for both:

gyp:
Size of out\release\chrome.exe is 1.044992 MB
      name:   mem size  ,  disk size
     .text:  0.638265 MB
     .rsrc:  0.074792 MB

gn:
Size of out\gn_release_official\chrome.exe is 1.152000 MB
      name:   mem size  ,  disk size
     .text:  0.694089 MB
     .rsrc:  0.125496 MB

The .rsrc section difference remains, which makes sense. The .text section difference is smaller, but still there. The other sections are close enough that the differences don't matter.

According to sizeviewer.py:
~3500 bytes of .text difference is in third_party
~900 bytes of .text difference is in base
but after that the differences just diminish away into five bytes here, and ten bytes there.

Comparing dia2dump -p output showed some differences (_sha256_block_data_order, _bn_sub_words) from boringssl assembly language functions but those are already in .lib files, so they must be being pulled in, indirectly or otherwise, by .obj files that are linked directly.

Looking at the .rsp file shows ~300 extra .obj files being linked in directly. Iterating through the relevant projects is looking promising - ~50 KB of code saved on the first one I tried.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/d4ab8b5452afb86f8915a94adcbe629e51480a2f

commit d4ab8b5452afb86f8915a94adcbe629e51480a2f
Author: brettw <brettw@chromium.org>
Date: Wed Jul 13 02:31:01 2016

Remove dependencies on installer util strings

The two references to this target were added by me long before I understood how the installer strings work (see comment at the top of chrome/installer/util/BUILD.gn).

This adds assert_no_deps at the toplevel to catch accidental regressions.

This shrinks chrome.dll by ~182,000 bytes.

BUG= 624274 

Review-Url: https://codereview.chromium.org/2134423002
Cr-Commit-Position: refs/heads/master@{#404916}

[modify] https://crrev.com/d4ab8b5452afb86f8915a94adcbe629e51480a2f/chrome/BUILD.gn
[modify] https://crrev.com/d4ab8b5452afb86f8915a94adcbe629e51480a2f/chrome/browser/BUILD.gn
[modify] https://crrev.com/d4ab8b5452afb86f8915a94adcbe629e51480a2f/chrome/browser/ui/BUILD.gn

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/d4ab8b5452afb86f8915a94adcbe629e51480a2f

commit d4ab8b5452afb86f8915a94adcbe629e51480a2f
Author: brettw <brettw@chromium.org>
Date: Wed Jul 13 02:31:01 2016

Remove dependencies on installer util strings

The two references to this target were added by me long before I understood how the installer strings work (see comment at the top of chrome/installer/util/BUILD.gn).

This adds assert_no_deps at the toplevel to catch accidental regressions.

This shrinks chrome.dll by ~182,000 bytes.

BUG= 624274 

Review-Url: https://codereview.chromium.org/2134423002
Cr-Commit-Position: refs/heads/master@{#404916}

[modify] https://crrev.com/d4ab8b5452afb86f8915a94adcbe629e51480a2f/chrome/BUILD.gn
[modify] https://crrev.com/d4ab8b5452afb86f8915a94adcbe629e51480a2f/chrome/browser/BUILD.gn
[modify] https://crrev.com/d4ab8b5452afb86f8915a94adcbe629e51480a2f/chrome/browser/ui/BUILD.gn

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/fe9b427bc2c5c840906daef07c6dd6789cfc6447

commit fe9b427bc2c5c840906daef07c6dd6789cfc6447
Author: brucedawson <brucedawson@chromium.org>
Date: Fri Jul 15 02:02:51 2016

Reduce chrome.exe .text size by 50 KB with static_library

Also improves build times.

chrome.exe's .text (code) section in gn builds is about 55 KB larger
than in gyp builds (32-bit official). This is largely or completely
because of the use of source sets. The gn build links with about 320
.obj files whereas the gyp build links with ten. The linker pulls in
all of those .obj files and then fails to do a perfect job of discarding
the unused code and data.

This change gets rid of ~50 KB of this discrepancy by changing several
build targets from source_set to static_library and reducing the number
of .obj files to be linked to 72.

The size gain came from the change to components/variations, but the
other changes help to ensure that the size gain sticks, and help with
build speeds.

This change reduces the build times for relinking the affected
binaries (mostly chrome.dll and chrome_child.dll) in an official build
about 4-7%.

Some source_set targets could not be changed. These were annotated as
such to avoid wasting time on them in the future.

BUG= 624274 

Review-Url: https://codereview.chromium.org/2149483002
Cr-Commit-Position: refs/heads/master@{#405671}

[modify] https://crrev.com/fe9b427bc2c5c840906daef07c6dd6789cfc6447/breakpad/BUILD.gn
[modify] https://crrev.com/fe9b427bc2c5c840906daef07c6dd6789cfc6447/build/secondary/third_party/crashpad/crashpad/client/BUILD.gn
[modify] https://crrev.com/fe9b427bc2c5c840906daef07c6dd6789cfc6447/build/secondary/third_party/crashpad/crashpad/compat/BUILD.gn
[modify] https://crrev.com/fe9b427bc2c5c840906daef07c6dd6789cfc6447/build/secondary/third_party/crashpad/crashpad/handler/BUILD.gn
[modify] https://crrev.com/fe9b427bc2c5c840906daef07c6dd6789cfc6447/build/secondary/third_party/crashpad/crashpad/minidump/BUILD.gn
[modify] https://crrev.com/fe9b427bc2c5c840906daef07c6dd6789cfc6447/build/secondary/third_party/crashpad/crashpad/snapshot/BUILD.gn
[modify] https://crrev.com/fe9b427bc2c5c840906daef07c6dd6789cfc6447/build/secondary/third_party/crashpad/crashpad/util/BUILD.gn
[modify] https://crrev.com/fe9b427bc2c5c840906daef07c6dd6789cfc6447/build/secondary/third_party/libjpeg_turbo/BUILD.gn
[modify] https://crrev.com/fe9b427bc2c5c840906daef07c6dd6789cfc6447/components/bookmarks/common/BUILD.gn
[modify] https://crrev.com/fe9b427bc2c5c840906daef07c6dd6789cfc6447/components/browser_watcher/BUILD.gn
[modify] https://crrev.com/fe9b427bc2c5c840906daef07c6dd6789cfc6447/components/crash/content/app/BUILD.gn
[modify] https://crrev.com/fe9b427bc2c5c840906daef07c6dd6789cfc6447/components/variations/BUILD.gn
[modify] https://crrev.com/fe9b427bc2c5c840906daef07c6dd6789cfc6447/mojo/public/cpp/bindings/BUILD.gn
[modify] https://crrev.com/fe9b427bc2c5c840906daef07c6dd6789cfc6447/third_party/brotli/BUILD.gn
[modify] https://crrev.com/fe9b427bc2c5c840906daef07c6dd6789cfc6447/third_party/libpng/BUILD.gn
[modify] https://crrev.com/fe9b427bc2c5c840906daef07c6dd6789cfc6447/ui/gfx/BUILD.gn

Status update:

The .rsrc discrepancy in chrome.exe is explained by this comment. Once GYP is gone the 50 KB will be saved and the size difference will be ~6 KB (probably not worth tracking down).

# TODO(GYP_GONE) when GYP is gone, this line should be deleted, and the grit
# file updated to no longer list this as an output. We use resource bundles
# rather than .rc files, but the GYP build still expects this file to
# exist.
#"ui_unscaled_resources.rc",

On my local release 32-bit official builds:
    chrome.dll:
gyp: 37.572 MB
gn : 38.761 MB

    chrome_child.dll:
gyp: 47.253 MB
 gn: 46.847

So, gyp is will far ahead on chrome.dll, but gn is smaller on chrome_child.dll

On chrome.dll the gn build has a larger .text segment and a slightly larger .rdata section. It also has a 0.9 MB larger in-memory size for the .data segment, due to linking in many extra global arrays (cos, sin, etc.)

On chrome_child.dll the gn build has a 50 KB larger .rsrc file - this will also be fixed by the GYP_GONE change listed above. The gn build's .text segment is .459 MB smaller, for unknown reasons.

These tests were done after Brett's fix to make blink a library. The gyp/gn builds are not actually from identical repos, but they should be close enough.

So, next area of focus is chrome.dll - long build times ahead. The things I learned while working on chrome.exe should help. I expect there is some magic source_set which will fix it all.

http://bugs.icu-project.org/trac/ticket/12390 tracks the 16 KB gDigitCount in ICU array which Chrome gets two copies of. Resolving this will eventually save us 24-48 KB of data segment size in gyp and gn.

 Issue 629333  has been merged into this issue.

Tracking down the differences in the functions in the gyp and gn versions of chrome.dll was found to be fruitless due to the huge noise levels in the data. So, I used a custom version of dia2dump.exe to dump the largest globals and compare them. It is easier to see unnecessary globals, and it is hoped that addressing them will also cause some code bloat to be avoided.

ff_cos_65536_fixed (64 KiB plus ~64 KiB of mip-maps) is pulled in to chrome.dll because ffmpeg_internal is a source_set. Fixing this saves ~128 KiB of .data memory size and allows for future improvements.

ff_sin_65536 (128 KiB plus ~128 KiB of mip-maps) is defined in ffmpeg's rdft.c. Running linker_verbose_tracking.py on the verbose linker output shows that it is referenced by:
- service_worker_network_provider.obj, through source_set("child") in content/child
- render_frame_observer.obj, through source_set("renderer_sources") - changing this shrunk the .text segment by 180 KB
- render_widget_mus_connection.obj, source_set("mus") - changing this saved 263 KB of .data memory size

ff_cos_65536 (128 KiB plus ~128 KiB of mip-maps) is defined in fft_template.c which is #included by fft_float.c. It is referenced by:
- audio_video_metadata_extractor.obj and media_file_checker.obj, through source_set("base") in media/base - changing this saves 107 KB of .text and ~316 KB of in-memory data!

Total savings available from this tactic therefore are:
ff_cos_*_fixed: 128 KiB of .data memory size
ff_sin_*:       256 KiB of .data memory size, ~180 KB of .text
ff_cos_*:       316 KiB of .data memory size, ~107 KB of .text

The main remaining data discrepancies are listed below. The initialized data is particularly important because it reduces the binary size as well as the in-memory size.

Section 3 (uninitialized data):
53064	unigram_table
Section 2 (initialized data):
8192	gl::g_mantissa
12875	gpu::ApplyFramebufferAttachmentCMAAINTELResourceManager::cmaa_frag_s1_
15714	gpu::ApplyFramebufferAttachmentCMAAINTELResourceManager::cmaa_frag_s2_

Some of the numbers are a bit vague because the link times discourage doing perfect science, but the broad strokes are correct and I'm preparing to land these initial changes.

I'm curious if this will also reduce the Android build size then too!

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/third_party/ffmpeg/+/24ea727552a0eaa95c8de72f99cba8d70ae658cb

commit 24ea727552a0eaa95c8de72f99cba8d70ae658cb
Author: Bruce Dawson <brucedawson@chromium.org>
Date: Tue Jul 19 20:58:25 2016

Make ffmpeg a static_library in non-component builds

In non-component builds ffmpeg must be a static library because
otherwise its .obj files are linked directly into chrome.dll, causing
100s of KB of unneeded global variables and code to be pulled in, even
in official builds.

The immediate affect of this is for ff_cos_65536_fixed (and its
mip-maps), totaling almost 128 KiB of global variables to no longer be
linked in. This change also enables future changes that will keep many
other globals and code out of chrome.dll.

BUG= 624274 

Change-Id: I1831ef3882c87ec45db9e881475edf6e29a58dbc

[modify] https://crrev.com/24ea727552a0eaa95c8de72f99cba8d70ae658cb/BUILD.gn

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/0eceb16c8c8056c4cdd37589af42a2f1d1fd2407

commit 0eceb16c8c8056c4cdd37589af42a2f1d1fd2407
Author: brucedawson <brucedawson@chromium.org>
Date: Wed Jul 20 04:38:04 2016

Roll third_party\ffmpeg d45f90eac..24ea72755 (2 commits)

https://chromium.googlesource.com/chromium/third_party/ffmpeg.git/+log/d45f90eac6d5..24ea727552a0

$ git log d45f90eac..24ea72755 --date=short --no-merges --format='%ad %ae %s'
2016-07-19 brucedawson Make ffmpeg a static_library in non-component builds
2016-06-24 petar.jovanovic [MIPS] Add build configurations for mips64el Linux

TBR=dalecurtis@chromium.org
BUG= 624274 

Review-Url: https://codereview.chromium.org/2162053003
Cr-Commit-Position: refs/heads/master@{#406495}

[modify] https://crrev.com/0eceb16c8c8056c4cdd37589af42a2f1d1fd2407/DEPS

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/058b59c936aeceebdc94bc3677966e56fd73a7a8

commit 058b59c936aeceebdc94bc3677966e56fd73a7a8
Author: brucedawson <brucedawson@chromium.org>
Date: Wed Jul 20 17:54:34 2016

Reduce chrome.dll size by careful use of static_library

Linking with a source_set tells the linker to pull in all of the code
and data from the specified object files, and then discard those that
can be proved to be unnecessary. This can cause extraneous global
variables and code to be retained, which has led to a regression in the
size of chrome.dll in gn builds, compared to gyp builds.

This change avoids linking in three arrays from ffmpeg. All of these
arrays have multiple instances, each half the size of the previous, so
the listed sizes below should all be doubled. The necessary changes were
tracked down by using dia2dump to find large global variables and then
running a python script to find the chain of object files that caused
these global to be pulled in. The arrays are:

ff_cos_65536_fixed - 64 KB, fixed by a previous ffmpeg change which enables these fixes.
ff_sin_65536 - 128 KiB is defined in ffmpeg's rdft.c. Pulled in by:
- service_worker_network_provider.obj, through //content/child:child
- render_frame_observer.obj, through //content/public/renderer:renderer_sources
- render_widget_mus_connection.obj, //content/renderer/mus:mus
- resource_converter.obj, through //content/renderer:renderer
ff_cos_65536 - 128 KiB is defined in ffmpeg's fft_template.c which is #included by fft_float.c. Pulled in by:
- audio_video_metadata_extractor.obj and media_file_checker.obj, through //media/base:base

Changing ffmpeg and the five source_set targets to static_library targets (conditionally in some cases) means that these arrays no longer get pulled in.

The expected in-memory savings in the .data section are (64+128+128)*2 KiB = 640 KiB. Actual savings were 718 KiB. This does not affect file size.

In addition this saved 288 KiB of code in the .text section, and shrunk the read-only data section, for a 301 KiB file-size savings.

Before: size of chrome.dll is 38.808576 MB
      name:   mem size  ,  disk size
     .text: 30.999296 MB
    .rdata:  6.007834 MB
     .data:  1.447656 MB,  0.270336 MB

After: size of chrome.dll is 38.499840 MB
      name:   mem size  ,  disk size
     .text: 30.704163 MB
    .rdata:  6.006906 MB
     .data:  0.712680 MB,  0.270336 MB

Measurements were done on 32-bit official builds from hash b8c16c8de1. Still some more work to be done.

BUG= 624274 

Review-Url: https://codereview.chromium.org/2163823002
Cr-Commit-Position: refs/heads/master@{#406611}

[modify] https://crrev.com/058b59c936aeceebdc94bc3677966e56fd73a7a8/content/child/BUILD.gn
[modify] https://crrev.com/058b59c936aeceebdc94bc3677966e56fd73a7a8/content/public/renderer/BUILD.gn
[modify] https://crrev.com/058b59c936aeceebdc94bc3677966e56fd73a7a8/content/renderer/BUILD.gn
[modify] https://crrev.com/058b59c936aeceebdc94bc3677966e56fd73a7a8/content/renderer/mus/BUILD.gn
[modify] https://crrev.com/058b59c936aeceebdc94bc3677966e56fd73a7a8/media/base/BUILD.gn

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/058b59c936aeceebdc94bc3677966e56fd73a7a8

commit 058b59c936aeceebdc94bc3677966e56fd73a7a8
Author: brucedawson <brucedawson@chromium.org>
Date: Wed Jul 20 17:54:34 2016

Reduce chrome.dll size by careful use of static_library

Linking with a source_set tells the linker to pull in all of the code
and data from the specified object files, and then discard those that
can be proved to be unnecessary. This can cause extraneous global
variables and code to be retained, which has led to a regression in the
size of chrome.dll in gn builds, compared to gyp builds.

This change avoids linking in three arrays from ffmpeg. All of these
arrays have multiple instances, each half the size of the previous, so
the listed sizes below should all be doubled. The necessary changes were
tracked down by using dia2dump to find large global variables and then
running a python script to find the chain of object files that caused
these global to be pulled in. The arrays are:

ff_cos_65536_fixed - 64 KB, fixed by a previous ffmpeg change which enables these fixes.
ff_sin_65536 - 128 KiB is defined in ffmpeg's rdft.c. Pulled in by:
- service_worker_network_provider.obj, through //content/child:child
- render_frame_observer.obj, through //content/public/renderer:renderer_sources
- render_widget_mus_connection.obj, //content/renderer/mus:mus
- resource_converter.obj, through //content/renderer:renderer
ff_cos_65536 - 128 KiB is defined in ffmpeg's fft_template.c which is #included by fft_float.c. Pulled in by:
- audio_video_metadata_extractor.obj and media_file_checker.obj, through //media/base:base

Changing ffmpeg and the five source_set targets to static_library targets (conditionally in some cases) means that these arrays no longer get pulled in.

The expected in-memory savings in the .data section are (64+128+128)*2 KiB = 640 KiB. Actual savings were 718 KiB. This does not affect file size.

In addition this saved 288 KiB of code in the .text section, and shrunk the read-only data section, for a 301 KiB file-size savings.

Before: size of chrome.dll is 38.808576 MB
      name:   mem size  ,  disk size
     .text: 30.999296 MB
    .rdata:  6.007834 MB
     .data:  1.447656 MB,  0.270336 MB

After: size of chrome.dll is 38.499840 MB
      name:   mem size  ,  disk size
     .text: 30.704163 MB
    .rdata:  6.006906 MB
     .data:  0.712680 MB,  0.270336 MB

Measurements were done on 32-bit official builds from hash b8c16c8de1. Still some more work to be done.

BUG= 624274 

Review-Url: https://codereview.chromium.org/2163823002
Cr-Commit-Position: refs/heads/master@{#406611}

[modify] https://crrev.com/058b59c936aeceebdc94bc3677966e56fd73a7a8/content/child/BUILD.gn
[modify] https://crrev.com/058b59c936aeceebdc94bc3677966e56fd73a7a8/content/public/renderer/BUILD.gn
[modify] https://crrev.com/058b59c936aeceebdc94bc3677966e56fd73a7a8/content/renderer/BUILD.gn
[modify] https://crrev.com/058b59c936aeceebdc94bc3677966e56fd73a7a8/content/renderer/mus/BUILD.gn
[modify] https://crrev.com/058b59c936aeceebdc94bc3677966e56fd73a7a8/media/base/BUILD.gn

Sadly didn't seem to have any effect on Android:

dalecurtis@dalecurtis-macbookpro2 ~/Downloads $ l chrome-android-406609-/apks/lib/armeabi-v7a/*
-rw-r--r--@ 1 dalecurtis  eng  43978644 Jan  1  2001 chrome-android-406609-/apks/lib/armeabi-v7a/libchrome.so
-rw-r--r--@ 1 dalecurtis  eng     87436 Jan  1  2001 chrome-android-406609-/apks/lib/armeabi-v7a/libchromium_android_linker.so

dalecurtis@dalecurtis-macbookpro2 ~/Downloads $ l chrome-android-406618-/apks/lib/armeabi-v7a/*
-rw-r--r--@ 1 dalecurtis  eng  44003220 Jan  1  2001 chrome-android-406618-/apks/lib/armeabi-v7a/libchrome.so
-rw-r--r--@ 1 dalecurtis  eng     87436 Jan  1  2001 chrome-android-406618-/apks/lib/armeabi-v7a/libchromium_android_linker.so

Though ffmpeg on Android is built without video decoders so these may have never been pulled in.

Pity. But, maybe the next round of changes will help.

Then again, Windows is probably particularly susceptible to this sort of regressions because of its three PE (chrome.exe/chrome.dll/chrome_child.dll) architecture - it is easy for code to end up unnecessarily in the wrong place.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/e792bd734f3535995d3d34dba43259965b65c51e

commit e792bd734f3535995d3d34dba43259965b65c51e
Author: brucedawson <brucedawson@chromium.org>
Date: Thu Jul 21 18:34:21 2016

Shrink gn's chrome.dll - now smaller than gyp's

More work to shrink gn's chrome.dll

The three largest globals that were present in gn's chrome.dll but not in gyp's chrome.dll were eliminated by using /verbose linker output to track the object files that pulled them in and then conditionally changing source_set targets to static_library targets. Specifically:

unigram_table, in compact_enc_det.obj
- Referenced by TextResourceDecoder.obj from //third_party/WebKit/Source/core:html - some other source_set targets in this file were also modified

gpu::ApplyFramebufferAttachmentCMAAINTELResourceManager::cmaa_frag_s1_ and cmaa_frag_s2_, in gles2_cmd_apply_framebuffer_attachment_cmaa_intel.obj from //gpu/command_buffer/service:service_sources
- Referenced by gpu_command_buffer_stub.obj from //gpu/ipc/service:ipc_service_sources
- Referenced by gpu_video_decode_accelerator.obj from //media/gpu/ipc/service:service
- Referenced by gpu_child_thread.obj from //content/gpu:gpu_sources
- Referenced by gpu_video_decode_accelerator_factory.obj from //content/public/gpu:gpu_sources

As of R406709 this shrinks gn's 32-bit official chrome.dll file size from 38,907,904 bytes to 37,571,584 bytes - an unexpected 1,336,320 byte savings, mostly from the .text section. There is also ~67,000 bytes of memory-only savings in the zero-init part of the .data section.

At the same revision gyp's 32-bit official chrome.dll file size is 37,843,456 bytes - 271,872 bytes *larger* than the gn version.

There are still globals that are present in gn's chrome.dll but not gyp's chrome.dll, so the optimization technique can still be applied some more, but the priority is much lower now that gn is winning.

This is a follow-on to crrev.com/2163823002.

BUG= 624274 
CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.linux:linux_optional_gpu_tests_rel;master.tryserver.chromium.mac:mac_optional_gpu_tests_rel;master.tryserver.chromium.win:win_optional_gpu_tests_rel

Review-Url: https://codereview.chromium.org/2163933003
Cr-Commit-Position: refs/heads/master@{#406912}

[modify] https://crrev.com/e792bd734f3535995d3d34dba43259965b65c51e/content/gpu/BUILD.gn
[modify] https://crrev.com/e792bd734f3535995d3d34dba43259965b65c51e/content/public/gpu/BUILD.gn
[modify] https://crrev.com/e792bd734f3535995d3d34dba43259965b65c51e/gpu/command_buffer/service/BUILD.gn
[modify] https://crrev.com/e792bd734f3535995d3d34dba43259965b65c51e/gpu/ipc/service/BUILD.gn
[modify] https://crrev.com/e792bd734f3535995d3d34dba43259965b65c51e/media/gpu/ipc/service/BUILD.gn
[modify] https://crrev.com/e792bd734f3535995d3d34dba43259965b65c51e/third_party/WebKit/Source/core/BUILD.gn

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/5f1f5e67ac7cb8c7ff24e453a86c0413092f1acd

commit 5f1f5e67ac7cb8c7ff24e453a86c0413092f1acd
Author: achuith <achuith@chromium.org>
Date: Thu Jul 21 20:08:37 2016

Revert of Shrink gn's chrome.dll - now smaller than gyp's (patchset #1 id:1 of https://codereview.chromium.org/2163933003/ )

Reason for revert:
Request revert by author

Original issue's description:
> Shrink gn's chrome.dll - now smaller than gyp's
>
> More work to shrink gn's chrome.dll
>
> The three largest globals that were present in gn's chrome.dll but not in gyp's chrome.dll were eliminated by using /verbose linker output to track the object files that pulled them in and then conditionally changing source_set targets to static_library targets. Specifically:
>
> unigram_table, in compact_enc_det.obj
> - Referenced by TextResourceDecoder.obj from //third_party/WebKit/Source/core:html - some other source_set targets in this file were also modified
>
> gpu::ApplyFramebufferAttachmentCMAAINTELResourceManager::cmaa_frag_s1_ and cmaa_frag_s2_, in gles2_cmd_apply_framebuffer_attachment_cmaa_intel.obj from //gpu/command_buffer/service:service_sources
> - Referenced by gpu_command_buffer_stub.obj from //gpu/ipc/service:ipc_service_sources
> - Referenced by gpu_video_decode_accelerator.obj from //media/gpu/ipc/service:service
> - Referenced by gpu_child_thread.obj from //content/gpu:gpu_sources
> - Referenced by gpu_video_decode_accelerator_factory.obj from //content/public/gpu:gpu_sources
>
> As of R406709 this shrinks gn's 32-bit official chrome.dll file size from 38,907,904 bytes to 37,571,584 bytes - an unexpected 1,336,320 byte savings, mostly from the .text section. There is also ~67,000 bytes of memory-only savings in the zero-init part of the .data section.
>
> At the same revision gyp's 32-bit official chrome.dll file size is 37,843,456 bytes - 271,872 bytes *larger* than the gn version.
>
> There are still globals that are present in gn's chrome.dll but not gyp's chrome.dll, so the optimization technique can still be applied some more, but the priority is much lower now that gn is winning.
>
> This is a follow-on to crrev.com/2163823002.
>
> BUG= 624274 
> CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.linux:linux_optional_gpu_tests_rel;master.tryserver.chromium.mac:mac_optional_gpu_tests_rel;master.tryserver.chromium.win:win_optional_gpu_tests_rel
>
> Committed: https://crrev.com/e792bd734f3535995d3d34dba43259965b65c51e
> Cr-Commit-Position: refs/heads/master@{#406912}

TBR=brettw@chromium.org,brucedawson@chromium.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG= 624274 

Review-Url: https://codereview.chromium.org/2173453004
Cr-Commit-Position: refs/heads/master@{#406931}

[modify] https://crrev.com/5f1f5e67ac7cb8c7ff24e453a86c0413092f1acd/content/gpu/BUILD.gn
[modify] https://crrev.com/5f1f5e67ac7cb8c7ff24e453a86c0413092f1acd/content/public/gpu/BUILD.gn
[modify] https://crrev.com/5f1f5e67ac7cb8c7ff24e453a86c0413092f1acd/gpu/command_buffer/service/BUILD.gn
[modify] https://crrev.com/5f1f5e67ac7cb8c7ff24e453a86c0413092f1acd/gpu/ipc/service/BUILD.gn
[modify] https://crrev.com/5f1f5e67ac7cb8c7ff24e453a86c0413092f1acd/media/gpu/ipc/service/BUILD.gn
[modify] https://crrev.com/5f1f5e67ac7cb8c7ff24e453a86c0413092f1acd/third_party/WebKit/Source/core/BUILD.gn

So I don't lose this information, after relanding the last change the remaining medium-sized mismatched globals (in gn's chrome.dll but not in gyp's) are:

 4624: WebRtcIsac_kQKltLevelsShape
 3136: WebRtcIsac_kQKltLevelsGain
 2600: webrtc::acm2::ACMCodecDB::database_
 2592: WebRtcIsac_kKltT1Shape
 2048: fixed_divide
 1800: webrtc::acm2::ACMCodecDB::codec_settings_
 1032: nacl_global_rng
 1024: kf_low_motion_minq_8
 1024: rtc_minq_8
 1024: inter_minq_8
 1024: arfgf_low_motion_minq_8
 1024: sad_per_bit4lut_8
 1024: kf_high_motion_minq_8
 1024: fixed_invtbl8
 1024: sad_per_bit16lut_8
 1024: arfgf_high_motion_minq_8
  880: vpx_rv
  864: WebRtcIsac_kLpcMeansShape
  768: vp8_six_tap_mmx
  720: WebRtcIsac_kCos
  536: WebRtcIsac_kQMeanLag2Hi
  528: g_NaCl_log_gio
  512: webrtc::RandomVector::kRandomTable
  510: WebRtcIsac_kQPitchGainCdf

I'll open a separate bug to track investigating these, but this is lower priority.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/6e53a2e16dc26dec41494b02da83a1221b72e7c4

commit 6e53a2e16dc26dec41494b02da83a1221b72e7c4
Author: brucedawson <brucedawson@chromium.org>
Date: Sat Jul 23 19:35:47 2016

Reland of Shrink gn's chrome.dll - now smaller than gyp's (patchset #1 id:1 of https://codereview.chromium.org/2173453004/ )

Reason for revert (reland):
Problem is understood, mostly. Needed to slightly update the patch to fix it. Blink split counts were copied from gyp.

Original issue's description (with indents removed):
More work to shrink gn's chrome.dll

The three largest globals that were present in gn's chrome.dll but not in gyp's chrome.dll were eliminated by using /verbose linker output to track the object files that pulled them in and then conditionally changing source_set targets to static_library targets. Specifically:

unigram_table, in compact_enc_det.obj
- Referenced by TextResourceDecoder.obj from //third_party/WebKit/Source/core:html - some other source_set targets in this file were also modified

gpu::ApplyFramebufferAttachmentCMAAINTELResourceManager::cmaa_frag_s1_ and cmaa_frag_s2_, in gles2_cmd_apply_framebuffer_attachment_cmaa_intel.obj from //gpu/command_buffer/service:service_sources
- Referenced by gpu_command_buffer_stub.obj from //gpu/ipc/service:ipc_service_sources
- Referenced by gpu_video_decode_accelerator.obj from //media/gpu/ipc/service:service
- Referenced by gpu_child_thread.obj from //content/gpu:gpu_sources
- Referenced by gpu_video_decode_accelerator_factory.obj from //content/public/gpu:gpu_sources

As of R406709 this shrinks gn's 32-bit official chrome.dll file size from 38,907,904 bytes to 37,571,584 bytes - an unexpected 1,336,320 byte savings, mostly from the .text section. There is also ~67,000 bytes of memory-only savings in the zero-init part of the .data section.

At the same revision gyp's 32-bit official chrome.dll file size is 37,843,456 bytes - 271,872 bytes *larger* than the gn version.

There are still globals that are present in gn's chrome.dll but not gyp's chrome.dll, so the optimization technique can still be applied some more, but the priority is much lower now that gn is winning.

This is a follow-on to crrev.com/2163823002.

TBR=brettw@chromium.org,achuith@chromium.org
BUG= 624274 
CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.linux:linux_optional_gpu_tests_rel;master.tryserver.chromium.mac:mac_optional_gpu_tests_rel;master.tryserver.chromium.win:win_optional_gpu_tests_rel

Review-Url: https://codereview.chromium.org/2172033002
Cr-Commit-Position: refs/heads/master@{#407372}

[modify] https://crrev.com/6e53a2e16dc26dec41494b02da83a1221b72e7c4/content/gpu/BUILD.gn
[modify] https://crrev.com/6e53a2e16dc26dec41494b02da83a1221b72e7c4/content/public/gpu/BUILD.gn
[modify] https://crrev.com/6e53a2e16dc26dec41494b02da83a1221b72e7c4/gpu/command_buffer/service/BUILD.gn
[modify] https://crrev.com/6e53a2e16dc26dec41494b02da83a1221b72e7c4/gpu/ipc/service/BUILD.gn
[modify] https://crrev.com/6e53a2e16dc26dec41494b02da83a1221b72e7c4/media/gpu/ipc/service/BUILD.gn
[modify] https://crrev.com/6e53a2e16dc26dec41494b02da83a1221b72e7c4/third_party/WebKit/Source/core/BUILD.gn

R407372 on the sizes chart at https://chromeperf.appspot.com/group_report?bug_id=621802 shows the promises 1.34 MB size reduction in chrome.dll, from 39,042,600 to 37,701,100.

The original regression was from 37,292,500 to 38,308,900 - 1.016 MB - so I'm closing this as fixed.

The size has increased in the interim, for gyp and gn. Those causes may be worth investigating separately, but this bug is fixed.

P.S. Previous changes that show up on the graph include 404916 (installer strings) which shows up as a 196 KB shrink, and 406611 (sin/cos work) which shows up as a 230 KB shrink.

So, total file size savings of 1.77 MB appear to be confirmed.

The follow-on bug for future reductions is  bug 630755 . Also relevant is the size regression  bug 641017 .