Issue 663220 link

Starred by 1 user

Issue metadata

Status:	Untriaged
Owner:	----
Components:	Build
EstimatedDays:	----
NextAction:	----
OS:	----
Pri:	3
Type:	Bug

Blocking:
issue 787983

Changing linker settings causes excessive compilation

Project Member Reported by brucedaw...@chromium.org, Nov 8 2016

Issue description

When turning is_win_fastlink on or off we would, ideally, just relink the affected binaries. The reality is more complex because many of those binaries are used to generate headers or code, which is itself some extra work. Then, those generated headers and code, despite being unchanged, have updated file dates which forces recompilation.

In the end, when building the 'chrome' target build the settings below I see 27 binaries being relinked and 1322 compilations.

is_debug = false
is_component_build = false
symbol_level = 1
#is_win_fastlink = true

This was tested on Windows but the behavior is probably the same elsewhere.

The most robust fix would be for more generated files, such as gen/mojo/public/tools/bindings/cpp_templates.zip, to be written in such a way that the file data is only changed if the contents change. This is done for some generated files already.

Comment 1 by brucedaw...@chromium.org, Nov 9 2016

More carefully measured numbers, from the 2656 steps run when is_win_fastlink was enabled:

1266 cl.exe steps
 962 stamp steps
 151 lib.exe steps
 115 run_yasm.py steps
  58 tools/protoc_wrapper/protoc_wrapper.py steps
  24 pnacl_newlib/bin/x86_64-nacl-clang++.exe steps
  27 link.exe steps

Plus these (trimmed to fit the screen) miscellaneous build steps:

build/gn_run_binary.py genmacro.exe gen/third_party/yasm/include/nasm-macros.c nasm_standard_m
build/gn_run_binary.py genmacro.exe gen/third_party/yasm/include/win64-nasm.c win64_nasm_stdma
build/gn_run_binary.py genmacro.exe gen/third_party/yasm/include/win64-gas.c win64_gas_stdmac 
build/gn_run_binary.py genmodule.exe ../../third_party/yasm/source/patched-yasm/libyasm/module
build/gn_run_binary.py genstring.exe license_msg gen/third_party/yasm/include/license.c ../../
third_party/WebKit/Source/build/scripts/gperf.py ./character_data_generator gen/blink/platform
build/gn_run_binary.py genversion.exe gen/third_party/yasm/version.mac
build/gn_run_binary.py genperf.exe ../../third_party/yasm/source/patched-yasm/modules/arch/x86
build/gn_run_binary.py genperf.exe gen/third_party/yasm/include/x86insn_nasm.gperf gen/third_p
build/gn_run_binary.py genperf.exe ../../third_party/yasm/source/patched-yasm/modules/arch/x86
build/gn_run_binary.py re2c.exe -s -o gen/third_party/yasm/lc3bid.c ../../third_party/yasm/sou
build/gn_run_binary.py re2c.exe -b -o gen/third_party/yasm/gas-token.c ../../third_party/yasm/
build/gn_run_binary.py re2c.exe -b -o gen/third_party/yasm/nasm-token.c ../../third_party/yasm
build/gn_run_binary.py genperf.exe gen/third_party/yasm/include/x86insn_gas.gperf gen/third_pa
build/gn_run_binary.py flatc.exe -c -o gen/components/subresource_filter/core/common/flat ../.
v8/tools/run.py ./mkpeephole gen/v8/bytecode-peephole-table.cc
build/gn_run_binary.py genmacro.exe gen/third_party/yasm/include/nasm-version.c nasm_version_m
build/gn_run_binary.py bro.exe --force --input gen/components/resources/about_credits.html --o
tools/grit/grit.py -i ../../components/resources/components_resources.grd build -o gen/compone
chrome/browser/resources/safe_browsing/gen_file_type_proto.py -w -t win -i ../../chrome/browse
tools/grit/grit/format/repack.py chrome_200_percent.pak gen/blink/public/resources/blink_image
tools/grit/grit/format/repack.py chrome_100_percent.pak gen/blink/public/resources/blink_image
build/toolchain/win/tool_wrapper.py recursive-mirror libGLESv2.dll Packages/ui/libGLESv2.dll
build/toolchain/win/tool_wrapper.py recursive-mirror libEGL.dll Packages/ui/libEGL.dll
tools/grit/grit.py -i ../../chrome/browser/browser_resources.grd build -o gen/chrome --depdir 
build/write_buildflag_header.py --output base/allocator/features.h --rulename //base/allocator
build/write_buildflag_header.py --output base/debug/debugging_flags.h --rulename //base:debugg
mojo/public/tools/bindings/generate_type_mappings.py --output irt_x64/gen/ipc/mojom__type_mapp
build/rm.py --stamp irt_x64/gen/mojo/public/cpp/system/clean_up_stamp -f libsystem.dylib
mojo/public/tools/bindings/generate_type_mappings.py --output irt_x64/gen/mojo/public/interfac
build/write_build_date_header.py irt_x64/gen/base/generated_build_date.h default
third_party/dom_distiller_js/protoc_plugins/json_values_converter_tests.py --stamp gen/third_p
mojo/public/tools/bindings/mojom_bindings_generator.py --use_bundled_pylibs precompile -o D:/s
tools/grit/grit/format/repack.py resources.pak gen/blink/public/resources/blink_resources.pak 
build/toolchain/gcc_ar_wrapper.py  --output=irt_x64/obj/components/tracing/proto/libprotos.a -
build/toolchain/win/tool_wrapper.py recursive-mirror tracing_library.dll Packages/tracing/trac
mojo/public/tools/bindings/mojom_bindings_generator.py --use_bundled_pylibs generate ../../moj
mojo/public/tools/bindings/mojom_bindings_generator.py --use_bundled_pylibs generate ../../moj
mojo/public/tools/bindings/mojom_bindings_generator.py --use_bundled_pylibs generate ../../ipc
mojo/public/tools/bindings/mojom_bindings_generator.py --use_bundled_pylibs generate ../../moj
mojo/public/tools/bindings/mojom_bindings_generator.py --use_bundled_pylibs generate ../../ipc
mojo/public/tools/bindings/mojom_bindings_generator.py --use_bundled_pylibs generate ../../moj
build/toolchain/gcc_ar_wrapper.py  --output=irt_x64/obj/base/libbase.a --ar="../../native_clie
build/toolchain/gcc_ar_wrapper.py  --output=irt_x64/obj/mojo/public/cpp/bindings/libbindings.a
build/toolchain/gcc_ar_wrapper.py  --output=irt_x64/obj/ipc/libipc.a --ar="../../native_client
build/toolchain/gcc_ar_wrapper.py  --output=irt_x64/obj/components/tracing/libtracing.a --ar="
v8/tools/run.py ./mksnapshot --startup_src gen/v8/snapshot.cc --random-seed 314159265 --startu
gin/fingerprint/fingerprint_v8_snapshot.py --snapshot_file snapshot_blob.bin --natives_file na
build/toolchain/win/tool_wrapper.py recursive-mirror irt_x64/exe.unstripped/nacl_irt.nexe nacl
ppapi/native_client/irt_debuglink.py D:/src/chromium/src/native_client/toolchain/win_x86/pnacl
build/toolchain/win/tool_wrapper.py recursive-mirror test_ime_driver_library.dll Packages/test
build/toolchain/win/tool_wrapper.py recursive-mirror ui_library.dll Packages/ui/ui.library
build/win/reorder-imports.py -i initialexe -o . -a x64

Ideally the cl.exe steps could be skipped, by careful modification of other steps when they don't change their results.