Support non-using rsp file build on windows |
||
Issue descriptionninja takes around 25% cpu time to create/delete rsp file in windows. It becomes one of build speed limiting factor when goma backend has sufficiently warm cache. Let me support the build not using rsp files. CreateProcess supports 32,768 characters command line, it should be sufficient for current chrome building. https://msdn.microsoft.com/ja-jp/library/windows/desktop/ms682425(v=vs.85).aspx
,
Dec 19 2017
,
Dec 19 2017
Pasting in information from the duplicate bug: When doing a NOP build of Chrome (ninja -n chrome, make sure you have explicitly run "gn gen" first) the StartEdge and FinishCommand stages consume most of the time (use -d stats to see this). StartEdge takes ~20 s and FinishCommand takes ~8 s. The vast majority of this (on my workstation) is spent in creating and deleting .rsp files. This was confirmed when I did ETW profiling of the NOP build which showed that the vast majority of this time was in the OS - in kernel functions or perhaps in anti-virus. This overhead means that a NOP build of chrome (with jumbo and goma enabled so there are "only" 25,470 build steps) takes ~36 s. Experiments with disabling of .rsp creation and deletion shows that it reduces the NOP build time to ~8 s. The effect on 'real' builds is unknown but it is possible that the absolute savings may be similar, or at least the .rsp file creation savings may apply. In particular note that on -n builds of Chrome the RSP files account for more than 75% of elapsed time, or about 28 s. On real builds of Chrome this change seems to help ninja ramp up to full build speed slightly faster, although that is hard to tell for sure.
,
Jan 5 2018
The following revision refers to this bug: https://chrome-internal.googlesource.com/goma/client/+/ff39c26fa5c23c70b88179a3d0a5e499250dc90f commit ff39c26fa5c23c70b88179a3d0a5e499250dc90f Author: Takuto Ikuta <tikuta@google.com> Date: Fri Jan 05 09:12:48 2018
,
Jan 13 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/d2164444bcfdf6e4e5eadc55bcfb77fd74f9dba8 commit d2164444bcfdf6e4e5eadc55bcfb77fd74f9dba8 Author: Takuto Ikuta <tikuta@chromium.org> Date: Sat Jan 13 01:33:45 2018 Do not use rsp files for windows compile Creating and deleting rsp files is time consuming part when building on windows with warmed goma backend cache. To reduce such bottleneck, we can use all compiler flags directory in command line instead of using rsp file. This CL keeps using rsp file for linking because linking flags can be longer than compiles and the number of linking is relatively small compared to compile. I took 3 time build stats for 'chrome' on 48 thread Z840 windows 10 with following args.gn and -j 500. ``` goma_dir = "C:\\src\\goma_client\\client\\out\\Release" is_component_build = true is_debug = false strip_absolute_paths_from_debug_symbols = true symbol_level = 0 target_cpu = "x86" use_goma = true is_clang = false enable_nacl = false use_lld = true ``` * With this CL TotalSeconds: 279.9604737 TotalSeconds: 279.4099064 TotalSeconds: 273.8253059 Avg: 277.731895333333 * Without this CL TotalSeconds: 325.0155664 TotalSeconds: 319.4094433 TotalSeconds: 299.9709 Avg: 314.798636566667 Bug: 796021 Change-Id: Ice959c196b6879b39962a3b628cdcf531884ec36 Reviewed-on: https://chromium-review.googlesource.com/832593 Reviewed-by: Nico Weber <thakis@chromium.org> Reviewed-by: Scott Graham <scottmg@chromium.org> Reviewed-by: Bruce Dawson <brucedawson@chromium.org> Reviewed-by: Dirk Pranke <dpranke@chromium.org> Commit-Queue: Takuto Ikuta <tikuta@chromium.org> Cr-Commit-Position: refs/heads/master@{#529137} [modify] https://crrev.com/d2164444bcfdf6e4e5eadc55bcfb77fd74f9dba8/build/toolchain/win/BUILD.gn
,
Jan 14 2018
,
Jan 17 2018
I just tested "ninja.exe -n -C out\default chrome" on a freshly created out\default directory (no args.gn file). This used to be painfully slow, mostly from the creation and destruction of .rsp files (~36 s). It now takes ~9 s, or ~20 s if VsChromium is running (creating file notification objects takes a while). I don't remember if the ~36 s was with VsChromium running or not. The new 9/20 s times are with a non-jumbo build. A jumbo build would be faster. So, yeah. It's faster now.
,
Apr 5 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/74612b2ba5df3042eca38da71223f2a22114ba40 commit 74612b2ba5df3042eca38da71223f2a22114ba40 Author: Nico Weber <thakis@chromium.org> Date: Thu Apr 05 17:47:33 2018 Stop passing --rsp-quoting to clang-cl. We no longer use rsp files for compiles as of https://chromium-review.googlesource.com/832593 Bug: 796021 Change-Id: I2ffb54f5bf2a596025af9a43d9e06fd992234f48 Reviewed-on: https://chromium-review.googlesource.com/997994 Reviewed-by: Hans Wennborg <hans@chromium.org> Commit-Queue: Nico Weber <thakis@chromium.org> Cr-Commit-Position: refs/heads/master@{#548475} [modify] https://crrev.com/74612b2ba5df3042eca38da71223f2a22114ba40/build/toolchain/win/BUILD.gn |
||
►
Sign in to add a comment |
||
Comment 1 by brucedaw...@chromium.org
, Dec 19 2017