While investigating  crbug.com/617837 , slowdowns in win_chromium_rel_ng, I found that the main cause of slowdowns in gn builds is reduced parallelism compared to gyp.

In local tests on a 48-thread Z840 with GYP_LINK_CONCURRENCY set to (a very conservative) 5 the gyp build finished 181 s faster (total elapsed time in the table below) whereas the sum of the times for all of the steps was less for the gn build (the Total step time/50 row is the theoretically best time possible, assuming perfect parallelism):

                       gn          gyp
Total step time:    106562.9 s (108626.0 s)
Total step time/50:   2131.3 s (  2172.5 s)
Total elapsed time:   2390.0 s (  2209.3 s)

This can also be seen in the chrome://tracing view of the .ninja_log data of the last 400 s of each build - perfect parallelism versus highly limited parallelism. Images attached.

One reason for the reduced parallelism is compilation versus linking of unit tests. In general the unit test compiles happen earlier for gn builds than gyp builds - 7.5 minutes earlier on average. Meanwhile, gn's linking of *test*.exe binaries starts, on average, 10.5 minutes later. So, when gn goes to link the test executables that is all that it has left to do and the parallelism is throttled to GYP_LINK_CONCURRENCY levels, which can be extremely low on some machines. gyp builds, on the other hand, are compiling test files almost to the end.

The long gap between creating and using the test object files (18 minutes later for gn compared to gyp) probably also reduces disk cache efficiency.

On this particular test the gn build performance would improve by about 10% if parallelism matched gyp's levels.

On a recent test from the build machine the imperfect parallelism was slowing the build by about 12.8% compared to ideal, compared to 12.1% compared to ideal on local builds:
(https://build.chromium.org/p/chromium/builders/Win/builds/44684/steps/compile/logs/stdio, .ninja_log attached)
gyp comparison results from the build machines are no longer available.

brettw said:
- One thing that affects this is GN's order-only dependencies for all actions. This means that all action/action_foreach targets of all dependents must be complete before compiling any code.

- In GYP this is explicit and generally not set. This means that GYP can start compiling leaf node targets earlier.

- However, this is a feature and not a bug. The GYP build is frequently wrong in subtle ways because of incorrectly or insufficiently specified hard dependencies on generated files.

 

Deleted: parallel_gyp.PNG 14.5 KB

	parallel_gyp.PNG 14.5 KB View Download

Deleted: parallel_gn.png 19.8 KB

	parallel_gn.png 19.8 KB View Download

Deleted: .ninja_log 4.6 MB

.ninja_log 4.6 MB Download