I'm surprised you only have 3 threads?

`build/vs_toolchain.py get_toolchain_dir` could probably take advantage of hypothetical action caching too (I see at least 3 calls), but it's mostly hidden on my machine by having ~30 threads.

Deleted: win.json 5.4 MB

win.json 5.4 MB View Download

This is on my laptop, with 4 physical and 8 virtual cores. Maybe gn uses one thread per physical core?

The current core allocation is based on some timings I did on my desktops. Our old scheme used threads = 2x virtual CPU count but this actually caused some starvation for the main thread which is dispatching work, and decreasing it to the current rule 1/2 virtual CPU count (typically the # of physical cores), with a minimum of 3. I verified that this rul did not slow down my 6 core home desktop. It could be on the low end this needs tweaking and also that our workload has changed a bit.

It would be interesting to see what the time is for different thread counts on your computer. You can just pass --threads=8 for example on the command line.

Since the entire build is parameterized by toolchain, you can end up running a script to get some system value more than once. I've considered adding some kind of "toolchain invariant" annotation to imports to prevent this, but that will get confusing.

I tried it on my MacBook and I notice the script ran 2 extra times due to  bug 599892 .

3-6 seems best on my laptop, but it doesn't make a huge difference:

Nicos-MBP:src thakis$ gn gen out/gn
Done. Wrote 4696 targets from 842 files in 3548ms
Nicos-MBP:src thakis$ gn gen out/gn --threads=8
Done. Wrote 4696 targets from 842 files in 3592ms
Nicos-MBP:src thakis$ gn gen out/gn --threads=2
Done. Wrote 4696 targets from 842 files in 4236ms
Nicos-MBP:src thakis$ gn gen out/gn --threads=3
Done. Wrote 4696 targets from 842 files in 3356ms
Nicos-MBP:src thakis$ gn gen out/gn --threads=3
Done. Wrote 4696 targets from 842 files in 3386ms
Nicos-MBP:src thakis$ gn gen out/gn --threads=4
Done. Wrote 4696 targets from 842 files in 3155ms
Nicos-MBP:src thakis$ gn gen out/gn --threads=5
Done. Wrote 4696 targets from 842 files in 3275ms
Nicos-MBP:src thakis$ gn gen out/gn --threads=6
Done. Wrote 4696 targets from 842 files in 3300ms
Nicos-MBP:src thakis$ gn gen out/gn --threads=7
Done. Wrote 4696 targets from 842 files in 3459ms
Nicos-MBP:src thakis$ gn gen out/gn --threads=8
Done. Wrote 4696 targets from 842 files in 3580ms
Nicos-MBP:src thakis$ gn gen out/gn --threads=9
Done. Wrote 4696 targets from 842 files in 3745ms
Nicos-MBP:src thakis$ gn gen out/gn --threads=10
Done. Wrote 4696 targets from 842 files in 4010ms

[mac triage]

It's a bit strange these scripts take so long when run under gn. If I run find_sdk.py manually, it takes 0.08s. In the gn trace, it takes 0.4s, 5x as long.

Added a suggestion to https://codereview.chromium.org/1752873002/#msg23 about speeding up sdk_info.py

Let's undupe this since it looks like it's possible to at least speed up sdk_info.py quite a bit.

Should we retitle this bug to be specifically about speeding up sdk_info.py?

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/268142df3c289fcd2605cd68f41e04ebade1067b

commit 268142df3c289fcd2605cd68f41e04ebade1067b
Author: sdefresne <sdefresne@chromium.org>
Date: Wed May 18 19:33:38 2016

[GN] Use "xcrun" instead of "xcodebuild" to get information on SDK.

Use "xcrun" instead of "xcodebuild" as it is much faster and available
on all version of OS X supported for building Chrome.

Reduce running time of "gn gen" from 2.5 to 2.2s on MacBook Pro
laptop.

BUG= 609541 

Review-Url: https://codereview.chromium.org/1993653002
Cr-Commit-Position: refs/heads/master@{#394505}

[modify] https://crrev.com/268142df3c289fcd2605cd68f41e04ebade1067b/build/config/mac/sdk_info.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/d50140bae570f519cea29a8b20e58511032cbfd7

commit d50140bae570f519cea29a8b20e58511032cbfd7
Author: thakis <thakis@chromium.org>
Date: Wed May 18 22:00:06 2016

mac: Make `find_sdk.py --print_sdk_path` much faster.

Use xcrun instead of xcodebuild if --print_sdk_path is passed.  Reduces time
to run `gn gen` from 2.2s to 1.95s on my MacBook Pro.

(In the gyp build, we don't pass --print_sdk_path to this script, so I didn't
notice this script had this slow path until now.)

While here, also stop pretending that we support Xcodes older than 4.3.

BUG= 609541 

Review-Url: https://codereview.chromium.org/1988023006
Cr-Commit-Position: refs/heads/master@{#394565}

[modify] https://crrev.com/d50140bae570f519cea29a8b20e58511032cbfd7/build/mac/find_sdk.py

Thanks to brettw's, sdefresne's, and my change, this is now slightly below 2s, over 33% faster. Looks like most low-hanging fruits are picked.

Thanks! :-)