New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 621999 link

Starred by 2 users

Issue metadata

Status: WontFix
Owner:
Closed: Oct 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Windows
Pri: 2
Type: Bug

Blocking:
issue 82385



Sign in to add a comment

Switching the CrWinClang bot to gn regressed its cycle time by 50%

Project Member Reported by thakis@chromium.org, Jun 21 2016

Issue description

https://build.chromium.org/p/chromium.fyi/stats/CrWinClang

See the four builds on the far right.

That build is doing an official 32-bit static library release build (no LTCG since clang and link.exe can't make that happen). It does clobber builds each time, but uses goma.

I thought most large source_sets are static_libraries now -- is such a huge increase in build time expected?
 
Screen Shot 2016-06-21 at 2.29.45 PM.png
284 KB View Download
On the upside, the generate_build_files step is 500% faster - saves about 40 seconds. Look on the bright side?

On local VC++ static_library builds I see 32%/25% increases in build times on 32-bit and 64-bit respectively, so same ball park.

shared_library builds are much less affected.

I believe there are still some large source sets due to hitting the 4 GB limit on .lib files. Is there an easy way to dump the names and contents of all the source sets?
generate_build_files.PNG
31.5 KB View Download
Blocking: 354261

Comment 3 by brettw@chromium.org, Jun 21 2016

List all source sets with their sources:
  gn desc out/Default "//*" sources --type=source_set

A large percentage of the build is still source sets. For the 32-bit size issue chrome/browser and chrome/browser/ui are still source sets on official builds. And most small targets that aren't components are also source sets.

content and gpu are also still source sets because they need to be for the component build, and I don't want to introduce a new target type for "source set in component build and static library in static build" because nobody will understand that. gpu should be more significantly refactored which may make this easier, but that's hard to do while still supporting GYP.

I tried changing a bunch of source sets and it's really hard in general due to the way that things get combined into components. The speed advantages of static libraries are also nonlinear because if some stuff is still source sets, it will bring in stuff from static libraries that might otherwise be skipped.

Comment 4 by thakis@chromium.org, Jun 23 2016

https://build.chromium.org/p/chromium.fyi/stats/ClangToTWin64 was affected as well (not quite as badly). It also does official builds.

How about giving gn something like msvs_shard so that big source_sets can be static_libraries in official builds as well? It's a bit yucky, but not that horrible, and it seems to help a lot.
Screen Shot 2016-06-23 at 5.16.20 PM.png
181 KB View Download
Labels: -Pri-3 Pri-2
We see similar regressions in cycle time on the 'Win x64 Builder' on chromium.perf; the cycle time seems to maybe be up to 4x slower. See  bug 619949 
Status: Available (was: Untriaged)

Comment 7 by thakis@chromium.org, Jun 28 2016

See also bug 623661. from there: "brett is working on a msvs_shard equivalent; see https://codereview.chromium.org/2095043005/"

Comment 8 by thakis@chromium.org, Jun 28 2016

Blocking: 82385
A request for future build-time regression bugs - we should grab the .ninja_log files from before/after the regression and attach them to the bug, to allow analysis of what went wrong after the fact.

I should have thought to grab them last week.
Blocking: -354261
As suggested by laforge@ and a conversation w/ the monorail folks, I'm going to try tracking GN-Migration related issues by *just* using the Proj-GN-Migration label, and not using blocking/rollup bugs, so that we can use blocking for just tasks that truly need to be completed before other tasks can make progress.
Labels: -Proj-GN-Migration
Owner: thakis@chromium.org
@thakis - do you still want this open?

I'm clearing the Proj-GN-Migration label since this clearly didn't block the GN migration :).
It's still true that the bot is much slower than it used to be. But if nobody's looking at it, then having this bug open isn't of much value either.
Status: WontFix (was: Available)
:-(

Sign in to add a comment