Keep an eye on Xcode performance and index size (~/Library/Developer/Xcode/DerivedData/*/*/*/*/*/db.xcindexd) when doing so, that's what caused the separation in the first place.  Sometimes the index would jump from a ~300 MB to 20-30GB.  At that point loading Xcode would take a very long time, as would Xcode usage.

Thank you for letting me know this, Justin. However, one thing I don't quite understand  is that Xcode will need to index all the files in either cases, so if it's not the number of files that causes the index size to explode, then what exactly it it that makes the difference? Do you have any insights on this?

Thanks,
Yuke

Before gyp had an hybrid mode (i.e. generate an Xcode project that invoked ninja, on which the gn generation is based), the gyp generator was generating one project per target defined in gyp and created dependencies between those projects. As most of the projects were depending on say "base" or "net", those projects were loaded and indexed multiple times (in fact, almost all projects depend on "base", so it was loaded N^2 times). This may be what explain the catastrophic behaviour. However, this is just an hypothesis.

Since we don't have access to how Xcode is implemented we cannot know what is/was causing the poor behaviour. So we need to ensure that we have a way to get back to the current projects if the performance are not good enough.

I have experimented with this for two days, and the index size looks more or less like following:

liaoyuke@liaoyuke-macpro bling $ ./check_index_size.sh
472M	/Users/liaoyuke/Library/Developer/Xcode/DerivedData/all-cfbjsqnxnazdvvbdvxkdokiishny//Index/_tempKR33y9/macosx10.12-x86_64/all.xcindex/db.xcindexdb
158M	/Users/liaoyuke/Library/Developer/Xcode/DerivedData/all-cfbjsqnxnazdvvbdvxkdokiishny//Index/Debug/iphoneos10.0-arm64/all.xcindex/db.xcindexdb
 96M	/Users/liaoyuke/Library/Developer/Xcode/DerivedData/all-cfbjsqnxnazdvvbdvxkdokiishny//Index/Debug/iphonesimulator10.0-i386/all.xcindex/db.xcindexdb
788M	/Users/liaoyuke/Library/Developer/Xcode/DerivedData/all-cfbjsqnxnazdvvbdvxkdokiishny//Index/Debug/iphonesimulator10.0-x86_64/all.xcindex/db.xcindexdb
848M	/Users/liaoyuke/Library/Developer/Xcode/DerivedData/all-cfbjsqnxnazdvvbdvxkdokiishny//Index/Debug/macosx10.12-x86_64/all.xcindex/db.xcindexdb
224K	/Users/liaoyuke/Library/Developer/Xcode/DerivedData/all-cfbjsqnxnazdvvbdvxkdokiishny//Index/UnspecifiedConfig/UnspecifiedTarget/all.xcindex/db.xcindexdb

And the sizes have been pretty stable during the past two days.

Should I monitor the index size for a little bit longer? Or it's good to go implementing it with an "escape hatch"?

Looks fine to me. I'd say implement as is. We can always roll back the code if performance turns out to be poor on the long term.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/de19976ca12bb4c42acd5bfb5a83a69047d6e7b4

commit de19976ca12bb4c42acd5bfb5a83a69047d6e7b4
Author: liaoyuke <liaoyuke@chromium.org>
Date: Wed Jan 18 20:43:17 2017

Consolidate Xcode Project Setup

1. Remove 'sources.xcodeproj'
2. Add a 'sources' target under products project for files indexing.
3. Delete xctests/ folder to fix duplicate xctest file copies.

BUG= 679110 

Review-Url: https://codereview.chromium.org/2623203004
Cr-Commit-Position: refs/heads/master@{#444469}

[modify] https://crrev.com/de19976ca12bb4c42acd5bfb5a83a69047d6e7b4/ios/build/tools/convert_gn_xcodeproj.py
[modify] https://crrev.com/de19976ca12bb4c42acd5bfb5a83a69047d6e7b4/tools/gn/xcode_object.cc
[modify] https://crrev.com/de19976ca12bb4c42acd5bfb5a83a69047d6e7b4/tools/gn/xcode_writer.cc
[modify] https://crrev.com/de19976ca12bb4c42acd5bfb5a83a69047d6e7b4/tools/gn/xcode_writer.h