1) We can limit it
2) It's possible, I'd need to modify puppet to allow that.

3 and 4 I've just not had time to really work on. The gem in particular we need to install is the json gem. It fails when it doesn't have ruby.h available.

Ping?

Still haven't had time to dig into this. It's on my plate for next week.

Ok, after re-evaluating our bootstrap scripts, it looks like just installing Xcode CommandLine Tools is enough to bootstrap the machine (gems and python modules seem to install fine contrary to when I first tried this). xcodebuild will still throw an error if called and Xcode is not installed:

xcode-select: error: tool 'xcodebuild' requires Xcode, but active developer directory '/Library/Developer/CommandLineTools' is a command line tools instance

It's worth noting that other users of Mac hosts require Xcode installed and that this new set up will cause a another configuration drift, unless it's going to be broadcasted that Xcode is now part of the toolchain for all of the other Chromium projects.

I'll still need to address #2 in the OP as well as tightening up the perms for xcodebuild. I'd also like to reimage build9-m1 to have it mirror what I just tested to see if it fails in any odd way.

Not having Xcode installed would be nice, since if there's an error in the hermetic scripts and Xcode is installed the compile might fall back to the installed xcode-select and weirdness may ensue.

"""
It's worth noting that other users of Mac hosts require Xcode installed and that this new set up will cause a another configuration drift, unless it's going to be broadcasted that Xcode is now part of the toolchain for all of the other Chromium projects.
"""

I think that we should move all Mac hosts to use hermetics builds. 

> Not having Xcode installed would be nice, since if there's an error in the hermetic scripts and Xcode is installed the compile might fall back to the installed xcode-select and weirdness may ensue.

So you're suggesting all Mac builders get reimaged? Because that's what that sounds like. That's not going to be a very fast process.

> I think that we should move all Mac hosts to use hermetics builds. 

How could this be communicated? We (systems) don't have a list of "this is all who uses Macs in our infra".

> How could this be communicated? We (systems) don't have a list of "this is all who uses Macs in our infra".

This process should be transparent (the compiled results should be indistinguishable), so we could just send email to the relevant teams [skia, blink, etc.] when we update their builders to use hermetic builds.

That being said, this is not a high priority - I was just suggesting that image drift is not a requirement - the whole point of hermetic builds is to decouple compile configuration from OS image/manual bot configuration.

Issue 589491 has been merged into this issue.

Now that this has been rolling out and smut tightened up xcodebuild perms, what's left to do here from my side? I can only think of trying a freshly deployed bot without system-wide Xcode installed at all, just the CLT package.

dba@ that would be a good test.  We had to make some changes to work on systems that did have Xcode installed, so it's possible a non-XCode system is broken.  Can we test an xcode-less bot on FYI?

I can set one up, or is there one that I could just redeploy?

Also, do we need to be testing different OS X versions? I believe we've only been testing 10.11.

I reverted the config for 'Chromium 10.11 Force Mac Toolchain' here:
https://codereview.chromium.org/1946023004/diff/1/scripts/slave/recipe_modules/chromium_tests/chromium_fyi.py

Perhaps we can put it back, and reuse that?

That's a lot of bots, i'd rather just target a couple of bots with each version (10.9, 10.10, and 10.11).

I know that's FYI and all.. but I'd rather not break a large amount of them.

I am suggesting only testing on the 'Chromium 10.11 Force Mac Toolchain' bot.

Sorry, it wasn't clear from that CL where it touched a bunch of other things.

Redeployed build9-m1 with no system Xcode. However it looks likes its been failing runs for a while.

I reverted the config in cl 1946023004 when hermetic was enabled everywhere.  Sent over https://codereview.chromium.org/2005523003/ to get the config back.  thanks!

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build.git/+/772692cbb72e1195ed8d3386c098281ef480d6f5

commit 772692cbb72e1195ed8d3386c098281ef480d6f5
Author: justincohen <justincohen@chromium.org>
Date: Tue May 24 21:51:45 2016

Restore Force Mac Toolchain config.

BUG= 601506 

Review-Url: https://codereview.chromium.org/2005523003

[modify] https://crrev.com/772692cbb72e1195ed8d3386c098281ef480d6f5/scripts/slave/recipe_modules/chromium_tests/chromium_fyi.py
[add] https://crrev.com/772692cbb72e1195ed8d3386c098281ef480d6f5/scripts/slave/recipes/chromium.expected/full_chromium_fyi_Chromium_Mac_10_11_Force_Mac_Toolchain.json

Looks like the nacl integration script fails on the true xcodeless bot, i'll take a look:

https://build.chromium.org/p/chromium.fyi/builders/Chromium%20Mac%2010.11%20Force%20Mac%20Toolchain/builds/3223

Everything else looks green.

nacl_integration looks green now: https://build.chromium.org/p/chromium.fyi/builders/Chromium%20Mac%2010.11%20Force%20Mac%20Toolchain/builds/3238

phajdan@ landed a fix to pass env to test bots here: https://codereview.chromium.org/2014373002

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infra/puppet/+/ae84ed8b45bc5c7de4bcfd22d802c8f8fd32f083

commit ae84ed8b45bc5c7de4bcfd22d802c8f8fd32f083
Author: smut <smut@google.com>
Date: Wed Apr 27 21:27:31 2016

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infra/puppet/+/b39e7a5abe37393da49d46cf48a9e6c24ad9c9eb

commit b39e7a5abe37393da49d46cf48a9e6c24ad9c9eb
Author: smut <smut@google.com>
Date: Fri May 06 19:18:58 2016

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chrome/tools/build_limited/scripts/slave/+/fac06be1b9367e675daba4c163c4184ab862326b

commit fac06be1b9367e675daba4c163c4184ab862326b
Author: recipe-roller <recipe-roller@chromium.org>
Date: Tue May 24 21:58:32 2016

Don't think theres anything else for me to do here.

Justin - feel free to dupe this if you've got another bug you're doing this work in.