Justin, could you take a look?  Seems like it might be related to hermetic building?

If system headers really change from under us, then we need to use -MD instead of -MMD in the mac toolchain gni file (this will tell the compiler to make a cc file depend on system include headers in addition to user headers). However, this will make incremental builds noticeably slower (way more files to stat). Alternatively, we could try to add an explicit dep on some file that always gets touched when Xcode is modified.

The Best Fix (tm) for this is imho to put Xcode at a content-addressed directory name, so that when Xcode changes, the path to it changes too, the -isysroot flag we pass to clang changes, and ninja's commandline tracking (instead of its mtime tracking) causes full rebuilds. That's what we do on Windows.

Why can't CIPD preserve the mtime when restoring an Xcode cache for the same version?

How often does the CIPD cache get pruned?  If this happens a lot and we go with the content-addressed directory name after every prune we are going to get lots of full rebuilds

That said, content-addressed directory names for different versions of Xcode does make sense, since that should cause a full rebuild.

thakis: thanks for the info!

Xcode is already installed in a folder that includes xcode build version - e.g. cache/xcode_ios_9c40b.app - see above. So it is content-addressed in this sense.

I think it's indeed fine to rebuild only when the path changes, though it still creates an implicit dependency on the internal implementation of how infra delivers toolchains, and might break again in the future if we change it again. For cleanliness, I'd be inclined to try the -MD argument and see what performance hit we actually get.

The caveat is that currently, even though ninja doesn't rebuild the headers, it still wants them to be newer than the source, and this causes the (unnecessary) crash. With your suggested path-based fix, will that check also be done for the path instead of mtime?

Note that the error is from clang, not ninja. clang insists on checking pch and h timestamps. So either the cache needs to copy the original mtimes, or use different paths for the same revision with different mtimes.

Re: #c3: on buildbot slaves at the moment the cache is pruned pretty much after every build. I'm not entirely sure why, but it only added 2 min of unpacking time from local CIPD cache, and buildbot slaves will go away hopefully by the end of Q1. So I didn't particularly care to investigate. On LUCI bots, we'll be using better managed "named caches" which do persist a lot better, if the tester swarming bots are any indication.

If this would cause too much of a rebuild, I'll look closer at how this cache is currently managed and see if I can make it stay longer. But it's still a cache that will be pruned occasionally, and technically we don't provide any guarantees about the Xcode path, at least not today. This is, obviously, negotiable, but needs to be an explicit design decision on infra's part, otherwise we may break it again in the future.

Thought's on justincohen's

> Why can't CIPD preserve the mtime when restoring an Xcode cache for the same version?

above?

Re: #c5 - I see. In this case, I'd say let's make sure ninja recompiles the headers the way clang expects them. And I'll look into the CIPD mtime story, maybe it can be changed.

Here's a quick find so far: https://godoc.org/github.com/luci/luci-go/cipd/client/cipd

  DeterministicZip = zip archive with deterministic ordering of files and stripped timestamps

Deterministic archiving is fundamental to CIPD, since its packages are content-addressed by their hash, and if the original timestamps are stripped, I'm guessing CIPD has nothing to restore.

> let's make sure ninja recompiles the headers the way clang expects them

Sure, the question is just how to achieve that :-) Ideally we can find a way without requiring stat()ing of all system header files on each build, like we've been able to since the beginning of time.

https://godoc.org/github.com/luci/luci-go/cipd/client/cipd makes sense for binaries that we build ourselves, but maybe less for versioned binary blobs that we get from vendors (like Xcode). So maybe cipd could grow a "retain timestamp". +vadim show wrote that page according to github blame.

Having said that, justincohen reported seeing this issue locally after Xcode updates too, so if we were able to fix this outside of cipd that'd help devs too presumably.

Yes, I think we're on the same page, both #c10 and #c11 :-)

The reason we were able to avoid stat()'ing system files all this time was because these files were statically put there mainly by sysadmins at the time of machine deployment. This is now changing rapidly, since it's no longer feasible for sysadmins to add new dependencies to the machines directly as frequently as we'd like them to. With the upcoming LUCI / swarming, machines will also be reusable across builds with potentially different requirements, which adds to the complexity and volatility of the dependencies.

IMHO, it's safest to start with a definitely correct approach of stat()'ing all the system headers, at least in a sample tryjob (no committed CLs yet), and see what performance hit we may get. If it's not too bad, maybe we'll just go with that. That would be the most robust way to solve it. If it is too expensive, then we'll look into optimizations.

In particular, we can create an explicit dependency on something inside Xcode (pretty much any file always known to exist) and rebuild based on that.

Additionally, I'll look into persisting buildbot slave's cache better. If it's easy to fix, it may be worth it even though it'll be replaced by LUCI soon.

Changing to -MD the first time will cause a full rebuild since the compile command is different.  What we want to test is the first rebuild after the change to -MD plus a CIPD cache rebuild.  How do we test that with a trybot?

I'd reuse ios-simulator-cronet for this (since it's not yet using goma), re-enable PCH on it in the CL (in particular, this will force it to run compile for that CL), and update the -MD arg (I'm hoping this can also be done src-side in the same CL?).

Then trigger the tryjob a couple of times. This bot only has 2 machines, so if it hits two different machines, a third run will surely test what we want.

We can add "preserve timestamps" flag to cipd for this particular case if it makes sense. The primary reason we strip them is to avoid generating "different" packages from same reproducibly built code. Xcode package is built by hand, and not frequently, so it's less important there.

Thanks, Vadim.

Before we consider that, does ninja force a full recompile if it detects that Xcode version has changed, regardless of its timestamps?
E.g. what would happen if the same build first requests Xcode v.1, then Xcode v.2, and reuses the same build dir?

@sergeyberezin: we embed the xcode version into the command line: https://cs.chromium.org/chromium/src/build/config/mac/BUILD.gn?l=31 . When the command line changes, ninja recompiles.

That's great, thanks! In this case, preserving timestamps would indeed solve this problem.

To make this right though, we explicitly need to define this guarantee on the infra side, so future changes to the pipeline (or new pipelines for other toolchains / system dependencies) don't break this guarantee. I'm mentioning it here explicitly FTR.

It's seems like preserving timestamps might be the best solution going forward.  Flipping over to infra@

I don't know if this bug needs an owner just yet. I'll sit on it for a bit, and if I don't get time to do it, will make it available. I don't believe it's urgent enough at the moment.

Putting into the CIPD component since this is a CIPD bug.

Moving discussion from https://groups.google.com/a/google.com/forum/#!topic/chrome-mac-dev/vpFre579JDY to this bug:

On Mon, Feb 12, 2018 at 1:18 PM Nico Weber <thakis@google.com> wrote:
> I suppose what's different on dev machines is that the cipd-installed xcode doesn't get wiped all the time, so it'll end up a in a different directory each time it gets revved (which means the flags will change and ninja will rebuild). Is that correct?

Not quite - on dev machines it'll always install to the same place. However, ninja will still recompile for a new Xcode. On bots, however, I'm thinking of changing the install path per version, to utilized managed caches. And preserving timestamps will be more relevant for the bots, since caches will occasionally get evicted.

> And this is a one-time only thing because for the initial deploy, cipd writes to the same place as the old installer?

I misremembered in my first reply, and thought the same problem with headers will come up when Xcode gets updated. But apparently it shouldn't, according to #c17. However, you also mentioned in #c11 that this problem might still be happening despite #c17.

So, I'm inclined to take a closer look at CIPD and what it takes to add this feature. If it's relatively easy, maybe I'll just do that.

Why don't we do a content-hash-dirname thing on dev machines too? We do for Windows, and it's needed for devs to get full rebuilds on Xcode rev, no?

That's a separate feature that we can do later if needed; I'd like to keep this change minimal if possible. I don't see how it would solve the timestamp issue though, since ninja would still not rebuild headers when Xcode changes, and clang would still complain. According to #c17, updating Xcode version in the same place should still result in full recompile, so I don't think unique dir names are strictly required for a rebuild (though I agree they seem cleaner).

One thing I indeed forgot - and thank you for bringing this bug to my attention - on the bots, when Xcode cache expires (which will happen more frequently than on dev's machines), builds that still use precompiled headers will fail until clobbered. So I still need to look into CIPD and timestamps just for that reason.

WIP CL: https://crrev.com/c/920751 (not yet ready to review, just FYI)

> I don't see how it would solve the timestamp issue though, since ninja would still not rebuild headers when Xcode changes

Ninja reruns edges whose command line changes. So if the new Xcode is at a different path, we pass a different -isysroot to all compilers, causing ninja to rerun all compile edges.

I see, thanks for explaining.

Anyway, I still think it's a separate feature that we can add later. For now, I'm focused on the CIPD preserving timestamps, which is blocking the primary issue of upgrading the Xcode pipeline.

In leu of a design doc for the CIPD CL:

- I'm adding flags to preserve file mtime and user-writable flag (since I'm at it, and Xcode needs both);
- These flags will propagate through the stack that creates a package, and zip archive will have writable flag and non-zero mtime on the affected files in the archive;
- I initially planned to save these flags to the package manifest, but realized they are not actually needed there, so maybe I'll revert those changes;
- At install time, I will *not* look at the manifest, and instead rely on each file's mtime and writable bit and set those on the created file.

As a result, the old (default) behavior of CIPD should not change; that is, it'll produce identical archives and hashes as before the change, and installing a package will correctly restore the extra bits if they were set.

A WIP CL is up:

However, my initial design in #c29 doesn't work because of https://chromium.googlesource.com/infra/luci/luci-go/+/master/cipd/client/cipd/local/builder.go#142
where zipped files are always saved as "writable", but that bit later is ignored.

I can change the initial file mode to 0400, but that will change the hash of the same zip files.
So the alternative seems to be to add "preserve writable" flag to the manifest and initialize the file mode bits correctly only when this flag is set.

Forgot to add the CL link: https://crrev.com/c/920751 

I think we can make stuff always writable, it's not a big deal.

Great! Then I won't need to change the manifest. It's unfortunate that we're stuck with this u+w bit though... I wonder if we can come up with a mechanism to transition to u-w by default without breaking the world. But that'll be later, in a separate and likely Pri-3 bug.

CLs in review:
- https://crrev.com/c/920751 (CIPD)
- https://crrev.com/c/963691 (mac_toolchain)

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/7ed42e5cd8886a738295152f9840f04d6a6eb17b

commit 7ed42e5cd8886a738295152f9840f04d6a6eb17b
Author: Sergey Berezin <sergeyberezin@google.com>
Date: Fri Mar 16 00:03:15 2018

CIPD: add option to preserve mtime and writable bit

BUG= 803158 
R=vadimsh@chromium.org

Change-Id: Ib6921d922ccb0855e9a66e6bb3ea5f5e1071d501
Reviewed-on: https://chromium-review.googlesource.com/920751
Commit-Queue: Sergey Berezin <sergeyberezin@chromium.org>
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>

[modify] https://crrev.com/7ed42e5cd8886a738295152f9840f04d6a6eb17b/cipd/client/cipd/client.go
[modify] https://crrev.com/7ed42e5cd8886a738295152f9840f04d6a6eb17b/cipd/client/cipd/client_test.go
[modify] https://crrev.com/7ed42e5cd8886a738295152f9840f04d6a6eb17b/cipd/client/cipd/local/builder.go
[modify] https://crrev.com/7ed42e5cd8886a738295152f9840f04d6a6eb17b/cipd/client/cipd/local/builder_test.go
[modify] https://crrev.com/7ed42e5cd8886a738295152f9840f04d6a6eb17b/cipd/client/cipd/local/deployer_test.go
[modify] https://crrev.com/7ed42e5cd8886a738295152f9840f04d6a6eb17b/cipd/client/cipd/local/files.go
[modify] https://crrev.com/7ed42e5cd8886a738295152f9840f04d6a6eb17b/cipd/client/cipd/local/files_test.go
[modify] https://crrev.com/7ed42e5cd8886a738295152f9840f04d6a6eb17b/cipd/client/cipd/local/json_descs.go
[modify] https://crrev.com/7ed42e5cd8886a738295152f9840f04d6a6eb17b/cipd/client/cipd/local/pkgdef.go
[modify] https://crrev.com/7ed42e5cd8886a738295152f9840f04d6a6eb17b/cipd/client/cipd/local/reader.go
[modify] https://crrev.com/7ed42e5cd8886a738295152f9840f04d6a6eb17b/cipd/client/cipd/local/reader_test.go
[modify] https://crrev.com/7ed42e5cd8886a738295152f9840f04d6a6eb17b/cipd/client/cipd/local/testing.go
[modify] https://crrev.com/7ed42e5cd8886a738295152f9840f04d6a6eb17b/cipd/client/cli/main.go

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/e3f2e013238540fbe0596bc1288bbfd4551de8e1

commit e3f2e013238540fbe0596bc1288bbfd4551de8e1
Author: Sergey Berezin <sergeyberezin@google.com>
Date: Fri Mar 16 00:31:43 2018

Roll luci-go to 7ed42e5c

Relevant changes:
  7ed42e5c CIPD: add option to preserve mtime and writable bit
  dbef627f [milo] Add ViewURL to GetCompressedMasterJSON result.
  ef5003bf [token-server] Validate certificate authorities config file.

BUG= 803158 
R=vadimsh@chromium.org

Change-Id: I54f4e2b3b3f94693710fb47c47ac28d0e5dbbd0d
Reviewed-on: https://chromium-review.googlesource.com/965394
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
Commit-Queue: Sergey Berezin <sergeyberezin@chromium.org>

[modify] https://crrev.com/e3f2e013238540fbe0596bc1288bbfd4551de8e1/DEPS

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/3de4acd82efd92d616bd918f4e325d28d6515c05

commit 3de4acd82efd92d616bd918f4e325d28d6515c05
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Sat Mar 17 03:32:42 2018

[cipd] Fix handling of file mode flags for packages with format "1".

It has regressed in https://github.com/luci/luci-go/commit/7ed42e5cd88. This
CL introduced package format "1.1" (with slightly different semantics for +w bit
and modification time propagation), but it never tested that packages with
format "1" still are unpacked correctly. They aren't.

R=iannucci@chromium.org, sergeyberezin@chromium.org
BUG= 803158 

Change-Id: Id968310353680a740e95675333e99ea9db9623b4
Reviewed-on: https://chromium-review.googlesource.com/967430
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>

[modify] https://crrev.com/3de4acd82efd92d616bd918f4e325d28d6515c05/cipd/client/cipd/local/builder.go
[modify] https://crrev.com/3de4acd82efd92d616bd918f4e325d28d6515c05/cipd/client/cipd/local/builder_test.go
[modify] https://crrev.com/3de4acd82efd92d616bd918f4e325d28d6515c05/cipd/client/cipd/local/json_descs.go
[modify] https://crrev.com/3de4acd82efd92d616bd918f4e325d28d6515c05/cipd/client/cipd/local/reader.go
[modify] https://crrev.com/3de4acd82efd92d616bd918f4e325d28d6515c05/cipd/client/cipd/local/reader_test.go
[modify] https://crrev.com/3de4acd82efd92d616bd918f4e325d28d6515c05/cipd/client/cli/main.go

I'm looking for places where CIPD version is pinned, so I can roll it (unless someone else is going to roll it - please LMK!). Here's what I found so far:

depot_tools.git//cipd_client_version
infradata/config.git:
  //configs/gce-backend/metadata/tricium/startup.sh
  //configs/gce-backend-dev/metadata/tricium/startup.sh
  // configs/chrome-swarming/settings.cfg
  // configs/chromium-swarm-dev/settings.cfg
  // configs/chromium-swarm/settings.cfg
  // configs/omnibot-legion-swarming-server/settings.cfg
  // configs/omnibot-swarming-server/settings.cfg
  // configs/touch-swarming/settings.cfg

build.git:
  https://cs.chromium.org/chromium/build/scripts/slave/cipd_bootstrap_v2.py?l=21&rcl=926645a389c7db8d81f80f38140ac502dda2f0df

Anything else I missed? Does this cover all the LUCI and buildbot builders and swarming bots?

Ah, here's another place: https://chrome-internal.googlesource.com/infra/puppet/+/master/puppetm/etc/puppet/hieradata/cipd.yaml#195

Looks like the latest CIPD version is git_revision:4d19637ec2c3d1efd8c6a1b05285118b786919e2 , so I'll start rolling it to canaries where those are available.

We'll need to update vpython at the same time, since it links to CIPD guts internally.

Considering this deployment is potentially dangerous I propose to do it at least in two stages:

1. Less important stuff first:
  a) CIPD pin in settings.cfg for all Swarming servers except chromium-swarm.
  b) vpython pin in swarming_task_template_canary.json in cr-buildbucket config.
  c) 'canary' pin for cipd in cipd.yaml in puppet.
  d) STAGING_CIPD_VERSION in cipd_bootstrap_v2.py for Buildbot.
  e) vpython STAGING pin in cipd_bootstrap_v2.py.
  f) startup.sh for dev tricium.
 
2. Let it sit for several hours, or preferably overnight. Confirm nothing breaks.

3. Update everything else:
  a) CIPD pin in settings.cfg for chromium-swarm.
  b) Promote swarming_task_template_canary.json to stable via 'bbroll' tool.
  c) 'stable' pin for cipd in cipd.yaml in puppet.
  d) DEFAULT_CIPD_VERSION in cipd_bootstrap_v2.py for Buildbot.
  e) vpython default pin in cipd_bootstrap_v2.py.
  f) startup.sh for prod tricium.
  g) cipd_client_version in depot_tools.
  h) vpython pin in cipd_manifest.txt in depot_tools.

Great, thanks, Vadim, for the detailed instructions!

I got the first CL for puppet: https://crrev.com/i/592823 
will prepare the other CLs for #1 shortly.

Since I'll be OOO Fri, I'm OK keeping the canary till Monday and deploying it to stable then. (It also appears CIPD canary in puppet was different from stable for a long while...)

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infra/puppet/+/0d4280eedc981b8dfc8475d08d0b583ed983fbed

commit 0d4280eedc981b8dfc8475d08d0b583ed983fbed
Author: Sergey Berezin <sergeyberezin@google.com>
Date: Thu Mar 22 00:31:43 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/1829dcbcf1e7f5b46fdbd72764db935174915524

commit 1829dcbcf1e7f5b46fdbd72764db935174915524
Author: Sergey Berezin <sergeyberezin@google.com>
Date: Thu Mar 22 18:22:02 2018

Roll CIPD staging to git_revision:4d19637ec2c3d1efd8c6a1b05285118b786919e2

Relevant changes:
  fe4e3046 [cipd] Treat io.ErrUnexpectedEOF as a corruption error.
  0d68f886 Proofread CIPD README.md and fix links
  e18d39eb [cipd] Rename cipd/client/cipd/common into cipd/common to use it from server.
  1db11af6 [cipd] Fix how the file size is presented in the progress output.
  3de4acd8 [cipd] Fix handling of file mode flags for packages with format "1".
  7ed42e5c CIPD: add option to preserve mtime and writable bit

BUG= 803158 
R=vadimsh@google.com

Change-Id: I904bf43bdc381b3f459e49642684dd174b5ae8eb
Reviewed-on: https://chromium-review.googlesource.com/974756
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
Commit-Queue: Sergey Berezin <sergeyberezin@chromium.org>

[modify] https://crrev.com/1829dcbcf1e7f5b46fdbd72764db935174915524/scripts/slave/cipd_bootstrap_v2.py

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infradata/config/+/b3f0a17ee25991b3672eab5d14925f12e6a5d9d7

commit b3f0a17ee25991b3672eab5d14925f12e6a5d9d7
Author: Sergey Berezin <sergeyberezin@google.com>
Date: Thu Mar 22 19:06:18 2018

Spot-checking chrome-swarming and touch-swarming tasks with the new CIPD version seems to WAI. I'm going to let it bake till Monday and push to stable then.

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infradata/config/+/48a002e4c84497febc829e8ccfd23eb8084470dc

commit 48a002e4c84497febc829e8ccfd23eb8084470dc
Author: Sergey Berezin <sergeyberezin@google.com>
Date: Tue Mar 27 20:40:23 2018

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infradata/config/+/9b96c0f6c36fff09ca927bb9d2ef1d06251e4982

commit 9b96c0f6c36fff09ca927bb9d2ef1d06251e4982
Author: Sergey Berezin <sergeyberezin@google.com>
Date: Tue Mar 27 20:40:48 2018

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infra/puppet/+/abab6a79a69671288c6be4c87dfc053a11a46b4b

commit abab6a79a69671288c6be4c87dfc053a11a46b4b
Author: Sergey Berezin <sergeyberezin@google.com>
Date: Tue Mar 27 21:02:40 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/491e42dd3ce33cc2f761706b4111119d98d33fbc

commit 491e42dd3ce33cc2f761706b4111119d98d33fbc
Author: Sergey Berezin <sergeyberezin@google.com>
Date: Wed Mar 28 00:08:30 2018

Roll CIPD and vpython prod to 4d19637

Relevant changes:
  fe4e3046 [cipd] Treat io.ErrUnexpectedEOF as a corruption error.
  0d68f886 Proofread CIPD README.md and fix links
  e18d39eb [cipd] Rename cipd/client/cipd/common into cipd/common to use it from server.
  1db11af6 [cipd] Fix how the file size is presented in the progress output.
  3de4acd8 [cipd] Fix handling of file mode flags for packages with format "1".
  7ed42e5c CIPD: add option to preserve mtime and writable bit

BUG= 803158 
R=martiniss@chromium.org

Change-Id: Ic09e30ed4a1fbd8cdcfbc7b47845ddfe53aff48b
Reviewed-on: https://chromium-review.googlesource.com/982551
Reviewed-by: Stephen Martinis <martiniss@chromium.org>
Commit-Queue: Sergey Berezin <sergeyberezin@chromium.org>

[modify] https://crrev.com/491e42dd3ce33cc2f761706b4111119d98d33fbc/scripts/slave/cipd_bootstrap_v2.py

The first ios-simulator build to use the new CIPD successfully installed Xcode: https://ci.chromium.org/buildbot/tryserver.chromium.mac/ios-simulator/422050

I think it's safe to land the depot_tools change now.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/depot_tools/+/96fc33383b6d11e2c21cad26d131873ea67fb1ad

commit 96fc33383b6d11e2c21cad26d131873ea67fb1ad
Author: Sergey Berezin <sergeyberezin@google.com>
Date: Wed Mar 28 02:13:33 2018

Roll CIPD and vpython versions to git_revision:4d19637ec2c3d1efd8c6a1b05285118b786919e2

Bug:  803158 
Change-Id: Ie42a726a85d86ff4d62d141a6413886787760835
R=martiniss@chromium.org
Reviewed-on: https://chromium-review.googlesource.com/982599
Reviewed-by: Stephen Martinis <martiniss@chromium.org>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Commit-Queue: Sergey Berezin <sergeyberezin@chromium.org>

[modify] https://crrev.com/96fc33383b6d11e2c21cad26d131873ea67fb1ad/cipd_client_version
[modify] https://crrev.com/96fc33383b6d11e2c21cad26d131873ea67fb1ad/cipd_manifest.txt

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/092f4974c74d29429e2b08961df4b1ac9b9841a2

commit 092f4974c74d29429e2b08961df4b1ac9b9841a2
Author: depot-tools-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com <depot-tools-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Date: Wed Mar 28 07:21:36 2018

Roll src/third_party/depot_tools/ fe68c91e4..96fc33383 (2 commits)

https://chromium.googlesource.com/chromium/tools/depot_tools.git/+log/fe68c91e470a..96fc33383b6d

$ git log fe68c91e4..96fc33383 --date=short --no-merges --format='%ad %ae %s'
2018-03-27 sergeyberezin Roll CIPD and vpython versions to git_revision:4d19637ec2c3d1efd8c6a1b05285118b786919e2
2018-03-27 agable fetch: stop specifying .DEPS.git for any repos

Created with:
  roll-dep src/third_party/depot_tools
BUG= chromium:803158 ,chromium:824734


The AutoRoll server is located here: https://depot-tools-chromium-roll.skia.org

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.


TBR=agable@chromium.org

Change-Id: I4046988bc9c279f7ca9d2064b5f4c7d4193c24bf
Reviewed-on: https://chromium-review.googlesource.com/983001
Reviewed-by: depot-tools-chromium-autoroll <depot-tools-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Commit-Queue: depot-tools-chromium-autoroll <depot-tools-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#546429}
[modify] https://crrev.com/092f4974c74d29429e2b08961df4b1ac9b9841a2/DEPS