It's also strange that a succeeding build has and cctest.exe with 52786688. This is quite some fluctuation from build to build.
See https://isolateserver.appspot.com/browse?namespace=default-gzip&hash=b5f4d405afb3db4b01fab47141c4350d4ee2669b

@jochen: Wasn't also cctest supposed to be shared library or is that still blocked on something? If it were shared, we could shard the dlls.

As a shot in the dark, I'll land a landmine to rule out an orphan dll that gets added as suggested in  issue 560522 .

no, cctest is the last remaining full binary :-/

The following revision refers to this bug:
  https://chromium.googlesource.com/v8/v8.git/+/e96076ee4293352fc899cd89a1ba653e60021a44

commit e96076ee4293352fc899cd89a1ba653e60021a44
Author: machenbach <machenbach@chromium.org>
Date: Mon Nov 28 08:23:02 2016

[build] Landmine for flaky win compile problems.

BUG= chromium:668958 
TBR=jochen@chromium.org

Review-Url: https://codereview.chromium.org/2531183002
Cr-Commit-Position: refs/heads/master@{#41296}

[modify] https://crrev.com/e96076ee4293352fc899cd89a1ba653e60021a44/gypfiles/get_landmines.py

Didn't happen anymore after landmine. So maybe it was a stray dll...

Now the problem started appearing again on a random CL:
https://build.chromium.org/p/client.v8/builders/V8%20Win64%20-%20debug/builds/13739

Not sure if the MSVS2015 switch will help here, but we better get it done to rule it out. Marking as blocking.

It's not totally clear *what* file is exceeding the 2 GB limit. My guess is it is the .ilk file (used for incremental linking) in which case VS 2015 will help a lot because the limit is raised to 4 GB. 

If that's not the problem then it might be the .pdb file. .pdb files are reused and due to fragmentation and *mumble* *mumble* *who* *knows* they can grow significantly over a repeated series of non-clobber builds. This was causing build flakes that were addressed by deleting the .pdb before every link. This doesn't waste significant time and means that the PDBs are always as small as possible. See this CL: https://codereview.chromium.org/2519803003

If it's not one of those then ??? but knowing which file was hitting the problem would be nice.

Thanks for the pointers! So far it didn't happen again. I'll close this for now again and keep it in my archive. If it happens again, I'll take a look for further action.