Steps to repro and do this (for future reference)

c:\src\gclient\src>type out\gnrelease64\args.gn
# Build arguments go here. Examples:
#   is_component_build = true
#   is_debug = false
# See "gn args <out_dir> --list" for available build arguments.
is_component_build = false
is_debug = false
target_cpu = "x64"
enable_nacl = false
remove_webcore_debug_symbols = true
is_chrome_branded = true

c:\src\gclient\src>echo gn > out\gnrelease64\mb_type

c:\src\gclient\src>python tools\mb\mb.py isolate out/gnrelease64 sbox_integration_tests

c:\src\gclient\src>python tools\swarming_client\isolate.py archive -s out\gnrelease64\sbox_integration_tests.isolated -I isolateserver.appspot.com
93797136e4c0b7ba179a3cfa7a5b668860319209  sbox_integration_tests

c:\src\gclient\src>python tools\swarming_client\swarming.py run -S chromium-swarm.appspot.com -I isolateserver.appspot.com -s 93797136e4c0b7ba179a3cfa7a5b668860319209 -d cpu x86-64 -d os Windows-10 -d pool Chrome -d machine_type n1-standard-16
Priority was reset to 100
Triggered task: wfh@google.com/cpu=x86-64_machine_type=n1-standard-16_os=Windows-10_pool=Chrome/93797136e4c0b7ba179a3cfa7a5b668860319209
win10-8f430bd8-21h5: 37bdf4495de80610

https://chromium-swarm.appspot.com/task?id=37bdf4495de80610

running --single-process-tests on GCE:

 [  FAILED  ] LpcPolicyTest.TestValidProcessHeaps
  [  FAILED  ] PolicyTargetTest.DesktopPolicy
  [  FAILED  ] ProcessMitigationsTest.CheckWin81DynamicCode_BaseCase
  [  FAILED  ] ProcessMitigationsTest.CheckWin81DynamicCode_TestMitigation
  [  FAILED  ] ProcessMitigationsTest.CheckWin10DynamicCodeOptOut_BaseCase
  [  FAILED  ] ProcessMitigationsTest.CheckWin10DynamicCodeOptOut_TestMitigation
  [  FAILED  ] ProcessMitigationsTest.CheckWin10DynamicCodeOptOut_TestMitigationWithOptOut
  [  FAILED  ] ProcessMitigationsTest.CheckWin10ImageLoadPreferSys32_Success
  [  FAILED  ] ProcessMitigationsTest.CheckWin10ImageLoadPreferSys32_Failure
  [  FAILED  ] CFGSupportTests.MsIndirectFailure

running locally:

[  FAILED  ] LpcPolicyTest.TestValidProcessHeaps
[  FAILED  ] PolicyTargetTest.DesktopPolicy
[  FAILED  ] ProcessMitigationsTest.CheckWin10ImageLoadPreferSys32_Success
[  FAILED  ] ProcessMitigationsTest.CheckWin10ImageLoadPreferSys32_Failure
[  FAILED  ] CFGSupportTests.MsIndirectFailure

both are running 10.0.14393

seems like we should fix the failing local tests at the same time.

Re: sbox_validation_tests that I built myself with isolate hash cbce3d93660439cde5ded23285927c546dd3912a passes fine, but the one from the win7 builder with isolate hash 4f88b7c543bd0830295fa6f4dcdfa9c8e0aaeeaf fails.

gn args to build my cbce3d93660439cde5ded23285927c546dd3912a is

is_component_build = false
is_debug = false
target_cpu = "x64"
enable_nacl = false
remove_webcore_debug_symbols = true
is_chrome_branded = true

gn args to build win7-bot 4f88b7c543bd0830295fa6f4dcdfa9c8e0aaeeaf is

goma_dir = "C:\\b\\c\\goma_client"
is_component_build = false
is_debug = false
strip_absolute_paths_from_debug_symbols = true
symbol_level = 1
target_cpu = "x86"
use_goma = true

can confirm when I build sbox_validation_tests with these gn args, then the tests fail. so the problem on sbox_validation_tests seems to be x86 only?

so one of the root cause issues here seems to be that FindCsrPortHeap is failing when running 32-bit on 64-bit. it seems all heaps have the same flags, perhaps there is no csrss heap with wow64?

https://chromium-review.googlesource.com/c/602615 fixes the csrss issue (disable CSRSS lockdown on 32-bit...) this makes validation_tests pass on all platforms fine.

but there's still failing tests (all on 14393) on sbox_integration_tests:

64-bit:

[  FAILED  ] PolicyTargetTest.DesktopPolicy
[  FAILED  ] ProcessMitigationsTest.CheckWin10ImageLoadPreferSys32_Success
[  FAILED  ] ProcessMitigationsTest.CheckWin10ImageLoadPreferSys32_Failure
[  FAILED  ] CFGSupportTests.MsIndirectFailure

32-bit on 64-bit:

[  FAILED  ] PolicyTargetTest.DesktopPolicy
[  FAILED  ] ProcessMitigationsTest.CheckWin10ImageLoadPreferSys32_Success
[  FAILED  ] ProcessMitigationsTest.CheckWin10ImageLoadPreferSys32_Failure

32-bit on 32-bit:

[  FAILED  ] FilePolicyTest.AllowNtCreatePatternRule
[  FAILED  ] FilePolicyTest.TestQueryAttributesFile
[  FAILED  ] PolicyTargetTest.DesktopPolicy
[  FAILED  ] ProcessMitigationsTest.CheckWin10ImageLoadPreferSys32_Success
[  FAILED  ] ProcessMitigationsTest.CheckWin10ImageLoadPreferSys32_Failure

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/fb04104d0e09de58995e9776d77bab6e6da14f09

commit fb04104d0e09de58995e9776d77bab6e6da14f09
Author: Will Harris <wfh@chromium.org>
Date: Wed Aug 09 18:54:27 2017

Disable CSRSS lockdown on 32-bit.

Also, disable the heap validation tests as calling HeapLock on certain
heaps can cause exception.

BUG=751809

Cq-Include-Trybots: master.tryserver.chromium.win:win10_chromium_x64_rel_ng
Change-Id: I805aeb1ea0ce326fbb32f1a7d92bb7b80431e095
Reviewed-on: https://chromium-review.googlesource.com/602615
Commit-Queue: Will Harris <wfh@chromium.org>
Reviewed-by: Penny MacNeil <pennymac@chromium.org>
Cr-Commit-Position: refs/heads/master@{#493079}
[modify] https://crrev.com/fb04104d0e09de58995e9776d77bab6e6da14f09/content/common/sandbox_win.cc
[modify] https://crrev.com/fb04104d0e09de58995e9776d77bab6e6da14f09/sandbox/win/src/lpc_policy_test.cc
[modify] https://crrev.com/fb04104d0e09de58995e9776d77bab6e6da14f09/sandbox/win/src/sandbox_policy.h
[modify] https://crrev.com/fb04104d0e09de58995e9776d77bab6e6da14f09/sandbox/win/src/sandbox_policy_base.cc
[modify] https://crrev.com/fb04104d0e09de58995e9776d77bab6e6da14f09/sandbox/win/src/sandbox_policy_base.h

PreferSys32 fixes up for review: https://chromium-review.googlesource.com/c/611310 

I'll look at CFG next.

CFG fix up for review: https://chromium-review.googlesource.com/c/617817

Down to only PolicyTargetTest.DesktopPolicy now.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/99d57e43708d07a32abedc4e954adae6af2d2a5c

commit 99d57e43708d07a32abedc4e954adae6af2d2a5c
Author: Penny MacNeil <pennymac@chromium.org>
Date: Thu Aug 17 20:45:16 2017

[Windows Sandbox Tests] ProcessMitigationsTest*PreferSys32* failures.

Tests started failing at some point.  We also didn't have Win10 14393 coverage on the waterfall.

This CL:
- Enforces hijack_shim_dll export name.
- Ensures file access to all three test-related hijack DLL paths (via TestRunner).
- Adds a new baseline test for the implicit link, import resolution.

R=wfh@chromium.org
TEST=sbox_integration_tests.exe, ProcessMitigationsTest.CheckWin10ImageLoadPreferSys32_*
BUG=751809

Cq-Include-Trybots: master.tryserver.chromium.win:win10_chromium_x64_rel_ng
Change-Id: Idbd43954c432ab26e54ee7ceadd78f3176dae2e3
Reviewed-on: https://chromium-review.googlesource.com/611310
Commit-Queue: Penny MacNeil <pennymac@chromium.org>
Reviewed-by: Will Harris <wfh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#495308}
[modify] https://crrev.com/99d57e43708d07a32abedc4e954adae6af2d2a5c/sandbox/win/BUILD.gn
[modify] https://crrev.com/99d57e43708d07a32abedc4e954adae6af2d2a5c/sandbox/win/src/process_mitigations_imageload_unittest.cc
[delete] https://crrev.com/1c8f7c51b843419bf3e80e45604be43fa98db118/sandbox/win/tests/integration_tests/hijack_dlls.h
[modify] https://crrev.com/99d57e43708d07a32abedc4e954adae6af2d2a5c/sandbox/win/tests/integration_tests/hijack_shim_dll.cc
[add] https://crrev.com/99d57e43708d07a32abedc4e954adae6af2d2a5c/sandbox/win/tests/integration_tests/hijack_shim_dll.def
[add] https://crrev.com/99d57e43708d07a32abedc4e954adae6af2d2a5c/sandbox/win/tests/integration_tests/hijack_shim_dll.h

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/29cf09bdc9d46841b5e4239148160bc5dd4c2510

commit 29cf09bdc9d46841b5e4239148160bc5dd4c2510
Author: Liam Murphy <liamjm@chromium.org>
Date: Fri Aug 18 23:24:32 2017

CRSSS lockdown. Support different heap structures for Win32.

This allows CSRSS lockdown to be supported on x86.
Win32 on x64 (Wow64) is not supported. This has significantly
different heaps and ALPC Ports, so will need further investigation.
Any changes to support this will be in a fresh CL.

Bug: 751809
Cq-Include-Trybots: master.tryserver.chromium.win:win10_chromium_x64_rel_ng
Change-Id: I3e850135350ebd4d72d32505bcbcde66c1a17465
Reviewed-on: https://chromium-review.googlesource.com/621784
Reviewed-by: Will Harris <wfh@chromium.org>
Commit-Queue: Liam Murphy <liamjm@chromium.org>
Cr-Commit-Position: refs/heads/master@{#495741}
[modify] https://crrev.com/29cf09bdc9d46841b5e4239148160bc5dd4c2510/sandbox/win/src/heap_helper.cc
[modify] https://crrev.com/29cf09bdc9d46841b5e4239148160bc5dd4c2510/sandbox/win/src/lpc_policy_test.cc

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/2ae19c2e002babbadb58e3d02cf22bb77ea810d8

commit 2ae19c2e002babbadb58e3d02cf22bb77ea810d8
Author: Penny MacNeil <pennymac@chromium.org>
Date: Thu Aug 31 20:55:37 2017

[Windows Sandbox Tests] Temporarily disable CFG check test.

Currently failing. Will move to checkbins.py once it is running on CQ.
(ref crbug/761066).

TBR=wfh
BUG=751809

Cq-Include-Trybots: master.tryserver.chromium.win:win10_chromium_x64_rel_ng
Change-Id: I686e26d16ddd09b286d2bb46c4a0f8fb9fa6ec85
Reviewed-on: https://chromium-review.googlesource.com/646530
Reviewed-by: Penny MacNeil <pennymac@chromium.org>
Commit-Queue: Penny MacNeil <pennymac@chromium.org>
Cr-Commit-Position: refs/heads/master@{#498993}
[modify] https://crrev.com/2ae19c2e002babbadb58e3d02cf22bb77ea810d8/sandbox/win/tests/integration_tests/cfi_unittest.cc

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/876fdbbf2b7f57bcfccceac3b0b7d4e817f56c7e

commit 876fdbbf2b7f57bcfccceac3b0b7d4e817f56c7e
Author: Liam Murphy <liamjm@chromium.org>
Date: Sat Sep 02 17:12:06 2017

Don't use dangerous HeapLock(), tests failing on GCE Win10 Bot.

Use safer HeapQueryInformation().

Eg failure:
https://build.chromium.org/p/chromium.fyi/builders/Chromium%20Win%2010%20GCE%20Tests/builds/374


Bug: 751809
Cq-Include-Trybots: master.tryserver.chromium.win:win10_chromium_x64_rel_ng
Change-Id: I2817d5ebe00cdf3dba8a091c9eb5e8b7d0a53cb5
Reviewed-on: https://chromium-review.googlesource.com/648190
Commit-Queue: Will Harris <wfh@chromium.org>
Reviewed-by: Will Harris <wfh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#499398}
[modify] https://crrev.com/876fdbbf2b7f57bcfccceac3b0b7d4e817f56c7e/sandbox/win/src/lpc_policy_test.cc

This issue has been marked as started, but has no owner. Making available.