The problem now is that ios-simulator.json is where we have tests running on main and the CQ. This is currently only unit tests, where it has ios_enable_code_signing=false as a gn arg. This doesn't work for EarlGrey tests which loads OCHamcrest as a dynamic framework, and needs the framework to be signed.

Is there a way to enable code signing for specific targets?
Do we not want to enable code signing for unit tests?
Do we need a separate master for EarlGrey tests on main/CQ?

Assigning to liaoyuke@ to follow up, as I'll be OOO after today.

Saving sdefresne@'s email response FTR:

Since Xcode 8 all simulator have to be code signed (it can be an adhoc signature). This does not requires a mobile provisioning profile (since there is no device). Thus you must not set ios_enable_code_signing=false unless you plan to manually code sign later (for example when you build official build and plan to code sign them with kokoro).

See the following CLs for details:
https://codereview.chromium.org/2397433002
https://codereview.chromium.org/2231723002
https://codereview.chromium.org/2221773002

TL;DR: do not use ios_enable_code_signing=false except for official builds.

Thanks! Looks like we should enable code signing on simulators. 

baxley@ has already enabled it in this CL: https://chromium-review.googlesource.com/c/530161/, which passed try jobs. Given that he will be OOO for more than a week, I'll just send the CL to CQ for him.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/502d45ed84592a1f2a4ea40f0dd06edc370800e3

commit 502d45ed84592a1f2a4ea40f0dd06edc370800e3
Author: Mike Baxley <baxley@chromium.org>
Date: Thu Jun 15 18:24:12 2017

Add ios_chrome_integration_egtests to main waterfall and CQ.

This test has been reliable on FYI (1 failure out of 400 runs). Only
adding for one configuration. Once that proves reliable, will add
some more configurations and maybe suites.

Bug:  638141 
Change-Id: Iba69cd98f0bcb96318e814e992d8b32d4f97f179
Reviewed-on: https://chromium-review.googlesource.com/530161
Reviewed-by: smut <smut@google.com>
Reviewed-by: Sylvain Defresne <sdefresne@chromium.org>
Reviewed-by: Rohit Rao <rohitrao@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Reviewed-by: Sergey Berezin <sergeyberezin@chromium.org>
Commit-Queue: Yuke Liao <liaoyuke@chromium.org>
Cr-Commit-Position: refs/heads/master@{#479760}
[modify] https://crrev.com/502d45ed84592a1f2a4ea40f0dd06edc370800e3/ios/build/bots/chromium.fyi/ios-simulator.json
[modify] https://crrev.com/502d45ed84592a1f2a4ea40f0dd06edc370800e3/ios/build/bots/chromium.mac/ios-simulator.json
[add] https://crrev.com/502d45ed84592a1f2a4ea40f0dd06edc370800e3/ios/build/bots/tests/eg_tests.json

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/4c626c53509997a7b65151b1c9ed2d4b158ab766

commit 4c626c53509997a7b65151b1c9ed2d4b158ab766
Author: Mike Baxley <baxley@chromium.org>
Date: Thu Jun 29 08:28:14 2017

Reduce configurations of iOS unit tests that don't depend on UI scaling.

Remove the following unit test suites from the iOS CQ for iPhone 6s+,
iPhone 5, and iPad Air 2:
boringssl_crypto_tests
boringssl_ssl_tests
crypto_unittests
google_apis_unittests
ios_net_unittests
net_unittests
services_unittests
sql_unittests
url_unittests

These tests still run on iOS 10 iPhone 6s, iOS 10 iPad Retina,
iOS 9 iPhone 5, and iOS 9 iPad Air 2.

Bug:  638141 
Change-Id: I131fc1c755a8cda558b222ee9bc5026fce60574b
Reviewed-on: https://chromium-review.googlesource.com/552154
Commit-Queue: Mike Baxley <baxley@chromium.org>
Reviewed-by: Eugene But <eugenebut@chromium.org>
Cr-Commit-Position: refs/heads/master@{#483316}
[modify] https://crrev.com/4c626c53509997a7b65151b1c9ed2d4b158ab766/ios/build/bots/chromium.mac/ios-simulator.json
[modify] https://crrev.com/4c626c53509997a7b65151b1c9ed2d4b158ab766/ios/build/bots/tests/common_tests.json
[add] https://crrev.com/4c626c53509997a7b65151b1c9ed2d4b158ab766/ios/build/bots/tests/screen_size_dependent_tests.json

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/67cac625fdc89728d1706ac45a0fda58e99333e6

commit 67cac625fdc89728d1706ac45a0fda58e99333e6
Author: Mike Baxley <baxley@chromium.org>
Date: Tue Jul 11 05:42:28 2017

Add web shell test suite to the CQ and main waterfall.

This test has been stable on FYI, and having it on the CQ will
help catch regressions sooner.

Bug:  638141 
Change-Id: Ie318f62993402a0f1b89f4aff47c821a0310722f
Reviewed-on: https://chromium-review.googlesource.com/557861
Reviewed-by: Eugene But <eugenebut@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Mike Baxley <baxley@chromium.org>
Cr-Commit-Position: refs/heads/master@{#485531}
[modify] https://crrev.com/67cac625fdc89728d1706ac45a0fda58e99333e6/ios/build/bots/tests/eg_tests.json