New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 682287 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Last visit > 30 days ago
Closed: Mar 2017
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug

Blocked on:
issue 674583

Blocking:
issue 673253



Sign in to add a comment

Building EarlGrey tests for Chrome on iOS in Release

Project Member Reported by sdefresne@chromium.org, Jan 18 2017

Issue description

When building in release (and by extension official) all assertions code need to be removed. This works for Chrome assertions (i.e. DCHECK) as they are compiled to no-op if NDEBUG is defined, but require defining NS_BLOCK_ASSERTIONS to disable GTMDevAssert [1].

The define was added with https://codereview.chromium.org/2637853002 which broke the downstream bots [2] as we build some of our tests in release [3]. The CL was reverted to fix the tree, but we need to be able to disable GTMDevAssert in release otherwise an assert failure will cause an application crash [4].

The issue is that EarlGrey fails compilation with an error if NS_BLOCK_ASSERTIONS is defined [5].

Our options at this point are:
1. disable all EarlGrey tests when building in release, define NS_BLOCK_ASSERTIONS in release & official,
2. disable all EarlGrey tests when building in official, only define NS_BLOCK_ASSERTIONS in official,
3. change EarlGrey to build when NS_BLOCK_ASSERTIONS is defined,
4. change google_toolbox_for_mac to add another way to disable GTMDevAsserts.

If we go with option 1., we'll have to change our device bot to build in debug (this may be a good thing or not, I don't know why they build in release, it is possible this is because debug is too slow to tests on some devices).

If we go with option 2., we'll have to change the official bots to not run the EarlGrey tests and change BUILD.gn to remove dependency on EarlGrey tests in official build (this will make the build slightly faster but would slightly reduce confidence of the official build, though I'm not sure anyone check the result of those tests).

Going with option 3. or option 4. would require getting the corresponding team involved. I don't know whether option 3. is technically feasible (it is possible that EarlGrey do require assertions if it uses exceptions in its code path, but then ARC is not exception safe, so it means that our EarlGrey tests may misbehave in case of errors). Option 4. is IMO worse than option 3. if it is feasible as option 4. mean that other assertion disabled by NS_BLOCK_ASSERTIONS will still be present in official builds (in particular NSAssert).

Currently, I think the best option is 3. and thus should be evaluated first. Mike, can you get in touch with the EarlGrey team and see whether it is feasible for them to support building with NS_BLOCK_ASSERTIONS defined?

[1]: see third_party/google_toolbox_for_mac/src/GTMDefines.h, lines 127-140.

[2]: this is how device builder downstream are configured -- this may be a mistake, I'm unsure about those ones -- and also how the official builder are configured -- this is definitely not a mistake.

[3]: https://luci-logdog.appspot.com/v/?s=chrome%2Fbb%2Finternal.bling.main%2Fdevice-builder%2F11896%2F%2B%2Frecipes%2Fsteps%2Fcompile%2F0%2Fstdout

[4]: see http://crbug.com/645222.

[5]: see ios/third_party/earl_grey/src/EarlGrey.pch, lines 26-28.
 

Comment 1 by baxley@chromium.org, Jan 18 2017

Blockedon: 674583
Thank you for the detailed bug.

The latest version of EG doesn't have NS_BLOCK_ASSERTIONS in EarlGrey.pch, so once the roll is complete, this may be fixed. There's a long press problem in WKWebView, where I have a workaround and am discussing with EarlGrey the best way to do it.

Once the roll is complete, if the problem isn't fixed then I'll look at more of the options.

Comment 2 by baxley@chromium.org, Mar 23 2017

NS_BLOCK_ASSERTIONS now work with EarlGrey in our tests.

Comment 3 by baxley@chromium.org, Mar 23 2017

Status: Fixed (was: Assigned)

Sign in to add a comment