Not sure why this was assigned to me, no idea what Sierra is, or why this is a release blocker.

macOS Sierra is the next version of macOS slated for release in September. Someone needs to investigate these test failures to determine their impact - they may or may not be release blockers.

mmenke: You were cc-ed because you typically know about Network-related things. Did you misread a cc as a owner?

Thanks for the context!  I was concerned this was a regression of some sort (recently merged a CL or two to M53, but none of my CLs were anywhere near this layer!)  Happy to help redirect this bug.

I know nothing about this layer, but someone who deals with SSL-related things should have some idea on how to direct this bug.

If you send an email to the people on the CC list about how to test with macOS Sierra, I'm sure we'd be happy to assist.

erikchen: The //net stack goes on a network triage rotation, so if you're ever unsure, simply set Internals>Network and Untriaged and the next triage rotation in https://chromium.googlesource.com/chromium/src/+/master/net/docs/bug-triage.md will help direct appropriately. We do this, in part, to avoid swamping specific people with triage interrupts :)

Also, it would help speed things along if you actually included the failure details from the tests that failed. We try to log sufficient diagnostics to make it easier to understand from the failure log itself the issue, while simply knowing the tests failed doesn't help narrow it down.

Assigning back to Erik for providing more information as requested in #5.

Note: This issue is marked as Beta blocker, M53 is scheduled to be promoted to Beta next week (07/27) please resolve asap.

I've attached more detailed errors.

macOS Sierra can only be installed on non-corp machines. I have one in MTV you're welcome to borrow, although I'm also going to be using it to look into errors from other test suites.

Deleted: net_unittest_sierra_failures.txt 9.6 KB

net_unittest_sierra_failures.txt 9.6 KB View Download

Yeah, none of these should end up being a Release-Block, but we should update the test expectations.

It would look like Sierra is more stringent about rejecting MD5, which is good. These unittests were to make sure MD5 was rejected on platforms it's supported, and it's looking like it's (partially) no longer supported in Sierra.

There are also some changes to expect with SHA-1, but it doesn't *look* like they're at play here. 

Removing RB-B, dropping priority.

We're trying to roll out 10.12 to the infra fleet, so this is now higher priority

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/99c0d7490554f43ed1b14d315a39be5f4c5e3fc0

commit 99c0d7490554f43ed1b14d315a39be5f4c5e3fc0
Author: mattm <mattm@chromium.org>
Date: Tue Dec 06 20:53:21 2016

Update CertVerifyProcTest.LargeKey for Sierra (Mac OS 10.12)

On 10.12 a large key causes a recoverable error instead of a fatal error,
and one of the chain status codes is CSSMERR_TP_NOT_TRUSTED, so the
cert_status is now (CERT_STATUS_AUTHORITY_INVALID | CERT_STATUS_INVALID).
Update the test to just check if CERT_STATUS_INVALID is present in
cert_status so that it will pass on any version of mac.

BUG= 629712 

Review-Url: https://codereview.chromium.org/2550333003
Cr-Commit-Position: refs/heads/master@{#436714}

[modify] https://crrev.com/99c0d7490554f43ed1b14d315a39be5f4c5e3fc0/net/cert/cert_verify_proc_unittest.cc

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/c74f2ca3e10529dc7a17eaa0bbcb0c1f8876029b

commit c74f2ca3e10529dc7a17eaa0bbcb0c1f8876029b
Author: mattm <mattm@chromium.org>
Date: Wed Dec 07 01:33:09 2016

Update CertVerifyProcTest.RejectWeakKeys for Sierra (Mac OS 10.12)

Starting with Mac OS 10.12, weak keys result in a kSecTrustResultRecoverableTrustFailure with a CSSMERR_TP_INVALID_CERTIFICATE result code and CSSMERR_TP_INVALID_CERTIFICATE chain status codes. This results in a CERT_STATUS_INVALID result of VerifyInternal, but the os-independent ExaminePublicKeys function still adds CERT_STATUS_WEAK_KEY.
(Previously they were a kSecTrustResultRecoverableTrustFailure with a CSSMERR_TP_VERIFY_ACTION_FAILED result code and CSSMERR_CSP_UNSUPPORTED_KEY_SIZE chain status codes, so VerifyInternal could directly map that to CERT_STATUS_WEAK_KEY.)

BUG= 629712 

Review-Url: https://codereview.chromium.org/2556953002
Cr-Commit-Position: refs/heads/master@{#436823}

[modify] https://crrev.com/c74f2ca3e10529dc7a17eaa0bbcb0c1f8876029b/net/cert/cert_verify_proc_unittest.cc

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/cacd8f757ae49b69e074c0b538c9660ab78a0c83

commit cacd8f757ae49b69e074c0b538c9660ab78a0c83
Author: mattm <mattm@chromium.org>
Date: Tue Dec 13 02:59:18 2016

Update CertVerifyProcWeakDigestTest for Sierra (Mac OS 10.12)

Starting with Mac OS 10.12, path building stops at the first weak digest (md2 or md5).

BUG= 629712 

Review-Url: https://codereview.chromium.org/2558983002
Cr-Commit-Position: refs/heads/master@{#438007}

[modify] https://crrev.com/cacd8f757ae49b69e074c0b538c9660ab78a0c83/net/cert/cert_verify_proc_unittest.cc

Bugdroid seems to have missed one of the CLs tagged with this bug number: https://codereview.chromium.org/2559623002 (Skip CertVerifyProcTest.MacCRLIntermediate on Sierra)

net_unittests is now passing on Sierra for me.