New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 621553 link

Starred by 5 users

Issue metadata

Status: Archived
Owner:
Last visit > 30 days ago
Closed: Dec 13
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 3
Type: Bug


Show other hotlists

Hotlists containing this issue:
Chrome-Bug-Cleanup


Sign in to add a comment

TLS: Certificate verification failed, error 19 when roaming between AP's

Reported by yyefet@chromium.org, Jun 20 2016

Issue description

Version: 51.0.2704.79
OS: ChromeOS

Customer Environment:
ISE version 1.3.0.876
AP Models: Cisco 1142, 3502, 2602, 2702. 
2 SSID's with the same name.

*May be related to crbug 613417*

Issue Description:
Enterprise customer is using 802.1x - PEAP-MSCHAPv2 to connect their Chromebooks to the network.  


Two wifi profiles have been setup:
1) Initially, the devices were 'shared' so a device level certificate was installed and set to auto connect as the first wifi profile.
2)  The second wifi profile was pushed from the admin console on the user level (auto connect set).

*Each wifi profile connects just fine when chosen manually.
The certificates get validated and authentication completes without an issue whether you choose the user wifi profile, or the device profile individually.

Prior to 51 upgrade, devices were able to successfully roam between profiles/AP's in the network. (e.g.: moving from one AP with profile 1 (user wifi profile/cert) would successfully switch to profile 2 (device wifi profile/cert) when roaming to AP 2)

Customer states that around April 8th, their users have been reporting connectivity issues, specifically in areas between AP's (where the wifi profiles should handoff).


Cisco ISE Errors:
12321 PEAP failed SSL/TLS handshake because the client rejected the ISE local-certificate


ChromeOS (debug log) Errors:
2016-05-30T09:06:12.970897+02:00 WARNING wpa_supplicant[355]: TLS: Certificate verification failed, error 19 (self signed certificate in certificate chain) depth 3 for '[REMOVED PII]' 
2016-05-30T09:06:12.970926+02:00 NOTICE wpa_supplicant[355]: wlan0: CTRL-EVENT-EAP-TLS-CERT-ERROR reason=1 depth=3 subject='[REMOVED PII]' err='self signed certificate in certificate chain' 
2016-05-30T09:06:12.970938+02:00 DEBUG wpa_supplicant[355]: EAP: Status notification: remote certificate verification (param=self signed certificate in certificate chain) 
2016-05-30T09:06:12.971353+02:00 NOTICE wpa_supplicant[355]: OpenSSL: openssl_handshake - SSL_connect error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed 
2016-05-30T09:06:12.988042+02:00 ERR shill[1182]: [ERROR:supplicant_eap_state_handler.cc(64)] EAP: Unexpected remote certificate verification parameter: self signed certificate in certificate chain 

which ultimately results in: 

2016-05-30T09:06:12.979250+02:00 DEBUG wpa_supplicant[355]: EAPOL authentication completed - result=FAILURE 
 

The question here is why does the certificate verification fail during roaming but succeeds during manual selection of the same wifi profile/cert.


What steps will reproduce the problem?
(1) setup 2 SSID's, and 2 wifi profiles (PEAP-MSCHAPv2) - 1 device level cert, 1 user cert.
(2) roam between AP's
(3) receive failure.



What is the expected output?
Roaming between AP's/wifi profiles should be smooth.

What do you see instead?
Failed certificate validation


PII Protected Device Logs:
https://drive.google.com/a/google.com/folderview?id=0B6fESMmJITTNYUhEMnhPYUxkX1U&usp=sharing


 
Cc: cernekee@chromium.org snanda@chromium.org
+Kevin +Sameer

Not sure if this is caused by any M51 changes?
Owner: cernekee@chromium.org
Status: Assigned (was: Untriaged)
The failed case, which uses the :78:8f AP, is:

2016-05-30T09:06:10.004046+02:00 INFO shill[1182]: [INFO:service.cc(411)] Service 4: state Idle -> Associating

2016-05-30T09:06:12.969104+02:00 DEBUG wpa_supplicant[355]: SSL: Received packet(len=715) - Flags 0x01
2016-05-30T09:06:12.969116+02:00 DEBUG wpa_supplicant[355]: OpenSSL: RX ver=0x0 content_type=256
2016-05-30T09:06:12.969126+02:00 DEBUG wpa_supplicant[355]: OpenSSL: RX ver=0x301 content_type=22
2016-05-30T09:06:12.969158+02:00 DEBUG wpa_supplicant[355]: SSL: (where=0x1001 ret=0x1)
2016-05-30T09:06:12.969173+02:00 DEBUG wpa_supplicant[355]: SSL: SSL_connect:SSLv3 read server hello A
2016-05-30T09:06:12.969184+02:00 DEBUG wpa_supplicant[355]: OpenSSL: RX ver=0x0 content_type=256
2016-05-30T09:06:12.969239+02:00 DEBUG wpa_supplicant[355]: OpenSSL: RX ver=0x301 content_type=22
2016-05-30T09:06:12.970897+02:00 WARNING wpa_supplicant[355]: TLS: Certificate verification failed, error 19 (self signed certificate in certificate chain) depth 3 for '/C=SE/O=AddTrust AB/OU=AddTrust External TTP Network/CN=AddTrust External CA Root'

The successful case, which uses the :78:80 AP, is:

2016-05-30T09:07:11.759134+02:00 INFO shill[1182]: [INFO:service.cc(411)] Service 4: state Idle -> Associating

2016-05-30T09:07:11.849727+02:00 DEBUG wpa_supplicant[355]: SSL: Received packet(len=715) - Flags 0x01
2016-05-30T09:07:11.849737+02:00 DEBUG wpa_supplicant[355]: OpenSSL: RX ver=0x0 content_type=256
2016-05-30T09:07:11.849748+02:00 DEBUG wpa_supplicant[355]: OpenSSL: RX ver=0x301 content_type=22
2016-05-30T09:07:11.849758+02:00 DEBUG wpa_supplicant[355]: SSL: (where=0x1001 ret=0x1)
2016-05-30T09:07:11.854015+02:00 DEBUG wpa_supplicant[355]: SSL: SSL_connect:SSLv3 read server hello A
2016-05-30T09:07:11.854049+02:00 DEBUG wpa_supplicant[355]: OpenSSL: RX ver=0x0 content_type=256
2016-05-30T09:07:11.854061+02:00 DEBUG wpa_supplicant[355]: OpenSSL: RX ver=0x301 content_type=22
2016-05-30T09:07:11.854072+02:00 DEBUG wpa_supplicant[355]: TLS: tls_verify_cb - preverify_ok=1 err=0 (ok) ca_cert_verify=1 depth=3 buf='/C=SE/O=AddTrust AB/OU=AddTrust External TTP Network/CN=AddTrust External CA Root'

In the middle I see:

2016-05-30T09:06:49.573766+02:00 INFO shill[1182]: [INFO:manager.cc(488)] (Re-)configured service 4 from new profile.

Did this coincide with a login event?  In var/log/ui/ui.20160530-090610 I see:

[8605:8713:0530/090650:ERROR:onc_certificate_importer_impl.cc(249)] Error ( net::ERR_IMPORT_CA_CERT_NOT_CA ) importing certificate of type Authority
[8605:8713:0530/090650:ERROR:onc_certificate_importer_impl.cc(111)] Cannot parse certificate at index 0
[8605:8713:0530/090650:ERROR:connection_factory_impl.cc(367)] Failed to connect to MCS endpoint with error -21
[8605:8605:0530/090650:ERROR:device_event_log_impl.cc(141)] [09:06:50.382] Network: onc_certificate_importer_impl.cc:125 ONC Certificate Import Error

And in var/log/messages:

var/log/messages:148:2016-05-30T09:06:49.519211+02:00 WARNING session_manager[8582]: [WARNING:device_policy_service.cc(400)] Could not verify that owner key belongs to this user.
var/log/messages:151:2016-05-30T09:06:49.579592+02:00 INFO session_manager[8582]: [INFO:session_manager_impl.cc(344)] Starting user session
var/log/messages:157:2016-05-30T09:06:49.981585+02:00 INFO session_manager[8582]: [INFO:policy_service.cc(188)] Persisted policy to disk.


In the chrome://policy dump there are two different configurations that cover this SSID:

The device configuration (GUID 42d668b8-0307-4a08-9afd-1e73923d82ba) does not specify ServerCARefs and it sets UseSystemCAs to false.  This means that wpa_supplicant will expect a self-signed cert, per the ONC spec.

The user configuration (GUID c4c66491-6b03-4424-9c94-84487e5e9a66) specifies a ServerCARef and sets UseSystemCAs to true.  This means that supplicant will accept a cert that has a chain of trust to either a system CA or to the CA certificate.  It will reject certs with an unknown chain of trust.

Both of these configurations use PEAP (user/pass authentication with no client cert).  Both have AutoConnect enabled.

> Version: 51.0.2704.79

chrome_version.pdf says it is using 50.0.2661.103.  Was the problem seen with both versions?


Some things to check:

1) Try to figure out what the ONC Certificate Import Error is complaining about, and whether it is a regression.  Check file:///var/log/ui/ui.LATEST after logging in to see if it is reproducible.

2) If you delete the device policy entirely, and allow the Chromebook to connect with the user policy (with CA checking enabled), does this work at all?  And if so, are there still problems involving roaming?

3) What about the inverse?

4) Is there documentation that says it is OK to define the same network in device+user profiles using different parameters?  Or is this relying on undocumented behavior?

5) If this is an explicitly supported feature, where is the code that enforces the precedence order?

6) Can you verify that the user policy is able to connect to both the :78:80 and :78:8f APs, and that the difference cannot be accounted for by e.g. different TLS configurations on each AP?


The PushProfile message in the middle leads me to wonder whether the user configuration is rejecting the cert but the device configuration is accepting it.  But that seems inconsistent with the theory that a user login happened at 09:06:49 and that caused a device->user profile change for /service/4.

Comment 3 by yyefet@chromium.org, Jul 14 2016

Quick update:

I reached out to the customer to gather the additional information/tests.
Hello!
This bug is receiving this notice because there has been no acknowledgment of its existence in quite a bit of time
- If you are currently working on this bug, please provide an update.
- If you are currently affected by this bug, please update with your current symptoms and relevant logs.

If there has been no updates provided by EOD Wednesday, 12/12/18 (5pm EST), this bug will be archived and can be re-opened at any time deemed necessary.

Thank you!
Status: Archived (was: Assigned)
Due to lack of action this bug has been Archived. If work is still being done on this issue or you are still experiencing this issue please feel free to re-open with the appropriate information.

Sign in to add a comment