2016-03-24T11:56:59.642791+00:00 INFO shill[2454]: [INFO:supplicant_eap_state_handler.cc(77)] EAP: Authentication starting.
2016-03-24T11:56:59.671308+00:00 NOTICE wpa_supplicant[2219]: eth0: CTRL-EVENT-EAP-PROPOSED-METHOD vendor=0 method=13 -> NAK

This means that the RADIUS server is proposing method 13 (EAP-TLS) and the Chromebox is refusing this, presumably because although EAP credentials are supplied for Ethernet, these credentials aren't for EAP-TLS (certificate), but some other authentication method.

We actually use PEAP on our network and the credentials saved in the chrome profile is for peap/mschapv2 but that should not change the eap identity request sent from the meraki switch in any way. On the network, we are seeing no response from the chromebox to the meraki challenge. The authentication simply times out and the switch sets the port to the unauth vlan.

From the Chrome device's perspective, it has credentials for some non-certificate authentication (no reason to believe it's not PEAP/MSCHAPv2):

2016-03-24T11:56:54.648415+00:00 DEBUG shill[2454]: [VERBOSE2:eap_credentials.cc(273)] (eap_credentials) Connectable. !EAP-TLS and has a password.

but it's seeing a RADIUS request for EAP-TLS:

2016-03-24T11:56:59.671308+00:00 NOTICE wpa_supplicant[2219]: eth0: CTRL-EVENT-EAP-PROPOSED-METHOD vendor=0 method=13 -> NAK

Feel free to share more detailed logging, which you can do by opening a crosh shell (Ctrl-Alt-T) and entering "wpa_debug excessive" in order to get a more detailed result from wpa_supplicant as to what it is sending and receiving.

Sorry I don't know how to get that stuff. That was from the tech in charge. I do have the network pcaps. Perhaps that behavior can be explained by the slight difference in the EAP request from the HP and the Meraki switches. The HP EAP request has "User name:" in the request but the Meraki does not. Since the chromebox does not see User Name, it assumes cert auth instead of user/pass auth.

The procedure (crosh/wpa_debug) should be performed on the chromebox.  The "cert auth" decision is not being made by the chromebox or the network switch.  It's being performed by the remote authentication RADIUS server that the switch is configured to relay EAP to.

In our environment, the chromebox never responds to the EAP request that is sent by the Meraki switch (as seen in the pcaps) so no request is ever received on the radius server. A windows pc plugged into the same port will respond and authenticate fine. I will try to figure out how to get the wpa debugs so I can capture it next time I am onsite.

Sorry the directions you gave were not very clear. I ran wpa_debug debug in crosh and then enabled 802.1x on the meraki switch port and plugged in the chromebox. Nothing was output on the console. Here is the net.log. Do you require any other files from the debug capture? Nothing changed from the switch perspective. The pcap shows the switch sending an eap request but no response from the chromebox.

Deleted: net.log 267 KB

net.log 267 KB View Download

For the entirety of net.log there are no messages of the form "wpa_supplicant[xxx]: eth0:" in the logs as they are in the logs shared by djeche@ in the original report, so it appears the issue originally described isn't happening anymore or you have run into a different issue.  Do you have any idea what happened there that would be different?

I don't know how David got that. Those are not what I provided in my ticket. I have tried with multiple switches in the meraki 220 family and the results are the same. The chromeboxes do not respond to the eap challenge. Plug a Windows machine into the same port and everything works as expected. No combination of settings on the meraki switches makes a difference. For some reason the chromeboxes do not recognize that they are being challenged for credentials. The only differences in the eap requests from the hp and meraki is the hp has 8021x version 2001 vs meraki's version 2004 and the hp has identity: user name while the meraki identity parameter is empty. 

The logs shown on the initial post are from my test environment. I was testing various things in order to try better understand the issue in question.

I was under the assumption that the device might not be reacting to EAP packet's simply because it was not sync the data appropriately from the admin panel. I have replicated the customer as best I can. 

[16:23:18] NetworkPropertiesUpdated: Ethernet EAP Parameters (/service/0)
[16:23:18] InitialPropertiesReceived: /service/0: Ethernet EAP Parameters State: idle Visible: 0
[16:23:18] RequestUpdate: /service/0
[16:23:18] NetworkPropertyUpdated: Ethernet EAP Parameters.ProxyConfig = "{\"mode\":\"direct\"}"
[16:23:18] NetworkPropertyUpdated: Ethernet EAP Parameters.GUID = "{e9244e97-62e1-4acf-874f-fdc1f79fae75}"
[16:23:18] RequestUpdate: /service/0
[16:23:18] NetworkPropertyUpdated: Ethernet EAP Parameters.Profile = "/profile/chronos/shill"
[16:23:18] RequestScan
[16:23:18] Configure: etherneteap.UIData="{\"onc_source\":\"user_policy\"}"
[16:23:18] Configure: etherneteap.Type="etherneteap"
[16:23:18] Configure: etherneteap.SaveCredentials=true
[16:23:18] Configure: etherneteap.ProxyConfig="{\"mode\":\"direct\"}"
[16:23:18] Configure: etherneteap.Profile="/profile/chronos/shill"
[16:23:18] Configure: etherneteap.GUID="{e9244e97-62e1-4acf-874f-fdc1f79fae75}"
[16:23:18] Configure: etherneteap.EAP.UseSystemCAs=false
[16:23:18] Configure: etherneteap.EAP.Password=******
[16:23:18] Configure: etherneteap.EAP.Identity="djechetesting"
[16:23:18] Configure: etherneteap.EAP.EAP="PEAP"

I can see from this that the policy and credential's are syncing properly. After this configuration has taken place. There is no more mention EAP with-in the log information. I have tried observing various chrome urls for extra info. 

It is like the chromebox is not seeing the EAP packet's on the network. This could be the explanation to why the device is not responding and why the is no log information within the logs provided in the link. 

I will provide more time accurate information and log's once I have done further test's

Thanks David. Here are the eap requests from each switch for posterity's sake. Everything from the chrome logs, network packet captures, and nps logs point to the fact that the chromebox does not recognize it is receiving an eap challenge.

Deleted: merakiEapPacket.pcap 100 bytes

merakiEapPacket.pcap 100 bytes Download

Deleted: HPEapPacket.pcap 100 bytes

HPEapPacket.pcap 100 bytes Download

Summarizing:

The HP pcap:

Ethernet II, Src: Hewlett-_06:02:00 (00:19:bb:06:02:00), Dst: Nearest (01:80:c2:00:00:03)
    Destination: Nearest (01:80:c2:00:00:03)
        Address: Nearest (01:80:c2:00:00:03)
        .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
        .... ...1 .... .... .... .... = IG bit: Group address (multicast/broadcast)
    Source: Hewlett-_06:02:00 (00:19:bb:06:02:00)
        Address: Hewlett-_06:02:00 (00:19:bb:06:02:00)
        .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
        .... ...0 .... .... .... .... = IG bit: Individual address (unicast)
    Type: 802.1X Authentication (0x888e)
    Padding: 000000000000000000000000000000000000000000000000...
802.1X Authentication
    Version: 802.1X-2001 (1)
    Type: EAP Packet (0)
    Length: 15
    Extensible Authentication Protocol
        Code: Request (1)
        Id: 1
        Length: 15
        Type: Identity (1)
        Identity: User name:

0000  01 80 c2 00 00 03 00 19 bb 06 02 00 88 8e 01 00   ................
0010  00 0f 01 01 00 0f 01 55 73 65 72 20 6e 61 6d 65   .......User name
0020  3a 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   :...............
0030  00 00 00 00 00 00 00 00 00 00 00 00               ............

The Meraki EAP packet:

Ethernet II, Src: Meraki_ce:29:b6 (88:15:44:ce:29:b6), Dst: Nearest (01:80:c2:00:00:03)
    Destination: Nearest (01:80:c2:00:00:03)
        Address: Nearest (01:80:c2:00:00:03)
        .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
        .... ...1 .... .... .... .... = IG bit: Group address (multicast/broadcast)
    Source: Meraki_ce:29:b6 (88:15:44:ce:29:b6)
        Address: Meraki_ce:29:b6 (88:15:44:ce:29:b6)
        .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
        .... ...0 .... .... .... .... = IG bit: Individual address (unicast)
    Type: 802.1X Authentication (0x888e)
    Padding: 000000000000000000000000000000000000000000000000...
802.1X Authentication
    Version: 802.1X-2004 (2)
    Type: EAP Packet (0)
    Length: 5
    Extensible Authentication Protocol
        Code: Request (1)
        Id: 15
        Length: 5
        Type: Identity (1)
        Identity: 

0000  01 80 c2 00 00 03 88 15 44 ce 29 b6 88 8e 02 00   ........D.).....
0010  00 05 01 0f 00 05 01 00 00 00 00 00 00 00 00 00   ................
0020  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0030  00 00 00 00 00 00 00 00 00 00 00 00               ............

They are sent to the same destination address, are the same Ethernet type (0x888e), have slightly different version (see below) and are both of type "request".

The code in question is https://android.googlesource.com/platform/system/connectivity/shill/+/master/eap_listener.cc .  Assuming the kernel delivers this packet to userspace in the same way (due to the same destination MAC address and Ethernet type), there are only 3 ways out of the eap listener method that will not trigger authentication.  They each contain a message that will be preceded by "eap_listener.cc" in /var/log/net.log, even if debugging flags are not turned.

The fact that you are not seeing these logs point to the fact that either the hardware isn't passing these frames to the kernel, or the kernel isn't passing them to userspace over to the listening socket.  Passing this over to the Chrome OS PM for further triage and to find someone to further repro and root cause.

Kevin, not sure if this is related to all the other recent 802.1x issues?

The big supplicant change happened on M51.  The customer who reported the regression on wifi indicated that 8053.0.0, the first build after updating supplicant, showed the problem.  8052.0.0, the last build with the old supplicant version, did not.  If the problem described in this bug was seen way back on M48/M49, it is something else.

The issues described in #c3 (attempt to use a password for EAP-TLS certificate auth) and #c7 (no wpa_supplicant logs on the afflicted interface) do not match what we see in  bug 599595 , either.

I am able to successfully connect to our own corp network using EAP-TLS on a wired interface using a link (Pixel 1) on beta channel: 7834.42.0 / 49.0.2623.59.  net.log shows a bunch of CTRL-EVENT messages from supplicant at NOTICE loglevel, starting with "eth0: Associated with..." and ending with CTRL-EVENT-CONNECTED.

For this particular issue it seem's as though the problem may be related to hardware not passing packet's up the stack. When you connected to the corp network was it via ethernet or wireless ?

We are experiencing this issue on chrome boxes using wired network's with EAP-PEAP method. MS-CHAPv2 is used for user authentication. Is it possible to attempt confirm that the this method is not were the problem is. I have attempted to connect via WiFi with out much problem, but when it comes to ethernet that when we experience most issue's.

I am starting to doubt my set up, so I will attempt this once again with a direct ethernet connection when I get the opportunity. 

This only happens on wired. We do use PEAP/MSCHAPV2 for wired and wireless. The CFM does not behave well when put on the guest network. It keeps getting deauthed by the switch. This is about to be a major problem as we are setting up all of our sites with meraki switches. Our chromebox for meetings devices that we pay annual support for are about to become useless to us.

> Ensure that policy has been deployed to device via Chrome://policy

I don't have a way to enroll in this domain, so I configured the ethernet parameters by hand locally; see attached screenshot.

Prior to configuring these parameters, I saw this error when replaying the mera802.1.pcap file:

2016-05-03T12:12:00.472126-07:00 INFO shill[1326]: [INFO:ethernet.cc(415)] EAP Service lacks 802.1X credentials; not doing EAP authentication.

After configuring the network, I see this:

2016-05-03T12:17:04.975379-07:00 DEBUG wpa_supplicant[548]: eth0: Added interface eth0
2016-05-03T12:17:04.975416-07:00 DEBUG wpa_supplicant[548]: eth0: State: INACTIVE -> DISCONNECTED
2016-05-03T12:17:04.979190-07:00 WARNING shill[24410]: [WARNING:eap_credentials.cc(130)] PopulateSupplicantProperties: No certificate authorities are configured. Server certificates will be accepted unconditionally.
[...]
2016-05-03T12:17:04.988198-07:00 INFO shill[24410]: [INFO:ethernet.cc(406)] Supplicant state changed to disconnected
2016-05-03T12:17:04.988291-07:00 INFO shill[24410]: [INFO:ethernet.cc(406)] Supplicant state changed to associated
2016-05-03T12:17:05.659021-07:00 DEBUG wpa_supplicant[548]: l2_packet_receive: src=88:15:44:ce:29:b6 len=46
2016-05-03T12:17:05.659041-07:00 DEBUG wpa_supplicant[548]: eth0: RX EAPOL from 88:15:44:ce:29:b6
2016-05-03T12:17:05.659047-07:00 DEBUG wpa_supplicant[548]: EAPOL: Received EAP-Packet frame
2016-05-03T12:17:05.659052-07:00 DEBUG wpa_supplicant[548]: EAPOL: SUPP_PAE entering state RESTART
2016-05-03T12:17:05.659057-07:00 DEBUG wpa_supplicant[548]: EAP: EAP entering state INITIALIZE
2016-05-03T12:17:05.659062-07:00 DEBUG wpa_supplicant[548]: EAP: EAP entering state IDLE
2016-05-03T12:17:05.659067-07:00 DEBUG wpa_supplicant[548]: EAPOL: SUPP_PAE entering state AUTHENTICATING
2016-05-03T12:17:05.659071-07:00 DEBUG wpa_supplicant[548]: EAPOL: SUPP_BE entering state REQUEST
2016-05-03T12:17:05.659077-07:00 DEBUG wpa_supplicant[548]: EAPOL: getSuppRsp
2016-05-03T12:17:05.659081-07:00 DEBUG wpa_supplicant[548]: EAP: EAP entering state RECEIVED
2016-05-03T12:17:05.659086-07:00 DEBUG wpa_supplicant[548]: EAP: Received EAP-Request id=187 method=1 vendor=0 vendorMethod=0
2016-05-03T12:17:05.659091-07:00 DEBUG wpa_supplicant[548]: EAP: EAP entering state IDENTITY
2016-05-03T12:17:05.659095-07:00 NOTICE wpa_supplicant[548]: eth0: CTRL-EVENT-EAP-STARTED EAP authentication started
[...]
2016-05-03T12:17:05.659354-07:00 INFO shill[24410]: [INFO:ethernet.cc(393)] In EAPEventTask with status started, parameter 
2016-05-03T12:17:05.659362-07:00 INFO shill[24410]: [INFO:supplicant_eap_state_handler.cc(77)] EAP: Authentication starting.

(more context is attached)

This is running a local build of canary channel which roughly corresponds to 52.0.2721.0 / 8272.0.0.  shill and Chrome are built from the ToT.

One possibility is that the newer version of wpa_supplicant accounts for the different result.  Another possibility is that we aren't quite using the same configuration (maybe some part of the ONC configuration is getting lost in translation).

Deleted: log.txt 28.9 KB

log.txt 28.9 KB View Download

Deleted: peap.png 67.8 KB

	peap.png 67.8 KB View Download

Yes we already have our policy set to not validate server certs.

Thanks,
Ben

@cernekee

Do you think that the root of the issue could be related to how we handle self signed cert's ?

Also it could be an ONC configuration problem. Have you tried pulling the config for Admin console you may see different result's.

There's no cert in the mera802.1.pcap file, so if that test case still reproduces the failure I don't think it's related to certs.