Haswell (v3.8 kernel): USB NIC enumeration failing with "device not accepting address [0-9]*, error -71" |
||||||||||||||
Issue description
I'm trolling through provisioning and repairs jobs for USB ethernet failures. Clapper seems to fail more than any other x86 platform with this particular error sequence:
2017-11-21T18:26:46.855315-08:00 ERR kernel: [ 4.609794] usb 1-2.4: device not accepting address 10, error -71
2017-11-21T18:26:46.855318-08:00 ERR kernel: [ 5.437241] usb 1-2-port4: Cannot enable. Maybe the USB cable is bad?
2017-11-21T18:26:46.855320-08:00 ERR kernel: [ 5.438825] usb 1-2-port4: unable to enumerate USB device
John, I've assigned to you so you can tell me what is actually connected to all the clapper machines at this port. If it's a Servo V4, please reassign to Nick.
(There are other errors causing USB ethernet on this platform and other platforms - just filing this since it seems significant).
Gory details below.
find 2017-11* -name \*message\* | xargs grep -l -m1 "unable to enumerate USB device" | xargs fgrep -m 1 -e DMI: -e "0.000000] Machine" -e "0.000000] machine" | cut -d : -f 7 | sort | uniq -c
17 Acer Peppy, BIOS Google_Peppy.4389.117.0 03/02/2017
3 Dell Wolf, BIOS Google_Wolf.4389.24.62 07/30/2015
4 Google Asuka/Asuka, BIOS Google_Asuka.7820.314.0 06/08/2017
5 GOOGLE Auron_Paine, BIOS Google_Auron_paine.6301.58.98 01/06/2017
20 GOOGLE Banjo, BIOS Google_Banjo.5216.334.4 04/09/2015
1 GOOGLE Banon, BIOS Google_Banon.7287.313.0 11/29/2016
10 GOOGLE Banon, BIOS Google_Banon.7287.373.0 08/13/2017
13 GOOGLE Candy, BIOS Google_Candy.5216.310.57 03/28/2016
7 Google Caroline/Caroline, BIOS Google_Caroline.7820.329.0 08/18/2017
2 Google Cave/Cave, BIOS Google_Cave.7820.329.0 08/18/2017
105 GOOGLE Clapper, BIOS Google_Clapper.5216.199.7 08/22/2014
1 Google Coral/Coral, BIOS Google_Coral.9984.0.0 09/28/2017
1 GOOGLE Cyan, BIOS Google_Cyan.7287.57.137 08/20/2017
8 GOOGLE Enguarde, BIOS Google_Enguarde.5216.201.10 12/06/2014
3 GOOGLE Expresso, BIOS Google_Expresso.5216.223.3 08/20/2014
8 GOOGLE Gnawty, BIOS Google_Gnawty.5216.239.109 01/15/2017
27 GOOGLE Gnawty, BIOS Google_Gnawty.5216.239.123 04/23/2017
1 GOOGLE Guado, BIOS Google_Guado.6301.108.4 04/01/2015
13 GOOGLE Heli, BIOS Google_Heli.5216.392.4 08/19/2015
1 Google Jaq
1 GOOGLE Link, BIOS Google_Link.2695.1.169 11/18/2015
2 Google Pyro/Pyro, BIOS Google_Pyro.9042.110.0 05/16/2017
2 Google Pyro/Pyro, BIOS Google_Pyro.9042.87.1 04/28/2017
7 GOOGLE Quawks, BIOS Google_Quawks.5216.204.61 03/03/2016
3 Google Reef/Reef, BIOS Google_Reef.9042.110.0 05/16/2017
2 GOOGLE Relm, BIOS Google_Relm.7287.341.0 03/18/2017
3 GOOGLE Relm, BIOS Google_Relm.7287.370.0 08/06/2017
5 Google Sand/Sand, BIOS Google_Sand.9042.140.0 08/22/2017
1 Google sentry/sentry, BIOS Google_Sentry.7820.156.0 08/19/2016
4 Google sentry/sentry, BIOS Google_Sentry.7820.314.0 06/08/2017
1 GOOGLE Squawks, BIOS Google_Squawks.5216.152.76 03/04/2016
19 GOOGLE Tricky, BIOS Google_Tricky.5829.18.0 02/05/2015
2 GOOGLE Ultima, BIOS Google_Ultima.7287.131.100 08/19/2017
8 GOOGLE Wizpig, BIOS Google_Wizpig.7287.333.0 02/22/2017
6 Hewlett-Packard Falco, BIOS Google_Falco.4389.92.0 12/09/2014
1 HP Snappy/Snappy, BIOS Google_Snappy.9042.154.0 10/19/2017
8 LG Electronics Monroe, BIOS Google_Monroe.4921.17.0 03/21/2014
8 NVIDIA Tegra SoC (Flattened Device Tree), model
87 SAMSUNG EXYNOS5 (Flattened Device Tree), model
This count could be biased to how many provisioning jobs were run or how many clapper devices are in the lab or how many are connected to Servo V4. I don't know. But it looks significant even with such a bias.
Among x86 platforms which share the same USB topography:
find 2017-11* -name \*message\* | xargs grep -l -m1 "usb 1-2-port4" | xargs fgrep -m 1 -e DMI: | cut -d : -f 7 | sort | uniq -c
1 GOOGLE Banon, BIOS Google_Banon.7287.313.0 11/29/2016
8 GOOGLE Banon, BIOS Google_Banon.7287.373.0 08/13/2017
4 Google Caroline/Caroline, BIOS Google_Caroline.7820.329.0 08/18/2017
104 GOOGLE Clapper, BIOS Google_Clapper.5216.199.7 08/22/2014
1 GOOGLE Gnawty, BIOS Google_Gnawty.5216.239.109 01/15/2017
3 GOOGLE Gnawty, BIOS Google_Gnawty.5216.239.123 04/23/2017
7 GOOGLE Heli, BIOS Google_Heli.5216.392.4 08/19/2015
4 GOOGLE Quawks, BIOS Google_Quawks.5216.204.61 03/03/2016
2 GOOGLE Relm, BIOS Google_Relm.7287.341.0 03/18/2017
3 GOOGLE Relm, BIOS Google_Relm.7287.370.0 08/06/2017
2 GOOGLE Ultima, BIOS Google_Ultima.7287.131.100 08/19/2017
7 GOOGLE Wizpig, BIOS Google_Wizpig.7287.333.0 02/22/2017
I'll take a look at Exynos example next.
I've attached an example /var/log/messages file from a failed clapper job:
2017-11-21/chromeos4-row4-rack3-host4/1127613-provision/provision/log/messages.preserved
I'm guessing this is a servo issue since most of the time this device is present:
443 1-2.4: Manufacturer: ATMEL
443 1-2.4: New USB device found, idVendor=03eb, idProduct=2ff4
443 1-2.4: Product: ATm32U4DFU
443 1-2.4: SerialNumber: 1.0.0
Full output of the query was:
find 2017-11* -name \*message\* | xargs grep "usb 1-2.4: " | cut -d ":" -f 6- | sed 's;.*usb;;' | sort | uniq -c | sort -n
1 1-2.4: device not accepting address 20, error -71
1 1-2.4: new low-speed USB device number 20 using xhci_hcd
1 1-2.4: New USB device found, idVendor=04b4, idProduct=6502
1 1-2.4: New USB device found, idVendor=04b4, idProduct=6572
1 1-2.4: New USB device strings: Mfr=0, Product=0, SerialNumber=0
1 1-2.4: SerialNumber: 00000001b6
1 1-2.4: SerialNumber: 0000000456
1 1-2.4: USB disconnect, device number 10
...
102 1-2.4: new low-speed USB device number 10 using xhci_hcd
108 1-2.4: device not accepting address 10, error -71
113 1-2.4: new low-speed USB device number 9 using xhci_hcd
116 1-2.4: new low-speed USB device number 7 using xhci_hcd
123 1-2.4: new high-speed USB device number 8 using xhci_hcd
126 1-2.4: New USB device found, idVendor=05e3, idProduct=0608
127 1-2.4: New USB device strings: Mfr=0, Product=1, SerialNumber=0
127 1-2.4: Product: USB2.0 Hub
134 1-2.4: device descriptor read/64, error -71
185 1-2.4: device not accepting address 6, error -71
215 1-2.4: new full-speed USB device number 7 using xhci_hcd
271 1-2.4: new full-speed USB device number 6 using xhci_hcd
443 1-2.4: Manufacturer: ATMEL
443 1-2.4: New USB device found, idVendor=03eb, idProduct=2ff4
443 1-2.4: Product: ATm32U4DFU
443 1-2.4: SerialNumber: 1.0.0
457 1-2.4: New USB device strings: Mfr=1, Product=2, SerialNumber=3
498 1-2.4: Device not responding to set address.
608 1-2.4: device descriptor read/64, error -32
622 1-2.4: Device not responding to setup address.
,
Dec 2 2017
It looks like they all use the standard SMSC NIC (Startech USB 2.0). If you like we can replace them with the Linksys USB 3.0 ones to see if it helps. chromeos4-row4-rack3-host1,clapper,smsc75xx chromeos4-row4-rack3-host2,clapper,smsc75xx chromeos4-row4-rack3-host3,clapper,smsc75xx chromeos4-row4-rack3-host4,clapper,smsc75xx chromeos4-row4-rack3-host5,clapper,smsc75xx chromeos4-row4-rack3-host6,clapper,smsc75xx chromeos4-row4-rack4-host1,clapper,smsc75xx chromeos4-row4-rack4-host2,clapper,smsc75xx chromeos4-row4-rack4-host3,clapper,smsc75xx chromeos4-row4-rack4-host4,clapper,smsc75xx chromeos4-row4-rack4-host5,clapper,smsc75xx chromeos4-row4-rack4-host6,clapper,smsc75xx chromeos4-row4-rack3-host19,, chromeos4-row4-rack4-host7,clapper,smsc75xx chromeos4-row4-rack4-host10,clapper,smsc75xx chromeos4-row4-rack4-host11,clapper,smsc75xx chromeos4-row4-rack4-host12,clapper,smsc75xx chromeos4-row4-rack3-host7,clapper,smsc75xx chromeos4-row4-rack3-host8,clapper,smsc75xx chromeos4-row4-rack3-host9,clapper,smsc75xx chromeos4-row4-rack3-host10,clapper,smsc75xx chromeos4-row4-rack3-host11,clapper,smsc75xx chromeos4-row4-rack3-host12,clapper,smsc75xx chromeos4-row4-rack3-host14,clapper,smsc75xx chromeos4-row4-rack3-host16,clapper,smsc75xx chromeos4-row4-rack2-host19,clapper,smsc75xx chromeos4-row4-rack2-host20,clapper,smsc75xx chromeos4-row4-rack2-host21,clapper,smsc75xx chromeos4-row4-rack2-host22,clapper,smsc75xx chromeos4-row4-rack3-host13,clapper,smsc75xx chromeos4-row4-rack3-host15,clapper,smsc75xx chromeos4-row4-rack3-host17,clapper,smsc75xx chromeos4-row4-rack3-host18,clapper,smsc75xx chromeos4-row4-rack3-host20,clapper,smsc75xx chromeos4-row4-rack3-host21,clapper,smsc75xx chromeos4-row4-rack3-host22,clapper,smsc75xx chromeos4-row4-rack4-host8,clapper,smsc75xx chromeos4-row4-rack4-host9,clapper,smsc75xx chromeos4-row4-rack4-host13,clapper,smsc75xx chromeos4-row4-rack4-host14,clapper,smsc75xx chromeos2-row11-rack3-host5,clapper,iwlwifi,smsc75xx(?)
,
Dec 4 2017
The USB Ethernet (SMSC7500) is not on 1-2.4. The errors appear to be with Servo V3. :(
Who "owns" servo v3 support?
2017-12-03T21:20:58.299801-08:00 INFO kernel: [ 10.954078] smsc75xx 1-1:1.0 eth1: link up, 1000Mbps, full-duplex, lpa 0xC5E1
localhost ~ # fgrep 1-2-port4 /var/log/messages*
2017-12-04T05:12:03.244890+00:00 ERR kernel: [ 3.076214] usb 1-2-port4: Cannot enable. Maybe the USB cable is bad?
2017-12-04T05:12:03.244918+00:00 ERR kernel: [ 4.052000] usb 1-2-port4: Cannot enable. Maybe the USB cable is bad?
2017-12-04T05:12:03.244921+00:00 ERR kernel: [ 4.880619] usb 1-2-port4: Cannot enable. Maybe the USB cable is bad?
2017-12-04T05:12:03.244924+00:00 ERR kernel: [ 5.701469] usb 1-2-port4: Cannot enable. Maybe the USB cable is bad?
2017-12-04T05:12:03.244927+00:00 ERR kernel: [ 5.701734] usb 1-2-port4: unable to enumerate USB device
...
Looking at chromeos4-row4-rack3-host1.cros:
hwid = CLAPPER D3A-C5M-J6A-A4X # Hardware ID
localhost ~ # lsusb
Bus 001 Device 008: ID 0424:ec00 Standard Microsystems Corp. SMSC9512/9514 Fast Ethernet Adapter
Bus 001 Device 005: ID 0424:9514 Standard Microsystems Corp. SMC9514 Hub
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 006: ID 8087:07dc Intel Corp.
Bus 001 Device 004: ID 5986:0657 Acer, Inc
Bus 001 Device 003: ID 05e3:0610 Genesys Logic, Inc. 4-port hub
Bus 001 Device 002: ID 0424:7500 Standard Microsystems Corp. LAN7500 Ethernet 10/100/1000 Adapter
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
localhost ~ # lsusb -vt
/: Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/1p, 5000M
/: Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/6p, 480M
|__ Port 1: Dev 2, If 0, Class=Vendor Specific Class, Driver=smsc75xx, 480M
|__ Port 2: Dev 3, If 0, Class=Hub, Driver=hub/4p, 480M
|__ Port 1: Dev 5, If 0, Class=Hub, Driver=hub/5p, 480M
|__ Port 1: Dev 8, If 0, Class=Vendor Specific Class, Driver=smsc95xx, 480M
|__ Port 3: Dev 4, If 0, Class=Video, Driver=uvcvideo, 480M
|__ Port 3: Dev 4, If 1, Class=Video, Driver=uvcvideo, 480M
|__ Port 4: Dev 6, If 0, Class=Wireless, Driver=btusb, 12M
|__ Port 4: Dev 6, If 1, Class=Wireless, Driver=btusb, 12M
localhost 1-2 # echo 0 > authorized
localhost 1-2 # echo 1 > authorized
localhost 1-2 # dmesg
...
[59225.129737] usb 1-2.1: USB disconnect, device number 5
[59225.129750] usb 1-2.1.1: USB disconnect, device number 8
[59225.129971] smsc95xx 1-2.1.1:1.0 eth0: unregister 'smsc95xx' usb-0000:00:14.0-2.1.1, smsc95xx USB 2.0 Ethernet
[59225.130002] smsc95xx 1-2.1.1:1.0 eth0: hardware isn't capable of remote wakeup
[59228.510480] usb 1-2: reset high-speed USB device number 3 using xhci_hcd
[59228.679319] hub 1-2:1.0: USB hub found
[59228.680279] hub 1-2:1.0: 4 ports detected
[59228.683424] usb 1-2: authorized to connect
[59228.951369] usb 1-2.1: new high-speed USB device number 12 using xhci_hcd
[59229.025893] usb 1-2.1: New USB device found, idVendor=0424, idProduct=9514
[59229.025904] usb 1-2.1: New USB device strings: Mfr=0, Product=0, SerialNumber=0
[59229.027255] hub 1-2.1:1.0: USB hub found
[59229.027294] hub 1-2.1:1.0: 5 ports detected
[59229.093006] usb 1-2.4: new low-speed USB device number 13 using xhci_hcd
[59229.917054] usb 1-2-port4: Cannot enable. Maybe the USB cable is bad?
[59229.990431] usb 1-2.1.1: new high-speed USB device number 14 using xhci_hcd
[59230.074628] usb 1-2.1.1: New USB device found, idVendor=0424, idProduct=ec00
[59230.074640] usb 1-2.1.1: New USB device strings: Mfr=0, Product=0, SerialNumber=0
[59230.078080] smsc95xx v1.0.4
[59230.121081] smsc95xx 1-2.1.1:1.0 eth0: register 'smsc95xx' at usb-0000:00:14.0-2.1.1, smsc95xx USB 2.0 Ethernet, 6e:b4:f1:02:e4:46
[59230.177375] smsc95xx 1-2.1.1:1.0 eth0: hardware isn't capable of remote wakeup
[59230.177459] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
localhost 1-2 # pwd
/sys/bus/usb/devices/1-2
and the output from "lsusb -vt" doesn't change.
,
Dec 4 2017
,
Dec 5 2017
,
Jan 25 2018
This is still happening but clapper isn't the worst offender:
grundler <2107>grep -l -R -m1 "device not accepting address [0-9]*, error -71" 2018-01-* | xargs fgrep -m1 " DMI:" | cut -d ":" -f 7 | sort | uniq -c | sort -n
1 GOOGLE Candy, BIOS Google_Candy.5216.310.57 03/28/2016
1 Google Caroline/Caroline, BIOS Google_Caroline.7820.349.0 11/17/2017
1 Google Coral/Coral, BIOS Google_Coral.10068.27.0 12/04/2017
1 GOOGLE Kefka, BIOS Google_Kefka.7287.379.0 09/17/2017
1 HP Snappy/Snappy, BIOS Google_Snappy.9042.158.0 11/08/2017
2 Google Asuka/Asuka, BIOS Google_Asuka.7820.314.0 06/08/2017
2 GOOGLE Banjo, BIOS Google_Banjo.5216.334.4 04/09/2015
2 Google Pyro/Pyro, BIOS Google_Pyro.9042.110.0 05/16/2017
2 GOOGLE Ultima, BIOS Google_Ultima.7287.131.100 08/19/2017
3 GOOGLE Gnawty, BIOS Google_Gnawty.5216.239.123 04/23/2017
4 GOOGLE Relm, BIOS Google_Relm.7287.370.0 08/06/2017
4 Google Sand/Sand, BIOS Google_Sand.9042.140.0 08/22/2017
4 Hewlett-Packard Falco, BIOS Google_Falco.4389.92.0 12/09/2014
5 Google Caroline/Caroline, BIOS Google_Caroline.7820.356.0 12/05/2017
5 GOOGLE Clapper, BIOS Google_Clapper.5216.199.7 08/22/2014
5 Google Eve/Eve, BIOS Google_Eve.9584.107.0 11/07/2017
6 GOOGLE Mccloud, BIOS Google_Mccloud.5827.14.0 10/03/2014
9 Hewlett-Packard Zako, BIOS Google_Zako.5219.30.0 10/07/2014
11 GOOGLE Banon, BIOS Google_Banon.7287.373.0 08/13/2017
13 Google Reef/Reef, BIOS Google_Reef.9042.110.0 05/16/2017
19 Google Cave/Cave, BIOS Google_Cave.7820.329.0 08/18/2017
20 GOOGLE Panther, BIOS Google_Panther.4920.24.26 10/03/2014
20 Google sentry/sentry, BIOS Google_Sentry.7820.314.0 06/08/2017
20 LG Electronics Monroe, BIOS Google_Monroe.4921.17.0 03/21/2014
21 GOOGLE Tricky, BIOS Google_Tricky.5829.18.0 02/05/2015
24 GOOGLE Wizpig, BIOS Google_Wizpig.7287.381.0 10/01/2017
49 Acer Peppy, BIOS Google_Peppy.4389.117.0 03/02/2017
51 GOOGLE Quawks, BIOS Google_Quawks.5216.204.61 03/03/2016
75 GOOGLE Auron_Paine, BIOS Google_Auron_paine.6301.58.98 01/06/2017
And just in case there are different FW versions:
1 GOOGLE Candy
1 Google Coral/Coral
1 GOOGLE Kefka
1 HP Snappy/Snappy
2 Google Asuka/Asuka
2 GOOGLE Banjo
2 Google Pyro/Pyro
2 GOOGLE Ultima
3 GOOGLE Gnawty
4 GOOGLE Relm
4 Google Sand/Sand
4 Hewlett-Packard Falco
5 GOOGLE Clapper
5 Google Eve/Eve
6 Google Caroline/Caroline
6 GOOGLE Mccloud
9 Hewlett-Packard Zako
11 GOOGLE Banon
14 Google Reef/Reef
19 Google Cave/Cave
20 GOOGLE Panther
20 Google sentry/sentry
20 LG Electronics Monroe
21 GOOGLE Tricky
24 GOOGLE Wizpig
49 Acer Peppy
54 GOOGLE Quawks
75 GOOGLE Auron_Paine
Manually looking up the chipset:
1 GOOGLE Candy Baytrail
1 Google Coral/Coral TBD
1 GOOGLE Kefka Braswell
1 HP Snappy/Snappy ApolloLake
2 Google Asuka/Asuka Skylake
2 GOOGLE Banjo Baytrail
2 Google Pyro/Pyro Apollolake
2 GOOGLE Ultima Braswell
3 GOOGLE Gnawty Baytrail
4 GOOGLE Relm Braswell
4 Google Sand/Sand Apollolake
4 Hewlett-Packard Falco Haswell
5 GOOGLE Clapper Baytrail
5 Google Eve/Eve Kabylake
6 Google Caroline/Caroline skylake
6 GOOGLE Mccloud Haswell
9 Hewlett-Packard Zako Haswell
11 GOOGLE Banon Braswell
14 Google Reef/Reef ApolloLake
19 Google Cave/Cave Skylake
20 GOOGLE Panther Haswell
20 Google sentry/sentry Skylake
20 LG Electronics Monroe Haswell
21 GOOGLE Tricky Haswell
24 GOOGLE Wizpig Braswell
49 Acer Peppy Haswell
54 GOOGLE Quawks Baytrail
75 GOOGLE Auron_Paine Broadwell
Sorting by chipset seemed interesting thing to do:
5 Kabylake (Eve)
21 = 14+4+2+1 Apollolake (Reef, Sand, Pyro, Snappy,
47 = 20+19+6+2 Skylake (sentry,cave,caroline, Asuka
42 = 24+11+4+2+1 Braswell (Wizpig, Banon, Relm, Ultima, kefka)
65 = 54+5+3+2+1 Baytrail (Quawks, Clapper, Gnawty, Banjo, Candy)
75 Broadwell (Paine)
129 = 49+21+20+20+9+6+4 Haswell (Peppy, Tricky, Monroe, Panther, Zako, Mccloud, Falco)
Suggests two things to me:
1) Haswell systems have some categorical problem (with peppy being the worst offender)
2) Paine is the only broadwell system: Something is wrong with this specific platform
I'll dig into "Paine" issues since that seems to be the "lowest hanging fruit" right now.
,
Jan 25 2018
BTW, the distribution looks different to what I posted in comment #1 - clapper is (mostly) no longer the issue:
find 2018-01* -name \*message\* | xargs grep -l -m1 "unable to enumerate USB device" | xargs fgrep -m 1 -e DMI: -e "0.000000] Machine" -e "0.000000] machine" | cut -d : -f 7 | cut -d ',' -f 1 | sort | uniq -c | sort -n
1 GOOGLE Candy
1 Google IPQ4019/Gale
1 GOOGLE Kefka
1 HP Snappy/Snappy
2 Google Asuka/Asuka
2 GOOGLE Banjo
2 Google Coral/Coral
2 Google Jaq
2 Google Pyro/Pyro
2 GOOGLE Ultima
3 GOOGLE Gnawty
4 GOOGLE Relm
4 Google Sand/Sand
5 Google Eve/Eve
5 Hewlett-Packard Falco
5 SAMSUNG Stumpy
6 GOOGLE Clapper
11 GOOGLE Banon
14 Google Reef/Reef
21 Google sentry/sentry
24 Google Caroline/Caroline
25 GOOGLE Cyan
25 GOOGLE Wizpig
40 Google Cave/Cave
57 GOOGLE Quawks
79 GOOGLE Auron_Paine
83 Acer Peppy
130 SAMSUNG EXYNOS5 (Flattened Device Tree)
Except "Acer Peppy" tops the list when looking for a related but different message - which I think is more accurate since it represents actual failure.
ARM systems report the USB configuration header failure with a different error message due to a different driver (not intel) but share the code which reports the enumeration error. So I might have to look into this as well.
,
Jan 25 2018
,
Feb 17 2018
Complete numbers for Jan 2018 and haswell devices are a clear winner just looking at anything which reported more than 10 failures:
# grep -l -R -m1 "device not accepting address [0-9]*, error -71" 2018-01-* | xargs fgrep -m1 -e " DMI:" -e "0.000000] Machine" -e "0.000000] machine" | cut -d ":" -f 7,8 | sort | uniq -c | sort -n
...
10 GOOGLE Candy, BIOS Google_Candy.5216.310.57 03/28/2016
13 GOOGLE Banon, BIOS Google_Banon.7287.373.0 08/13/2017
14 Hewlett-Packard Falco, BIOS Google_Falco.4389.92.0 12/09/2014
19 Google Reef/Reef, BIOS Google_Reef.9042.110.0 05/16/2017
20 Google Eve/Eve, BIOS Google_Eve.9584.107.0 11/07/2017
22 Google Cave/Cave, BIOS Google_Cave.7820.329.0 08/18/2017
22 GOOGLE Clapper, BIOS Google_Clapper.5216.199.7 08/22/2014
31 GOOGLE Wizpig, BIOS Google_Wizpig.7287.381.0 10/01/2017
45 GOOGLE Panther, BIOS Google_Panther.4920.24.26 10/03/2014
50 Google sentry/sentry, BIOS Google_Sentry.7820.314.0 06/08/2017
50 Hewlett-Packard Zako, BIOS Google_Zako.5219.30.0 10/07/2014
51 GOOGLE Mccloud, BIOS Google_Mccloud.5827.14.0 10/03/2014
58 GOOGLE Quawks, BIOS Google_Quawks.5216.204.61 03/03/2016
76 Acer Peppy, BIOS Google_Peppy.4389.117.0 03/02/2017
79 GOOGLE Auron_Paine, BIOS Google_Auron_paine.6301.58.98 01/06/2017
85 GOOGLE Tricky, BIOS Google_Tricky.5829.18.0 02/05/2015
183 LG Electronics Monroe, BIOS Google_Monroe.4921.17.0 03/21/2014
230 NVIDIA Tegra SoC (Flattened Device Tree), model: Google Nyan Kitty Rev 0-3
Adding Doug since he might be interested in or want to fork a new bug for Nyan Kitty.
It's interesting that Nyan Blaze is not listed...and that 129 "SAMSUNG EXYNOS5" disappeared (had one report not shown here). *sigh*. Will need to sort out what happened to the data since I thought I had recollected all the logs properly. Maybe not.
,
May 3 2018
I was previously only counting a fraction of the incidents. Here is a more recent and hopefully more complete count for April 2018.
# top 10 offenders
...
1303 GOOGLE Banon, BIOS Google_Banon.7287.373.0 08/13/2017
1389 GOOGLE Auron_Paine, BIOS Google_Auron_paine.6301.58.98 01/06/2017
1546 Google Caroline/Caroline, BIOS Google_Caroline.7820.356.0 12/05/2017
1596 GOOGLE Mccloud, BIOS Google_Mccloud.5827.14.0 10/03/2014
1796 Hewlett-Packard Zako, BIOS Google_Zako.5219.30.0 10/07/2014
1888 GOOGLE Clapper, BIOS Google_Clapper.5216.199.7 08/22/2014
1927 Google Eve/Eve, BIOS Google_Eve.9584.107.0 11/07/2017
2052 LG Electronics Monroe, BIOS Google_Monroe.4921.17.0 03/21/2014
2834 GOOGLE Panther, BIOS Google_Panther.4920.24.26 10/03/2014
4675 GOOGLE Tricky, BIOS Google_Tricky.5829.18.0 02/05/2015
4968 NVIDIA Tegra SoC (Flattened Device Tree), model
(Yes, that's actually 11; I'm not counting "Nvidia Tegra SoC")
Haswell: (Tricky, Panther, Monroe, Zako, Mccloud,) difficult to prioritize since Haswell are running chromeos-3.8 kernels - will be very difficult to backport any fixes to - yet Haswell is by far seeing the most of these failures. What's more odd is these are mostly Chrome Boxes and have loads of USB ports and built-in PCI ethernet. All are based on beltino reference board. :/
Kabylake : (Eve) running chromeos-4.4 and has a very high failure number. Fixes should be easier to apply here. I'll look into this first.
Baytrail (Clapper) also a 4.4 kernel.
Skylake: (Caroline) 3.18 kernel
Braswell (Banon) 3.18 kernel
Broadwell: (Auron_Paine) 3.14 kernel
# full list in alphabetical order
find 2018-03* -name \*message\* | xargs grep -l "device not accepting address [0-9]*, error -71" | xargs fgrep -m 1 -e DMI: -e "0.000000] Machine" -e "0.000000] machine" | cut -d : -f 7 | sort | uniq -c
5 Acer Lars/Lars, BIOS Google_Lars.7820.317.0 06/08/2017
623 Acer Peppy, BIOS Google_Peppy.4389.117.0 03/02/2017
43 Dell Wolf, BIOS Google_Wolf.4389.24.62 07/30/2015
562 Google Asuka/Asuka, BIOS Google_Asuka.7820.314.0 06/08/2017
1389 GOOGLE Auron_Paine, BIOS Google_Auron_paine.6301.58.98 01/06/2017
932 GOOGLE Banjo, BIOS Google_Banjo.5216.334.4 04/09/2015
1303 GOOGLE Banon, BIOS Google_Banon.7287.373.0 08/13/2017
17 GOOGLE Buddy, BIOS Google_Buddy.6301.202.5 01/09/2016
38 GOOGLE Candy, BIOS Google_Candy.5216.310.57 03/28/2016
1546 Google Caroline/Caroline, BIOS Google_Caroline.7820.356.0 12/05/2017
1 Google Cave/Cave, BIOS Google_Cave.7820.242.0 12/20/2016
2 Google Cave/Cave, BIOS Google_Cave.7820.288.0 03/21/2017
709 Google Cave/Cave, BIOS Google_Cave.7820.347.0 11/10/2017
1888 GOOGLE Clapper, BIOS Google_Clapper.5216.199.7 08/22/2014
10 Google Coral/Coral, BIOS Google_Coral.10068.27.0 12/04/2017
142 Google Coral/Coral, BIOS Google_Coral.10068.34.0 12/15/2017
68 Google Coral/Coral, BIOS Google_Coral.10068.37.0 12/15/2017
3 Google Coral/Coral, BIOS Google_Coral.10068.39.0 12/15/2017
130 Google Coral/Coral, BIOS Google_Coral.10068.45.0 02/08/2018
86 Google Coral/Coral, BIOS Google_Coral.10068.47.0 02/08/2018
92 GOOGLE Cyan, BIOS Google_Cyan.7287.57.145 10/07/2017
2 GOOGLE Edgar, BIOS Google_Edgar.7287.167.96 10/22/2017
73 GOOGLE Enguarde, BIOS Google_Enguarde.5216.201.10 12/06/2014
1927 Google Eve/Eve, BIOS Google_Eve.9584.107.0 11/07/2017
2 Google Eve/Eve, BIOS Google_Eve.9584.95.0 09/27/2017
1 Google Fievel
53 GOOGLE Gnawty, BIOS Google_Gnawty.5216.239.109 01/15/2017
142 GOOGLE Gnawty, BIOS Google_Gnawty.5216.239.123 04/23/2017
1 GOOGLE Guado, BIOS Google_Guado.6301.108.4 04/01/2015
632 GOOGLE Heli, BIOS Google_Heli.5216.392.4 08/19/2015
170 Google Jaq
123 GOOGLE Kefka, BIOS Google_Kefka.7287.379.0 09/17/2017
1 GOOGLE Link, BIOS Google_Link.2695.1.169 11/18/2015
1596 GOOGLE Mccloud, BIOS Google_Mccloud.5827.14.0 10/03/2014
100 GOOGLE Ninja, BIOS Google_Ninja.5216.383.7 08/10/2015
1 GOOGLE Panther, BIOS Google_Panther.4920.24.22 03/05/2014
2834 GOOGLE Panther, BIOS Google_Panther.4920.24.26 10/03/2014
1 Google Poppy/Poppy, BIOS Google_Poppy.10068.0.0 10/26/2017
1127 Google Pyro/Pyro, BIOS Google_Pyro.9042.110.0 05/16/2017
960 GOOGLE Quawks, BIOS Google_Quawks.5216.204.61 03/03/2016
687 Google Reef/Reef, BIOS Google_Reef.9042.110.0 05/16/2017
4 GOOGLE Reks, BIOS Google_Reks.7287.133.103 08/20/2017
1262 GOOGLE Relm, BIOS Google_Relm.7287.370.0 08/06/2017
6 GOOGLE Samus, BIOS Google_Samus.6300.276.0 08/17/2016
984 Google Sand/Sand, BIOS Google_Sand.9042.140.0 08/22/2017
505 Google sentry/sentry, BIOS Google_Sentry.7820.314.0 06/08/2017
444 GOOGLE Squawks, BIOS Google_Squawks.5216.152.76 03/04/2016
70 GOOGLE Sumo, BIOS Google_Sumo.5216.382.5 08/08/2015
1 GOOGLE Terra, BIOS Google_Terra.7287.154.102 08/19/2017
162 GOOGLE Tidus, BIOS Google_Tidus.6301.109.17 05/31/2015
1 Google Tiger
4675 GOOGLE Tricky, BIOS Google_Tricky.5829.18.0 02/05/2015
285 GOOGLE Ultima, BIOS Google_Ultima.7287.131.100 08/19/2017
953 GOOGLE Wizpig, BIOS Google_Wizpig.7287.381.0 10/01/2017
3 Hewlett-Packard Chell/Chell, BIOS Google_Chell.7820.314.0 06/08/2017
384 Hewlett-Packard Falco, BIOS Google_Falco.4389.92.0 12/09/2014
1796 Hewlett-Packard Zako, BIOS Google_Zako.5219.30.0 10/07/2014
7 HP Snappy/Snappy, BIOS Google_Snappy.9042.136.0 08/22/2017
964 HP Snappy/Snappy, BIOS Google_Snappy.9042.167.0 12/18/2017
42 HP Soraka/Soraka, BIOS Google_Soraka.10337.0.0 01/24/2018
2052 LG Electronics Monroe, BIOS Google_Monroe.4921.17.0 03/21/2014
4968 NVIDIA Tegra SoC (Flattened Device Tree), model
5 SAMSUNG EXYNOS5 (Flattened Device Tree), model
1 Toshiba Leon, BIOS Google_Leon.4389.61.46 08/26/2014
,
May 4 2018
Kirtika reproduced this at her desk (random chance) and the log has an interesting sequence (look for "usb 2-3"): 2018-05-03T14:41:31.976073-07:00 WARNING kernel: [ 11.019190] xhci_hcd 0000:00:14.0: Timeout while waiting for setup device command 2018-05-03T14:41:31.976107-07:00 ERR kernel: [ 11.019223] usb 1-4: hub failed to enable device, error -62 2018-05-03T14:41:31.979083-07:00 WARNING kernel: [ 11.022602] usb 2-3: Device not responding to setup address. 2018-05-03T14:41:32.132046-07:00 INFO kernel: [ 11.175209] usb 1-4: new high-speed USB device number 3 using xhci_hcd 2018-05-03T14:41:32.180073-07:00 ERR kernel: [ 11.223184] usb 2-3: device not accepting address 2, error -71 2018-05-03T14:41:32.331130-07:00 INFO kernel: [ 11.375094] usb 1-4: New USB device found, idVendor=0bda, idProduct=57cf, bcdDevice= 0.03 2018-05-03T14:41:32.331165-07:00 INFO kernel: [ 11.375117] usb 1-4: New USB device strings: Mfr=3, Product=1, SerialNumber=2 2018-05-03T14:41:32.331169-07:00 INFO kernel: [ 11.375146] usb 1-4: Product: HD WebCam 2018-05-03T14:41:32.331171-07:00 INFO kernel: [ 11.375156] usb 1-4: Manufacturer: HD WebCam 2018-05-03T14:41:32.331195-07:00 INFO kernel: [ 11.375165] usb 1-4: SerialNumber: ^GNC2141103Q60208A7BLM03 2018-05-03T14:41:32.335064-07:00 INFO kernel: [ 11.378283] platform vpd: Driver vpd requests probe deferral 2018-05-03T14:41:32.487070-07:00 INFO kernel: [ 11.530220] usb 1-5: new full-speed USB device number 4 using xhci_hcd 2018-05-03T14:41:32.652052-07:00 INFO kernel: [ 11.695462] usb 1-5: New USB device found, idVendor=8087, idProduct=0a2a, bcdDevice= 0.01 2018-05-03T14:41:32.652083-07:00 INFO kernel: [ 11.695486] usb 1-5: New USB device strings: Mfr=0, Product=0, SerialNumber=0 ... 2018-05-03T14:41:34.260059-07:00 WARNING kernel: [ 13.303135] hub 2-0:1.0: Could not enable port 3 after 2000 ms [but then what happened? What caused this?] 2018-05-03T14:41:34.716049-07:00 INFO kernel: [ 13.759213] usb 2-3: new SuperSpeed USB device number 3 using xhci_hcd 2018-05-03T14:41:34.727052-07:00 INFO kernel: [ 13.770542] usb 2-3: New USB device found, idVendor=0bda, idProduct=8152, bcdDevice=30.00 2018-05-03T14:41:34.727084-07:00 INFO kernel: [ 13.770566] usb 2-3: New USB device strings: Mfr=1, Product=2, SerialNumber=6 2018-05-03T14:41:34.727088-07:00 INFO kernel: [ 13.770580] usb 2-3: Product: USB 10/100/1000 LAN 2018-05-03T14:41:34.727091-07:00 INFO kernel: [ 13.770591] usb 2-3: Manufacturer: Realtek 2018-05-03T14:41:34.727093-07:00 INFO kernel: [ 13.770600] usb 2-3: SerialNumber: 000001000000 2018-05-03T14:41:34.728033-07:00 INFO kernel: [ 13.771607] usb-storage 2-3:1.0: USB Mass Storage device detected 2018-05-03T14:41:34.728054-07:00 INFO kernel: [ 13.771843] scsi host0: usb-storage 2-3:1.0 [Storage!! "mist" will find and recover this device to networking. But it means the USB subsystem didn't meet the timing requirements for USB state transitions.] 2018-05-03T14:41:35.148053-07:00 INFO kernel: [ 14.191166] usb 2-3: USB disconnect, device number 3 ... 2018-05-03T14:41:35.106070-07:00 INFO kernel: [ 14.515410] usb 2-3: reset SuperSpeed USB device number 4 using xhci_hcd 2018-05-03T14:41:35.152018-07:00 INFO kernel: [ 14.561533] r8152 2-3:1.0 eth0: v1.09.9 2018-05-03T14:41:35.152035-07:00 INFO kernel: [ 14.561643] platform vpd: Driver vpd requests probe deferral 2018-05-03T14:41:35.168849-07:00 INFO kernel: [ 14.578194] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready ... [ but someone still wants cdc_ether driver? Maybe mist noticed this and decided it should load the driver? ] 2018-05-03T14:41:35.190846-07:00 INFO kernel: [ 14.600469] usbcore: registered new interface driver cdc_ether ... 2018-05-03T14:41:37.754552-07:00 INFO kernel: [ 17.230059] r8152 2-3:1.0 eth0: carrier on 2018-05-03T14:41:37.754621-07:00 INFO kernel: [ 17.230251] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready ... 2018-05-03T14:41:38.625587-07:00 INFO avahi-daemon[1785]: Joining mDNS multicast group on interface eth0.IPv4 with address 100.127.30.156. 2018-05-03T14:41:38.625644-07:00 INFO avahi-daemon[1785]: New relevant interface eth0.IPv4 for mDNS. 2018-05-03T14:41:38.625666-07:00 INFO avahi-daemon[1785]: Registering new address record for 100.127.30.156 on eth0.IPv4. I'm just wondering if CPU utilization during boot time is causing USB tasks to not run in time. :/ But the "could not enable port 3" message seems like a big red flag.
,
May 4 2018
Same machine as comment #11: Hardware name: GOOGLE Cyan, BIOS Google_Cyan.7287.57.125 06/06/2017 But a later boot: 2018-05-03T14:59:40.020188-07:00 INFO kernel: [ 0.999736] usb 2-3: new SuperSpeed USB device number 2 using xhci_hcd 2018-05-03T14:59:40.020191-07:00 INFO kernel: [ 1.011197] usb 2-3: New USB device found, idVendor=0bda, idProduct=8153, bcdDevice=30.00 2018-05-03T14:59:40.020194-07:00 INFO kernel: [ 1.011213] usb 2-3: New USB device strings: Mfr=1, Product=2, SerialNumber=6 2018-05-03T14:59:40.020198-07:00 INFO kernel: [ 1.011227] usb 2-3: Product: USB 10/100/1000 LAN 2018-05-03T14:59:40.020201-07:00 INFO kernel: [ 1.011236] usb 2-3: Manufacturer: Realtek 2018-05-03T14:59:40.020204-07:00 INFO kernel: [ 1.011246] usb 2-3: SerialNumber: 000001000000 ... 2018-05-03T14:59:40.044691-07:00 INFO kernel: [ 2.056829] usb 2-3: reset SuperSpeed USB device number 2 using xhci_hcd [wtf? log time and kernel time don't agree on the delta from previous entry :/ In other words, can't trust /var/log/messages time stamps. :( And there is no output explaining why the interface was reset... It looked like it came up correctly. ] ... 2018-05-03T14:59:40.051066-07:00 INFO kernel: [ 2.095314] r8152 2-3:1.0 eth0: v1.09.9 ... 2018-05-03T14:59:41.362021-07:00 INFO kernel: [ 3.406370] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready ... [hrm... "link is ready" message is missing... stuff was dropped on the floor somewhere along the way. :( ] 2018-05-03T14:59:43.794271-07:00 INFO avahi-daemon[1768]: Joining mDNS multicast group on interface eth0.IPv4 with address 100.127.30.156. 2018-05-03T14:59:43.794320-07:00 INFO avahi-daemon[1768]: New relevant interface eth0.IPv4 for mDNS. 2018-05-03T14:59:43.794347-07:00 INFO avahi-daemon[1768]: Network interface enumeration completed. 2018-05-03T14:59:43.794370-07:00 INFO avahi-daemon[1768]: Registering new address record for 100.127.30.156 on eth0.IPv4. ...
,
Jun 25 2018
,
Jun 25 2018
,
Jun 25 2018
|
||||||||||||||
►
Sign in to add a comment |
||||||||||||||
Comment 1 by nsanders@chromium.org
, Dec 2 2017