servod broken on some boards
Reported by
jrbarnette@chromium.org,
Mar 11 2016
|
||||||||
Issue description
Using R51-8041.0.0, servod on lars fails to start.
Other boards have been affected: when we upgraded the lab
to R51-8041.0.0, servod went offline on ~387 servo hosts.
Here's a sample of the failure from a lars servo beaglebone:
2016-03-11 16:35:05,810 - SystemConfig - INFO - Loading XML config /usr/lib/python2.7/site-packages/servo/data/ec_common.xml
Traceback (most recent call last):
File "/usr/lib/python-exec/python2.7/servod", line 9, in <module>
load_entry_point('servo===0.0.1-4c7d647', 'console_scripts', 'servod')()
File "/usr/lib/python2.7/site-packages/servo/servod.py", line 512, in main
main_function()
File "/usr/lib/python2.7/site-packages/servo/servod.py", line 462, in main_function
scfg.add_cfg_file(cfg_file)
File "/usr/lib/python2.7/site-packages/servo/system_config.py", line 161, in add_cfg_file
self.add_cfg_file(element.find('name').text)
File "/usr/lib/python2.7/site-packages/servo/system_config.py", line 161, in add_cfg_file
self.add_cfg_file(element.find('name').text)
File "/usr/lib/python2.7/site-packages/servo/system_config.py", line 224, in add_cfg_file
% (tag, name, element_str))
servo.system_config.SystemConfigError: Duplicate control uart4_pty without 'clobber_ok' key
<control>
<name>uart4_pty</name>
<alias>raw_usbpd_uart_pty</alias>
<doc>Psuedo-terminal (pty) thats connnected to USB PD MCU's
uart console</doc>
<params cmd="get" drv="uart" interface="1" subtype="pty">
</params>
</control>
,
Mar 11 2016
Servo in the lab has been rolled back to the previous version, R50-7850.0.0. This bug is blocking a requested update for new features.
,
Mar 11 2016
Are all the hosts lars? Or, do you have a list of the boards that failed?
,
Mar 11 2016
add tree sheriffs
,
Mar 11 2016
There should probably be a test that checks that servo works... For example the test could lockup the kernels on the machines and if servo can't recover it, fail the test. If that was part of the CQ we'd be set against that type of problem.
,
Mar 11 2016
My early guess at the problem in this particular case is making changes and only verifying with servo v2 instead of on v3 since most developers don't use v3 at their desks. This seems like an error right on startup of servo, so it's most likely the case that it wasn't tested with servo v3. I do agree that having some basic sanity checks for servo in the CQ might be worthwhile. It's hard to do it as an individual since there are so many different boards.
,
Mar 11 2016
I've reproduced the failure and am currently testing a fix. I'm assuming that the other boards that are failing (or would fail) are glados, chell, kunimitsu. Essentially, all boards that inherit from the servo_glados_overlay.xml. Some other ideas that are coming to mind to prevent this kind of thing in the future: - What if we had a pool with 1 or maybe 2 DUTs per board (depending upon availability), whose roles are to vet the next beaglebone_servo update? That way you can just upgrade that pool and check for basic servo functionality (and/or other tests) before rolling out to the rest of the lab?
,
Mar 11 2016
Verified locally, CL here: https://chromium-review.googlesource.com/#/c/332224/ Once that lands, feel free to update the lab images.
,
Mar 11 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/hdctools/+/d373751e74e5890d1e094599dc58d9b3dfa9386d commit d373751e74e5890d1e094599dc58d9b3dfa9386d Author: Aseda Aboagye <aaboagye@google.com> Date: Fri Mar 11 18:53:09 2016 servo: Fix glados overlay for servo v3. The glados boards and it's variants have repurposed the JTAG pins for servo v2 to be used for the USB PD UART. This was introduced as uart4. However, servo v3 already had a uart4 control so the glados overlay was causing a conflict immediately causing servod to fail upon start. This commit simply allows the glados overlay to clobber the existing uart4 for servo v3. BUG= chromium:594163 BRANCH=None TEST=cros_workon hdctools --board beaglebone_servo; Built a test image and loaded it onto a servo v3. Verified that servo no longer crashed upon starting for the board of lars, glados, chell, and kunimitsu. TEST=Using servo v3, was able to get an EC console. Change-Id: Ieb98a943a667f20fde123c22d8b02aaa4274f507 Signed-off-by: Aseda Aboagye <aaboagye@google.com> Reviewed-on: https://chromium-review.googlesource.com/332224 Trybot-Ready: Aseda Aboagye <aaboagye@chromium.org> Tested-by: Aseda Aboagye <aaboagye@chromium.org> Reviewed-by: Shawn N <shawnn@chromium.org> Commit-Queue: Aseda Aboagye <aaboagye@chromium.org> Commit-Queue: Shawn N <shawnn@chromium.org> [modify] https://crrev.com/d373751e74e5890d1e094599dc58d9b3dfa9386d/servo/data/servo_glados_overlay.xml
,
Mar 11 2016
,
Apr 11 2016
,
Apr 27 2016
,
May 23 2016
Bulk verified |
||||||||
►
Sign in to add a comment |
||||||||
Comment 1 by jrbarnette@chromium.org
, Mar 11 2016