[chameleon] Chameleon unpingable, unsshable, unscreenable from DUT |
||||
Issue descriptionObserved two chameleon boards in lab today (9-4-8-4, 9-4-7-4) that were unpingable using their IP address, unsshable, and unscreenable from the DUT they were attached to. Manual inspection of the boards noted that the chameleon image was on the LCD but the LCD was not backlit with the blue light (@josephsih has a picture of this) The RN42 kit attached was also not lit with any LED lights. Only a manual reset of the board fixed the problem. Any ideas of what was going on?
,
Aug 8 2017
I still could not ping 9-4-8-4. But I found that chameleon at 9-4-7-4 has no space left on device. Which is https://bugs.chromium.org/p/chromium/issues/detail?id=747841. After removing the log file /www/logs/lighttpd.error.log, device should behave normally. I have not seen boards that is unscreenable from the DUT. If that board repro this issue constantly we should look into it further. Thanks!
,
Aug 8 2017
Oops, I mistyped earlier. The two boards are 9-4-8-3 and 9-4-7-4.
,
Aug 9 2017
Ok! 9-4-8-3 looks fine now.
,
Aug 9 2017
Great, I was able to clear the logs on 9-4-8-3 to free up space. We were able to reset the boards and have them working normally but the problem was that they were unpingable, unsshable, and unscreenable in the first place. Is there a automated or remote solution so that we can avoid manually resetting?
,
Aug 10 2017
It is not clear what was the state of the chameleon. We can put some logic like "reboot if it can not get IP for 1 days" in chameleond, but we are not sure whether that logic will be running when this issue happens. Did you try to connect to it through screen after the reboot ? Hardware issue on USB serial port is one of the possible reasons.
,
Aug 11 2017
We were able to screen into it after the manual reboot. Seems like the logic won't be running. Since the state is unclear, let's just be aware of this issue (I placed it in the Chameleon Debug Doc) and reopen this bug if it happens again.
,
Aug 23 2017
This happened again with 9-4-8-1, 9-4-8-2, 9-4-8-3, 9-4-7-2, 9-4-7-4. I've asked Chromestop for additional chameleon boards to replace these five.
,
Sep 13 2017
Happening again with 9-4-8-1, 9-4-7-2, 9-4-7-3. Two of these boards (9-4-8-1, 9-4-7-2) failed before according to #8. Will replace these two and check the status of the other failing board in lab.
,
Sep 13 2017
,
Sep 15 2017
I've replaced 9-4-8-1, 9-4-7-2, and 9-4-7-3 chameleon boards. |
||||
►
Sign in to add a comment |
||||
Comment 1 by rjahagir@chromium.org
, Aug 8 2017