New issue
Advanced search Search tips

Issue 599287 link

Starred by 1 user

Issue metadata

Status: Available
Owner: ----
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 3
Type: Feature



Sign in to add a comment

Add a Chameleon Verifier

Reported by jrbarnette@chromium.org, Mar 30 2016

Issue description

Consider this recent repair failure:
    http://cautotest/tko/retrieve_logs.cgi?job=/results/hosts/chromeos2-row1-rack4-host1/53113715-repair/

The status.log is devoid of diagnosis:
    START   ----    repair  timestamp=1459352600    localtime=Mar 30 08:43:20
    END FAIL        ----    repair  timestamp=1459352741    localtime=Mar 30 08:45:41

Looking in the autoserv.DEBUG log file, you see that the reason is
that host creation failed because chameleon wasn't working.

We don't want that.  Either the status.log file should show a repair
failure along the lines of "chameleon failed", or repair should pass.
Given that chameleon DUTs are used for chameleon testing, failing verify
and repair is probably better.

Relevant excerpt from the autoserv.DEBUG file:
03/30 08:45:41.157 ERROR|            repair:0037| [Errno 110] Connection timed out
Traceback (most recent call last):
  File "/usr/local/autotest/server/control_segments/repair", line 20, in repair
    try_lab_servo=True)
  File "/usr/local/autotest/server/hosts/factory.py", line 224, in create_target_machine
    return create_host(machine, **kwargs)
  File "/usr/local/autotest/server/hosts/factory.py", line 160, in create_host
    host_instance = host_class(hostname, **args)
  File "/usr/local/autotest/server/hosts/base_classes.py", line 58, in __init__
    super(Host, self).__init__(*args, **dargs)
  File "/usr/local/autotest/client/common_lib/hosts/base_classes.py", line 70, in __init__
    self._initialize(*args, **dargs)
  File "/usr/local/autotest/server/hosts/cros_host.py", line 312, in _initialize
    dut=self.hostname, chameleon_args=chameleon_args)
  File "/usr/local/autotest/server/hosts/chameleon_host.py", line 217, in create_chameleon_host
    return ChameleonHost(chameleon_host=chameleon_hostname)
  File "/usr/local/autotest/server/hosts/base_classes.py", line 58, in __init__
    super(Host, self).__init__(*args, **dargs)
  File "/usr/local/autotest/client/common_lib/hosts/base_classes.py", line 70, in __init__
    self._initialize(*args, **dargs)
  File "/usr/local/autotest/server/hosts/chameleon_host.py", line 68, in _initialize
    self.hostname, chameleon_port)
  File "/usr/local/autotest/client/cros/chameleon/chameleon.py", line 51, in __init__
    hostname, port)
  File "/usr/local/autotest/client/cros/chameleon/chameleon.py", line 73, in _create_server_proxy
    raise ChameleonConnectionError(e)
ChameleonConnectionError: [Errno 110] Connection timed out
03/30 08:45:41.162 INFO |        server_job:0128| END FAIL	----	repair	timestamp=1459352741	localtime=Mar 30 08:45:41	

 
Summary: Dead chameleon breaks repair diagnosis (was: Dead chameleon causes repair failure)
Owner: jrbarnette@chromium.org
Summary: Chameleon verifier should use new flow (was: Dead chameleon breaks repair diagnosis)
@ Richard - are you able to make this change? 

We prefer "the status.log file should show a repair failure along the lines of "chameleon failed"," so that we don't silently lose them. 
Owner: ----
Summary: Add a Chameleon Verifier (was: Chameleon verifier should use new flow )
I think we need two changes to get to the desired function:
  * In CrosHost.initialize(), catch Chameleon creation failures,
    and initialize the chameleon instance variable with None,
    or something similar.
  * Add a verifier to cros_repair.py that checks for "there's
    a chameleon host, but the chameleon instance variable is
    None".

I can do this, but it's not currently high on my priority list.
Other people should be able to do it, too.


Comment 4 by autumn@chromium.org, Apr 11 2016

Owner: jrbarnette@chromium.org

Comment 5 by autumn@chromium.org, Apr 11 2016

Labels: Hotlist-Fixit

Comment 6 by benhenry@google.com, Apr 26 2016

Components: Infra>Client>ChromeOS
Labels: -Infra-ChromeOS
Project Member

Comment 7 by sheriffbot@chromium.org, Apr 27 2017

Labels: Hotlist-Recharge-Cold
Status: Untriaged (was: Available)
This issue has been available for more than 365 days, and should be re-evaluated. Please re-triage this issue.
The Hotlist-Recharge-Cold label is applied for tracking purposes, and should not be removed after re-triaging the issue.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
Labels: -Type-Bug Type-Feature
Owner: ----
Status: Available (was: Untriaged)
Let's call this a feature request, not a bug.

Deficiencies in creation/initialization of Chameleon still
sometimes cause problems, especially at deployment and repair
time.  So, this is still relevant.  It's also still something
I don't have time for, and not something that requires my
particular attention.

Components: -Infra>Client>ChromeOS Infra>Client>ChromeOS>Test

Sign in to add a comment