New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 722632 link

Starred by 1 user

Issue metadata

Status: Duplicate
Owner: ----
Closed: May 2017
Cc:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 1
Type: Bug

Blocking:
issue 718397



Sign in to add a comment

provision failure: The host has wrong cros-version label

Project Member Reported by afakhry@chromium.org, May 16 2017

Issue description

This failure is currently affecting tricky-tot-chrome-pfq-informational.

What does this failure mean? What can we do to fix it?

05/15 14:20:45.527 WARNI|         cros_host:1396| cros-version label "cros-version:tricky-tot-chrome-pfq-informational/R60-9555.0.0-b4417" does not match release version 9554.0.0. Removing the label.
05/15 14:20:46.530 DEBUG|             retry:0155| Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0)
05/15 14:20:46.530 INFO |    connectionpool:0188| Starting new HTTP connection (1): 172.25.66.62
05/15 14:20:46.582 DEBUG|    connectionpool:0362| "POST /cautotest/cros_version_label_mismatch HTTP/1.1" 201 125
05/15 14:20:46.584 INFO |              base:0063| POST http://172.25.66.62:9800/cautotest/cros_version_label_mismatch [status:201 request:0.054s]
05/15 14:20:46.584 DEBUG|              base:0065| > {"hostname": "chromeos4-row2-rack3-host6", "time_recorded": 1494883246.529442}
05/15 14:20:46.584 DEBUG|              base:0066| < {"_index":"cautotest_2016_09","_type":"cros_version_label_mismatch","_id":"AVwN_CHEpdQHjru4VfTN","_version":1,"created":true}
05/15 14:20:46.585 ERROR|              base:0073| No handlers could be found for logger "elasticsearch.trace"
05/15 14:20:46.585 ERROR|            repair:0332| Failed: Legacy host verification checks
Traceback (most recent call last):
  File "/usr/local/autotest/client/common_lib/hosts/repair.py", line 329, in _verify_host
    self.verify(host)
  File "/usr/local/autotest/server/hosts/repair.py", line 55, in verify
    host.verify_software()
  File "/usr/local/autotest/server/hosts/cros_host.py", line 1543, in verify_software
    self.verify_cros_version_label()
  File "/usr/local/autotest/server/hosts/cros_host.py", line 1403, in verify_cros_version_label
    raise error.AutoservError('The host has wrong cros-version label.')
AutoservError: The host has wrong cros-version label.
05/15 14:20:46.588 INFO |        server_job:0184| 	FAIL	----	verify.cros	timestamp=1494883246	localtime=May 15 14:20:46	The host has wrong cros-version label.
05/15 14:20:46.588 INFO |            repair:0105| Skipping this operation: All host verification checks pass
05/15 14:20:46.588 DEBUG|            repair:0106| The following dependencies failed:
05/15 14:20:46.588 DEBUG|            repair:0108|     Legacy host verification checks
05/15 14:20:46.589 ERROR|           control:0075| Provision failed due to Exception.
Traceback (most recent call last):
  File "/usr/local/autotest/results/hosts/chromeos4-row2-rack3-host6/1246347-provision/20171505141148/control.srv", line 52, in provision_machine
    host.verify()
  File "/usr/local/autotest/server/hosts/cros_host.py", line 1547, in verify
    self._repair_strategy.verify(self)
  File "/usr/local/autotest/client/common_lib/hosts/repair.py", line 685, in verify
    self._verify_root._verify_host(host, silent)
  File "/usr/local/autotest/client/common_lib/hosts/repair.py", line 326, in _verify_host
    self._verify_dependencies(host, silent)
  File "/usr/local/autotest/client/common_lib/hosts/repair.py", line 199, in _verify_dependencies
    self._verify_list(host, self._dependency_list, silent)
  File "/usr/local/autotest/client/common_lib/hosts/repair.py", line 188, in _verify_list
    raise AutoservVerifyDependencyError(self, failures)
AutoservVerifyDependencyError: The host has wrong cros-version label.
05/15 14:20:46.589 INFO |        server_job:0184| END FAIL	----	provision	timestamp=1494883246	localtime=May 15 14:20:46	
05/15 14:20:46.589 ERROR|        server_job:0809| Exception escaped control file, job aborting:
Traceback (most recent call last):
  File "/usr/local/autotest/server/server_job.py", line 801, in run
    self._execute_code(server_control_file, namespace)
  File "/usr/local/autotest/server/server_job.py", line 1301, in _execute_code
    execfile(code_file, namespace, namespace)
  File "/usr/local/autotest/results/hosts/chromeos4-row2-rack3-host6/1246347-provision/20171505141148/control.srv", line 109, in <module>
    job.parallel_simple(provision_machine, machines)
  File "/usr/local/autotest/server/server_job.py", line 625, in parallel_simple
    return_results=return_results)
  File "/usr/local/autotest/server/subcommand.py", line 93, in parallel_simple
    function(arg)
  File "/usr/local/autotest/results/hosts/chromeos4-row2-rack3-host6/1246347-provision/20171505141148/control.srv", line 100, in provision_machine
    raise Exception('')
Exception
 
Cc: davidri...@chromium.org
+sheriff

I've seen this provision failure on a number of spot-checked paladin runs.

My guess is there's some bug in ToT that is flakily preventing devices from booting, causing them to fail to update.
Mergedinto: 722599
Status: Duplicate (was: Untriaged)

Sign in to add a comment