New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 768501 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner: ----
Closed: May 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Bug



Sign in to add a comment

ShardRemoveFromProductionMasterTask failed to remove shard with empty boards

Project Member Reported by nxia@chromium.org, Sep 25 2017

Issue description

Fatal error: run() received nonzero return code 1 while executing!

Requested: /usr/local/autotest//cli/atest shard remove_board -l  chromeos-skunk-1.mtv.corp.google.com
Executed: /bin/bash -l -c "/usr/local/autotest//cli/atest shard remove_board -l  chromeos-skunk-1.mtv.corp.google.com"

Aborting.
2017-09-25 12:59:49,314 ERRO| Traceback (most recent call last):
  File "/usr/local/google/home/nxia/chromiumos/chromeos-admin/venv/server_management_lib/tasks/task.py", line 66, in run
    self._run()
  File "/usr/local/google/home/nxia/chromiumos/chromeos-admin/venv/server_management_lib/tasks/atomic_shard.py", line 461, in _run
    self._remove_boards_from_shard()
  File "/usr/local/google/home/nxia/chromiumos/chromeos-admin/venv/server_management_lib/tasks/atomic_common.py", line 93, in decorated_func
    return func(self)
  File "/usr/local/google/home/nxia/chromiumos/chromeos-admin/venv/server_management_lib/tasks/atomic_shard.py", line 413, in _remove_boards_from_shard
    api.run(cmd)
  File "/usr/local/google/home/nxia/.cache/cros_venv/venv-2.7.6-5ac5c779eb68fedf8c205112bd9aced9/local/lib/python2.7/site-packages/fabric/network.py", line 677, in host_prompting_wrapper
    return func(*args, **kwargs)
  File "/usr/local/google/home/nxia/.cache/cros_venv/venv-2.7.6-5ac5c779eb68fedf8c205112bd9aced9/local/lib/python2.7/site-packages/fabric/operations.py", line 1088, in run
    shell_escape=shell_escape, capture_buffer_size=capture_buffer_size,
  File "/usr/local/google/home/nxia/.cache/cros_venv/venv-2.7.6-5ac5c779eb68fedf8c205112bd9aced9/local/lib/python2.7/site-packages/fabric/operations.py", line 952, in _run_command
    error(message=msg, stdout=out, stderr=err)
  File "/usr/local/google/home/nxia/.cache/cros_venv/venv-2.7.6-5ac5c779eb68fedf8c205112bd9aced9/local/lib/python2.7/site-packages/fabric/utils.py", line 358, in error
    return func(message)
  File "/usr/local/google/home/nxia/.cache/cros_venv/venv-2.7.6-5ac5c779eb68fedf8c205112bd9aced9/local/lib/python2.7/site-packages/fabric/utils.py", line 54, in abort
    raise env.abort_exception(msg)
FabricException: run() received nonzero return code 1 while executing!

Requested: /usr/local/autotest//cli/atest shard remove_board -l  chromeos-skunk-1.mtv.corp.google.com
Executed: /bin/bash -l -c "/usr/local/autotest//cli/atest shard remove_board -l  chromeos-skunk-1.mtv.corp.google.com"

2017-09-25 12:59:49,314 ERRO| run() received nonzero return code 1 while executing!

Requested: /usr/local/autotest//cli/atest shard remove_board -l  chromeos-skunk-1.mtv.corp.google.com
Executed: /bin/bash -l -c "/usr/local/autotest//cli/atest shard remove_board -l  chromeos-skunk-1.mtv.corp.google.com"
2017-09-25 12:59:49,315 INFO| Printing out task report.
{
  "sub_reports": [],
  "exception": "FabricException('run() received nonzero return code 1 while executing!\\n\\nRequested: /usr/local/autotest//cli/atest shard remove_board -l  chromeos-skunk-1.mtv.corp.google.com\\nExecuted: /bin/bash -l -c \"/usr/local/autotest//cli/atest shard remove_board -l  chromeos-skunk-1.mtv.corp.google.com\"',)",
  "is_successful": false,
  "description": "run() received nonzero return code 1 while executing!\n\nRequested: /usr/local/autotest//cli/atest shard remove_board -l  chromeos-skunk-1.mtv.corp.google.com\nExecuted: /bin/bash -l -c \"/usr/local/autotest//cli/atest shard remove_board -l  chromeos-skunk-1.mtv.corp.google.com\"",
  "arguments_used": {
    "board_labels": [
      ""
    ],
    "prod_master": "cautotest",
    "host_server": "chromeos-skunk-1.mtv.corp.google.com"
  },
  "task_name": "ShardRemoveFromProductionMasterTask"



I'll remove the shards from atest server and atest shard to avoid unnecessary alerts.

 
For the server mgmt failure, it will be fixed by https://chrome-internal-review.googlesource.com/c/chromeos/chromeos-admin/+/461652

Labels: -OS-Android
Status: Fixed (was: Untriaged)
I'm pretty sure we fixed this problem, right?

Sign in to add a comment