Can't provision DB servers. |
|||
Issue descriptionThis command: bin/run_server_task DatabaseProvisionTask --host_server cros-bighd-0001.mtv.corp.google.com Fails with this puppet error: [cros-bighd-0001.mtv.corp.google.com] sudo: HOME=/root/ FACTER_server_type="autotest-db" /root/chromeos-admin/puppet/sync_and_run_puppet -f [cros-bighd-0001.mtv.corp.google.com] out: HEAD is now at 500052d Fix virtualenv bug when VIRTUALENV_ALWAYS_COPY is set [cros-bighd-0001.mtv.corp.google.com] out: puppet/run_puppet: line 26: GEM: readonly variable [cros-bighd-0001.mtv.corp.google.com] out: /root/chromeos-admin//puppet/bin/ensure_gem_deps: GEM=gem [cros-bighd-0001.mtv.corp.google.com] out: /root/chromeos-admin//puppet/bin/ensure_gem_deps: GEM_HOME not set; installing for user [cros-bighd-0001.mtv.corp.google.com] out: Error: Could not find data item classes in any Hiera data file and no default supplied at /root/chromeos-admin/puppet/manifests/site.pp:3 on node cros-bighd-0001.mtv.corp.google.com [cros-bighd-0001.mtv.corp.google.com] out: Error: Could not find data item classes in any Hiera data file and no default supplied at /root/chromeos-admin/puppet/manifests/site.pp:3 on node cros-bighd-0001.mtv.corp.google.com [cros-bighd-0001.mtv.corp.google.com] out: Fatal error: sudo() received nonzero return code 1 while executing! Requested: HOME=/root/ FACTER_server_type="autotest-db" /root/chromeos-admin/puppet/sync_and_run_puppet -f Executed: sudo -S -p 'sudo password:' /bin/bash -l -c "HOME=/root/ FACTER_server_type=\"autotest-db\" /root/chromeos-admin/puppet/sync_and_run_puppet -f" Aborting. 2018-02-13 12:42:32,812 ERRO| Traceback (most recent call last): File "/usr/local/google/home/dgarrett/sand/chromeos/chromeos-admin/venv/server_management_lib/tasks/task.py", line 66, in run self._run() File "/usr/local/google/home/dgarrett/sand/chromeos/chromeos-admin/venv/server_management_lib/tasks/atomic_common.py", line 203, in _run self._deploy_puppet() File "/usr/local/google/home/dgarrett/sand/chromeos/chromeos-admin/venv/server_management_lib/tasks/atomic_common.py", line 94, in decorated_func return func(self) File "/usr/local/google/home/dgarrett/sand/chromeos/chromeos-admin/venv/server_management_lib/tasks/atomic_common.py", line 178, in _deploy_puppet fabfile.deploy_puppet(server_type=self.server_type) File "/usr/local/google/home/dgarrett/sand/chromeos/chromeos-admin/venv/server_management_lib/fabfile.py", line 177, in deploy_puppet _run_puppet(server_type=server_type) File "/usr/local/google/home/dgarrett/sand/chromeos/chromeos-admin/venv/server_management_lib/fabfile.py", line 211, in _run_puppet sudo('HOME=/root/ FACTER_server_type="%s" %s' % (server_type, puppet_cmd)) File "/usr/local/google/home/dgarrett/.cache/cros_venv/venv-2.7.13-95de6b4f9b30bb6fc148ee4eccd758dc/local/lib/python2.7/site-packages/fabric/network.py", line 677, in host_prompting_wrapper return func(*args, **kwargs) File "/usr/local/google/home/dgarrett/.cache/cros_venv/venv-2.7.13-95de6b4f9b30bb6fc148ee4eccd758dc/local/lib/python2.7/site-packages/fabric/operations.py", line 1146, in sudo capture_buffer_size=capture_buffer_size, File "/usr/local/google/home/dgarrett/.cache/cros_venv/venv-2.7.13-95de6b4f9b30bb6fc148ee4eccd758dc/local/lib/python2.7/site-packages/fabric/operations.py", line 952, in _run_command error(message=msg, stdout=out, stderr=err) File "/usr/local/google/home/dgarrett/.cache/cros_venv/venv-2.7.13-95de6b4f9b30bb6fc148ee4eccd758dc/local/lib/python2.7/site-packages/fabric/utils.py", line 358, in error return func(message) File "/usr/local/google/home/dgarrett/.cache/cros_venv/venv-2.7.13-95de6b4f9b30bb6fc148ee4eccd758dc/local/lib/python2.7/site-packages/fabric/utils.py", line 54, in abort raise env.abort_exception(msg) FabricException: sudo() received nonzero return code 1 while executing! Requested: HOME=/root/ FACTER_server_type="autotest-db" /root/chromeos-admin/puppet/sync_and_run_puppet -f Executed: sudo -S -p 'sudo password:' /bin/bash -l -c "HOME=/root/ FACTER_server_type=\"autotest-db\" /root/chromeos-admin/puppet/sync_and_run_puppet -f" 2018-02-13 12:42:32,812 ERRO| sudo() received nonzero return code 1 while executing! Requested: HOME=/root/ FACTER_server_type="autotest-db" /root/chromeos-admin/puppet/sync_and_run_puppet -f Executed: sudo -S -p 'sudo password:' /bin/bash -l -c "HOME=/root/ FACTER_server_type=\"autotest-db\" /root/chromeos-admin/puppet/sync_and_run_puppet -f" 2018-02-13 12:42:32,813 INFO| Printing out task report. { "sub_reports": [], "exception": "FabricException('sudo() received nonzero return code 1 while executing!\\n\\nRequested: HOME=/root/ FACTER_server_type=\"autotest-db\" /root/chromeos-admin/puppet/sync_and_run_puppet -f\\nExecuted: sudo -S -p \\'sudo password:\\' /bin/bash -l -c \"HOME=/root/ FACTER_server_type=\\\\\"autotest-db\\\\\" /root/chromeos-admin/puppet/sync_and_run_puppet -f\"',)", "is_successful": false, "description": "sudo() received nonzero return code 1 while executing!\n\nRequested: HOME=/root/ FACTER_server_type=\"autotest-db\" /root/chromeos-admin/puppet/sync_and_run_puppet -f\nExecuted: sudo -S -p 'sudo password:' /bin/bash -l -c \"HOME=/root/ FACTER_server_type=\\\"autotest-db\\\" /root/chromeos-admin/puppet/sync_and_run_puppet -f\"", "arguments_used": { "host_server": "cros-bighd-0001.mtv.corp.google.com" }, "task_name": "DatabaseProvisionTask" }
,
Feb 13 2018
As Allen pointed out: admin/venv/server_management_lib/tasks/atomic_database.py has a hardcoded role of "autotest-db", which isn't valid. I'm working around this by locally hard coding to "autotest-db-slave-secondary" which matches the role listed in puppet. Then leaving this bug around, since pulling the real role from autotest seems long and complicated.
,
Feb 13 2018
,
Feb 13 2018
The following revision refers to this bug: https://chrome-internal.googlesource.com/chromeos/chromeos-admin/+/c2ca631ecf71e95d73bfdd0cd1a661e36d0bd00e commit c2ca631ecf71e95d73bfdd0cd1a661e36d0bd00e Author: Shuqian Zhao <shuqianz@chromium.org> Date: Tue Feb 13 22:10:23 2018
,
Feb 13 2018
|
|||
►
Sign in to add a comment |
|||
Comment 1 by dgarr...@chromium.org
, Feb 13 2018