New issue
Advanced search Search tips

Issue 796767 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Jan 2018
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Bug

Blocking:
issue 794630



Sign in to add a comment

sml: first run of puppet fails in ensure_devserver_config

Project Member Reported by pprabhu@chromium.org, Dec 21 2017

Issue description

ensure_devserver_config references chromeos-admin/puppet/cache/...
but that directory is (as the name suggests) a cache, isn't checked in, and doesn't exist when provisioning a server.

 SCHEDULER max_parse_processes]/Ini_setting[shadow_config /usr/local/autotest/shadow_config.ini SCHEDULER max_parse_processes]/ensure: created
[chromeos-server118.mtv.corp.google.com] out: Notice: /Stage[main]/Autotest::Shadow_config/Exec[append devserver config]/returns: /root/chromeos-admin/puppet/bin/gem_env: line 7: /root/chromeos-admin/pupp
et/cache/gem/bin/hiera: No such file or directory
[chromeos-server118.mtv.corp.google.com] out: Notice: /Stage[main]/Autotest::Shadow_config/Exec[append devserver config]/returns: Traceback (most recent call last):
[chromeos-server118.mtv.corp.google.com] out: Notice: /Stage[main]/Autotest::Shadow_config/Exec[append devserver config]/returns:   File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
[chromeos-server118.mtv.corp.google.com] out: Notice: /Stage[main]/Autotest::Shadow_config/Exec[append devserver config]/returns:     "__main__", fname, loader, pkg_name)
[chromeos-server118.mtv.corp.google.com] out: Notice: /Stage[main]/Autotest::Shadow_config/Exec[append devserver config]/returns:   File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
[chromeos-server118.mtv.corp.google.com] out: Notice: /Stage[main]/Autotest::Shadow_config/Exec[append devserver config]/returns:     exec code in run_globals
[chromeos-server118.mtv.corp.google.com] out: Notice: /Stage[main]/Autotest::Shadow_config/Exec[append devserver config]/returns:   File "/root/chromeos-admin/venv/admin/scripts/ensure_devserver_config.py
", line 117, in <module>
[chromeos-server118.mtv.corp.google.com] out: Notice: /Stage[main]/Autotest::Shadow_config/Exec[append devserver config]/returns:     sys.exit(main(sys.argv))
[chromeos-server118.mtv.corp.google.com] out: Notice: /Stage[main]/Autotest::Shadow_config/Exec[append devserver config]/returns:   File "/root/chromeos-admin/venv/admin/scripts/ensure_devserver_config.py
", line 34, in main
[chromeos-server118.mtv.corp.google.com] out: Notice: /Stage[main]/Autotest::Shadow_config/Exec[append devserver config]/returns:     servers = serverdb.get_servers_from_master_db()
[chromeos-server118.mtv.corp.google.com] out: Notice: /Stage[main]/Autotest::Shadow_config/Exec[append devserver config]/returns:   File "/root/chromeos-admin/venv/admin/serverdb.py", line 61, in get_serv
ers_from_master_db
[chromeos-server118.mtv.corp.google.com] out: Notice: /Stage[main]/Autotest::Shadow_config/Exec[append devserver config]/returns:     with contextlib.closing(mysqldb.connect(**_lookup_master_db_config()))
 \
[chromeos-server118.mtv.corp.google.com] out: Notice: /Stage[main]/Autotest::Shadow_config/Exec[append devserver config]/returns:   File "/root/chromeos-admin/venv/admin/serverdb.py", line 79, in _lookup_
master_db_config
[chromeos-server118.mtv.corp.google.com] out: Notice: /Stage[main]/Autotest::Shadow_config/Exec[append devserver config]/returns:     host=hiera.lookup('site_data::db::cautotest::host'),
[chromeos-server118.mtv.corp.google.com] out: Notice: /Stage[main]/Autotest::Shadow_config/Exec[append devserver config]/returns:   File "/root/chromeos-admin/venv/admin/hiera.py", line 22, in lookup
[chromeos-server118.mtv.corp.google.com] out: Notice: /Stage[main]/Autotest::Shadow_config/Exec[append devserver config]/returns:     return subprocess.check_output([_HIERA_PATH, key]).rstrip()
[chromeos-server118.mtv.corp.google.com] out: Notice: /Stage[main]/Autotest::Shadow_config/Exec[append devserver config]/returns:   File "/usr/lib/python2.7/subprocess.py", line 573, in check_output
[chromeos-server118.mtv.corp.google.com] out: Notice: /Stage[main]/Autotest::Shadow_config/Exec[append devserver config]/returns:     raise CalledProcessError(retcode, cmd, output=output)
[chromeos-server118.mtv.corp.google.com] out: Notice: /Stage[main]/Autotest::Shadow_config/Exec[append devserver config]/returns: subprocess.CalledProcessError: Command '['/root/chromeos-admin/puppet/bin/
hiera', 'site_data::db::cautotest::host']' returned non-zero exit status 1
[chromeos-server118.mtv.corp.google.com] out: Error: /bin/bash -c '/root/chromeos-admin/bin/ensure_devserver_config --environment prod /usr/local/autotest/shadow_config.ini' returned 1 instead of one of [
0]
[chromeos-server118.mtv.corp.google.com] out: Error: /Stage[main]/Autotest::Shadow_config/Exec[append devserver config]/returns: change from notrun to 0 failed: /bin/bash -c '/root/chromeos-admin/bin/ensu
re_devserver_config --environment prod /usr/local/autotest/shadow_config.ini' returned 1 instead of one of [0]
 
Blocking: 794630
Actual error:

chromeos-test@chromeos-server118:~$ sudo su
root@chromeos-server118:/usr/local/google/home/chromeos-test# root/chromeos-admin/bin/ensu
bash: root/chromeos-admin/bin/ensu: No such file or directory
root@chromeos-server118:/usr/local/google/home/chromeos-test# re_devserver_config --environment prod /usr/local/autotest/shadow_config.ini^C
root@chromeos-server118:/usr/local/google/home/chromeos-test# /root/chromeos-admin/bin/ensure_devserver_config --environment prod /usr/local/autotest/shadow_config.ini
/root/chromeos-admin/puppet/bin/gem_env: line 7: /root/chromeos-admin/puppet/cache/gem/bin/hiera: No such file or directory
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/root/chromeos-admin/venv/admin/scripts/ensure_devserver_config.py", line 117, in <module>
    sys.exit(main(sys.argv))
  File "/root/chromeos-admin/venv/admin/scripts/ensure_devserver_config.py", line 34, in main
    servers = serverdb.get_servers_from_master_db()
  File "/root/chromeos-admin/venv/admin/serverdb.py", line 61, in get_servers_from_master_db
    with contextlib.closing(mysqldb.connect(**_lookup_master_db_config())) \
  File "/root/chromeos-admin/venv/admin/serverdb.py", line 79, in _lookup_master_db_config
    host=hiera.lookup('site_data::db::cautotest::host'),
  File "/root/chromeos-admin/venv/admin/hiera.py", line 22, in lookup
    return subprocess.check_output([_HIERA_PATH, key]).rstrip()
  File "/usr/lib/python2.7/subprocess.py", line 573, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command '['/root/chromeos-admin/puppet/bin/hiera', 'site_data::db::cautotest::host']' returned non-zero exit status 1
Workaround from ayatane@:

Run 'make bundle_install'

It didn't like one of the options so I removed it (so now we have some unnecessary documentation installed on chromeos-server118, meh)

root@chromeos-server118:~/chromeos-admin/puppet# git diff
diff --git a/puppet/Makefile b/puppet/Makefile
index 2ab8c17c..fc0b30c0 100644
--- a/puppet/Makefile
+++ b/puppet/Makefile
@@ -5,7 +5,7 @@ export GEM_PATH := $(GEM_HOME)
 export BUNDLE_SILENCE_ROOT_WARNING := true

 GEM := gem
-GEM_INSTALL_FLAGS := --no-document
+#GEM_INSTALL_FLAGS := --no-document

 BUNDLE := $(GEM_HOME)/bin/bundle
 RAKE := $(GEM_HOME)/bin/rake

But that fixed the immediate problem.
Status: Started (was: Untriaged)
https://chrome-internal-review.googlesource.com/c/chromeos/chromeos-admin/+/534145

#3, make bundle_install GEM_INSTALL_FLAGS=

or

make bundle_install 'GEM_INSTALL_FLAGS=--no-rdoc --no-ri'
Project Member

Comment 6 by bugdroid1@chromium.org, Dec 21 2017

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/chromeos-admin/+/4307d462a96a73cf1e6fc6bb78435737a9371aa3

commit 4307d462a96a73cf1e6fc6bb78435737a9371aa3
Author: Allen Li <ayatane@chromium.org>
Date: Thu Dec 21 02:30:12 2017

Status: Fixed (was: Started)

Sign in to add a comment