New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 682962 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner: ----
Closed: Jan 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 2
Type: Bug



Sign in to add a comment

Puppet configs seem to be wrong for GCE builders on chromiumos waterfall.

Project Member Reported by dgarr...@chromium.org, Jan 20 2017

Issue description

We just added GCE builders to the chromiumos waterfall for the first time, and puppet doesn't seem to be configuring them correctly.

An example failure:

cbuildbot: Unhandled exception:
Traceback (most recent call last):
  File "/b/build/slave/arm-generic-paladin-master/build/chromite/bin/cbuildbot", line 168, in <module>
    DoMain()
  File "/b/build/slave/arm-generic-paladin-master/build/chromite/bin/cbuildbot", line 164, in DoMain
    commandline.ScriptWrapperMain(FindTarget)
  File "/b/build/slave/arm-generic-paladin-master/build/chromite/lib/commandline.py", line 834, in ScriptWrapperMain
    ret = target(argv[1:])
  File "/b/build/slave/arm-generic-paladin-master/build/chromite/scripts/cbuildbot.py", line 1317, in main
    logging.info('One stage exited early: %s', ex)
  File "/b/build/slave/arm-generic-paladin-master/build/chromite/scripts/cbuildbot.py", line 1279, in main
    stack.Add(_SetupConnections, options, build_config)
  File "/b/build/slave/arm-generic-paladin-master/build/chromite/lib/cros_build_lib.py", line 1247, in Add
    obj = functor(*args, **kwargs)
  File "/b/build/slave/arm-generic-paladin-master/build/chromite/scripts/cbuildbot.py", line 1047, in _SetupConnections
    db = cidb.CIDBConnectionFactory.GetCIDBConnectionForBuilder()
  File "/b/build/slave/arm-generic-paladin-master/build/chromite/lib/cidb.py", line 1598, in GetCIDBConnectionForBuilder
    return self.GetInstance()
  File "/b/build/slave/arm-generic-paladin-master/build/chromite/lib/factory.py", line 103, in GetInstance
    return self._types[self.setup_type]()
  File "/b/build/slave/arm-generic-paladin-master/build/chromite/lib/factory.py", line 119, in wrapper
    cached_value.append(function())
  File "/b/build/slave/arm-generic-paladin-master/build/chromite/lib/cidb.py", line 1490, in <lambda>
    lambda: CIDBConnection(constants.CIDB_PROD_BOT_CREDS)),
  File "/b/build/slave/arm-generic-paladin-master/build/chromite/lib/cidb.py", line 616, in __init__
    db_credentials_dir, *args, **kwargs)
  File "/b/build/slave/arm-generic-paladin-master/build/chromite/lib/cidb.py", line 242, in __init__
    listeners=[self._listener_class()])
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/__init__.py", line 332, in create_engine
    return strategy.create(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/strategies.py", line 64, in create
    dbapi = dialect_cls.dbapi(**dbapi_args)
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/connectors/mysqldb.py", line 57, in dbapi
    return __import__('MySQLdb')
ImportError: No module named MySQLdb
@@@STEP_CURSOR@cbuildbot [arm-generic-paladin]@@@
step returned non-zero exit code: 1
@@@STEP_FAILURE@@@
 
Cc: d...@chromium.org dshi@chromium.org friedman@chromium.org nxia@chromium.org
Owner: ----
Only one chromiumos builder is on the CQ. I'm taking it offline, which will cause it's builds to fall back to physical builders.

Other incremental and full builders will continue failing, but are more 'advisory'.
Actually, they slaves all seem to have recovered if I was patient enough.

Though the initial builds all failed.

Comment 3 by dshi@chromium.org, Jan 23 2017

Is this fixed?
Status: WontFix (was: Untriaged)
Yes. I didn't understand when/how the first puppet runs happened.

This is expected behavior, but should never last for more than about 30 minutes after a builder is brought up.

Sign in to add a comment