New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 767099 link

Starred by 1 user

Issue metadata

Status: Archived
Owner:
Last visit > 30 days ago
Closed: Nov 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug

Blocking:
issue 754036



Sign in to add a comment

services on skunk-3 failed to start

Project Member Reported by nxia@chromium.org, Sep 20 2017

Issue description

I successfully provisioned skunk-3, and then added it to production. services on skunk-3 failed to start because of the following errors and kept sending alerts.

10:31:43 ERROR| Uncaught exception, terminating monitor_db.
Traceback (most recent call last):
  File "/usr/local/autotest/scheduler/monitor_db.py", line 164, in main_without_exception_handling
    initialize()
  File "/usr/local/autotest/scheduler/monitor_db.py", line 222, in initialize
    _db_manager = scheduler_lib.ConnectionManager()
  File "/usr/local/autotest/server/site_utils.py", line 87, in __call__
    *args, **kwargs)
  File "/usr/local/autotest/scheduler/scheduler_lib.py", line 75, in __init__
    setup_django_environment.enable_autocommit()
  File "/usr/local/autotest/frontend/setup_django_environment.py", line 21, in enable_autocommit
    _enable_autocommit_by_name('default')
  File "/usr/local/autotest/frontend/setup_django_environment.py", line 14, in _enable_autocommit_by_name
    connections[name].cursor()
  File "/usr/local/autotest/site-packages/django/db/backends/__init__.py", line 326, in cursor
    cursor = util.CursorWrapper(self._cursor(), self)
  File "/usr/local/autotest/site-packages/django/db/backends/mysql/base.py", line 405, in _cursor
    self.connection = Database.connect(**kwargs)
  File "/usr/local/autotest/site-packages/MySQLdb/__init__.py", line 81, in Connect
    return Connection(*args, **kwargs)
  File "/usr/local/autotest/site-packages/MySQLdb/connections.py", line 187, in __init__
    super(Connection, self).__init__(*args, **kwargs2)
OperationalError: (1045, "Access denied for user 'chromeosqa-admin'@'localhost' (using password: YES)")




 

Comment 1 by nxia@chromium.org, Sep 20 2017

Cc: pprabhu@chromium.org ayatane@chromium.org
ayantane@/pprabhu@, is it possible this is caused by puppet ? 
This is a new shard, right?

This looks like the shard was not able to connect to its local database.
I'd 
- verify that shadow config is sane: The AFE DB is local.
- Using the username/password in there, connect to the db manually. This should fail.
  - This could mean that setup_dev_autotest actually failed midway while provisioning the shard, and your database isn't setup correctly.

Comment 3 by nxia@chromium.org, Sep 20 2017

Cc: shuqianz@chromium.org
Thanks, pprabhu@.

shuqianz@, how can I verify if the db was setup correctly? can I setup it again? 

Comment 4 by nxia@chromium.org, Nov 2 2017

Status: Archived (was: Untriaged)

Comment 5 by nxia@chromium.org, Mar 12 2018

Blocking: 754036

Sign in to add a comment