New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 794008 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Dec 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Bug



Sign in to add a comment

staging tests: scheduler, host_scheduler failing to connect to local mysql

Project Member Reported by pprabhu@chromium.org, Dec 12 2017

Issue description

Traceback (most recent call last):
  File "/usr/local/autotest/scheduler/monitor_db.py", line 182, in main_without_exception_handling
    initialize()
  File "/usr/local/autotest/scheduler/monitor_db.py", line 260, in initialize
    _db_manager = scheduler_lib.ConnectionManager()
  File "/usr/local/autotest/server/site_utils.py", line 87, in __call__
    *args, **kwargs)
  File "/usr/local/autotest/scheduler/scheduler_lib.py", line 75, in __init__
    setup_django_environment.enable_autocommit()
  File "/usr/local/autotest/frontend/setup_django_environment.py", line 22, in enable_autocommit
    _enable_autocommit_by_name('global')
  File "/usr/local/autotest/frontend/setup_django_environment.py", line 14, in _enable_autocommit_by_name
    connections[name].cursor()
  File "/usr/local/autotest/site-packages/django/db/backends/__init__.py", line 326, in cursor
    cursor = util.CursorWrapper(self._cursor(), self)
  File "/usr/local/autotest/site-packages/django/db/backends/mysql/base.py", line 405, in _cursor
    self.connection = Database.connect(**kwargs)
  File "/usr/local/autotest/site-packages/MySQLdb/__init__.py", line 81, in Connect
    return Connection(*args, **kwargs)
  File "/usr/local/autotest/site-packages/MySQLdb/connections.py", line 187, in __init__
    super(Connection, self).__init__(*args, **kwargs2)
OperationalError: (1045, "Access denied for user 'cros-infra-admin'@'localhost' (using password: YES)")

I suspect we somehow deployed the wrong mysql credentials to the staging master.

+allen who recently changed this.
 
Owner: pprabhu@chromium.org
Status: Started (was: Untriaged)
Bug was introduced here: https://chrome-internal-review.googlesource.com/c/chromeos/chromeos-admin/+/520144/7/puppet/modules/autotest/templates/shadow_config/push_master.ini.erb#b10

Before, global_db_user and global_db_password were unset. Now, they fallback to the value in common.yaml, which is wrong.
Project Member

Comment 2 by bugdroid1@chromium.org, Dec 12 2017

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/chromeos-admin/+/a900c02b1d65b78665d8322a30a41f8c5c49f4a0

commit a900c02b1d65b78665d8322a30a41f8c5c49f4a0
Author: Prathmesh Prabhu <pprabhu@chromium.org>
Date: Tue Dec 12 01:52:05 2017

Status: Fixed (was: Started)
Cc: dgarr...@chromium.org jrbarnette@chromium.org
+deputies FYI. 
I've kicked a manual test_push to see how far it gets. Will rinse and repeat until test_push passes.

Sign in to add a comment