New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 671250 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner:
Last visit > 30 days ago
Closed: Sep 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug



Sign in to add a comment

shard_client sends spam e-mail from out-of-production shard

Reported by jrbarnette@chromium.org, Dec 5 2016

Issue description

This is related to bug 670000.  Server chromeos-server59.hot
was configured as a shard, but then removed from production.
The system was still trying to run shard_client.  The result
was a failure that spammed the system with e-mail like this:

Subject: monitor_db exception
chromeos-server59.hot.corp.google.com / 15689 / 09:40:27 12/05/16
EXCEPTION: Uncaught exception. Terminating shard_client.
Traceback (most recent call last):
 File "/usr/local/autotest/scheduler/shard/shard_client.py", line 386, in main
   main_without_exception_handling()
 File "/usr/local/autotest/scheduler/shard/shard_client.py", line
411, in main_without_exception_handling
   _heartbeat_client = get_shard_client()
 File "/usr/local/autotest/scheduler/shard/shard_client.py", line
378, in get_shard_client
   shard_hostname = _get_shard_hostname_and_ensure_running_on_shard()
 File "/usr/local/autotest/scheduler/shard/shard_client.py", line
359, in _get_shard_hostname_and_ensure_running_on_shard
   'To run the shard client, shard_hostname must neither be None nor '
HeartbeatOnlyAllowedInShardModeException: To run the shard client,
shard_hostname must neither be None nor empty.

The spam caused the sending account to be shut off, causing other
mail (routine and urgent) to be shut off with it.

 
Owner: shuqianz@chromium.org
Status: Assigned (was: Available)
Project Member

Comment 2 by bugdroid1@chromium.org, Dec 6 2016

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/chromeos-admin/+/0bb2650134d6fa50e9c37270b84ef4d1832a294f

commit 0bb2650134d6fa50e9c37270b84ef4d1832a294f
Author: Shuqian Zhao <shuqianz@chromium.org>
Date: Mon Dec 05 21:15:03 2016

Project Member

Comment 3 by bugdroid1@chromium.org, Jan 5 2017

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/chromeos-admin/+/e048a83ad098186c8fb9ecf08d558b7a8a2f1fd5

commit e048a83ad098186c8fb9ecf08d558b7a8a2f1fd5
Author: Shuqian Zhao <shuqianz@chromium.org>
Date: Tue Dec 06 03:15:53 2016

Cc: haoweiw@chromium.org englab-sys-cros@google.com
Labels: -Pri-1 Pri-2
Status: WontFix (was: Assigned)
Thanks to the monarch monitoring project, this doesn't apply any more

Sign in to add a comment