I took a look at chromeos2-row3-rack4-host5.  I found
that the trouble starts when the test kills rsyslogd.
The symptoms include:
  * rsyslogd can't restart.
  * The DUT answers to ping, but not to ssh.

System logs from the DUT are attached.  The job history
(with log references) is below.

chromeos2-row3-rack4-host5
    2016-05-13 10:40:01  OK http://cautotest/tko/retrieve_logs.cgi?job=/results/hosts/chromeos2-row3-rack4-host5/55302644-repair/
    2016-05-13 07:07:18  NO http://cautotest/tko/retrieve_logs.cgi?job=/results/hosts/chromeos2-row3-rack4-host5/55295821-repair/
    2016-05-13 06:58:13  -- http://cautotest/tko/retrieve_logs.cgi?job=/results/hosts/chromeos2-row3-rack4-host5/55295453-verify/
    2016-05-13 06:32:18  NO http://cautotest/tko/retrieve_logs.cgi?job=/results/hosts/chromeos2-row3-rack4-host5/55294550-repair/
    2016-05-13 06:23:18  -- http://cautotest/tko/retrieve_logs.cgi?job=/results/hosts/chromeos2-row3-rack4-host5/55294261-verify/
    2016-05-13 05:56:04  NO http://cautotest/tko/retrieve_logs.cgi?job=/results/hosts/chromeos2-row3-rack4-host5/55293372-repair/
    2016-05-13 05:47:23  -- http://cautotest/tko/retrieve_logs.cgi?job=/results/hosts/chromeos2-row3-rack4-host5/55293067-verify/
    2016-05-13 03:01:59  NO http://cautotest/tko/retrieve_logs.cgi?job=/results/hosts/chromeos2-row3-rack4-host5/55287591-repair/
    2016-05-13 02:53:15  -- http://cautotest/tko/retrieve_logs.cgi?job=/results/hosts/chromeos2-row3-rack4-host5/55287288-verify/
    2016-05-13 01:57:05  NO http://cautotest/tko/retrieve_logs.cgi?job=/results/hosts/chromeos2-row3-rack4-host5/55285191-repair/
    2016-05-13 01:48:53  -- http://cautotest/tko/retrieve_logs.cgi?job=/results/hosts/chromeos2-row3-rack4-host5/55284838-verify/
    2016-05-13 01:23:48  NO http://cautotest/tko/retrieve_logs.cgi?job=/results/hosts/chromeos2-row3-rack4-host5/55283943-repair/
    2016-05-13 01:14:18  -- http://cautotest/tko/retrieve_logs.cgi?job=/results/hosts/chromeos2-row3-rack4-host5/55283586-verify/
    2016-05-13 00:53:18  NO http://cautotest/tko/retrieve_logs.cgi?job=/results/hosts/chromeos2-row3-rack4-host5/55282759-repair/
    2016-05-13 00:44:16  -- http://cautotest/tko/retrieve_logs.cgi?job=/results/hosts/chromeos2-row3-rack4-host5/55282407-cleanup/
    2016-05-12 23:57:39  -- http://cautotest/tko/retrieve_logs.cgi?job=/results/63132246-chromeos-test/
    2016-05-12 23:53:49  OK http://cautotest/tko/retrieve_logs.cgi?job=/results/hosts/chromeos2-row3-rack4-host5/55280414-reset/

Deleted: chromeos2-row3-rack4-host5.tar.gz 1.4 MB

chromeos2-row3-rack4-host5.tar.gz 1.4 MB Download

Long term, this feels like a bug in rsyslogd.  Short term, we can
deal with the problem by removing rsyslogd from the list of processes
tested by logging_CrashServices.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/aa92076c4c4c99d38d19ac7c296618ec3c7d6b26

commit aa92076c4c4c99d38d19ac7c296618ec3c7d6b26
Author: Richard Barnette <jrbarnette@chromium.org>
Date: Fri May 13 19:31:33 2016

[autotest] Remove rsyslogd from testing in logging_CrashServices

The logging_CrashServices test has a list of processes that it kills
in order to test that dumps are properly gathered.  One of the
victims is rsyslogd.  Killing rsyslogd sometimes causes the DUT to
go offline.  This removes rsyslogd from the list of services to
kill, until the service can be made more robust.

BUG=chromium:611786
TEST=None

Change-Id: I0365f01733d3e80d81a91f020163cb5a9c053a76
Reviewed-on: https://chromium-review.googlesource.com/344634
Commit-Ready: Richard Barnette <jrbarnette@chromium.org>
Tested-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Shuqian Zhao <shuqianz@chromium.org>

[modify] https://crrev.com/aa92076c4c4c99d38d19ac7c296618ec3c7d6b26/client/site_tests/logging_CrashServices/logging_CrashServices.py