New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 783449 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Last visit > 30 days ago
Closed: Nov 2017
Cc:
EstimatedDays: ----
NextAction: ----
OS: Linux
Pri: 1
Type: Bug

Blocking:
issue 720219



Sign in to add a comment

unittest flake in autotest_lib.site_utils.lxc.container_pool.client_unittest.ClientTests.testConnection

Project Member Reported by pprabhu@chromium.org, Nov 9 2017

Issue description

Killed a CQ run

https://luci-milo.appspot.com/buildbot/chromeos/gale-paladin/4451

autotest-0.0.2-r8821: FAIL: testConnection (autotest_lib.site_utils.lxc.container_pool.client_unittest.ClientTests)
autotest-0.0.2-r8821: Tests a basic client connection.
autotest-0.0.2-r8821: ----------------------------------------------------------------------
autotest-0.0.2-r8821: Traceback (most recent call last):
autotest-0.0.2-r8821:   File "/build/gale/tmp/portage/chromeos-base/autotest-0.0.2-r8821/work/autotest-0.0.2/site_utils/lxc/container_pool/client_unittest.py", line 64, in testConnection
autotest-0.0.2-r8821:     self.assertIsNotNone(host)
autotest-0.0.2-r8821: AssertionError: unexpectedly None
autotest-0.0.2-r8821: 
autotest-0.0.2-r8821: ----------------------------------------------------------------------
autotest-0.0.2-r8821: Ran 3 tests in 1.018s
autotest-0.0.2-r8821: 
autotest-0.0.2-r8821: FAILED (failures=1)
autotest-0.0.2-r8821: Traceback (most recent call last):
autotest-0.0.2-r8821:   File "./utils/unittest_suite.py", line 294, in <module>
autotest-0.0.2-r8821:     main()
autotest-0.0.2-r8821:   File "./utils/unittest_suite.py", line 281, in main
autotest-0.0.2-r8821:     errors = find_and_run_tests(absolute_start, options)
autotest-0.0.2-r8821:   File "./utils/unittest_suite.py", line 264, in find_and_run_tests
autotest-0.0.2-r8821:     pe.run_until_completion()
autotest-0.0.2-r8821:   File "/build/gale/tmp/portage/chromeos-base/autotest-0.0.2-r8821/work/autotest-0.0.2/utils/parallel.py", line 67, in run_until_completion
autotest-0.0.2-r8821:     self._run(self.ready_to_run.pop())
autotest-0.0.2-r8821:   File "/build/gale/tmp/portage/chromeos-base/autotest-0.0.2-r8821/work/autotest-0.0.2/utils/parallel.py", line 53, in _run
autotest-0.0.2-r8821:     function()
autotest-0.0.2-r8821:   File "./utils/unittest_suite.py", line 254, in <lambda>
autotest-0.0.2-r8821:     run_module = lambda module=module_names: run_test(module, options)
autotest-0.0.2-r8821:   File "./utils/unittest_suite.py", line 159, in run_test
autotest-0.0.2-r8821:     raise TestFailure(msg)
autotest-0.0.2-r8821: __main__.TestFailure: autotest_lib.site_utils.lxc.container_pool.client_unittest had 1 failures and 0 errors.
 
Blocking: 720219
Also in shard_dir_unittest 

https://logs.chromium.org/v/?s=chromeos%2Fbb%2Fchromeos%2Fchell-paladin%2F3114%2F%2B%2Frecipes%2Fsteps%2FUnitTest%2F0%2Fstdout

I think we are being too optimistic in trying to run unit tests that create and destroy containers. Are these really unit tests?

Comment 3 by kenobi@google.com, Nov 10 2017

Apologies for the flakes.

This test isn't creating and destroying containers, it's a test of the code that manages the socket connections that will enable clients to communicate with the container pool service.

This test was accidentally assigned a timeout of 0 instead of TIMEOUT for establishing the client connection, so there is a bit of a race against the setup code starting the other end of the connection for test.  Should be an easy fix.
Status: Assigned (was: Unconfirmed)
Project Member

Comment 7 by bugdroid1@chromium.org, Nov 10 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/82ed447d5d1fcea7376721596a2d0ea65ede48a4

commit 82ed447d5d1fcea7376721596a2d0ea65ede48a4
Author: Ben Kwa <kenobi@google.com>
Date: Fri Nov 10 20:23:33 2017

[autotest] Fix flakey ClientTest

A test was accidentally assigned a timeout of 0 instead of the test timeout,
which is resulting in flakey behaviour when it races against the setup code.

BUG= chromium:783449 
TEST=Run test repeatedly on a system with simulated heavy load.  Observe flake
rates >1/100 before the fix; after the fix, 500 runs with no failures.

Change-Id: I486228cc6cd71048241dbc8a3b2d19944c7e7723
Reviewed-on: https://chromium-review.googlesource.com/763919
Trybot-Ready: Ben Kwa <kenobi@chromium.org>
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
Tested-by: Ben Kwa <kenobi@chromium.org>
Commit-Queue: Ben Kwa <kenobi@chromium.org>

[modify] https://crrev.com/82ed447d5d1fcea7376721596a2d0ea65ede48a4/site_utils/lxc/container_pool/client_unittest.py

Comment 8 by kenobi@chromium.org, Nov 10 2017

Status: Fixed (was: Assigned)

Sign in to add a comment