New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 811943 link

Starred by 3 users

Issue metadata

Status: Fixed
Owner:
Closed: Feb 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 1
Type: ----



Sign in to add a comment

lxc_functional_test in VerifyContainerTask can't pass

Project Member Reported by xixuan@chromium.org, Feb 13 2018

Issue description

[cros-full-0026.mtv.corp.google.com] sudo: python /usr/local/autotest//site_utils/lxc/lxc_functional_test.py -v -d chromeos2-row1-rack2-host1 -r  chromeos2-devservertest
[cros-full-0026.mtv.corp.google.com] out: ERROR:root:ERROR:
[cros-full-0026.mtv.corp.google.com] out: Traceback (most recent call last):
[cros-full-0026.mtv.corp.google.com] out:   File "/usr/local/autotest//site_utils/lxc/lxc_functional_test.py", line 359, in <module>
[cros-full-0026.mtv.corp.google.com] out:     main(options)
[cros-full-0026.mtv.corp.google.com] out:   File "/usr/local/autotest//site_utils/lxc/lxc_functional_test.py", line 335, in main
[cros-full-0026.mtv.corp.google.com] out:     unittest_setup.verify_user()
[cros-full-0026.mtv.corp.google.com] out:   File "/usr/local/autotest/site_utils/lxc/unittest_setup.py", line 41, in verify_user
[cros-full-0026.mtv.corp.google.com] out:     raise EnvironmentError('Unittests should not be run as root.')
[cros-full-0026.mtv.corp.google.com] out: EnvironmentError: Unittests should not be run as root.
[cros-full-0026.mtv.corp.google.com] out: Traceback (most recent call last):
[cros-full-0026.mtv.corp.google.com] out:   File "/usr/local/autotest//site_utils/lxc/lxc_functional_test.py", line 359, in <module>
[cros-full-0026.mtv.corp.google.com] out:     main(options)
[cros-full-0026.mtv.corp.google.com] out:   File "/usr/local/autotest//site_utils/lxc/lxc_functional_test.py", line 335, in main
[cros-full-0026.mtv.corp.google.com] out:     unittest_setup.verify_user()
[cros-full-0026.mtv.corp.google.com] out:   File "/usr/local/autotest/site_utils/lxc/unittest_setup.py", line 41, in verify_user
[cros-full-0026.mtv.corp.google.com] out:     raise EnvironmentError('Unittests should not be run as root.')
[cros-full-0026.mtv.corp.google.com] out: EnvironmentError: Unittests should not be run as root.
[cros-full-0026.mtv.corp.google.com] out: INFO:root:Cleaning up temporary directory /usr/local/autotest/containers/container_test_rJFM0Y.
[cros-full-0026.mtv.corp.google.com] out: DEBUG:root:Running 'sudo lxc-ls -P /usr/local/autotest/containers/container_test_rJFM0Y -f -F name,state'
[cros-full-0026.mtv.corp.google.com] out: DEBUG:root:Running 'sudo rm -rf "/usr/local/autotest/containers/container_test_rJFM0Y"'
[cros-full-0026.mtv.corp.google.com] out: Traceback (most recent call last):
[cros-full-0026.mtv.corp.google.com] out:   File "/usr/local/autotest//site_utils/lxc/lxc_functional_test.py", line 369, in <module>
[cros-full-0026.mtv.corp.google.com] out:     lxc.ContainerBucket(TEMP_DIR).destroy_all()
[cros-full-0026.mtv.corp.google.com] out:   File "/usr/local/autotest/site_utils/lxc/container_bucket.py", line 73, in __init__
[cros-full-0026.mtv.corp.google.com] out:     raise e
[cros-full-0026.mtv.corp.google.com] out: autotest_lib.client.common_lib.error.ContainerError: No container found in directory /usr/local/autotest/containers/container_test_rJFM0Y with name of base_09.
[cros-full-0026.mtv.corp.google.com] out: 
 

Comment 1 by xixuan@chromium.org, Feb 13 2018

Change api.sudo() to api.run(), it runs for more time, but there's still error:

[cros-full-0026.mtv.corp.google.com] out: Traceback (most recent call last):
[cros-full-0026.mtv.corp.google.com] out:   File "/usr/local/autotest//site_utils/lxc/lxc_functional_test.py", line 369, in <module>
[cros-full-0026.mtv.corp.google.com] out:     lxc.ContainerBucket(TEMP_DIR).destroy_all()
[cros-full-0026.mtv.corp.google.com] out:   File "/usr/local/autotest/site_utils/lxc/container_bucket.py", line 139, in destroy_all
[cros-full-0026.mtv.corp.google.com] out:     containers = self.get_all().values()
[cros-full-0026.mtv.corp.google.com] out:   File "/usr/local/autotest/site_utils/lxc/container_bucket.py", line 100, in get_all
[cros-full-0026.mtv.corp.google.com] out:     if containers[info["name"]]:
[cros-full-0026.mtv.corp.google.com] out: KeyError: 'base_09'
[cros-full-0026.mtv.corp.google.com] out: 

Comment 2 by xixuan@chromium.org, Feb 13 2018

[cros-full-0026.mtv.corp.google.com] out: 2018-02-13 12:18:40,684.684 DEBUG|         container:0326|       MainThread(139710031771456)| Command <sudo lxc-attach -P /usr/local/autotest/containers/container_test_xNzUhQ -n test_123_1518553106_61864 -- bash -c "curl --head http://storage.googleapis.com/abci-ssp/autotest-containers/base_09.tar.xz"> failed, rc=6, Command returned non-zero exit status
[cros-full-0026.mtv.corp.google.com] out: * Command: 
[cros-full-0026.mtv.corp.google.com] out:     sudo lxc-attach -P /usr/local/autotest/containers/container_test_xNzUhQ
[cros-full-0026.mtv.corp.google.com] out:     -n test_123_1518553106_61864 -- bash -c "curl --head
[cros-full-0026.mtv.corp.google.com] out:     http://storage.googleapis.com/abci-ssp/autotest-containers/base_09.tar.xz"
[cros-full-0026.mtv.corp.google.com] out: Exit status: 6
[cros-full-0026.mtv.corp.google.com] out: Duration: 0.35431599617
[cros-full-0026.mtv.corp.google.com] out: 
[cros-full-0026.mtv.corp.google.com] out: stderr:
[cros-full-0026.mtv.corp.google.com] out:   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
[cros-full-0026.mtv.corp.google.com] out:                                  Dload  Upload   Total   Spent    Left  Speed
[cros-full-0026.mtv.corp.google.com] out: 
[cros-full-0026.mtv.corp.google.com] out:   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (6) Could not resolve host: storage.googleapis.com
Cc: dshi@chromium.org

Comment 4 by snanda@chromium.org, Feb 13 2018

Saw similar failure in this build:
https://luci-milo.appspot.com/buildbot/chromeos/cyan-paladin/5277

https://pantheon.corp.google.com/storage/browser/chromeos-autotest-results/176558562-chromeos-test/chromeos4-row11-rack11-host9/ssp_logs/debug/

stderr:
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (6) Could not resolve host: storage.googleapis.com
02/13 13:24:15.495 DEBUG|             utils:0214| Running 'sudo lxc-attach -P /usr/local/autotest/containers -n test_176558562_1518556718_143137 -- bash -c "curl --head http://storage.googleapis.com/abci-ssp/autotest-containers/base_09.tar.xz"'
02/13 13:24:15.773 DEBUG|         container:0326| Command <sudo lxc-attach -P /usr/local/autotest/containers -n test_176558562_1518556718_143137 -- bash -c "curl --head http://storage.googleapis.com/abci-ssp/autotest-containers/base_09.tar.xz"> failed, rc=6, Command returned non-zero exit status
* Command: 
    sudo lxc-attach -P /usr/local/autotest/containers -n
    test_176558562_1518556718_143137 -- bash -c "curl --head
    http://storage.googleapis.com/abci-ssp/autotest-containers/base_09.tar.xz"
Exit status: 6
Duration: 0.258385181427




Project Member

Comment 5 by bugdroid1@chromium.org, Feb 14 2018

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/chromeos-admin/+/9c9d04347fa4fe9227282198e7589fb842456c6d

commit 9c9d04347fa4fe9227282198e7589fb842456c6d
Author: Xixuan Wu <xixuan@chromium.org>
Date: Wed Feb 14 01:52:12 2018

Comment 6 by xixuan@chromium.org, Feb 14 2018

Failure in #4 is because the paladin cannot download the base_09 packages, which is not the same as this bug.

This bug is due to 'key missing' error here: http://shortn/_BFHwb6t7SM

Fix: https://chromium-review.googlesource.com/c/chromiumos/third_party/autotest/+/918063
Project Member

Comment 7 by bugdroid1@chromium.org, Feb 14 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/d63fb812554c233f43d1f629a4a8911474678234

commit d63fb812554c233f43d1f629a4a8911474678234
Author: Xixuan Wu <xixuan@chromium.org>
Date: Wed Feb 14 12:07:26 2018

autotest: fix lxc destroy bug.

BUG= chromium:811943 
TEST=Ran "lxc_functional_test.py -v -d chromeos2-row1-rack2-host1 -r  chromeos2-devservertest"
on new drone.

Change-Id: I07bd39bb6fe06b36f575c2b342008f007dd9bf50
Reviewed-on: https://chromium-review.googlesource.com/918063
Commit-Ready: Xixuan Wu <xixuan@chromium.org>
Tested-by: Xixuan Wu <xixuan@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>

[modify] https://crrev.com/d63fb812554c233f43d1f629a4a8911474678234/site_utils/lxc/container_bucket.py

Cc: mattmallett@chromium.org pprabhu@chromium.org
 Issue 812318  has been merged into this issue.

Comment 9 by xixuan@chromium.org, Feb 20 2018

Owner: xixuan@chromium.org
Status: Fixed (was: Untriaged)

Sign in to add a comment