New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 787936 link

Starred by 0 users

Issue metadata

Status: Duplicate
Merged: issue 788455
Owner:
Last visit > 30 days ago
Closed: Nov 2017
Cc:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 1
Type: Bug



Sign in to add a comment

dummy_PassServer.sanity failures on reef-chrome-pfq

Project Member Reported by achuith@chromium.org, Nov 22 2017

Issue description

Cc: xixuan@chromium.org
Owner: nxia@chromium.org
https://cs.corp.google.com/chromeos_public/src/third_party/autotest/files/server/site_tests/dummy_PassServer/dummy_PassServer.py

This seems to be a test to verify provisioning, and I believe this is a lab failure.
autoserv.DEBUG log lines that seem relevant:
11/21 22:21:41.874 DEBUG|             utils:0212| Running 'sudo test -e "/usr/local/autotest/containers/test_157781264_1511331553_25220/container_id.p"'
11/21 22:21:42.210 DEBUG|             utils:0212| Running 'sudo cat /usr/local/autotest/containers/test_157781264_1511331553_25220/container_id.p'
11/21 22:21:42.719 DEBUG|         container:0368| Destroying container /usr/local/autotest/containers/test_157781264_1511331553_25220
11/21 22:21:42.720 DEBUG|             utils:0212| Running 'sudo lxc-destroy -P /usr/local/autotest/containers -n test_157781264_1511331553_25220 -f'
11/21 22:21:47.514 INFO |        server_job:0213| FAIL	----	----	timestamp=1511331707	localtime=Nov 21 22:21:47	Failed to setup container for test: Command <sudo lxc-start -P /usr/local/autotest/containers -n test_157781264_1511331553_25220 -d> failed, rc=1, Command returned non-zero exit status
  * Command: 
      sudo lxc-start -P /usr/local/autotest/containers -n
      test_157781264_1511331553_25220 -d
  Exit status: 1
  Duration: 6.20587110519
  
  stderr:
  lxc-start: tools/lxc_start.c: main: 366 The container failed to start.
  lxc-start: tools/lxc_start.c: main: 368 To get more details, run the container in foreground mode.
  lxc-start: tools/lxc_start.c: main: 370 Additional information can be obtained by setting the --logfile and --logpriority options.. Check logs in ssp_logs folder for more details.


Full log at:
https://storage.cloud.google.com/chromeos-autotest-results/157781264-chromeos-test/chromeos6-row4-rack9-host9/ssp_logs/debug/autoserv.DEBUG
Cc: dshi@chromium.org kenobi@chromium.org ihf@chromium.org
Adding Ben, who has done some work on autotest containers:
https://chromium-review.googlesource.com/c/chromiumos/third_party/autotest/+/775675


Comment 5 by nxia@chromium.org, Nov 22 2017

Cc: -kenobi@chromium.org n...@chromiu.org
Owner: kenobi@chromium.org
kenobi@, can you please take a look at the ssp failure?
Status: Assigned (was: Untriaged)

Comment 7 by dshi@chromium.org, Nov 23 2017

chromeos-server119:/usr/local/autotest$ python site_utils/lxc/lxc_functional_test.py -v -s
sudo lxc-start -P /usr/local/autotest/containers/container_test_oAIXSZ -n test_123_1511395080_32245 -d --logfile /tmp/l1

$ cat /tmp/l1
      lxc-start 20171122235934.791 ERROR    lxc_conf - conf.c:instantiate_veth:2685 - failed to create veth pair (vethJVQLCR and vethM7DGGM): Operation not supported
      lxc-start 20171122235934.803 ERROR    lxc_conf - conf.c:lxc_create_network:3029 - failed to create netdev
      lxc-start 20171122235934.803 ERROR    lxc_start - start.c:lxc_spawn:1103 - Failed to create the network.
      lxc-start 20171122235934.803 ERROR    lxc_start - start.c:__lxc_start:1358 - Failed to spawn container "test_123_1511395080_32245".
      lxc-start 20171122235940.329 ERROR    lxc_start_ui - tools/lxc_start.c:main:366 - The container failed to start.
      lxc-start 20171122235940.330 ERROR    lxc_start_ui - tools/lxc_start.c:main:368 - To get more details, run the container in foreground mode.
      lxc-start 20171122235940.330 ERROR    lxc_start_ui - tools/lxc_start.c:main:370 - Additional infor

I tried to restart lxc and lxc-net, neither helps. We might have to reboot the server. The bottomline is that the server can't run any test with ssp.

Comment 8 by nxia@chromium.org, Nov 25 2017

the recent reef-paladin builds are failing on this, going to mark reef-paladin as experimental temporarily

https://uberchromegw.corp.google.com/i/chromeos/builders/reef-paladin/builds/4367

Comment 9 by nxia@chromium.org, Nov 25 2017

https://luci-milo.appspot.com/buildbot/chromeos/reef-paladin/4350

https://pantheon.corp.google.com/storage/browser/chromeos-autotest-results/157995457-chromeos-test/chromeos6-row3-rack12-host13/ssp_logs/debug/


11/22 19:51:02.363 DEBUG|             utils:0212| Running 'sudo lxc-destroy -P /usr/local/autotest/containers -n test_157995457_1511408948_1884 -f'
11/22 19:51:06.230 INFO |        server_job:0213| FAIL	----	----	timestamp=1511409066	localtime=Nov 22 19:51:06	Failed to setup container for test: Command <sudo lxc-start -P /usr/local/autotest/containers -n test_157995457_1511408948_1884 -d> failed, rc=1, Command returned non-zero exit status
  * Command: 
      sudo lxc-start -P /usr/local/autotest/containers -n
      test_157995457_1511408948_1884 -d
  Exit status: 1
  Duration: 6.34547901154
  
  stderr:
  lxc-start: tools/lxc_start.c: main: 366 The container failed to start.
  lxc-start: tools/lxc_start.c: main: 368 To get more details, run the container in foreground mode.
  lxc-start: tools/lxc_start.c: main: 370 Additional information can be obtained by setting the --logfile and --logpriority options.. Check logs in ssp_logs folder for more details.
Cc: jclinton@chromium.org
Cc: nsanders@chromium.org

Comment 12 by dshi@chromium.org, Nov 26 2017

Lab will be closed next Monday, why don't we restart the server? Assign
this to deputy please.
Mergedinto: 788455
Status: Duplicate (was: Assigned)

Sign in to add a comment