Several hosts (audioboxes in Linda Vista lab) are not running tests due to a Provisioning failure when querying the build from devserver |
|||||
Issue description
This issue seems to only be effecting audioboxes @ chromeos15
Full failure reason: staging artifacts=full_payload,stateful,autotest_packages files= for {BOARD}-release/R66-10445.0.0 failed;HTTP OK not accompanied by 'Success'.
From a Lars log:
02/28 03:50:36.440 DEBUG|provision_AutoUpda:0062| Start provisioning <remote host: chromeos15-audiobox8-host1> to lars-release/R66-10445.0.0.
02/28 03:50:36.440 INFO | dev_server:0798| Getting devservers for host: chromeos15-audiobox8-host1
02/28 03:50:36.501 DEBUG| dev_server:0819| The host chromeos15-audiobox8-host1 (100.115.127.194) is in a restricted subnet. Try to locate a devserver inside subnet 100.115.124.0:22.
02/28 03:50:36.502 DEBUG| dev_server:0770| Pick one healthy devserver from ['http://100.115.127.250:8082', 'http://100.115.127.248:8082', 'http://100.115.127.249:8082', 'http://100.115.127.246:8082', 'http://100.115.127.245:8082']
02/28 03:50:36.502 DEBUG| dev_server:0774| Check health for http://100.115.127.248:8082
02/28 03:50:36.503 DEBUG| utils:0214| Running 'ssh 100.115.127.248 'curl "http://100.115.127.248:8082/check_health?"''
02/28 03:50:37.174 DEBUG| dev_server:0779| Pick http://100.115.127.248:8082
02/28 03:50:37.174 INFO | dev_server:1244| Staging artifacts on devserver http://100.115.127.248:8082: build=lars-release/R66-10445.0.0, artifacts=['full_payload', 'stateful', 'autotest_packages'], files=, archive_url=gs://chromeos-image-archive/lars-release/R66-10445.0.0
02/28 03:50:37.175 DEBUG| utils:0214| Running 'ssh 100.115.127.248 'curl "http://100.115.127.248:8082/stage?artifacts=full_payload,stateful,autotest_packages&files=&async=True&archive_url=gs://chromeos-image-archive/lars-release/R66-10445.0.0"''
02/28 03:50:37.673 DEBUG| dev_server:1186| response for RPC: '<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"\n"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">\n<html>\n<head>\n <meta http-equiv="Content-Type" content="text/html; charset=utf-8"></meta>\n <title>500 Internal Server Error</title>\n <style type="text/css">\n #powered_by {\n margin-top: 20px;\n border-top: 2px solid black;\n font-style: italic;\n }\n\n #traceback {\n color: red;\n }\n </style>\n</head>\n <body>\n <h2>500 Internal Server Error</h2>\n <p>The server encountered an unexpected condition which prevented it from fulfilling the request.</p>\n <pre id="traceback">Traceback (most recent call last):\n File "/usr/lib/python2.7/dist-packages/cherrypy/_cprequest.py", line 656, in respond\n response.body = self.handler()\n File "/usr/lib/python2.7/dist-packages/cherrypy/lib/encoding.py", line 188, in __call__\n self.body = self.oldhandler(*args, **kwargs)\n File "/usr/lib/python2.7/dist-packages/cherrypy/_cpdispatch.py", line 34, in __call__\n return self.callable(*self.args, **self.kwargs)\n File "/home/chromeos-test/chromiumos/src/platform/dev/devserver.py", line 885, in stage\n dl.Download(factory, async=async)\n File "/home/chromeos-test/chromiumos/src/platform/dev/downloader.py", line 178, in Download\n common_util.MkDirP(self._build_dir)\n File "/home/chromeos-test/chromiumos/src/platform/dev/common_util.py", line 59, in MkDirP\n os.makedirs(directory)\n File "/usr/lib/python2.7/os.py", line 157, in makedirs\n mkdir(name, mode)\nOSError: [Errno 30] Read-only file system: \'/home/chromeos-test/images/lars-release/R66-10445.0.0\'\n</pre>\n <div id="powered_by">\n <span>Powered by <a href="http://www.cherrypy.org">CherryPy 3.2.2</a></span>\n </div>\n </body>\n</html>\n'
02/28 03:50:37.675 INFO | dev_server:1580| Requesting contents from devserver http://100.115.127.248:8082 for image lars-release/R66-10445.0.0
02/28 03:50:37.676 DEBUG| utils:0214| Running 'ssh 100.115.127.248 'curl "http://100.115.127.248:8082/list_image_dir?archive_url=gs://chromeos-image-archive/lars-release/R66-10445.0.0"''
02/28 03:50:38.167 INFO | dev_server:1585| gs://chromeos-image-archive/lars-release/R66-10445.0.0 has not been staged on this devserver.
02/28 03:50:38.167 DEBUG| test:0410| Test failed due to staging artifacts=full_payload,stateful,autotest_packages files= for lars-release/R66-10445.0.0 failed;HTTP OK not accompanied by 'Success'.. Exception log follows the after_iteration_hooks.
peach_pi
https://pantheon.corp.google.com/storage/browser/chromeos-autotest-results/179975702-chromeos-test
lars
https://pantheon.corp.google.com/storage/browser/chromeos-autotest-results/179974377-chromeos-test
link
https://pantheon.corp.google.com/storage/browser/chromeos-autotest-results/179930465-chromeos-test
nyan_big
https://pantheon.corp.google.com/storage/browser/chromeos-autotest-results/179920928-chromeos-test
peppy
https://pantheon.corp.google.com/storage/browser/chromeos-autotest-results/179935285-chromeos-test
,
Feb 28 2018
,
Mar 5 2018
Handing to deputy
,
Mar 5 2018
devserver was down because https://chrome-internal-review.googlesource.com/c/chromeos/chromeos-admin/+/579410 and it had been resolved by reverting the change crbug.com/817645
,
Mar 5 2018
All these boards are now running tests.
Only lars has some weird SpecialTaskActionException issue - File "/usr/local/autotest/client/common_lib/cros/dev_server.py", line 1199, in _call_and_wait
raise DevServerException(error_message)
TestFail: staging artifacts=full_payload,stateful,autotest_packages files= for link-release/R67-10461.0.0 failed;HTTP OK not accompanied by 'Success'.
host page: http://cros-full-0009.mtv.corp.google.com/afe/#tab_id=view_host&object_id=7814
,
Mar 5 2018
|
|||||
►
Sign in to add a comment |
|||||
Comment 1 by ka...@chromium.org
, Feb 28 2018Components: Infra>Client>ChromeOS
Owner: nxia@chromium.org