chromeos2-row8-rack9-host14 unable to reach devserver |
|||||||
Issue descriptionProvision failed due to "client-autotest.tar.bz2 could not be fetched from any of the repos ['http://100.115.245.198:8082/static/kevin-paladin/R58-9280.0.0-rc2/autotest/packages']" https://pantheon.corp.google.com/storage/browser/chromeos-autotest-results/hosts/chromeos2-row8-rack9-host14/59966813-provision/debug/ It has been caused kevin paladin failed for three times.
,
Feb 13 2017
Summary of the bug: build failure: https://uberchromegw.corp.google.com/i/chromeos/builders/kevin-paladin/builds/198 The log of the failed provsion: https://pantheon.corp.google.com/storage/browser/chromeos-autotest-results/hosts/chromeos2-row8-rack9-host14/59966813-provision/debug/ It complains that client-autotest.tar.bz2 could not be fetched from any of the repos ['http://100.115.245.198:8082/static/kevin-paladin/R58-9280.0.0-rc2/autotest/packages' I manually download the autotest_package.tar from the image https://pantheon.corp.google.com/storage/browser/chromeos-image-archive/kevin-paladin/R58-9280.0.0-rc2/. The client-autotest.tar.bz2 is actually there
,
Feb 13 2017
Try to stage the artifact for the same build again? It could be some devserver issue. Also, check the timestamp of that client-autotest.tar.bz2 on GS, maybe there is a timing issue?
,
Feb 13 2017
Yeah, I think all three builds 198, 197, 196 were trying to stage for the same build and failed. The timestamp for the autotest_packages.tar is 2/13/17 5:41 AM. So, I don't think it is a timing issue that can affect all three builds.
,
Feb 14 2017
ping
,
Feb 14 2017
looking
,
Feb 14 2017
It's because host chromeos2-row8-rack9-host14.cros (100.115.231.55) fails to run 'wget --connect-timeout=15 --retry-connrefused --wait=5 -nv http://100.115.245.198:8082/static/kevin-paladin/R58-9280.0.0-rc2/autotest/packages/client-autotest.tar.bz2 -O /tmp/sysinfo/autoserv-4oNwST/packages/client-autotest.tar.bz2' command. I ssh to the host and manually run this cmd, still fail. Instead I tried to ssh chromeos2-row6-rack5-host17.cros (100.115.228.226), and successfully executes this cmd. Usually the host will find a devserver in its subnet and should be able to run wget. I think there's some network config/connection issue between host chromeos2-row8-rack9-host14.cros and devserver 100.115.245.198. So reassign to deputy to file a bug to engLab.
,
Feb 14 2017
,
Feb 14 2017
$ dut-status -f chromeos2-row8-rack9-host14
chromeos2-row8-rack9-host14
2017-02-14 09:25:39 -- http://cautotest/tko/retrieve_logs.cgi?job=/results/hosts/chromeos2-row8-rack9-host14/59983972-provision/
2017-02-14 08:18:32 OK http://cautotest/tko/retrieve_logs.cgi?job=/results/hosts/chromeos2-row8-rack9-host14/59983270-repair/
2017-02-14 06:45:58 -- http://cautotest/tko/retrieve_logs.cgi?job=/results/hosts/chromeos2-row8-rack9-host14/59982087-provision/
2017-02-14 05:33:17 OK http://cautotest/tko/retrieve_logs.cgi?job=/results/hosts/chromeos2-row8-rack9-host14/59981280-repair/
2017-02-14 04:01:15 -- http://cautotest/tko/retrieve_logs.cgi?job=/results/hosts/chromeos2-row8-rack9-host14/59979979-provision/
2017-02-14 02:42:58 OK http://cautotest/tko/retrieve_logs.cgi?job=/results/hosts/chromeos2-row8-rack9-host14/59979040-repair/
2017-02-14 01:08:35 -- http://cautotest/tko/retrieve_logs.cgi?job=/results/hosts/chromeos2-row8-rack9-host14/59978033-provision/
2017-02-13 23:56:02 OK http://cautotest/tko/retrieve_logs.cgi?job=/results/hosts/chromeos2-row8-rack9-host14/59977262-repair/
2017-02-13 22:25:42 -- http://cautotest/tko/retrieve_logs.cgi?job=/results/hosts/chromeos2-row8-rack9-host14/59976051-provision/
2017-02-13 21:02:33 OK http://cautotest/tko/retrieve_logs.cgi?job=/results/hosts/chromeos2-row8-rack9-host14/59974796-repair/
2017-02-13 19:28:09 -- http://cautotest/tko/retrieve_logs.cgi?job=/results/hosts/chromeos2-row8-rack9-host14/59973211-provision/
2017-02-13 18:13:48 OK http://cautotest/tko/retrieve_logs.cgi?job=/results/hosts/chromeos2-row8-rack9-host14/59971829-repair/
2017-02-13 16:42:27 -- http://cautotest/tko/retrieve_logs.cgi?job=/results/hosts/chromeos2-row8-rack9-host14/59970425-provision/
2017-02-13 15:15:27 OK http://cautotest/tko/retrieve_logs.cgi?job=/results/hosts/chromeos2-row8-rack9-host14/59969179-repair/
2017-02-13 13:43:08 -- http://cautotest/tko/retrieve_logs.cgi?job=/results/hosts/chromeos2-row8-rack9-host14/59968106-provision/
2017-02-13 13:06:39 OK http://cautotest/tko/retrieve_logs.cgi?job=/results/hosts/chromeos2-row8-rack9-host14/59967817-verify/
2017-02-13 12:31:24 OK http://cautotest/tko/retrieve_logs.cgi?job=/results/hosts/chromeos2-row8-rack9-host14/59967641-repair/
2017-02-13 10:58:09 -- http://cautotest/tko/retrieve_logs.cgi?job=/results/hosts/chromeos2-row8-rack9-host14/59966813-provision/
,
Feb 14 2017
Issue 692161 has been merged into this issue.
,
Feb 14 2017
This took down several paladin runs. I've locked the DUT in question. Raised to P0. We can't leave the CQ broken for extended periods like this.
,
Feb 14 2017
to haowei@ any idea if there's been some network reconfiguration that would cause this?
,
Feb 14 2017
,
Feb 21 2017
Question. How we name the path in the software? If this commend is correct. wget --connect-timeout=15 --retry-connrefused --wait=5 -nv http://100.115.245.198:8082/static/kevin-paladin/R58-9280.0.0-rc2/autotest/packages/client-autotest.tar.bz2 -O /tmp/sysinfo/autoserv-4oNwST/packages/client-autotest.tar.bz2' I don't see the path above in that server. I only can find the package of Kevin-paladin here. chromeos-test@chromeos2-devserver6:~/images/kevin-paladin$ pwd /home/chromeos-test/images/kevin-paladin And the build number is chromeos-test@chromeos2-devserver6:~/images/kevin-paladin$ ls R58-9304.0.0-rc3
,
Feb 21 2017
I tested with current build: wget --connect-timeout=15 --retry-connrefused --wait=5 -nv http://100.115.245.198:8082/static/kevin-paladin/R58-9304.0.0-rc3/autotest/packages/client-autotest.tar.bz2 -O /tmp/1.tar.bz2 Looks good to me
,
Feb 28 2017
Pri-0 bugs are critical regressions or serious emergencies, and this bug has not been updated in three days. Could you please provide an update, or adjust the priority to a more appropriate level if applicable? If a fix is in active development, please set the status to Started. Thanks for your time! To disable nags, add the Disable-Nags label. For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Mar 15 2017
Pri-0 bugs are critical regressions or serious emergencies, and this bug has not been updated in three days. Could you please provide an update, or adjust the priority to a more appropriate level if applicable? If a fix is in active development, please set the status to Started. Thanks for your time! To disable nags, add the Disable-Nags label. For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Apr 10 2017
I'm going to close this bug due to no action. Please reopen it if issue persists. |
|||||||
►
Sign in to add a comment |
|||||||
Comment 1 by shuqianz@chromium.org
, Feb 13 2017