Devserver should be skipped if check_health fails. |
|||||
Issue descriptiondaisy_skate CQ failed: https://uberchromegw.corp.google.com/i/chromeos/waterfall?builder=daisy_skate-paladin Debug log: https://pantheon.corp.google.com/storage/browser/chromeos-autotest-results/100519721-chromeos-test/chromeos4-row9-rack5-host9/debug """ 02/10 09:44:02.135 ERROR| dev_server:0427| Devserver call failed: "http://100.108.1.152:8082/check_health?", timeout: 2.0 seconds, Error: ... 500 Internal Server Error The server encountered an unexpected condition which prevented it from fulfilling the request. Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/cherrypy/_cprequest.py", line 656, in respond response.body = self.handler() File "/usr/lib/python2.7/dist-packages/cherrypy/lib/encoding.py", line 188, in __call__ self.body = self.oldhandler(*args, **kwargs) File "/usr/lib/python2.7/dist-packages/cherrypy/_cpdispatch.py", line 34, in __call__ return self.callable(*self.args, **self.kwargs) File "/usr/local/google/home/chromeos-test/chromiumos/src/platform/dev/devserver.py", line 1510, in check_health apache_client_count = self._get_process_count('apache') File "/usr/local/google/home/chromeos-test/chromiumos/src/platform/dev/devserver.py", line 1489, in _get_process_count 'pgrep -fc "%s"' % process_cmd_pattern, shell=True)) File "/usr/lib/python2.7/subprocess.py", line 566, in check_output process = Popen(stdout=PIPE, *popenargs, **kwargs) File "/usr/lib/python2.7/subprocess.py", line 710, in __init__ errread, errwrite) File "/usr/lib/python2.7/subprocess.py", line 1215, in _execute_child errpipe_read, errpipe_write = self.pipe_cloexec() File "/usr/lib/python2.7/subprocess.py", line 1167, in pipe_cloexec r, w = os.pipe() OSError: [Errno 24] Too many open files """
,
Feb 10 2017
Retooling this into a fixit bug. Essentially if we hit an error when checking the health of the devserver, we should go on to the next one.
,
Jun 21 2017
,
Jun 21 2017
hmm, actually we did do "when devserver health checking failed, go on to the next one", but the log doesn't clearly show that. Will fix this bug by adding more logs. Another problem is this is actually a crash_devserver check fail, which won't be the real reason for a CQ failure.
,
Jun 22 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/b273f6fab2b1fc34b60f091151dd1984acbfeb46 commit b273f6fab2b1fc34b60f091151dd1984acbfeb46 Author: xixuan <xixuan@chromium.org> Date: Thu Jun 22 10:34:48 2017 autotest: Clearly log the process of devserver health check. As crbug.com/691009 shown, we don't have clear logs for how to iterate devserver health check. BUG= chromium:691009 TEST=Ran unittest. Change-Id: I155bdaaecb6b7f7e48abf028e7945cc950165f4e Reviewed-on: https://chromium-review.googlesource.com/543755 Commit-Ready: Xixuan Wu <xixuan@chromium.org> Tested-by: Xixuan Wu <xixuan@chromium.org> Reviewed-by: Xixuan Wu <xixuan@chromium.org> [modify] https://crrev.com/b273f6fab2b1fc34b60f091151dd1984acbfeb46/client/common_lib/cros/dev_server.py
,
Jun 22 2017
,
Jan 22 2018
|
|||||
►
Sign in to add a comment |
|||||
Comment 1 by sbasi@chromium.org
, Feb 10 2017