New issue
Advanced search Search tips

Issue 601142 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Apr 2016
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: ----



Sign in to add a comment

gae_ts_mon (Python): unhandled exception connecting to PubSub

Project Member Reported by sergeybe...@chromium.org, Apr 6 2016

Issue description

Exception in thread Thread-216:
Traceback (most recent call last):
  File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/threading.py", line 569, in __bootstrap_inner
    self.run()
  File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/threading.py", line 522, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/base/data/home/apps/s~chromium-swarm/1978-f84626f.391772511941781570/gae_ts_mon/config.py", line 95, in _flush_metrics
    interface.flush()
  File "/base/data/home/apps/s~chromium-swarm/1978-f84626f.391772511941781570/gae_ts_mon/common/interface.py", line 108, in flush
    state.global_monitor.send(proto)
  File "/base/data/home/apps/s~chromium-swarm/1978-f84626f.391772511941781570/gae_ts_mon/common/monitors.py", line 164, in send
    body=body).execute(num_retries=5)
  File "/base/data/home/apps/s~chromium-swarm/1978-f84626f.391772511941781570/gae_ts_mon/third_party/oauth2client/util.py", line 132, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/base/data/home/apps/s~chromium-swarm/1978-f84626f.391772511941781570/gae_ts_mon/third_party/googleapiclient/http.py", line 716, in execute
    body=self.body, headers=self.headers)
  File "/base/data/home/apps/s~chromium-swarm/1978-f84626f.391772511941781570/gae_ts_mon/third_party/oauth2client/util.py", line 132, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/base/data/home/apps/s~chromium-swarm/1978-f84626f.391772511941781570/gae_ts_mon/third_party/oauth2client/client.py", line 538, in new_request
    redirections, connection_type)
  File "/base/data/home/apps/s~chromium-swarm/1978-f84626f.391772511941781570/gae_ts_mon/httplib2_utils.py", line 198, in request
    uri, method, body, *args, **kwargs)
  File "/base/data/home/apps/s~chromium-swarm/1978-f84626f.391772511941781570/gae_ts_mon/third_party/httplib2/__init__.py", line 1551, in request
    (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
  File "/base/data/home/apps/s~chromium-swarm/1978-f84626f.391772511941781570/gae_ts_mon/third_party/httplib2/__init__.py", line 1293, in _request
    (response, content) = self._conn_request(conn, request_uri, method, body, headers)
  File "/base/data/home/apps/s~chromium-swarm/1978-f84626f.391772511941781570/gae_ts_mon/third_party/httplib2/__init__.py", line 1249, in _conn_request
    response = conn.getresponse()
  File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/gae_override/httplib.py", line 536, in getresponse
    'An error occured while connecting to the server: %s' % e)
error: An error occured while connecting to the server: Unable to fetch URL: https://pubsub.googleapis.com/v1/projects/chrome-infra-mon-pubsub/topics/monacq:publish?alt=json

https://chromium-swarm.appspot.com/restricted/ereporter2/request/5703c50100ff010251123a2d660001737e6368726f6d69756d2d737761726d0001313937382d6638343632366600010138

https://chromium-swarm.appspot.com/restricted/ereporter2/request/5703c4be00ff0e1400ffdba664b10001737e6368726f6d69756d2d737761726d0001313937382d6638343632366600010152

https://chromium-swarm.appspot.com/restricted/ereporter2/request/5703c48400ff0306d204258b5b0001737e6368726f6d69756d2d737761726d0001313937382d6638343632366600010129


 
gae_override/httplib.py:536 re-raises socket.error which is tracked and re-raised in httplib2_utils.py:202, and which needs to be caught in ts_mon/common/monitors.py:165, but currently isn't.

Fix: add all the remaining exceptions from httplib2_utils.py to monitors.py:165.
Status: Fixed (was: Assigned)

Comment 5 by benhenry@google.com, Apr 27 2016

Components: Infra>Monitoring
Labels: -Infra-Monitoring

Sign in to add a comment