New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 730156 link

Starred by 1 user

Issue metadata

Status: Archived
Owner: ----
Closed: Mar 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug



Sign in to add a comment

"balance-pool --all-boards cq" gets 500 internal error.

Project Member Reported by dgarr...@chromium.org, Jun 6 2017

Issue description

dgarrett$balance-pool --all-boards cq 
Default max broken boards calculated to be 9 for cq pool
Traceback (most recent call last):
  File "/usr/local/bin/balance-pool", line 599, in <module>
    main(sys.argv)
  File "/usr/local/bin/balance-pool", line 576, in main
    if _too_many_broken_boards(inventory, pool, arguments):
  File "/usr/local/bin/balance-pool", line 446, in _too_many_broken_boards
    if counts.get_broken(pool) != 0]
  File "/usr/local/google/home/dgarrett/sand/clean/src/third_party/autotest/files/site_utils/lab_inventory.py", line 345, in get_broken
    return self._count_pool(_PoolCounts.get_broken, pool)
  File "/usr/local/google/home/dgarrett/sand/clean/src/third_party/autotest/files/site_utils/lab_inventory.py", line 292, in _count_pool
    return get_pool_count(self._pools[pool])
  File "/usr/local/google/home/dgarrett/sand/clean/src/third_party/autotest/files/site_utils/lab_inventory.py", line 211, in get_broken
    return len(self.get_broken_list())
  File "/usr/local/google/home/dgarrett/sand/clean/src/third_party/autotest/files/site_utils/lab_inventory.py", line 205, in get_broken_list
    if h.last_diagnosis()[0] == status_history.BROKEN]
  File "/usr/local/google/home/dgarrett/sand/clean/src/third_party/autotest/files/server/lib/status_history.py", line 657, in last_diagnosis
    self._init_status_task()
  File "/usr/local/google/home/dgarrett/sand/clean/src/third_party/autotest/files/server/lib/status_history.py", line 586, in _init_status_task
    self._afe, self._host.id, self.end_time)
  File "/usr/local/google/home/dgarrett/sand/clean/src/third_party/autotest/files/server/lib/status_history.py", line 281, in get_status_task
    task = afe.get_host_status_task(host_id, query_end)
  File "/usr/local/google/home/dgarrett/sand/clean/src/third_party/autotest/files/server/frontend.py", line 651, in get_host_status_task
    host_id=host_id, end_time=end_time)
  File "/usr/local/google/home/dgarrett/sand/clean/src/third_party/autotest/files/server/frontend.py", line 107, in run
    result = utils.strip_unicode(rpc_call(**dargs))
  File "/usr/local/google/home/dgarrett/sand/clean/src/third_party/autotest/files/frontend/afe/json_rpc/proxy.py", line 123, in __call__
    raise BuildException(resp['error'])
autotest_lib.frontend.afe.json_rpc.proxy.JSONRPCException: HTTPError: HTTP Error 500: Internal Server Error
Traceback (most recent call last):
  File "/usr/local/autotest/frontend/afe/json_rpc/serviceHandler.py", line 120, in dispatchRequest
    results['result'] = self.invokeServiceEndpoint(meth, args)
  File "/usr/local/autotest/frontend/afe/json_rpc/serviceHandler.py", line 160, in invokeServiceEndpoint
    return meth(*args)
  File "/usr/local/autotest/frontend/afe/rpc_handler.py", line 270, in new_fn
    return f(*args, **keyword_args)
  File "/usr/local/autotest/frontend/afe/rpc_interface.py", line 1406, in get_host_status_task
    host_id=host_id, end_time=end_time)
  File "/usr/local/autotest/server/frontend.py", line 107, in run
    result = utils.strip_unicode(rpc_call(**dargs))
  File "/usr/local/autotest/frontend/afe/json_rpc/proxy.py", line 114, in __call__
    respdata = urllib2.urlopen(request).read()
  File "/usr/lib/python2.7/urllib2.py", line 127, in urlopen
    return _opener.open(url, data, timeout)
  File "/usr/lib/python2.7/urllib2.py", line 410, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/urllib2.py", line 523, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/urllib2.py", line 448, in error
    return self._call_chain(*args)
  File "/usr/lib/python2.7/urllib2.py", line 382, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 531, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 500: Internal Server Error

 
If I run it for a specific board, it works. I suspect there is a single board which breaks, but don't know which one, and don't understand the code well enough to quickly add instrumentation.
Owner: ----
Status: Available (was: Untriaged)

Comment 3 by aut...@google.com, Jun 7 2017

Cc: jrbarnette@chromium.org
@jrbarnette - is there an easy way to get a list of all board names to run that way?
> @jrbarnette - is there an easy way to get a list of all board names to run that way?

I think what would be needed would be the ability for error handling
to report which board failed.  Probably, that's possible.  I'm not
especially familiar with this part of the code, so I don't have any
special insights into how to make it so, or how hard it would be.

Comment 5 by aut...@google.com, Jun 12 2017

Labels: Hotlist-Fixit
Status: Archived (was: Available)

Sign in to add a comment