New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 812015 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Feb 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Bug



Sign in to add a comment

Not enough Hana DUTs in CQ

Project Member Reported by pho...@chromium.org, Feb 13 2018

Issue description

Traceback (most recent call last):
  File "/usr/local/autotest/frontend/afe/json_rpc/serviceHandler.py", line 109, in dispatchRequest
    results['result'] = self.invokeServiceEndpoint(meth, args)
  File "/usr/local/autotest/frontend/afe/json_rpc/serviceHandler.py", line 147, in invokeServiceEndpoint
    return meth(*args)
  File "/usr/local/autotest/frontend/afe/rpc_handler.py", line 270, in new_fn
    return f(*args, **keyword_args)
  File "/usr/local/autotest/frontend/afe/rpc_utils.py", line 1167, in replacement
    return func(**kwargs)
  File "/usr/local/autotest/frontend/afe/rpc_interface.py", line 1032, in create_job
    require_ssp=require_ssp)
  File "/usr/local/autotest/frontend/afe/rpc_utils.py", line 826, in create_job_common
    metahost_objects=metahost_objects)
  File "/usr/local/autotest/frontend/afe/rpc_utils.py", line 515, in create_new_job
    check_job_metahost_dependencies(metahost_objects, dependencies)
  File "/usr/local/autotest/frontend/afe/rpc_utils.py", line 355, in check_job_metahost_dependencies
    % (metahost.name, ', '.join(job_dependencies)))
NoEligibleHostException: No hosts within board:hana satisfy arc, cts_abi_x86, sparse_coverage_2, cros-version:hana-paladin/R66-10400.0.0-rc3, board:hana, pool:cq.


 

Comment 1 by pho...@chromium.org, Feb 14 2018

Hm, balance_pools says there already are enough hana duts.

Comment 2 by pho...@chromium.org, Feb 17 2018

It happened again:

https://luci-milo.appspot.com/buildbot/chromeos/hana-paladin/2323
logs here: https://pantheon.corp.google.com/storage/browser/chromeos-autotest-results/177259044-chromeos-test/hostless/debug

NoEligibleHostException: No hosts within board:hana satisfy arc, cts_abi_x86, sparse_coverage_2, cros-version:hana-paladin/R66-10410.0.0-rc2, board:hana, pool:cq.

This will probably keep breaking the CQ.

Comment 3 by pho...@chromium.org, Feb 17 2018

Cc: -dgarr...@chromium.org
Owner: dgarr...@chromium.org
Status: Assigned (was: Untriaged)
Marking hana-paladin experimental. 

Comment 4 by pho...@chromium.org, Feb 17 2018

Also affecting stumpy: NoEligibleHostException: No hosts within board:stumpy satisfy hw_video_acc_vp9, cros-version:stumpy-paladin/R66-10410.0.0-rc2, board:stumpy, pool:cq.

Comment 5 by pho...@chromium.org, Feb 17 2018

#4 refers to the same CQ run as the hana build.

build: https://luci-milo.appspot.com/buildbot/chromeos/stumpy-paladin/31074
logs: https://pantheon.corp.google.com/storage/browser/chromeos-autotest-results/177255933-chromeos-test/hostless/debug

FTR, balance_pool thinks they have enough DUTs (although there are no spare stumpy DUTs. Will need to open a bug to repair them).

  20:28  (~) >> balance_pool cq hana
hana cq pool: Target of 7 is above minimum.
Transferring 0 DUTs from cq to suites.
Transferring 0 DUTs from suites to cq.

  20:29  (~) >> balance_pool cq stumpy
stumpy cq pool: Target of 8 is above minimum.

Balancing ['model:stumpy'] cq pool:
Total 8 DUTs, 6 working, 2 broken, 0 reserved.
Target is 8 working DUTs; grow pool by 2 DUTs.
['model:stumpy'] suites pool has 0 spares available for balancing pool cq
ERROR: Not enough spares: need 2, only have 0.
Transferring 0 DUTs from cq to suites.
Transferring 0 DUTs from suites to cq.

Comment 6 by pho...@chromium.org, Feb 17 2018

The Hana run had some "failed" tests, but they have a weird exception having to do with the attach.%d file from drone_manager:

suite: http://cautotest-prod/afe/#tab_id=view_job&object_id=177259374
logs: https://pantheon.corp.google.com/storage/browser/chromeos-autotest-results/177259374-chromeos-test/chromeos6-row3-rack3-host21/debug/

02/16 17:42:57.452 ERROR|          autoserv:0763| Uncaught Exception, exit_code = 1.
Traceback (most recent call last):
  File "/usr/local/autotest/server/autoserv", line 755, in main
    use_ssp)
  File "/usr/local/autotest/server/autoserv", line 471, in run_autoserv
    test_retry, **kwargs)
  File "/usr/local/autotest/server/server_job.py", line 379, in __init__
    control, raise_warnings=False)
  File "/usr/local/autotest/client/common_lib/control_data.py", line 394, in parse_control
    mod = compiler.parseFile(path)
  File "/usr/lib/python2.7/compiler/transformer.py", line 40, in parseFile
    f = open(path, "U")
IOError: [Errno 2] No such file or directory: '/usr/local/autotest/results/drone_tmp/attach.491'

Status: Started (was: Assigned)
Cc: nxia@chromium.org waihong@chromium.org norvez@chromium.org
Resolved? hana paladin is passing. It is still marked experimental in tree status.
Status: Fixed (was: Started)
Removed the experimental in the tree status.

Sign in to add a comment