New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 654033 link

Starred by 1 user

Issue metadata

Status: Duplicate
Owner: ----
Closed: Oct 2016
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 0
Type: Bug



Sign in to add a comment

Stumpy paladin keeps failing to run HWTests

Project Member Reported by denniskempin@google.com, Oct 7 2016

Issue description

The stumpy paladin seems to be having issues. 
https://uberchromegw.corp.google.com/i/chromeos/builders/stumpy-paladin/builds/26353

It looks like there are no DUTs to run most of the tests, those will just be marked as N/A, which I think seems to be correct behavior. 


  video_ChromeRTCHWDecodeUsed                 TEST_NA: Test skipped, no DUTs with expected labels found
....

However there are a few tests that time out:


  audio_Microphone                          [ FAILED ]
  audio_Microphone                            ABORT: Timed out, did not run.
  graphics_GLMark2.bvt-cq                   [ FAILED ]
  graphics_GLMark2.bvt-cq                     ABORT: Timed out, did not run.
  network_DestinationVerification           [ FAILED ]
  network_DestinationVerification             ABORT: Timed out, did not run.

Now I have been trying to dig through the autotest suite: 

http://cautotest.corp.google.com/tko/compose_query.cgi?columns=test&rows=hostname&condition=tag~%2779732002-%25%27&title=Report

Which shows the same results. However the links to the test logs always point to the hostless folder, which does not contain any logs about what happened running those test cases: https://pantheon.corp.google.com/storage/browser/chromeos-autotest-results/79732002-chromeos-test/hostless/

So I am not sure where to go from here.

I'm keeping an eye on the current build which is running HWTests right now: http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=79767843

If this issue persists we really need to figure out what's happening here. 

 
Labels: -Pri-1 Pri-0
Cc: kevcheng@chromium.org
The suite is definitely timing out, still digging to figure out why though (from https://pantheon.corp.google.com/storage/browser/chromeos-autotest-results/79732002-chromeos-test/hostless/debug/autoserv.DEBUG)

10/07 05:44:20.974 DEBUG|     dynamic_suite:0554| Waiting on suite.
10/07 07:15:29.918 INFO |        server_job:0153| START	----	stumpy-paladin/R55-8872.0.0-rc2/bvt-cq/audio_Microphone	timestamp=1475849729	localtime=Oct 07 07:15:29	
10/07 07:15:29.919 INFO |        server_job:0153| 	ABORT	----	stumpy-paladin/R55-8872.0.0-rc2/bvt-cq/audio_Microphone	timestamp=1475849729	localtime=Oct 07 07:15:29	
10/07 07:15:29.919 INFO |        server_job:0153| END ABORT	----	stumpy-paladin/R55-8872.0.0-rc2/bvt-cq/audio_Microphone	timestamp=1475849729	localtime=Oct 07 07:15:29	
10/07 07:15:30.219 INFO |        server_job:0153| START	----	stumpy-paladin/R55-8872.0.0-rc2/bvt-cq/graphics_GLMark2.bvt-cq	timestamp=1475849730	localtime=Oct 07 07:15:30	
10/07 07:15:30.220 INFO |        server_job:0153| 	ABORT	----	stumpy-paladin/R55-8872.0.0-rc2/bvt-cq/graphics_GLMark2.bvt-cq	timestamp=1475849730	localtime=Oct 07 07:15:30	
10/07 07:15:30.221 INFO |        server_job:0153| END ABORT	----	stumpy-paladin/R55-8872.0.0-rc2/bvt-cq/graphics_GLMark2.bvt-cq	timestamp=1475849730	localtime=Oct 07 07:15:30	
10/07 07:15:30.528 INFO |        server_job:0153| START	----	stumpy-paladin/R55-8872.0.0-rc2/bvt-cq/network_DestinationVerification	timestamp=1475849730	localtime=Oct 07 07:15:30	
10/07 07:15:30.528 INFO |        server_job:0153| 	ABORT	----	stumpy-paladin/R55-8872.0.0-rc2/bvt-cq/network_DestinationVerification	timestamp=1475849730	localtime=Oct 07 07:15:30	
10/07 07:15:30.528 INFO |        server_job:0153| END ABORT	----	stumpy-paladin/R55-8872.0.0-rc2/bvt-cq/network_DestinationVerification	timestamp=1475849730	localtime=Oct 07 07:15:30
Cc: shuqianz@chromium.org
Seems to be a sharding issue, from Dan:

the test jobs all have shard_id null
but board:stumpy is sharded
so the shard didn't run the tests b/c they are not marked as sharded
the master didn't run the tests b/c board stumpy is sharded
some bug in shard or that board, i think

This is in regards to http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=79795940 (bvt-cq suite I kicked off).


Seems to be that the shard handling stumpys was in a bad state (see crbug.com/652545).  I've fixed that up and let's see if jobs are getting scheduled now.
sweet, duts are provisioning now, I'll cancel my suite and let's watch the next cq's bvt-cq run.

http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=79799212
Mergedinto: 652545
Status: Duplicate (was: Untriaged)

Sign in to add a comment