The last successful grunt-release build was R71-11124.0.0 @ 2018-10-03 11:13. Since then every build has failed in HWTest:
https://cros-goldeneye.corp.google.com/chromeos/legoland/builderHistory?buildConfig=grunt-release
The most recent run (R71-11138.0.0 [0]) failed with "HWTest [sanity] [grunt] / TestLabFailure / [Test-Logs]: Suite job: ABORT". Note, however, that HWTest [sanity] [careena] passed.
[0] https://cros-goldeneye.corp.google.com/chromeos/healthmonitoring/buildDetails?buildbucketId=8933241305994312032
AFAICT, it looks like stdout shows that the test suite was scheduled but just never ran and eventually timed out after 90 minutes.
Will return from run_suite with status: OK
06:23:20: INFO: Re-run swarming_cmd to avoid buildbot salency check.
06:23:20: INFO: RunCommand: /b/swarming/w/ir/cache/cbuild/repository/chromite/third_party/swarming.client/swarming.py run --swarming chromeos-proxy.appspot.com --task-summary-json /b/swarming/w/ir/tmp/t/cbuildbot-tmpQeOQ6E/tmp3x13aX/temp_summary.json --print-status-updates --timeout 14400 --raw-cmd --task-name grunt-release/R71-11138.0.0-sanity --dimension os Ubuntu-14.04 --dimension pool default --io-timeout 14400 --hard-timeout 14400 --expiration 1200 '--tags=priority:Build' '--tags=suite:sanity' '--tags=build:grunt-release/R71-11138.0.0' '--tags=task_name:grunt-release/R71-11138.0.0-sanity' '--tags=board:grunt' -- /usr/local/autotest/site_utils/run_suite.py --build grunt-release/R71-11138.0.0 --board grunt --model grunt --suite_name sanity --pool bvt --file_bugs True --priority Build --timeout_mins 180 --retry True --max_retries 5 --minimum_duts 1 --suite_min_duts 1 --offload_failures_only False --job_keyvals "{'cidb_build_stage_id': 95065195L, 'cidb_build_id': 3016594, 'datastore_parent_key': ('Build', 3016594, 'BuildStage', 95065195L)}" -m 246463015
07:55:50: INFO: RetriableHttp: attempt 1 receiving status 503, will retry
08:00:21: INFO: Re-run swarming_cmd to avoid buildbot salency check.
08:00:21: INFO: RunCommand: /b/swarming/w/ir/cache/cbuild/repository/chromite/third_party/swarming.client/swarming.py run --swarming chromeos-proxy.appspot.com --task-summary-json /b/swarming/w/ir/tmp/t/cbuildbot-tmpQeOQ6E/tmp3x13aX/temp_summary.json --print-status-updates --timeout 14400 --raw-cmd --task-name grunt-release/R71-11138.0.0-sanity --dimension os Ubuntu-14.04 --dimension pool default --io-timeout 14400 --hard-timeout 14400 --expiration 1200 '--tags=priority:Build' '--tags=suite:sanity' '--tags=build:grunt-release/R71-11138.0.0' '--tags=task_name:grunt-release/R71-11138.0.0-sanity' '--tags=board:grunt' -- /usr/local/autotest/site_utils/run_suite.py --build grunt-release/R71-11138.0.0 --board grunt --model grunt --suite_name sanity --pool bvt --file_bugs True --priority Build --timeout_mins 180 --retry True --max_retries 5 --minimum_duts 1 --suite_min_duts 1 --offload_failures_only False --job_keyvals "{'cidb_build_stage_id': 95065195L, 'cidb_build_id': 3016594, 'datastore_parent_key': ('Build', 3016594, 'BuildStage', 95065195L)}" -m 246463015
08:25:46: INFO: OAuth token TTL expired, auto-refreshing (attempt 1/2)
08:25:46: INFO: Refreshing access_token
08:32:35: INFO: OAuth token TTL expired, auto-refreshing (attempt 1/2)
08:32:35: INFO: Refreshing access_token
08:38:59: INFO: OAuth token TTL expired, auto-refreshing (attempt 1/2)
08:38:59: INFO: Refreshing access_token
08:57:54: INFO: RetriableHttp: attempt 1 receiving status 503, will retry
09:03:42: INFO: RetriableHttp: attempt 1 receiving status 503, will retry
[1;33m09:24:24: WARNING: Exception is not retriable return code: 3; command: /b/swarming/w/ir/cache/cbuild/repository/chromite/third_party/swarming.client/swarming.py run --swarming chromeos-proxy.appspot.com --task-summary-json /b/swarming/w/ir/tmp/t/cbuildbot-tmpQeOQ6E/tmp3x13aX/temp_summary.json --print-status-updates --timeout 14400 --raw-cmd --task-name grunt-release/R71-11138.0.0-sanity --dimension os Ubuntu-14.04 --dimension pool default --io-timeout 14400 --hard-timeout 14400 --expiration 1200 '--tags=priority:Build' '--tags=suite:sanity' '--tags=build:grunt-release/R71-11138.0.0' '--tags=task_name:grunt-release/R71-11138.0.0-sanity' '--tags=board:grunt' -- /usr/local/autotest/site_utils/run_suite.py --build grunt-release/R71-11138.0.0 --board grunt --model grunt --suite_name sanity --pool bvt --file_bugs True --priority Build --timeout_mins 180 --retry True --max_retries 5 --minimum_duts 1 --suite_min_duts 1 --offload_failures_only False --job_keyvals "{'cidb_build_stage_id': 95065195L, 'cidb_build_id': 3016594, 'datastore_parent_key': ('Build', 3016594, 'BuildStage', 95065195L)}" -m 246463015
Triggered task: grunt-release/R71-11138.0.0-sanity
Comment 1 by djkurtz@google.com
, Oct 8