I tried logging into it (aka chromeos15-infra-devserver3.cros) to inspect it, but failed permissions checks. Bumping to deputy.

Can you include a link to the specific suite job that failed to produce results? That would ease investigation.

This link gives us the info of all Not_Run:

https://stainless.corp.google.com/search?view=matrix&row=queued_date&col=board&first_date=2017-11-28&last_date=2018-01-24&suite=wifi_matfunc&status=NOT_RUN&exclude_cts=true&exclude_not_run=false&exclude_non_release=false&exclude_au=true&exclude_acts=true&exclude_retried=true&exclude_non_production=true

For some devices the error message is displayed as:
Provision failed: staging artifacts=full_payload,stateful,autotest_packages files= for caroline-release/R65-10292.0.0 failed;HTTP OK not accompanied by 'Success'.

link here:
https://stainless.corp.google.com/search?exclude_retried=true&first_date=2017-11-28&master_builder_name=&builder_name_number=&shard=&exclude_acts=true&builder_name=&master_builder_name_number=&owner=&retry=&exclude_cts=true&exclude_non_production=true&hostname=&board=%5Ecaroline%24&test=&exclude_not_run=false&build=&status=NOT_RUN&reason=&waterfall=&suite=wifi_matfunc&last_date=2018-01-24&exclude_non_release=false&exclude_au=true&model=&view=list&queued_date=%5E20180109%24

For other devices there is no log to check why they were not run or scheduled.

Can you please let me know where to check for scheduler details for specific logs and I can be of more help?

Re #2, One of the failed example job: https://chromeos-proxy.appspot.com/task?id=3b4c348ae7f78c10&refresh=10&show_raw=1

akeshet@ - hope link from xixuan@ in #4 helps. If not, are there instructions on how to look up specific suite jobs that failed?

I don't see any issues at my end. The devserver, or course, is listed and has been up for 50 days.

Hostname     : chromeos15-infra-devserver3.cros.corp.google.com
Status       : primary
Roles        : devserver
Attributes   : {u'restricted_subnet': u'100.115.124.0/22', u'port': u'8082'}
Date Created : 2017-11-01 15:26:18
Date Modified: 2017-11-01 15:26:18
Note         : None

chromeos-test@chromeos-15-infra-devserver3:~$ uptime
 13:11:20 up 50 days, 22:41,  2 users,  load average: 0.00, 0.00, 0.00


I can give it a quick reboot to see if that resolved. Please advise.

Joe

past deputy -> current deputy

I don't have permissions to view the example job in #4.

I believe this bug has been fixed: a recent successful suite: http://chromeos-server158.cbf.corp.google.com/afe/#tab_id=view_job&object_id=173404062

Re #10, I just add you to the administrator. (PS I don't know why adding the mdb group doesn't work...)

Yes this seems to be fixed.

Was it something the infra folks did to fix this or rebooting the devsever (in #7) did the trick?

No changes on the infra side.

#7 did the trick. I checked, the devserver service is just restarted at that time.

Thanks for the update xixuan@

closing this bug.