Stainless link - https://stainless.corp.google.com/search?view=matrix&row=build&col=board&first_date=2018-01-17&last_date=2018-02-13&suite=%5Ewifi%5C_matfunc%24&status=GOOD&status=WARN&status=FAIL&status=ERROR&status=ABORT&status=ALERT&reason=hostapd+process+terminated&exclude_cts=true&exclude_not_run=true&exclude_non_release=false&exclude_au=true&exclude_acts=true&exclude_retried=true&exclude_non_production=true

I already have started working to update the stumpy testbed-ap image:
  crbug.com/809792 

Is this bug sort of a dup of 809792?

The netperf-2.7 tmpdir fix needed to land first though. I'm back to working on this though.

Re. if this a dup ... maybe? but this issue is only observed on couple routers. 

I tried to manually update chromeos15-row3-rack2-host1-router to R50-7849.0.0-b13 but it fails to restore stateful partition, complains about python command not found, so i tried dev_install --reinstall but that fails as well. 




$ cros flash ssh://chromeos15-row3-rack2-host1-router xbuddy://remote/trybot-stumpy-test-ap/R50-7849.0.0-b13
16:38:10: NOTICE: Preparing to update the remote device chromeos15-row3-rack2-host1-router
16:38:13: NOTICE: Using image trybot-stumpy-test-ap/R50-7849.0.0-b13/chromiumos_test_image.bin
16:38:31: WARNING: Cannot start devserver:
16:38:31: WARNING: Warning: Permanently added 'chromeos15-row3-rack2-host1-router,100.115.126.157' (RSA) to the list of known hosts.
bash: python: command not found


Cannot start devserver! The stateful partition may be corrupted: bash: python: command not found

Attempt to restore the stateful partition? (yes/No)? yes
16:39:06: WARNING: Restoring the stateful partition
16:41:56: WARNING: Cannot start devserver:
16:41:56: WARNING: Warning: Permanently added 'chromeos15-row3-rack2-host1-router,100.115.126.157' (RSA) to the list of known hosts.
Traceback (most recent call last):
  File "/mnt/stateful_partition/cros-flash/tmp.7CUwd2jWCY/src/devserver.py", line 59, in <module>
    import cherrypy
  File "/usr/local/lib64/python2.7/site-packages/cherrypy/__init__.py", line 62, in <module>
    from cherrypy._cpcompat import urljoin as _urljoin, urlencode as _urlencode
  File "/usr/local/lib64/python2.7/site-packages/cherrypy/_cpcompat.py", line 208, in <module>
    from http.cookies import SimpleCookie, CookieError
ImportError: No module named http.cookies

16:41:56: ERROR: Device update failed.
16:41:58: ERROR: cros flash failed before completing.
16:41:58: ERROR: ('Unable to restore stateful partition: %s', DevserverCannotStartError('ImportError: No module named http.cookies',))




..........................below is dev_install try on the stumpy - tried with different devservers.................


localhost ~ # dev_install --reinstall
To clean up, we will run:
  rm -rf /usr/local/
Any content you have stored in there will be deleted.
Remove all installed packages now? (y/N) y
Removed all installed packages.
Reinstalling emerge.
Starting installation of developer packages.
First, we download the necessary files.
Devserver URL set to:  http://build26-b2.chrome.corp.google.com:8080
Use it as the binhost? (y/N) N
Downloading https://commondatastorage.googleapis.com/chromeos-dev-installer/board/stumpy/canary-R35-5613.0.2014_03_10_1714/packages/sys-apps/sandbox-2.6-r1.tbz2
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:--  0:00:04 --:--:--     0
curl: (22) The requested URL returned error: 403 Forbidden        <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
ERROR(dev_install): Could not download package.
ERROR(dev_install): Command curl --fail -o /usr/local/portage/packages/sys-apps/sandbox-2.6-r1.tbz2 https://commondatastorage.googleapis.com/chromeos-dev-installer/board/stumpy/canary-R35-5613.0.2014_03_10_1714/packages/sys-apps/sandbox-2.6-r1.tbz2 failed.
localhost ~ # dev_install --reinstall
To clean up, we will run:
  rm -rf /usr/local/
Any content you have stored in there will be deleted.
Remove all installed packages now? (y/N) y
Removed all installed packages.
Reinstalling emerge.
Starting installation of developer packages.
First, we download the necessary files.
Devserver URL set to:  http://build26-b2.chrome.corp.google.com:8080
Use it as the binhost? (y/N) y
Downloading http://build26-b2.chrome.corp.google.com:8080/static/pkgroot/stumpy/packages/sys-apps/sandbox-2.6-r1.tbz2   
curl: (6) Couldn't resolve host 'build26-b2.chrome.corp.google.com'        <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
ERROR(dev_install): Could not download package.
ERROR(dev_install): Command curl --fail -o /usr/local/portage/packages/sys-apps/sandbox-2.6-r1.tbz2 http://build26-b2.chrome.corp.google.com:8080/static/pkgroot/stumpy/packages/sys-apps/sandbox-2.6-r1.tbz2 failed.

Sounds like bad routers, and it's not happening any more.