Recent failures were actually on tricky. Recent example:

https://uberchromegw.corp.google.com/i/chromeos.chrome/builders/tricky-tot-chrome-pfq-informational/builds/159

This has been pretty widespread across many boards: https://bugs.chromium.org/p/chromium/issues/list?can=2&q=Failed%20to%20install%20device%20image%20using%20payload%20at%20label%3Aautofiled


There seems to be two major flavors of this error:
-Failed to install device image using payload at ... SSH reports a generic error (255) which is probably a lab network failure
- Failed to install device image using payload at ... command execution error

+shuqianz so she can enlighten us about these failure messages.

Erroe 1:
Failed to install device image using payload at ... SSH reports a generic error (255) which is probably a lab network failure.

It implies that we fail to ssh into the dut before running update. It could be a network flake.

Error 2:
Failed to install device image using payload at ... command execution error
It implies that update_engine fails. It is mostly because we failed to talk to the devserver or the omaha server. Just the logs of https://uberchromegw.corp.google.com/i/chromeos.chrome/builders/tricky-tot-chrome-pfq-informational/builds/159. It failed to talk to the omaha server.
03/21 09:33:03.856 DEBUG|        base_utils:0268| [stdout] [0321/093303:ERROR:update_engine_client.cc(218)] Update failed, current operation is UPDATE_STATUS_IDLE, last error code is ErrorCode::kOmahaErrorInHTTPResponse(37)

I don't turn on the SSH switch when several SSH errors happens.

cros_host:0758| Autoupdate did not complete.
03/28 05:21:37.547 ERROR|provision_AutoUpda:0138| Failed to install device image using payload at http://100.107.160.1:8082/update/edgar-release/R50-7978.34.0 on chromeos4-row12-rack7-host5. SSH reports a generic error (255) which is probably a lab network failure

Seems like not a dev_server problem, list_image_dir of dev_server has successfully finished.

Similar provision errors seen on x86-zgb-paladin

https://uberchromegw.corp.google.com/i/chromeos/builders/x86-zgb-paladin/builds/6112