This looks like something that won't be fun to work on yet will be educational and tractable for me to do between unibuild stuff. Taking this for now.

Thanks!  abiusx@ and smbarber did some work getting ssh to talk to VMs on test builds that might be useful here too.

I hear derat@ is looking at making autotest go away completely so it may be easier to do what we want with his new thing.

Oh please yes! Everything I've found so far horrifies me. I'll chat with Dan and see what's up.

I think we'll need SSH access to the VM regardless. I'd recommend trying to keep whatever work is needed for this outside of the Autotest codebase as much as possible, though.[1]

Is the intent that this be some kind of wrapper script that starts a VM and then runs a test command that interacts with it, or would it be something completely different?

1. Autotest already has plenty of code, and adding more is unlikely to improve matters.

Is anything happening w/ this issue?

Not yet; was hopeful that Tast would get the Autoserv protocol support soonish. That would make it a viable replacement for Autotest.

By "get the Autoserv protocol support", do you mean gain the capability to execute Autotest-based tests written in Python and parse the results, or something else? I wrote some code that sort of did that a while back but never checked it in because it didn't seem particularly useful (compared to just writing tests in Go and getting additional benefits like fast deployment to DUTs).

If the main blocker here is being able to run Tast-based tests on real hardware in the lab, that's tracked by  issue 782515  and hopefully almost working.

Jason, let's chat briefly to figure out what's needed.

 issue 782515  is what I was thinking of. Thanks for the link.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/7a8084b92f5ae68d77ee2ea762ad0fd5031774f8

commit 7a8084b92f5ae68d77ee2ea762ad0fd5031774f8
Author: Dylan Reid <dgreid@chromium.org>
Date: Wed Feb 14 12:06:58 2018

Add vm_CrosVmStart

Test the crosvm can load the kernel and rootfs image that is provided
by the component updater.

This is the most basic test, similar tests will be added to make sure
maitred starts and that lxd is functioning.

BUG= 725692 
TEST=this is a test.

Change-Id: Ia7ec135c136067fc24dac705acb8158da7e59d65
Signed-off-by: Dylan Reid <dgreid@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/905509
Reviewed-by: Eric Caruso <ejcaruso@chromium.org>

[add] https://crrev.com/7a8084b92f5ae68d77ee2ea762ad0fd5031774f8/client/site_tests/vm_CrosVmStart/control
[add] https://crrev.com/7a8084b92f5ae68d77ee2ea762ad0fd5031774f8/client/site_tests/vm_CrosVmStart/vm_CrosVmStart.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/5e209c3c2146fa096ed8a9dd9993646bfd9cc3f4

commit 5e209c3c2146fa096ed8a9dd9993646bfd9cc3f4
Author: Dylan Reid <dgreid@chromium.org>
Date: Tue Feb 27 03:28:52 2018

chromeos-base: Add vm_CrosVmStart ebuild

This test is used to check that crosvm works on boards with
USE=kvm_host.

BUG= 725692 
TEST=test_that eve vm_CrosVmStart

Change-Id: I309908fd8a7dab063939a0083f7afb002ac14d82
Reviewed-on: https://chromium-review.googlesource.com/930487
Commit-Ready: Dylan Reid <dgreid@chromium.org>
Tested-by: Dylan Reid <dgreid@chromium.org>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>

[modify] https://crrev.com/5e209c3c2146fa096ed8a9dd9993646bfd9cc3f4/chromeos-base/autotest-all/autotest-all-0.0.1.ebuild
[rename] https://crrev.com/5e209c3c2146fa096ed8a9dd9993646bfd9cc3f4/chromeos-base/autotest-all/autotest-all-0.0.1-r44.ebuild
[add] https://crrev.com/5e209c3c2146fa096ed8a9dd9993646bfd9cc3f4/chromeos-base/autotest-tests-vm-host/autotest-tests-vm-host-9999.ebuild

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/82fe2867c0621fa4ca542309e348b6b1bb2bdcde

commit 82fe2867c0621fa4ca542309e348b6b1bb2bdcde
Author: Dylan Reid <dgreid@chromium.org>
Date: Fri Mar 02 00:13:32 2018

vm_CrosVmStart: Add to bvt-cq

Add basic cros vm functionality tests to the cq so failing changes won't
be merged.

BUG= 725692 
TEST=This is a test.

Change-Id: I2b83391bd62b9b3a16c49c939ccf2dd7559cb225
Signed-off-by: Dylan Reid <dgreid@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/941785
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/82fe2867c0621fa4ca542309e348b6b1bb2bdcde/client/site_tests/vm_CrosVmStart/control

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform2/+/0415e7bf95c1d09b63c903ae8fa6facda6c4de73

commit 0415e7bf95c1d09b63c903ae8fa6facda6c4de73
Author: Stephen Barber <smbarber@chromium.org>
Date: Wed Mar 28 03:35:04 2018

common-mk: add support for go protobufs

Add basic support for outputting go protobufs in preparation for
system_api being available for go clients.

BUG= chromium:725692 
TEST=emerge-eve system_api

Change-Id: I6bbda40688ec2e77f0a310599f0a0dc0fc09fd39
Reviewed-on: https://chromium-review.googlesource.com/981905
Commit-Ready: Stephen Barber <smbarber@chromium.org>
Tested-by: Stephen Barber <smbarber@chromium.org>
Reviewed-by: Dan Erat <derat@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/0415e7bf95c1d09b63c903ae8fa6facda6c4de73/common-mk/protoc.gypi

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/a37da168bda5317ccd213f98ae0c40587ab7ab4b

commit a37da168bda5317ccd213f98ae0c40587ab7ab4b
Author: Stephen Barber <smbarber@chromium.org>
Date: Wed Mar 28 23:40:38 2018

system_api: install go package for vm_concierge

This will allow direct usage of the vm_concierge dbus API from go.

BUG= chromium:725692 
TEST=emerge-eve system_api
CQ-DEPEND=CL:981906

Change-Id: I67ac91c8f7bfff46b32f447f74440db4d1b3c8e8
Reviewed-on: https://chromium-review.googlesource.com/981908
Commit-Ready: Stephen Barber <smbarber@chromium.org>
Tested-by: Stephen Barber <smbarber@chromium.org>
Reviewed-by: Rahul Chaudhry <rahulchaudhry@chromium.org>

[modify] https://crrev.com/a37da168bda5317ccd213f98ae0c40587ab7ab4b/chromeos-base/system_api/system_api-9999.ebuild

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/system_api/+/8d6a9e161bfbefd3339620fd0934b08cda27fed6

commit 8d6a9e161bfbefd3339620fd0934b08cda27fed6
Author: Stephen Barber <smbarber@chromium.org>
Date: Wed Mar 28 23:40:37 2018

vm_concierge: generate go protos

Generate go from vm_concierge protobufs. These will be installed
to a path suitable for installation via CROS_GO_PACKAGES.

BUG= chromium:725692 
TEST=emerge-eve system_api
CQ-DEPEND=CL:981905

Change-Id: I47fc32a9afdb92ee9a453451bea0483842cd1b46
Reviewed-on: https://chromium-review.googlesource.com/981906
Commit-Ready: Stephen Barber <smbarber@chromium.org>
Tested-by: Stephen Barber <smbarber@chromium.org>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/8d6a9e161bfbefd3339620fd0934b08cda27fed6/system_api.gyp
[modify] https://crrev.com/8d6a9e161bfbefd3339620fd0934b08cda27fed6/dbus/vm_concierge/service.proto

Hopefully jclinton won't mind me reassigning this to myself.

Right now I have a basic tast test to ensure the VM starts up via concierge.

Nice work!

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/tast-tests/+/614b78bd30d90762b0565236c8632af3f9c13571

commit 614b78bd30d90762b0565236c8632af3f9c13571
Author: Daniel Erat <derat@chromium.org>
Date: Sat May 12 11:00:51 2018

tast-tests: Add cryptohome package.

Move cryptohome-related code out of the chrome package and
into a new cryptohome package.

BUG= chromium:725692 
TEST=local tests matched by all '(bvt)' pass

Change-Id: I64bf72dcce27ef539d86e6e7d5c2418ede163895
Reviewed-on: https://chromium-review.googlesource.com/1053628
Commit-Ready: Dan Erat <derat@chromium.org>
Tested-by: Dan Erat <derat@chromium.org>
Reviewed-by: Stephen Barber <smbarber@chromium.org>

[add] https://crrev.com/614b78bd30d90762b0565236c8632af3f9c13571/src/chromiumos/tast/local/cryptohome/cryptohome.go
[delete] https://crrev.com/5c7d82fffa8b4f71dabcf3027e6d86a14419ccee/src/chromiumos/tast/local/chrome/cryptohome.go
[modify] https://crrev.com/614b78bd30d90762b0565236c8632af3f9c13571/src/chromiumos/tast/local/chrome/chrome.go

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/17fdcc9dbed797bf8e4bd88fbfd820609e2e4cec

commit 17fdcc9dbed797bf8e4bd88fbfd820609e2e4cec
Author: Stephen Barber <smbarber@chromium.org>
Date: Sun May 13 01:12:58 2018

tast-local-tests-cros: add deps on system_api and protobuf

Allow tast local tests to use protobufs and system_api.

BUG= chromium:725692 
TEST=emerge tast-local-tests-cros

Change-Id: I66cdfdd8417cdfe1fbbefcfdf5cf59cc69ae2a32
Reviewed-on: https://chromium-review.googlesource.com/989012
Commit-Ready: Stephen Barber <smbarber@chromium.org>
Tested-by: Stephen Barber <smbarber@chromium.org>
Reviewed-by: Dan Erat <derat@chromium.org>

[modify] https://crrev.com/17fdcc9dbed797bf8e4bd88fbfd820609e2e4cec/chromeos-base/tast-local-tests-cros/tast-local-tests-cros-9999.ebuild

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/e80b8755aeaad5f371facdbd4877238481a33ff2

commit e80b8755aeaad5f371facdbd4877238481a33ff2
Author: Stephen Barber <smbarber@chromium.org>
Date: Tue May 22 10:19:13 2018

tast-use-flags: add kvm_host USE flag

kvm_host is used to gate whether VMs are enabled on a device.

BUG= chromium:725692 
TEST=emerge-eve tast-use-flags

Change-Id: Ieb4c647ced0efa179c2e19933b805fa2e330bfa8
Reviewed-on: https://chromium-review.googlesource.com/1068137
Commit-Ready: Stephen Barber <smbarber@chromium.org>
Tested-by: Stephen Barber <smbarber@chromium.org>
Reviewed-by: Dan Erat <derat@chromium.org>

[add] https://crrev.com/e80b8755aeaad5f371facdbd4877238481a33ff2/chromeos-base/tast-use-flags/tast-use-flags-0.0.1-r1.ebuild
[modify] https://crrev.com/e80b8755aeaad5f371facdbd4877238481a33ff2/chromeos-base/tast-use-flags/tast-use-flags-0.0.1.ebuild

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/tast/+/e14825680ed4851058cdc0a6455435d2b4643ba6

commit e14825680ed4851058cdc0a6455435d2b4643ba6
Author: Stephen Barber <smbarber@chromium.org>
Date: Tue May 22 10:19:13 2018

tast: Add vm_host software dep

The vm_host software dep requires USE=kvm_host for
virtualization support.

BUG= chromium:725692 
TEST=vm.StartTerminaVM
CQ-DEPEND=CL:1068137

Change-Id: I6a0b02db17bee87797c0bffabbe994715557a6fb
Reviewed-on: https://chromium-review.googlesource.com/1068147
Commit-Ready: Stephen Barber <smbarber@chromium.org>
Tested-by: Stephen Barber <smbarber@chromium.org>
Reviewed-by: Dan Erat <derat@chromium.org>

[modify] https://crrev.com/e14825680ed4851058cdc0a6455435d2b4643ba6/src/chromiumos/cmd/local_test_runner/main.go
[modify] https://crrev.com/e14825680ed4851058cdc0a6455435d2b4643ba6/docs/test_dependencies.md

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/tast-tests/+/1adce6faa870da72575dc6fec49166b54183b42e

commit 1adce6faa870da72575dc6fec49166b54183b42e
Author: Stephen Barber <smbarber@chromium.org>
Date: Wed May 23 01:45:42 2018

tast-tests: Add User accessor for Chrome struct

BUG= chromium:725692 
TEST=tast tests build

Change-Id: Ie9949ca67a005fd32ac7f5511fe89643cec25bba
Reviewed-on: https://chromium-review.googlesource.com/1068207
Commit-Ready: Stephen Barber <smbarber@chromium.org>
Tested-by: Stephen Barber <smbarber@chromium.org>
Reviewed-by: Dan Erat <derat@chromium.org>

[modify] https://crrev.com/1adce6faa870da72575dc6fec49166b54183b42e/src/chromiumos/tast/local/chrome/chrome.go

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/system_api/+/d4a67e156cc64f8de5cd9d414e6f42623600e890

commit d4a67e156cc64f8de5cd9d414e6f42623600e890
Author: Stephen Barber <smbarber@chromium.org>
Date: Wed May 23 08:25:39 2018

vm_concierge: export goprotos as system_api/vm_concierge_proto

Match golang protobuf conventions by exporting system_api/vm_concierge
to system_api/vm_concierge_proto.

BUG= chromium:725692 
TEST=emerge system_api
CQ-DEPEND=CL:1068134

Change-Id: I5a363e84986f224eca6d6d4aeec618669e60c4e7
Reviewed-on: https://chromium-review.googlesource.com/1068132
Commit-Ready: Stephen Barber <smbarber@chromium.org>
Tested-by: Stephen Barber <smbarber@chromium.org>
Reviewed-by: Dan Erat <derat@chromium.org>
Reviewed-by: Rahul Chaudhry <rahulchaudhry@chromium.org>

[modify] https://crrev.com/d4a67e156cc64f8de5cd9d414e6f42623600e890/system_api.gyp
[modify] https://crrev.com/d4a67e156cc64f8de5cd9d414e6f42623600e890/dbus/vm_concierge/service.proto

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/45ed8199bed8ea526ed6d2f04c14be3da3635686

commit 45ed8199bed8ea526ed6d2f04c14be3da3635686
Author: Stephen Barber <smbarber@chromium.org>
Date: Wed May 23 08:25:39 2018

system_api: use system_api/vm_concierge_proto as go package

Match golang protobuf conventions by exporting system_api/vm_concierge
to system_api/vm_concierge_proto.

BUG= chromium:725692 
TEST=emerge system_api
CQ-DEPEND=CL:1068132

Change-Id: I4e56c44fb6dce3a948f965b8cefd4b3273bc2c31
Reviewed-on: https://chromium-review.googlesource.com/1068134
Commit-Ready: Stephen Barber <smbarber@chromium.org>
Tested-by: Stephen Barber <smbarber@chromium.org>
Reviewed-by: Dan Erat <derat@chromium.org>

[modify] https://crrev.com/45ed8199bed8ea526ed6d2f04c14be3da3635686/chromeos-base/system_api/system_api-9999.ebuild

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/tast-tests/+/7e6bd85a753eb3f05c54929de10b4c2798e6a7eb

commit 7e6bd85a753eb3f05c54929de10b4c2798e6a7eb
Author: Stephen Barber <smbarber@chromium.org>
Date: Wed May 23 19:50:48 2018

tast-tests: add concierge daemon support package

Add a package that handles starting VMs and (eventually) containers with
concierge.

BUG= chromium:725692 
TEST=vm.StartTerminaVM
CQ-DEPEND=CL:1068132

Change-Id: I9529c29a8d1330f549b7fa2e016ab366862d5f85
Reviewed-on: https://chromium-review.googlesource.com/988675
Commit-Ready: Stephen Barber <smbarber@chromium.org>
Tested-by: Stephen Barber <smbarber@chromium.org>
Reviewed-by: Dan Erat <derat@chromium.org>

[modify] https://crrev.com/7e6bd85a753eb3f05c54929de10b4c2798e6a7eb/src/chromiumos/tast/local/dbusutil/constants.go
[add] https://crrev.com/7e6bd85a753eb3f05c54929de10b4c2798e6a7eb/src/chromiumos/tast/local/vm/concierge.go

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/tast-tests/+/87cbbed3cd87a938d759d739b9ed32eafbceca55

commit 87cbbed3cd87a938d759d739b9ed32eafbceca55
Author: Stephen Barber <smbarber@chromium.org>
Date: Wed May 23 19:50:49 2018

tast-tests: add vm.StartTerminaVM

Add a basic test that starts a Termina VM with concierge.

BUG= chromium:725692 
TEST=vm.StartTerminaVM

Change-Id: Ifbdecb60a42efd064ed77db9c38bb3e69a4b1499
Reviewed-on: https://chromium-review.googlesource.com/988676
Commit-Ready: Stephen Barber <smbarber@chromium.org>
Tested-by: Stephen Barber <smbarber@chromium.org>
Reviewed-by: Dan Erat <derat@chromium.org>

[modify] https://crrev.com/87cbbed3cd87a938d759d739b9ed32eafbceca55/src/chromiumos/tast/local/bundles/cros/main.go
[add] https://crrev.com/87cbbed3cd87a938d759d739b9ed32eafbceca55/src/chromiumos/tast/local/bundles/cros/vm/start_termina_vm.go

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform2/+/44fb94cf44f0f66e18e85a22f6c13e5de6cbd6f2

commit 44fb94cf44f0f66e18e85a22f6c13e5de6cbd6f2
Author: Stephen Barber <smbarber@chromium.org>
Date: Tue Jun 26 19:52:05 2018

vm_tools: vsh: don't assume stdin is a tty

When used in headless mode (e.g. from a tast test), vsh will be given pipes
for stdio. Don't assume that /dev/tty is available. Also handle EOF from stdin
correctly.

BUG= chromium:725692 
TEST=tast experiment

Change-Id: Iea237956797667e1edf26071020900c66a0fc821
Reviewed-on: https://chromium-review.googlesource.com/1113911
Commit-Ready: Stephen Barber <smbarber@chromium.org>
Tested-by: Stephen Barber <smbarber@chromium.org>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>

[modify] https://crrev.com/44fb94cf44f0f66e18e85a22f6c13e5de6cbd6f2/vm_tools/vsh/vsh.cc
[modify] https://crrev.com/44fb94cf44f0f66e18e85a22f6c13e5de6cbd6f2/vm_tools/vsh/vsh_client.cc
[modify] https://crrev.com/44fb94cf44f0f66e18e85a22f6c13e5de6cbd6f2/vm_tools/vsh/vsh_client.h

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/tast-tests/+/fa5a75c6c601799cf6bbd8d9b91766b9ae1ae7ee

commit fa5a75c6c601799cf6bbd8d9b91766b9ae1ae7ee
Author: Stephen Barber <smbarber@chromium.org>
Date: Thu Jul 26 13:29:08 2018

tast-tests: vm: add support for tremplin

This adds support for launching containers via tremplin, and allows
running commands in the VM and container.

BUG= chromium:725692 
TEST=vm.StartTerminaVM

Change-Id: I60b5c77d190d72512f5de83ce3249ca6f36bacac
Reviewed-on: https://chromium-review.googlesource.com/1115969
Commit-Ready: Stephen Barber <smbarber@chromium.org>
Tested-by: Stephen Barber <smbarber@chromium.org>
Reviewed-by: Dan Erat <derat@chromium.org>

[add] https://crrev.com/fa5a75c6c601799cf6bbd8d9b91766b9ae1ae7ee/src/chromiumos/tast/local/vm/vm.go
[add] https://crrev.com/fa5a75c6c601799cf6bbd8d9b91766b9ae1ae7ee/src/chromiumos/tast/local/vm/util.go
[modify] https://crrev.com/fa5a75c6c601799cf6bbd8d9b91766b9ae1ae7ee/src/chromiumos/tast/local/vm/concierge.go
[add] https://crrev.com/fa5a75c6c601799cf6bbd8d9b91766b9ae1ae7ee/src/chromiumos/tast/local/vm/container.go
[modify] https://crrev.com/fa5a75c6c601799cf6bbd8d9b91766b9ae1ae7ee/src/chromiumos/tast/local/bundles/cros/vm/start_termina_vm.go