I think...

Also, I saw a CL for this.

This is a possibility that I am exploring. The other option would be to add a separate builder.

OK, I created a CL:
https://chromium-review.googlesource.com/c/535934/

And ran it with cbuildbot:
https://uberchromegw.corp.google.com/i/chromiumos.tryserver/builders/chromium_pfq_informational/builds/22

But got the following error:
Could not initialize SDL(No available video device) - exiting

achuith@ - Any thoughts here?

This is pretty awesome actually. 

Try --disable-kvm

I've never seen this error, tbh. Wonder if we can log into a baremetal builder and poke around?

Failing command: 16:18:33: INFO: RunCommand: sudo 'CROS_CACHEDIR=/tmp/cbuildbot-tmpXrsIVB/cbuildbot-tmpsbkocM/chrome-sdk-cachejGW_lD/cache' 'CROS_SUDO_KEEP_ALIVE=unknown' -- /usr/bin/qemu-system-x86_64 -m 2G -smp 4 -vga cirrus -daemonize -pidfile /tmp/cbuildbot-tmpXrsIVB/cbuildbot-tmpsbkocM/cros_vm/kvm.pid -chardev 'pipe,id=control_pipe,path=/tmp/cbuildbot-tmpXrsIVB/cbuildbot-tmpsbkocM/cros_vm/kvm.monitor' -serial file:/tmp/cbuildbot-tmpXrsIVB/cbuildbot-tmpsbkocM/cros_vm/kvm.monitor.serial -mon 'chardev=control_pipe' -net 'nic,model=virtio' -net 'user,hostfwd=tcp::9222-:22' -drive 'file=/tmp/cbuildbot-tmpXrsIVB/cbuildbot-tmpsbkocM/chrome-sdk-cachejGW_lD/cache/chrome-sdk/tarballs/amd64-generic+__b__cbuild__repository__trybot_archive__trybot-amd64-generic-tot-chromium-pfq-informational__R61-9651.0.0-b22+chromiumos_qemu_image.tar.xz/chromiumos_qemu_image.bin,index=0,media=disk,cache=unsafe' -enable-kvm

Update: I was able to avoid the error with --no-display, and it worked on several boards!

I am experimenting with which ones to run it on, and how to configure that correctly (e.g. don't try to run it on arm boards :) ), will update again soon.

Oh, and I ran locally with  --disable-kvm but it was *sloooow* so let's see what we can get working with just --no-display.

Ok, thanks for the update. I was working under the assumption that you were seeing the error after using --no-display. The smoke suite VM tests use a similar option --no_graphics, which does the same thing as --no-display.

Yup, we don't need --disable-kvm for these bots, since they are able to run the VM smoke suite with KVM.

https://bugs.chromium.org/p/chromium/issues/detail?id=782190
https://bugs.chromium.org/p/chromium/issues/detail?id=710629

The above bugs are a regression in cros_vm as a result of deprecating cirrus support. We will need to run cros_run_vm_test on the chromeos builders to guard against future regressions.

Ben / Achuith - I am handing this over to you two to decide what to do here, since it is related to issue 840967.

The intention here is to ensure that a CrOS update does not break the VMTests run in chromeos-{amd64|daisy}-generic-rel, so it would make sense to run the same tests in the PFQ that we run there.

The CL I put up before should serve as a reasonable reference:
https://chromium-review.googlesource.com/c/chromiumos/chromite/+/535934

But we probably need to update the arguments passed to cros_run_vm_test.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/3d335bee0f8d8fa3c56c89d38270afbe012a7b68

commit 3d335bee0f8d8fa3c56c89d38270afbe012a7b68
Author: Achuith Bhandarkar <achuith@chromium.org>
Date: Mon Dec 24 03:19:31 2018

commands: Use autoninja.

* Prefer autoninja instead of setting up the number of jobs.
* Prefer chromiumos_preflight instead of manual targets.

BUG= chromium:733394 
TEST=bots

Change-Id: Ia695e91fbffd793747e3d0cfaa8f703e5c09aa1c
Reviewed-on: https://chromium-review.googlesource.com/1382941
Commit-Ready: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com>
Tested-by: Achuith Bhandarkar <achuith@chromium.org>
Reviewed-by: Steven Bennetts <stevenjb@chromium.org>
Reviewed-by: Ben Pastene <bpastene@chromium.org>

[modify] https://crrev.com/3d335bee0f8d8fa3c56c89d38270afbe012a7b68/cbuildbot/commands.py
[modify] https://crrev.com/3d335bee0f8d8fa3c56c89d38270afbe012a7b68/cbuildbot/commands_unittest.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/ad5955aeec3f9d6d396444d739910cacbfb00c61

commit ad5955aeec3f9d6d396444d739910cacbfb00c61
Author: Achuith Bhandarkar <achuith@chromium.org>
Date: Tue Jan 08 22:33:37 2019

chrome_stages: Add cros_run_vm_test.

Add cros_run_vm_test to TestSimpleChromeStage.

BUG= chromium:733394 
TEST=amd64-generic-chromium-pfq

Change-Id: I1dac0013179ee12179c3285db8ebe1aa89133292
Reviewed-on: https://chromium-review.googlesource.com/1381066
Commit-Ready: Achuith Bhandarkar <achuith@chromium.org>
Tested-by: Achuith Bhandarkar <achuith@chromium.org>
Reviewed-by: Steven Bennetts <stevenjb@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>

[modify] https://crrev.com/ad5955aeec3f9d6d396444d739910cacbfb00c61/cbuildbot/commands.py
[modify] https://crrev.com/ad5955aeec3f9d6d396444d739910cacbfb00c61/cbuildbot/stages/chrome_stages.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/27a5a4c68a0c933a3e65f25475d2985cf0b709c8

commit 27a5a4c68a0c933a3e65f25475d2985cf0b709c8
Author: Nicolas Boichat <drinkcat@chromium.org>
Date: Wed Jan 09 07:21:11 2019

Revert "chrome_stages: Add cros_run_vm_test."

This reverts commit ad5955aeec3f9d6d396444d739910cacbfb00c61.

Reason for revert: Breaks -release and PFQ

Original change's description:
> chrome_stages: Add cros_run_vm_test.
> 
> Add cros_run_vm_test to TestSimpleChromeStage.
> 
> BUG= chromium:733394 
> TEST=amd64-generic-chromium-pfq
> 
> Change-Id: I1dac0013179ee12179c3285db8ebe1aa89133292
> Reviewed-on: https://chromium-review.googlesource.com/1381066
> Commit-Ready: Achuith Bhandarkar <achuith@chromium.org>
> Tested-by: Achuith Bhandarkar <achuith@chromium.org>
> Reviewed-by: Steven Bennetts <stevenjb@chromium.org>
> Reviewed-by: Don Garrett <dgarrett@chromium.org>

BUG= chromium:733394 
BUG= chromium:920096 
TEST=N/A

Change-Id: I728ff2895964dea45c74f5be02a6b4dc153fb574
Reviewed-on: https://chromium-review.googlesource.com/c/1401883
Tested-by: Nicolas Boichat <drinkcat@chromium.org>
Trybot-Ready: Nicolas Boichat <drinkcat@chromium.org>
Reviewed-by: Leo Zhang <googleo@google.com>

[modify] https://crrev.com/27a5a4c68a0c933a3e65f25475d2985cf0b709c8/cbuildbot/commands.py
[modify] https://crrev.com/27a5a4c68a0c933a3e65f25475d2985cf0b709c8/cbuildbot/stages/chrome_stages.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/4f3f4e17214542f140b7c905ed753a37da2e358b

commit 4f3f4e17214542f140b7c905ed753a37da2e358b
Author: Achuith Bhandarkar <achuith@chromium.org>
Date: Fri Jan 11 17:48:02 2019

chrome_stages: Add cros_run_vm_test.

Add cros_run_vm_test to TestSimpleChromeStage.

BUG= chromium:733394 
TEST=amd64-generic-chromium-pfq

Change-Id: I050f953c82b8c631224d4b38646af8ec29d8e022
Reviewed-on: https://chromium-review.googlesource.com/1403881
Commit-Ready: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com>
Tested-by: Achuith Bhandarkar <achuith@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>

[modify] https://crrev.com/4f3f4e17214542f140b7c905ed753a37da2e358b/cbuildbot/commands.py
[modify] https://crrev.com/4f3f4e17214542f140b7c905ed753a37da2e358b/cbuildbot/stages/chrome_stages.py

Recently amd64-generic-chromium-pfq run:
https://cros-goldeneye.corp.google.com/chromeos/healthmonitoring/buildDetails?buildbucketId=8924338787658569904

Build logs:
https://luci-logdog.appspot.com/logs/chromeos/buildbucket/cr-buildbucket.appspot.com/8924338787658569904/+/steps/TestSimpleChromeWorkflow/0/stdout

We can see that cros_run_vm_test is running and succeeding.