New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 832374 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Jun 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 1
Type: Bug

Blocked on: View detail
issue 828607
issue 844102
issue 832379

Blocking:
issue 840967
issue 836447



Sign in to add a comment

Spin up a non-blocking bot that runs tests on cros vms.

Project Member Reported by bpastene@chromium.org, Apr 12 2018

Issue description

Going to look identical to https://ci.chromium.org/buildbot/chromium.chromiumos/chromeos-amd64-generic-rel/ + some tests.

Blocked on https://chromium-review.googlesource.com/c/chromium/src/+/994062 landing to at least get the tests running in VMs, but will set up the bot anyway.

Filing this to track the multitude of config changes needed to create a new bot.
 
Blockedon: 832379
Labels: -Pri-3 Pri-1
Project Member

Comment 2 by bugdroid1@chromium.org, Apr 13 2018

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infradata/config/+/6c9359e5eab580f2f3c2cdbeb461c0e2c83ffb1a

commit 6c9359e5eab580f2f3c2cdbeb461c0e2c83ffb1a
Author: Benjamin Pastene <bpastene@chromium.org>
Date: Fri Apr 13 00:51:38 2018

Project Member

Comment 3 by bugdroid1@chromium.org, Apr 13 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/f06103dc4019133ade92a65bbaac674b1c6da669

commit f06103dc4019133ade92a65bbaac674b1c6da669
Author: Benjamin Pastene <bpastene@chromium.org>
Date: Fri Apr 13 01:32:11 2018

Add recipe config for new chromeos-amd64-generic-rel-vm-tests bot.

Copied from the dict at
https://cs.chromium.org/chromium/build/scripts/slave/recipe_modules/chromium_tests/chromium_chromiumos.py?l=79

TBR=jbudorick@chromium.org

Bug:  832374 
Change-Id: I2a9d73c18b01732c5a8210f7fed223fa4413d446
Reviewed-on: https://chromium-review.googlesource.com/1011330
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Reviewed-by: Ben Pastene <bpastene@chromium.org>

[modify] https://crrev.com/f06103dc4019133ade92a65bbaac674b1c6da669/scripts/slave/recipe_modules/chromium_tests/chromium_fyi.py

Project Member

Comment 4 by bugdroid1@chromium.org, Apr 13 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/ed2aa1ee3ef5706aab8f58b401d4816364b42e1e

commit ed2aa1ee3ef5706aab8f58b401d4816364b42e1e
Author: Benjamin Pastene <bpastene@chromium.org>
Date: Fri Apr 13 17:00:13 2018

Add LUCI configs for new chromeos-amd64-generic-rel-vm-tests bot.

TBR=jbudorick@chromium.org

Bug:  832374 
Change-Id: I7500d930c5e8de66661e10867e35333d13d04355
Reviewed-on: https://chromium-review.googlesource.com/1011622
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Reviewed-by: Ben Pastene <bpastene@chromium.org>
Cr-Commit-Position: refs/heads/master@{#550656}
[modify] https://crrev.com/ed2aa1ee3ef5706aab8f58b401d4816364b42e1e/infra/config/global/cr-buildbucket.cfg
[modify] https://crrev.com/ed2aa1ee3ef5706aab8f58b401d4816364b42e1e/infra/config/global/luci-scheduler.cfg

Project Member

Comment 5 by bugdroid1@chromium.org, Apr 17 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/c87dfaf09b63ddfe3438f8da391f7ba94030634e

commit c87dfaf09b63ddfe3438f8da391f7ba94030634e
Author: Benjamin Pastene <bpastene@chromium.org>
Date: Tue Apr 17 03:03:42 2018

Add mb config and test spec for CrOS vm tester bot.

https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/chromeos-amd64-generic-rel-vm-tests

https://chromium-review.googlesource.com/c/chromium/tools/build/+/1013289 needs
to land before the tests actually do anything. Hopefully I'll be able to land
that sometime soon. This at least gets the configs set up.

Bug:  832374 
Change-Id: I93665b1307c82e84e5a5d83d2092a792786573f6
Reviewed-on: https://chromium-review.googlesource.com/1014484
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Cr-Commit-Position: refs/heads/master@{#551216}
[modify] https://crrev.com/c87dfaf09b63ddfe3438f8da391f7ba94030634e/testing/buildbot/chromium.fyi.json
[modify] https://crrev.com/c87dfaf09b63ddfe3438f8da391f7ba94030634e/testing/buildbot/test_suites.pyl
[modify] https://crrev.com/c87dfaf09b63ddfe3438f8da391f7ba94030634e/testing/buildbot/waterfalls.pyl
[modify] https://crrev.com/c87dfaf09b63ddfe3438f8da391f7ba94030634e/tools/mb/mb_config.pyl

Project Member

Comment 6 by bugdroid1@chromium.org, Apr 17 2018

Labels: merge-merged-testbranch
The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/ed2aa1ee3ef5706aab8f58b401d4816364b42e1e

commit ed2aa1ee3ef5706aab8f58b401d4816364b42e1e
Author: Benjamin Pastene <bpastene@chromium.org>
Date: Fri Apr 13 17:00:13 2018

Add LUCI configs for new chromeos-amd64-generic-rel-vm-tests bot.

TBR=jbudorick@chromium.org

Bug:  832374 
Change-Id: I7500d930c5e8de66661e10867e35333d13d04355
Reviewed-on: https://chromium-review.googlesource.com/1011622
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Reviewed-by: Ben Pastene <bpastene@chromium.org>
Cr-Commit-Position: refs/heads/master@{#550656}
[modify] https://crrev.com/ed2aa1ee3ef5706aab8f58b401d4816364b42e1e/infra/config/global/cr-buildbucket.cfg
[modify] https://crrev.com/ed2aa1ee3ef5706aab8f58b401d4816364b42e1e/infra/config/global/luci-scheduler.cfg

Project Member

Comment 7 by bugdroid1@chromium.org, Apr 18 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/b47ca36ae32a08b718f7885f0fbda26a4ab08a65

commit b47ca36ae32a08b718f7885f0fbda26a4ab08a65
Author: Benjamin Pastene <bpastene@chromium.org>
Date: Wed Apr 18 19:16:57 2018

Fix name of cros vm test bot in mb_config.

Whoops

TBR=dpranke@chromium.org

Bug:  832374 
Change-Id: Ib2caf7bcb5d6cc8e15312122d8cb24f6eb0ca23c
Reviewed-on: https://chromium-review.googlesource.com/1015408
Reviewed-by: Ben Pastene <bpastene@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Commit-Position: refs/heads/master@{#551775}
[modify] https://crrev.com/b47ca36ae32a08b718f7885f0fbda26a4ab08a65/tools/mb/mb_config.pyl

Project Member

Comment 8 by bugdroid1@chromium.org, Apr 18 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/fab64fff87d182b38cecccce779a57b682c7fa3e

commit fab64fff87d182b38cecccce779a57b682c7fa3e
Author: Benjamin Pastene <bpastene@chromium.org>
Date: Wed Apr 18 22:21:16 2018

Clobber the cros-chrome-sdk cache everytime when fetching artifacts.

Adds 2-3 minutes to every cros-flavored "gclient runhooks", which
is unfortunate.

Should be unnecessary when/if crbug.com/834134 is fixed.

Bug:  832374 
Change-Id: I2696fc736350ecbe2b641112d2140db66118a00e
Reviewed-on: https://chromium-review.googlesource.com/1016137
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Commit-Position: refs/heads/master@{#551850}
[modify] https://crrev.com/fab64fff87d182b38cecccce779a57b682c7fa3e/DEPS

Project Member

Comment 9 by bugdroid1@chromium.org, Apr 19 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/b254d023b48d450f39ef865746317f164c2ac549

commit b254d023b48d450f39ef865746317f164c2ac549
Author: Benjamin Pastene <bpastene@chromium.org>
Date: Thu Apr 19 17:11:47 2018

Don't wrap runhooks in the cros chrome-sdk for simplechrome bots.

And point all other invocations of it to the new cache location. (Used
to be "src/../.cros_cache" but now it's "src/build/cros_cache")

And also set the gclient var cros_board so the gclient hook downloads
the right board for each bot.

Recipe-Nontrivial-Roll: build_limited_scripts_slave
Bug:  832374 , 732531
Change-Id: I9aea093d7a6e519922eb375cf82bc0da1ebd0336
Reviewed-on: https://chromium-review.googlesource.com/1013289
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>

[modify] https://crrev.com/b254d023b48d450f39ef865746317f164c2ac549/scripts/slave/recipe_modules/chromium/tests/run_mb.expected/cros_board.json
[modify] https://crrev.com/b254d023b48d450f39ef865746317f164c2ac549/scripts/slave/recipe_modules/chromium/examples/full.py
[modify] https://crrev.com/b254d023b48d450f39ef865746317f164c2ac549/scripts/slave/recipe_modules/chromium/api.py
[modify] https://crrev.com/b254d023b48d450f39ef865746317f164c2ac549/scripts/slave/recipe_modules/chromium_tests/chromium_fyi.py
[modify] https://crrev.com/b254d023b48d450f39ef865746317f164c2ac549/scripts/slave/README.recipes.md
[modify] https://crrev.com/b254d023b48d450f39ef865746317f164c2ac549/scripts/slave/recipe_modules/chromium_tests/chromium_chromiumos.py
[modify] https://crrev.com/b254d023b48d450f39ef865746317f164c2ac549/scripts/slave/recipe_modules/filter/tests/analyze.expected/basic.json
[modify] https://crrev.com/b254d023b48d450f39ef865746317f164c2ac549/scripts/slave/recipe_modules/chromium/gclient_config.py
[modify] https://crrev.com/b254d023b48d450f39ef865746317f164c2ac549/scripts/slave/recipe_modules/chromium/tests/compile.expected/chromeos.json
[modify] https://crrev.com/b254d023b48d450f39ef865746317f164c2ac549/scripts/slave/recipe_modules/chromium/tests/runhooks.expected/chromeos.json
[add] https://crrev.com/b254d023b48d450f39ef865746317f164c2ac549/scripts/slave/recipe_modules/chromium/examples/full.expected/chromeos_simplechrome.json
[modify] https://crrev.com/b254d023b48d450f39ef865746317f164c2ac549/scripts/slave/recipes/swarming/staging.expected/cros.json

Cc: achuith@chromium.org
Hurrah it works!
https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/chromeos-amd64-generic-rel-vm-tests/524

And the tests are actually passing 100%:
https://chromium-swarm.appspot.com/task?id=3cf7def6e69e6410
https://chromium-swarm.appspot.com/task?id=3cf7defaa9054810
https://chromium-swarm.appspot.com/task?id=3cf7deff74c78c10

Though the steps are purple since the result-collecting logic is a little confused. We're not pulling the test-results json file from the vm after the test finishes, so the recipe doesn't actually know the detailed results. I'll make the test wrapper pull it from the vm before finishing.

Also, sharding isn't supported atm, but that's just a matter of passing down the "--shard-index" and "--shard-count" gtest args down to the test. I'll get to that too.
Project Member

Comment 11 by bugdroid1@chromium.org, Apr 20 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/be4dea1713bc7a1c0461744289cfee9c3b88acd4

commit be4dea1713bc7a1c0461744289cfee9c3b88acd4
Author: Benjamin Pastene <bpastene@chromium.org>
Date: Fri Apr 20 16:50:54 2018

Add standard gtest args to the cros vm_test wrapper.

This adds support for test sharding and json results.

See https://ci.chromium.org/buildbot/tryserver.chromium.chromiumos/chromeos-amd64-generic-rel/103454
for this patch in action.

Bug:  832374 
Change-Id: I1a1bea2412e25af9d5ba7a8daf0b6cb34e7b6ae4
Reviewed-on: https://chromium-review.googlesource.com/1020535
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Commit-Position: refs/heads/master@{#552356}
[modify] https://crrev.com/be4dea1713bc7a1c0461744289cfee9c3b88acd4/build/chromeos/run_vm_test.py

The issues in #10 are solved, and now we got a fully green build:
https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/chromeos-amd64-generic-rel-vm-tests/655

\o/

So now all gtests should be ready to start running in the VMs. That covers ~90% of our tests. To get the rest I still need to cover:

- telemetry/catapult unittests: Unlike gtest binaries, this expects to run on the host and be given hostname and port of cros device. I think I'll just need to call cros_vm --start/--stop before/afterwards and pass it localhost and VM port. (see also https://chromium.googlesource.com/chromiumos/docs/+/master/cros_vm.md#run-telemetry-unit-tests).

- vm sanity test: This is the default behavior of cros_run_vm_test when no --cmd is specified. IIUC, it just opens chrome, loads some basic pages/runs some JS and closes. We'd probably want to run this on the bots as well. Just need to make sure we build chrome first (I believe the target chromiumos_preflight takes care of that) and deploy it to the device. cros_run_vm_test can take care of that, so this should be straightforward.
Great!
Project Member

Comment 14 by bugdroid1@chromium.org, Apr 21 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/6e06160799aafc332b225a52ee9bf130211d2b0f

commit 6e06160799aafc332b225a52ee9bf130211d2b0f
Author: Benjamin Pastene <bpastene@chromium.org>
Date: Sat Apr 21 00:09:37 2018

Add a few more suites to cros vm test bot.

And double shard crypto_unittests to test sharding & merging logic.

Bug:  832374 
Change-Id: I165160b5482efb505861f7f4bd0e0f1497923efc
Reviewed-on: https://chromium-review.googlesource.com/1022255
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Commit-Position: refs/heads/master@{#552525}
[modify] https://crrev.com/6e06160799aafc332b225a52ee9bf130211d2b0f/testing/buildbot/chromium.fyi.json
[modify] https://crrev.com/6e06160799aafc332b225a52ee9bf130211d2b0f/testing/buildbot/test_suites.pyl

Cc: nednguyen@chromium.org
Jotting down my current plan for the last two test types.

telemetry:
- break the data of generate_vm_runner_script target into a group: (that's what handles the logic of adding the vm image to the isolate)
https://codesearch.chromium.org/chromium/src/build/config/chromeos/rules.gni?rcl=dcd041159fc60f661b18aa13b9349bff634b532c&l=23
- add that group as a dep to the telemetry test target for chromeos: (so that all telemetry isolates have the vm image)
https://codesearch.chromium.org/chromium/src/chrome/test/BUILD.gn?rcl=30c65c50c700772bfe56e7a6a01724fc74367733&l=5529
- wrap the test call in build/chromeos/run_vm_test.py which will handle launching/tearing down the VM
https://codesearch.chromium.org/chromium/src/testing/buildbot/gn_isolate_map.pyl?l=1062
or maybe https://codesearch.chromium.org/chromium/src/tools/mb/mb.py?l=1068
or maybe create a whole new target? (not sure about this part yet)
- then I think it should be good to go

Another option is to let telemetry handle the isolating of the VM deps as well as the startup/teardown of the VM, but I'd rather we keep these all in one place (ie: //build/config/chromeos/rules.gni and //build/chromeos/run_vm_test.py") to avoid duplication.


vm sanity test:
- create a dumb test target (maybe "chromeos_vm_sanity_test")
https://codesearch.chromium.org/chromium/src/chromeos/BUILD.gn
- it won't do much but depend on //chrome
- since it's a test, a vm wrapper script will get generated:
https://codesearch.chromium.org/chromium/src/testing/test.gni?rcl=0f2bbbfbc10159502a98aebd3e6cf2f44a145c6d&l=360
- then I hardcode special logic in the script that runs the vm sanity test when "--test-exe" is chromeos_vm_sanity_test (needed because there won't be an actual chromeos_vm_sanity_test test binary)
https://codesearch.chromium.org/chromium/src/build/chromeos/run_vm_test.py?l=60
we definitely want to run the VM sanity test - it captures a good 80% of the chromium CLs that break CrOS.

All of these tests will run on the same bot chromeos-amd64-generic-rel-vm-tests, correct?

https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/chromeos-amd64-generic-rel-vm-tests
Noted; I'll be sure to get the sanity test up and running.

And the chromeos-amd64-generic-rel-vm-tests bot is just my playground for now. Once we're ready to run these tests on the CQ, we'll add the tests to the real amd64-generic simplechrome bot (this one's actually sheriffed):
https://ci.chromium.org/buildbot/chromium.chromiumos/chromeos-amd64-generic-rel/
And its CQ bot:
https://ci.chromium.org/buildbot/tryserver.chromium.chromiumos/chromeos-amd64-generic-rel/
Project Member

Comment 18 by bugdroid1@chromium.org, May 2 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/e5e7dc3762a1b18646810303e106da173ac603b5

commit e5e7dc3762a1b18646810303e106da173ac603b5
Author: Ben Pastene <bpastene@chromium.org>
Date: Wed May 02 01:42:14 2018

Add a host-side mode of execution for cros vm test runner.

This will add a positional arg to run_vm_test.py that can take one of two
values:

- "host-cmd": Runs a command on the host. Just launches the vm then
immediately calls the passed in cmd on the host. It's assumed the cmd will
handle data deps, ssh commands, etc.

- "vm-test": Runs a test in the vm. Takes care of spawning the vm, pushing
the test and its deps to the vm, executing the test, and tears down the vm.
(This is the current default behavior.)

Bug:  832374 
Change-Id: Ib8bd1b2e7dd393f8a4578979fa164381a8342ef2
Reviewed-on: https://chromium-review.googlesource.com/1038747
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Cr-Commit-Position: refs/heads/master@{#555279}
[modify] https://crrev.com/e5e7dc3762a1b18646810303e106da173ac603b5/build/chromeos/create_vm_test_script.py
[modify] https://crrev.com/e5e7dc3762a1b18646810303e106da173ac603b5/build/chromeos/run_vm_test.py
[modify] https://crrev.com/e5e7dc3762a1b18646810303e106da173ac603b5/build/config/chromeos/rules.gni

Blocking: 836447
Project Member

Comment 20 by bugdroid1@chromium.org, May 4 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/c60bf502e3e92c32c7c297a429563c83db02277e

commit c60bf502e3e92c32c7c297a429563c83db02277e
Author: Ben Pastene <bpastene@chromium.org>
Date: Fri May 04 01:10:33 2018

Create gn target that builds the cros-vm-launcher.

This "cros_vm_launcher" target generates a script at build time (located
at $out_dir/bin/launch_cros_vm) that you prepend to any other command
and it will ensure a cros VM is running for the duration of the command.

Also add it as a dependency to telemetry unittests. Invoking it in a bot's
tests will come in a later CL.

Bug: 828607,  832374 
Change-Id: I56ee169c81127bf8cac1114b719f1cdeeb2c3d31
Reviewed-on: https://chromium-review.googlesource.com/1040117
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Commit-Position: refs/heads/master@{#555945}
[modify] https://crrev.com/c60bf502e3e92c32c7c297a429563c83db02277e/chromeos/BUILD.gn
[modify] https://crrev.com/c60bf502e3e92c32c7c297a429563c83db02277e/tools/perf/chrome_telemetry_build/BUILD.gn

Project Member

Comment 21 by bugdroid1@chromium.org, May 4 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/8ab6954de550aebdf36d72e899910731fe9b6186

commit 8ab6954de550aebdf36d72e899910731fe9b6186
Author: Ben Pastene <bpastene@chromium.org>
Date: Fri May 04 04:08:24 2018

Add telemetry_perf_unittests to the cros vm test bot.

Will all likely fail until I fix  crbug.com/839187 

This'll at least get them running.

Bug:  836447 ,  832374 
Change-Id: I1d206ab931230652b4ba3dc8ba519f386a80da96
Reviewed-on: https://chromium-review.googlesource.com/1043258
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Cr-Commit-Position: refs/heads/master@{#555974}
[modify] https://crrev.com/8ab6954de550aebdf36d72e899910731fe9b6186/testing/buildbot/chromium.fyi.json
[modify] https://crrev.com/8ab6954de550aebdf36d72e899910731fe9b6186/testing/buildbot/test_suites.pyl
[modify] https://crrev.com/8ab6954de550aebdf36d72e899910731fe9b6186/testing/buildbot/waterfalls.pyl
[modify] https://crrev.com/8ab6954de550aebdf36d72e899910731fe9b6186/tools/mb/mb.py

telemetry's looking good:
https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/chromeos-amd64-generic-rel-vm-tests/1872

However it doesn't seem to run a lot of benchmarks:
* Passed: 88 (88 expected, 0 unexpected)
* Skipped: 169 (169 expected, 0 unexpected)

It must skip a lot on CrOS by default, so probably WAI?

Moving onto the sanity test...
Blocking: 840967
Project Member

Comment 24 by bugdroid1@chromium.org, May 9 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/c55041d7fbb3cc88a500ec407b11ce24d21ed8c6

commit c55041d7fbb3cc88a500ec407b11ce24d21ed8c6
Author: Ben Pastene <bpastene@chromium.org>
Date: Wed May 09 20:27:42 2018

Move cros vm tests to their own swarming pool.

Should help with keeping the vm image warm in the bots' caches.

Currently 20 bots in the pool. Will scale up if needed.

TBR=dpranke@chromium.org

Bug: 840967,  832374 
Change-Id: Ia09fdde0a94d0d798cd1c5073f0651b1ec92ac66
Reviewed-on: https://chromium-review.googlesource.com/1052824
Reviewed-by: Ben Pastene <bpastene@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Commit-Position: refs/heads/master@{#557303}
[modify] https://crrev.com/c55041d7fbb3cc88a500ec407b11ce24d21ed8c6/testing/buildbot/chromium.fyi.json
[modify] https://crrev.com/c55041d7fbb3cc88a500ec407b11ce24d21ed8c6/testing/buildbot/waterfalls.pyl

Project Member

Comment 25 by bugdroid1@chromium.org, May 11 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/5b0d0dd175c71ae0cc403897ec0a1adbb2844939

commit 5b0d0dd175c71ae0cc403897ec0a1adbb2844939
Author: Achuith Bhandarkar <achuith@chromium.org>
Date: Fri May 11 19:10:49 2018

cros_run_vm_test: Support for --host-cmd

BUG= chromium:832374 
TEST=cros_run_vm_test --host-cmd -- test_that localhost:9222 login_Cryptohome
Fail cases:
cros_run_vm_test test_that localhost:9222 login_Cryptohome
cros_run_vm_test --host-cmd test_that localhost:9222 login_Cryptohome
cros_run_vm_test --host-cmd

Change-Id: I32875d59bd61f7dfdd34795bdc0322984c050d0f
Reviewed-on: https://chromium-review.googlesource.com/1053099
Commit-Ready: Achuith Bhandarkar <achuith@chromium.org>
Tested-by: Achuith Bhandarkar <achuith@chromium.org>
Reviewed-by: Achuith Bhandarkar <achuith@chromium.org>
Reviewed-by: Ben Pastene <bpastene@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/5b0d0dd175c71ae0cc403897ec0a1adbb2844939/scripts/cros_run_vm_test.py

Project Member

Comment 26 by bugdroid1@chromium.org, May 11 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/f812c075051dd302e9bae4572715bce9a78f8bd9

commit f812c075051dd302e9bae4572715bce9a78f8bd9
Author: chromite-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com <chromite-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Date: Fri May 11 21:16:45 2018

Roll src/third_party/chromite/ 746c45d5d..7657336de (3 commits)

https://chromium.googlesource.com/chromiumos/chromite.git/+log/746c45d5d8b1..7657336de212

$ git log 746c45d5d..7657336de --date=short --no-merges --format='%ad %ae %s'
2018-05-08 mikenichols [chromite] Capture job_name and git timing for ts_mon in cbuildbot runs.
2018-05-10 khmel arc: Retry copying resource in case race condition for android uprev
2018-05-09 achuith cros_run_vm_test: Support for --host-cmd

Created with:
  roll-dep src/third_party/chromite
BUG= chromium:829289 , chromium:841937 , chromium:832374 


The AutoRoll server is located here: https://chromite-chromium-roll.skia.org

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.


TBR=chrome-os-gardeners@chromium.org

Change-Id: I7712f93eb43ad2661af0fe11a71a9b228ce916ad
Reviewed-on: https://chromium-review.googlesource.com/1055816
Commit-Queue: Chromite Chromium Autoroll <chromite-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Reviewed-by: Chromite Chromium Autoroll <chromite-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#558020}
[modify] https://crrev.com/f812c075051dd302e9bae4572715bce9a78f8bd9/DEPS

Project Member

Comment 27 by bugdroid1@chromium.org, May 14 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/83b500c126930271f4da9c246c768d61ada04a84

commit 83b500c126930271f4da9c246c768d61ada04a84
Author: Ben Pastene <bpastene@chromium.org>
Date: Mon May 14 18:00:28 2018

Add a sanity-test mode to the cros vm test runner.

When a test command is specified, the run_cros_vm_test will run the vm
sanity test. This test is baked into the VM and smoke checks the system
browser by loading empty pages and running some simple javascript.

Since it depends on the system browser, we need to first deploy a locally
built Chrome to the VM (via the --deploy arg). The deploy script needs to
know the gn args that built the browser to know which libs to copy over.
Since we don't have the full set of args at test-time, give it a dummy
set of args to trigger the default deploying behavior.

Bug:  832374 
Change-Id: I84f7fdd94672fcb220eaa4d9625395c1d0ab7636
Reviewed-on: https://chromium-review.googlesource.com/1050981
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Cr-Commit-Position: refs/heads/master@{#558372}
[modify] https://crrev.com/83b500c126930271f4da9c246c768d61ada04a84/build/chromeos/run_vm_test.py

Project Member

Comment 28 by bugdroid1@chromium.org, May 14 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/9ec70f930a6c40ee7f5a3ac56418de2718593392

commit 9ec70f930a6c40ee7f5a3ac56418de2718593392
Author: Ben Pastene <bpastene@chromium.org>
Date: Mon May 14 21:09:22 2018

Add GN target for building the cros VM sanity test.

The actual test binary is actually baked into the VM image. The gn
target here just builds chrome and a wrapper script around the test
runner.

Bug:  832374 
Change-Id: I062c80966abb6a234511ba125cdfb5c86367d9ad
Reviewed-on: https://chromium-review.googlesource.com/1057961
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Cr-Commit-Position: refs/heads/master@{#558459}
[modify] https://crrev.com/9ec70f930a6c40ee7f5a3ac56418de2718593392/build/config/chromeos/rules.gni
[modify] https://crrev.com/9ec70f930a6c40ee7f5a3ac56418de2718593392/chromeos/BUILD.gn

Project Member

Comment 29 by bugdroid1@chromium.org, May 15 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/9cba3376c389e164353cf417e70b926b86eea095

commit 9cba3376c389e164353cf417e70b926b86eea095
Author: Ben Pastene <bpastene@chromium.org>
Date: Tue May 15 03:45:05 2018

Run the cros vm sanity test on the fyi cros vm bot.

Bug:  832374 
Change-Id: I13d52c97809ec4da1e2f2987d0b76c9d9be91810
Reviewed-on: https://chromium-review.googlesource.com/1058507
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Commit-Position: refs/heads/master@{#558602}
[modify] https://crrev.com/9cba3376c389e164353cf417e70b926b86eea095/testing/buildbot/chromium.fyi.json
[modify] https://crrev.com/9cba3376c389e164353cf417e70b926b86eea095/testing/buildbot/gn_isolate_map.pyl
[modify] https://crrev.com/9cba3376c389e164353cf417e70b926b86eea095/testing/buildbot/test_suites.pyl

Project Member

Comment 30 by bugdroid1@chromium.org, May 16 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/39f23e84c22a9815c1537e05aeb38490ef1d6ad9

commit 39f23e84c22a9815c1537e05aeb38490ef1d6ad9
Author: Ben Pastene <bpastene@chromium.org>
Date: Wed May 16 02:13:18 2018

Create a simple json results file when running the cros VM sanity test.

This uses build/android's gtest result lib. The results will contain only
a single test, and will either be PASS or FAIL depending on the return code.

Bug:  832374 
Change-Id: I7294206340b17fda3f9f93a5085cd95b9b5faceb
Reviewed-on: https://chromium-review.googlesource.com/1060023
Reviewed-by: John Budorick <jbudorick@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Commit-Position: refs/heads/master@{#558929}
[modify] https://crrev.com/39f23e84c22a9815c1537e05aeb38490ef1d6ad9/build/chromeos/run_vm_test.py
[modify] https://crrev.com/39f23e84c22a9815c1537e05aeb38490ef1d6ad9/build/config/chromeos/rules.gni

Sanity test is up and running on the fyi bot:
https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/chromeos-amd64-generic-rel-vm-tests/2320

The flakiness dashboard is even tracking it:
https://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=cros_vm_sanity_test

Although, passing runs are white w/ a question mark... That's weird. I woulda expected them to be colored green. I'll need to look into that. Maybe I got something wrong.

And we've had one sanity test failure since turning it on:
https://chromium-swarm.appspot.com/user/task/3d8297ad222d8d10

Maybe some type of flake? I see "ERROR:root:Failed with DevToolsClientConnectionError while starting the browser backend." in the logs. We'll probably need to sort that out before turning the test on in the CQ.
Did a little playing around with base_unittests. We're seeing the following failures on that suite:
FileUtilTest.ChangeFilePermissionsAndWrite
ChromeOSMemoryPressureMonitorTest.CheckMemoryPressure
FileUtilTest.ChangeDirectoryPermissionsAndEnumerate
FileUtilTest.ChangeFilePermissionsAndRead
TaskSchedulerWorkerTest.BumpPriorityOfAliveThreadDuringShutdown

Turns out that all the file permission tests fail if we're running as root in the VM, but pass if we're running as chronos. Which is a little strange. I honestly don't know if the proper fix is to get the tests compatible with running as root, or not running them as root...

ChromeOSMemoryPressureMonitorTest.CheckMemoryPressure still fails however, which is also a little strange, since that test looks to be explicitly targeted for chromeos.
Blockedon: 844102
Should be straightforward to run as chronos using sudo -u chronos, right? That seems like a good approach?

I'm guessing the test memory pressure monitor is somehow interacting with the chromeos system memory pressure monitor.
Yeah, I think we'd want to add the option to run as non-root. We'd have to make sure that all the files pushed to the VM are readable/writable/executable as the new user. Might be best to add that to cros_run_vm_test. Maybe add an --as-user option?

And the memory pressure test is beyond me, so I filed bug 844102 for it.
Sure, sg. I'll take a look.
Project Member

Comment 37 by bugdroid1@chromium.org, May 18 2018

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infradata/config/+/61f8c2175d506421d95472da33de8764c2299dda

commit 61f8c2175d506421d95472da33de8764c2299dda
Author: Ben Pastene <bpastene@chromium.org>
Date: Fri May 18 00:44:29 2018

Project Member

Comment 38 by bugdroid1@chromium.org, May 18 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/2b1ce59023459ba15057ad15320094f73088c20e

commit 2b1ce59023459ba15057ad15320094f73088c20e
Author: Ben Pastene <bpastene@chromium.org>
Date: Fri May 18 17:42:29 2018

Enable the CrOS sanity test on the amd64-generic CQ bot as an experiment

Since it's 100% experiment, it won't block CLs if it fails. (In theory,
at least.)

https://chrome-internal-review.googlesource.com/c/infradata/config/+/628212
increases test pool size in preparation for this.

Bug:  832374 
Change-Id: Ie520e3cb75aec5c0df9ad497daa1cc72321c52f0
Reviewed-on: https://chromium-review.googlesource.com/1065198
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Reviewed-by: Achuith Bhandarkar <achuith@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Commit-Position: refs/heads/master@{#559943}
[modify] https://crrev.com/2b1ce59023459ba15057ad15320094f73088c20e/testing/buildbot/chromium.chromiumos.json
[modify] https://crrev.com/2b1ce59023459ba15057ad15320094f73088c20e/testing/buildbot/test_suites.pyl

Although... the sanity test is the first test on the bot to have a dependency on chrome itself, so now compile/test times have shot-up. The slave pool is a bit small for the new load:
http://shortn/_3OXNwpAJXr

I'll add some more bots.
Project Member

Comment 41 by bugdroid1@chromium.org, May 18 2018

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infradata/config/+/a9cc4d5256b4559ebef5cdf3749ef9d2314bea5f

commit a9cc4d5256b4559ebef5cdf3749ef9d2314bea5f
Author: Ben Pastene <bpastene@chromium.org>
Date: Fri May 18 20:15:30 2018

Looks like the sanity test (like telemetry) can hang indefinitely every now and then without any output and eventually time out:
https://chromium-swarm.appspot.com/task?id=3d8d95ebeb998410
https://chromium-swarm.appspot.com/task?id=3d8da9134094f510
https://chromium-swarm.appspot.com/task?id=3d8dd43c1f4c3910

Managed to hop on a bot while it was doing this. SSH'ed into the vm and the sanity test was alive. strace'ing it seemed to indicate that it was doing *something*, but I'm not the best at deciphering syscalls. The chrome logs had some interesting errors tho:

localhost ~ # cat /var/log/chrome/chrome | grep ERROR
[2635:2635:0518/164212.961158:ERROR:vaapi_wrapper.cc(313)] vaInitialize failed: unknown libva error
[2589:2589:0518/164213.027230:ERROR:input_method_manager_impl.cc(1080)] IMEEngine for "fgoepimhcoialccpbmpnnblemnepkkao" is not registered
[2635:2635:0518/164213.055287:ERROR:sandbox_linux.cc(378)] InitializeSandbox() called with multiple threads in process gpu-process. This error can be safely ignored in VMTests.
[2589:2589:0518/164213.475956:ERROR:object_proxy.cc(616)] Failed to call method: org.chromium.PowerManager.GetScreenBrightnessPercent: object_path= /org/chromium/PowerManager: org.freedesktop.DBus.Error.NotSupported: Backlight uninitialized
[2635:2651:0518/164214.303108:ERROR:drm_device.cc(751)] CTM is empty. Expected a 3x3 matrix.
[2635:2651:0518/164214.303918:ERROR:drm_display.cc(181)] Failed to set color correction for display: crtc_id = 27

Looks like most of those are display/screen related things, so maybe not relevant? The task timed out before I could dig in further, so I'll bump the timeout and give it another shot the next time I find one.
This style of test (though not this exact one) runs in the CQ/PFQ/waterfall hundreds of times a day, so it ought to be stable. 

The chrome errors are all bogus unfortunately :/ We should know more about what's going on with streaming output once I've enabled that for vm_sanity. Unfortunately that CL is stuck behind the refactor CL; hopefully we can get both squared away soon.
Noted. It's a very small subset of our test runs that seem to hang (< 1%). And it's not just confined to the sanity test, telemetry_perf_unittests can sometimes hang in the exact same way:
https://chromium-swarm.appspot.com/task?id=3d9c1da69e0f4110
https://chromium-swarm.appspot.com/task?id=3d9bb87871cf8d10

Still keeping an eye out for an reoccurrence.
Another thing that we can do is add a retry (or more than 1 retry) to the test which will almost certainly mask the problem. It does improve the possibility of flaky failures going through, and not detecting an increase in the flakiness of this particular phenomenon, etc.
Found a hung telemetry test. From scanning the process tree of the test on host, it looked like it was trying to restart the ui on the VM. The ui logs on the VM has an interesting looking stack trace. I also found a hung sanity test, and its ui logs had the same stack trace. Seems to indicate that this might be relevant. Any ideas?



localhost ~ # cat /var/log/ui/ui.LATEST 
libva error: va_getDriverName() failed with unknown libva error,driver_name=(null)
[2518:2518:0523/102800.481656:ERROR:vaapi_wrapper.cc(313)] vaInitialize failed: unknown libva error
libEGL warning: MESA-LOADER: failed to retrieve device information

libEGL warning: DRI2: failed to open vgem (search paths /usr/lib64/dri)
libEGL warning: MESA-LOADER: failed to retrieve device information

libEGL warning: DRI2: failed to open virtio_gpu (search paths /usr/lib64/dri)
libEGL warning: No hardware driver found, falling back to software rendering
[2518:2518:0523/102800.592065:ERROR:sandbox_linux.cc(378)] InitializeSandbox() called with multiple threads in process gpu-process. This error can be safely ignored in VMTests.
[2477:2477:0523/102800.650649:ERROR:input_method_manager_impl.cc(1080)] IMEEngine for "fgoepimhcoialccpbmpnnblemnepkkao" is not registered

DevTools listening on ws://127.0.0.1:37832/devtools/browser/c9ba565b-b9d5-4adf-9b2c-771827a848b5
device-enumerator: scan all dirs
  device-enumerator: scanning /sys/bus
  device-enumerator: scanning /sys/class
device-enumerator: scan all dirs
  device-enumerator: scanning /sys/bus
  device-enumerator: scanning /sys/class
[2477:2477:0523/102801.065581:ERROR:object_proxy.cc(616)] Failed to call method: org.chromium.PowerManager.GetScreenBrightnessPercent: object_path= /org/chromium/PowerManager: org.freedesktop.DBus.Error.NotSupported: Backlight uninitialized
device-enumerator: scan all dirs
  device-enumerator: scanning /sys/bus
  device-enumerator: scanning /sys/class
[2518:2532:0523/102802.205807:ERROR:drm_device.cc(751)] CTM is empty. Expected a 3x3 matrix.
[2518:2532:0523/102802.205924:ERROR:drm_display.cc(181)] Failed to set color correction for display: crtc_id = 27
[2477:2477:0523/102811.093821:ERROR:object_proxy.cc(616)] Failed to call method: org.chromium.CryptohomeInterface.RemoveFirmwareManagementParameters: object_path= /org/chromium/Cryptohome: org.freedesktop.DBus.Error.ServiceUnknown: The name org.chromium.Cryptohome was not provided by any .service files
[2477:2477:0523/102811.094524:FATAL:optional.h(583)] Check failed: storage_.is_populated_. 
#0 0x5939bb43c1ec <unknown>
#1 0x5939bb3c8f80 <unknown>
#2 0x5939ba532cb8 <unknown>
#3 0x5939ba51da35 <unknown>
#4 0x5939bbdb51a7 <unknown>
#5 0x5939b996f24e <unknown>
#6 0x5939bbdd2d88 <unknown>
#7 0x5939bbdd5853 <unknown>
#8 0x5939bbdd3156 <unknown>
#9 0x5939bbdd60b1 <unknown>
#10 0x5939bb412839 <unknown>
#11 0x5939bb412a72 <unknown>
#12 0x5939bb3b6fcc <unknown>
#13 0x5939bb3ce997 <unknown>
#14 0x5939bb3cee77 <unknown>
#15 0x5939bb44cf09 <unknown>
#16 0x5939bb3eec65 <unknown>
#17 0x5939bb0c7858 <unknown>
#18 0x5939b9a7b574 <unknown>
#19 0x5939b9a7dde2 <unknown>
#20 0x5939b9a77bc1 <unknown>
#21 0x5939bb0b5654 <unknown>
#22 0x5939bb0be2ca <unknown>
#23 0x5939bb0b3511 <unknown>
#24 0x5939b8f87b1f <unknown>
#25 0x782cb0134736 __libc_start_main
#26 0x5939b8f87949 <unknown>

Received signal 6
#0 0x5939bb43c1ec <unknown>
#1 0x5939bb43bd51 <unknown>
#2 0x782cb0d622e0 <unknown>
#3 0x782cb0147dd2 gsignal
#4 0x782cb0149bf6 abort
#5 0x5939bb43aae5 <unknown>
#6 0x5939bb3c93b6 <unknown>
#7 0x5939ba532cb8 <unknown>
#8 0x5939ba51da35 <unknown>
#9 0x5939bbdb51a7 <unknown>
#10 0x5939b996f24e <unknown>
#11 0x5939bbdd2d88 <unknown>
#12 0x5939bbdd5853 <unknown>
#13 0x5939bbdd3156 <unknown>
#14 0x5939bbdd60b1 <unknown>
#15 0x5939bb412839 <unknown>
#16 0x5939bb412a72 <unknown>
#17 0x5939bb3b6fcc <unknown>
#18 0x5939bb3ce997 <unknown>
#19 0x5939bb3cee77 <unknown>
#20 0x5939bb44cf09 <unknown>
#21 0x5939bb3eec65 <unknown>
#22 0x5939bb0c7858 <unknown>
#23 0x5939b9a7b574 <unknown>
#24 0x5939b9a7dde2 <unknown>
#25 0x5939b9a77bc1 <unknown>
#26 0x5939bb0b5654 <unknown>
#27 0x5939bb0be2ca <unknown>
#28 0x5939bb0b3511 <unknown>
#29 0x5939b8f87b1f <unknown>
#30 0x782cb0134736 __libc_start_main
#31 0x5939b8f87949 <unknown>
  r8: 0000000000000000  r9: 0000782cafc0c740 r10: 0000000000000008 r11: 0000000000000202
 r12: 00007ffe1f3ab459 r13: 00007ffe1f3aba00 r14: 00007ffe1f3ab490 r15: 00007ffe1f3ab458
  di: 00000000000009ad  si: 00000000000009ad  bp: 00007ffe1f3ab420  bx: 00007ffe1f3ab490
  dx: 0000000000000006  ax: 0000000000000000  cx: 0000782cb0147dd2  sp: 00007ffe1f3ab2f8
  ip: 0000782cb0147dd2 efl: 0000000000000202 cgf: 0000000000000033 erf: 0000000000000000
 trp: 0000000000000000 msk: 0000000000000000 cr2: 0000000000000000
[end of stack trace]
Calling _exit(1). Core file will not be generated.
libva error: va_getDriverName() failed with unknown libva error,driver_name=(null)
[2762:2762:0523/102811.781412:ERROR:vaapi_wrapper.cc(313)] vaInitialize failed: unknown libva error
libEGL warning: MESA-LOADER: failed to retrieve device information

libEGL warning: DRI2: failed to open vgem (search paths /usr/lib64/dri)
libEGL warning: MESA-LOADER: failed to retrieve device information

libEGL warning: DRI2: failed to open virtio_gpu (search paths /usr/lib64/dri)
libEGL warning: No hardware driver found, falling back to software rendering
[2762:2762:0523/102811.899753:ERROR:sandbox_linux.cc(378)] InitializeSandbox() called with multiple threads in process gpu-process. This error can be safely ignored in VMTests.
[2709:2709:0523/102811.914590:ERROR:input_method_manager_impl.cc(1080)] IMEEngine for "fgoepimhcoialccpbmpnnblemnepkkao" is not registered

DevTools listening on ws://127.0.0.1:33163/devtools/browser/1425a412-1baf-47b6-afa4-e2d31afa0dc9
device-enumerator: scan all dirs
  device-enumerator: scanning /sys/bus
  device-enumerator: scanning /sys/class
device-enumerator: scan all dirs
  device-enumerator: scanning /sys/bus
  device-enumerator: scanning /sys/class
[2709:2709:0523/102812.633663:ERROR:object_proxy.cc(616)] Failed to call method: org.chromium.PowerManager.GetScreenBrightnessPercent: object_path= /org/chromium/PowerManager: org.freedesktop.DBus.Error.NotSupported: Backlight uninitialized
[2762:2776:0523/102813.836394:ERROR:drm_device.cc(751)] CTM is empty. Expected a 3x3 matrix.
[2762:2776:0523/102813.836512:ERROR:drm_display.cc(181)] Failed to set color correction for display: crtc_id = 27
device-enumerator: scan all dirs
  device-enumerator: scanning /sys/bus
  device-enumerator: scanning /sys/class
Also, I wonder if not stripping the symbols when deploying chrome to the VM would help with the unsymbolized stack trace. 'Cause all those "unknowns" aren't very helpful.
The most useful thing would be to know where exactly in the sequence we're failing, and we won't know that without the stdout of vm_sanity :/

You can use --no-strip, but I believe the binary becomes enormous. We don't have space on the root partition for this binary, but I think deploy knows how to copy the file to a different partition and set up the symlinks. You can try it? 
Project Member

Comment 49 by bugdroid1@chromium.org, May 25 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/3bd6ea8c07ab4ae998354ed0d07615efbe21129c

commit 3bd6ea8c07ab4ae998354ed0d07615efbe21129c
Author: Ben Pastene <bpastene@chromium.org>
Date: Fri May 25 19:06:47 2018

Bump execution and IO timeout to 60min on cros vm FYI tester.

This'll give me more time to debug hung VM tasks. As it is, 20min isn't
really enough time for my ssh'ing to yield much results.

TBR=dpranke@chromium.org

Bug:  832374 
Change-Id: I58f9b065190c1029c495756d914304abcca7b5a0
Reviewed-on: https://chromium-review.googlesource.com/1070450
Reviewed-by: Ben Pastene <bpastene@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Commit-Position: refs/heads/master@{#561963}
[modify] https://crrev.com/3bd6ea8c07ab4ae998354ed0d07615efbe21129c/testing/buildbot/chromium.fyi.json
[modify] https://crrev.com/3bd6ea8c07ab4ae998354ed0d07615efbe21129c/testing/buildbot/waterfalls.pyl

It looks like the flake rate of the sanity test jumped significantly sometime over the weekend:
https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/chromeos-amd64-generic-rel?limit=200

Previously it was failing around 1% of tests, but now it's around 15-20%. It seems to be generally caused by browser/devtools startup timeouts. I wonder what changed...
Yup, and I've seen more than one failure stack.
Using the nostrip + mount option like you suggested, I managed to get a more symbolized version of the trace in #46:

[2695:2695:0529/180758.401359:ERROR:object_proxy.cc(616)] Failed to call method: org.chromium.CryptohomeInterface.RemoveFirmwareManagementParameters: object_path= /org/chromium/Cryptohome: org.freedesktop.DBus.Error.ServiceUnknown: The name org.chromium.Cryptohome was not provided by any .service files
[2695:2695:0529/180758.401911:FATAL:optional.h(583)] Check failed: storage_.is_populated_. 
#0 0x557cbc98070c base::debug::StackTrace::StackTrace()
#1 0x557cbc90e930 logging::LogMessage::~LogMessage()
#2 0x557cbba60bd8 chromeos::AutoEnrollmentController::OnFirmwareManagementParametersRemoved()
#3 0x557cbba4b805 _ZN4base8internal7InvokerINS0_9BindStateIMN8chromeos26EasyUnlockGetKeysOperationEFvNS_8OptionalIN10cryptohome9BaseReplyEEEEJNS_7WeakPtrIS4_EEEEEFvS8_EE7RunOnceEPNS0_13BindStateBaseEOS8_
#4 0x557cbd2dcf57 chromeos::(anonymous namespace)::CryptohomeClientImpl::OnBaseReplyMethod()
#5 0x557cbae9473e _ZN4base8internal7InvokerINS0_9BindStateIMN8download19InProgressCacheImplEFvNS_12OnceCallbackIFvvEEERKNSt3__16vectorIcNS8_9allocatorIcEEEEEJNS_7WeakPtrIS4_EES7_EEEFvSE_EE7RunOnceEPNS0_13BindStateBaseESE_
#6 0x557cbd2fab68 dbus::ObjectProxy::OnCallMethod()
#7 0x557cbd2fd633 _ZN4base8internal7InvokerINS0_9BindStateIMN4dbus11ObjectProxyEFvRKNSt3__112basic_stringIcNS5_11char_traitsIcEENS5_9allocatorIcEEEESD_NS_12OnceCallbackIFvPNS3_8ResponseEEEESG_PNS3_13ErrorResponseEEJ13scoped_refptrIS4_ESB_SB_SI_EEEFvSG_SK_EE7RunOnceEPNS0_13BindStateBaseESG_SK_
#8 0x557cbd2faf36 dbus::ObjectProxy::RunResponseOrErrorCallback()
#9 0x557cbd2fde91 _ZN4base8internal7InvokerINS0_9BindStateIMN4dbus11ObjectProxyEFvNS4_19ReplyCallbackHolderENS_9TimeTicksEPNS3_8ResponseEPNS3_13ErrorResponseEEJ13scoped_refptrIS4_ES5_S6_S8_SA_EEEFvvEE7RunOnceEPNS0_13BindStateBaseE
#10 0x557cbc958419 base::(anonymous namespace)::PostTaskAndReplyRelay::RunTaskAndPostReply()
#11 0x557cbc958652 _ZN4base8internal7InvokerINS0_9BindStateIPFvNS_12_GLOBAL__N_121PostTaskAndReplyRelayEEJS4_EEEFvvEE7RunOnceEPNS0_13BindStateBaseE
#12 0x557cbc8fc97c base::debug::TaskAnnotator::RunTask()
#13 0x557cbc914357 base::MessageLoop::RunTask()
#14 0x557cbc914837 base::MessageLoop::DoWork()
#15 0x557cbc991429 base::MessagePumpLibevent::Run()
#16 0x557cbc9346c5 base::RunLoop::Run()
#17 0x557cbc607ac8 ChromeBrowserMainParts::MainMessageLoopRun()
#18 0x557cbafa1174 content::BrowserMainLoop::RunMainMessageLoopParts()
#19 0x557cbafa3a72 content::BrowserMainRunnerImpl::Run()
#20 0x557cbaf9d761 content::BrowserMain()
#21 0x557cbc5f58a4 content::ContentMainRunnerImpl::Run()
#22 0x557cbc5fe51a service_manager::Main()
#23 0x557cbc5f3761 content::ContentMain()
#24 0x557cba4a449f ChromeMain
#25 0x7c86dcf44736 __libc_start_main
#26 0x557cba4a42c9 _start

Looks like OnFirmwareManagementParametersRemoved() is dereferencing a null/empty pointer. Poking around, I found https://chromium-review.googlesource.com/c/chromium/src/+/1072718 which landed earlier today and removed a pointer dereference in that method. Maybe that'll help things? Let's wait and see...
Status: Fixed (was: Assigned)
Prob fair to say that this bug has outgrown its original intent.

The fyi bot (where I'm putting tests that I'm still working on) is up and running at:
https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/chromeos-amd64-generic-rel-vm-tests

So I'll close this out and file other bugs to track all the issues I've mentioned previously above. 
Project Member

Comment 54 by bugdroid1@chromium.org, Jun 11 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/cdd784010f13bcf6bda05887ef458eb3d937631c

commit cdd784010f13bcf6bda05887ef458eb3d937631c
Author: Ben Pastene <bpastene@chromium.org>
Date: Mon Jun 11 20:44:34 2018

cros_run_vm_test: Add option that runs the test as user chronos.

Some of chromium's tests fail if the test is run as root, but pass if they
run as a less-privileged user. This adds an option to run the test as chronos.

This temporarily changes the username field in the RemoteDevice:
https://codesearch.chromium.org/chromium/src/third_party/chromite/lib/remote_access.py?rcl=a05ffccfb494de4c72d52b13b3bc53bdcf81a562&l=271
And changes it back to root after the test.

BUG= chromium:832374 
TEST=./bin/cros_run_vm_test --start --as-chronos --cmd -- "whoami ; whoami && whoami"

Change-Id: I46ee3fe0f7c2c800483cc2392cc8cc795621e243
Reviewed-on: https://chromium-review.googlesource.com/1079644
Commit-Ready: Ben Pastene <bpastene@chromium.org>
Tested-by: Ben Pastene <bpastene@chromium.org>
Reviewed-by: Achuith Bhandarkar <achuith@chromium.org>

[modify] https://crrev.com/cdd784010f13bcf6bda05887ef458eb3d937631c/lib/remote_access.py
[modify] https://crrev.com/cdd784010f13bcf6bda05887ef458eb3d937631c/scripts/cros_run_vm_test.py

Sign in to add a comment