vboot_reference doesn't run all unit tests |
|||||
Issue descriptionI just noticed that the vboot_reference package doesn't run all unit tests by default. When I do FEATURES=test emerge-falco vboot_reference it doesn't run the tests under the runtestscripts target (like run_cgpt_tests.sh). When I instead do USE=-minimal FEATURES=test emerge-falco vboot_reference it runs them just fine and fails because there's an existing bug in test_sign_keyblocks.sh. Now, I get that there might be reasons to not run all tests under all circumstances, but the fact that we have broken code checked in suggests that no part of the CQ or canaries ever runs these at all. Is that WAI? If not, how is the whole USE=minimal thing intended to work and how do we fix this?
,
Oct 20 2016
+Stefan for triage.
,
Oct 21 2016
Daisuke's already started fixing this, in https://chromium-review.googlesource.com/396462 Daisuke, can we just expand that to run all the unit tests? Is there a downside?
,
Oct 26 2016
I don't know the history of this 'MINIMAL' and don't think I'm the right person fixing it. 'util' target builds x86 binaries for x86 boards and ARM binaries for ARM boards. So, I think those unit tests using utilities shouldn't run on a build server. It's probably a good idea to run board-agnostic test on a build server: FEATURES=test emerge vboot_reference That should run all unit tests. If it's not possible (without a big infra change), we may as well run the unit tests on each board but that'll force test only binaries and data to be installed on each board.
,
Oct 26 2016
vboot_reference has a lot of stuff in it. MINIMAL is set when building things to be installed on a Chromebook, because it doesn't need everything. It looks like a bunch of utility/ tools are all that are left off now, but once upon a time there were a bunch of additional things too (bitmap editors, etc). The other side-effect of MINIMAL is that the "make install" target needs to put things in different places. For example, the make_dev_ssh.sh script goes into /usr/share/vboot/bin on Chromebooks, but can just be put in /usr/bin/ for the chroot or into ./build/install_for_test/bin/ when running "make runtests".
,
Oct 27 2016
I'm pretty sure we have ways of running non-x86 unittests on build servers (I think it "just works" if you configure it, it will run qemu for you and everything). It might not be necessary to run all these tests for all boards, but it wouldn't really hurt either. Right now they're simply run nowhere which is a problem. I think we have a chroot paladin as well (https://uberchromegw.corp.google.com/i/chromiumos/builders/chromiumos-sdk), so if we could get them to just run on there that would probably be good enough as well.
,
Nov 2
|
|||||
►
Sign in to add a comment |
|||||
Comment 1 by wfrichar@chromium.org
, Apr 21 2016