Gracefully skip or abort peerd_* autotests on devices that do not have peerd |
||
Issue descriptionRelated issues: b/77830981, b/35561534 The peerd_* autotests (peerd_AdvertiseServices, peerd_DiscoverServices) are only supported on devices that have peerd installed, which does not include most Chrome OS devices. When run on devices without peerd, the tests fail with obscure error messages: ERROR: Command <start peerd PEERD_LOG_LEVEL=3> failed, rc=1, Command returned non-zero exit status peerd is included in the test image by autotest-tests-peerd, which is only a dependency of chromeos-base/autotest-all when USE=peerd is set. However, these autotests are part of suite:smoke, and as such, it is often that folks try to run them. Can we do some combination of: (a) remove these tests from the generic suite:smoke in favor of a more specific "peerd" test suite. (b) have the autotests check at runtime whether they are supposed to run, and if not fail gracefully. Assigning over to benchan for thoughts.
,
Apr 13 2018
I agree. In my opinion all tests that test a feature that does not exist on a target should spew a warning and pass, not fail cryptically. This requires the test writer to a) determine if the feature should be present on the target b) test the feature Usually only b) is implemented by CrOS developers causing problems when tests are run outside of the hardware lab where images are carefully paired with matching autotest bundles. Notice CTS has solved this issue precisely as described above: two separate checks, a) using the support of the build system which leaves build information - in this case the peerd flag - on the DUT's image. Also adding Dan, as he has been looking at adding this step automatically to the tast framework, so the flag/check might already be available on DUTs.
,
Apr 13 2018
My plan for Tast was/is to make the test framework provide a way for tests to declare system image dependencies in the form of USE flags. In other words, peerd-related tests would declare that they require the peerd USE flag to be set, and those tests would be automatically skipped on boards that don't have the flag. |
||
►
Sign in to add a comment |
||
Comment 1 by djkurtz@chromium.org
, Apr 13 2018