Mirror some (all?) of chrome's tests ran during the chromeos-chrome ebuild onto Chrome's CQ |
||
Issue descriptionCrOS's chrome ebuild optionally includes building (and presumably running later in the build) some of chromium's tests: https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/eac472a94ae05b36ace11282f2c1cef8bbcc81ba/chromeos-base/chromeos-chrome/chromeos-chrome-9999.ebuild#751 As part of bug 732531, we'd like to catch browser breakages on CrOS earlier (ie: on chromium's CQ/waterfalls). We've set up a bot on chromium's CQ running browser tests in cros VMs: https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/chromeos-amd64-generic-rel Presumably, the suites ran in the chromeos-chrome ebuild were added because they were especially good at catching browser regressions on cros. Consequently, they'd be a good candidate for the VM bot on chromium's CQ. I'll look into adding them. + some people that have added tests to the chrome-chromeos ebuild in the past
,
Jul 16
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/39d823d709c6cd01eb8647e6aa466eb9e6171520 commit 39d823d709c6cd01eb8647e6aa466eb9e6171520 Author: Ben Pastene <bpastene@chromium.org> Date: Mon Jul 16 23:07:42 2018 Add test jpg files to jpeg_decode_accelerator_unittest's data. I'd like to run this test (and others) on chromeos bots on chrome's CQ. Chrome's test dependency management works a little different than cros's/autotest's. We push to the cros device only the test target's runtime deps (as determined by GN). Consequently, we get ENOENTs due to these missing images. This'll fix that. Bug: 860066 Cq-Include-Trybots: luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel Change-Id: I9b2fc1deed0bbefcfa9fccc51969571b8056f30e Reviewed-on: https://chromium-review.googlesource.com/1138752 Reviewed-by: Daniele Castagna <dcastagna@chromium.org> Commit-Queue: Ben Pastene <bpastene@chromium.org> Cr-Commit-Position: refs/heads/master@{#575465} [modify] https://crrev.com/39d823d709c6cd01eb8647e6aa466eb9e6171520/media/gpu/BUILD.gn
,
Jul 17
All of the video/image encode/decode suites don't seem to work on VMs. Things fail with "unknown libva error" and "Failed creating a VDA". It looks like their autotest equivalents depend on what appear to be hardware capabilities: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/master/client/site_tests/video_VideoDecodeAccelerator/control.h264#37 Not surprising then that they fail on VMs. I tried running them on a real dut and things seemed to work fine there. Logistically, we'd have trouble putting cros hardware on chrome's CQ, so those suites will probably have to run on post-commit/informational dut testers. Not quite the CQ, but should still catch breakages relatively quick.
,
Jul 17
Yup, these tests have a hardware dependency. There may be some subset we could run in the VM, or modifications we can make to these tests, but I don't think it's worth it (Ilja may have some thoughts). And there is value in these tests running in the chrome CQ because we do see chrome gfx/video breakages, but this isn't low-hanging fruit. We already have continuous informational builders in the chromeos-chrome view that gardeners monitor so probably additional builders doing the same thing is not that useful?
,
Jul 17
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/7d5467abddf0d2bcb7079874ab6bd80cb293d7c2 commit 7d5467abddf0d2bcb7079874ab6bd80cb293d7c2 Author: Ben Pastene <bpastene@chromium.org> Date: Tue Jul 17 23:51:01 2018 Add media_unittests to cros VM tester. Some tests are failing, so this adds them to a filter file and adds it in a la fuchsia. Bug: 860066, 840967 Change-Id: I6ebe67a0d4a5ab5e58c543524d36ada6ae55219f Reviewed-on: https://chromium-review.googlesource.com/1136884 Reviewed-by: John Budorick <jbudorick@chromium.org> Commit-Queue: Ben Pastene <bpastene@chromium.org> Cr-Commit-Position: refs/heads/master@{#575846} [modify] https://crrev.com/7d5467abddf0d2bcb7079874ab6bd80cb293d7c2/build/config/chromeos/rules.gni [modify] https://crrev.com/7d5467abddf0d2bcb7079874ab6bd80cb293d7c2/testing/buildbot/chromium.chromiumos.json [modify] https://crrev.com/7d5467abddf0d2bcb7079874ab6bd80cb293d7c2/testing/buildbot/filters/BUILD.gn [add] https://crrev.com/7d5467abddf0d2bcb7079874ab6bd80cb293d7c2/testing/buildbot/filters/chromeos.media_unittests.filter [modify] https://crrev.com/7d5467abddf0d2bcb7079874ab6bd80cb293d7c2/testing/buildbot/test_suites.pyl
,
Jul 19
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/ba2572a6bdfb93aff61c9133464eec544a571d71 commit ba2572a6bdfb93aff61c9133464eec544a571d71 Author: Ben Pastene <bpastene@chromium.org> Date: Thu Jul 19 17:21:31 2018 Add sandbox_linux_unittests to cros VM bot. Bug: 860066, 840967 Change-Id: I5dbc65b55d423f7a79de63dcdf6c6f5dd36c463b Reviewed-on: https://chromium-review.googlesource.com/1137439 Reviewed-by: John Budorick <jbudorick@chromium.org> Commit-Queue: Ben Pastene <bpastene@chromium.org> Cr-Commit-Position: refs/heads/master@{#576535} [modify] https://crrev.com/ba2572a6bdfb93aff61c9133464eec544a571d71/testing/buildbot/chromium.chromiumos.json [modify] https://crrev.com/ba2572a6bdfb93aff61c9133464eec544a571d71/testing/buildbot/test_suites.pyl
,
Jul 19
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/74ad5377d3cbe059f589b388deea6a077d8e2c2d commit 74ad5377d3cbe059f589b388deea6a077d8e2c2d Author: Ben Pastene <bpastene@chromium.org> Date: Thu Jul 19 17:21:35 2018 mb: Add support for modifying a simplechrome builder's GN args. And utilize it to add the arg "ozone_platform_headless=true" to the fyi tester and run ozone_gl_unittests. Context: There are a subset of tests that prefer to run with the "headless" ozone platform (see http://crbug.com/859246#c1). The chrome-chromeos ebuild (and by association, the cros chrome-sdk) whitelists the ozone platforms that are supported: https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/45c432cbf1fac8cf655dd5c28c543097fdaefd43/chromeos-base/chromeos-chrome/chromeos-chrome-9999.ebuild#356 For the amd64-generic board, it turns off auto_platform detection, and whitelists only GBM. Consequently, using those GN args it generates, we can't run in headless mode on our bots. So we need to whitelist 'headless' mode. This is done by adding the "ozone_platform_headless=true" GN arg. We can do this by either: - Adding it to the ebuild. Or - Making MB add it during "mb.py gen" time. This chooses the latter option because the ebuild's configuration is (FWIU) optimized for building a production grade version of the Chrome binary on ChromeOS. The 'headless' mode of ozone is (also FWIU) more of debugging/test mode and isn't suitable for prod builds of Chrome. So barring any "debug" mode of an ebuild that I'm not aware of, this goes with the mb option. Bug: 860066 Change-Id: I07234c67c2e493375525da227b8754de2a20fdb7 Reviewed-on: https://chromium-review.googlesource.com/1142740 Commit-Queue: Ben Pastene <bpastene@chromium.org> Reviewed-by: Dirk Pranke <dpranke@chromium.org> Cr-Commit-Position: refs/heads/master@{#576536} [modify] https://crrev.com/74ad5377d3cbe059f589b388deea6a077d8e2c2d/testing/buildbot/chromium.fyi.json [modify] https://crrev.com/74ad5377d3cbe059f589b388deea6a077d8e2c2d/testing/buildbot/test_suites.pyl [modify] https://crrev.com/74ad5377d3cbe059f589b388deea6a077d8e2c2d/tools/mb/mb.py [modify] https://crrev.com/74ad5377d3cbe059f589b388deea6a077d8e2c2d/tools/mb/mb_config.pyl |
||
►
Sign in to add a comment |
||
Comment 1 by bpastene@chromium.org
, Jul 16