Avoid running non-Chrome autotests in Chrome PFQ |
||||||||||||
Issue descriptionWe have bvt-inline, bvt-cq, smoke, etc. The pfq builders currently run bvt-cq and in some cases smoke. The pfq builders don't need to verify chromeos - we should only run autotests that have a dependency on chrome. I propose a new suite bvt-pfq that contains the set of bvt-cq tests that depend on chrome. It's unlikely that this will improve flakiness much since the non-chrome tests are pretty stable, but it will save time. There are currently about 60 tests in bvt-cq: https://cs.corp.google.com/search/?q=package:%5Echromeos_public$+file:control+suite:bvt-cq
,
Apr 26 2016
Sounds reasonable. Can we call it bvt-chrome instead? The term pfq is both ambiguous and outdated, I'd like to phase it out.
,
Apr 26 2016
We are pretty strongly tied to 'pfq' for better or for worse. That said, I am OK with bvt-chrome, the meaning is pretty clear.
,
Apr 26 2016
,
Apr 26 2016
,
May 3 2016
achuith@ can you find someone on the UI team to do this?
,
May 4 2016
,
Jul 19 2016
,
May 11 2017
,
May 11 2017
Yes, please.
,
May 11 2017
I'll spend some more time looking at this soon. I think I have consensus on how to do it now. Unfortunately, it requires some work on the way that we define suites. Some docs: https://docs.google.com/document/d/17weKafY3tuLHi1q_6S3bYeu7vFqiZLnUGWCmO1cKqlQ/edit#heading=h.bf2t7w9czh0z https://docs.google.com/document/d/100tJ42pTExpjoK0q-aNBWcJtJdJS9gRDddMal5XO0gY/edit#heading=h.r24rz68abqia
,
May 11 2017
+zel since we were talking about this in the hallway yesterday.
,
May 15 2017
,
May 25 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/f748eb2b874845706a71a3db78702b9618c7ae0f commit f748eb2b874845706a71a3db78702b9618c7ae0f Author: Daniel Erat <derat@chromium.org> Date: Thu May 25 13:00:56 2017 autotest: Add desktopui_ChromeSanity client test. Add desktopui_ChromeSanity, a basic integration test for Chrome. It checks the following: - login works - session_manager emits SessionStateChanged signal - the user's cryptohome is mounted - a simple web page can be loaded BUG=chromium:606600 TEST=test passes Change-Id: I66ad02334d00144514b96e0714806dae61ca73ad Reviewed-on: https://chromium-review.googlesource.com/436766 Tested-by: Dan Erat <derat@chromium.org> Trybot-Ready: Dan Erat <derat@chromium.org> Reviewed-by: Ilja H. Friedel <ihf@chromium.org> [add] https://crrev.com/f748eb2b874845706a71a3db78702b9618c7ae0f/client/site_tests/desktopui_ChromeSanity/control [add] https://crrev.com/f748eb2b874845706a71a3db78702b9618c7ae0f/client/site_tests/desktopui_ChromeSanity/desktopui_ChromeSanity.py [add] https://crrev.com/f748eb2b874845706a71a3db78702b9618c7ae0f/client/site_tests/desktopui_ChromeSanity/test.html
,
May 26 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/e74afec585327833a3a362eb723a4684ac8069b1 commit e74afec585327833a3a362eb723a4684ac8069b1 Author: Daniel Erat <derat@chromium.org> Date: Fri May 26 16:47:47 2017 autotest-chrome: Add desktopui_ChromeSanity test. BUG=chromium:606600 TEST=none Change-Id: I0eea4c62813e136e2b8b256fac4967e6342317b7 Reviewed-on: https://chromium-review.googlesource.com/515212 Commit-Ready: Dan Erat <derat@chromium.org> Tested-by: Dan Erat <derat@chromium.org> Reviewed-by: Ilja H. Friedel <ihf@chromium.org> [modify] https://crrev.com/e74afec585327833a3a362eb723a4684ac8069b1/chromeos-base/autotest-chrome/autotest-chrome-9999.ebuild
,
Jan 26 2018
I don't think we ever reached full consensus about the right way to do this (see docs in #11). Our current method of defining suites via e.g. "suite:bvt-inline" test attributes makes this very difficult to do, and my efforts to rework Autotest to use attribute expressions (issue 714865) were stymied by the separate code paths for running suites in the lab vs. when using test_that (and me not understanding anything about the lab :-/). (aside: I've kept this goal in mind when working on Tast, and in fact define suites there using boolean expressions. See https://chromium.googlesource.com/chromiumos/platform/tast/+/HEAD/docs/running_tests.md#specifying-which-tests-to-run and https://chromium.googlesource.com/chromiumos/platform/tast/+/HEAD/docs/test_attributes.md ) |
||||||||||||
►
Sign in to add a comment |
||||||||||||
Comment 1 by jrbarnette@chromium.org
, Apr 26 2016