Test platform_CUPSDaemon seems to be in the wrong suite
Reported by
jrbarnette@chromium.org,
Oct 21 2016
|
|||||||
Issue description
The test platform_CUPSDaemon is in the bvt-inline suite.
A cursory glance at the purpose of this test suggests that's
the wrong suite, and that bvt-cq would be preferable.
Here are the relevant criteria for a test in bvt-inline
(taken from test_suites/control.bvt-inline in the Autotest
sources):
2. A test failure should indicate one or more of the following
impacts is possible:
* The failure may impact the stability of the test lab.
* The failure may prevent discovery of other bugs.
* The failure may block ordinary development tasks.
That is, bvt-inline is meant to find major test-stoppers.
The focus is on login and security tests. Operationally,
a failure in bvt-inline is meant to trigger engagement
from the sheriffs. Failures in this suite also block
all follow-on testing of other features; the build is
classified as DOA.
Here are the relevant criteria for a test in bvt-cq
(taken from test_suites/control.bvt-cq in the Autotest
sources):
2. A test failure should indicate that the product is not fit for
release on any channel.
The bvt-cq test is meant to find bugs in specific subsystems
that while critical, don't prevent using the system for most
testing or development purposes. CLs must pass bvt-cq to be
accepted into the tree, but failures in the canary should
generally be treated as "file a bug against the failing subsystem."
,
Sep 14 2017
,
Sep 14 2017
Presumably kalin@ will be modifying this? https://chromium-review.googlesource.com/c/chromiumos/third_party/autotest/+/666379
,
Sep 15 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/c912f30aee89f7141cee7680228a6d6d837da93f commit c912f30aee89f7141cee7680228a6d6d837da93f Author: Kalin Stoyanov <kalin@chromium.org> Date: Fri Sep 15 03:14:14 2017 Move platform_CUPSDaemon to bvt-cq BUG= chromium:658090 TEST=Test is running successfully now on the bvt-inline suite Change-Id: Ifc3956e7832e705c2edbddc8c2f2cee054aec83d Reviewed-on: https://chromium-review.googlesource.com/666379 Commit-Ready: Kalin Stoyanov <kalin@chromium.org> Tested-by: Kalin Stoyanov <kalin@chromium.org> Reviewed-by: Sean Kau <skau@chromium.org> Reviewed-by: Brian Norris <briannorris@chromium.org> [modify] https://crrev.com/c912f30aee89f7141cee7680228a6d6d837da93f/client/site_tests/platform_CUPSDaemon/control
,
Sep 15 2017
,
Nov 10 2017
From test health - it is still flaky (race condition) - https://cros-goldeneye.corp.google.com/chromeos/healthmonitoring/testDetails?testName=platform_CUPSDaemon&suite=&daysBack=30&board=&architecture=&boardFamily=&buildConfig=release&reason=&version=&milestone=64&dut= and it already affects CQ - https://uberchromegw.corp.google.com/i/chromeos/builders/peppy-paladin/builds/17049
,
Nov 10 2017
,
Nov 10 2017
Actually, not certain if this is the right bug (to me it's not really a bvt-cq vs bvt-inline question, both of the run in the CQ anyway) but P1 because flake in this test is impacting the CQ. I'm open to moving the flake chase to a separate bug though.
,
Nov 10 2017
How about disabling this test until the race condition is fixed?
,
Nov 10 2017
Yes, we can move the test back to bvt-perbuild. I'll open a new bug. Thia one should be resolved.
,
Nov 10 2017
|
|||||||
►
Sign in to add a comment |
|||||||
Comment 1 by ka...@chromium.org
, Sep 14 2017