New issue
Advanced search Search tips

Issue 922231 link

Starred by 2 users

Issue metadata

Status: Untriaged
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 2
Type: Bug



Sign in to add a comment

Add Tast tests for DLC functionality

Project Member Reported by xiaochu@chromium.org, Jan 15

Issue description

Rather than lumping all DLC related tests (install, update, mount, etc.) into one tast test. I proposed to have separate tests for each functionalities. Benefits are that we have modular testing that we can selectively run on different suites (pre-cq, cq, release builders, etc.) and less flakeness for each test.

This bug specifically tracks the update_engine related tests.
 
Cc: nya@chromium.org hidehiko@chromium.org
I don't have any objections to smaller, more-modular tests, as long as they're fast and don't require lengthy setup that could otherwise be shared (e.g. restarting Chrome, waiting for ARC to boot, etc.).

But just to reply to a few parts:

> we can selectively run on different suites (pre-cq, cq, release builders, etc.)

Particularly if the tests are fast, the goal should be to have all of them running in the CQ. Note that there's no concept of running on the pre-CQ vs. the CQ: if a test runs on the CQ, it will also run on VMs in the pre-CQ (as long as it's capable of running on VMs, which I assume that all of these tests will be).

> less flakeness for each test

I don't think that this is necessarily true. More tests often means more flakiness, as the same setup steps (which have a risk of encountering races, DUT or network failures, etc.) are being performed multiple times rather than just once. This isn't as much of a concern if setup doesn't involve restarting Chrome, though.

Comment 2 by ahass...@chromium.org, Jan 16 (6 days ago)

Cc: dhadd...@chromium.org
Are we gonna leverage the already existing many update engine autotests that exist and they run at many levels of the infra like CQ, canaries and release buidlers, once in a while (every few days) builders, etc... The update tests are kind of treated special in stainless and GE because they need special provisioning, devserver, etc.

+dhaddock@ as the owner of update engine tests.

Comment 3 by xiaochu@chromium.org, Jan 16 (6 days ago)

i don't think a tast local test can inter-operate with a local autotest easily. so unfortunately we can't reuse the existing update_engine autotests. i'm aware of many efforts of porting existing autotests into tast tests and interested in discussing about plans for update_engine if that's necessary.

the goal of this bug is to have some coverage in pre-cq/cq on update_engine for DLC functionalities. afaik, current update_engine autotests don't have coverage for it.

Comment 4 by derat@chromium.org, Jan 17 (6 days ago)

Summary: Add Tast tests for DLC functionality (was: update_engine: a tast test)
(updating the title to reflect the goal)

Sign in to add a comment