New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 591279 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner:
Closed: Oct 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 1
Type: Bug



Sign in to add a comment

Start running audio and external display tests as part of bvt-cq

Project Member Reported by ka...@chromium.org, Mar 2 2016

Issue description

Audio test candidates:
- audio_AudioBasicExternalMicrophone
- audio_AudioBasicHeadphone

External Display tests candidates
 - display_Resolution.extended 
 - display_HotPlugAtSuspend.extended
 - display_HotPlugAtBoot.extended

Devices to start these tests on bvt-cq suite: veyron_jerry, peach_pi, link, and squawks.

Test results for last three weeks: https://wmatrix.googleplex.com/unfiltered?suites=chameleon_audio_perbuild,chameleon_hdmi_perbuild&tests=display_Resolution.extended,display_HotPlugAtSuspend.extended,display_HotPlugAtBoot.extended&platforms=veyron_jerry,peach_pi,link,squawks&days_back=21
 

Comment 1 by ka...@chromium.org, Mar 2 2016

Components: OS>Kernel
Labels: -Pri-2 Pri-1

Comment 2 by ka...@chromium.org, Mar 2 2016

The URL posted is for display tests only.

Here is the URL for audio tests: https://wmatrix.googleplex.com/unfiltered?suites=chameleon_audio_perbuild,chameleon_hdmi_perbuild&tests=audio_AudioBasicExternalMicrophone,audio_AudioBasicHeadphone&platforms=veyron_jerry,peach_pi,link,squawks&days_back=14

The 'scp' failures reason are the ones motivated us to start this effort.
Thanks Kalin!
These four boards looks stable enough to me.
We definitely don't want to block cq too often.
Should we dedicate some of these devices to bvt-cq ?

Comment 4 by ka...@chromium.org, Mar 2 2016

Mmm, ok, lets 
Per go/chameleons-in-lab view we can do this with jerry and link. I misread there are four pi DUTs with chameleon.

veyron_jerry - chromeos2-row1-rack4-host1
veyron_jerry - chromeos2-row1-rack4-host2
veyron_jerry - chromeos2-row1-rack4-host3
veyron_jerry - chromeos2-row1-rack4-host4
link - chromeos2-row4-rack3-host8
link - chromeos2-row4-rack3-host9
link - chromeos2-row4-rack2-host12
link - chromeos2-row4-rack2-host7

If we insist on squawks we can file a request for two more boards to be equipped with chameleon(respectively with audio board too)

I'll file requests for 
1) adding four audio board to each of jerry and link 
2) adding two more chameleon boards to squawks DUTs and four audio boards.

Now, is there a chance that having audio jack plugged could break anything on bvt-cq? We should probably run the suite against few boards with the same setup in cassandra lab.

Comment 5 by ka...@chromium.org, Mar 2 2016

Cc: dshi@chromium.org
Labels: OS-Chrome
+dshi for any suggestions about adding tests to bvt.
Dan, what would be the guideline we should follow?

Comment 6 by dshi@chromium.org, Mar 2 2016

Cc: sbasi@chromium.org
+infra deputy

The basic rules for tests in cq suite are:
1. The test needs to be constantly passing on canary builds for the last two weeks.
2. The test run time should be relatively short, around 2min tops.

Comment 7 by ka...@chromium.org, Mar 2 2016

Rule 1.The test needs to be constantly passing on canary builds for the last two weeks.
- How do we arrange this? What test suite to assign to?


Rule 2. The test run time should be relatively short, around 2min tops.

 - display_Resolution.extended - Duration [s]	153 - GOOD
 - display_HotPlugAtSuspend.extended - Duration [s]	218 - GOD
 - display_HotPlugAtBoot.extended - Duration [s]	385 - this should be fixed by having new (control.bvt) test that goes through a single item in PLUG_CONFIGS 

 - audio_AudioBasicExternalMicrophone - Duration [s]	205 - GOOD
 - audio_AudioBasicHeadphone - Duration [s]	226 - GOOD

Comment 8 by dshi@chromium.org, Mar 2 2016

suite bvt-perbuild should be a good candidate. The suite runs on canary builds.

3-4 min is a bit long, but not too bad, I think it should be OK.

Comment 9 by ka...@chromium.org, Mar 2 2016

Ohh, right. I folded the running time when looking at duration. I'll follow up on making faster test runs for  most of the tests.
We only want to run some of the stable tests on several boards, e.g. veyron_jerry, peach_pi, link, and squawks. Is it doable?

About the speed, we can customize the tests (using another control.bvt) to run only a subset of test items.

Comment 11 by dshi@chromium.org, Mar 2 2016

It will be easy for a single board, you can add board:xxx to the DEPENDENCIES attribute. Are these boards share some common label? I think it will be kind of messy. You might have to create individual test control file for each board.
OK, then to keep it simple we can start with a single board - Jerry.

Tasks to start with
- Add audio board to the veyron_jerry DUTs - chromeos2-row1-rack4-host1...host4
- modify test to accomodate faster execution.
- create separate tests with control files pointing to 
DEPENDECIES = 'board:veyron_jerry audio_board' and 
SUITE = 'bvt-perbuild'


If we observe failures that fail only on specific board family, we should add audio boards to such board DUTs.
https://buganizer.corp.google.com/u/0/issues/27477762 is tracking the deployment task.

Comment 14 by ka...@chromium.org, Mar 28 2016

Four audio boards are added to the veyron_jerry DUTs chromeos2-row1-rack4-host1...host4

Next steps

1) Make tests faster 
- display_Resolution.extended
- audio_AudioBasicHeadphone
- audio_AudioBasicExternalMicrophone

2) Start running the tests for bvt-perbuild

3) Add few more tests
- display_HotPlugAtSuspend.mirrored
- audio_AudioBasicUSBPlayback

4) Cover more devices

Comment 15 by ka...@chromium.org, Oct 24 2016

Status: WontFix (was: Untriaged)

Sign in to add a comment