New issue
Advanced search Search tips

Issue 871439 link

Starred by 2 users

Issue metadata

Status: Fixed
Owner:
Closed: Aug 21
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 1
Type: ----

Blocking:
issue 867282
issue 873695



Sign in to add a comment

skylab: add SSP support

Project Member Reported by xixuan@chromium.org, Aug 6

Issue description

Cc: pprabhu@chromium.org ayatane@chromium.org
Owner: pprabhu@chromium.org
Status: Assigned (was: Untriaged)
Blocking: 867282
None of the jobs in skylab are running with SSP at the moment.
The problem is twofold,
- skylab_swarming_worker isn't running lucifer with --require-ssp
- We need to push the check for SSP disabled via control file inside autoserv. Currently, job_reporter does this check via https://cs.corp.google.com/chromeos_public/src/third_party/autotest/files/venv/lucifer/jobx.py?l=80 but we don't want skylab_swarming_worker / lucifer parsing the control file at all.


Status: Started (was: Assigned)
Summary: skylab: add SSP support (was: dummy_PassServer.ssp fails in push_to_prod)
Skylab can't yet run with SSP for a different reason: autoserv only looks on the local disc for control file to execute. 
https://cs.corp.google.com/chromeos_public/src/third_party/autotest/files/server/autoserv?l=327

autoserv needs to be taught to load control file from a devserver when running tests with SSP (and when a control file is provided as the argument)
Basic SSP support in:
https://chromium-review.googlesource.com/c/chromiumos/infra/lucifer/+/1166295
https://chromium-review.googlesource.com/c/chromiumos/third_party/autotest/+/1172069/

Still need to make skylab respect control file directives to skip SSP.
That requires pushing the check into autoserv from job_reporter.
Blocking: 873695
Last CL to respect REQUIRE_SSP directive in control files: https://chromium-review.googlesource.com/c/chromiumos/third_party/autotest/+/1173137

Project Member

Comment 8 by bugdroid1@chromium.org, Aug 14

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/492ef46336e762101af35ad06669a6d3be551d12

commit 492ef46336e762101af35ad06669a6d3be551d12
Author: Prathmesh Prabhu <pprabhu@chromium.org>
Date: Tue Aug 14 00:48:53 2018

autotest: Pass in more specific arguments

BUG= chromium:871439 
TEST=None

Change-Id: I99b7a48ce7e99cfb8ff880719dbf91bd4d8b76be
Reviewed-on: https://chromium-review.googlesource.com/1172069
Commit-Ready: Prathmesh Prabhu <pprabhu@chromium.org>
Tested-by: Prathmesh Prabhu <pprabhu@chromium.org>
Reviewed-by: Xixuan Wu <xixuan@chromium.org>

[modify] https://crrev.com/492ef46336e762101af35ad06669a6d3be551d12/server/autoserv

Project Member

Comment 9 by bugdroid1@chromium.org, Aug 14

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/412d1514e177bcb3b79f77b5814448bac587f0be

commit 412d1514e177bcb3b79f77b5814448bac587f0be
Author: Prathmesh Prabhu <pprabhu@chromium.org>
Date: Tue Aug 14 00:48:54 2018

autotest: Stage control file inside SSP container for skylab

In skylab, control file name is passed in and autoserv finds the control
file to execute using the name.

When running a test with SSP, autoserv re-executes inside the container
with somewhat modified commandline. For skylab, autoserv now passes in
--control-file unchanged into the container. autoserv execution inside
the container finds and stages the control file. Because the autotest
checkout inside the container is from the test artifacts for the target
build, this results in the control file being staged from the right
build.

BUG= chromium:871439 
TEST=Manual.

Change-Id: Ide2b1173a40b08bfc598537c15731d613da3eb19
Reviewed-on: https://chromium-review.googlesource.com/1172070
Commit-Ready: Prathmesh Prabhu <pprabhu@chromium.org>
Tested-by: Prathmesh Prabhu <pprabhu@chromium.org>
Reviewed-by: Xixuan Wu <xixuan@chromium.org>

[modify] https://crrev.com/412d1514e177bcb3b79f77b5814448bac587f0be/server/autoserv

Project Member

Comment 10 by bugdroid1@chromium.org, Aug 14

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/infra/lucifer/+/c539dd8bc5ad90a1385acea46869934e8079fd0b

commit c539dd8bc5ad90a1385acea46869934e8079fd0b
Author: Prathmesh Prabhu <pprabhu@chromium.org>
Date: Tue Aug 14 21:30:52 2018

skylab_swarming_worker: Run server tests with SSP

BUG= chromium:871439 
TEST=None
CQ-DEPEND=CL:1172070

Change-Id: Idce678c1f7ced5b6da42d0cbb35f5d903e484835
Reviewed-on: https://chromium-review.googlesource.com/1166295
Commit-Ready: Prathmesh Prabhu <pprabhu@chromium.org>
Tested-by: Prathmesh Prabhu <pprabhu@chromium.org>
Reviewed-by: Allen Li <ayatane@chromium.org>

[modify] https://crrev.com/c539dd8bc5ad90a1385acea46869934e8079fd0b/src/lucifer/cmd/skylab_swarming_worker/internal/lucifer/lucifer.go

dummy_PassServer.ssp is now green in staging: https://chromium-swarm-dev.appspot.com/task?id=3f5c4a793f283010

Currenly, dummy_PassServer.nossp does not properly test that the test is indeed running outside of the container. Nor is it a part of the staging test suite.
I'll update the test / suite such that it will start failing on staging.
Project Member

Comment 12 by bugdroid1@chromium.org, Aug 20

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/f62d8d7d1c3d8010eb13b47e749c1ab0b2d5a552

commit f62d8d7d1c3d8010eb13b47e749c1ab0b2d5a552
Author: Prathmesh Prabhu <pprabhu@chromium.org>
Date: Mon Aug 20 17:35:49 2018

skylab_staging: Add dummy_PassServer.nossp to staging test

This test is required to ensure that skylab respects test control's
directive to disable SSP.

BUG= chromium:871439 
TEST=None

Change-Id: I5ba39d4f86c2caff7b54ce33a4278e84c5a3b710
Reviewed-on: https://chromium-review.googlesource.com/1178470
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
Reviewed-by: Allen Li <ayatane@chromium.org>
Tested-by: Prathmesh Prabhu <pprabhu@chromium.org>

[modify] https://crrev.com/f62d8d7d1c3d8010eb13b47e749c1ab0b2d5a552/server/site_tests/dummy_PassServer/control.nossp

Project Member

Comment 13 by bugdroid1@chromium.org, Aug 21

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/80f9cb840adff3c28cbec4df5753a76d530df346

commit 80f9cb840adff3c28cbec4df5753a76d530df346
Author: Prathmesh Prabhu <pprabhu@chromium.org>
Date: Tue Aug 21 00:12:10 2018

autotest: Fail a test when staging SSP artifacts fails

autoserv would continue a test without SSP even when --require-ssp is
set if SSP setup failed. This is bad behaviour because users expect the
test to have used SSP. Instead, fail the test.

BUG= chromium:871439 
TEST=TBD

Change-Id: Ibb4f2357e972dfd3b2576adb1e88ae9f045a26fe
Reviewed-on: https://chromium-review.googlesource.com/1172076
Commit-Ready: Prathmesh Prabhu <pprabhu@chromium.org>
Tested-by: Prathmesh Prabhu <pprabhu@chromium.org>
Reviewed-by: Xixuan Wu <xixuan@chromium.org>

[modify] https://crrev.com/80f9cb840adff3c28cbec4df5753a76d530df346/server/autoserv_parser.py
[modify] https://crrev.com/80f9cb840adff3c28cbec4df5753a76d530df346/server/autoserv

Project Member

Comment 14 by bugdroid1@chromium.org, Aug 21

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/83b598cc6c165d854ede063abbb3fb6f9f8cd449

commit 83b598cc6c165d854ede063abbb3fb6f9f8cd449
Author: Prathmesh Prabhu <pprabhu@chromium.org>
Date: Tue Aug 21 00:12:11 2018

autoserv: Infer require_ssp from control file passed to autoserv

Unlike Autotest, Skylab does not pass in a control file to execute by
autoserv, nor does Skylab parse the control file to obtain the options
specified within control files.

Instead, Skylab passes in the name of the control file to execute and
autoserv must infer and respect the options set by a test via its
control file.

This CL makes autoserv respect the REQUIRE_SSP directive in the control
file, if the control file was passed in via --control-name

BUG= chromium:871439 
TEST=manual skylab task for dummy_ServerPass.ssp and .nossp

Change-Id: I367d11d7bf74fafed2ae7ca2184856ca707f8367
Reviewed-on: https://chromium-review.googlesource.com/1173137
Commit-Ready: Prathmesh Prabhu <pprabhu@chromium.org>
Tested-by: Prathmesh Prabhu <pprabhu@chromium.org>
Reviewed-by: Xixuan Wu <xixuan@chromium.org>

[modify] https://crrev.com/83b598cc6c165d854ede063abbb3fb6f9f8cd449/server/autoserv

Project Member

Comment 15 by bugdroid1@chromium.org, Aug 21

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/7b05eccacee948cdac4bcc365c18656c3af006fb

commit 7b05eccacee948cdac4bcc365c18656c3af006fb
Author: Prathmesh Prabhu <pprabhu@chromium.org>
Date: Tue Aug 21 00:12:11 2018

autotest: Delete unnecessary, broken job_folder inference

BUG= chromium:871439 
TEST=manual skylab task.

Change-Id: I0549902e6ef924e91f0d64931f3fd42461b16a98
Reviewed-on: https://chromium-review.googlesource.com/1173420
Commit-Ready: Prathmesh Prabhu <pprabhu@chromium.org>
Tested-by: Prathmesh Prabhu <pprabhu@chromium.org>
Reviewed-by: Xixuan Wu <xixuan@chromium.org>

[modify] https://crrev.com/7b05eccacee948cdac4bcc365c18656c3af006fb/site_utils/job_directories.py
[modify] https://crrev.com/7b05eccacee948cdac4bcc365c18656c3af006fb/server/autoserv

Status: Fixed (was: Started)
There are some cleanup CLs still in the CQ, but the actual bug is fixed.

Sign in to add a comment