New issue
Advanced search Search tips

Issue 773819 link

Starred by 2 users

Issue metadata

Status: Verified
Owner:
Closed: Oct 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 2
Type: Feature



Sign in to add a comment

tast: results.json should include details about tests

Project Member Reported by derat@chromium.org, Oct 11 2017

Issue description

It'd be nice if the tast executable included serialized testing.Test structs in results.json files:

- People looking at results can read tests' descriptions.
- cbuildbot test stages can avoid failing for errors in tests that have a "flaky" attribute set.
- Probably more stuff that I haven't thought of yet.

To do this, the local_tests and remote_tests executables will need to pass the structs to the tast executable; they should probably be included in control.TestStart messages.
 
Project Member

Comment 1 by bugdroid1@chromium.org, Oct 15 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/tast/+/b12d1e0ca6914518ee57bf1693c335c1ccc6e77b

commit b12d1e0ca6914518ee57bf1693c335c1ccc6e77b
Author: Daniel Erat <derat@chromium.org>
Date: Fri Oct 13 21:45:11 2017

tast: Include test details in results.json.

Make the local_tests and remote_tests executables pass
testing.Test structs to the main tast executable, and then
make the tast executable include them in results.json files.

BUG= chromium:773819 
TEST=added unit tests; also checked that test details appear
     in results.json when using -build=true, and also that
     things don't blow up when either using new tast with
     old local_tests or old tast with new local_tests

Change-Id: I9e520fe8295bbcd2d5abdcb262ea3478de0b8896
Reviewed-on: https://chromium-review.googlesource.com/714535
Commit-Ready: Dan Erat <derat@chromium.org>
Tested-by: Dan Erat <derat@chromium.org>
Reviewed-by: Jason Clinton <jclinton@chromium.org>

[modify] https://crrev.com/b12d1e0ca6914518ee57bf1693c335c1ccc6e77b/src/chromiumos/tast/cmd/run/results_test.go
[modify] https://crrev.com/b12d1e0ca6914518ee57bf1693c335c1ccc6e77b/src/chromiumos/tast/common/runner/runner_test.go
[modify] https://crrev.com/b12d1e0ca6914518ee57bf1693c335c1ccc6e77b/src/chromiumos/tast/common/control/control.go
[modify] https://crrev.com/b12d1e0ca6914518ee57bf1693c335c1ccc6e77b/src/chromiumos/tast/common/testing/test.go
[modify] https://crrev.com/b12d1e0ca6914518ee57bf1693c335c1ccc6e77b/src/chromiumos/tast/common/control/control_test.go
[modify] https://crrev.com/b12d1e0ca6914518ee57bf1693c335c1ccc6e77b/src/chromiumos/tast/common/testing/test_test.go
[modify] https://crrev.com/b12d1e0ca6914518ee57bf1693c335c1ccc6e77b/src/chromiumos/tast/cmd/run/results.go
[modify] https://crrev.com/b12d1e0ca6914518ee57bf1693c335c1ccc6e77b/src/chromiumos/tast/common/runner/runner.go

Comment 2 by derat@chromium.org, Oct 20 2017

Status: Verified (was: Started)
Project Member

Comment 3 by bugdroid1@chromium.org, Jan 22 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/tast/+/8f05ad10016e65edb014284ac7528d24d376df8d

commit 8f05ad10016e65edb014284ac7528d24d376df8d
Author: Daniel Erat <derat@chromium.org>
Date: Mon Jan 22 18:13:39 2018

tast: Drop control.TestStart.Name field.

Drop the Name field from TestStart control messages. The
test name is now available via the Test field's Name field
(and has been for months).

BUG= chromium:773819 
TEST=unit tests pass; also ran a local test and checked
     that results.json contains test info

Change-Id: Id59ee6898b85bcc47316202bd9a2c8546c37046a
Reviewed-on: https://chromium-review.googlesource.com/877799
Commit-Ready: Dan Erat <derat@chromium.org>
Tested-by: Dan Erat <derat@chromium.org>
Reviewed-by: Jason Clinton <jclinton@chromium.org>

[modify] https://crrev.com/8f05ad10016e65edb014284ac7528d24d376df8d/src/chromiumos/cmd/tast/run/results_test.go
[modify] https://crrev.com/8f05ad10016e65edb014284ac7528d24d376df8d/src/chromiumos/tast/control/control_test.go
[modify] https://crrev.com/8f05ad10016e65edb014284ac7528d24d376df8d/src/chromiumos/tast/bundle/bundle.go
[modify] https://crrev.com/8f05ad10016e65edb014284ac7528d24d376df8d/src/chromiumos/tast/control/control.go
[modify] https://crrev.com/8f05ad10016e65edb014284ac7528d24d376df8d/src/chromiumos/cmd/tast/run/results.go
[modify] https://crrev.com/8f05ad10016e65edb014284ac7528d24d376df8d/src/chromiumos/tast/bundle/bundle_test.go

Comment 4 by derat@chromium.org, Jan 23 2018

Components: Tests>Tast

Sign in to add a comment