Improve tast Autotest's handling of tast executable failures |
||
Issue descriptionRight now, the tast.py Autotest server test doesn't do a great job of handling failed executions of the tast command. If the initial "tast list" command succeeds (so we have a list of expected tests) but "tast run" fails (so we don't parse results.json to get the results of individual tests), tast.py doesn't write the individual tests to status.log. Furthermore, the tast command only writes a results.json file after it's done running all tests. If it manages to run a partial set of tests but then dies for some reason, tast.py doesn't get any results. I should make tast write a file containing a stream of JSON result objects while it's reading control messages from test runners.
,
May 18 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/platform/tast/+/b5589dfd6993092cb0a04a481539c1acdf151551 commit b5589dfd6993092cb0a04a481539c1acdf151551 Author: Daniel Erat <derat@chromium.org> Date: Fri May 18 02:35:03 2018 tast: Write streamed_results.jsonl. Make the tast command write TestResults objects to a new streamed_results.jsonl file while reading control messages so we'll still have partial results if the tast process is killed mid-run. The new file is in "JSON Lines" format, i.e. newline-terminated JSON objects written by json.Encoder. Started-but-not-completed tests will have end times of 0001-01-01T00:00:00Z in the new file. BUG= chromium:842453 TEST=added/updated unit tests; also updated resultsHandler to call os.Exit(1) abruptly and checked that streamed_results.jsonl contains full TestResult objects for completed tests and a partial object (i.e. with a zero-time 'end' property) for the in-progress test Change-Id: I32e26986457bcca2a9d73ef4d87eeb94f3792e8a Reviewed-on: https://chromium-review.googlesource.com/1063177 Commit-Ready: Dan Erat <derat@chromium.org> Tested-by: Dan Erat <derat@chromium.org> Reviewed-by: Dan Erat <derat@chromium.org> [modify] https://crrev.com/b5589dfd6993092cb0a04a481539c1acdf151551/src/chromiumos/cmd/tast/run/results_test.go [modify] https://crrev.com/b5589dfd6993092cb0a04a481539c1acdf151551/src/chromiumos/cmd/tast/run/results.go
,
May 22 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/026f07476817332572fee92677f1f3291627905d commit 026f07476817332572fee92677f1f3291627905d Author: Daniel Erat <derat@chromium.org> Date: Tue May 22 04:17:56 2018 autotest: Update tast server test to read streamed results. Update the tast server test to read test results from streamed_results.jsonl rather than results.json. The tast executable writes results to streamed_results.jsonl as they're received, so this allows partial results to be reported by Autotest even if the tast process is killed before it finishes. BUG= chromium:842453 TEST=updated tast to die after writing results for a few tests and checked status.log to confirm that a) the completed tests are still reported as passing, b) the test that was in progress when the tast process exited gets a 'Test did not finish' failure, and c) the not-run tests get 'Test was not run' failures Change-Id: I72b08ec0bbaf83328a27c60d7d6a7156eebf397d Reviewed-on: https://chromium-review.googlesource.com/1065254 Commit-Ready: Dan Erat <derat@chromium.org> Tested-by: Dan Erat <derat@chromium.org> Reviewed-by: Ilja H. Friedel <ihf@chromium.org> [modify] https://crrev.com/026f07476817332572fee92677f1f3291627905d/server/site_tests/tast/tast.py
,
May 25 2018
|
||
►
Sign in to add a comment |
||
Comment 1 by bugdroid1@chromium.org
, May 15 2018