New issue
Advanced search Search tips

Issue 785613 link

Starred by 1 user
Status: Duplicate
Merged: issue 794707
Owner: ----
Closed: Dec 2017
Cc:
ayatane@chromium.org
ihf@chromium.org
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug



Sign in to add a comment

betty-vmtest-pre-cq stability issues

Project Member Reported by pprabhu@chromium.org, Nov 16 2017 
I've noticed betty-pre-cq's VMTests fail flakily sometimes.

e.g.: https://luci-milo.appspot.com/buildbot/chromiumos.tryserver/pre_cq/68242

The blamed CL https://chromium-review.googlesource.com/c/chromiumos/third_party/autotest/+/772310 can not break anything (should actually just be chumped)

I remember seeing this before as well.
This bug is about being able to track the betty-pre-cq health / protect it from being flaky.

Comment 1 by ihf@chromium.org, Nov 16 2017 

It passed all tests except for
/tmp/cbuildbotx1qLBP/smoke_suite/test_harness/all/SimpleTestVerify/1_autotest_tests/results-05-cheets_StartAndroid.smoke                                                               [  FAILED  ]
/tmp/cbuildbotx1qLBP/smoke_suite/test_harness/all/SimpleTestVerify/1_autotest_tests/results-05-cheets_StartAndroid.smoke                                                                 FAIL: Autotest client terminated unexpectedly: DUT rebooted during the test run.

11/15 16:15:44.957 INFO | test_runner_utils:0199| autoserv| 
11/15 16:15:44.957 INFO | test_runner_utils:0199| autoserv| tko parser: parsing partial test cheets_StartAndroid.sanity cheets_StartAndroid.sanity
11/15 16:15:44.957 INFO | test_runner_utils:0199| autoserv| tko parser: RUNNING: RUNNING
11/15 16:15:44.957 INFO | test_runner_utils:0199| autoserv| Subdir: cheets_StartAndroid.sanity
11/15 16:15:44.957 INFO | test_runner_utils:0199| autoserv| Testname: cheets_StartAndroid.sanity
11/15 16:15:44.957 INFO | test_runner_utils:0199| autoserv| 
11/15 16:15:44.957 INFO | test_runner_utils:0199| autoserv| tko parser: update RUNNING reason: Autotest client terminated unexpectedly: DUT rebooted during the test run.
11/15 16:15:44.958 INFO | test_runner_utils:0199| autoserv| tko parser: parsing test cheets_StartAndroid.sanity cheets_StartAndroid.sanity
11/15 16:15:44.958 INFO | test_runner_utils:0199| autoserv| tko parser: ADD: FAIL


I don't think any state was saved over the VM reboot? So, not enough data from the lab. Will need local repro.

Comment 2 by pprabhu@chromium.org, Nov 16 2017 

This is VMTests, so lab isn't involved. The test runs on the builder.
Also, we freeze and upload the VM images when VMTests fail, but it looks like VMTestStage itself timed out before the images could be archived :(

Comment 4 by ihf@chromium.org, Nov 17 2017 

Some of the second link might be hitting a 1h timeout. Other than that not much to say.

Comment 7 by pprabhu@chromium.org, Dec 15 2017

Mergedinto: 794707
Status: Duplicate (was: Untriaged)
Enough with the anecdotes. jkop@ has real data.
►

Sign in to add a comment