video_PlaybackPerf and video_WebRtcPerf flaky due to AC power assertion |
|||||||
Issue description
video_PlaybackPerf is flaky, most of failures are control.power_* (aka video_PowerConsumption).
File "/usr/local/autotest/tests/video_PlaybackPerf/video_PlaybackPerf.py", line 205, in test_power
self._power_status.assert_battery_state(BATTERY_INITIAL_CHARGED_MIN)
File "/usr/local/autotest/cros/power_status.py", line 477, in assert_battery_state
'Running on AC power. Please remove AC power cable.')
The test turned off AC power and detected AC power status but AC is still ON.
This is not pure random. It failed very often on certain boards.
,
May 22 2017
I found power_Consumption is failing with the same issue.
Moreover, the health of power_Consumption test is worse than video tests (many yellow and red)
Given the AC state is flaky but is good most of time, I feel maybe we should workaround it first.
+if self._power_status.on_ac():
+ raise error.TestNAError('
self._power_status.assert_battery_state(BATTERY_INITIAL_CHARGED_MIN)
What do you think?
,
May 22 2017
I don't understand this workaround. Can you explain more?
,
May 23 2017
What the test did now: 1. turn off AC power. 2. verify it is battery and has enough capacity 3. run test cases My proposal: 1. turn off AC power 2. if it is on AC, skip this test (raise TestNAError) 3. verify it is battery and has enough capacity 4. run test cases In order words, the test is disabled if AC is not in expected state. I feel this is acceptable because the possibility is not too high (only on 15 boards. each about 50%).
,
May 26 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/27ee2110cb95c07d4686f67e9cfa7c343d01e2dc commit 27ee2110cb95c07d4686f67e9cfa7c343d01e2dc Author: Kuang-che Wu <kcwu@chromium.org> Date: Fri May 26 00:16:08 2017 autotest: fix AC power assertion "Turn off AC power" should be done before battery assertion. The code is also revised after video_PlaybackPerf.py to ensure properly tear down. BUG= chromium:723968 TEST=video_WebRtcPerf Change-Id: Ib54c1f423340453fcb41d0d370d1710f421a0bff Reviewed-on: https://chromium-review.googlesource.com/513766 Commit-Ready: Kuang-che Wu <kcwu@chromium.org> Tested-by: Kuang-che Wu <kcwu@chromium.org> Reviewed-by: Pin-chih Lin <johnylin@chromium.org> [modify] https://crrev.com/27ee2110cb95c07d4686f67e9cfa7c343d01e2dc/client/site_tests/video_WebRtcPerf/video_WebRtcPerf.py
,
May 26 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/9cca67a5c9d069d37c9c8d83958445fedf1f3587 commit 9cca67a5c9d069d37c9c8d83958445fedf1f3587 Author: Kuang-che Wu <kcwu@chromium.org> Date: Fri May 26 13:38:56 2017 autotest: disable video power tests if AC state is unexpected BUG= chromium:723968 TEST=video_PlaybackPerf and video_WebRtcPerf Change-Id: I15deccbc60c942a3f9787d398134f4c85fcdecfe Reviewed-on: https://chromium-review.googlesource.com/517004 Commit-Ready: Kuang-che Wu <kcwu@chromium.org> Tested-by: Kuang-che Wu <kcwu@chromium.org> Reviewed-by: Pin-chih Lin <johnylin@chromium.org> [modify] https://crrev.com/9cca67a5c9d069d37c9c8d83958445fedf1f3587/client/site_tests/video_WebRtcPerf/video_WebRtcPerf.py [modify] https://crrev.com/9cca67a5c9d069d37c9c8d83958445fedf1f3587/client/site_tests/video_PlaybackPerf/video_PlaybackPerf.py
,
Jun 1 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/f4610bdd340f30567836fff719b1d4563a544170 commit f4610bdd340f30567836fff719b1d4563a544170 Author: Kuang-che Wu <kcwu@chromium.org> Date: Thu Jun 01 05:03:16 2017 Revert "autotest: disable video power tests if AC state is unexpected" This reverts commit 9cca67a5c9d069d37c9c8d83958445fedf1f3587. Reason for revert: break video_WebRtcPerf Original change's description: > autotest: disable video power tests if AC state is unexpected > > BUG= chromium:723968 > TEST=video_PlaybackPerf and video_WebRtcPerf > > Change-Id: I15deccbc60c942a3f9787d398134f4c85fcdecfe > Reviewed-on: https://chromium-review.googlesource.com/517004 > Commit-Ready: Kuang-che Wu <kcwu@chromium.org> > Tested-by: Kuang-che Wu <kcwu@chromium.org> > Reviewed-by: Pin-chih Lin <johnylin@chromium.org> > BUG= chromium:723968 Change-Id: I06e5a32538c2f22584290aefc8e94d5a74d8bdb0 Reviewed-on: https://chromium-review.googlesource.com/517842 Reviewed-by: Kuang-che Wu <kcwu@chromium.org> Tested-by: Kuang-che Wu <kcwu@chromium.org> [modify] https://crrev.com/f4610bdd340f30567836fff719b1d4563a544170/client/site_tests/video_WebRtcPerf/video_WebRtcPerf.py [modify] https://crrev.com/f4610bdd340f30567836fff719b1d4563a544170/client/site_tests/video_PlaybackPerf/video_PlaybackPerf.py
,
Jun 1 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/f4610bdd340f30567836fff719b1d4563a544170 commit f4610bdd340f30567836fff719b1d4563a544170 Author: Kuang-che Wu <kcwu@chromium.org> Date: Thu Jun 01 05:03:16 2017 Revert "autotest: disable video power tests if AC state is unexpected" This reverts commit 9cca67a5c9d069d37c9c8d83958445fedf1f3587. Reason for revert: break video_WebRtcPerf Original change's description: > autotest: disable video power tests if AC state is unexpected > > BUG= chromium:723968 > TEST=video_PlaybackPerf and video_WebRtcPerf > > Change-Id: I15deccbc60c942a3f9787d398134f4c85fcdecfe > Reviewed-on: https://chromium-review.googlesource.com/517004 > Commit-Ready: Kuang-che Wu <kcwu@chromium.org> > Tested-by: Kuang-che Wu <kcwu@chromium.org> > Reviewed-by: Pin-chih Lin <johnylin@chromium.org> > BUG= chromium:723968 Change-Id: I06e5a32538c2f22584290aefc8e94d5a74d8bdb0 Reviewed-on: https://chromium-review.googlesource.com/517842 Reviewed-by: Kuang-che Wu <kcwu@chromium.org> Tested-by: Kuang-che Wu <kcwu@chromium.org> [modify] https://crrev.com/f4610bdd340f30567836fff719b1d4563a544170/client/site_tests/video_WebRtcPerf/video_WebRtcPerf.py [modify] https://crrev.com/f4610bdd340f30567836fff719b1d4563a544170/client/site_tests/video_PlaybackPerf/video_PlaybackPerf.py
,
Jun 2 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/bbeb0f7d8f443f78d56120d01122e15217cd29c9 commit bbeb0f7d8f443f78d56120d01122e15217cd29c9 Author: Kuang-che Wu <kcwu@chromium.org> Date: Fri Jun 02 06:49:31 2017 Reland: autotest: disable video power tests if AC state is unexpected BUG= chromium:723968 TEST=video_PlaybackPerf and video_WebRtcPerf Change-Id: I28db6830b20ece7c9e4135bd9451d77093919c35 Reviewed-on: https://chromium-review.googlesource.com/520829 Commit-Ready: Kuang-che Wu <kcwu@chromium.org> Tested-by: Kuang-che Wu <kcwu@chromium.org> Reviewed-by: Pin-chih Lin <johnylin@chromium.org> [modify] https://crrev.com/bbeb0f7d8f443f78d56120d01122e15217cd29c9/client/site_tests/video_WebRtcPerf/video_WebRtcPerf.py [modify] https://crrev.com/bbeb0f7d8f443f78d56120d01122e15217cd29c9/client/site_tests/video_PlaybackPerf/video_PlaybackPerf.py
,
Jun 2 2017
,
Jun 2 2017
,
Jun 3 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/24bce6abc501dab8d699a65f6f4d6f7a0fd35f79 commit 24bce6abc501dab8d699a65f6f4d6f7a0fd35f79 Author: Kuang-che Wu <kcwu@chromium.org> Date: Sat Jun 03 12:52:30 2017 Revert "Reland: autotest: disable video power tests if AC state is unexpected" This reverts commit bbeb0f7d8f443f78d56120d01122e15217cd29c9. Reason for revert: break video_WebRtcPerf Original change's description: > Reland: autotest: disable video power tests if AC state is unexpected > > BUG= chromium:723968 > TEST=video_PlaybackPerf and video_WebRtcPerf > > Change-Id: I28db6830b20ece7c9e4135bd9451d77093919c35 > Reviewed-on: https://chromium-review.googlesource.com/520829 > Commit-Ready: Kuang-che Wu <kcwu@chromium.org> > Tested-by: Kuang-che Wu <kcwu@chromium.org> > Reviewed-by: Pin-chih Lin <johnylin@chromium.org> > BUG= chromium:723968 Change-Id: I416877feb3416b4b2cb29bb9368fcb17d852ca13 Reviewed-on: https://chromium-review.googlesource.com/523344 Commit-Ready: Kuang-che Wu <kcwu@chromium.org> Tested-by: Kuang-che Wu <kcwu@chromium.org> Reviewed-by: Pin-chih Lin <johnylin@chromium.org> [modify] https://crrev.com/24bce6abc501dab8d699a65f6f4d6f7a0fd35f79/client/site_tests/video_WebRtcPerf/video_WebRtcPerf.py [modify] https://crrev.com/24bce6abc501dab8d699a65f6f4d6f7a0fd35f79/client/site_tests/video_PlaybackPerf/video_PlaybackPerf.py
,
Jun 6 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/67ca2fbe02a39f41df72a492afdddde4ecb223f0 commit 67ca2fbe02a39f41df72a492afdddde4ecb223f0 Author: Kuang-che Wu <kcwu@chromium.org> Date: Tue Jun 06 06:49:37 2017 autotest: disable video power tests if AC state is unexpected BUG= chromium:723968 TEST=video_PlaybackPerf and video_WebRtcPerf Change-Id: Idcdec044039fbc6751bc9ae8fc6857521b2cf3aa Reviewed-on: https://chromium-review.googlesource.com/523484 Commit-Ready: Kuang-che Wu <kcwu@chromium.org> Tested-by: Kuang-che Wu <kcwu@chromium.org> Reviewed-by: Pin-chih Lin <johnylin@chromium.org> [modify] https://crrev.com/67ca2fbe02a39f41df72a492afdddde4ecb223f0/client/site_tests/video_WebRtcPerf/video_WebRtcPerf.py [modify] https://crrev.com/67ca2fbe02a39f41df72a492afdddde4ecb223f0/client/site_tests/video_PlaybackPerf/video_PlaybackPerf.py
,
Jun 6 2017
The CL is landed. Let's wait few days to see how it goes.
,
Jun 12 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/ff4277082660d587f18f4cbc46fffbc3741393e2 commit ff4277082660d587f18f4cbc46fffbc3741393e2 Author: Kuang-che Wu <kcwu@chromium.org> Date: Mon Jun 12 08:20:53 2017 Revert "autotest: disable video power tests if AC state is unexpected" This reverts commit 9cca67a5c9d069d37c9c8d83958445fedf1f3587. Reason for revert: break video_WebRtcPerf Original change's description: > autotest: disable video power tests if AC state is unexpected > > BUG= chromium:723968 > TEST=video_PlaybackPerf and video_WebRtcPerf > > Change-Id: I15deccbc60c942a3f9787d398134f4c85fcdecfe > Reviewed-on: https://chromium-review.googlesource.com/517004 > Commit-Ready: Kuang-che Wu <kcwu@chromium.org> > Tested-by: Kuang-che Wu <kcwu@chromium.org> > Reviewed-by: Pin-chih Lin <johnylin@chromium.org> > BUG= chromium:723968 Change-Id: I06e5a32538c2f22584290aefc8e94d5a74d8bdb0 Reviewed-on: https://chromium-review.googlesource.com/517842 Reviewed-by: Kuang-che Wu <kcwu@chromium.org> Tested-by: Kuang-che Wu <kcwu@chromium.org> (cherry picked from commit f4610bdd340f30567836fff719b1d4563a544170) Reviewed-on: https://chromium-review.googlesource.com/530208 Commit-Queue: Kuang-che Wu <kcwu@chromium.org> Trybot-Ready: Kuang-che Wu <kcwu@chromium.org> Reviewed-by: Pin-chih Lin <johnylin@chromium.org> [modify] https://crrev.com/ff4277082660d587f18f4cbc46fffbc3741393e2/client/site_tests/video_WebRtcPerf/video_WebRtcPerf.py [modify] https://crrev.com/ff4277082660d587f18f4cbc46fffbc3741393e2/client/site_tests/video_PlaybackPerf/video_PlaybackPerf.py
,
Jun 12 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/af1e50dc83578c77fb950c06bbd2ffab15d4d32a commit af1e50dc83578c77fb950c06bbd2ffab15d4d32a Author: Kuang-che Wu <kcwu@chromium.org> Date: Mon Jun 12 08:39:38 2017 autotest: disable video power tests if AC state is unexpected BUG= chromium:723968 TEST=video_PlaybackPerf and video_WebRtcPerf Change-Id: Idcdec044039fbc6751bc9ae8fc6857521b2cf3aa Reviewed-on: https://chromium-review.googlesource.com/523484 Commit-Ready: Kuang-che Wu <kcwu@chromium.org> Tested-by: Kuang-che Wu <kcwu@chromium.org> Reviewed-by: Pin-chih Lin <johnylin@chromium.org> (cherry picked from commit 67ca2fbe02a39f41df72a492afdddde4ecb223f0) Reviewed-on: https://chromium-review.googlesource.com/530209 Commit-Queue: Kuang-che Wu <kcwu@chromium.org> Trybot-Ready: Kuang-che Wu <kcwu@chromium.org> [modify] https://crrev.com/af1e50dc83578c77fb950c06bbd2ffab15d4d32a/client/site_tests/video_WebRtcPerf/video_WebRtcPerf.py [modify] https://crrev.com/af1e50dc83578c77fb950c06bbd2ffab15d4d32a/client/site_tests/video_PlaybackPerf/video_PlaybackPerf.py
,
Jun 14 2017
,
Jun 19 2017
https://cros-goldeneye.corp.google.com/chromeos/healthmonitoring/testDetails?testName=video_PlaybackPerf&suite&daysBack=7&version&milestone |
|||||||
►
Sign in to add a comment |
|||||||
Comment 1 by kcwu@chromium.org
, May 18 2017