video_PlaybackPerf failing since 9378.0.0 |
||||||||||||
Issue descriptionFrom wmatrix, video_PlaybackPerf started to fail since 9378.0.0. This is a new failure so please take a look soon.
,
Mar 22 2017
According to log, it seems to be lab network issue.
File "/usr/local/telemetry/src/third_party/catapult/telemetry/telemetry/core/local_server.py", line 152, in StartServer
assert self._platform_backend.network_controller_backend.is_initialized
AssertionError
Besides, I tested elm/R59-9378.0.0 with "test_that elm video_PlaybackPerf.power_vp9" for three times and all passed.
,
Mar 22 2017
+Vin for network issue. Daniel. Can you also try "test_that elm video_PlaybackPerf.vp9"?
,
Mar 22 2017
"test_that elm video_PlaybackPerf.vp9" passed as well. I also tried the server side test with "test_that elm video_PowerConsumption.vp8" and it failed although I'm not sure whether the test needs a special environment setup. There is no error message in the log. 04:03:39 INFO | autoserv| Starting master ssh connection '/usr/bin/ssh -a -x -N -o ControlMaster=yes -o ControlPath=/tmp/_autotmp_HvpQu2ssh-master/socket -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o BatchMode=yes -o ConnectTimeout=30 -o ServerAliveInterval=900 -o ServerAliveCountMax=3 -o ConnectionAttempts=4 -o Protocol=2 -l root -p 22 elm' 04:03:40 INFO | autoserv| get_network_stats: at-end RXbytes 59407475 TXbytes 2546651 04:03:43 INFO | autoserv| record_state_duration failed: job_or_task_id=None, hostname=elm, status=Running ------------------------------------------------------------------------------ /tmp/test_that_results_4kdQUO/results-1-video_PowerConsumption.vp8 [ FAILED ] ------------------------------------------------------------------------------
,
Mar 23 2017
+dshi, akeshet. Have you seen errors like this?
File "/usr/local/telemetry/src/third_party/catapult/telemetry/telemetry/core/local_server.py", line 152, in StartServer
assert self._platform_backend.network_controller_backend.is_initialized
AssertionError
,
Mar 23 2017
I've never seen such error. How about try test_that with :lab: option to run it in lab?
,
Mar 23 2017
Daniel. Please try lab option.
,
Mar 24 2017
The lab is still closed. Will try when it comes back.
,
Mar 28 2017
This is still failing in tot.
,
Apr 20 2017
I got INFRA_FAILURE while trying to run it in lab.
$ test_that :lab: --max_runtime_mins 600 -b elm -i 'elm-release/R59-9378.0.0' video_PlaybackPerf.vp9
INFO:root:About to start lab suite with command ['/mnt/host/source/src/third_party/autotest/files/site_utils/run_suite.py', '--board=elm', '--build=elm-release/R59-9378.0.0', '--suite_name=test_that_wrapper', '--pool=suites', '--max_runtime_mins=600', '--suite_args=:lab: -b elm -i elm-release/R59-9378.0.0 video_PlaybackPerf.vp9'].
Autotest instance: localhost
04-16-2017 [20:08:00] Submitted create_suite_job rpc
Unhandled run_suite exception: retry exception (function="create_suite()"), timeout = 600s
Traceback (most recent call last):
File "/mnt/host/source/src/third_party/autotest/files/site_utils/run_suite.py", line 1831, in main
code, output_dict = main_without_exception_handling(options)
File "/mnt/host/source/src/third_party/autotest/files/site_utils/run_suite.py", line 1635, in main_without_exception_handling
job_id = create_suite(afe, options)
File "/mnt/host/source/src/third_party/autotest/files/client/common_lib/cros/retry.py", line 233, in func_retry
raise error.TimeoutException(exception_message)
TimeoutException: retry exception (function="create_suite()"), timeout = 600s
Will return from run_suite with status: INFRA_FAILURE
,
Apr 21 2017
The failure stack:
04/14 07:59:10.109 WARNI| test:0615| The test failed with the following exception
Traceback (most recent call last):
File "/usr/local/autotest/common_lib/test.py", line 609, in _exec
_call_test_function(self.execute, *p_args, **p_dargs)
File "/usr/local/autotest/common_lib/test.py", line 823, in _call_test_function
raise error.UnhandledTestFail(e)
UnhandledTestFail: Unhandled AssertionError:
Traceback (most recent call last):
File "/usr/local/autotest/common_lib/test.py", line 817, in _call_test_function
return func(*args, **dargs)
File "/usr/local/autotest/common_lib/test.py", line 470, in execute
dargs)
File "/usr/local/autotest/common_lib/test.py", line 347, in _call_run_once_with_retry
postprocess_profiled_run, args, dargs)
File "/usr/local/autotest/common_lib/test.py", line 380, in _call_run_once
self.run_once(*args, **dargs)
File "/usr/local/autotest/tests/video_PlaybackPerf/video_PlaybackPerf.py", line 122, in run_once
keyvals = self.test_power(local_path)
File "/usr/local/autotest/tests/video_PlaybackPerf/video_PlaybackPerf.py", line 220, in test_power
return self.test_playback(local_path, get_power)
File "/usr/local/autotest/tests/video_PlaybackPerf/video_PlaybackPerf.py", line 258, in test_playback
self.start_playback(cr, local_path)
File "/usr/local/autotest/tests/video_PlaybackPerf/video_PlaybackPerf.py", line 72, in start_playback
cr.browser.platform.SetHTTPServerDirectories(self.bindir)
File "/usr/local/telemetry/src/third_party/catapult/telemetry/telemetry/core/platform.py", line 410, in SetHTTPServerDirectories
self.StartLocalServer(server)
File "/usr/local/telemetry/src/third_party/catapult/telemetry/telemetry/core/platform.py", line 377, in StartLocalServer
self._local_server_controller.StartServer(server)
File "/usr/local/telemetry/src/third_party/catapult/telemetry/telemetry/core/local_server.py", line 152, in StartServer
assert self._platform_backend.network_controller_backend.is_initialized
AssertionError
,
Apr 21 2017
+hiroh@ who has also been looking at failures in this test.
,
Apr 21 2017
video_WebRtcPerf has the same problem. Maybe all tests that use SetHTTPServerDirectories are broken.
R59-9461.0.0 auron_paine
09:16:01.899 WARNI| test:0615| The test failed with the following exception
Traceback (most recent call last):
File "/usr/local/autotest/common_lib/test.py", line 609, in _exec
_call_test_function(self.execute, *p_args, **p_dargs)
File "/usr/local/autotest/common_lib/test.py", line 823, in _call_test_function
raise error.UnhandledTestFail(e)
UnhandledTestFail: Unhandled AssertionError:
Traceback (most recent call last):
File "/usr/local/autotest/common_lib/test.py", line 817, in _call_test_function
return func(*args, **dargs)
File "/usr/local/autotest/common_lib/test.py", line 470, in execute
dargs)
File "/usr/local/autotest/common_lib/test.py", line 347, in _call_run_once_with_retry
postprocess_profiled_run, args, dargs)
File "/usr/local/autotest/common_lib/test.py", line 380, in _call_run_once
self.run_once(*args, **dargs)
File "/usr/local/autotest/tests/video_WebRtcPerf/video_WebRtcPerf.py", line 163, in run_once
keyvals = self.test_power(local_path)
File "/usr/local/autotest/tests/video_WebRtcPerf/video_WebRtcPerf.py", line 291, in test_power
return self.test_webrtc(local_path, get_power)
File "/usr/local/autotest/tests/video_WebRtcPerf/video_WebRtcPerf.py", line 212, in test_webrtc
self.start_loopback(cr)
File "/usr/local/autotest/tests/video_WebRtcPerf/video_WebRtcPerf.py", line 104, in start_loopback
cr.browser.platform.SetHTTPServerDirectories(self.bindir)
File "/usr/local/telemetry/src/third_party/catapult/telemetry/telemetry/core/platform.py", line 410, in SetHTTPServerDirectories
self.StartLocalServer(server)
File "/usr/local/telemetry/src/third_party/catapult/telemetry/telemetry/core/platform.py", line 377, in StartLocalServer
self._local_server_controller.StartServer(server)
File "/usr/local/telemetry/src/third_party/catapult/telemetry/telemetry/core/local_server.py", line 152, in StartServer
assert self._platform_backend.network_controller_backend.is_initialized
AssertionError
09:16:02.533 ERROR| parallel:0026| child process failed
,
Apr 21 2017
video_WebRtcPerf also started to fail in 9378.0.0.
,
Apr 21 2017
Re #10, Lab has some network issue in these two days due to lots of chrome crash. (See http://crbug.com/713004) It is getting recovered, maybe you can try it again.
,
Apr 21 2017
We need to add init_network_controller=True. See https://bugs.chromium.org/p/chromium/issues/detail?id=701629 for reference.
,
Apr 21 2017
Please also help add init_network_controller=True for video_WebRtcPerf.
,
Apr 22 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/646f5c8ea8a81489c5e22290033132e8be2137b3 commit 646f5c8ea8a81489c5e22290033132e8be2137b3 Author: Daniel Hung-yu Wu <hywu@chromium.org> Date: Sat Apr 22 04:57:45 2017 video tests: set init_network_controller=True Set init_network_controller=True while using SetHTTPServerDirectories. BUG= chromium:703015 Change-Id: I86b3a87d410f7b0ebdd3f2d932efa5f1c177ba46 Reviewed-on: https://chromium-review.googlesource.com/484203 Commit-Ready: Wu-cheng Li <wuchengli@chromium.org> Tested-by: Wu-cheng Li <wuchengli@chromium.org> Reviewed-by: Wu-cheng Li <wuchengli@chromium.org> [modify] https://crrev.com/646f5c8ea8a81489c5e22290033132e8be2137b3/client/site_tests/video_WebRtcPerf/video_WebRtcPerf.py [modify] https://crrev.com/646f5c8ea8a81489c5e22290033132e8be2137b3/client/site_tests/video_PlaybackPerf/video_PlaybackPerf.py
,
Apr 24 2017
,
May 5 2017
,
May 5 2017
Can you please mark which OS this is for?
,
May 5 2017
,
May 6 2017
Your change meets the bar and is auto-approved for M59. Please go ahead and merge the CL to branch 3071 manually. Please contact milestone owner if you have questions. Owners: amineer@(Android), cmasso@(iOS), gkihumba@(ChromeOS), Abdul Syed@(Desktop) For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
May 8 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/c8f3593e710288d95b65f978bc0f3b41eacd741b commit c8f3593e710288d95b65f978bc0f3b41eacd741b Author: Daniel Hung-yu Wu <hywu@chromium.org> Date: Mon May 08 06:09:44 2017 video tests: set init_network_controller=True Set init_network_controller=True while using SetHTTPServerDirectories. BUG= chromium:703015 Change-Id: I86b3a87d410f7b0ebdd3f2d932efa5f1c177ba46 Reviewed-on: https://chromium-review.googlesource.com/484203 Commit-Ready: Wu-cheng Li <wuchengli@chromium.org> Tested-by: Wu-cheng Li <wuchengli@chromium.org> Reviewed-by: Wu-cheng Li <wuchengli@chromium.org> (cherry picked from commit 646f5c8ea8a81489c5e22290033132e8be2137b3) Reviewed-on: https://chromium-review.googlesource.com/497231 Commit-Queue: Hung-yu Wu <hywu@chromium.org> Tested-by: Hung-yu Wu <hywu@chromium.org> Trybot-Ready: Hung-yu Wu <hywu@chromium.org> Reviewed-by: Kuang-che Wu <kcwu@chromium.org> [modify] https://crrev.com/c8f3593e710288d95b65f978bc0f3b41eacd741b/client/site_tests/video_WebRtcPerf/video_WebRtcPerf.py [modify] https://crrev.com/c8f3593e710288d95b65f978bc0f3b41eacd741b/client/site_tests/video_PlaybackPerf/video_PlaybackPerf.py
,
May 10 2017
This issue has been approved for a merge. Please merge the fix to any appropriate branches as soon as possible! If all merges have been completed, please remove any remaining Merge-Approved labels from this issue. Thanks for your time! To disable nags, add the Disable-Nags label. For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
May 10 2017
,
May 15 2017
Verified on M59 using https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=video_PlaybackPerf&releases=59&days_back=30 |
||||||||||||
►
Sign in to add a comment |
||||||||||||
Comment 1 by hywu@chromium.org
, Mar 22 2017