Telemetry benchmark loading.desktop is flaky on ChromeOS |
|||
Issue description
Test command:
$ ./run_benchmark --browser=cros-chrome --remote=DUT_IP --story-tag-filter=typical loading.desktop
Some flaky pages:
TheOnion_cold
TheOnion_warm
AllRecipes_warm
example error log:
(ERROR) 2018-08-22 15:57:55,103 story_runner.ProcessError:100 Handleable error
Traceback (most recent call last):
File "chromium/src/third_party/catapult/telemetry/telemetry/internal/story_runner.py", line 126, in _RunStoryAndProcessErrorIfNeeded
state.RunStory(results)
File "chromium/src/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 75, in traced_function
return func(*args, **kwargs)
File "chromium/src/third_party/catapult/telemetry/telemetry/page/shared_page_state.py", line 300, in RunStory
self._current_page.Run(self)
File "chromium/src/third_party/catapult/telemetry/telemetry/page/__init__.py", line 99, in Run
self.RunPageInteractions(action_runner)
File "chromium/src/tools/perf/page_sets/page_cycler_story.py", line 42, in RunPageInteractions
_WEB_CONTENTS_TIMEOUT)
File "chromium/src/third_party/catapult/common/py_utils/py_utils/__init__.py", line 148, in WaitFor
(timeout, GetConditionString()))
TimeoutException: Timed out while waiting 180s for HasReachedQuiescence.
,
Aug 22
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/915214c2cce83f264385e0db77d3d4468aed1301 commit 915214c2cce83f264385e0db77d3d4468aed1301 Author: Kuo-Hsin Yang <vovoy@chromium.org> Date: Wed Aug 22 15:45:03 2018 Disable flaky loading pages on ChromeOS NOTRY=true Bug: chromium:876636 Change-Id: I516f4f2920b7fa99c1cab1b0ab4dcef0053f0564 Reviewed-on: https://chromium-review.googlesource.com/1184740 Commit-Queue: Juan Antonio Navarro Pérez <perezju@chromium.org> Reviewed-by: Juan Antonio Navarro Pérez <perezju@chromium.org> Cr-Commit-Position: refs/heads/master@{#585026} [modify] https://crrev.com/915214c2cce83f264385e0db77d3d4468aed1301/tools/perf/expectations.config
,
Sep 6
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/0da7dce9a83cbad841f925002a4ddfee8f2afcf7 commit 0da7dce9a83cbad841f925002a4ddfee8f2afcf7 Author: Ned Nguyen <nednguyen@google.com> Date: Thu Sep 06 00:58:52 2018 Disable flaky loading tests on win 10 perf config NOTRY=true TBR=kouhei@chromium.org, ksakamoto@chromium.org Bug: 876636, 879833 Change-Id: I6d2963cbc6ca33da5e62e68a17d19330786f864a Reviewed-on: https://chromium-review.googlesource.com/1208960 Reviewed-by: Ned Nguyen <nednguyen@google.com> Commit-Queue: Ned Nguyen <nednguyen@google.com> Cr-Commit-Position: refs/heads/master@{#589076} [modify] https://crrev.com/0da7dce9a83cbad841f925002a4ddfee8f2afcf7/tools/perf/expectations.config
,
Sep 12
AllRecipes_cold is also flaky on ChromeOS
Error log:
(ERROR) 2018-09-11 18:35:21,391 page_test_results.Fail:540 Failure recorded: Exception raised running AllRecipes_cold
(INFO) 2018-09-11 18:35:21,399 chrome_tracing_agent.RecordClockSyncMarker:180 Chrome version: 3544
(INFO) 2018-09-11 18:35:21,404 chrome_tracing_agent._RemoveTraceConfigFile:294 Remove trace config file in /tmp/chrome-trace-config.json
(INFO) 2018-09-11 18:36:21,496 exception_formatter.PrintFormattedException:19 Try printing formatted exception: None None None
Exception raised when cleaning story run:
Traceback (most recent call last):
_RunStoryAndProcessErrorIfNeeded at /home/chromeos-test/images/veyron_jaq-release/R71-11057.0.0/telemetry_src/src/third_party/catapult/telemetry/telemetry/internal/story_runner.py:153
test.DidRunStory(state.platform, results)
DidRunStory at /home/chromeos-test/images/veyron_jaq-release/R71-11057.0.0/telemetry_src/src/third_party/catapult/telemetry/telemetry/web_perf/timeline_based_measurement.py:288
trace_result, _ = platform.tracing_controller.StopTracing()
StopTracing at /home/chromeos-test/images/veyron_jaq-release/R71-11057.0.0/telemetry_src/src/third_party/catapult/telemetry/telemetry/core/tracing_controller.py:53
return self._tracing_controller_backend.StopTracing()
StopTracing at /home/chromeos-test/images/veyron_jaq-release/R71-11057.0.0/telemetry_src/src/third_party/catapult/telemetry/telemetry/internal/platform/tracing_controller_backend.py:157
'\n'.join(raised_exception_messages))
TracingException: Exceptions raised when trying to stop tracing:
Traceback (most recent call last):
File "/home/chromeos-test/images/veyron_jaq-release/R71-11057.0.0/telemetry_src/src/third_party/catapult/telemetry/telemetry/internal/platform/tracing_controller_backend.py", line 145, in StopTracing
agent.CollectAgentTraceData(builder)
File "/home/chromeos-test/images/veyron_jaq-release/R71-11057.0.0/telemetry_src/src/third_party/catapult/telemetry/telemetry/internal/platform/tracing_agent/chrome_tracing_agent.py", line 248, in CollectAgentTraceData
'\n'.join(raised_exception_messages))
ChromeTracingStoppedError: Exceptions raised when trying to collect Chrome devtool tracing:
Error when collecting Chrome tracing on devtools at port 37607:
Traceback (most recent call last):
File "/home/chromeos-test/images/veyron_jaq-release/R71-11057.0.0/telemetry_src/src/third_party/catapult/telemetry/telemetry/internal/platform/tracing_agent/chrome_tracing_agent.py", line 237, in CollectAgentTraceData
client.CollectChromeTracingData(trace_data_builder)
File "/home/chromeos-test/images/veyron_jaq-release/R71-11057.0.0/telemetry_src/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/devtools_client_backend.py", line 489, in CollectChromeTracingData
self._tracing_backend.CollectTraceData(trace_data_builder, timeout)
File "/home/chromeos-test/images/veyron_jaq-release/R71-11057.0.0/telemetry_src/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/tracing_backend.py", line 237, in CollectTraceData
self._CollectTracingData(trace_data_builder, timeout)
File "/home/chromeos-test/images/veyron_jaq-release/R71-11057.0.0/telemetry_src/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/tracing_backend.py", line 279, in _CollectTracingData
'the timeout amount.' % elapsed_time)
TracingTimeoutException: Only received partial trace data due to timeout after 60.0071730614 seconds. If the trace data is big, you may want to increase the timeout amount.
,
Sep 12
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/2de8f4f0f966637c76152b54aee275dcc49fbbb9 commit 2de8f4f0f966637c76152b54aee275dcc49fbbb9 Author: Kuo-Hsin Yang <vovoy@chromium.org> Date: Wed Sep 12 11:56:10 2018 Disable loading.desktop/AllRecipes_cold on ChromeOS Bug: chromium:876636 Change-Id: Iad48e0c8256d612ec971add32adceed39bffd1b5 Reviewed-on: https://chromium-review.googlesource.com/1220931 Commit-Queue: Ned Nguyen <nednguyen@google.com> Reviewed-by: Ned Nguyen <nednguyen@google.com> Cr-Commit-Position: refs/heads/master@{#590648} [modify] https://crrev.com/2de8f4f0f966637c76152b54aee275dcc49fbbb9/tools/perf/expectations.config
,
Sep 24
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/49bd0e64188e49be0c9061f75b75190accc72a05 commit 49bd0e64188e49be0c9061f75b75190accc72a05 Author: Ned Nguyen <nednguyen@google.com> Date: Mon Sep 24 13:30:48 2018 Disable loading.desktop/TheOnion_warm on win 7 This is flaky on 'Win 7 Nvidia GPU Perf' Bug:876636 Change-Id: Ide6e0ceff3b7963f86b7160345a3239542d476c1 TBR=kouhei@chromium.org, ksakamoto@chromium.org No-Try: true Change-Id: Ide6e0ceff3b7963f86b7160345a3239542d476c1 Reviewed-on: https://chromium-review.googlesource.com/1240316 Reviewed-by: Ned Nguyen <nednguyen@google.com> Commit-Queue: Ned Nguyen <nednguyen@google.com> Cr-Commit-Position: refs/heads/master@{#593521} [modify] https://crrev.com/49bd0e64188e49be0c9061f75b75190accc72a05/tools/perf/expectations.config
,
Jan 16
(6 days ago)
,
Jan 16
(6 days ago)
|
|||
►
Sign in to add a comment |
|||
Comment 1 by nednguyen@chromium.org
, Aug 22