telemetry_perf_unittests (browse_accessibility:tools:gmail_compose) failing on chromium.chromiumos/Linux ChromiumOS Tests (1) |
||||||
Issue descriptiontelemetry_perf_unittests failing on chromium.chromiumos/Linux ChromiumOS Tests (1) Builders failed on: - Linux ChromiumOS Tests (1): https://build.chromium.org/p/chromium.chromiumos/builders/Linux%20ChromiumOS%20Tests%20%281%29 5 fails + 1 exception out of last 6 builds.
,
Sep 27 2017
Actually the test is rather flaky. It failed for the first time in https://build.chromium.org/p/chromium.chromiumos/builders/Linux%20ChromiumOS%20Tests%20%281%29/builds/45063. After that there were 7 green builds out of 26 builds.
,
Sep 27 2017
The test is benchmarks.system_health_smoke_test.SystemHealthBenchmarkSmokeTest.system_health.memory_desktop.browse_accessibility:tools:gmail_compose.
,
Sep 27 2017
The log is from https://build.chromium.org/p/chromium.chromiumos/builders/Linux%20ChromiumOS%20Tests%20%281%29/builds/45063 ========== [27/27] benchmarks.system_health_smoke_test.SystemHealthBenchmarkSmokeTest.system_health.memory_desktop.browse_accessibility:tools:gmail_compose failed unexpectedly 113.7266s: [ RUN ] browse_accessibility:tools:gmail_compose@{'case': 'browse_accessibility', 'group': 'tools'} Traceback (most recent call last): File "/b/s/w/ir/third_party/catapult/telemetry/telemetry/internal/story_runner.py", line 105, in _RunStoryAndProcessErrorIfNeeded state.RunStory(results) File "/b/s/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 52, in traced_function return func(*args, **kwargs) File "/b/s/w/ir/third_party/catapult/telemetry/telemetry/page/shared_page_state.py", line 332, in RunStory self._current_page.Run(self) File "/b/s/w/ir/third_party/catapult/telemetry/telemetry/page/__init__.py", line 114, in Run shared_state.page_test.RunNavigateSteps(self, current_tab) File "/b/s/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 52, in traced_function return func(*args, **kwargs) File "/b/s/w/ir/third_party/catapult/telemetry/telemetry/page/legacy_page_test.py", line 152, in RunNavigateSteps page.RunNavigateSteps(action_runner) File "/b/s/w/ir/tools/perf/page_sets/system_health/accessibility_stories.py", line 88, in RunNavigateSteps action_runner.WaitForElement(selector='#\\:gr') File "/b/s/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 52, in traced_function return func(*args, **kwargs) File "/b/s/w/ir/third_party/catapult/telemetry/telemetry/internal/actions/action_runner.py", line 290, in WaitForElement timeout_in_seconds=timeout_in_seconds)) File "/b/s/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 52, in traced_function return func(*args, **kwargs) File "/b/s/w/ir/third_party/catapult/telemetry/telemetry/internal/actions/action_runner.py", line 63, in _RunAction action.RunAction(self._tab) File "/b/s/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 52, in traced_function return func(*args, **kwargs) File "/b/s/w/ir/third_party/catapult/telemetry/telemetry/internal/actions/wait.py", line 22, in RunAction wait=True, timeout_in_seconds=self.timeout_in_seconds) File "/b/s/w/ir/third_party/catapult/telemetry/telemetry/internal/actions/page_action.py", line 127, in EvaluateCallbackWithElement tab.WaitForJavaScriptCondition(code, timeout=timeout_in_seconds) File "/b/s/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 52, in traced_function return func(*args, **kwargs) File "/b/s/w/ir/third_party/catapult/telemetry/telemetry/internal/browser/web_contents.py", line 239, in WaitForJavaScriptCondition return self._inspector_backend.WaitForJavaScriptCondition(*args, **kwargs) File "/b/s/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 52, in traced_function return func(*args, **kwargs) File "/b/s/w/ir/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_backend.py", line 302, in WaitForJavaScriptCondition e.message + '\n' + debug_message) TimeoutException: Timed out while waiting 60s for IsJavaScriptExpressionTrue. Console output: [ FAILED ] browse_accessibility:tools:gmail_compose@{'case': 'browse_accessibility', 'group': 'tools'} (91605 ms) ==========
,
Sep 27 2017
,
Sep 27 2017
Trying to disable the test, but this type is not in the guide :)
,
Sep 27 2017
I am not able to disable it, I do not know which storyset the test belongs to. I am reverting the CL instead. Please update the sheriff guide https://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium#TOC-How-do-I-disable-a-flaky-test- with a way to disable your tests.
,
Sep 27 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/9edd0dee2801e92d01cc480da7607fbaf8e0f98a commit 9edd0dee2801e92d01cc480da7607fbaf8e0f98a Author: vitaliii <vitaliii@chromium.org> Date: Wed Sep 27 09:32:26 2017 Revert "Add accessibility coverage to system_health benchmark." This reverts commit a5edee6703dcbcdad0a19ef19a4636641c9af7b9. Reason for revert: browse_accessibility:tools:gmail_compose is flaky. I would happily disable just this test, but I don't know how. I tried, but I do not know which story set it belongs to. Please update the sheriff guide https://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium#TOC-How-do-I-disable-a-flaky-test- with a description how to disable your tests. Original change's description: > Add accessibility coverage to system_health benchmark. > > Adds new stories that exercise things that sometimes cause performance > problems for accessibility: live regions, large data tables, scrolling > large complex sites, and typing in a contenteditable, and expands the > system_health benchmark to track accessibility metrics. > > Bug: 712417 > Change-Id: I8384083146650b3fae35ea223892053a7e0c0be4 > Reviewed-on: https://chromium-review.googlesource.com/663208 > Commit-Queue: Dominic Mazzoni <dmazzoni@chromium.org> > Reviewed-by: Ned Nguyen <nednguyen@google.com> > Reviewed-by: Juan Antonio Navarro Pérez <perezju@chromium.org> > Cr-Commit-Position: refs/heads/master@{#504475} TBR=dmazzoni@chromium.org,perezju@chromium.org,nednguyen@google.com Change-Id: Ib3357cca3154eee6925e3a12eacc3dbb0b46526f No-Presubmit: true No-Tree-Checks: true No-Try: true Bug: 712417 , 769217 Reviewed-on: https://chromium-review.googlesource.com/686454 Reviewed-by: vitaliii <vitaliii@chromium.org> Commit-Queue: vitaliii <vitaliii@chromium.org> Cr-Commit-Position: refs/heads/master@{#504609} [modify] https://crrev.com/9edd0dee2801e92d01cc480da7607fbaf8e0f98a/tools/perf/benchmarks/system_health.py [modify] https://crrev.com/9edd0dee2801e92d01cc480da7607fbaf8e0f98a/tools/perf/page_sets/data/system_health_desktop.json [delete] https://crrev.com/051a3c9b3603f18c80aa98c7411ec3f6cc4e6674/tools/perf/page_sets/data/system_health_desktop_059.wprgo.sha1 [delete] https://crrev.com/051a3c9b3603f18c80aa98c7411ec3f6cc4e6674/tools/perf/page_sets/system_health/accessibility_stories.py [modify] https://crrev.com/9edd0dee2801e92d01cc480da7607fbaf8e0f98a/tools/perf/page_sets/system_health/story_tags.py [modify] https://crrev.com/9edd0dee2801e92d01cc480da7607fbaf8e0f98a/tools/perf/page_sets/system_health/system_health_story.py
,
Sep 27 2017
,
Sep 27 2017
The test is now disabled so we can lower Pri. +rnephew, and ned, we should think about improving the disabling mechanism + docs as sheriffs are finding it confusing.
,
Sep 27 2017
I updated the instruction in https://sites.google.com/a/chromium.org/dev/developers/tree-sheriffs/sheriff-details-chromium, PTAL.
,
Sep 27 2017
Thank you for a quick update. > "on the test method (example)." Could you make the "(example)" a link?
,
Sep 27 2017
#12: I added example link.
,
Sep 27 2017
thank you!
,
Oct 5 2017
This test still seems to be flaking, see https://chromium-swarm.appspot.com/task?id=3904c93aee210310&refresh=10&show_raw=1
,
Oct 5 2017
Oh, I should have mentioned, that failure (which matches the trace above) is on Mac.
,
Nov 16 2017
Fixed in a different bug |
||||||
►
Sign in to add a comment |
||||||
Comment 1 by vitaliii@chromium.org
, Sep 27 2017