New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 769217 link

Starred by 1 user

Issue metadata

Status: Archived
Owner:
Closed: Nov 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Linux , Mac
Pri: 2
Type: Bug



Sign in to add a comment

telemetry_perf_unittests (browse_accessibility:tools:gmail_compose) failing on chromium.chromiumos/Linux ChromiumOS Tests (1)

Project Member Reported by vitaliii@chromium.org, Sep 27 2017

Issue description

telemetry_perf_unittests failing on chromium.chromiumos/Linux ChromiumOS Tests (1)

Builders failed on: 
- Linux ChromiumOS Tests (1): 
  https://build.chromium.org/p/chromium.chromiumos/builders/Linux%20ChromiumOS%20Tests%20%281%29


5 fails + 1 exception out of last 6 builds.

 
The exception seems to be a flake (see issue 769213).
Actually the test is rather flaky.

It failed for the first time in https://build.chromium.org/p/chromium.chromiumos/builders/Linux%20ChromiumOS%20Tests%20%281%29/builds/45063.

After that there were 7 green builds out of 26 builds.
The test is benchmarks.system_health_smoke_test.SystemHealthBenchmarkSmokeTest.system_health.memory_desktop.browse_accessibility:tools:gmail_compose.
The log is

from https://build.chromium.org/p/chromium.chromiumos/builders/Linux%20ChromiumOS%20Tests%20%281%29/builds/45063

==========
[27/27] benchmarks.system_health_smoke_test.SystemHealthBenchmarkSmokeTest.system_health.memory_desktop.browse_accessibility:tools:gmail_compose failed unexpectedly 113.7266s:
  [ RUN      ] browse_accessibility:tools:gmail_compose@{'case': 'browse_accessibility', 'group': 'tools'}
  Traceback (most recent call last):
    File "/b/s/w/ir/third_party/catapult/telemetry/telemetry/internal/story_runner.py", line 105, in _RunStoryAndProcessErrorIfNeeded
      state.RunStory(results)
    File "/b/s/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 52, in traced_function
      return func(*args, **kwargs)
    File "/b/s/w/ir/third_party/catapult/telemetry/telemetry/page/shared_page_state.py", line 332, in RunStory
      self._current_page.Run(self)
    File "/b/s/w/ir/third_party/catapult/telemetry/telemetry/page/__init__.py", line 114, in Run
      shared_state.page_test.RunNavigateSteps(self, current_tab)
    File "/b/s/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 52, in traced_function
      return func(*args, **kwargs)
    File "/b/s/w/ir/third_party/catapult/telemetry/telemetry/page/legacy_page_test.py", line 152, in RunNavigateSteps
      page.RunNavigateSteps(action_runner)
    File "/b/s/w/ir/tools/perf/page_sets/system_health/accessibility_stories.py", line 88, in RunNavigateSteps
      action_runner.WaitForElement(selector='#\\:gr')
    File "/b/s/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 52, in traced_function
      return func(*args, **kwargs)
    File "/b/s/w/ir/third_party/catapult/telemetry/telemetry/internal/actions/action_runner.py", line 290, in WaitForElement
      timeout_in_seconds=timeout_in_seconds))
    File "/b/s/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 52, in traced_function
      return func(*args, **kwargs)
    File "/b/s/w/ir/third_party/catapult/telemetry/telemetry/internal/actions/action_runner.py", line 63, in _RunAction
      action.RunAction(self._tab)
    File "/b/s/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 52, in traced_function
      return func(*args, **kwargs)
    File "/b/s/w/ir/third_party/catapult/telemetry/telemetry/internal/actions/wait.py", line 22, in RunAction
      wait=True, timeout_in_seconds=self.timeout_in_seconds)
    File "/b/s/w/ir/third_party/catapult/telemetry/telemetry/internal/actions/page_action.py", line 127, in EvaluateCallbackWithElement
      tab.WaitForJavaScriptCondition(code, timeout=timeout_in_seconds)
    File "/b/s/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 52, in traced_function
      return func(*args, **kwargs)
    File "/b/s/w/ir/third_party/catapult/telemetry/telemetry/internal/browser/web_contents.py", line 239, in WaitForJavaScriptCondition
      return self._inspector_backend.WaitForJavaScriptCondition(*args, **kwargs)
    File "/b/s/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 52, in traced_function
      return func(*args, **kwargs)
    File "/b/s/w/ir/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_backend.py", line 302, in WaitForJavaScriptCondition
      e.message + '\n' + debug_message)
  TimeoutException: Timed out while waiting 60s for IsJavaScriptExpressionTrue.
  Console output:
  
  
  [  FAILED  ] browse_accessibility:tools:gmail_compose@{'case': 'browse_accessibility', 'group': 'tools'} (91605 ms)
==========
Trying to disable the test, but this type is not in the guide :)
I am not able to disable it, I do not know which storyset the test belongs to.

I am reverting the CL instead.

Please update the sheriff guide https://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium#TOC-How-do-I-disable-a-flaky-test- with a way to disable your tests.
Project Member

Comment 8 by bugdroid1@chromium.org, Sep 27 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/9edd0dee2801e92d01cc480da7607fbaf8e0f98a

commit 9edd0dee2801e92d01cc480da7607fbaf8e0f98a
Author: vitaliii <vitaliii@chromium.org>
Date: Wed Sep 27 09:32:26 2017

Revert "Add accessibility coverage to system_health benchmark."

This reverts commit a5edee6703dcbcdad0a19ef19a4636641c9af7b9.

Reason for revert: browse_accessibility:tools:gmail_compose is flaky.
I would happily disable just this test, but I don't know how. I tried, but I do not know which story set it belongs to.

Please update the sheriff guide https://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium#TOC-How-do-I-disable-a-flaky-test- with a description how to disable your tests.

Original change's description:
> Add accessibility coverage to system_health benchmark.
> 
> Adds new stories that exercise things that sometimes cause performance
> problems for accessibility: live regions, large data tables, scrolling
> large complex sites, and typing in a contenteditable, and expands the
> system_health benchmark to track accessibility metrics.
> 
> Bug:  712417 
> Change-Id: I8384083146650b3fae35ea223892053a7e0c0be4
> Reviewed-on: https://chromium-review.googlesource.com/663208
> Commit-Queue: Dominic Mazzoni <dmazzoni@chromium.org>
> Reviewed-by: Ned Nguyen <nednguyen@google.com>
> Reviewed-by: Juan Antonio Navarro Pérez <perezju@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#504475}

TBR=dmazzoni@chromium.org,perezju@chromium.org,nednguyen@google.com

Change-Id: Ib3357cca3154eee6925e3a12eacc3dbb0b46526f
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug:  712417 , 769217 
Reviewed-on: https://chromium-review.googlesource.com/686454
Reviewed-by: vitaliii <vitaliii@chromium.org>
Commit-Queue: vitaliii <vitaliii@chromium.org>
Cr-Commit-Position: refs/heads/master@{#504609}
[modify] https://crrev.com/9edd0dee2801e92d01cc480da7607fbaf8e0f98a/tools/perf/benchmarks/system_health.py
[modify] https://crrev.com/9edd0dee2801e92d01cc480da7607fbaf8e0f98a/tools/perf/page_sets/data/system_health_desktop.json
[delete] https://crrev.com/051a3c9b3603f18c80aa98c7411ec3f6cc4e6674/tools/perf/page_sets/data/system_health_desktop_059.wprgo.sha1
[delete] https://crrev.com/051a3c9b3603f18c80aa98c7411ec3f6cc4e6674/tools/perf/page_sets/system_health/accessibility_stories.py
[modify] https://crrev.com/9edd0dee2801e92d01cc480da7607fbaf8e0f98a/tools/perf/page_sets/system_health/story_tags.py
[modify] https://crrev.com/9edd0dee2801e92d01cc480da7607fbaf8e0f98a/tools/perf/page_sets/system_health/system_health_story.py

Cc: perezju@chromium.org nednguyen@chromium.org
Components: Tests>Telemetry Tests>Flaky
Labels: OS-Linux Pri-1 Type-Bug
Owner: dmazz...@chromium.org
Status: Assigned (was: Started)
Cc: rnep...@chromium.org
Labels: -Pri-1 Pri-2
The test is now disabled so we can lower Pri.

+rnephew, and ned, we should think about improving the disabling mechanism + docs as sheriffs are finding it confusing.
Thank you for a quick update.

> "on the test method (example)." 

Could you make the "(example)" a link?
#12: I added example link. 
Labels: -Sheriff-Chromium
thank you!
Labels: OS-Mac
Oh, I should have mentioned, that failure (which matches the trace above) is on Mac.
Status: Archived (was: Assigned)
Fixed in a different bug

Sign in to add a comment