New issue
Advanced search Search tips

Issue 876636 link

Starred by 1 user

Issue metadata

Status: Assigned
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 3
Type: Bug



Sign in to add a comment

Telemetry benchmark loading.desktop is flaky on ChromeOS

Project Member Reported by vovoy@chromium.org, Aug 22

Issue description

Test command:
$ ./run_benchmark --browser=cros-chrome --remote=DUT_IP --story-tag-filter=typical loading.desktop

Some flaky pages:
TheOnion_cold
TheOnion_warm
AllRecipes_warm

example error log:
(ERROR) 2018-08-22 15:57:55,103 story_runner.ProcessError:100  Handleable error
Traceback (most recent call last):
  File "chromium/src/third_party/catapult/telemetry/telemetry/internal/story_runner.py", line 126, in _RunStoryAndProcessErrorIfNeeded
    state.RunStory(results)
  File "chromium/src/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 75, in traced_function
    return func(*args, **kwargs)
  File "chromium/src/third_party/catapult/telemetry/telemetry/page/shared_page_state.py", line 300, in RunStory
    self._current_page.Run(self)
  File "chromium/src/third_party/catapult/telemetry/telemetry/page/__init__.py", line 99, in Run
    self.RunPageInteractions(action_runner)
  File "chromium/src/tools/perf/page_sets/page_cycler_story.py", line 42, in RunPageInteractions
    _WEB_CONTENTS_TIMEOUT)
  File "chromium/src/third_party/catapult/common/py_utils/py_utils/__init__.py", line 148, in WaitFor
    (timeout, GetConditionString()))
TimeoutException: Timed out while waiting 180s for HasReachedQuiescence.
 
Cc: sullivan@google.com tdres...@chromium.org
HasReachedQuiescence can be flaky. IMO, the best fix is to manually update the wait condition of these pages to wait for some specific dom element.
Project Member

Comment 2 by bugdroid1@chromium.org, Aug 22

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/915214c2cce83f264385e0db77d3d4468aed1301

commit 915214c2cce83f264385e0db77d3d4468aed1301
Author: Kuo-Hsin Yang <vovoy@chromium.org>
Date: Wed Aug 22 15:45:03 2018

Disable flaky loading pages on ChromeOS

NOTRY=true

Bug: chromium:876636
Change-Id: I516f4f2920b7fa99c1cab1b0ab4dcef0053f0564
Reviewed-on: https://chromium-review.googlesource.com/1184740
Commit-Queue: Juan Antonio Navarro Pérez <perezju@chromium.org>
Reviewed-by: Juan Antonio Navarro Pérez <perezju@chromium.org>
Cr-Commit-Position: refs/heads/master@{#585026}
[modify] https://crrev.com/915214c2cce83f264385e0db77d3d4468aed1301/tools/perf/expectations.config

Project Member

Comment 3 by bugdroid1@chromium.org, Sep 6

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/0da7dce9a83cbad841f925002a4ddfee8f2afcf7

commit 0da7dce9a83cbad841f925002a4ddfee8f2afcf7
Author: Ned Nguyen <nednguyen@google.com>
Date: Thu Sep 06 00:58:52 2018

Disable flaky loading tests on win 10 perf config

NOTRY=true
TBR=kouhei@chromium.org, ksakamoto@chromium.org

Bug: 876636, 879833
Change-Id: I6d2963cbc6ca33da5e62e68a17d19330786f864a
Reviewed-on: https://chromium-review.googlesource.com/1208960
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Commit-Queue: Ned Nguyen <nednguyen@google.com>
Cr-Commit-Position: refs/heads/master@{#589076}
[modify] https://crrev.com/0da7dce9a83cbad841f925002a4ddfee8f2afcf7/tools/perf/expectations.config

AllRecipes_cold is also flaky on ChromeOS

Error log:

(ERROR) 2018-09-11 18:35:21,391 page_test_results.Fail:540  Failure recorded: Exception raised running AllRecipes_cold
(INFO) 2018-09-11 18:35:21,399 chrome_tracing_agent.RecordClockSyncMarker:180  Chrome version: 3544
(INFO) 2018-09-11 18:35:21,404 chrome_tracing_agent._RemoveTraceConfigFile:294  Remove trace config file in /tmp/chrome-trace-config.json
(INFO) 2018-09-11 18:36:21,496 exception_formatter.PrintFormattedException:19  Try printing formatted exception: None None None

Exception raised when cleaning story run: 

Traceback (most recent call last):
  _RunStoryAndProcessErrorIfNeeded at /home/chromeos-test/images/veyron_jaq-release/R71-11057.0.0/telemetry_src/src/third_party/catapult/telemetry/telemetry/internal/story_runner.py:153
    test.DidRunStory(state.platform, results)
  DidRunStory at /home/chromeos-test/images/veyron_jaq-release/R71-11057.0.0/telemetry_src/src/third_party/catapult/telemetry/telemetry/web_perf/timeline_based_measurement.py:288
    trace_result, _ = platform.tracing_controller.StopTracing()
  StopTracing at /home/chromeos-test/images/veyron_jaq-release/R71-11057.0.0/telemetry_src/src/third_party/catapult/telemetry/telemetry/core/tracing_controller.py:53
    return self._tracing_controller_backend.StopTracing()
  StopTracing at /home/chromeos-test/images/veyron_jaq-release/R71-11057.0.0/telemetry_src/src/third_party/catapult/telemetry/telemetry/internal/platform/tracing_controller_backend.py:157
    '\n'.join(raised_exception_messages))
TracingException: Exceptions raised when trying to stop tracing:
Traceback (most recent call last):
  File "/home/chromeos-test/images/veyron_jaq-release/R71-11057.0.0/telemetry_src/src/third_party/catapult/telemetry/telemetry/internal/platform/tracing_controller_backend.py", line 145, in StopTracing
    agent.CollectAgentTraceData(builder)
  File "/home/chromeos-test/images/veyron_jaq-release/R71-11057.0.0/telemetry_src/src/third_party/catapult/telemetry/telemetry/internal/platform/tracing_agent/chrome_tracing_agent.py", line 248, in CollectAgentTraceData
    '\n'.join(raised_exception_messages))
ChromeTracingStoppedError: Exceptions raised when trying to collect Chrome devtool tracing:
Error when collecting Chrome tracing on devtools at port 37607:
Traceback (most recent call last):
  File "/home/chromeos-test/images/veyron_jaq-release/R71-11057.0.0/telemetry_src/src/third_party/catapult/telemetry/telemetry/internal/platform/tracing_agent/chrome_tracing_agent.py", line 237, in CollectAgentTraceData
    client.CollectChromeTracingData(trace_data_builder)
  File "/home/chromeos-test/images/veyron_jaq-release/R71-11057.0.0/telemetry_src/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/devtools_client_backend.py", line 489, in CollectChromeTracingData
    self._tracing_backend.CollectTraceData(trace_data_builder, timeout)
  File "/home/chromeos-test/images/veyron_jaq-release/R71-11057.0.0/telemetry_src/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/tracing_backend.py", line 237, in CollectTraceData
    self._CollectTracingData(trace_data_builder, timeout)
  File "/home/chromeos-test/images/veyron_jaq-release/R71-11057.0.0/telemetry_src/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/tracing_backend.py", line 279, in _CollectTracingData
    'the timeout amount.' % elapsed_time)
TracingTimeoutException: Only received partial trace data due to timeout after 60.0071730614 seconds. If the trace data is big, you may want to increase the timeout amount.
Project Member

Comment 5 by bugdroid1@chromium.org, Sep 12

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/2de8f4f0f966637c76152b54aee275dcc49fbbb9

commit 2de8f4f0f966637c76152b54aee275dcc49fbbb9
Author: Kuo-Hsin Yang <vovoy@chromium.org>
Date: Wed Sep 12 11:56:10 2018

Disable loading.desktop/AllRecipes_cold on ChromeOS

Bug: chromium:876636
Change-Id: Iad48e0c8256d612ec971add32adceed39bffd1b5
Reviewed-on: https://chromium-review.googlesource.com/1220931
Commit-Queue: Ned Nguyen <nednguyen@google.com>
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Cr-Commit-Position: refs/heads/master@{#590648}
[modify] https://crrev.com/2de8f4f0f966637c76152b54aee275dcc49fbbb9/tools/perf/expectations.config

Project Member

Comment 6 by bugdroid1@chromium.org, Sep 24

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/49bd0e64188e49be0c9061f75b75190accc72a05

commit 49bd0e64188e49be0c9061f75b75190accc72a05
Author: Ned Nguyen <nednguyen@google.com>
Date: Mon Sep 24 13:30:48 2018

Disable loading.desktop/TheOnion_warm on win 7

This is flaky on 'Win 7 Nvidia GPU Perf'

Bug:876636
Change-Id: Ide6e0ceff3b7963f86b7160345a3239542d476c1

TBR=kouhei@chromium.org, ksakamoto@chromium.org

No-Try: true
Change-Id: Ide6e0ceff3b7963f86b7160345a3239542d476c1
Reviewed-on: https://chromium-review.googlesource.com/1240316
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Commit-Queue: Ned Nguyen <nednguyen@google.com>
Cr-Commit-Position: refs/heads/master@{#593521}
[modify] https://crrev.com/49bd0e64188e49be0c9061f75b75190accc72a05/tools/perf/expectations.config

Comment 7 by benhenry@google.com, Jan 16 (6 days ago)

Components: Test>Telemetry

Comment 8 by benhenry@google.com, Jan 16 (6 days ago)

Components: -Tests>Telemetry

Sign in to add a comment