New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 809207 link

Starred by 1 user

Issue metadata

Status: Duplicate
Merged: issue 780907
Owner:
Closed: Feb 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Bug-Regression



Sign in to add a comment

2458.3% regression in media.desktop at 533517:533601

Project Member Reported by nedngu...@google.com, Feb 5 2018

Issue description

This benchmark used to take 20 minutes, now it's taking 65 minutes, which is too much.
 
All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=809207

(For debugging:) Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?sid=3594733bf4c388e6c45d5cf273e5934d3e239de7423a6c5d8ce0e4df10594b4a


Bot(s) for this bug's original alert(s):

chromium-rel-mac-retina
Cc: -nedngu...@google.com nednguyen@chromium.org
Components: -Speed>Telemetry Speed>Benchmarks
Labels: -Pri-2 Pri-1
Owner: crouleau@chromium.org
Status: Assigned (was: Untriaged)
Cc: xhw...@chromium.org crouleau@chromium.org
Owner: charliea@chromium.org
+ Media Perf Sheriff.

Looks like a BattOr bug:

Traceback (most recent call last):
  File "/b/s/w/ir/third_party/catapult/telemetry/telemetry/internal/platform/tracing_controller_backend.py", line 78, in _CollectNonfatalException
    yield
  File "/b/s/w/ir/third_party/catapult/telemetry/telemetry/internal/platform/tracing_controller_backend.py", line 112, in StartTracing
    if agent.StartAgentTracing(config, timeout):
  File "/b/s/w/ir/third_party/catapult/telemetry/telemetry/internal/platform/tracing_agent/battor_tracing_agent.py", line 73, in StartAgentTracing
    self._battor.StartTracing()
  File "/b/s/w/ir/third_party/catapult/common/battor/battor/battor_wrapper.py", line 234, in StartTracing
    self._SendBattOrCommand(self._START_TRACING_CMD)
  File "/b/s/w/ir/third_party/catapult/common/battor/battor/battor_wrapper.py", line 358, in _SendBattOrCommand
    'Outputted: %s' % (cmd, status))
BattOrError: BattOr did not complete command 'StartTracing' correctly.
Outputted: [0201/010516.085844:FATAL:battor_agent_bin.cc(100)] Fatal error when communicating with the BattOr: TOO MANY COMMAND RETRIES

https://logs.chromium.org/v/?s=chrome%2Fbb%2Fchromium.perf%2FMac_Retina_Perf%2F1746%2F%2B%2Frecipes%2Fsteps%2Fmedia.desktop_on_Intel_GPU_on_Mac_on_Mac-10.11%2F0%2Fstdout
📍 Couldn't reproduce a difference.
https://pinpoint-dot-chromeperf.appspot.com/job/16d530d1840000
Status: duplicated (was: Assigned)
This is actually an interaction between two different bugs: 

- https://bugs.chromium.org/p/chromium/issues/detail?id=755981, where BattOr tests time out rather than Telemetry failing if a BattOr error occurs
- https://bugs.chromium.org/p/chromium/issues/detail?id=780907, which greatly increases the likelihood that, especially on Mac, BattOrs will fail when downloading traces and eventually cause the BattOr to become unresponsive

The BattOr on this mac became unresponsive, causing all BattOr-using benchmarks on this host to start to time out instead of running normally. The problem went away when the BattOr was reset (in response to another bug), as you can see by the recent drop in the benchmark duration.

aschulman@ has a pending CL (https://chromium-review.googlesource.com/c/chromium/src/+/908252) which should drastically improve the BattOr resilience.
Mergedinto: 780907
Status: Duplicate (was: duplicated)

Sign in to add a comment