New issue
Advanced search Search tips

Issue 736868 link

Starred by 1 user

Issue metadata

Status: Available
Owner: ----
Components:
EstimatedDays: ----
NextAction: ----
OS: Windows
Pri: 2
Type: Bug



Sign in to add a comment

system_health.memory_desktop flaky on win10 perf bot

Project Member Reported by rnep...@chromium.org, Jun 26 2017

Issue description

https://build.chromium.org/p/chromium.perf/builders/Win%2010%20Perf?numbuilds=200

multitab:misc:typical24 and load:search:yahoo
https://build.chromium.org/p/chromium.perf/builders/Win%2010%20Perf/builds/975
https://build.chromium.org/p/chromium.perf/builders/Win%2010%20Perf/builds/997

ARNING:root:Either tab has crashed or browser does not support taking tab screenshot. Skip taking screenshot on failure.
Traceback (most recent call last):
  File "c:\b\s\w\ir\third_party\catapult\telemetry\telemetry\internal\story_runner.py", line 99, in _RunStoryAndProcessErrorIfNeeded
    state.RunStory(results)
  File "c:\b\s\w\ir\third_party\catapult\common\py_trace_event\py_trace_event\trace_event_impl\decorators.py", line 75, in traced_function
    return func(*args, **kwargs)
  File "c:\b\s\w\ir\third_party\catapult\telemetry\telemetry\page\shared_page_state.py", line 296, in RunStory
    self._current_page.Run(self)
  File "c:\b\s\w\ir\third_party\catapult\telemetry\telemetry\page\__init__.py", line 112, in Run
    self.RunPageInteractions(action_runner)
  File "c:\b\s\w\ir\tools\perf\page_sets\system_health\system_health_story.py", line 143, in RunPageInteractions
    self._Measure(action_runner)
  File "c:\b\s\w\ir\tools\perf\page_sets\system_health\system_health_story.py", line 126, in _Measure
    action_runner.MeasureMemory(deterministic_mode=True)
  File "c:\b\s\w\ir\third_party\catapult\common\py_trace_event\py_trace_event\trace_event_impl\decorators.py", line 75, in traced_function
    return func(*args, **kwargs)
  File "c:\b\s\w\ir\third_party\catapult\telemetry\telemetry\internal\actions\action_runner.py", line 158, in MeasureMemory
    dump_id = self.tab.browser.DumpMemory()
  File "c:\b\s\w\ir\third_party\catapult\telemetry\telemetry\internal\browser\browser.py", line 342, in DumpMemory
    return self._browser_backend.DumpMemory(timeout)
  File "c:\b\s\w\ir\third_party\catapult\common\py_trace_event\py_trace_event\trace_event_impl\decorators.py", line 75, in traced_function
    return func(*args, **kwargs)
  File "c:\b\s\w\ir\third_party\catapult\telemetry\telemetry\internal\backends\chrome\chrome_browser_backend.py", line 279, in DumpMemory
    return self.devtools_client.DumpMemory(timeout)
  File "c:\b\s\w\ir\third_party\catapult\telemetry\telemetry\internal\backends\chrome_inspector\devtools_client_backend.py", line 402, in DumpMemory
    return self._tracing_backend.DumpMemory(timeout)
  File "c:\b\s\w\ir\third_party\catapult\telemetry\telemetry\internal\backends\chrome_inspector\tracing_backend.py", line 213, in DumpMemory
    'request:\n' + traceback.format_exc())
TracingUnrecoverableException: Exception raised while sending a Tracing.requestMemoryDump request:
Traceback (most recent call last):
  File "c:\b\s\w\ir\third_party\catapult\telemetry\telemetry\internal\backends\chrome_inspector\tracing_backend.py", line 204, in DumpMemory
    response = self._inspector_websocket.SyncRequest(request, timeout)
  File "c:\b\s\w\ir\third_party\catapult\telemetry\telemetry\internal\backends\chrome_inspector\inspector_websocket.py", line 110, in SyncRequest
    res = self._Receive(timeout)
  File "c:\b\s\w\ir\third_party\catapult\telemetry\telemetry\internal\backends\chrome_inspector\inspector_websocket.py", line 149, in _Receive
    data = self._socket.recv()
  File "c:\b\s\w\ir\third_party\catapult\telemetry\third_party\websocket-client\websocket\_core.py", line 293, in recv
    opcode, data = self.recv_data()
  File "c:\b\s\w\ir\third_party\catapult\telemetry\third_party\websocket-client\websocket\_core.py", line 310, in recv_data
    opcode, frame = self.recv_data_frame(control_frame)
  File "c:\b\s\w\ir\third_party\catapult\telemetry\third_party\websocket-client\websocket\_core.py", line 323, in recv_data_frame
    frame = self.recv_frame()
  File "c:\b\s\w\ir\third_party\catapult\telemetry\third_party\websocket-client\websocket\_core.py", line 357, in recv_frame
    return self.frame_buffer.recv_frame()
  File "c:\b\s\w\ir\third_party\catapult\telemetry\third_party\websocket-client\websocket\_abnf.py", line 336, in recv_frame
    self.recv_header()
  File "c:\b\s\w\ir\third_party\catapult\telemetry\third_party\websocket-client\websocket\_abnf.py", line 286, in recv_header
    header = self.recv_strict(2)
  File "c:\b\s\w\ir\third_party\catapult\telemetry\third_party\websocket-client\websocket\_abnf.py", line 371, in recv_strict
    bytes_ = self.recv(min(16384, shortage))
  File "c:\b\s\w\ir\third_party\catapult\telemetry\third_party\websocket-client\websocket\_core.py", line 427, in _recv
    return recv(self.sock, bufsize)
  File "c:\b\s\w\ir\third_party\catapult\telemetry\third_party\websocket-client\websocket\_socket.py", line 80, in recv
    bytes_ = sock.recv(bufsize)
error: [Errno 10054] An existing connection was forcibly closed by the remote host
INFO:root:Devtools client not yet ready: [Errno 10061] No connection could be made because the target machine actively refused it
INFO:root:Try printing formatted exception: None None None
Exception raised when cleaning story run: 
Traceback (most recent call last):
  _RunStoryAndProcessErrorIfNeeded at c:\b\s\w\ir\third_party\catapult\telemetry\telemetry\internal\story_runner.py:124
    test.DidRunStory(state.platform, results)
  DidRunStory at c:\b\s\w\ir\third_party\catapult\telemetry\telemetry\web_perf\timeline_based_measurement.py:309
    trace_result = platform.tracing_controller.StopTracing()
  StopTracing at c:\b\s\w\ir\third_party\catapult\telemetry\telemetry\core\tracing_controller.py:47
    return self._tracing_controller_backend.StopTracing()
  StopTracing at c:\b\s\w\ir\third_party\catapult\telemetry\telemetry\internal\platform\tracing_controller_backend.py:109
    self._IssueClockSyncMarker()
  _IssueClockSyncMarker at c:\b\s\w\ir\third_party\catapult\telemetry\telemetry\internal\platform\tracing_controller_backend.py:213
    self._RecordIssuerClockSyncMarker)
  RecordClockSyncMarker at c:\b\s\w\ir\third_party\catapult\telemetry\telemetry\internal\platform\tracing_agent\chrome_tracing_agent.py:177
    raise ChromeClockSyncError('Cannot issue clock sync. No devtools clients')
ChromeClockSyncError: Cannot issue clock sync. No devtools clients
Locals:
  devtools_clients                             : []
  record_controller_clock_sync_marker_callback : <bound method TracingControllerBackend._RecordIssuerClockSyncMarker of <telemetry.internal.platform.tracing_controller_backend.TracingControllerBackend object at 0x036D1590>>
  sync_id                                      : 'bfe916d2-6e65-4c0b-b843-ecdda053e9b4'
[  FAILED  ] multitab:misc:typical24@{'case': 'multitab', 'group': 'misc'} (71695 ms)
INFO:root:Try printing formatted exception: None None None
Traceback (most recent call last):
  RunBenchmark at c:\b\s\w\ir\third_party\catapult\telemetry\telemetry\internal\story_runner.py:398
    expectations=expectations, metadata=benchmark.GetMetadata())
  Run at c:\b\s\w\ir\third_party\catapult\telemetry\telemetry\internal\story_runner.py:262
    _RunStoryAndProcessErrorIfNeeded(story, results, state, test)
  _RunStoryAndProcessErrorIfNeeded at c:\b\s\w\ir\third_party\catapult\telemetry\telemetry\internal\story_runner.py:99
    state.RunStory(results)
  traced_function at c:\b\s\w\ir\third_party\catapult\common\py_trace_event\py_trace_event\trace_event_impl\decorators.py:75
    return func(*args, **kwargs)
  RunStory at c:\b\s\w\ir\third_party\catapult\telemetry\telemetry\page\shared_page_state.py:296
    self._current_page.Run(self)
  Run at c:\b\s\w\ir\third_party\catapult\telemetry\telemetry\page\__init__.py:112
    self.RunPageInteractions(action_runner)
  RunPageInteractions at c:\b\s\w\ir\tools\perf\page_sets\system_health\system_health_story.py:143
    self._Measure(action_runner)
  _Measure at c:\b\s\w\ir\tools\perf\page_sets\system_health\system_health_story.py:126
    action_runner.MeasureMemory(deterministic_mode=True)
  traced_function at c:\b\s\w\ir\third_party\catapult\common\py_trace_event\py_trace_event\trace_event_impl\decorators.py:75
    return func(*args, **kwargs)
  MeasureMemory at c:\b\s\w\ir\third_party\catapult\telemetry\telemetry\internal\actions\action_runner.py:158
    dump_id = self.tab.browser.DumpMemory()
  DumpMemory at c:\b\s\w\ir\third_party\catapult\telemetry\telemetry\internal\browser\browser.py:342
    return self._browser_backend.DumpMemory(timeout)
  traced_function at c:\b\s\w\ir\third_party\catapult\common\py_trace_event\py_trace_event\trace_event_impl\decorators.py:75
    return func(*args, **kwargs)
  DumpMemory at c:\b\s\w\ir\third_party\catapult\telemetry\telemetry\internal\backends\chrome\chrome_browser_backend.py:279
    return self.devtools_client.DumpMemory(timeout)
  DumpMemory at c:\b\s\w\ir\third_party\catapult\telemetry\telemetry\internal\backends\chrome_inspector\devtools_client_backend.py:402
    return self._tracing_backend.DumpMemory(timeout)
  DumpMemory at c:\b\s\w\ir\third_party\catapult\telemetry\telemetry\internal\backends\chrome_inspector\tracing_backend.py:213
    'request:\n' + traceback.format_exc())
TracingUnrecoverableException: Exception raised while sending a Tracing.requestMemoryDump request:
Traceback (most recent call last):
  File "c:\b\s\w\ir\third_party\catapult\telemetry\telemetry\internal\backends\chrome_inspector\tracing_backend.py", line 204, in DumpMemory
    response = self._inspector_websocket.SyncRequest(request, timeout)
  File "c:\b\s\w\ir\third_party\catapult\telemetry\telemetry\internal\backends\chrome_inspector\inspector_websocket.py", line 110, in SyncRequest
    res = self._Receive(timeout)
  File "c:\b\s\w\ir\third_party\catapult\telemetry\telemetry\internal\backends\chrome_inspector\inspector_websocket.py", line 149, in _Receive
    data = self._socket.recv()
  File "c:\b\s\w\ir\third_party\catapult\telemetry\third_party\websocket-client\websocket\_core.py", line 293, in recv
    opcode, data = self.recv_data()
  File "c:\b\s\w\ir\third_party\catapult\telemetry\third_party\websocket-client\websocket\_core.py", line 310, in recv_data
    opcode, frame = self.recv_data_frame(control_frame)
  File "c:\b\s\w\ir\third_party\catapult\telemetry\third_party\websocket-client\websocket\_core.py", line 323, in recv_data_frame
    frame = self.recv_frame()
  File "c:\b\s\w\ir\third_party\catapult\telemetry\third_party\websocket-client\websocket\_core.py", line 357, in recv_frame
    return self.frame_buffer.recv_frame()
  File "c:\b\s\w\ir\third_party\catapult\telemetry\third_party\websocket-client\websocket\_abnf.py", line 336, in recv_frame
    self.recv_header()
  File "c:\b\s\w\ir\third_party\catapult\telemetry\third_party\websocket-client\websocket\_abnf.py", line 286, in recv_header
    header = self.recv_strict(2)
  File "c:\b\s\w\ir\third_party\catapult\telemetry\third_party\websocket-client\websocket\_abnf.py", line 371, in recv_strict
    bytes_ = self.recv(min(16384, shortage))
  File "c:\b\s\w\ir\third_party\catapult\telemetry\third_party\websocket-client\websocket\_core.py", line 427, in _recv
    return recv(self.sock, bufsize)
  File "c:\b\s\w\ir\third_party\catapult\telemetry\third_party\websocket-client\websocket\_socket.py", line 80, in recv
    bytes_ = sock.recv(bufsize)
error: [Errno 10054] An existing connection was forcibly closed by the remote host
Locals:
  request : {'method': 'Tracing.requestMemoryDump', 'id': 0}
  timeout : 90
 
Project Member

Comment 5 by 42576172...@developer.gserviceaccount.com, Jun 26 2017


=== BISECT JOB RESULTS ===
NO Test failure found

Bisect Details
  Configuration: winx64_10_perf_bisect
  Benchmark    : system_health.memory_desktop
  Metric       : benchmark_duration/benchmark_duration

Revision             Exit Code      N
chromium@481176      0 +- N/A       5      good
chromium@481392      0 +- N/A       5      bad

Please refer to the following doc on diagnosing memory regressions:
  https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=load:search:yahoo system_health.memory_desktop

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8975696226830279072

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5788094789320704


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Project Member

Comment 6 by 42576172...@developer.gserviceaccount.com, Jun 26 2017


=== BISECT JOB RESULTS ===
NO Test failure found

Bisect Details
  Configuration: winx64_10_perf_bisect
  Benchmark    : system_health.memory_desktop
  Metric       : benchmark_duration/benchmark_duration

Revision             Exit Code      N
chromium@481176      0 +- N/A       5      good
chromium@481392      0 +- N/A       5      bad

Please refer to the following doc on diagnosing memory regressions:
  https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=multitab:misc:typical24 system_health.memory_desktop

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8975696296136182736

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5232515066888192


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Project Member

Comment 7 by 42576172...@developer.gserviceaccount.com, Jun 27 2017


=== BISECT JOB RESULTS ===
NO Test failure found

Bisect Details
  Configuration: winx64_10_perf_bisect
  Benchmark    : system_health.memory_desktop
  Metric       : benchmark_duration/benchmark_duration

Revision             Exit Code      N
chromium@481176      1 +- N/A       5      good
chromium@481392      1 +- N/A       5      bad

Please refer to the following doc on diagnosing memory regressions:
  https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests system_health.memory_desktop

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8975696198057003984

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5831462013632512


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Components: Tests>Telemetry
Labels: -Pri-1 Pri-2
Owner: rnep...@chromium.org
Status: Assigned (was: Untriaged)
rnephew@chromium.org: Is this still an issue?
Owner: ----
Status: Available (was: Assigned)
Components: Test>Telemetry
Components: -Tests>Telemetry

Sign in to add a comment