New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 654209 link

Starred by 2 users

Issue metadata

Status: Fixed
Owner:
Closed: Mar 2017
Cc:
EstimatedDays: ----
NextAction: ----
OS: Windows
Pri: 2
Type: Bug-Regression



Sign in to add a comment

Fix and reenable v8.browsing_desktop & v8.browsing_desktop_ignition on Win Perf bots

Project Member Reported by zh...@chromium.org, Oct 8 2016

Issue description

The tests have been failing on Win 10 High-DPI Perf (4) for a few weeks:
https://uberchromegw.corp.google.com/i/chromium.perf/builders/Win%2010%20High-DPI%20Perf%20%284%29?numbuilds=200

They are also very flaky on Win Zenbook Perf (4):
https://uberchromegw.corp.google.com/i/chromium.perf/builders/Win%20Zenbook%20Perf%20%284%29?numbuilds=200




C:\b\c\b\Win_10_High_DPI_Perf__4_\src\third_party\catapult\telemetry\telemetry\internal\image_processing\_bitmap.py:152: UserWarning: Using pure python png decoder, which could be very slow. To speed up, consider installing numpy & cv2 (OpenCV).
  'Using pure python png decoder, which could be very slow. To speed up, '
Traceback (most recent call last):
  File "C:\b\c\b\Win_10_High_DPI_Perf__4_\src\third_party\catapult\telemetry\telemetry\internal\story_runner.py", line 86, in _RunStoryAndProcessErrorIfNeeded
    state.RunStory(results)
  File "C:\b\c\b\Win_10_High_DPI_Perf__4_\src\third_party\catapult\common\py_trace_event\py_trace_event\trace_event_impl\decorators.py", line 75, in traced_function
    return func(*args, **kwargs)
  File "C:\b\c\b\Win_10_High_DPI_Perf__4_\src\third_party\catapult\telemetry\telemetry\page\shared_page_state.py", line 310, in RunStory
    self._current_page.Run(self)
  File "C:\b\c\b\Win_10_High_DPI_Perf__4_\src\third_party\catapult\telemetry\telemetry\page\__init__.py", line 109, in Run
    self.RunPageInteractions(action_runner)
  File "C:\b\c\b\Win_10_High_DPI_Perf__4_\src\tools\perf\page_sets\system_health\system_health_story.py", line 112, in RunPageInteractions
    self._DidLoadDocument(action_runner)
  File "C:\b\c\b\Win_10_High_DPI_Perf__4_\src\tools\perf\page_sets\system_health\browsing_stories.py", line 70, in _DidLoadDocument
    self._ReadNewsItem(action_runner)
  File "C:\b\c\b\Win_10_High_DPI_Perf__4_\src\tools\perf\page_sets\system_health\browsing_stories.py", line 75, in _ReadNewsItem
    action_runner.tab.WaitForDocumentReadyStateToBeComplete()
  File "C:\b\c\b\Win_10_High_DPI_Perf__4_\src\third_party\catapult\telemetry\telemetry\internal\browser\web_contents.py", line 70, in WaitForDocumentReadyStateToBeComplete
    'document.readyState == "complete"', timeout)
  File "C:\b\c\b\Win_10_High_DPI_Perf__4_\src\third_party\catapult\telemetry\telemetry\internal\browser\web_contents.py", line 136, in WaitForJavaScriptExpression
    e.message + '\n' + debug_message)
TimeoutException: Timed out while waiting 90s for IsJavaScriptExpressionTrue.
Console output:


INFO:root:Chrome version: 2885
ERROR:root:Error when deleting c:\users\chrome~1.lab\appdata\local\temp\tmpckqbtd, will try again at exit.
Traceback (most recent call last):
  File "C:\b\c\b\Win_10_High_DPI_Perf__4_\src\third_party\catapult\telemetry\telemetry\internal\platform\tracing_controller_backend.py", line 275, in CollectAgentTraceData
    os.remove(self._trace_log)
WindowsError: [Error 32] The process cannot access the file because it is being used by another process: 'c:\\users\\chrome~1.lab\\appdata\\local\\temp\\tmpckqbtd'
INFO:root:Try printing formatted exception: None None None

Exception raised when cleaning story run: 

Traceback (most recent call last):
  _RunStoryAndProcessErrorIfNeeded at C:\b\c\b\Win_10_High_DPI_Perf__4_\src\third_party\catapult\telemetry\telemetry\internal\story_runner.py:108
    test.DidRunStory(state.platform)
  DidRunStory at C:\b\c\b\Win_10_High_DPI_Perf__4_\src\third_party\catapult\telemetry\telemetry\web_perf\timeline_based_measurement.py:309
    platform.tracing_controller.StopTracing()
  StopTracing at C:\b\c\b\Win_10_High_DPI_Perf__4_\src\third_party\catapult\telemetry\telemetry\core\tracing_controller.py:47
    return self._tracing_controller_backend.StopTracing()
  StopTracing at C:\b\c\b\Win_10_High_DPI_Perf__4_\src\third_party\catapult\telemetry\telemetry\internal\platform\tracing_controller_backend.py:133
    '\n'.join(raised_exception_messages))
TracingControllerStoppedError: Exceptions raised when trying to stop tracing:
Traceback (most recent call last):
  File "C:\b\c\b\Win_10_High_DPI_Perf__4_\src\third_party\catapult\telemetry\telemetry\internal\platform\tracing_controller_backend.py", line 121, in StopTracing
    agent.CollectAgentTraceData(builder)
  File "C:\b\c\b\Win_10_High_DPI_Perf__4_\src\third_party\catapult\telemetry\telemetry\internal\platform\tracing_agent\chrome_tracing_agent.py", line 247, in CollectAgentTraceData
    '\n'.join(raised_exception_messages))
ChromeTracingStoppedError: Exceptions raised when trying to collect Chrome devtool tracing:
Error when collecting Chrome tracing on devtools at port 58127:
Traceback (most recent call last):
  File "C:\b\c\b\Win_10_High_DPI_Perf__4_\src\third_party\catapult\telemetry\telemetry\internal\platform\tracing_agent\chrome_tracing_agent.py", line 236, in CollectAgentTraceData
    client.CollectChromeTracingData(trace_data_builder)
  File "C:\b\c\b\Win_10_High_DPI_Perf__4_\src\third_party\catapult\telemetry\telemetry\internal\backends\chrome_inspector\devtools_client_backend.py", line 369, in CollectChromeTracingData
    self._tracing_backend.CollectTraceData(trace_data_builder, timeout)
  File "C:\b\c\b\Win_10_High_DPI_Perf__4_\src\third_party\catapult\telemetry\telemetry\internal\backends\chrome_inspector\tracing_backend.py", line 227, in CollectTraceData
    self._CollectTracingData(trace_data_builder, timeout)
  File "C:\b\c\b\Win_10_High_DPI_Perf__4_\src\third_party\catapult\telemetry\telemetry\internal\backends\chrome_inspector\tracing_backend.py", line 248, in _CollectTracingData
    self._inspector_websocket.DispatchNotifications(timeout)
  File "C:\b\c\b\Win_10_High_DPI_Perf__4_\src\third_party\catapult\telemetry\telemetry\internal\backends\chrome_inspector\inspector_websocket.py", line 134, in DispatchNotifications
    self._Receive(timeout)
  File "C:\b\c\b\Win_10_High_DPI_Perf__4_\src\third_party\catapult\telemetry\telemetry\internal\backends\chrome_inspector\inspector_websocket.py", line 168, in _Receive
    self._HandleAsyncResponse(result)
  File "C:\b\c\b\Win_10_High_DPI_Perf__4_\src\third_party\catapult\telemetry\telemetry\internal\backends\chrome_inspector\inspector_websocket.py", line 184, in _HandleAsyncResponse
    callback(result)
  File "C:\b\c\b\Win_10_High_DPI_Perf__4_\src\third_party\catapult\telemetry\telemetry\internal\backends\chrome_inspector\tracing_backend.py", line 80, in _GotChunkFromStream
    trace_string = ''.join(self._data)
MemoryError

 

Comment 1 by zh...@chromium.org, Oct 8 2016

Summary: v8.browsing_desktop & v8.browsing_desktop_ignition failure on Win Perf bots (was: v8.browsing_desktop & v8.browsing_desktop_ignition failure on chromium.perf at <revisionrange>)

Comment 2 by zh...@chromium.org, Oct 8 2016

Disable CL is up https://codereview.chromium.org/2401253002/
Project Member

Comment 3 by bugdroid1@chromium.org, Oct 8 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/1a039dd675016be7597d84dcef408e1b786a56d4

commit 1a039dd675016be7597d84dcef408e1b786a56d4
Author: zhenw <zhenw@chromium.org>
Date: Sat Oct 08 19:13:54 2016

Disable v8.browsing_desktop & v8.browsing_desktop_ignition on Win Perf bots

TBR=nednguyen@google.com
BUG= chromium:654209 

Review-Url: https://codereview.chromium.org/2401253002
Cr-Commit-Position: refs/heads/master@{#424086}

[modify] https://crrev.com/1a039dd675016be7597d84dcef408e1b786a56d4/tools/perf/benchmarks/v8_browsing.py

Comment 4 by zh...@chromium.org, Oct 9 2016

Labels: -Pri-1 Pri-2
Summary: Fix and reenable v8.browsing_desktop & v8.browsing_desktop_ignition on Win Perf bots (was: v8.browsing_desktop & v8.browsing_desktop_ignition failure on Win Perf bots)

Comment 5 by u...@chromium.org, Oct 25 2016

Cc: zh...@chromium.org
Owner: u...@chromium.org
Status: Assigned (was: Untriaged)
zhenw@, in future please let the benchmark owner know that the benchmark is getting disabled.

Comment 6 by u...@chromium.org, Oct 25 2016

Cc: nednguyen@chromium.org
I don't have Win 10 High-DPI Perf and Win Zenbook, so I cannot reproduce.

On my Windows machine the benchmark runs without failures.

Ned, is it possible to disable the benchmark only for Win 10 High-DPI Perf and Win Zenbook?
It would be ok to lose coverage on these bots, but we would like to get coverage on other windows bots.


Yes, we can disable them only on those low end win by overriding the ShouldDisable method (https://cs.chromium.org/search/?q=ShouldDisable+f:tools/perf&sq=package:chromium&type=cs).

In this case, it looks like the failure is the trace string too big, hence we got OOM'ed error. So probably the condition for disabling would be s.t about available memory of the platform.

Comment 8 by u...@chromium.org, Oct 25 2016

The error is timeout while waiting for JS condition. I guess the page is rendered differently on these machines.

Win 10 High-DPI is probably high-memory machine, so I think the physical memory check unfortunately won't help.

I see that GetDeviceTypeName() simply returns 'Desktop':
https://cs.chromium.org/chromium/src/third_party/catapult/telemetry/telemetry/internal/platform/desktop_platform_backend.py?rcl=0&l=26


Comment 9 by u...@chromium.org, Oct 25 2016

Do you know who owns the bots? Maybe there is a way to debug remotely.
ulan: once you confirm the device is windows platform, you are free to use whichever python method to query the state of machine.

Cc: eyaich@chromium.org
Infra labs owns the bots. I think the easiest thing to do would be to get remote access to one of the trybots. You can click the "Remote Access to Bots" link at http://go/chrome-infra for info on how to do that, although the documentation is in a deprecated/not-ready-yet state.
Cc: martiniss@chromium.org
https://chrome-internal.googlesource.com/infra/infra_internal/+/master/doc/ssh.md is the more up to date documentation on how to access bots.

You should be able to ssh onto it; with a windows bot, the command line might be limited. You should be able to remote desktop to it and use the windows command line if ssh-ing into the bot doesn't get you what you need.

Comment 15 by u...@chromium.org, Oct 26 2016

Thanks all! I'll try remote desktop.
Project Member

Comment 16 by bugdroid1@chromium.org, Dec 12 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/a79e188e6bde19959c4517ee0e9408f95594b405

commit a79e188e6bde19959c4517ee0e9408f95594b405
Author: ulan <ulan@chromium.org>
Date: Mon Dec 12 15:15:33 2016

Revert of Disable v8.browsing_desktop & v8.browsing_desktop_ignition on Win Perf bots (patchset #1 id:1 of https://codereview.chromium.org/2401253002/ )

Reason for revert:
Speculatively re-enabling to see if it still fails on the bots.

Original issue's description:
> Disable v8.browsing_desktop & v8.browsing_desktop_ignition on Win Perf bots
>
> TBR=nednguyen@google.com
> BUG= chromium:654209 
>
> Committed: https://crrev.com/1a039dd675016be7597d84dcef408e1b786a56d4
> Cr-Commit-Position: refs/heads/master@{#424086}

TBR=zhenw@chromium.org
# Not skipping CQ checks because original CL landed more than 1 days ago.
BUG= chromium:654209 

Review-Url: https://codereview.chromium.org/2562373002
Cr-Commit-Position: refs/heads/master@{#437875}

[modify] https://crrev.com/a79e188e6bde19959c4517ee0e9408f95594b405/tools/perf/benchmarks/v8_browsing.py

Status: Fixed (was: Assigned)

Sign in to add a comment