New issue
Advanced search Search tips

Issue 780502 link

Starred by 4 users

Issue metadata

Status: Fixed
Owner:
Closed: Nov 2017
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: ----
Type: ----



Sign in to add a comment

blink_perf.paint failing on Android

Project Member Reported by sullivan@chromium.org, Nov 1 2017

Issue description

255b4036-ef4f-4901-b864-116be0d6227d

Builders failed on: 
- Android Nexus5 Perf: 
  https://build.chromium.org/p/chromium.perf/builders/Android%20Nexus5%20Perf
- Android Nexus7v2 Perf: 
  https://build.chromium.org/p/chromium.perf/builders/Android%20Nexus7v2%20Perf


Range is 512790:512835 on Nexus 5 and 512799:512849 on Nexus 7. Trying a bisect.
 

=== BISECT JOB RESULTS ===
NO Test failure found

Bisect Details
  Configuration: android_nexus5_perf_bisect
  Benchmark    : blink_perf.paint
  Metric       : appending-text/appending-text.html

Revision             Exit Code      N
chromium@512789      0 +- N/A       20      good
chromium@512835      0 +- N/A       20      bad

Please refer to the following doc on diagnosing blink_perf regressions:
  https://chromium.googlesource.com/chromium/src/+/master/docs/speed/benchmark_harnesses/blink_perf.md

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=appending.text.html blink_perf.paint

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8964110631183856752


For feedback, file a bug with component Speed>Bisection
Hmmm, trying on the N7.
Cc: kojii@chromium.org
Owner: kojii@chromium.org
Status: Assigned (was: Available)

=== Auto-CCing suspected CL author kojii@chromium.org ===

Hi kojii@chromium.org, the bisect results pointed to your CL, please take a look at the
results.


=== BISECT JOB RESULTS ===
Test failure found with culprit

Suspected Commit
  Author : Koji Ishii
  Commit : 811fe6a25ddee5065812bf50733de963e58832cb
  Date   : Tue Oct 31 14:34:37 2017
  Subject: Add CrashKey to LazyLineBreakIterator on Android

Bisect Details
  Configuration: android_nexus7_perf_bisect
  Benchmark    : blink_perf.paint
  Metric       : complex-content-slow-scroll/complex-content-slow-scroll.html

Revision             Exit Code      N
chromium@512798      0 +- N/A       20      good
chromium@512824      0 +- N/A       20      good
chromium@512825      1 +- N/A       20      bad       <--
chromium@512826      1 +- N/A       20      bad
chromium@512828      1 +- N/A       20      bad
chromium@512831      1 +- N/A       20      bad
chromium@512837      1 +- N/A       20      bad
chromium@512849      1 +- N/A       20      bad

Please refer to the following doc on diagnosing blink_perf regressions:
  https://chromium.googlesource.com/chromium/src/+/master/docs/speed/benchmark_harnesses/blink_perf.md

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=complex.content.slow.scroll.html blink_perf.paint

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8964096330346497424


For feedback, file a bug with component Speed>Bisection

Comment 6 by kojii@chromium.org, Nov 2 2017

By "failing", is this bug talking about some crashes, or slower perf?

The slower is expected, I needed to add additional information to track crashes temporarily, and will remove once we got some crash reports.

If it's crashing, could someone point me where I can find stacks? I tried to understand links at #0 but not succeeding yet.
Cc: nedngu...@google.com jbudorick@chromium.org
The test failed, it looks like it's either timing out or crashing but the logs are pretty hard to read. +Ned for help reading the logs and +John in case there's a way to get a crash log. Logs:
https://logs.chromium.org/v/?s=chrome%2Fbb%2Fchromium.perf%2FAndroid_Nexus7v2_Perf%2F779%2F%2B%2Frecipes%2Fsteps%2Fblink_perf.paint_on_Android%2F0%2Fstdout

(ERROR) 2017-11-02 09:28:16,317 exceptions.__init__:76  Problem when trying to gather stack trace:
Traceback (most recent call last):
  File "/b/swarming/w/ir/third_party/catapult/telemetry/telemetry/core/exceptions.py", line 72, in __init__
    self._is_valid_dump, trace_output = app.GetStackTrace()
  File "/b/swarming/w/ir/third_party/catapult/telemetry/telemetry/internal/browser/browser.py", line 301, in GetStackTrace
    return self._browser_backend.GetStackTrace()
  File "/b/swarming/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 52, in traced_function
    return func(*args, **kwargs)
  File "/b/swarming/w/ir/third_party/catapult/telemetry/telemetry/internal/backends/chrome/android_browser_backend.py", line 276, in GetStackTrace
    return self.platform_backend.GetStackTrace()
  File "/b/swarming/w/ir/third_party/catapult/telemetry/telemetry/internal/platform/android_platform_backend.py", line 717, in GetStackTrace
    ret = Decorate('UI dump', '\n'.join(self.GetSystemUi().ScreenDump()))
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/android/app_ui.py", line 212, in ScreenDump
    return self._GetRootUiNode().Dump()
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/android/decorators.py", line 57, in timeout_retry_wrapper
    retry_if_func=retry_if_func)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/utils/timeout_retry.py", line 157, in Run
    error_log_func=error_log_func)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 186, in JoinAll
    self._JoinAll(watcher, timeout)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 158, in _JoinAll
    thread.ReraiseIfException()
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 81, in run
    self._ret = self._func(*self._args, **self._kwargs)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/utils/timeout_retry.py", line 150, in <lambda>
    child_thread = reraiser_thread.ReraiserThread(lambda: func(*args, **kwargs),
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl
    return f(*args, **kwargs)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/android/app_ui.py", line 201, in _GetRootUiNode
    check_return=True)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/android/decorators.py", line 51, in timeout_retry_wrapper
    return impl()
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl
    return f(*args, **kwargs)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/android/device_utils.py", line 1042, in RunShellCommand
    output = handle_large_output(cmd, large_output)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/android/device_utils.py", line 1011, in handle_large_output
    return handle_large_command(cmd)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/android/device_utils.py", line 993, in handle_large_command
    return handle_check_return(cmd)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/android/device_utils.py", line 984, in handle_check_return
    return run(cmd)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/android/device_utils.py", line 980, in run
    return self.adb.Shell(cmd)
  File "/b/swarming/w/ir/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 521, in Shell
    command, output, status=status, device_serial=self._device_serial)
AdbShellCommandFailedError: (device: 09275b25) shell command run via adb failed on the device:
  command: uiautomator dump /data/local/tmp/temp_file-de7714f8329ac
  exit status: 137
  output:
  - Killed 
(INFO) 2017-11-02 09:28:16,322 browser.DumpStateUponFailure:391  *************** BROWSER STANDARD OUTPUT ***************
(INFO) 2017-11-02 09:28:16,322 browser.DumpStateUponFailure:393  Cannot get standard output on Android
(INFO) 2017-11-02 09:28:16,322 browser.DumpStateUponFailure:396  *********** END OF BROWSER STANDARD OUTPUT ************
(INFO) 2017-11-02 09:28:16,322 browser.DumpStateUponFailure:398  ********************* BROWSER LOG *********************
(INFO) 2017-11-02 09:28:16,323 browser.DumpStateUponFailure:400  No log file
(INFO) 2017-11-02 09:28:16,323 browser.DumpStateUponFailure:403  ***************** END OF BROWSER LOG ******************
(INFO) 2017-11-02 09:28:16,324 cmd_helper._ValidateAndLogCommand:160  [host]> /b/swarming/w/ir/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 09275b25 shell '( /system/bin/screencap -p /data/local/tmp/temp_file-394301eafef1.png );echo %$?'
(INFO) 2017-11-02 09:28:17,640 cmd_helper._ValidateAndLogCommand:160  [host]> /b/swarming/w/ir/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 09275b25 pull /data/local/tmp/temp_file-394301eafef1.png /b/swarming/w/itrESnHA/tmpFcyocV.png
(INFO) 2017-11-02 09:28:17,948 cmd_helper._ValidateAndLogCommand:160  [host]> /b/swarming/w/ir/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb -s 09275b25 shell 'rm -f /data/local/tmp/temp_file-394301eafef1.png'
Traceback (most recent call last):
  File "/b/swarming/w/ir/third_party/catapult/telemetry/telemetry/internal/story_runner.py", line 104, in _RunStoryAndProcessErrorIfNeeded
    state.RunStory(results)
  File "/b/swarming/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 52, in traced_function
    return func(*args, **kwargs)
  File "/b/swarming/w/ir/third_party/catapult/telemetry/telemetry/page/shared_page_state.py", line 327, in RunStory
    self._current_page.Run(self)
  File "/b/swarming/w/ir/third_party/catapult/telemetry/telemetry/page/__init__.py", line 114, in Run
    shared_state.page_test.RunNavigateSteps(self, current_tab)
  File "/b/swarming/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 52, in traced_function
    return func(*args, **kwargs)
  File "/b/swarming/w/ir/third_party/catapult/telemetry/telemetry/page/legacy_page_test.py", line 152, in RunNavigateSteps
    page.RunNavigateSteps(action_runner)
  File "/b/swarming/w/ir/third_party/catapult/telemetry/telemetry/page/__init__.py", line 123, in RunNavigateSteps
    url, script_to_evaluate_on_commit=self.script_to_evaluate_on_commit)
  File "/b/swarming/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 52, in traced_function
    return func(*args, **kwargs)
  File "/b/swarming/w/ir/third_party/catapult/telemetry/telemetry/internal/actions/action_runner.py", line 187, in Navigate
    timeout_in_seconds=timeout_in_seconds))
  File "/b/swarming/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 52, in traced_function
    return func(*args, **kwargs)
  File "/b/swarming/w/ir/third_party/catapult/telemetry/telemetry/internal/actions/action_runner.py", line 63, in _RunAction
    action.RunAction(self._tab)
  File "/b/swarming/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 52, in traced_function
    return func(*args, **kwargs)
  File "/b/swarming/w/ir/third_party/catapult/telemetry/telemetry/internal/actions/navigate.py", line 27, in RunAction
    tab.WaitForDocumentReadyStateToBeInteractiveOrBetter(time_left_in_seconds)
  File "/b/swarming/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 52, in traced_function
    return func(*args, **kwargs)
  File "/b/swarming/w/ir/third_party/catapult/telemetry/telemetry/internal/browser/web_contents.py", line 103, in WaitForDocumentReadyStateToBeInteractiveOrBetter
    'document.readyState == "complete"', timeout=timeout)
  File "/b/swarming/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 52, in traced_function
    return func(*args, **kwargs)
  File "/b/swarming/w/ir/third_party/catapult/telemetry/telemetry/internal/browser/web_contents.py", line 239, in WaitForJavaScriptCondition
    return self._inspector_backend.WaitForJavaScriptCondition(*args, **kwargs)
  File "/b/swarming/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 52, in traced_function
    return func(*args, **kwargs)
  File "/b/swarming/w/ir/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_backend.py", line 289, in WaitForJavaScriptCondition
    return py_utils.WaitFor(IsJavaScriptExpressionTrue, timeout)
  File "/b/swarming/w/ir/third_party/catapult/common/py_utils/py_utils/__init__.py", line 129, in WaitFor
    res = condition()
  File "/b/swarming/w/ir/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_backend.py", line 286, in IsJavaScriptExpressionTrue
    return self._EvaluateJavaScript(condition, context_id, timeout)
  File "/b/swarming/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 52, in traced_function
    return func(*args, **kwargs)
  File "/b/swarming/w/ir/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_backend.py", line 41, in Inner
    inspector_backend._ConvertExceptionFromInspectorWebsocket(e)
  File "/b/swarming/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 52, in traced_function
    return func(*args, **kwargs)
  File "/b/swarming/w/ir/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_backend.py", line 38, in Inner
    return func(inspector_backend, *args, **kwargs)
  File "/b/swarming/w/ir/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_backend.py", line 518, in _EvaluateJavaScript
    return self._runtime.Evaluate(expression, context_id, timeout)
  File "/b/swarming/w/ir/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_runtime.py", line 44, in Evaluate
    res = self._inspector_websocket.SyncRequest(request, timeout)
  File "/b/swarming/w/ir/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_websocket.py", line 116, in SyncRequest
    res = self._Receive(timeout)
  File "/b/swarming/w/ir/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_websocket.py", line 155, in _Receive
    data = self._socket.recv()
  File "/b/swarming/w/ir/third_party/catapult/telemetry/third_party/websocket-client/websocket/_core.py", line 293, in recv
    opcode, data = self.recv_data()
  File "/b/swarming/w/ir/third_party/catapult/telemetry/third_party/websocket-client/websocket/_core.py", line 310, in recv_data
    opcode, frame = self.recv_data_frame(control_frame)
  File "/b/swarming/w/ir/third_party/catapult/telemetry/third_party/websocket-client/websocket/_core.py", line 323, in recv_data_frame
    frame = self.recv_frame()
  File "/b/swarming/w/ir/third_party/catapult/telemetry/third_party/websocket-client/websocket/_core.py", line 357, in recv_frame
    return self.frame_buffer.recv_frame()
  File "/b/swarming/w/ir/third_party/catapult/telemetry/third_party/websocket-client/websocket/_abnf.py", line 336, in recv_frame
    self.recv_header()
  File "/b/swarming/w/ir/third_party/catapult/telemetry/third_party/websocket-client/websocket/_abnf.py", line 286, in recv_header
    header = self.recv_strict(2)
  File "/b/swarming/w/ir/third_party/catapult/telemetry/third_party/websocket-client/websocket/_abnf.py", line 371, in recv_strict
    bytes_ = self.recv(min(16384, shortage))
  File "/b/swarming/w/ir/third_party/catapult/telemetry/third_party/websocket-client/websocket/_core.py", line 427, in _recv
    return recv(self.sock, bufsize)
  File "/b/swarming/w/ir/third_party/catapult/telemetry/third_party/websocket-client/websocket/_socket.py", line 83, in recv
    raise WebSocketTimeoutException(message)
TimeoutException: 
********************************************************************************
(/b/swarming/w/ir/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_backend.py:487 _ConvertExceptionFromInspectorWebsocket) The app is probably crashed:
Found Minidump: False
Stack Trace:
********************************************************************************
********************************************************************************
Standard output:
********************************************************************************
	Cannot get standard output on Android
********************************************************************************
System log:
(Not implemented)
********************************************************************************
Cc: wangxianzhu@chromium.org
Also cc-ing blink_perf.paint owner wangxianzhu as FYI
This is a test timeout:
containment-resize.html : https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/profiler-file-id_12-2017-11-02_09-30-0588444.png

complex-content-slow-scroll.html: https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/profiler-file-id_1-2017-11-02_09-30-0449713.png


Given kojii@'s CL regress the tests to the point that make them timed out, I suggest reverting the CL
Cc: mlippautz@google.com
 Issue 781166  has been merged into this issue.
Cc: -mlippautz@google.com mlippautz@chromium.org
Cc: mlippautz@google.com
 Issue 781167  has been merged into this issue.
Cc: -mlippautz@google.com
Cc: mustaq@chromium.org
 Issue 781245  has been merged into this issue.
 Issue 781245  has been merged into this issue.
 Issue 781252  has been merged into this issue.
 Issue 781253  has been merged into this issue.
 Issue 781252  has been merged into this issue.
 Issue 781252  has been merged into this issue.
Project Member

Comment 21 by bugdroid1@chromium.org, Nov 3 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/0ea04e064baa8615e5ef9702050ce6a605435c57

commit 0ea04e064baa8615e5ef9702050ce6a605435c57
Author: Annie Sullivan <sullivan@chromium.org>
Date: Fri Nov 03 18:38:38 2017

Revert "Add CrashKey to LazyLineBreakIterator on Android"

This reverts commit 811fe6a25ddee5065812bf50733de963e58832cb.

Reason for revert: This CL slowed down the blink_perf.paint test to the point where it times out, and also caused several other performance regressions.

Bug:  780502 ,  781166 ,  781167 ,  781245 ,  781245 

Original change's description:
> Add CrashKey to LazyLineBreakIterator on Android
> 
> This patch adds CrashKey to investigate where crashes occur.
> 
> LazyLineBreakIterator::IsBreakable() has inline and templates
> that the stack quality is low.
> 
> Bug: 756624
> Change-Id: I657b18aa1cb74b2587104352b4a3e826522b6ba8
> Reviewed-on: https://chromium-review.googlesource.com/737710
> Reviewed-by: Matt Falkenhagen <falken@chromium.org>
> Reviewed-by: Robert Sesek <rsesek@chromium.org>
> Reviewed-by: Luke Halliwell <halliwell@chromium.org>
> Reviewed-by: Dominik Röttsches <drott@chromium.org>
> Commit-Queue: Koji Ishii <kojii@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#512825}

TBR=falken@chromium.org,eae@chromium.org,kojii@chromium.org,drott@chromium.org,halliwell@chromium.org,rsesek@chromium.org

# Not skipping CQ checks because original CL landed > 1 day ago.

Bug: 756624
Change-Id: Ib5dbe2ea94fe719d96c2bbfd4a908f2d226aa8ae
Reviewed-on: https://chromium-review.googlesource.com/753529
Reviewed-by: Annie Sullivan <sullivan@chromium.org>
Commit-Queue: Annie Sullivan <sullivan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#513852}
[modify] https://crrev.com/0ea04e064baa8615e5ef9702050ce6a605435c57/android_webview/common/crash_reporter/crash_keys.cc
[modify] https://crrev.com/0ea04e064baa8615e5ef9702050ce6a605435c57/chrome/common/crash_keys.cc
[modify] https://crrev.com/0ea04e064baa8615e5ef9702050ce6a605435c57/chromecast/crash/cast_crash_keys.cc
[modify] https://crrev.com/0ea04e064baa8615e5ef9702050ce6a605435c57/third_party/WebKit/Source/platform/text/TextBreakIterator.cpp

 Issue 781258  has been merged into this issue.
 Issue 781262  has been merged into this issue.
 Issue 781254  has been merged into this issue.
 Issue 781201  has been merged into this issue.
 Issue 781256  has been merged into this issue.
 Issue 781254  has been merged into this issue.
 Issue 781256  has been merged into this issue.
Status: Fixed (was: Assigned)
Thank you sullivan@@ for the revert.
Cc: rmcilroy@chromium.org
 Issue 782744  has been merged into this issue.

Sign in to add a comment