New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 796386 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Dec 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 1
Type: Bug-Regression

Blocked on:
issue 842782

Blocking:
issue 796739
issue 744658
issue 796437



Sign in to add a comment

desktopui_MashLogin chrome crash in SystemInfoHandlerGpuObserver()

Project Member Reported by jamescook@chromium.org, Dec 19 2017

Issue description

First failing build:
https://ci.chromium.org/buildbot/chromeos.chrome/peach_pit-tot-chrome-pfq-informational/9035

This test attempts to launch chrome with --mash (out-of-process ash) and log in. See go/mustash for project details. It looks like chrome is crashing either on startup or right after login.

Unhandled WebSocketConnectionClosedException: Connection is already closed..
build artifacts:
https://storage.cloud.google.com/?arg=chromeos-image-archive/peach_pit-tot-chrome-pfq-informational/R65-10230.0.0-b9035.
results log: http://ubercautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/163820771-chromeos-test/chromeos6-row2-rack10-host20/debug/.
status log: http://ubercautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/163820771-chromeos-test/chromeos6-row2-rack10-host20/status.log.
job link: http://cautotest-prod.corp.google.com/afe/#tab_id=view_job&object_id=163820771.

Stack:
Crash reason:  SIGTRAP
Crash address: 0x0
Process uptime: not available

Thread 0 (crashed)
 0  chrome!content::protocol::SystemInfoHandlerGpuObserver::SystemInfoHandlerGpuObserver(std::__1::unique_ptr<content::protocol::SystemInfo::Backend::GetInfoCallback, std::__1::default_delete<content::protocol::SystemInfo::Backend::GetInfoCallback> >) + 0x117
     r0 = 0xb85f2ce0    r1 = 0xb1c9b775    r2 = 0xb85f2ce0    r3 = 0x00000010
     r4 = 0xbebfba60    r5 = 0xbebfb940    r6 = 0x00000000    r7 = 0xbebfb9a8
     r8 = 0xb61a4bb5    r9 = 0xb1c9b6a9   r10 = 0xb715ff88   r12 = 0xb2d4bc39
     fp = 0xb715fdc8    sp = 0xbebfb908    lr = 0xb2d39c95    pc = 0xb1c9b774
    Found by: given as instruction pointer in context
 1  chrome!base::debug::TaskAnnotator::RunTask(char const*, base::PendingTask*) + 0x72
     r4 = 0xbebfba60    r5 = 0xbebfb940    r6 = 0x00000000    r7 = 0xbebfb9a8
     r8 = 0xb61a4bb5    r9 = 0xb1c9b6a9   r10 = 0xb715ff88    fp = 0xb715fdc8
     sp = 0xbebfb908    pc = 0xb2d39c95
    Found by: call frame info
 2  chrome!base::MessageLoop::RunTask(base::PendingTask*) + 0x14a
     r4 = 0xbebfba18    r5 = 0xbebfba60    r6 = 0xbebfba68    r7 = 0xbebfba58
     r8 = 0x00000000    r9 = 0xb1c9b6a9   r10 = 0xb715ff88    fp = 0xb715ff20
     sp = 0xbebfb9b0    pc = 0xb2d4c711
    Found by: call frame info
 3  chrome!base::MessageLoop::DoDelayedWork(base::TimeTicks*) + 0xe8
     r4 = 0xb715ff20    r5 = 0xb713f608    r6 = 0xb715ff60    r7 = 0xbebfbb40
     r8 = 0x00000000    r9 = 0x4361cfca   r10 = 0x00000000    fp = 0x00000000
     sp = 0xbebfba60    pc = 0xb2d4cb87
    Found by: call frame info
 4  chrome!base::MessagePumpLibevent::Run(base::MessagePump::Delegate*) + 0x64
     r4 = 0xb713f600    r5 = 0xb715ff20    r6 = 0xb713f608    r7 = 0xbebfbb90
     r8 = 0xb77b7910    r9 = 0x00000000   r10 = 0x00000000    fp = 0x00000000
     sp = 0xbebfbb48    pc = 0xb2d4d6ef
    Found by: call frame info
 5  chrome!base::RunLoop::Run() + 0x2c
     r4 = 0xbebfbbe0    r5 = 0xb711ef0c    r6 = 0xb6d9968c    r7 = 0xbebfbba0
     r8 = 0xbebfbcd8    r9 = 0xbebfbd10   r10 = 0x00000000    fp = 0xbebfbe98
     sp = 0xbebfbb98    pc = 0xb2d64c1b
    Found by: call frame info
 6  chrome!ChromeBrowserMainParts::MainMessageLoopRun(int*) + 0x96
     r4 = 0xbebfbbe0    r5 = 0xb711ef0c    r6 = 0xb6d9968c    r7 = 0xbebfbc20
     r8 = 0xbebfbcd8    r9 = 0xbebfbd10   r10 = 0x00000000    fp = 0xbebfbe98
     sp = 0xbebfbba8    pc = 0xb2b18f15
    Found by: call frame info
 7  chrome!content::BrowserMainLoop::RunMainMessageLoopParts() + 0x34
     r4 = 0xb711ef00    r5 = 0xb6d9968c    r6 = 0xb712d604    r7 = 0xbebfbc88
     r8 = 0xbebfbcd8    r9 = 0xbebfbd10   r10 = 0x00000000    fp = 0xbebfbe98
     sp = 0xbebfbc28    pc = 0xb1c3acc7
    Found by: call frame info
 8  chrome!content::BrowserMainRunnerImpl::Run() + 0xa
     r4 = 0xb711ba90    r5 = 0xffffffff    r6 = 0xb712d604    r7 = 0xbebfbc98
     r8 = 0xbebfbcd8    r9 = 0xbebfbd10   r10 = 0x00000000    fp = 0xbebfbe98
     sp = 0xbebfbc90    pc = 0xb1c3cdc9
    Found by: call frame info
 9  chrome!content::BrowserMain(content::MainFunctionParams const&) + 0x10e
     r4 = 0xb711ba90    r5 = 0xffffffff    r6 = 0xb712d604    r7 = 0xbebfbd08
     r8 = 0xbebfbcd8    r9 = 0xbebfbd10   r10 = 0x00000000    fp = 0xbebfbe98
     sp = 0xbebfbca0    pc = 0xb1c37e31
    Found by: call frame info
10  chrome!content::ContentMainRunnerImpl::Run() + 0xa6
     r4 = 0xb711cdb0    r5 = 0xb710fda0    r6 = 0x00000000    r7 = 0xbebfbd50
     r8 = 0xbebfbd28    r9 = 0x00000000   r10 = 0x00000000    fp = 0xbebfbe98
     sp = 0xbebfbd10    pc = 0xb2b0ac6d
    Found by: call frame info
11  chrome!service_manager::Main(service_manager::MainParams const&) + 0x680
     r4 = 0xbebfbd6c    r5 = 0x00000000    r6 = 0xffffffff    r7 = 0xbebfbf68
     r8 = 0xbebfbf7c    r9 = 0x00000003   r10 = 0xbebfc194    fp = 0xbebfbe98
     sp = 0xbebfbd58    pc = 0xb2b10c53
    Found by: call frame info
12  chrome!content::ContentMain(content::ContentMainParams const&) + 0x36
     r4 = 0xbebfc00c    r5 = 0xbebfbf7c    r6 = 0xbebfbf70    r7 = 0xbebfbfb0
     r8 = 0xb710fda0    r9 = 0x00000000   r10 = 0xb6d92de8    fp = 0x00000000
     sp = 0xbebfbf70    pc = 0xb2b09d45
    Found by: call frame info

My best guess is zmo's CL https://chromium-review.googlesource.com/c/chromium/src/+/832933

Can you either revert, or figure out a way to support chrome --mash?

 

Comment 1 by kbr@chromium.org, Dec 19 2017

James, can this failure be reproduced on desktop Linux by building or running Chrome in a particular mode?

Cc: achuith@chromium.org
achuith, I'm OOO after today, if you're around could you help zmo@ figure out how to repro this autotest failure?

The code path apparently only runs when telemetry is active.

chrome --mash on linux-chromeos works fine after zmo's CL. I don't know how to repro on linux-chromeos. :-(

Comment 3 by kbr@chromium.org, Dec 20 2017

Blocking: 744658
Labels: -Type-Bug Type-Bug-Regression

Comment 4 by zmo@chromium.org, Dec 20 2017

Cc: sadrul@chromium.org
This is a CHECK I just added. It's not a regression - it's timing out before my CL, now simply I made it to crash.

Let me revert this behavior to not crash on chromeos. Then we can investigate how to fix the timeout on --mash.

Comment 5 by kbr@chromium.org, Dec 20 2017

zmo's CL in flight restoring the old behavior:
https://chromium-review.googlesource.com/835420

Comment 6 by kbr@chromium.org, Dec 20 2017

Blocking: 796437
Project Member

Comment 7 by bugdroid1@chromium.org, Dec 20 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/2312d0c8c2ace63272a0c287562d09ecf9a96922

commit 2312d0c8c2ace63272a0c287562d09ecf9a96922
Author: Zhenyao Mo <zmo@chromium.org>
Date: Wed Dec 20 02:14:36 2017

Make desktopui_MashLogin not crash.

This is a short term fix. Long term, we need to fix the underlying bug that
in mus-viz mode, browser process doesn't talk to GPU process.

BUG= 796386 
TEST=affected tests on chromeos bots
R=piman@chromium.org
TBR=pfeldman@chromium.org

Change-Id: I8fe8e7cbe62c39f035b5ba6cf01b8f82b7d4e241
Reviewed-on: https://chromium-review.googlesource.com/835420
Commit-Queue: Kenneth Russell <kbr@chromium.org>
Reviewed-by: Antoine Labour <piman@chromium.org>
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Cr-Commit-Position: refs/heads/master@{#525229}
[modify] https://crrev.com/2312d0c8c2ace63272a0c287562d09ecf9a96922/content/browser/devtools/protocol/system_info_handler.cc

Comment 8 by kbr@chromium.org, Dec 20 2017

Status: Fixed (was: Assigned)
Mo's patch seems to have mostly solved this crash:
https://ci.chromium.org/buildbot/chromeos.chrome/peach_pit-tot-chrome-pfq-informational/

There was still one flaky failure:
https://ci.chromium.org/buildbot/chromeos.chrome/peach_pit-tot-chrome-pfq-informational/9041

Logs here:
https://00e9e64bac52e90a72b4764590c2964265caf5d070ec131c5f-apidata.googleusercontent.com/download/storage/v1/b/chromeos-autotest-results/o/163964363-chromeos-test%2Fchromeos6-row2-rack11-host1%2Fdesktopui_MashLogin%2Fstatus?qk=AD5uMEsu1PnY18ofDpCpnY8ojYLmlae2Yh-UpjIAlH6MQF8LsfP4O-BGFC-mVY8P_3KvM9pFcQc0-1uFHD3EMadVMtUpfHUrUGk35KCNIGJKa_aZnjeqQRt-0dilriMEczEeW63l79cXFDCrTdOHOCVQdCjnDvSwl5DqeQzY3D_dEF31Yhdi3YpQWk8aCoAXnE6vcZBA9FQP2uYrFEeEQkzInyjmb0WUwnnePxfQpQLNkkfhxm5bZGRNzBlJ_cvyR5mLJ9lCp9kRhUxG7UT9lYUpdp4tYRm0gpeXsU3n9ubKC8D8qGDyNU6mANGie81LlTL_hDzbU-t1H0SoMquCYdPw1OEjmzznSvFeuPi0bHHOO4xRDL-J0aqkIGr8BIXpoLA_AXm5fJMOHr0a7ZKOFGlRf-0fJDLrUT8L7CzxkvZ-D35Mt0wPoyM_30i4XhcuOF1-3nB7KS52BiZrhtWEPRraRqtZLikShQ4oUvO19_Ont6JYWXlABUiMg-kYVS1I0myBSC80lBcWPXvMHHPR1mpuih3JFLpOIUdWIQrYh7y944fb3DXqmf1rI6IUJMQTNU4ooHMaVNd28JFjo550xrnigiC1M3GgrGdSvjAOS4fLgm8FxSgYZyHci3oYtgUi9o4cnWOMuQBdWHFwa2cxLttWxeLBT6-9aC-9R8du7EpQMgYUxq4LbR0ZaZuit_pAXawj3ggvedL7SKWRwAwaw9g8MLUQTgkfeg5ZXknkmv3dYA6em6Xw9Ok_sxU7TyJ7jsk8SgCbt3IMKaVNMAkNhXcylJbuG43uWMBwMoaZlhbm-9OBRqbv1yLXg_KB9BH6WFJCTlXBwvikq1hOGeLpzyhovoDiodC7wA

Stack trace:

  Traceback (most recent call last):
    File "/usr/local/autotest/common_lib/test.py", line 831, in _call_test_function
      return func(*args, **dargs)
    File "/usr/local/autotest/common_lib/test.py", line 495, in execute
      dargs)
    File "/usr/local/autotest/common_lib/test.py", line 362, in _call_run_once_with_retry
      postprocess_profiled_run, args, dargs)
    File "/usr/local/autotest/common_lib/test.py", line 400, in _call_run_once
      self.run_once(*args, **dargs)
    File "/usr/local/autotest/tests/desktopui_MashLogin/desktopui_MashLogin.py", line 38, in run_once
      with chrome.Chrome(extra_browser_args=mash_browser_args):
    File "/usr/local/autotest/common_lib/cros/chrome.py", line 171, in __init__
      self._browser = browser_to_create.Create(finder_options)
    File "/usr/local/telemetry/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome/cros_browser_finder.py", line 51, in Create
      browser_backend, self._platform_backend)
    File "/usr/local/telemetry/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome/cros_browser_with_oobe.py", line 14, in __init__
      backend, platform_backend)
    File "/usr/local/telemetry/src/third_party/catapult/telemetry/telemetry/internal/browser/browser.py", line 42, in __init__
      self._LogBrowserInfo()
    File "/usr/local/telemetry/src/third_party/catapult/telemetry/telemetry/internal/browser/browser.py", line 105, in _LogBrowserInfo
      if self.supports_system_info:
    File "/usr/local/telemetry/src/third_party/catapult/telemetry/telemetry/internal/browser/browser.py", line 273, in supports_system_info
      return self._browser_backend.supports_system_info
    File "/usr/local/telemetry/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome/chrome_browser_backend.py", line 315, in supports_system_info
      return self.GetSystemInfo() != None
    File "/usr/local/telemetry/src/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 52, in traced_function
      return func(*args, **kwargs)
    File "/usr/local/telemetry/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome/chrome_browser_backend.py", line 321, in GetSystemInfo
      return self.devtools_client.GetSystemInfo(timeout=30)
    File "/usr/local/telemetry/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/devtools_client_backend.py", line 455, in GetSystemInfo
      return self._system_info_backend.GetSystemInfo(timeout)
    File "/usr/local/telemetry/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/system_info_backend.py", line 19, in GetSystemInfo
      res = websocket.SyncRequest(req, timeout)
    File "/usr/local/telemetry/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_websocket.py", line 116, in SyncRequest
      res = self._Receive(timeout)
    File "/usr/local/telemetry/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_websocket.py", line 155, in _Receive
      data = self._socket.recv()
    File "/usr/local/telemetry/src/third_party/catapult/telemetry/third_party/websocket-client/websocket/_core.py", line 293, in recv
      opcode, data = self.recv_data()
    File "/usr/local/telemetry/src/third_party/catapult/telemetry/third_party/websocket-client/websocket/_core.py", line 310, in recv_data
      opcode, frame = self.recv_data_frame(control_frame)
    File "/usr/local/telemetry/src/third_party/catapult/telemetry/third_party/websocket-client/websocket/_core.py", line 323, in recv_data_frame
      frame = self.recv_frame()
    File "/usr/local/telemetry/src/third_party/catapult/telemetry/third_party/websocket-client/websocket/_core.py", line 357, in recv_frame
      return self.frame_buffer.recv_frame()
    File "/usr/local/telemetry/src/third_party/catapult/telemetry/third_party/websocket-client/websocket/_abnf.py", line 336, in recv_frame
      self.recv_header()
    File "/usr/local/telemetry/src/third_party/catapult/telemetry/third_party/websocket-client/websocket/_abnf.py", line 286, in recv_header
      header = self.recv_strict(2)
    File "/usr/local/telemetry/src/third_party/catapult/telemetry/third_party/websocket-client/websocket/_abnf.py", line 371, in recv_strict
      bytes_ = self.recv(min(16384, shortage))
    File "/usr/local/telemetry/src/third_party/catapult/telemetry/third_party/websocket-client/websocket/_core.py", line 427, in _recv
      return recv(self.sock, bufsize)
    File "/usr/local/telemetry/src/third_party/catapult/telemetry/third_party/websocket-client/websocket/_socket.py", line 93, in recv
      "Connection is already closed.")


I think this specific issue is basically addressed by Mo's patch. There is a secondary problem with some too-short timeouts in Telemetry which I'm going to try to address in  Issue 796437 . Finally, the planned work to fix mus-viz will be done in a follow-on bug. Closing this as Fixed.

Comment 9 by kbr@chromium.org, Dec 20 2017

Blocking: 796739
Components: -Internals>MUS Internals>Services>WindowService

Comment 11 by kbr@chromium.org, May 29 2018

Blockedon: 842782

Sign in to add a comment