ChromeOS telemetry tests failing with Unhandled DevtoolsTargetCrashException: Devtools target crashed |
||||
Issue description
Works on R53 8370.0.0 / 53.0.2746.0
Broken starting R53 8372.0.0 / 53.0.2748.0
The DevtoolsTargetCrashException is seen when running telemetry autotests, which in our case are CFM/hotrod tests. Hangouts kiosk app also fails to start with the following message "Hangouts is unable to connect to Google. We'll keep trying..." and error_007. The hangouts app launch failure can be reproduced manually as well with and without enrolling the device in CFM.
Device logs and test logs available at the link in the next comment.
Exception from test run debug log:
05/26 22:31:42.044 WARNI| test:0606| Autotest caught exception when running test:
Traceback (most recent call last):
File "/usr/local/autotest/common_lib/test.py", line 600, in _exec
_call_test_function(self.execute, *p_args, **p_dargs)
File "/usr/local/autotest/common_lib/test.py", line 810, in _call_test_function
raise error.UnhandledTestFail(e)
UnhandledTestFail: Unhandled DevtoolsTargetCrashException: Devtools target crashed
********************************************************************************
(/usr/local/telemetry/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_backend.py:341 _AddDebuggingInformation) Received a socket error in the browser connection and the tab no longer exists. The tab probably crashed.
********************************************************************************
(/usr/local/telemetry/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_backend.py:342 _AddDebuggingInformation) Debugger url: ws://127.0.0.1:54235/devtools/page/e5439e74-02c4-4f54-b9f2-beb712257c13
Found Minidump: False
Stack Trace:
********************************************************************************
Cannot get stack trace on CrOS
********************************************************************************
Standard output:
********************************************************************************
Cannot get standard output on CrOS
********************************************************************************
Traceback (most recent call last):
File "/usr/local/autotest/common_lib/test.py", line 804, in _call_test_function
return func(*args, **dargs)
File "/usr/local/autotest/common_lib/test.py", line 461, in execute
dargs)
File "/usr/local/autotest/common_lib/test.py", line 347, in _call_run_once_with_retry
postprocess_profiled_run, args, dargs)
File "/usr/local/autotest/common_lib/test.py", line 376, in _call_run_once
self.run_once(*args, **dargs)
File "/usr/local/autotest/tests/enterprise_CFM_Sanity/enterprise_CFM_Sanity.py", line 142, in run_once
self._hangouts_sanity_test(cfm_webview_context)
File "/usr/local/autotest/tests/enterprise_CFM_Sanity/enterprise_CFM_Sanity.py", line 42, in _hangouts_sanity_test
cfm_util.wait_for_oobe_start_page(webview_context)
File "/usr/local/autotest/common_lib/cros/cfm_util.py", line 83, in wait_for_oobe_start_page
"window.hrOobIsStartPageForTest() === true;", DEFAULT_TIMEOUT)
File "/usr/local/telemetry/src/third_party/catapult/telemetry/telemetry/internal/browser/web_contents.py", line 123, in WaitForJavaScriptExpression
util.WaitFor(IsJavaScriptExpressionTrue, timeout)
File "/usr/local/telemetry/src/third_party/catapult/telemetry/telemetry/core/util.py", line 86, in WaitFor
res = condition()
File "/usr/local/telemetry/src/third_party/catapult/telemetry/telemetry/internal/browser/web_contents.py", line 116, in IsJavaScriptExpressionTrue
return bool(self.EvaluateJavaScript(expr))
File "/usr/local/telemetry/src/third_party/catapult/telemetry/telemetry/internal/browser/web_contents.py", line 187, in EvaluateJavaScript
expr, context_id=None, timeout=timeout)
File "/usr/local/telemetry/src/third_party/catapult/telemetry/telemetry/internal/browser/web_contents.py", line 215, in EvaluateJavaScriptInContext
expr, context_id=context_id, timeout=timeout)
File "/usr/local/telemetry/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_backend.py", line 32, in inner
return func(inspector_backend, *args, **kwargs)
File "/usr/local/telemetry/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_backend.py", line 203, in EvaluateJavaScript
return self._runtime.Evaluate(expr, context_id, timeout)
File "/usr/local/telemetry/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_runtime.py", line 45, in Evaluate
res = self._inspector_websocket.SyncRequest(request, timeout)
File "/usr/local/telemetry/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_websocket.py", line 110, in SyncRequest
res = self._Receive(timeout)
File "/usr/local/telemetry/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_websocket.py", line 166, in _Receive
self._HandleNotification(result)
File "/usr/local/telemetry/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_websocket.py", line 179, in _HandleNotification
self._domain_handlers[domain_name](result)
File "/usr/local/telemetry/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_backend.py", line 288, in _HandleInspectorDomainNotification
raise exception
DevtoolsTargetCrashException: Devtools target crashed
********************************************************************************
(/usr/local/telemetry/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_backend.py:341 _AddDebuggingInformation) Received a socket error in the browser connection and the tab no longer exists. The tab probably crashed.
********************************************************************************
(/usr/local/telemetry/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_backend.py:342 _AddDebuggingInformation) Debugger url: ws://127.0.0.1:54235/devtools/page/e5439e74-02c4-4f54-b9f2-beb712257c13
Found Minidump: False
Stack Trace:
********************************************************************************
Cannot get stack trace on CrOS
********************************************************************************
Standard output:
********************************************************************************
Cannot get standard output on CrOS
********************************************************************************
Crash dump:
Operating system: Linux
0.0.0 Linux 3.8.11 #1 SMP Wed May 25 18:56:08 PDT 2016 x86_64
CPU: amd64
family 6 model 69 stepping 1
1 CPU
GPU: UNKNOWN
Crash reason: SIGSEGV
Crash address: 0x0
Process uptime: not available
Thread 0 (crashed)
0 chrome + 0x31a2d75
rax = 0x0000000000000000 rdx = 0x0000000000000000
rcx = 0x0000000000000000 rbx = 0x00001fb9c02e53a8
rsi = 0x000031206c231248 rdi = 0x000031206c3e06c0
rbp = 0x00007ffc8cf36d90 rsp = 0x00007ffc8cf36d40
r8 = 0x0000000000000004 r9 = 0x0000000000000000
r10 = 0x0000000000000001 r11 = 0x00007f1fcc1ae040
r12 = 0x00007ffc8cf36db0 r13 = 0x00007ffc8cf36e90
r14 = 0x0000000000000000 r15 = 0x00003beee248ac58
rip = 0x00007f1fc8607d75
Found by: given as instruction pointer in context
1 chrome + 0x2e45ebf
rbp = 0x00007ffc8cf36e10 rsp = 0x00007ffc8cf36da0
rip = 0x00007f1fc82aaebf
Found by: previous frame's frame pointer
2 chrome + 0x2e27f54
rbp = 0x00007ffc8cf36fa0 rsp = 0x00007ffc8cf36e20
rip = 0x00007f1fc828cf54
Found by: previous frame's frame pointer
3 chrome + 0x330e981
rbp = 0x00007ffc8cf370d0 rsp = 0x00007ffc8cf36fb0
rip = 0x00007f1fc8773981
Found by: previous frame's frame pointer
4 chrome + 0xf1e1bf
rbp = 0x00007ffc8cf37190 rsp = 0x00007ffc8cf370e0
rip = 0x00007f1fc63831bf
Found by: previous frame's frame pointer
5 chrome + 0xd8afef
rbp = 0x00007ffc8cf372b0 rsp = 0x00007ffc8cf371a0
rip = 0x00007f1fc61effef
Found by: previous frame's frame pointer
6 0x377da9f063c7
rbp = 0x00007ffc8cf372e0 rsp = 0x00007ffc8cf372c0
rip = 0x0000377da9f063c7
Found by: previous frame's frame pointer
7 0x377daa85193f
rbp = 0x00007ffc8cf37340 rsp = 0x00007ffc8cf372f0
rip = 0x0000377daa85193f
Found by: previous frame's frame pointer
8 0x377daa7dff5d
rbp = 0x00007ffc8cf37388 rsp = 0x00007ffc8cf37350
rip = 0x0000377daa7dff5d
Found by: previous frame's frame pointer
9 0x377da9f43b43
rbp = 0x00007ffc8cf373b8 rsp = 0x00007ffc8cf37398
rip = 0x0000377da9f43b43
Found by: previous frame's frame pointer
10 0x377da9f2742f
rbp = 0x00007ffc8cf37420 rsp = 0x00007ffc8cf373c8
rip = 0x0000377da9f2742f
Found by: previous frame's frame pointer
11 chrome + 0xdf751a
rbp = 0x00007ffc8cf374f0 rsp = 0x00007ffc8cf37430
rip = 0x00007f1fc625c51a
Found by: previous frame's frame pointer
12 chrome + 0xd7cf71
rbp = 0x00007ffc8cf37600 rsp = 0x00007ffc8cf37500
rip = 0x00007f1fc61e1f71
Found by: previous frame's frame pointer
13 chrome + 0x10a1cca
rbp = 0x00007ffc8cf376d0 rsp = 0x00007ffc8cf37610
rip = 0x00007f1fc6506cca
Found by: previous frame's frame pointer
14 chrome + 0x318df95
rbp = 0x00007ffc8cf37710 rsp = 0x00007ffc8cf376e0
rip = 0x00007f1fc85f2f95
Found by: previous frame's frame pointer
15 chrome + 0x318d6ca
rbp = 0x00007ffc8cf37830 rsp = 0x00007ffc8cf37720
rip = 0x00007f1fc85f26ca
Found by: previous frame's frame pointer
16 chrome + 0x2fd90f7
rbp = 0x00007ffc8cf37910 rsp = 0x00007ffc8cf37840
rip = 0x00007f1fc843e0f7
Found by: previous frame's frame pointer
17 chrome + 0x1149d6b
rbp = 0x00007ffc8cf37970 rsp = 0x00007ffc8cf37920
rip = 0x00007f1fc65aed6b
Found by: previous frame's frame pointer
18 chrome + 0x5370de8
rbp = 0x00007ffc8cf37990 rsp = 0x00007ffc8cf37980
rip = 0x00007f1fca7d5de8
Found by: previous frame's frame pointer
19 chrome + 0xc91d9b
rbp = 0x00007ffc8cf379b0 rsp = 0x00007ffc8cf379a0
rip = 0x00007f1fc60f6d9b
Found by: previous frame's frame pointer
20 chrome + 0xcaccdd
rbp = 0x00007ffc8cf37ab0 rsp = 0x00007ffc8cf379c0
rip = 0x00007f1fc6111cdd
Found by: previous frame's frame pointer
21 chrome + 0x11777de
rbp = 0x00007ffc8cf37cd0 rsp = 0x00007ffc8cf37ac0
rip = 0x00007f1fc65dc7de
Found by: previous frame's frame pointer
22 chrome + 0xcaccdd
rbp = 0x00007ffc8cf37dd0 rsp = 0x00007ffc8cf37ce0
rip = 0x00007f1fc6111cdd
Found by: previous frame's frame pointer
23 chrome + 0xc9bdf6
rbp = 0x00007ffc8cf37f80 rsp = 0x00007ffc8cf37de0
rip = 0x00007f1fc6100df6
Found by: previous frame's frame pointer
24 chrome + 0xc9c609
rbp = 0x00007ffc8cf37fe0 rsp = 0x00007ffc8cf37f90
rip = 0x00007f1fc6101609
Found by: previous frame's frame pointer
25 chrome + 0x15349f8
rbp = 0x00007ffc8cf38030 rsp = 0x00007ffc8cf37ff0
rip = 0x00007f1fc69999f8
Found by: previous frame's frame pointer
26 chrome + 0x151e449
rbp = 0x00007ffc8cf38080 rsp = 0x00007ffc8cf38040
rip = 0x00007f1fc6983449
Found by: previous frame's frame pointer
27 chrome + 0x3bf8656
rbp = 0x00007ffc8cf381e0 rsp = 0x00007ffc8cf38090
rip = 0x00007f1fc905d656
Found by: previous frame's frame pointer
28 chrome + 0x14e0350
rbp = 0x00007ffc8cf382a0 rsp = 0x00007ffc8cf381f0
rip = 0x00007f1fc6945350
Found by: previous frame's frame pointer
29 chrome + 0x14e05c1
rbp = 0x00007ffc8cf38320 rsp = 0x00007ffc8cf382b0
rip = 0x00007f1fc69455c1
Found by: previous frame's frame pointer
30 chrome + 0x14df15b
rbp = 0x00007ffc8cf38340 rsp = 0x00007ffc8cf38330
rip = 0x00007f1fc694415b
Found by: previous frame's frame pointer
31 chrome + 0x117c7bf
rbp = 0x00007ffc8cf383d0 rsp = 0x00007ffc8cf38350
rip = 0x00007f1fc65e17bf
Found by: previous frame's frame pointer
32 libc-2.19.so + 0x1ffd6
rbp = 0x00007ffc8cf384a0 rsp = 0x00007ffc8cf383e0
rip = 0x00007f1fc3a8cfd6
Found by: previous frame's frame pointer
33 chrome + 0xc8b630
rbp = 0x00007ffc8cf384a0 rsp = 0x00007ffc8cf38400
rip = 0x00007f1fc60f0630
Found by: stack scanning
34 chrome + 0x5cc98a0
rbp = 0x00007ffc8cf384a0 rsp = 0x00007ffc8cf38418
rip = 0x00007f1fcb12e8a0
Found by: stack scanning
35 ld-2.19.so + 0xf8ac
rbp = 0x00007ffc8cf384a0 rsp = 0x00007ffc8cf38470
rip = 0x00007f1fc52508ac
Found by: stack scanning
36 chrome + 0x117c5e8
rbp = 0x00007ffc8cf384a0 rsp = 0x00007ffc8cf38488
rip = 0x00007f1fc65e15e8
Found by: stack scanning
37 chrome + 0x117c611
rsp = 0x00007ffc8cf384b0 rip = 0x00007f1fc65e1611
Found by: stack scanning
38 0x7ffc8cfaf000
rsp = 0x00007ffc8cf385c0 rip = 0x00007ffc8cfaf000
Found by: stack scanning
39 chrome + 0x40
rsp = 0x00007ffc8cf38600 rip = 0x00007f1fc5465040
Found by: stack scanning
40 0x7f1fc5241000
rsp = 0x00007ffc8cf38630 rip = 0x00007f1fc5241000
Found by: stack scanning
41 chrome + 0x117c5e8
rsp = 0x00007ffc8cf38650 rip = 0x00007f1fc65e15e8
Found by: stack scanning
,
May 27 2016
,
May 27 2016
,
May 27 2016
Even with normal enrollment, you can't get past the sign-in screens. Also, if you open the browser by going to 'browse as guest' and try going to any website, it loads the site and crashes with the "Aw, Snap!" message within couple seconds.
,
May 27 2016
We saw similar errors on amd64-generic-telemetry starting at R53-8365: https://build.chromium.org/p/chromiumos.chromium/builders/amd64-generic-telemetry/builds/8973 But it went away at R53-8373: https://build.chromium.org/p/chromiumos.chromium/builders/amd64-generic-telemetry/builds/8985 You might want to try updating to 8373? More info at issue 614474
,
May 27 2016
It seems to be related to the chrome version rather than CrOS version. 53.0.2746.0 (8370.0.0) works, 53.0.2748.0 (8372.0.0 - 8375.0.0) does NOT work but the latest 53.0.2749.3 (8376.0.0) works too. I am marking this as wont fix since it is working on ToT. |
||||
►
Sign in to add a comment |
||||
Comment 1 by harpreet@chromium.org
, May 27 2016