New issue
Advanced search Search tips
Note: Color blocks (like █ or █) mean that a user may not be available. Tooltip shows the reason.

Issue 686193 link

Starred by 1 user
Status: WontFix
Owner: ----
Closed: Sep 10
Cc:
█ ravisadineni@chromium.org
█ warx@chromium.org
█ reve...@chromium.org
jamescook@chromium.org
achuith@chromium.org
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 2
Type: Bug



Sign in to add a comment

amd64-generic-telemetry failure in vmtest telemetry_UnitTests SimpleTestVerify, PlayActionTest.testPlayWaitForPlayTimeout and webservd crash

Project Member Reported by jamescook@chromium.org, Jan 27 2017 
Failing run:
https://build.chromium.org/p/chromiumos.chromium/builders/amd64-generic-telemetry/builds/10676

Failed on initial vmtest run and on retry run.

I don't see any telemetry-related changes on the chrome side blamelist.

From VMTest stdout:
10:57:44 INFO | autoserv| STATUS: START	telemetry_UnitTests.user	telemetry_UnitTests.user	timestamp=1485534680	localtime=Jan 27 10:31:20
10:57:44 INFO | autoserv| parsing partial test telemetry_UnitTests.user telemetry_UnitTests.user
10:57:44 INFO | autoserv| RUNNING: RUNNING
10:57:44 INFO | autoserv| Subdir: telemetry_UnitTests.user
10:57:44 INFO | autoserv| Testname: telemetry_UnitTests.user
10:57:44 INFO | autoserv| 
10:57:44 INFO | autoserv| 
10:57:44 INFO | autoserv| STATUS: FAIL	telemetry_UnitTests.user	telemetry_UnitTests.user	timestamp=1485536259	localtime=Jan 27 10:57:39	The unit tests of /usr/local/telemetry/src/third_party/catapult/telemetry failed.
10:57:44 INFO | autoserv| update RUNNING reason: The unit tests of /usr/local/telemetry/src/third_party/catapult/telemetry failed.

There are ~30 crashes in webservd


Crash reason:  SIGABRT
Crash address: 0x0
Process uptime: not available

Thread 0 (crashed)
 0  libc-2.23.so + 0x33e32
    rax = 0x0000000000000000   rdx = 0x0000000000000006
    rcx = 0x000078af899dee32   rbx = 0x000078af8a609847
    rsi = 0x000000000000045e   rdi = 0x000000000000045e
    rbp = 0x00007fff425565c0   rsp = 0x00007fff42556498
     r8 = 0x000056683c669910    r9 = 0x0000000000000000
    r10 = 0x0000000000000008   r11 = 0x0000000000000202
    r12 = 0x00007fff42556c18   r13 = 0x0000000000000012
    r14 = 0x000000000000001f   r15 = 0x000056683c654c80
    rip = 0x000078af899dee32
    Found by: given as instruction pointer in context
 1  libminijailpreload.so + 0xcc17
    rbp = 0x00007fff425565f0   rsp = 0x00007fff425565d0
    rip = 0x000078af8a607c17
    Found by: previous frame's frame pointer
 2  libc-2.23.so + 0x33eb0
    rbp = 0x00007fff42556a80   rsp = 0x00007fff42556600
    rip = 0x000078af899deeb0
    Found by: previous frame's frame pointer
 3  libmicrohttpd.so.10.32.0 + 0x985b
    rbp = 0x00007fff42556ae0   rsp = 0x00007fff42556a90
    rip = 0x000078af8a4cc85b
    Found by: previous frame's frame pointer
 4  libmicrohttpd.so.10.32.0 + 0xa39e
    rbp = 0x00007fff42556cc0   rsp = 0x00007fff42556af0
    rip = 0x000078af8a4cd39e
    Found by: previous frame's frame pointer
 5  libmicrohttpd.so.10.32.0 + 0x9bee
    rbp = 0x00007fff42556ce0   rsp = 0x00007fff42556cd0
    rip = 0x000078af8a4ccbee
    Found by: previous frame's frame pointer
 6  webservd + 0x29671
    rbp = 0x00007fff425574b0   rsp = 0x00007fff42556cf0
    rip = 0x000056683a7b3671
    Found by: previous frame's frame pointer
 7  webservd + 0x2ba78
    rbp = 0x00007fff42557500   rsp = 0x00007fff425574c0
    rip = 0x000056683a7b5a78
    Found by: previous frame's frame pointer
 8  libbase-core-395517.so + 0x8d6f2
    rbp = 0x00007fff42557620   rsp = 0x00007fff42557510
    rip = 0x000078af8a3196f2
    Found by: previous frame's frame pointer
 9  libbase-core-395517.so + 0xb16ae
    rbp = 0x00007fff42557890   rsp = 0x00007fff42557630
    rip = 0x000078af8a33d6ae
    Found by: previous frame's frame pointer
10  libbase-core-395517.so + 0xb1a3b
    rbp = 0x00007fff425578b0   rsp = 0x00007fff425578a0
    rip = 0x000078af8a33da3b
    Found by: previous frame's frame pointer
11  libbase-core-395517.so + 0xb1ddb
    rbp = 0x00007fff42557950   rsp = 0x00007fff425578c0
    rip = 0x000078af8a33dddb
    Found by: previous frame's frame pointer
12  libbase-core-395517.so + 0xb6b19
    rbp = 0x00007fff425579c0   rsp = 0x00007fff42557960
    rip = 0x000078af8a342b19
    Found by: previous frame's frame pointer
13  libbase-core-395517.so + 0xb11a1
    rbp = 0x00007fff42557b80   rsp = 0x00007fff425579d0
    rip = 0x000078af8a33d1a1
    Found by: previous frame's frame pointer
14  libbase-core-395517.so + 0xe22d0
    rbp = 0x00007fff42557be0   rsp = 0x00007fff42557b90
    rip = 0x000078af8a36e2d0
    Found by: previous frame's frame pointer
15  libbrillo-core-395517.so + 0x59d36
    rbp = 0x00007fff42557c30   rsp = 0x00007fff42557bf0
    rip = 0x000078af8a5d7d36
    Found by: previous frame's frame pointer
16  libbrillo-core-395517.so + 0x237b6
    rbp = 0x00007fff42557c60   rsp = 0x00007fff42557c40
    rip = 0x000078af8a5a17b6
    Found by: previous frame's frame pointer
17  webservd + 0xb435
    rbp = 0x00007fff42558360   rsp = 0x00007fff42557c70
    rip = 0x000056683a795435
    Found by: previous frame's frame pointer
18  libc-2.23.so + 0x20796
    rbp = 0x00007fff42558430   rsp = 0x00007fff42558370
    rip = 0x000078af899cb796
    Found by: previous frame's frame pointer
19  libminijailpreload.so + 0x7790
    rbp = 0x00007fff42558430   rsp = 0x00007fff42558390
    rip = 0x000078af8a602790
    Found by: stack scanning
20  webservd + 0x2f0a0
    rbp = 0x00007fff42558430   rsp = 0x00007fff425583a8
    rip = 0x000056683a7b90a0
    Found by: stack scanning
21  webservd + 0x2f0a0
    rbp = 0x00007fff42558430   rsp = 0x00007fff425583f0
    rip = 0x000056683a7b90a0
    Found by: stack scanning
22  libminijailpreload.so + 0x772c
    rbp = 0x00007fff42558430   rsp = 0x00007fff42558400
    rip = 0x000078af8a60272c
    Found by: stack scanning
23  ld-2.23.so + 0xfe10
    rbp = 0x00007fff42558430   rsp = 0x00007fff42558408
    rip = 0x000078af8a3ffe10
    Found by: stack scanning
24  webservd + 0xacc0
    rbp = 0x00007fff42558430   rsp = 0x00007fff42558418
    rip = 0x000056683a794cc0
    Found by: stack scanning
25  webservd + 0xace9
    rsp = 0x00007fff42558440   rip = 0x000056683a794ce9
    Found by: stack scanning
26  0x7fff4259a000
    rsp = 0x00007fff42558500   rip = 0x00007fff4259a000
    Found by: stack scanning
27  webservd + 0x40
    rsp = 0x00007fff42558540   rip = 0x000056683a78a040
    Found by: stack scanning
28  0x78af8a3f0000
    rsp = 0x00007fff42558570   rip = 0x000078af8a3f0000
    Found by: stack scanning
29  webservd + 0xacc0
    rsp = 0x00007fff42558590   rip = 0x000056683a794cc0
    Found by: stack scanning

I also see a single unit test failure:

01/27 08:40:30.253 INFO |run_chromeos_tests:0052| [192/1064] telemetry.internal.actions.play_unittest.PlayActionTest.testPlayWaitForPlayTimeout failed unexpectedly 13.9897s:
01/27 08:40:30.254 INFO |run_chromeos_tests:0052|   Detected media error while waiting for playing: Media error: error, code:4
01/27 08:40:30.254 INFO |run_chromeos_tests:0052|   Traceback (most recent call last):
01/27 08:40:30.255 INFO |run_chromeos_tests:0052|     File "/usr/local/telemetry/src/third_party/catapult/telemetry/telemetry/testing/browser_test_case.py", line 41, in WrappedMethod
01/27 08:40:30.255 INFO |run_chromeos_tests:0052|       method(self)
01/27 08:40:30.256 INFO |run_chromeos_tests:0052|     File "/usr/local/telemetry/src/third_party/catapult/telemetry/telemetry/internal/actions/play_unittest.py", line 73, in testPlayWaitForPlayTimeout
01/27 08:40:30.256 INFO |run_chromeos_tests:0052|       self.assertRaises(py_utils.TimeoutException, action.RunAction, self._tab)
01/27 08:40:30.257 INFO |run_chromeos_tests:0052|   AssertionError: TimeoutException not raised

I am uncertain if the two are related.

Comment 1 by jamescook@chromium.org, Jan 27 2017 

The test that fails is inconsistent. On the first VMTest run this test failed:
telemetry.internal.actions.repeatable_scroll_unittest.RepeatableScrollActionTest.testRepeatableScrollActionNoRepeats

Per /var/log/messages webservd seems to be starting OK, and I don't see crash_reporter getting invoked.

2017-01-27T17:07:02.477313+00:00 WARNING webservd[1137]: libminijail[1137]: logging seccomp filter failures
2017-01-27T17:07:02.477860+00:00 INFO webservd[1137]: [INFO:config.cc(101)] Loading default server configuration...
2017-01-27T17:07:02.480063+00:00 INFO webservd[1137]: [INFO:server.cc(185)] Generating new certificate...
<snip>
2017-01-27T17:07:02.561115+00:00 INFO webservd[1137]: [INFO:protocol_handler.cc(167)] Starting HTTP protocol handler on port: 80
2017-01-27T17:07:02.562119+00:00 INFO webservd[1137]: [INFO:protocol_handler.cc(272)] Protocol handler started
2017-01-27T17:07:02.562239+00:00 INFO webservd[1137]: [INFO:protocol_handler.cc(167)] Starting HTTPS protocol handler on port: 443
2017-01-27T17:07:02.573775+00:00 INFO webservd[1137]: [INFO:protocol_handler.cc(272)] Protocol handler started
2017-01-27T17:07:02.578783+00:00 INFO webservd[1137]: [INFO:server.cc(119)] Firewall service is on-line. Opening firewall for protocol handlers
2017-01-27T17:07:02.600204+00:00 INFO webservd[1137]: [INFO:server.cc(50)] Successfully opened up port 80 on interface 
2017-01-27T17:07:02.600208+00:00 INFO firewalld[647]: Punching hole for TCP port 443 on interface ''
<snip>
2017-01-27T17:07:02.626187+00:00 INFO webservd[1137]: [INFO:server.cc(50)] Successfully opened up port 443 on interface

Comment 2 by jamescook@chromium.org, Jan 27 2017

Labels: -Pri-1 Pri-2
The failure is flaky, the next build passed.

https://build.chromium.org/p/chromiumos.chromium/builders/amd64-generic-telemetry/builds/10677

I'm going to sit on this until I see it happen again.

Comment 3 by jamescook@chromium.org, Jan 30 2017

Cc: jamescook@chromium.org
Owner: ----
Status: Untriaged (was: Assigned)

Comment 4 by elijahtaylor@chromium.org, Sep 7

Components: Tests>Telemetry

Comment 5 by achuith@chromium.org, Sep 10

Status: WontFix (was: Untriaged)
Old bug
►

Sign in to add a comment