New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 666293 link

Starred by 2 users

Issue metadata

Status: Duplicate
Merged: issue 672382
Owner:
Closed: Dec 2016
Cc:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 1
Type: ----

Blocked on:
issue 672382



Sign in to add a comment

Bot android_n5x_swarming_rel is really flaky

Project Member Reported by sdefresne@chromium.org, Nov 17 2016

Issue description

The bot android_n5x_swarming_rel in the CQ has been really flaky those last two days. My CL https://codereview.chromium.org/2496573002 has been failing this bot many times (when it passed the bot, it failed to land due to patch conflict). I've seen failures on many of my other CLs thus I don't think my CL is causing the flakes (especially since my CL just add .h files to BUILD.gn files and does make any code change).

This make it especially painfull to land upstream CLs.

> Troopers, +cc android Sheriff, +cc noyau that also experienced the flakes.
 
Cc: primiano@chromium.org ssid@chromium.org
You seem to be hitting the following crash:


--------- beginning of crash
  	11-17 12:56:23.491  9692  9753 F libc    : Fatal signal 6 (SIGABRT), code -6 in tid 9753 (Compositor)
  	11-17 12:56:23.603   497   497 I SELinux : SELinux: Loaded file_contexts contexts from /file_contexts.
  	11-17 12:56:23.610   497   497 F DEBUG   : *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
  	11-17 12:56:23.610   497   497 F DEBUG   : Build fingerprint: 'google/bullhead/bullhead:6.0.1/MMB29Q/2480792:userdebug/dev-keys'
  	11-17 12:56:23.610   497   497 F DEBUG   : Revision: 'rev_1.0'
  	11-17 12:56:23.611   497   497 F DEBUG   : ABI: 'arm64'
  	11-17 12:56:23.611   497   497 F DEBUG   : pid: 9692, tid: 9753, name: Compositor  >>> org.chromium.chrome:sandboxed_process0 <<<
  	11-17 12:56:23.611   497   497 F DEBUG   : signal 6 (SIGABRT), code -6 (SI_TKILL), fault addr --------
  	11-17 12:56:23.626   497   497 F DEBUG   : Abort message: '[FATAL:memory_dump_manager.cc(337)] Check failed: (*mdp_iter)->task_runner && (*mdp_iter)->task_runner->RunsTasksOnCurrentThread(). MemoryDumpProvider "VpxVideoDecoder" attempted to unregister itself in a racy way. Please file a crbug.
  	11-17 12:56:23.626   497   497 F DEBUG   : #00 0x0000007f8e965f77 /data/app/org.chromium.chrome-1/base.apk+0x000000000116af77
  	11-17 12:56:23.626   497   497 F DEBUG   : #01 0x0000007f8e9d9e9b /data/app/org.chromium.chrome-1/base.apk+0x00000000011dee9b
  	11-17 12:56:23.626   497   497 F DEBUG   : #02 0x0000007f8f4e469b /data/app/org.chromium.chrome-1/base.apk+0x0000000001ce969b
  	11-17 12:56:23.626   497   497 F DEBUG   : #03 0x0000
  	11-17 12:56:23.626   497   497 F DEBUG   :     x0   0000000000000000  x1   0000000000002619  x2   0000000000000006  x3   0000000000000000
  	11-17 12:56:23.626   497   497 F DEBUG   :     x4   0000000000000000  x5   0000000000000001  x6   0000000000000000  x7   0000000000000000
  	11-17 12:56:23.626   497   497 F DEBUG   :     x8   0000000000000083  x9   0000007f9707d822  x10  0000007f9707d7d8  x11  0000007f9707dae8
  	11-17 12:56:23.626   497   497 F DEBUG   :     x12  0000007f9707d821  x13  0000000000000054  x14  0000007f92e36798  x15  0000007fb3737748
  	11-17 12:56:23.627   497   497 F DEBUG   :     x16  0000007fb3729568  x17  0000007fb36bc13c  x18  0000007fb3737738  x19  0000007f9707f500
  	11-17 12:56:23.627   497   497 F DEBUG   :     x20  0000007f9707f440  x21  0000000000000000  x22  0000000000000006  x23  0000007f94f2e000
  	11-17 12:56:23.627   497   497 F DEBUG   :     x24  0000007f94f2e000  x25  0000007f9707e300  x26  0000000000000000  x27  0000007fa65744b0
  	11-17 12:56:23.627   497   497 F DEBUG   :     x28  0000000000001000  x29  0000007f9707db20  x30  0000007fb36b98d8
  	11-17 12:56:23.627   497   497 F DEBUG   :     sp   0000007f9707db20  pc   0000007fb36bc144  pstate 0000000020000000
  	11-17 12:56:23.638   497   497 F DEBUG   : 
  	11-17 12:56:23.638   497   497 F DEBUG   : backtrace:
  	11-17 12:56:23.638   497   497 F DEBUG   :     #00 pc 000000000006a144  /system/lib64/libc.so (tgkill+8)
  	11-17 12:56:23.639   497   497 W debuggerd64: type=1400 audit(0.0:49): avc: denied { search } for name="org.chromium.chrome" dev="dm-2" ino=377106 scontext=u:r:debuggerd:s0 tcontext=u:object_r:app_data_file:s0:c512,c768 tclass=dir permissive=0
  	11-17 12:56:23.639   497   497 F DEBUG   :     #01 pc 00000000000678d4  /system/lib64/libc.so (pthread_kill+68)
  	11-17 12:56:23.639   497   497 F DEBUG   :     #02 pc 0000000000023838  /system/lib64/libc.so (raise+28)
  	11-17 12:56:23.639   497   497 F DEBUG   :     #03 pc 000000000001dfd8  /system/lib64/libc.so (abort+60)
  	11-17 12:56:23.639   497   497 F DEBUG   :     #04 pc 00000000001cc29c  /data/app/org.chromium.chrome-1/base.apk (offset 0xf85000)
  	11-17 12:56:23.639   497   497 F DEBUG   :     #05 pc 00000000001e6018  /data/app/org.chromium.chrome-1/base.apk (offset 0xf85000)
  	11-17 12:56:23.639   497   497 F DEBUG   :     #06 pc 0000000000259e98  /data/app/org.chromium.chrome-1/base.apk (offset 0xf85000)
  	11-17 12:56:23.639   497   497 F DEBUG   :     #07 pc 0000000000d64698  /data/app/org.chromium.chrome-1/base.apk (offset 0xf85000)
  	11-17 12:56:23.639   497   497 F DEBUG   :     #08 pc 0000000000d6470c  /data/app/org.chromium.chrome-1/base.apk (offset 0xf85000)
  	11-17 12:56:23.640   497   497 F DEBUG   :     #09 pc 00000000008a696c  /data/app/org.chromium.chrome-1/base.apk (offset 0xf85000)
  	11-17 12:56:23.640   497   497 F DEBUG   :     #10 pc 00000000009921dc  /data/app/org.chromium.chrome-1/base.apk (offset 0xf85000)
  	11-17 12:56:23.640   497   497 F DEBUG   :     #11 pc 00000000001c2bf4  /data/app/org.chromium.chrome-1/base.apk (offset 0xf85000)
  	11-17 12:56:23.640   497   497 F DEBUG   :     #12 pc 00000000008e3220  /data/app/org.chromium.chrome-1/base.apk (offset 0xf85000)
  	11-17 12:56:23.640   497   497 F DEBUG   :     #13 pc 00000000001c2bf4  /data/app/org.chromium.chrome-1/base.apk (offset 0xf85000)
  	11-17 12:56:23.640   497   497 F DEBUG   :     #14 pc 00000000026ec29c  /data/app/org.chromium.chrome-1/base.apk (offset 0xf85000)
  	11-17 12:56:23.640   497   497 F DEBUG   :     #15 pc 00000000026ec314  /data/app/org.chromium.chrome-1/base.apk (offset 0xf85000)
  	11-17 12:56:23.640   497   497 F DEBUG   :     #16 pc 00000000008a696c  /data/app/org.chromium.chrome-1/base.apk (offset 0xf85000)
  	11-17 12:56:23.640   497   497 F DEBUG   :     #17 pc 0000000002305938  /data/app/org.chromium.chrome-1/base.apk (offset 0xf85000)
  	11-17 12:56:23.641   497   497 F DEBUG   :     #18 pc 00000000023059a4  /data/app/org.chromium.chrome-1/base.apk (offset 0xf85000)
  	11-17 12:56:23.641   497   497 F DEBUG   :     #19 pc 000000000230e078  /data/app/org.chromium.chrome-1/base.apk (offset 0xf85000)
  	11-17 12:56:23.641   497   497 F DEBUG   :     #20 pc 00000000001c0878  /data/app/org.chromium.chrome-1/base.apk (offset 0xf85000)
  	11-17 12:56:23.641   497   497 F DEBUG   :     #21 pc 00000000001cc9f4  /data/app/org.chromium.chrome-1/base.apk (offset 0xf85000)
  	11-17 12:56:23.641   497   497 F DEBUG   :     #22 pc 00000000001f0784  /data/app/org.chromium.chrome-1/base.apk (offset 0xf85000)
  	11-17 12:56:23.641   497   497 F DEBUG   :     #23 pc 00000000001f0b3c  /data/app/org.chromium.chrome-1/base.apk (offset 0xf85000)
  	11-17 12:56:23.641   497   497 F DEBUG   :     #24 pc 00000000001f11e0  /data/app/org.chromium.chrome-1/base.apk (offset 0xf85000)
  	11-17 12:56:23.641   497   497 F DEBUG   :     #25 pc 00000000001f4b88  /data/app/org.chromium.chrome-1/base.apk (offset 0xf85000)
  	11-17 12:56:23.641   497   497 F DEBUG   :     #26 pc 00000000001f3340  /data/app/org.chromium.chrome-1/base.apk (offset 0xf85000)
  	11-17 12:56:23.642   497   497 F DEBUG   :     #27 pc 00000000002167d4  /data/app/org.chromium.chrome-1/base.apk (offset 0xf85000)
  	11-17 12:56:23.642   497   497 F DEBUG   :     #28 pc 0000000000246e40  /data/app/org.chromium.chrome-1/base.apk (offset 0xf85000)
  	11-17 12:56:23.642   497   497 F DEBUG   :     #29 pc 0000000000247a60  /data/app/org.chromium.chrome-1/base.apk (offset 0xf85000)
  	11-17 12:56:23.642   497   497 F DEBUG   :     #30 pc 000000000024060c  /data/app/org.chromium.chrome-1/base.apk (offset 0xf85000)
  	11-17 12:56:23.642   497   497 F DEBUG   :     #31 pc 0000000000066d24  /system/lib64/libc.so (__pthread_start(void*)+52)
  	11-17 12:56:23.642   497   497 F DEBUG   :     #32 pc 000000000001eb84  /system/lib64/libc.so (__start_thread+16)
  	11-17 12:56:23.838   712  3163 D audio_hw_primary: disable_audio_route: reset and update mixer path: audio-ull-playback



ssid@ / primiano@ any thoughts?
Cc: perezju@chromium.org
Owner: nednguyen@chromium.org
Status: Assigned (was: Untriaged)
It seems the system_health.memory_mobile.browse:media:youtube test is the source of the flake. It was added to the bot in https://codereview.chromium.org/2493203002/
I can help display the test, but Primiano, can you take a look into the race condition?

  	11-17 12:56:23.626   497   497 F DEBUG   : Abort message: '[FATAL:memory_dump_manager.cc(337)] Check failed: (*mdp_iter)->task_runner && (*mdp_iter)->task_runner->RunsTasksOnCurrentThread(). MemoryDumpProvider "VpxVideoDecoder" attempted to unregister itself in a racy way. Please file a crbug.
Cc: -primiano@chromium.org nednguyen@chromium.org
Owner: primiano@chromium.org
Project Member

Comment 5 by bugdroid1@chromium.org, Nov 17 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/1d8f7b58d0e1050a776f7ff861873a3288d1c7fa

commit 1d8f7b58d0e1050a776f7ff861873a3288d1c7fa
Author: nednguyen <nednguyen@google.com>
Date: Thu Nov 17 23:10:41 2016

Disable browse:media:youtube system health smoke test

BUG= 666293 
TBR=primiano@chromium.org

Review-Url: https://codereview.chromium.org/2511843002
Cr-Commit-Position: refs/heads/master@{#433006}

[modify] https://crrev.com/1d8f7b58d0e1050a776f7ff861873a3288d1c7fa/tools/perf/benchmarks/system_health_smoke_test.py

Cc: dcasta...@chromium.org hjd@chromium.org
Components: -Infra Internals>Tracing
Labels: -Restrict-View-Google Hotlist-MemoryInfra
I need a bit of history on this: is the problem that we recently added a youtube page and that is flaky? Or is the problem that suddenly the youtube page became flaky?

In either case this seems a bug with the VPX memory dump provider. If it hits that check it's because it is unregistering from a different thread.

+dcastagna any idea why VpxVideoDecoder::MemoryPool::MemoryPool could be destroyed on a different page when playing a youtube page?

+hjd can you plz check and see if we can build up a mini repro case?

-RVG as there doesn't seem to be anything secret here
Do you mean on a different thread? MemoryPool seems to be RefCountedThreadSafe, that probably means it can be destroyed on any thread.
This is a dup of  crbug.com/637409 , no?

Comment 8 by kbr@chromium.org, Dec 8 2016

Blockedon: 672382

Comment 9 by treib@chromium.org, Dec 8 2016

Cc: yolandyan@chromium.org
 Issue 672141  has been merged into this issue.
Mergedinto: 672382
Status: Duplicate (was: Assigned)

Sign in to add a comment