New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 663293 link

Starred by 2 users

Issue metadata

Status: Verified
Owner:
Last visit 16 days ago
Closed: Dec 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Linux
Pri: 1
Type: Bug-Regression



Sign in to add a comment

Browser crash when accessing the webcam (webrtc related)

Reported by doronf.p...@gmail.com, Nov 8 2016

Issue description

UserAgent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.28 Safari/537.36

Steps to reproduce the problem:
1. Browse to: https://test.webrtc.org/
2. Run the test
3. See "Aw, Snap!"

What is the expected behavior?
Tests should succeed with no crash.

What went wrong?
I was able to get a stack trace when running in debug:

Received signal 11 SEGV_MAPERR fffffffd4fd2d633
#0 0x5605a6b441fe <unknown>
#1 0x5605a6b445bb <unknown>
#2 0x7f5325ed3e20 <unknown>
#3 0x7f531e88ef12 vpx_codec_destroy
#4 0x5605aac1dd72 <unknown>
#5 0x5605aac08761 <unknown>
#6 0x5605aac0909b <unknown>
#7 0x5605aac09170 <unknown>
#8 0x5605aac0d114 <unknown>
#9 0x5605aac0d526 <unknown>
#10 0x5605aabef215 <unknown>
#11 0x5605aa36b615 <unknown>
#12 0x5605aa36b639 <unknown>
#13 0x7f5325eca494 start_thread
#14 0x7f531c0685dd clone
  r8: 0000000000000009  r9: 00007f531ebf1ac0 r10: 00007f52ff5ad740 r11: fffffffd4fd2d613
 r12: 00001082f988bc40 r13: 00001082f946a008 r14: 00001082f988b970 r15: 0000000000000064
  di: 0000000000000079  si: 00001082f93736e0  bp: 00001082f949be98  bx: 00001082f960ddc0
  dx: fffffffd4fd2d613  ax: 0000000000000001  cx: 0000000000010000  sp: 00007f52ff5ad860
  ip: 00007f531e88ef12 efl: 0000000000010202 cgf: 002b000000000033 erf: 0000000000000005
 trp: 000000000000000e msk: 0000000000000000 cr2: fffffffd4fd2d633
[end of stack trace]

Crashed report ID: 

How much crashed? Just one tab

Is it a problem with a plugin? N/A 

Did this work before? Yes Not sure.

Chrome version: 55.0.2883.28  Channel: n/a
OS Version: Linux 4.8.6-gentoo #1 SMP PREEMPT Sun Nov 6 16:15:05 IST 2016 x86_64 Intel(R) Core(TM) i7-5600U CPU @ 2.60GHz GenuineIntel GNU/Linux
Flash Version: 

I'm using a videconf application created by https://bluejeans.com/ and I have the same problem there.

It worked well for me until version chromium-53.0.2785.143 and the problem started with version chromium-54.0.2840.59
 
Updating that the problem still exists in chromium-55.0.2883.44.
Stack trace:

Received signal 11 SEGV_MAPERR fffffffd4f6f79cc
#0 0x56121242fc6e <unknown>
#1 0x56121243002b <unknown>
#2 0x7f66f9d56e20 <unknown>
#3 0x7f66f2711f12 vpx_codec_destroy
#4 0x561216547122 <unknown>
#5 0x561216531a51 <unknown>
#6 0x56121653238b <unknown>
#7 0x561216532460 <unknown>
#8 0x561216536414 <unknown>
#9 0x561216536826 <unknown>
#10 0x561216518405 <unknown>
#11 0x561215c8cba5 <unknown>
#12 0x561215c8cbc9 <unknown>
#13 0x7f66f9d4d494 start_thread
#14 0x7f66efeeb5dd clone
  r8: 0000000000000009  r9: 00007f66f2a74ac0 r10: 00002483bfdbf160 r11: fffffffd4f6f79ac
 r12: 00002483bfbdf560 r13: 00002483bfd34008 r14: 00002483bfbdf480 r15: 0000000000000064
  di: 0000000000000079  si: 00002483bfdbfdc0  bp: 00002483bfd2ee98  bx: 00002483c0042500
  dx: fffffffd4f6f79ac  ax: 0000000000000001  cx: 0000000000010000  sp: 00007f66d2601860
  ip: 00007f66f2711f12 efl: 0000000000010202 cgf: 002b000000000033 erf: 0000000000000005
 trp: 000000000000000e msk: 0000000000000000 cr2: fffffffd4f6f79cc
[end of stack trace]
Cc: kkaluri@chromium.org
Labels: Needs-Feedback
Unable to reproduce this issue on Ubuntu 14.04 with chrome beta 55.0.2883.28.
These are the steps i have followed 
1. Opened the google chrome
2. Opened the url "https://test.webrtc.org/"
3. Clicked on the start button

No Crash is observed. please look into the attached screencast and let us know your observations.

Thank you...
Issue 663293.webm
5.4 MB View Download
Hi,
thanks for taking the time. Attaching my screen shot.
The main difference I notice is that I'm not using Flash (I'm avoising close source binaries). Could this be a reason?
chromium.webm
1.5 MB View Download
Labels: -Pri-2 M-55 Pri-1
Unable to reproduce the issue, even with flash disabled.

Please look into the attached screen-cast and let us know your observations.
Issue 663293 (2).webm
5.7 MB View Download
Cc: ranjitkan@chromium.org pbomm...@chromium.org brajkumar@chromium.org
Missed the below details in the above comment:

Issue was rechecked on updated version of Chrome beta 55.0.2883.52 on Ubuntu 14.04

doronf.public@ Could you please update your chrome to latest beta version and let us know if the issue still exists. Also can you please help us with a crash ID generated from chrome://crashes. This will help us to triage the issue better.

Thanks.!
Cc: tnakamura@chromium.org
Components: Blink>WebRTC
Hey,
sorry for the delay (it takes me ~4 hours to compile each version).
Attaching a screenshot of the same crash on Chrome beta 55.0.2883.52 on Gentoo (Portage 2.3.0 (python 3.4.3-final-0, default/linux/amd64/13.0/no-multilib, gcc-4.9.3, glibc-2.22-r4, 4.8.6-gentoo x86_64)).

Unfortunately I'm missing the option to generate crash reports as you can see in the attached png file. Let me know how to enable it.

In the meantime, here's the generated stack trace:
Received signal 11 SEGV_MAPERR fffffffd5357b838
#0 0x55950a758f5e <unknown>
#1 0x55950a75931b <unknown>
#2 0x7f9bec4e1e20 <unknown>
#3 0x7f9be4e9cf12 vpx_codec_destroy
#4 0x55950e871a02 <unknown>
#5 0x55950e85c331 <unknown>
#6 0x55950e85cc6b <unknown>
#7 0x55950e85cd40 <unknown>
#8 0x55950e860cf4 <unknown>
#9 0x55950e861106 <unknown>
#10 0x55950e842ce5 <unknown>
#11 0x55950dfb7455 <unknown>
#12 0x55950dfb7479 <unknown>
#13 0x7f9bec4d8494 start_thread
#14 0x7f9be26765dd clone
  r8: 0000000000000009  r9: 00007f9be51ffac0 r10: 00007f9bc3d9a740 r11: fffffffd5357b818
 r12: 00003897892b5300 r13: 0000389789525008 r14: 00003897892b5130 r15: 0000000000000064
  di: 0000000000000079  si: 000038978933e6e0  bp: 000038978951be98  bx: 00003897897dc940
  dx: fffffffd5357b818  ax: 0000000000000001  cx: 0000000000010000  sp: 00007f9bc3d9a860
  ip: 00007f9be4e9cf12 efl: 0000000000010202 cgf: 002b000000000033 erf: 0000000000000005
 trp: 000000000000000e msk: 0000000000000000 cr2: fffffffd5357b838
[end of stack trace]
 
55.0.2883.52.webm
1.9 MB View Download
Screenshot_20161120_131001.png
67.7 KB View Download
Cc: tommi@chromium.org magjed@chromium.org
reaching out to  tommi@ and magjed@ from Webrtc to check if they can help us get this issue checked.
Updating on something I tired; 
In Gentoo the default is to use system-ffmpeg.
I changed it to use the bundled ffmpeg, to see if this helps.
Sadly this is not helping and I got the same results. This means that the
media codecs are not the issue.
I don't think this is a webcam issue since vpx_codec_destroy is part of the stack trace.

doronf.public@ - Can you confirm this by only trying the webcam with e.g. https://webrtc.github.io/samples/src/content/getusermedia/gum/?

You should probably add someone that works with video codecs here. If someone can reproduce, we should be able to bisect this.
Components: Internals>Media>Codecs
cc'ing dalecurtis@ and 	hubbe@ to get more insights on Video codecs.
Cc: jzern@chromium.org fgalligan@chromium.org
+vpx folk
Confirming magjed's claim using-
https://webrtc.github.io/samples/src/content/getusermedia/gum/?

(Attached screenshot).
webrtc_20161123_100019.png
586 KB View Download

Comment 15 by jzern@chromium.org, Nov 23 2016

It's hard to say whether this is a vpx issue, a webrtc one or something else without a more complete stacktrace. The destroy call will crash if an uninitialized (non-zeroed) codec context is passed to it for instance.

Comment 16 by tommi@chromium.org, Nov 23 2016

There should be an option named "Automatically send usage statistics and crash reports to Google" that you'd need to enable for crash dumps to work.  However, could it be that you're not actually using Google Chrome, but rather Chromium?
@tommi I added a screenshot (PNG) to show I do not have the option.
And yes I am using Chromium which worked well until (including) chromium-53.0.2785.143. Starting chromium-54.0.2840.59 this issue showed up.

Comment 18 by tommi@chromium.org, Nov 24 2016

Since we don't have symbols or even an offset from vpx_codec_destroy, we don't know much about the crash. vpx_codec_destroy might not be on the stack at all but shows up there due to being the closest public export (I'm theorizing). We don't distribute or support chromium builds so I don't suppose you could try the same using chrome? 
If you can tell me how to add symbols, I'll recompile chromium and give you the information.

As for Chrome, I was able to install  www-client/google-chrome version 54.0.2840.100 (64-bit) . This is the binary distribution which comes with Flash and everything else bundled. I can confirm that the issue did NOT reproduce on Chrome.

However I'd still like to find the problem in Chromium. What would be the next step?

Comment 20 by tommi@chromium.org, Nov 24 2016

Interesting.  Well, I guess attaching a debugger would be the next step.  Also make sure you run Chromium from the command line in case you can get some log entries that can help.
doronf.public@ Is there any latest update available on this issue? Could you please respond for the comment #20 .

Thanks!
Hi,
I attached gdb and attached is the output I got.
The main thing I noticed is:
0x00007fbd9cf91f12 in vpx_codec_destroy () from /usr/lib64/libvpx.so.3

I'm now recompiling chromium with a newer version of libvpx (media-libs/libvpx-1.6.0-r1:0/4::gentoo) and if an API changed it may resolve the issue. I'll check and let you know.

In the meantime if you have any other idea, let me know.
chromium-gdb.log
17.9 KB View Download

Comment 23 by jzern@chromium.org, Nov 28 2016

What version is your system libvpx? There have been fixes around shutdown when dealing with threading, most that come to mind are in the decoder when dealing with corrupted data, though.
If you can install the debug symbols for your system lib you should be able to get a better backtrace.
@jzern I was using libvpx-1.5.0 and moved to libvpx-1.6.0-r1 with the hope
it'll resolve the issue, but sadly it did not.

I'll look into adding debug symbols for a better trace and update.
Hi,
It took me a while to recompile Chromium with the symbols.

I'm now running it with:
chromium --disable-extensions --disable-plugins --no-sandbox --renderer-cmd-prefix="xterm -e gdb -ex run --args"

and I can see the symbols are being used:
Reading symbols from /usr/lib64/chromium-browser/chrome...Reading symbols from /usr/lib64/debug//usr/lib64/chromium-browser/chrome.debug...done.
done.

Now the backtrace produces this:
(gdb) backtrace
#0  vpx_codec_destroy (ctx=0x333501307680) at /usr/src/debug/media-libs/libvpx-1.6.0-r1/libvpx-1.6.0/vpx/src/vpx_codec.c:91
#1  0x000055555b8edea2 in webrtc::VP8DecoderImpl::Release() ()
#2  0x000055555b8d87d1 in webrtc::VCMCodecDataBase::ReleaseDecoder(webrtc::VCMGenericDecoder*) const ()
#3  0x000055555b8d910b in webrtc::VCMCodecDataBase::CreateAndInitDecoder(webrtc::VCMEncodedFrame const&, webrtc::VideoCodec*) const ()
#4  0x000055555b8d91e0 in webrtc::VCMCodecDataBase::GetDecoder(webrtc::VCMEncodedFrame const&, webrtc::VCMDecodedFrameCallback*) ()
#5  0x000055555b8dd194 in webrtc::vcm::VideoReceiver::Decode(webrtc::VCMEncodedFrame const&) ()
#6  0x000055555b8dd5a6 in webrtc::vcm::VideoReceiver::Decode(unsigned short) ()
#7  0x000055555b8bf185 in webrtc::internal::VideoReceiveStream::DecodeThreadFunction(void*) ()
#8  0x000055555b0360b5 in rtc::PlatformThread::Run() ()
#9  0x000055555b0360d9 in rtc::PlatformThread::StartThread(void*) ()
#10 0x00007ffff7bc6494 in start_thread () from /lib64/libpthread.so.0
#11 0x00007fffedcd25dd in clone () from /lib64/libc.so.6

I'm attaching the full log of the gdb session. Please take a look and let me know if something else is needed.
chromium-gdb2.log
5.1 KB View Download
Owner: mflodman@chromium.org
Status: Assigned (was: Unconfirmed)
Magnus can you take a look at this?

From what it looks like, here's what's happening.

1. We're inside VCMCodecDataBase::CreateAndInitDecoder.
2. The call to VP8DecoderImpl::InitDecode fails.
  - Here, I'm wondering if this part is buggy.  (really hope this isn't supposed to be reference counting!)

int VP8DecoderImpl::InitDecode(const VideoCodec* inst, int number_of_cores) {
  int ret_val = Release();  <--- ???
  if (ret_val < 0) {
    return ret_val;
  }
  ...

3. Release() gets called again when InitDecode fails and that's where we crash.


vpx_codec_destroy could be hitting this case:

  if (!ctx)
    res = VPX_CODEC_INVALID_PARAM;
  else if (!ctx->iface || !ctx->priv)
    res = VPX_CODEC_ERROR;  <---  Here.
  ...


Another thing, and perhaps more likely, is that vpx_codec_dec_init is failing inside InitDecode.  In that case, we've allocated a |decoder_| instance, failed to initialize it but don't free the |decoder_|.  VP8DecoderImpl::Release() will try to use that object, but it's not in a known, reliable state.
Yes, the second case you describe seems to be the issue. From looking at the code, it's not safe to call vpx_codec_destroy if vpx_codec_dec_init failed, because the |decoder_| memory will be uninitialized.

I think we should do this change:
   if (vpx_codec_dec_init(decoder_, vpx_codec_vp8_dx(), &cfg, flags)) {
+    delete decoder_;
+    decoder_ = nullptr;
     return WEBRTC_VIDEO_CODEC_MEMORY;
   }

We should probably also do this, just in case:
   if (decoder_ == NULL) {
     decoder_ = new vpx_codec_ctx_t;
+    memset(decoder_, 0, sizeof(*decoder_));
   }
Cc: mflodman@chromium.org
Owner: magjed@chromium.org
Thanks for fixing this so quickly magjed!
Project Member

Comment 29 by bugdroid1@chromium.org, Dec 2 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/external/webrtc.git/+/5c71166dffb49510092bf87f271b6ea590acd1f1

commit 5c71166dffb49510092bf87f271b6ea590acd1f1
Author: magjed <magjed@webrtc.org>
Date: Fri Dec 02 10:46:18 2016

VP8DecoderImpl: Fix uninitialized memory crash

It is not safe to call vpx_codec_destroy if vpx_codec_dec_init failed,
because the |decoder_| memory will be uninitialized. See the bug for
more info.

BUG= chromium:663293 

Review-Url: https://codereview.webrtc.org/2541163007
Cr-Commit-Position: refs/heads/master@{#15381}

[modify] https://crrev.com/5c71166dffb49510092bf87f271b6ea590acd1f1/webrtc/modules/video_coding/codecs/vp8/vp8_impl.cc

Project Member

Comment 30 by bugdroid1@chromium.org, Dec 2 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/external/webrtc.git/+/5c71166dffb49510092bf87f271b6ea590acd1f1

commit 5c71166dffb49510092bf87f271b6ea590acd1f1
Author: magjed <magjed@webrtc.org>
Date: Fri Dec 02 10:46:18 2016

VP8DecoderImpl: Fix uninitialized memory crash

It is not safe to call vpx_codec_destroy if vpx_codec_dec_init failed,
because the |decoder_| memory will be uninitialized. See the bug for
more info.

BUG= chromium:663293 

Review-Url: https://codereview.webrtc.org/2541163007
Cr-Commit-Position: refs/heads/master@{#15381}

[modify] https://crrev.com/5c71166dffb49510092bf87f271b6ea590acd1f1/webrtc/modules/video_coding/codecs/vp8/vp8_impl.cc

Hi guys,
just wanted to let you know I manually applied the patch on 55.0.2883.75 (the code is not the same), and the fix works.

How can I know which release will include this fix?
Labels: Merge-Request-56
Status: Verified (was: Assigned)
Thanks for verifying the fix. It will be included in M57 by default. The fix is simple and safe, so I think we should merge it to M56 if this is an important issue.

Comment 33 by dimu@chromium.org, Dec 13 2016

Labels: -Merge-Request-56 Merge-Approved-56 Hotlist-Merge-Approved
Your change meets the bar and is auto-approved for M56 (branch: 2924)
Project Member

Comment 34 by bugdroid1@chromium.org, Dec 13 2016

Labels: merge-merged-56
The following revision refers to this bug:
  https://chromium.googlesource.com/external/webrtc.git/+/ac061a9ab6ca6690acb6284c1ee34860927acfd7

commit ac061a9ab6ca6690acb6284c1ee34860927acfd7
Author: Magnus Jedvert <magjed@webrtc.org>
Date: Tue Dec 13 10:53:26 2016

VP8DecoderImpl: Fix uninitialized memory crash

It is not safe to call vpx_codec_destroy if vpx_codec_dec_init failed,
because the |decoder_| memory will be uninitialized. See the bug for
more info.

BUG= chromium:663293 
TBR=mflodman,tommi

Review-Url: https://codereview.webrtc.org/2566383003 .
Cr-Commit-Position: refs/branch-heads/56@{#7}
Cr-Branched-From: 613152af11d6e9a4d046af3c48a7be7642dfcc68-refs/heads/master@{#15101}

[modify] https://crrev.com/ac061a9ab6ca6690acb6284c1ee34860927acfd7/webrtc/modules/video_coding/codecs/vp8/vp8_impl.cc

Labels: -Hotlist-Merge-Approved -Merge-Approved-56

Sign in to add a comment