Able to reproduce this on 54.0.2800.2/8619.0.0 (Official Build) dev-channel peach_pit.

Stack trace of 7a038f0e00000000 

Thread 0 CRASHED [SIGSEGV @ 0xa8d9b000 ] MAGIC SIGNATURE THREAD
0xb51b051c	(chrome + 0x02d8951c )	CopyRow_NEON
0xb2e15205	(chrome + 0x009ee205 )	CopyPlane
0xb2e1005f	(chrome + 0x009e905f )	I420Copy
0xb4409333	(chrome + 0x01fe2333 )	content::RTCVideoEncoder::Impl::EncodeOneFrame()
0xb44082fd	(chrome + 0x01fe12fd )	base::internal::Invoker<base::internal::BindState<void (content::RTCVideoEncoder::Impl::*)(webrtc::VideoFrame const*, bool, base::WaitableEvent*, int*), scoped_refptr<content::RTCVideoEncoder::Impl>, webrtc::VideoFrame const*, bool, base::WaitableEvent*, int*>, void ()>::Run(base::internal::BindStateBase*)
0xb2b0697b	(chrome + 0x006df97b )	base::debug::TaskAnnotator::RunTask(char const*, base::PendingTask const&)
0xb2afbdc7	(chrome + 0x006d4dc7 )	base::MessageLoop::DoWork()
0xb2afc299	(chrome + 0x006d5299 )	base::MessagePumpDefault::Run(base::MessagePump::Delegate*)
0xb4788a4d	(chrome + 0x02361a4d )	base::RunLoop::Run()
0xb4775007	(chrome + 0x0234e007 )	base::MessageLoop::Run()
0xb479a187	(chrome + 0x02373187 )	base::Thread::ThreadMain()
0xb4798a5d	(chrome + 0x02371a5d )	base::(anonymous namespace)::ThreadFunc(void*)
0xb23f66b9	(libpthread-2.19.so -pthread_create.c:311 )	start_thread
0xb1e0365b	(libc-2.19.so + 0x0009e65b )	clone

Is this only seen on peach-pit?

Albert do you know who would be the best owner for this?

RTC video decode crash. posciak@ can you route further?

jansson@ could someone from webrtc test team check if this is a recent regression?

Not reproduce the issue on 52.0.02743.85/8350.60.0 - Pit
Gmail is working fine while doing hangout video call

I don't see anything here (crashes in the DB etc.) that indicates the crash is happening in M52. It looks only happening in 54.0.2800.2.

 sc00335628@: please give the steps that repro'd this in beta 52.0.02743.85/8350.60.0 because we cant reproduce it.
Marking this as non-stable blocker for M52. Please reopen if you have crash logs for M52.   

In response to Comment#9 : Checked the issue on 52.0.02743.85/8350.60.0 beta of Pit. Now am no longer able to reproduce it.

How frequently is this happening on M54?

I tried 8634.0.0 and it is always reproducible.

I could not reproduce the crash by visiting apprtc.appspot.com/?debug=loopback&vsc=VP8.

I enabled the logs in rtc_video_encoder.cc and v4l2_video_encoder_accelerator.cc. Here are the logs.

Deleted: chrome 60.1 KB

chrome 60.1 KB View Download

Deleted: ui.LATEST 8.0 KB

ui.LATEST 8.0 KB Download

Deleted: messages 1.0 MB

messages 1.0 MB View Download

I disabled the logs in v4l2_video_encoder_accelerator.cc. It looks the size of the first input frame is incorrect. RtcVideoEncoder was initialized with 320x180 and the first frame was 1280x720.


[1:39:0727/165407:VERBOSE1:rtc_video_encoder.cc(762)] RTCVideoEncoder(): codec type=0
[1:39:0727/165407:VERBOSE1:rtc_video_encoder.cc(774)] InitEncode(): codecType=0, width=176, height=144, startBitrate=200
[1:21:0727/165407:VERBOSE3:rtc_video_encoder.cc(301)] Impl::CreateAndInitializeVEA()
[1:21:0727/165407:VERBOSE3:rtc_video_encoder.cc(426)] Impl::RequireBitstreamBuffers(): input_count=2, input_coded_size=176x144, output_buffer_size=2097152
[1:39:0727/165407:VERBOSE3:rtc_video_encoder.cc(845)] RegisterEncodeCompleteCallback()
[1:21:0727/165407:VERBOSE3:rtc_video_encoder.cc(692)] RegisterEncodeCompleteCallback()
[1:39:0727/165407:VERBOSE3:rtc_video_encoder.cc(766)] ~RTCVideoEncoder
[1:39:0727/165407:VERBOSE3:rtc_video_encoder.cc(864)] Release()
[1:21:0727/165407:VERBOSE3:rtc_video_encoder.cc(403)] Impl::Destroy()
[1:39:0727/165407:VERBOSE1:rtc_video_encoder.cc(762)] RTCVideoEncoder(): codec type=0
[1:39:0727/165407:VERBOSE1:rtc_video_encoder.cc(774)] InitEncode(): codecType=0, width=320, height=180, startBitrate=150
[1:21:0727/165407:VERBOSE3:rtc_video_encoder.cc(301)] Impl::CreateAndInitializeVEA()
[1:21:0727/165407:VERBOSE3:rtc_video_encoder.cc(426)] Impl::RequireBitstreamBuffers(): input_count=2, input_coded_size=320x192, output_buffer_size=2097152
[1:39:0727/165407:VERBOSE3:rtc_video_encoder.cc(845)] RegisterEncodeCompleteCallback()
[1:21:0727/165407:VERBOSE3:rtc_video_encoder.cc(692)] RegisterEncodeCompleteCallback()
[1:39:0727/165407:VERBOSE1:rtc_video_encoder.cc(762)] RTCVideoEncoder(): codec type=0
[1:39:0727/165407:VERBOSE1:rtc_video_encoder.cc(774)] InitEncode(): codecType=0, width=640, height=360, startBitrate=150
[1:21:0727/165407:VERBOSE3:rtc_video_encoder.cc(301)] Impl::CreateAndInitializeVEA()
[1:21:0727/165407:VERBOSE3:rtc_video_encoder.cc(475)] Impl::BitstreamBufferReady(): bitstream_buffer_id=0, payload_size=0, key_frame=0, timestamp ms=0
[1:21:0727/165407:VERBOSE3:rtc_video_encoder.cc(705)] ReturnEncodedImage(): bitstream_buffer_id=0, picture_id=15106
[1:21:0727/165407:VERBOSE2:rtc_video_encoder.cc(749)] ReturnEncodedImage(): encoded_image_callback_ returned -1
[1:21:0727/165407:VERBOSE3:rtc_video_encoder.cc(375)] Impl::UseOutputBitstreamBufferIndex(): bitstream_buffer_id=0
[1:21:0727/165407:VERBOSE3:rtc_video_encoder.cc(426)] Impl::RequireBitstreamBuffers(): input_count=2, input_coded_size=640x368, output_buffer_size=2097152
[1:39:0727/165407:VERBOSE3:rtc_video_encoder.cc(845)] RegisterEncodeCompleteCallback()
[1:21:0727/165407:VERBOSE3:rtc_video_encoder.cc(692)] RegisterEncodeCompleteCallback()
[1:39:0727/165407:VERBOSE1:rtc_video_encoder.cc(762)] RTCVideoEncoder(): codec type=0
[1:39:0727/165407:VERBOSE1:rtc_video_encoder.cc(774)] InitEncode(): codecType=0, width=1280, height=720, startBitrate=700
[1:21:0727/165407:VERBOSE3:rtc_video_encoder.cc(301)] Impl::CreateAndInitializeVEA()
[1:21:0727/165407:VERBOSE3:rtc_video_encoder.cc(475)] Impl::BitstreamBufferReady(): bitstream_buffer_id=0, payload_size=0, key_frame=0, timestamp ms=0
[1:21:0727/165407:VERBOSE3:rtc_video_encoder.cc(705)] ReturnEncodedImage(): bitstream_buffer_id=0, picture_id=26490
[1:21:0727/165407:VERBOSE2:rtc_video_encoder.cc(749)] ReturnEncodedImage(): encoded_image_callback_ returned -1
[1:21:0727/165407:VERBOSE3:rtc_video_encoder.cc(375)] Impl::UseOutputBitstreamBufferIndex(): bitstream_buffer_id=0
[1:21:0727/165407:VERBOSE3:rtc_video_encoder.cc(426)] Impl::RequireBitstreamBuffers(): input_count=2, input_coded_size=1280x720, output_buffer_size=2097152
[1:39:0727/165407:VERBOSE3:rtc_video_encoder.cc(845)] RegisterEncodeCompleteCallback()
[1:21:0727/165407:VERBOSE3:rtc_video_encoder.cc(692)] RegisterEncodeCompleteCallback()
[1:39:0727/165407:VERBOSE3:rtc_video_encoder.cc(888)] SetRates(): new_bit_rate=150, frame_rate=60
[1:21:0727/165407:VERBOSE3:rtc_video_encoder.cc(390)] Impl::RequestEncodingParametersChange(): bitrate=150, framerate=60
[1:39:0727/165407:VERBOSE3:rtc_video_encoder.cc(888)] SetRates(): new_bit_rate=150, frame_rate=60
[1:21:0727/165407:VERBOSE3:rtc_video_encoder.cc(390)] Impl::RequestEncodingParametersChange(): bitrate=150, framerate=60
[1:39:0727/165407:VERBOSE3:rtc_video_encoder.cc(888)] SetRates(): new_bit_rate=700, frame_rate=60
[1:21:0727/165407:VERBOSE3:rtc_video_encoder.cc(390)] Impl::RequestEncodingParametersChange(): bitrate=700, framerate=60
[1:39:0727/165407:WARNING:rtc_video_encoder.cc(813)] Encodeinput_frame DataY=0x9840a000, DataU=0x984eb000, DataV=0x98523400, StrideY=1280, StrideU=640, StrideV=640, width=1280, height=720
[1:21:0727/165407:VERBOSE3:rtc_video_encoder.cc(332)] Impl::Enqueue()
[1:21:0727/165407:VERBOSE3:rtc_video_encoder.cc(562)] Impl::EncodeOneFrame()
[1:21:0727/165407:WARNING:rtc_video_encoder.cc(610)] Before I420Copy, next_frame DataY=0x9840a000, DataU=0x984eb000, DataV=0x98523400, StrideY=1280, StrideU=640, StrideV=640, width=1280, height=720, input_frame_coded_size_=320x192, input_visible_size_=320x180

Deleted: chrome 17.9 KB

chrome 17.9 KB View Download

Tomas. Can someone in WebRTC team take a look?

I'm pretty sure this CL is the culprit: https://codereview.webrtc.org/2099483002/

That CL causes SimulcastEncoderAdapter to stop scaling frames that have native handles. SimulcastEncoderAdapter will receive WebRtcVideoFrameAdapter frames that do have native handles, and pass them unscaled to the RTCVideoEncoder subencoders which will cause the segfault we see here. That's why the RTCVideoEncoder that was initialized with 320x180 receives 1280x720 frames. That's also why this issue doesn't reproduce with apprtc.appspot.com, because it doesn't use simulcast.

In the culprit CL, there was a discussion if the CL would cause problems for HW encoders, and e.g. this comment by pbos@ is spot on:
"+wuchengli@, I believe RTCVideoEncoder on ChromeOS can use textures. So I think this might break simulcast on ChromeOS?"
with the reply:
"RTCVideoEncoder does not handle textures. The native handle passed to RTCVideoEncoder has to be I420. The default SupportsNativeHandle returns false. RTCVideoEncoder are subencoders. So streaminfo.encoder->SupportsNativeHandle() will return false and this should be fine?"
It's true that RTCVideoEncoder::SupportsNativeHandle() and thereby also SimulcastEncoderAdapter::SupportsNativeHandle() returns false, but that won't help anything in this case. If SupportsNativeHandle() returns false, the VideoSender [0] will call NativeToI420Buffer() to convert the frames to ordinary I420 frames, but WebRtcVideoFrameAdapter will return itself [1] and the resulting buffer will still have a native handle. So SimulcastEncoderAdapter will still receive frames with native handles, and pass them unscaled to RTCVideoEncoder.

I think we should either:
 - Revert https://codereview.webrtc.org/2099483002/
 - Return a buffer in WebRtcVideoFrameAdapter::NativeToI420Buffer() with no native handle.
 - Start scaling the incoming frames in RTCVideoEncoder.

The last option has better performance, because we can avoid the frame copy in RTCVideoEncoder and scale directly to shared memory, but the fix is more complicated than the others.

[0] https://cs.chromium.org/chromium/src/third_party/webrtc/modules/video_coding/video_sender.cc?rcl=0&l=293
[1] https://cs.chromium.org/chromium/src/content/renderer/media/webrtc/webrtc_video_frame_adapter.cc?rcl=1469603944&l=56

Can someone take a look and confirm that the above CL is the culprit?

Assuming it is, the RTCVideoEncoder option sounds like the one we'd like and we need someone to do the implementation (wuchengli?).  If we can avoid reverting the cl in webrtc, then that would be preferable.

I agree with your solution options magjed. With that CL[0], frames are no longer scaled in SimulcastEncoderAdapter. So, we basically give that responsibility to Chromium, and it needs to happen on RTCVideoEncoder, which currently lacking.

I agree that last option would be ideal. It shouldn't complicated as the incoming frames are guaranteed to be CPU mappable with this check[1]. I will work on a fix.

[0] https://codereview.webrtc.org/2099483002/
[1] https://cs.chromium.org/chromium/src/content/renderer/media/webrtc/webrtc_video_frame_adapter.cc?rcl=1469603944&l=56

Any update on this one? 

Waiting for OWNERS on this CL: https://codereview.chromium.org/2182183007/ 

it seems like CL has been lgtm'd, good to be submitted?

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/35ad929a8daded62262cdd91018dc50350483bb3

commit 35ad929a8daded62262cdd91018dc50350483bb3
Author: emircan <emircan@chromium.org>
Date: Tue Aug 02 19:57:20 2016

Handle scaling frames in RTCVideoEncoder

This CL adds support for scaling incoming video frames in RTCVideoEncoder.
Earlier, we expected WebRTC to only send frames that have the same size
as the initialized size. However, after last changes we can have different
sizes.

I replaced libyuv::I420Copy with libyuv::I420Scale, but note that it still
does only a copy when the gven input and output sizes are the same:
https://cs.chromium.org/chromium/src/third_party/libyuv/source/scale.cc?rcl=0&l=1400

BUG= 630577 
TEST=Tested Hangouts call on veyron_jerry. Added RTCVideoEncoderUnittest.

Review-Url: https://codereview.chromium.org/2182183007
Cr-Commit-Position: refs/heads/master@{#409286}

[modify] https://crrev.com/35ad929a8daded62262cdd91018dc50350483bb3/content/content_tests.gypi
[modify] https://crrev.com/35ad929a8daded62262cdd91018dc50350483bb3/content/renderer/media/gpu/rtc_video_encoder.cc
[add] https://crrev.com/35ad929a8daded62262cdd91018dc50350483bb3/content/renderer/media/gpu/rtc_video_encoder_unittest.cc
[modify] https://crrev.com/35ad929a8daded62262cdd91018dc50350483bb3/media/BUILD.gn
[modify] https://crrev.com/35ad929a8daded62262cdd91018dc50350483bb3/media/media.gyp
[modify] https://crrev.com/35ad929a8daded62262cdd91018dc50350483bb3/media/renderers/mock_gpu_video_accelerator_factories.cc
[add] https://crrev.com/35ad929a8daded62262cdd91018dc50350483bb3/media/video/mock_video_encode_accelerator.cc
[add] https://crrev.com/35ad929a8daded62262cdd91018dc50350483bb3/media/video/mock_video_encode_accelerator.h

Verified on build 8743.65.0

Moving old issues out of Internal>Graphics to delete this obsolete component ( crbug.com/685425  for details)