Call from chrome 58 to ios Spark client leads to no Media
Reported by
arung...@cisco.com,
May 12 2017
|
||||||||||||||||||||||||||||||||||
Issue descriptionUserAgent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.47 Safari/537.36 Steps to reproduce the problem: 1. Login with http://web.ciscospark.com 2. download spark Ios client (https://itunes.apple.com/us/app/cisco-spark/id833967564?mt=8) 3. Make a call from the spark web client to ios client What is the expected behavior? There should be media flowing between both the clients (Audio and video) When software encoding is enabled on the ios custom client it starts working What went wrong? There is a decode error on the webrtc internals and media doesnt flow between them Did this work before? Yes chrome 57 Chrome version: >58 Channel: stable OS Version: OS X 10.12.4 Flash Version: Error Logs on the Chrome side [83772:52547:0410/111132.303309:ERROR:rtc_video_decoder.cc(183)] Decoding error occurred. [83772:52547:0410/111132.303392:ERROR:h264_decoder_impl.cc(330)] avcodec_decode_video2 error: -1094995529 [83772:52547:0410/111132.342950:ERROR:h264_decoder_impl.cc(330)] avcodec_decode_video2 error: -1094995529
,
May 12 2017
,
May 12 2017
,
May 12 2017
,
May 12 2017
So both the HW decoder and the SW fallback decoder fail, which indicates there is something wrong with the stream coming to the decoder. According to http://stackoverflow.com/questions/12780931/ffmpeg-exit-status-1094995529, the error 1094995529 means invalid data. Philip: do you think this could be missing SPS/PPS?
,
May 12 2017
It works on chrome 57 and firefox tho.
,
May 12 2017
,
May 18 2017
To login with http://web.ciscospark.com or on app, you have to add the email ID . It would send you a activation mail click it add a new password and use it to login on your ios app as well ..
,
May 18 2017
magjed@, can you do a quick comparative test with AppRTC to see what happens there between iOS and Chrome? This is what I see when I repro: [1:118:0518/145651.529178:ERROR:h264_decoder_impl.cc(330)] avcodec_decode_video2 error: -1094995529 [1:118:0518/145651.556927:ERROR:h264_decoder_impl.cc(330)] avcodec_decode_video2 error: -1094995529 [1:118:0518/145651.588681:ERROR:h264_decoder_impl.cc(330)] avcodec_decode_video2 error: -1094995529 [1:118:0518/145651.628411:ERROR:h264_decoder_impl.cc(330)] avcodec_decode_video2 error: -1094995529
,
May 19 2017
Attaching an rtp dump (SSRC 641303390), that doesn't play with video_replay either: [h264 @ 0x7fdc4c000b40] decode_slice_header error [h264 @ 0x7fdc4c000b40] no frame! [h264 @ 0x7fdc4c000b40] non-existing PPS 0 referenced [h264 @ 0x7fdc4c000b40] decode_slice_header error [h264 @ 0x7fdc4c000b40] no frame! [h264 @ 0x7fdc4c000b40] non-existing PPS 0 referenced ... The encoded bitstream doesn't play with VLC or ffmpeg either.
,
May 23 2017
What exactly am I being asked to do here? Test a H264 video call between AppRTCMobile on iOS and https://appr.tc/ on Chrome Mac? That works without errors as far as I know.
,
May 23 2017
Yep with hardware acceleration (Hardware encoding and decoding) turned on on AppRTCMobile
,
May 23 2017
Yes please double check that it still work for Chrome 58.
,
May 24 2017
I checked and it still works. It was from an iOS device using HW to Chrome 58 on Mac.
,
May 24 2017
,
May 24 2017
Niklas, how did you capture/convert the .rtp/.h264 files in c#10? The .rtp dump starts with FU-A (non-IDR slice NAL) while the .h264 starts with SPS (perhaps truncated) then what looks like corrupted PPS (perhaps truncated) then IDR. Do you have the raw .pcap available (and SRTP key to decrypt)?
,
May 24 2017
Attached, key: iDRdqDQMfKEh1xlk0K9LvO/gNj3a0ldicEcMAONB
,
May 25 2017
Attaching decrypted rtp.pcapng using https://github.com/gteissier/srtp-decrypt H.264 RTP looks good. Starts with STAP-A containing SPS+PPS, then FU-A IDR. So your playback issues are due to bad tools not supporting STAP-A. Chrome supports SPS+PPS in STAP-A, right? SPS has no VUI. I recall an ancient issue, maybe 2 years ago, where Chromebook could not decode H.264 streams without VUI, because it specifically required num_reorder_frames and max_dec_frame_buffering. Is that still an issue? Note that many/most encoders don't include these optional parameters, often no VUI at all. So Chrome should not depend on them. Did something change from 57 to 58 to make it more sensitive to VUI parameters like num_reorder_frames / max_dec_frame_buffering?
,
May 25 2017
We'll take a look, what profile parameter are you using for VideoToolbox in the Spark client?
,
May 25 2017
Baseline Profile Level 3.1 kVTProfileLevel_H264_Baseline_3_1 SPS profile-level=42001F Does Chrome require Constraint Set 0/1 Flags (middle byte of profile-level) to be set to indicate Constrained Baseline Profile? For the best possible interop, it should never reject streams based on those flags.
,
May 26 2017
Philip and Stefan, what do you make of this? I can't play this with video_replay, but I don't get any errors or warnings either (I enable logging in video_replay). Attaching another example that should start with a keyframe (SSRC:1486094762 )
,
May 27 2017
Looks like missing VUI in SPS is the problem. With VUI, chrome decodes and renders. Without VUI, no video is rendered. This was working in 57, so must be some regression in the SPS VUI rewriter or related code. I also see some changes for POC Type setting some reodering defaults which may need a closer look.
,
May 27 2017
This is what Chrome should do for the best interop. If received SPS has no VUI buffering/timing info, and profile=0x42 (baseline), then set default buffering/timing info to: - max_dec_frame_buffering = max_num_ref_frames - max_num_reorder_frames = 0 - Do this regardless of pic_order_cnt_type (POC Type). The H.264 standard specifies the defaults should be 16, but the above is best for real-world interop. Also: - Ignore all Constraint Set Flags in SPS and SDP. - Verify STAP-A and FU-A parsing.
,
May 30 2017
I'm not aware that vui rewriting has changed since M57, but maybe I'm not looking at the right place. Niklas, when running video_replay (./out/Default/video_replay -codec H264 -input_file=sparkiOS_decr2.rtp --payload_type=100 --ssrc=1486094762) on this file I get a number of errors such as: [h264 @ 0x7fc7f0000b80] Invalid NAL unit 0, skipping. [h264 @ 0x7fc7f0000b80] Overread VUI by 8 bits [h264 @ 0x7fc7f0000b80] non-existing PPS 0 referenced [h264 @ 0x7fc7f0000b80] decode_slice_header error [h264 @ 0x7fc7f0000b80] no frame! (h264_decoder_impl.cc:330): avcodec_decode_video2 error: -1094995529 (generic_decoder.cc:173): Failed to decode frame with timestamp 1695797664, error code: -1 Seems like it could be related to VUI. Philip, have you seen issues like these before?
,
May 30 2017
Re #25, I was running with the -decoder_bitstream_filename flag, and I now realize that this flag skips the decoding all together. I can repro the decoding errors without that flag.
,
May 31 2017
This problem is directly caused by the new jitter buffer. video_replay works fine with https://chromium.googlesource.com/external/webrtc/+/27378f39ced81acb1c2a61808e5e42fcf65d4b8d but breaks with https://chromium.googlesource.com/external/webrtc/+/e5bd70223d603c99ef4043313c523d7276330a8e
,
May 31 2017
Also works fine with https://chromium.googlesource.com/external/webrtc/+/8c61924b5671b568a1953820961e4ec24e19e4c6 that was a H264 change right before the jitter buffer switch.
,
Jun 1 2017
after analyzing the pcap file for the IOs call it looks like chrome doesn't establish audio connection with the IOS device . I can only see one outgoing RTP connection which is for video , there is no audio connection.
,
Jun 2 2017
Philip, we should test this rtp dump with the reordering fix and see if that solves the problem.
,
Jun 12 2017
,
Jun 13 2017
Philip, I took a look and found: Received: 5377 StapA received Received PPS: 0, 0 Create frame with first seq: 5377, ts: 1695797664 Received: 5390 Decode: 1695797664 Received FuA referring to PPS: 0 Insert to jb (codec_database.cc:507): Initializing decoder with payload type '100'. Received: 5391 Insert to jb [h264 @ 0x7fbc30000b80] Invalid NAL unit 0, skipping. [h264 @ 0x7fbc30000b80] Overread VUI by 8 bits [h264 @ 0x7fbc30000b80] non-existing PPS 0 referenced [h264 @ 0x7fbc30000b80] decode_slice_header error [h264 @ 0x7fbc30000b80] no frame! The first NAL unit of this frame should be the PPS, but for some reason we mess it up. Could something go wrong in sps_pps_tracker.cc? All following frames fail because of this.
,
Jun 13 2017
I think I have found what is causing this error. When we actually do rewrite the VUI in RtpDepacketizerH264::ProcessStapAOrSingleNalu we do not update |nalu.size| with the new size. Later in H264SpsPpsTracker::CopyAndFixBitstream we save the SPS bitstream, but not all of it since it's length is incorrect.
,
Jun 13 2017
Thanks, that sounds reasonable to me given the errors we get from the decoder. It also fits with how we changed the code between 57 and 58, since we now copy those NAL units and store them, so if we copy the wrong length we'll end up with broken NAL units. Previously the incorrect length wasn't really used.
,
Jun 15 2017
Can we get a fix to test?
,
Jun 16 2017
We're still working on it unfortunately. It turned out to be a bit complicated, but Philip is on it. Hopefully Monday.
,
Jun 21 2017
The following revision refers to this bug: https://chromium.googlesource.com/external/webrtc.git/+/83c97da5931956ae8596d2efcfdea282fdc86d2d commit 83c97da5931956ae8596d2efcfdea282fdc86d2d Author: philipel <philipel@webrtc.org> Date: Wed Jun 21 14:22:40 2017 Only append SPS/PPS to bitstream if supplied out of band. BUG= chromium:721597 Review-Url: https://codereview.webrtc.org/2945853002 Cr-Commit-Position: refs/heads/master@{#18701} [modify] https://crrev.com/83c97da5931956ae8596d2efcfdea282fdc86d2d/webrtc/modules/rtp_rtcp/source/rtp_format_h264.cc [modify] https://crrev.com/83c97da5931956ae8596d2efcfdea282fdc86d2d/webrtc/modules/video_coding/codecs/h264/include/h264_globals.h [modify] https://crrev.com/83c97da5931956ae8596d2efcfdea282fdc86d2d/webrtc/modules/video_coding/h264_sps_pps_tracker.cc [modify] https://crrev.com/83c97da5931956ae8596d2efcfdea282fdc86d2d/webrtc/modules/video_coding/h264_sps_pps_tracker_unittest.cc [modify] https://crrev.com/83c97da5931956ae8596d2efcfdea282fdc86d2d/webrtc/video/rtp_video_stream_receiver_unittest.cc
,
Jun 21 2017
The following revision refers to this bug: https://chromium.googlesource.com/external/webrtc.git/+/83c97da5931956ae8596d2efcfdea282fdc86d2d commit 83c97da5931956ae8596d2efcfdea282fdc86d2d Author: philipel <philipel@webrtc.org> Date: Wed Jun 21 14:22:40 2017 Only append SPS/PPS to bitstream if supplied out of band. BUG= chromium:721597 Review-Url: https://codereview.webrtc.org/2945853002 Cr-Commit-Position: refs/heads/master@{#18701} [modify] https://crrev.com/83c97da5931956ae8596d2efcfdea282fdc86d2d/webrtc/modules/rtp_rtcp/source/rtp_format_h264.cc [modify] https://crrev.com/83c97da5931956ae8596d2efcfdea282fdc86d2d/webrtc/modules/video_coding/codecs/h264/include/h264_globals.h [modify] https://crrev.com/83c97da5931956ae8596d2efcfdea282fdc86d2d/webrtc/modules/video_coding/h264_sps_pps_tracker.cc [modify] https://crrev.com/83c97da5931956ae8596d2efcfdea282fdc86d2d/webrtc/modules/video_coding/h264_sps_pps_tracker_unittest.cc [modify] https://crrev.com/83c97da5931956ae8596d2efcfdea282fdc86d2d/webrtc/video/rtp_video_stream_receiver_unittest.cc
,
Jun 21 2017
This fix solves the problem for me when doing video replay, will try on Chrome when available.
,
Jun 22 2017
The fix missed today's canary, but should be in tomorrow's.
,
Jun 23 2017
We believe this should now be fixed in Canary. Let us know if that is not the case.
,
Jun 29 2017
Verified on Chrome. Since this is a regression, what do you think about merging this to 60? Is this fix depending on other non-merged changes?
,
Jun 30 2017
,
Jun 30 2017
This bug requires manual review: M60 has already been promoted to the beta branch, so this requires manual review Please contact the milestone owner if you have questions. Owners: amineer@(Android), cmasso@(iOS), josafat@(ChromeOS), bustamante@(Desktop) For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Jun 30 2017
This bug meets the bar for merge to M60. Approving merge to branch 3112.
,
Jul 3 2017
,
Jul 3 2017
The following revision refers to this bug: https://chromium.googlesource.com/external/webrtc.git/+/5bcbe1f6c496cf1d1f9dcc3a1c39ac0f302f17ed commit 5bcbe1f6c496cf1d1f9dcc3a1c39ac0f302f17ed Author: philipel <philipel@webrtc.org> Date: Mon Jul 03 13:10:19 2017 Only append SPS/PPS to bitstream if supplied out of band. BUG= chromium:721597 R=holmer@google.com, stefan@webrtc.org Review-Url: https://codereview.webrtc.org/2945853002 Cr-Original-Commit-Position: refs/heads/master@{#18701} Review-Url: https://codereview.webrtc.org/2969863002 . Cr-Commit-Position: refs/branch-heads/60@{#6} Cr-Branched-From: c61bf947b4ac31f3500858ffcae6fee39d799930-refs/heads/master@{#18252} [modify] https://crrev.com/5bcbe1f6c496cf1d1f9dcc3a1c39ac0f302f17ed/webrtc/modules/rtp_rtcp/source/rtp_format_h264.cc [modify] https://crrev.com/5bcbe1f6c496cf1d1f9dcc3a1c39ac0f302f17ed/webrtc/modules/video_coding/codecs/h264/include/h264_globals.h [modify] https://crrev.com/5bcbe1f6c496cf1d1f9dcc3a1c39ac0f302f17ed/webrtc/modules/video_coding/h264_sps_pps_tracker.cc [modify] https://crrev.com/5bcbe1f6c496cf1d1f9dcc3a1c39ac0f302f17ed/webrtc/modules/video_coding/h264_sps_pps_tracker_unittest.cc [modify] https://crrev.com/5bcbe1f6c496cf1d1f9dcc3a1c39ac0f302f17ed/webrtc/video/rtp_stream_receiver_unittest.cc |
||||||||||||||||||||||||||||||||||
►
Sign in to add a comment |
||||||||||||||||||||||||||||||||||
Comment 1 by brajkumar@chromium.org
, May 12 2017