Issue metadata
Sign in to add a comment
|
webrtc: Chrome displaying blank video if H264 stream has SEI NAL packets
Reported by
hitz.inf...@gmail.com,
May 5 2017
|
||||||||||||||||||||||||
Issue description
UserAgent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.96 Safari/537.36
Example URL:
Steps to reproduce the problem:
1. Dial WebRTC call from Chrome to native software client on Windows using Intel Media SDK hardware video encoder
2. Chrome displays blank video while software client receives proper video
[Note: We are trying this in our internal setup]
What is the expected behavior?
Chrome displays remote video
What went wrong?
Chrome doesn't display remote video.
Did this work before? N/A
Is it a problem with Flash or HTML5? N/A
Does this work in other browsers? N/A
Chrome version: 58.0.3029.96 Channel: stable
OS Version: OS X 10.12.2
Flash Version: Shockwave Flash 25.0 r0
Contents of chrome://gpu:
Graphics Feature Status
Canvas: Hardware accelerated
Flash: Hardware accelerated
Flash Stage3D: Hardware accelerated
Flash Stage3D Baseline profile: Hardware accelerated
Compositing: Hardware accelerated
Multiple Raster Threads: Enabled
Native GpuMemoryBuffers: Hardware accelerated
Rasterization: Hardware accelerated
Video Decode: Hardware accelerated
Video Encode: Hardware accelerated
VPx Video Decode: Hardware accelerated
WebGL: Hardware accelerated
WebGL2: Hardware accelerated
Driver Bug Workarounds
add_and_true_to_loop_condition
adjust_src_dst_region_for_blitframebuffer
decode_encode_srgb_for_generatemipmap
disable_framebuffer_cmaa
disable_multimonitor_multisampling
disable_webgl_rgb_multisampling_usage
emulate_abs_int_function
get_frag_data_info_bug
init_two_cube_map_levels_before_copyteximage
msaa_is_slow
pack_parameters_workaround_with_pack_buffer
rebind_transform_feedback_before_resume
regenerate_struct_names
remove_invariant_and_centroid_for_essl3
reset_base_mipmap_level_before_texstorage
rewrite_texelfetchoffset_to_texelfetch
scalarize_vec_and_mat_constructor_args
set_zero_level_before_generating_mipmap
unfold_short_circuit_as_ternary_operation
unpack_alignment_workaround_with_unpack_buffer
unpack_image_height_workaround_with_unpack_buffer
use_intermediary_for_copy_texture_image
use_unused_standard_shared_blocks
Problems Detected
Multisampling is buggy on OSX when multiple monitors are connected: 237931
Applied Workarounds: disable_multimonitor_multisampling
Unfold short circuit on Mac OS X: 307751
Applied Workarounds: unfold_short_circuit_as_ternary_operation
Always rewrite vec/mat constructors to be consistent: 398694
Applied Workarounds: scalarize_vec_and_mat_constructor_args
Mac drivers handle struct scopes incorrectly: 403957
Applied Workarounds: regenerate_struct_names
On Intel GPUs MSAA performance is not acceptable for GPU rasterization: 527565
Applied Workarounds: msaa_is_slow
glGenerateMipmap fails if the zero texture level is not set on some Mac drivers: 560499
Applied Workarounds: set_zero_level_before_generating_mipmap
Pack parameters work incorrectly with pack buffer bound: 563714
Applied Workarounds: pack_parameters_workaround_with_pack_buffer
Alignment works incorrectly with unpack buffer bound: 563714
Applied Workarounds: unpack_alignment_workaround_with_unpack_buffer
copyTexImage2D fails when reading from IOSurface on multiple GPU types.: 581777
Applied Workarounds: use_intermediary_for_copy_texture_image
Multisample renderbuffers with format GL_RGB8 have performance issues on Intel GPUs.: 607130
Applied Workarounds: disable_webgl_rgb_multisampling_usage
Limited enabling of Chromium GL_INTEL_framebuffer_CMAA: 535198
Applied Workarounds: disable_framebuffer_cmaa
glGetFragData{Location|Index} works incorrectly on Max: 638340
Applied Workarounds: get_frag_data_info_bug
glResumeTransformFeedback works incorrectly on Intel GPUs: 638514
Applied Workarounds: rebind_transform_feedback_before_resume
glTexStorage* are buggy when base mipmap level is not 0: 640506
Applied Workarounds: reset_base_mipmap_level_before_texstorage
Result of abs(i) where i is an integer in vertex shader is wrong: 642227
Applied Workarounds: emulate_abs_int_function
Rewrite texelFetchOffset to texelFetch for Intel Mac: 642605
Applied Workarounds: rewrite_texelfetchoffset_to_texelfetch
Rewrite condition in for and while loops for Intel Mac: 644669
Applied Workarounds: add_and_true_to_loop_condition
Decode and encode before generateMipmap for srgb format textures on macosx: 634519
Applied Workarounds: decode_encode_srgb_for_generatemipmap
Init first two levels before CopyTexImage2D for cube map texture on Intel Mac 10.12: 648197
Applied Workarounds: init_two_cube_map_levels_before_copyteximage
Insert statements to reference all members in unused std140/shared blocks on Mac: 618464
Applied Workarounds: use_unused_standard_shared_blocks
Tex(Sub)Image3D performs incorrectly when uploading from unpack buffer with GL_UNPACK_IMAGE_HEIGHT greater than zero on Intel Macs: 654258
Applied Workarounds: unpack_image_height_workaround_with_unpack_buffer
adjust src/dst region if blitting pixels outside read framebuffer on Mac: 644740
Applied Workarounds: adjust_src_dst_region_for_blitframebuffer
Mac driver GL 4.1 requires invariant and centroid to match between shaders: 639760, 641129
Applied Workarounds: remove_invariant_and_centroid_for_essl3
Disable KHR_blend_equation_advanced until cc shaders are updated: 661715
Version Information
Data exported 5/5/2017, 6:17:29 PM
Chrome version Chrome/58.0.3029.96
Operating system Mac OS X 10.12.2
Software rendering list version 12.20
Driver bug list version 9.36
ANGLE commit id 461d9a3060e3
2D graphics backend Skia/58 4c81ba6ba3a3270db809bf7d4c3bc782694a56a4
Command Line Args Chrome.app/Contents/MacOS/Google Chrome --flag-switches-begin --flag-switches-end
Driver Information
Initialization time 67
In-process GPU false
Passthrough Command Decoder false
Sandboxed true
GPU0 VENDOR = 0x8086, DEVICE= 0x0a2e *ACTIVE*
Optimus false
Optimus false
AMD switchable false
Driver vendor
Driver version 10.22.25
Driver date
Pixel shader version 4.10
Vertex shader version 4.10
Max. MSAA samples 8
Machine model name MacBookPro
Machine model version 11.1
GL_VENDOR Intel Inc.
GL_RENDERER Intel Iris OpenGL Engine
GL_VERSION 4.1 INTEL-10.22.25
GL_EXTENSIONS GL_ARB_blend_func_extended GL_ARB_draw_buffers_blend GL_ARB_draw_indirect GL_ARB_ES2_compatibility GL_ARB_explicit_attrib_location GL_ARB_gpu_shader_fp64 GL_ARB_gpu_shader5 GL_ARB_instanced_arrays GL_ARB_internalformat_query GL_ARB_occlusion_query2 GL_ARB_sample_shading GL_ARB_sampler_objects GL_ARB_separate_shader_objects GL_ARB_shader_bit_encoding GL_ARB_shader_subroutine GL_ARB_shading_language_include GL_ARB_tessellation_shader GL_ARB_texture_buffer_object_rgb32 GL_ARB_texture_cube_map_array GL_ARB_texture_gather GL_ARB_texture_query_lod GL_ARB_texture_rgb10_a2ui GL_ARB_texture_storage GL_ARB_texture_swizzle GL_ARB_timer_query GL_ARB_transform_feedback2 GL_ARB_transform_feedback3 GL_ARB_vertex_attrib_64bit GL_ARB_vertex_type_2_10_10_10_rev GL_ARB_viewport_array GL_EXT_debug_label GL_EXT_debug_marker GL_EXT_framebuffer_multisample_blit_scaled GL_EXT_texture_compression_s3tc GL_EXT_texture_filter_anisotropic GL_EXT_texture_sRGB_decode GL_APPLE_client_storage GL_APPLE_container_object_shareable GL_APPLE_flush_render GL_APPLE_object_purgeable GL_APPLE_rgb_422 GL_APPLE_row_bytes GL_APPLE_texture_range GL_ATI_texture_mirror_once GL_NV_texture_barrier
Disabled Extensions GL_KHR_blend_equation_advanced GL_KHR_blend_equation_advanced_coherent
Window system binding vendor
Window system binding version
Window system binding extensions
Direct rendering Yes
Reset notification strategy 0x0000
GPU process crash count 0
Compositor Information
Tile Update Mode Zero-copy
Partial Raster Enabled
GpuMemoryBuffers Status
ATC Software only
ATCIA Software only
DXT1 Software only
DXT5 Software only
ETC1 Software only
R_8 GPU_READ_CPU_READ_WRITE, GPU_READ_CPU_READ_WRITE_PERSISTENT
RG_88 Software only
BGR_565 Software only
RGBA_4444 Software only
RGBX_8888 Software only
RGBA_8888 GPU_READ, SCANOUT, SCANOUT_CPU_READ_WRITE
BGRX_8888 GPU_READ, SCANOUT, SCANOUT_CPU_READ_WRITE
BGRA_8888 GPU_READ, SCANOUT, SCANOUT_CPU_READ_WRITE, GPU_READ_CPU_READ_WRITE, GPU_READ_CPU_READ_WRITE_PERSISTENT
YVU_420 Software only
YUV_420_BIPLANAR GPU_READ_CPU_READ_WRITE, GPU_READ_CPU_READ_WRITE_PERSISTENT
UYVY_422 GPU_READ_CPU_READ_WRITE, GPU_READ_CPU_READ_WRITE_PERSISTENT
We have observed from Chromium logs that when SEI packets are present in received H264 stream, then even the first Keyframe is getting discarded. H264 NALUs have to be in following sequence to reproduce this issue: SPS, PPS, SEI, IDR, SEI, Delta-Frame, SEI, Delta-Frame, SEI, Delta-Frame,...... SPS, PPS, SEI, IDR, Delta-Frame. That is when SEI packets are present with every IDR frame, then Chrome won't display remote video.
We have also observed occasional video freeze issues when H264 NALUs are being received in this sequence: SPS, PPS, IDR, Delta-Frame, Delta-Frame, SPS, PPS, SEI, IDR, Delta-Frame. That is when SEI packets are not present in all of the IDR frames.
Please find attached network trace with RTP packets having H264 stream for the blank video scenario. Also attaching event log taken on Chrome for the call and Chromium logs with additional logging around frames processing.
,
May 9 2017
This issue looks similar to the issue id: 717884. Hence, merging into the issue id: 717884. Please feel free to undupe the same if not the case. Thanks...!!
,
May 9 2017
Issue we are facing is different than reported in 717884, as we don't see errors observed in that ticket. As updated in the ticket description above that for H264 stream which doesn't have SEI NAL packet in first IDR frame, we do see video on Chrome and thereafter freeze issues whenever SEI packets arrive. So video does decode for our H264 stream, it is just the handling of SEI packets in the H264 stream which causes IDR frames to get discarded. We see blank video on Chrome when SEI packets are there with every IDR frame from the beginning. We have been updating https://groups.google.com/forum/#!msg/discuss-webrtc/dt_A8pv_nBU/KEW1-kpCBQAJ forum post for this issue. Comment from Philip in this thread talk about handling of SEI packets and review request https://codereview.chromium.org/2769863007/. Fix we have tried also touches code modified in this review request for handling of SEI packets. Hence requesting to track this as separate issue. Sorry but I don't seem to have permission to unmark this issue as duplicate. Also is Blink>WebRTC>Video right component for this issue? Thanks.
,
May 11 2017
,
May 12 2017
,
May 12 2017
,
May 12 2017
,
May 18 2017
Hi, Any update on this issue? Please let us know if fix for this issue is going to be part of M59 release. Thanks.
,
May 18 2017
Hi, We are working on improving support for H264, hopefully it will be done next week.
,
May 18 2017
Thanks for the update. Hitesh
,
May 19 2017
The following revision refers to this bug: https://chromium.googlesource.com/external/webrtc.git/+/e87c87651f36650e76827671c4ec2a5c436ef5a9 commit e87c87651f36650e76827671c4ec2a5c436ef5a9 Author: philipel <philipel@webrtc.org> Date: Fri May 19 13:38:50 2017 Check H264 NALUs for frametype and insert SPS/PPS packets into the PacketBuffer. BUG= chromium:719095 Review-Url: https://codereview.webrtc.org/2889163003 Cr-Commit-Position: refs/heads/master@{#18214} [modify] https://crrev.com/e87c87651f36650e76827671c4ec2a5c436ef5a9/webrtc/modules/video_coding/frame_object.cc [modify] https://crrev.com/e87c87651f36650e76827671c4ec2a5c436ef5a9/webrtc/modules/video_coding/h264_sps_pps_tracker.cc [modify] https://crrev.com/e87c87651f36650e76827671c4ec2a5c436ef5a9/webrtc/modules/video_coding/h264_sps_pps_tracker_unittest.cc
,
May 23 2017
Hitesh, could you please verify that the fix landed in #11. It is part of canary build 60.0.3108.0 or newer.
,
May 24 2017
I will verify and update. Thanks.
,
May 25 2017
We found fix to be working. Thanks. Blank video issue has been resolved for the scenario when SEI packet comes with every IDR & Delta frame. Video freeze issue also doesn't happen frequently too for the scenario when SEI packet arrives randomly every few seconds. We did observe occasional video freeze (like every few minutes) but not like what it used to be before. We need to do more testing around this, as occasional video freeze could be due to packet loss. We were seeing Aw,snap/crash issue yesterday occasionally with Canary v60.0.3109.0, but no more seeing it today with v60.0.3110.0. Is this fix going to be available as part of M59 beta builds? Regards, Hitesh
,
May 25 2017
Thanks for verifying! Hopefully we'll be able to get this into 59.
,
May 25 2017
This bug requires manual review: We are only 11 days from stable. Please contact the milestone owner if you have questions. Owners: amineer@(Android), cmasso@(iOS), gkihumba@(ChromeOS), Abdul Syed@(Desktop) For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
May 25 2017
Since we are only less than a week way from Stable, can you please confirm following 1)if this is super urgent or if it can wait until M60? 2)Is it a safe merge, verified in canary and dev, and with enough automated test coverage? 3)We are already way past branch point for M59, and our bar is very high for merges.
,
May 25 2017
We need this fix to support basic Video calls with Desktop clients using Intel Media SDK hardware video encoder and improve Video quality with any client which sends SEI NAL packets in H264 video stream. Thanks, Hitesh
,
May 26 2017
I think it would be very good to get this into 59 since many users have been suffering from this in 58. It has been in Canary for some time and been verified by some of the affected users. It should have reached dev by now. hitz.inifitech, could you verify this in dev channel as well? FYI, we'd also have to merge https://bugs.chromium.org/p/chromium/issues/detail?id=725494, which has already been approved.
,
May 26 2017
We are still seeing blank video issue with Dev builds on Windows and Mac with Version 60.0.3107.4 (Official Build) dev (64-bit). Looks like fix is still not available in Dev Channel. Thanks, Hitesh
,
May 26 2017
Based on comment 20, seems like this issue is still occurring. Rejecting M59 merge.
,
May 26 2017
The fix in #11 did not make it into 60.0.3107.4. The WebRTC roll with this fix was made on May 22.
,
May 26 2017
This bug requires manual review: We are only 10 days from stable. Please contact the milestone owner if you have questions. Owners: amineer@(Android), cmasso@(iOS), gkihumba@(ChromeOS), Abdul Syed@(Desktop) For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
May 26 2017
If this has not been tested in Dev yet, I'm not comfortable taking this merge in this late in the cycle, especially with M59 stable next week. My recommendation is to wait until M60.
,
May 30 2017
,
May 30 2017
,
May 30 2017
,
May 30 2017
Per chat, discussed this is a safe merge, well tested and verified, and it is impacting a sizeable number of users. Approving merge for M59.
,
May 30 2017
The following revision refers to this bug: https://chromium.googlesource.com/external/webrtc.git/+/b9ca68003f9dd88241dd2022dc9c077b91bd2ea2 commit b9ca68003f9dd88241dd2022dc9c077b91bd2ea2 Author: philipel <philipel@webrtc.org> Date: Tue May 30 21:12:14 2017 Check H264 NALUs for frametype and insert SPS/PPS packets into the PacketBuffer. BUG= chromium:719095 Review-Url: https://codereview.webrtc.org/2889163003 Cr-Original-Commit-Position: refs/heads/master@{#18214} Review-Url: https://codereview.webrtc.org/2916433003 . Cr-Commit-Position: refs/branch-heads/59@{#14} Cr-Branched-From: 10d095d4f743bc16f8e486e156c48a6d023b32c5-refs/heads/master@{#17657} [modify] https://crrev.com/b9ca68003f9dd88241dd2022dc9c077b91bd2ea2/webrtc/modules/video_coding/frame_object.cc [modify] https://crrev.com/b9ca68003f9dd88241dd2022dc9c077b91bd2ea2/webrtc/modules/video_coding/h264_sps_pps_tracker.cc [modify] https://crrev.com/b9ca68003f9dd88241dd2022dc9c077b91bd2ea2/webrtc/modules/video_coding/h264_sps_pps_tracker_unittest.cc
,
May 30 2017
,
Jun 1 2017
I faced this issue with M58 on linux x64 machine. will https://chromium.googlesource.com/external/webrtc.git/+/b9ca68003f9dd88241dd2022dc9c077b91bd2ea2 work out on H264 blank issue?
,
Jun 1 2017
derriclyns@, If you have a stream where every frame start with a SEI then that CL should fix it.
,
Jun 7 2017
Issue 717884 has been merged into this issue.
,
Jun 13 2017
We have verified the fix with latest Chrome Version 59.0.3071.86 (Official Build) (64-bit). Thanks, Hitesh
,
Jun 14 2017
Setting to Verified based on #35. |
|||||||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||||||
Comment 1 by hitz.inf...@gmail.com
, May 8 20174.1 KB
4.1 KB Download