MEDIA_LOG when AVC IDR slice occurs in a frame, but the container didn't mark the frame as a keyframe
Reported by
jeroen.t...@theoplayer.com,
Jul 5
|
||||||||||
Issue descriptionUserAgent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36 Example URL: https://cdn.theoplayer.com/bugreport/index.html Steps to reproduce the problem: 1. Go to https://cdn.theoplayer.com/bugreport/index.html or open the index.html file in the attached zip file. 2. Click the 2nd or 4th button to trigger the playback issue. To check what scenario each button triggers, please read https://cdn.theoplayer.com/bugreport/README.md or README.md file in the attached zip file. What is the expected behavior? All the cases should result in normal playback. What went wrong? We add certain segments to the buffer. Based on whether we add the segments individually, add a mp4 creating by concatenating the segments to the buffer or specify that mp4 as the source of the video element, the behavior differs. The 2nd and 4th scenario fail to start playback as the video element reports an empty buffer. All other scenarios do work. Did this work before? N/A Is it a problem with Flash or HTML5? HTML5 Does this work in other browsers? Yes Chrome version: 67.0.3396.99 Channel: stable OS Version: 10.0 Flash Version: Contents of chrome://gpu: Graphics Feature Status Canvas: Hardware accelerated CheckerImaging: Disabled Flash: Hardware accelerated Flash Stage3D: Hardware accelerated Flash Stage3D Baseline profile: Hardware accelerated Compositing: Hardware accelerated Multiple Raster Threads: Enabled Native GpuMemoryBuffers: Software only. Hardware acceleration disabled Rasterization: Hardware accelerated Surface Synchronization: Enabled Video Decode: Hardware accelerated Viz Service Display Compositor: Disabled WebGL: Hardware accelerated WebGL2: Hardware accelerated Driver Bug Workarounds clear_uniforms_before_first_program_use decode_encode_srgb_for_generatemipmap disable_accelerated_vpx_decode disable_discard_framebuffer disable_framebuffer_cmaa exit_on_context_lost force_cube_complete msaa_is_slow scalarize_vec_and_mat_constructor_args texsubimage_faster_than_teximage Problems Detected Some drivers are unable to reset the D3D device in the GPU process sandbox Applied Workarounds: exit_on_context_lost TexSubImage is faster for full uploads on ANGLE Applied Workarounds: texsubimage_faster_than_teximage Clear uniforms before first program use on all platforms: 124764, 349137 Applied Workarounds: clear_uniforms_before_first_program_use Always rewrite vec/mat constructors to be consistent: 398694 Applied Workarounds: scalarize_vec_and_mat_constructor_args ANGLE crash on glReadPixels from incomplete cube map texture: 518889 Applied Workarounds: force_cube_complete On Intel GPUs MSAA performance is not acceptable for GPU rasterization: 527565 Applied Workarounds: msaa_is_slow Framebuffer discarding can hurt performance on non-tilers: 570897 Applied Workarounds: disable_discard_framebuffer Use GL_INTEL_framebuffer_CMAA on ChromeOS: 535198 Applied Workarounds: disable_framebuffer_cmaa Disable KHR_blend_equation_advanced until cc shaders are updated: 661715 Applied Workarounds: disable(GL_KHR_blend_equation_advanced), disable(GL_KHR_blend_equation_advanced_coherent) Decode and Encode before generateMipmap for srgb format textures on Windows: 634519 Applied Workarounds: decode_encode_srgb_for_generatemipmap VPx decoding is too slow on Intel Broadwell, Skylake, and CherryView: 616318 Applied Workarounds: disable_accelerated_vpx_decode Don't expose disjoint_timer_query extensions to WebGL: 808744 Native GpuMemoryBuffers have been disabled, either via about:flags or command line. Disabled Features: native_gpu_memory_buffers Viz service display compositor is not enabled by default. Disabled Features: viz_display_compositor Checker-imaging has been disabled via finch trial or the command line. Disabled Features: checker_imaging Version Information Data exported 2018-07-05T07:36:52.821Z Chrome version Chrome/67.0.3396.99 Operating system Windows NT 10.0.17134 Software rendering list URL https://chromium.googlesource.com/chromium/src/+/a337fbf3c2ab8ebc6b64b0bfdce73a20e2e2252b/gpu/config/software_rendering_list.json Driver bug list URL https://chromium.googlesource.com/chromium/src/+/a337fbf3c2ab8ebc6b64b0bfdce73a20e2e2252b/gpu/config/gpu_driver_bug_list.json ANGLE commit id 702006f4a07e 2D graphics backend Skia/67 baf6686f92df805d3e25e80a0f3c79597cb3a6a5- Command Line "C:\Program Files (x86)\Google\Chrome\Application\chrome.exe" --flag-switches-begin --flag-switches-end Driver Information Initialization time 173 In-process GPU false Passthrough Command Decoder false Direct Composition true Supports overlays true Sandboxed true GPU0 VENDOR = 0x8086, DEVICE= 0x191b *ACTIVE* Optimus false AMD switchable false Desktop compositing Aero Glass Diagonal Monitor Size of \\.\DISPLAY1 24.0" Diagonal Monitor Size of \\.\DISPLAY1 15.5" Diagonal Monitor Size of \\.\DISPLAY2 15.5" Diagonal Monitor Size of \\.\DISPLAY3 15.5" DX12 true Vulkan true Driver vendor Intel Corporation Driver version 22.20.16.4771 Driver date 8-13-2017 Pixel shader version 5.0 Vertex shader version 5.0 Max. MSAA samples 16 Machine model name Machine model version GL_VENDOR Google Inc. GL_RENDERER ANGLE (Intel(R) HD Graphics 530 Direct3D11 vs_5_0 ps_5_0) GL_VERSION OpenGL ES 2.0 (ANGLE 2.1.0.702006f4a07e) GL_EXTENSIONS GL_ANGLE_client_arrays GL_ANGLE_depth_texture GL_ANGLE_framebuffer_blit GL_ANGLE_framebuffer_multisample GL_ANGLE_instanced_arrays GL_ANGLE_lossy_etc_decode GL_ANGLE_pack_reverse_row_order GL_ANGLE_program_cache_control GL_ANGLE_request_extension GL_ANGLE_robust_client_memory GL_ANGLE_texture_compression_dxt3 GL_ANGLE_texture_compression_dxt5 GL_ANGLE_texture_usage GL_ANGLE_translated_shader_source GL_CHROMIUM_bind_generates_resource GL_CHROMIUM_bind_uniform_location GL_CHROMIUM_color_buffer_float_rgb GL_CHROMIUM_color_buffer_float_rgba GL_CHROMIUM_copy_compressed_texture GL_CHROMIUM_copy_texture GL_CHROMIUM_sync_query GL_EXT_blend_minmax GL_EXT_color_buffer_half_float GL_EXT_debug_marker GL_EXT_discard_framebuffer GL_EXT_disjoint_timer_query GL_EXT_draw_buffers GL_EXT_frag_depth GL_EXT_map_buffer_range GL_EXT_occlusion_query_boolean GL_EXT_read_format_bgra GL_EXT_robustness GL_EXT_sRGB GL_EXT_shader_texture_lod GL_EXT_texture_compression_dxt1 GL_EXT_texture_compression_s3tc_srgb GL_EXT_texture_filter_anisotropic GL_EXT_texture_format_BGRA8888 GL_EXT_texture_rg GL_EXT_texture_storage GL_EXT_unpack_subimage GL_KHR_debug GL_NV_EGL_stream_consumer_external GL_NV_fence GL_NV_pack_subimage GL_NV_pixel_buffer_object GL_OES_EGL_image GL_OES_EGL_image_external GL_OES_compressed_ETC1_RGB8_texture GL_OES_depth32 GL_OES_element_index_uint GL_OES_get_program_binary GL_OES_mapbuffer GL_OES_packed_depth_stencil GL_OES_rgb8_rgba8 GL_OES_standard_derivatives GL_OES_surfaceless_context GL_OES_texture_float GL_OES_texture_float_linear GL_OES_texture_half_float GL_OES_texture_half_float_linear GL_OES_texture_npot GL_OES_vertex_array_object Disabled Extensions GL_KHR_blend_equation_advanced GL_KHR_blend_equation_advanced_coherent Disabled WebGL Extensions EXT_disjoint_timer_query EXT_disjoint_timer_query_webgl2 Window system binding vendor Google Inc. (adapter LUID: 0000000000015621) Window system binding version 1.4 (ANGLE 2.1.0.702006f4a07e) Window system binding extensions EGL_EXT_create_context_robustness EGL_ANGLE_d3d_share_handle_client_buffer EGL_ANGLE_d3d_texture_client_buffer EGL_ANGLE_surface_d3d_texture_2d_share_handle EGL_ANGLE_query_surface_pointer EGL_ANGLE_window_fixed_size EGL_ANGLE_keyed_mutex EGL_ANGLE_surface_orientation EGL_ANGLE_direct_composition EGL_NV_post_sub_buffer EGL_KHR_create_context EGL_EXT_device_query EGL_KHR_image EGL_KHR_image_base EGL_KHR_gl_texture_2D_image EGL_KHR_gl_texture_cubemap_image EGL_KHR_gl_renderbuffer_image EGL_KHR_get_all_proc_addresses EGL_KHR_stream EGL_KHR_stream_consumer_gltexture EGL_NV_stream_consumer_gltexture_yuv EGL_ANGLE_flexible_surface_compatibility EGL_ANGLE_stream_producer_d3d_texture EGL_ANGLE_create_context_webgl_compatibility EGL_CHROMIUM_create_context_bind_generates_resource EGL_CHROMIUM_sync_control EGL_EXT_pixel_format_float EGL_KHR_surfaceless_context EGL_ANGLE_display_texture_share_group EGL_ANGLE_create_context_client_arrays EGL_ANGLE_program_cache_control EGL_ANGLE_robust_resource_initialization EGL_ANGLE_create_context_extensions_enabled Direct rendering Yes Reset notification strategy 0x8252 GPU process crash count 0 Compositor Information Tile Update Mode One-copy Partial Raster Enabled GpuMemoryBuffers Status ATC Software only ATCIA Software only DXT1 Software only DXT5 Software only ETC1 Software only R_8 Software only R_16 Software only RG_88 Software only BGR_565 Software only RGBA_4444 Software only RGBX_8888 GPU_READ, SCANOUT RGBA_8888 GPU_READ, SCANOUT BGRX_8888 Software only BGRX_1010102 Software only RGBX_1010102 Software only BGRA_8888 Software only RGBA_F16 Software only YVU_420 Software only YUV_420_BIPLANAR Software only UYVY_422 Software only Display(s) Information Info Display[2528732444] bounds=[0,0 1920x1200], workarea=[0,0 1920x1160], scale=1, external. Color space information {primaries:BT709, transfer:IEC61966_2_1, matrix:RGB, range:FULL} Bits per color component 8 Bits per pixel 24 Video Acceleration Information Decode h264 baseline up to 4096x2304 pixels Decode h264 baseline up to 2304x4096 pixels Decode h264 main up to 4096x2304 pixels Decode h264 main up to 2304x4096 pixels Decode h264 high up to 4096x2304 pixels Decode h264 high up to 2304x4096 pixels Encode h264 baseline up to 3840x2176 pixels and/or 30.000 fps Encode h264 main up to 3840x2176 pixels and/or 30.000 fps Encode h264 high up to 3840x2176 pixels and/or 30.000 fps Diagnostics 0 b3DAccelerationEnabled true b3DAccelerationExists true bAGPEnabled true bAGPExistenceValid true bAGPExists true bCanRenderWindow true bDDAccelerationEnabled true bDriverBeta false bDriverDebug false bDriverSigned false bDriverSignedValid false bNoHardware false dwBpp 32 dwDDIVersion 12 dwHeight 1200 dwRefreshRate 59 dwWHQLLevel 0 dwWidth 1920 iAdapter 0 lDriverSize 59869552 lMiniVddSize 0 szAGPStatusEnglish Enabled szAGPStatusLocalized Enabled szChipType Intel(R) HD Graphics Family szD3DStatusEnglish Enabled szD3DStatusLocalized Enabled szDACType Internal szDDIVersionEnglish 12 szDDIVersionLocalized 12 szDDStatusEnglish Enabled szDDStatusLocalized Enabled szDXVAHDEnglish Supported szDXVAModes ModeMPEG2_A ModeMPEG2_C ModeWMV9_C ModeVC1_C szDescription Intel(R) HD Graphics 530 szDeviceId 0x191B szDeviceIdentifier {D7B78E66-5A5B-11CF-4B64-1945BDC2DB35} szDeviceName \\.\DISPLAY1 szDisplayMemoryEnglish 8262 MB szDisplayMemoryLocalized 8262 MB szDisplayModeEnglish 1920 x 1200 (32 bit) (59Hz) szDisplayModeLocalized 1920 x 1200 (32 bit) (59Hz) szDriverAssemblyVersion 22.20.16.4771 szDriverAttributes Final Retail szDriverDateEnglish 8/13/2017 2:00:00 AM szDriverDateLocalized 8/13/2017 02:00:00 szDriverLanguageEnglish English szDriverLanguageLocalized English szDriverModelEnglish WDDM 2.1 szDriverModelLocalized WDDM 2.1 szDriverName C:\WINDOWS\System32\DriverStore\FileRepository\igdlh64.inf_amd64_f10b0c0616a02072\igdumdim64.dll,C:\WINDOWS\System32\DriverStore\FileRepository\igdlh64.inf_amd64_f10b0c0616a02072\igd10iumd64.dll,C:\WINDOWS\System32\DriverStore\FileRepository\igdlh64.inf_amd64_f10b0c0616a02072\igd10iumd64.dll,C:\WINDOWS\System32\DriverStore\FileRepository\igdlh64.inf_amd64_f10b0c0616a02072\igd12umd64.dll szDriverNodeStrongName oem12.inf:5f63e5340a14bf6c:iSKLD_w10_DS:22.20.16.4771:pci\ven_8086&dev_191b szDriverSignDate Unknown szDriverVersion 22.20.0016.4771 szKeyDeviceID Enum\PCI\VEN_8086&DEV_191B&SUBSYS_65091558&REV_06 szKeyDeviceKey \Registry\Machine\System\CurrentControlSet\Control\Video\{CCDEF62C-65BF-11E8-ADE0-80FA5B38C880}\0000 szManufacturer Intel Corporation szMiniVdd unknown szMiniVddDateEnglish Unknown szMiniVddDateLocalized unknown szMonitorMaxRes Unknown szMonitorName Generic PnP Monitor szNotesEnglish No problems found. szNotesLocalized No problems found. szOverlayEnglish Supported szRankOfInstalledDriver 00D12001 szRegHelpText Unknown szRevision Unknown szRevisionId 0x0006 szSubSysId 0x65091558 szTestResultD3D7English Not run szTestResultD3D7Localized Not run szTestResultD3D8English Not run szTestResultD3D8Localized Not run szTestResultD3D9English Not run szTestResultD3D9Localized Not run szTestResultDDEnglish Not run szTestResultDDLocalized Not run szVdd unknown szVendorId 0x8086 Log Messages GpuProcessHostUIShim: The GPU process exited normally. Everything is okay. GpuProcessHostUIShim: The GPU process exited normally. Everything is okay.
,
Jul 6
+mse I haven't looked at the file yet and am no expert, but some thoughts: Isn't a fragmented mp4 required to start with a keyframe? So the last two cases are not quite the same as manually cutting the first segment? Assuming you do have proper keyframes, just skipping the segments without re-timecoding them isn't sufficient. So dropping just the first segment probably requires you to either seek to the right time code or set timestampOffset appropriately.
,
Jul 10
I agree that the last two cases are different from the first four, I added them for completeness. As for your remarks, here is some additional info: - The segments were analyzed with a ISOBMFF analyzer which reported that they all start with a key frame. - The provided code seeks to the start of buffer. This results in an error in the two failing cases (2 and 4) since the video element reports the buffer as empty.
,
Jul 10
Thanks for the details. I took a look and I'm not sure why it's not playing. There are no errors emitted which is a problem at least. Adjusting timestampOffset does not fix anything. One thing that I did notice is that ffmpeg doesn't report the individual packets as being flagged as keyframes, even though cat init.dash + 1.dash > out.mp4 into Mp4box shows the first sample as marked as keyframe. I wonder if somehow we're dropping data thinking it's not a keyframe.
,
Jul 10
I'll take a look now that I'm back from vacation. Thanks for your patience.
,
Jul 10
MP4 has multiple ways of marking coded frames as keyframes. I'll check the files to see what Chrome thinks about them -- it's likely that no keyframes were ever detected by Chrome, hence nothing buffered. Seek in an unbuffered MSE media element itself shouldn't cause error -- if the error is coming from the seek operation itself in Chrome, that could also be a bug. Regardless, I'll look today.
,
Jul 10
,
Jul 10
I don't think there's any error thrown other than an unchecked access to .buffered.start(0) when buffered.length==0. So the hypothesis that it's just not detecting the key frame markings seems plausible.
,
Jul 10
It looks like the media only has precisely 1 keyframe: the very first frame in the first segment. MSE drops any non-keyframes parsed since the beginning of parsing (or since the last discontinuity detected by the coded frame processing algorithm, or after an explicit abort() operation). So nothing buffered in cases 2 and 4 is working as intended, assuming Chrome is correctly identifying there is no keyframe in the media segments appended in those cases. Further, no parse error or other error is expected per specification in such case. Examining the frame sizes in cases 2 and 4, there are periodic spikes; possibly for frames incorrectly marked non-keyframes. I've attached some log captures from a local build for each of cases 1, 2, and 4, filtered to show each coded frame parsed from the ISO-BMFF stream, in the sequence they are emitted to be processed by the MSE coded frame processing algorithm. Cases 2 and 4 logs do not differ. The latter part of case 1 matches case 2.
,
Jul 10
@#9 clarification of my wording: It is working as intended that nothing is buffered in cases 2 and 4 (since no keyframes were parsed).
,
Jul 10
@#9 more clarification: MSE drops any non-keyframes parsed since the beginning of parsing...until a keyframe is parsed.
,
Jul 10
It's not clear to me that they aren't keyframes though, since init + 1 plays: http://xorax.sea.corp.google.com/mstest/segments/out-1.mp4 MP4Box also thinks that file has a keyframe at the beginning, but neither ffmpeg nor Chrome think so... so it's unclear how mp4box is determining that.
,
Jul 10
+sandersd who might know why; see c#12.
,
Jul 10
Chrome MSE MP4 parser detected a keyframe, but only in the first segment. It's logic has lots of comments due to the fairly complicated legacy around various media encodings not marking their ISOBMFF keyframes quite right: https://chromium.googlesource.com/chromium/src/+/542807b25b6e7befcb71e69597fbc45dc3a52e6a/media/formats/mp4/track_run_iterator.cc#191 MP4Box might have some lenience different from Chrome MSE, and maybe just for the first frame in the media. sandersd@, please take a closer look. Thanks!
,
Jul 10
One thing to note is that some other implementations of MSE might incorrectly be parsing and *not ignoring* sidx boxes, which can provide information like "starts_with_sap" that can inform whether or not the first indexed access unit of that sidx is perhaps a keyframe. Using such sidx information in an MSE implementation would not be compliant with the MSE ISOBMFF byte stream format (https://www.w3.org/TR/mse-byte-stream-format-isobmff/). In particular: "Valid top-level boxes such as pdin, free, and sidx are allowed to appear before the moov box. These boxes must be accepted and ignored by the user agent and are not considered part of the initialization segment in this specification." and "Valid top-level boxes defined in ISO/IEC 14496-12 other than ftyp, moov, styp, moof, and mdat are allowed to appear between the end of an initialization segment or media segment and before the beginning of a new media segment. These boxes must be accepted and ignored by the user agent and are not considered part of the media segment in this specification."
,
Jul 12
One other bit of info I've now recalled is that other MSE implementations may base their determination of "keyframe-ness" for some codecs based on codec-specific information, disregarding or overriding any such indication in the container itself. For example, an IDR in H.264 in MP4 might be determined as a keyframe by such implementations even if the MP4 doesn't mark the frame as such. In Chrome MSE, we rely on correctly muxed streams. We even have a complementary case bug 584384 . Dan, perhaps, as a complement to fixing bug 584384 , we could also MEDIA_LOG when a codec-bitstream keyframe is not marked keyframe in the container. WDYT? Please also look into the media for this bug to see if that's what's really happening here.
,
Jul 12
I took a look; Dan please correct if I'm wrong: In the sample media, there are multiple coded frames with nal_unit_type 5 (IDR slice); each corresponds to one of the longer periodic codec frames. But only the very first of those is determined to be a keyframe by our parsing of the MP4 container metadata. This explains how other MSE implementations (and perhaps ffmpeg) can be more lenient: they cross layers and use information from the codec bitstream to determine keyframe-ness. jeroen.tempels@theoplayer.com: To get this stream to work in Chrome MSE, please remux it with correct keyframe indications in the ISO-BMFF container, not just in the encoded video frames.
,
Jul 12
,
Jul 12
From chat w/Dan, the route to MEDIA_LOG when kIDRSplice is seen in any NALU for a coded frame, but the container didn't mark the frame as a keyframe, appears to be the best way forward. Note also that the MSE spec for the ISO-BMFF bytestream defines random access points in terms of the container, not the coded frames it contains. Some other MSE implementations may be providing more lenience for handling incorrectly muxed media streams w.r.t. the spec; there are workarounds for handling such streams (in worst case, transmuxing in JS app before appending to MSE). I'll see about adding a LIMITED_MEDIA_LOG soon.
,
Jul 13
Thanks for the all the information and investigation! I will remux the stream.
,
Jul 20
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/8005190febec38ff366befdab72cbd4b15e9a0c4 commit 8005190febec38ff366befdab72cbd4b15e9a0c4 Author: Matt Wolenetz <wolenetz@chromium.org> Date: Fri Jul 20 00:32:07 2018 MSE: Remove old AVC and HEVC IsValidAnnexB vector wrapper Chromium's allowed use of C++11's std::vector::data for quite a while now. In anticipation of some other refactoring coming soon to the underlying code, this CL removes the workaround previously in place for IsValidAnnexB(std::vector, ...) for each of MSE's AVC and HEVC bitstream conversion utilities. BUG= 860420 , 584384 Change-Id: I15f1cedebaa37bbac91cb62456b8322dc70e34c4 Reviewed-on: https://chromium-review.googlesource.com/1144218 Reviewed-by: Dan Sanders <sandersd@chromium.org> Reviewed-by: Sergey Volk <servolk@chromium.org> Commit-Queue: Matthew Wolenetz <wolenetz@chromium.org> Cr-Commit-Position: refs/heads/master@{#576720} [modify] https://crrev.com/8005190febec38ff366befdab72cbd4b15e9a0c4/media/formats/mp4/avc.cc [modify] https://crrev.com/8005190febec38ff366befdab72cbd4b15e9a0c4/media/formats/mp4/avc.h [modify] https://crrev.com/8005190febec38ff366befdab72cbd4b15e9a0c4/media/formats/mp4/avc_unittest.cc [modify] https://crrev.com/8005190febec38ff366befdab72cbd4b15e9a0c4/media/formats/mp4/hevc.cc [modify] https://crrev.com/8005190febec38ff366befdab72cbd4b15e9a0c4/media/formats/mp4/hevc.h
,
Jul 24
,
Jul 24
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/568cc6a9efbcc2cbfbc33d5442dbaa7eaf5ae6ae commit 568cc6a9efbcc2cbfbc33d5442dbaa7eaf5ae6ae Author: Matt Wolenetz <wolenetz@chromium.org> Date: Tue Jul 24 22:52:59 2018 MSE: refactor mp4 AnnexB validation to also report keyframe-ness In preparation for logging when there appears to be a mismatch between MSE MP4 keyframe metadata and the encoded video bitstream's keyframe metadata, this change incorporates basic analysis of the latter as part of the existing Annex-B bitstream validation for MSE MP4 video. Note that mp4::BitstreamConverter::AnalysisResult contains base::Optional<bool> for each of conformance and keyframe-ness fields. If a field doesn't have a value, that portion of the result was not analyzed. Note that keyframe analysis is implemented only for AVC currently, but even that is skipped if the frame is detected as non-conformant before enough indications of whether or not it is a keyframe are detected. Neither AVC-DV nor HEVC AnnexB analyses do any actual keyframe analysis, since such was either skipped or not yet implemented previously, respectively. A subsequent CL will use the newly reported MSE MP4 video bitstream converter's keyframe analysis results, if that analysis was done, in reporting to chrome://media-internals when the bitstream keyframe-ness mismatches that of the mp4 container for a coded frame. BUG= 860420 , 584384 Cq-Include-Trybots: luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel Change-Id: I169c7774070ad232c86658bcd7803160323993ad Reviewed-on: https://chromium-review.googlesource.com/1144456 Commit-Queue: Matthew Wolenetz <wolenetz@chromium.org> Reviewed-by: Sergey Volk <servolk@chromium.org> Reviewed-by: Dan Sanders <sandersd@chromium.org> Cr-Commit-Position: refs/heads/master@{#577717} [modify] https://crrev.com/568cc6a9efbcc2cbfbc33d5442dbaa7eaf5ae6ae/media/filters/ffmpeg_demuxer_unittest.cc [modify] https://crrev.com/568cc6a9efbcc2cbfbc33d5442dbaa7eaf5ae6ae/media/formats/mp4/avc.cc [modify] https://crrev.com/568cc6a9efbcc2cbfbc33d5442dbaa7eaf5ae6ae/media/formats/mp4/avc.h [modify] https://crrev.com/568cc6a9efbcc2cbfbc33d5442dbaa7eaf5ae6ae/media/formats/mp4/avc_unittest.cc [modify] https://crrev.com/568cc6a9efbcc2cbfbc33d5442dbaa7eaf5ae6ae/media/formats/mp4/bitstream_converter.cc [modify] https://crrev.com/568cc6a9efbcc2cbfbc33d5442dbaa7eaf5ae6ae/media/formats/mp4/bitstream_converter.h [modify] https://crrev.com/568cc6a9efbcc2cbfbc33d5442dbaa7eaf5ae6ae/media/formats/mp4/hevc.cc [modify] https://crrev.com/568cc6a9efbcc2cbfbc33d5442dbaa7eaf5ae6ae/media/formats/mp4/hevc.h [modify] https://crrev.com/568cc6a9efbcc2cbfbc33d5442dbaa7eaf5ae6ae/media/formats/mp4/mp4_stream_parser.cc
,
Jul 25
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/61697367404dc4c7033e389556f01a652f2b6516 commit 61697367404dc4c7033e389556f01a652f2b6516 Author: Matt Wolenetz <wolenetz@chromium.org> Date: Wed Jul 25 00:42:07 2018 MSE: Log when MP4 keyframe-ness mismatches the compressed media Logs to chrome://media-internals when the container's keyframe metadata for a frame mismatches the contents of that frame (if the latter was determined by the respective bitstream converter's Analyze() implementation, of whom only AVC currently does any keyframe analysis). BUG= 860420 , 584384 TEST=Repro crbug 860420 shows logs, new MP4StreamParserTest.AVC_Key* and media Change-Id: Ic87eab8726ad76b36b569df08ab2212b267008c2 Reviewed-on: https://chromium-review.googlesource.com/1149153 Reviewed-by: Dan Sanders <sandersd@chromium.org> Commit-Queue: Matthew Wolenetz <wolenetz@chromium.org> Cr-Commit-Position: refs/heads/master@{#577751} [modify] https://crrev.com/61697367404dc4c7033e389556f01a652f2b6516/media/formats/mp4/mp4_stream_parser.cc [modify] https://crrev.com/61697367404dc4c7033e389556f01a652f2b6516/media/formats/mp4/mp4_stream_parser.h [modify] https://crrev.com/61697367404dc4c7033e389556f01a652f2b6516/media/formats/mp4/mp4_stream_parser_unittest.cc [modify] https://crrev.com/61697367404dc4c7033e389556f01a652f2b6516/media/test/data/README [add] https://crrev.com/61697367404dc4c7033e389556f01a652f2b6516/media/test/data/bear-640x360-v-2frames-keyframe-is-non-sync-sample_frag.mp4 [add] https://crrev.com/61697367404dc4c7033e389556f01a652f2b6516/media/test/data/bear-640x360-v-2frames-nonkeyframe-is-sync-sample_frag.mp4 [add] https://crrev.com/61697367404dc4c7033e389556f01a652f2b6516/media/test/data/bear-640x360-v-2frames_frag.mp4
,
Jul 25
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/9a4096bed9f7071637393d67a2d2fe9f554bfd45 commit 9a4096bed9f7071637393d67a2d2fe9f554bfd45 Author: Zhenyao Mo <zmo@chromium.org> Date: Wed Jul 25 17:50:12 2018 Revert "MSE: Log when MP4 keyframe-ness mismatches the compressed media" This reverts commit 61697367404dc4c7033e389556f01a652f2b6516. Reason for revert: crbug.com/867520 Original change's description: > MSE: Log when MP4 keyframe-ness mismatches the compressed media > > Logs to chrome://media-internals when the container's keyframe metadata > for a frame mismatches the contents of that frame (if the latter was > determined by the respective bitstream converter's Analyze() > implementation, of whom only AVC currently does any keyframe analysis). > > BUG= 860420 , 584384 > TEST=Repro crbug 860420 shows logs, new MP4StreamParserTest.AVC_Key* and media > > Change-Id: Ic87eab8726ad76b36b569df08ab2212b267008c2 > Reviewed-on: https://chromium-review.googlesource.com/1149153 > Reviewed-by: Dan Sanders <sandersd@chromium.org> > Commit-Queue: Matthew Wolenetz <wolenetz@chromium.org> > Cr-Commit-Position: refs/heads/master@{#577751} TBR=wolenetz@chromium.org,sandersd@chromium.org Change-Id: I8437af99f24bf04910ad56931a9e91f960a59469 No-Presubmit: true No-Tree-Checks: true No-Try: true Bug: 860420 , 584384 Reviewed-on: https://chromium-review.googlesource.com/1150243 Reviewed-by: Zhenyao Mo <zmo@chromium.org> Commit-Queue: Zhenyao Mo <zmo@chromium.org> Cr-Commit-Position: refs/heads/master@{#577968} [modify] https://crrev.com/9a4096bed9f7071637393d67a2d2fe9f554bfd45/media/formats/mp4/mp4_stream_parser.cc [modify] https://crrev.com/9a4096bed9f7071637393d67a2d2fe9f554bfd45/media/formats/mp4/mp4_stream_parser.h [modify] https://crrev.com/9a4096bed9f7071637393d67a2d2fe9f554bfd45/media/formats/mp4/mp4_stream_parser_unittest.cc [modify] https://crrev.com/9a4096bed9f7071637393d67a2d2fe9f554bfd45/media/test/data/README [delete] https://crrev.com/352492fdd7794a1c7672bee0e081764a667ebd9b/media/test/data/bear-640x360-v-2frames-keyframe-is-non-sync-sample_frag.mp4 [delete] https://crrev.com/352492fdd7794a1c7672bee0e081764a667ebd9b/media/test/data/bear-640x360-v-2frames-nonkeyframe-is-sync-sample_frag.mp4 [delete] https://crrev.com/352492fdd7794a1c7672bee0e081764a667ebd9b/media/test/data/bear-640x360-v-2frames_frag.mp4
,
Jul 25
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/8fba4f3d294510d3e8317f950c01138d956fd554 commit 8fba4f3d294510d3e8317f950c01138d956fd554 Author: Zhenyao Mo <zmo@chromium.org> Date: Wed Jul 25 17:50:57 2018 Revert "MSE: refactor mp4 AnnexB validation to also report keyframe-ness" This reverts commit 568cc6a9efbcc2cbfbc33d5442dbaa7eaf5ae6ae. Reason for revert: <INSERT REASONING HERE> Original change's description: > MSE: refactor mp4 AnnexB validation to also report keyframe-ness > > In preparation for logging when there appears to be a mismatch between > MSE MP4 keyframe metadata and the encoded video bitstream's keyframe > metadata, this change incorporates basic analysis of the latter as part > of the existing Annex-B bitstream validation for MSE MP4 video. > > Note that mp4::BitstreamConverter::AnalysisResult contains > base::Optional<bool> for each of conformance and keyframe-ness fields. > If a field doesn't have a value, that portion of the result was not > analyzed. > > Note that keyframe analysis is implemented only for AVC currently, but > even that is skipped if the frame is detected as non-conformant before > enough indications of whether or not it is a keyframe are detected. > Neither AVC-DV nor HEVC AnnexB analyses do any actual keyframe analysis, > since such was either skipped or not yet implemented previously, > respectively. > > A subsequent CL will use the newly reported MSE MP4 video bitstream > converter's keyframe analysis results, if that analysis was done, in > reporting to chrome://media-internals when the bitstream keyframe-ness > mismatches that of the mp4 container for a coded frame. > > BUG= 860420 , 584384 > > Cq-Include-Trybots: luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel > Change-Id: I169c7774070ad232c86658bcd7803160323993ad > Reviewed-on: https://chromium-review.googlesource.com/1144456 > Commit-Queue: Matthew Wolenetz <wolenetz@chromium.org> > Reviewed-by: Sergey Volk <servolk@chromium.org> > Reviewed-by: Dan Sanders <sandersd@chromium.org> > Cr-Commit-Position: refs/heads/master@{#577717} TBR=wolenetz@chromium.org,sandersd@chromium.org,servolk@chromium.org Change-Id: If4d14f0cb6fb0779255b9d7eb3d5b2b8c20e5be1 No-Presubmit: true No-Tree-Checks: true No-Try: true Bug: 860420 , 584384 Cq-Include-Trybots: luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel Reviewed-on: https://chromium-review.googlesource.com/1150244 Reviewed-by: Zhenyao Mo <zmo@chromium.org> Commit-Queue: Zhenyao Mo <zmo@chromium.org> Cr-Commit-Position: refs/heads/master@{#577969} [modify] https://crrev.com/8fba4f3d294510d3e8317f950c01138d956fd554/media/filters/ffmpeg_demuxer_unittest.cc [modify] https://crrev.com/8fba4f3d294510d3e8317f950c01138d956fd554/media/formats/mp4/avc.cc [modify] https://crrev.com/8fba4f3d294510d3e8317f950c01138d956fd554/media/formats/mp4/avc.h [modify] https://crrev.com/8fba4f3d294510d3e8317f950c01138d956fd554/media/formats/mp4/avc_unittest.cc [modify] https://crrev.com/8fba4f3d294510d3e8317f950c01138d956fd554/media/formats/mp4/bitstream_converter.cc [modify] https://crrev.com/8fba4f3d294510d3e8317f950c01138d956fd554/media/formats/mp4/bitstream_converter.h [modify] https://crrev.com/8fba4f3d294510d3e8317f950c01138d956fd554/media/formats/mp4/hevc.cc [modify] https://crrev.com/8fba4f3d294510d3e8317f950c01138d956fd554/media/formats/mp4/hevc.h [modify] https://crrev.com/8fba4f3d294510d3e8317f950c01138d956fd554/media/formats/mp4/mp4_stream_parser.cc
,
Jul 25
,
Jul 25
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/a9b7a993e82e84c44dd16f332e55da569ab016c9 commit a9b7a993e82e84c44dd16f332e55da569ab016c9 Author: Matt Wolenetz <wolenetz@chromium.org> Date: Wed Jul 25 23:12:39 2018 Reland "MSE: refactor mp4 AnnexB validation to also report keyframe-ness" This is a reland of 568cc6a9efbcc2cbfbc33d5442dbaa7eaf5ae6ae, which had been reverted in 8fba4f3d294510d3e8317f950c01138d956fd554 ( bug 867520 ) due to failing link on windows builders with proprietary_codecs in their build configuration ( bug 867536 ). Versus the original change, this one includes an additional MEDIA_EXPORT on the inner type "media::mp4::BitstreamConverter::AnalysisResult". Original change's description: > MSE: refactor mp4 AnnexB validation to also report keyframe-ness > > In preparation for logging when there appears to be a mismatch between > MSE MP4 keyframe metadata and the encoded video bitstream's keyframe > metadata, this change incorporates basic analysis of the latter as part > of the existing Annex-B bitstream validation for MSE MP4 video. > > Note that mp4::BitstreamConverter::AnalysisResult contains > base::Optional<bool> for each of conformance and keyframe-ness fields. > If a field doesn't have a value, that portion of the result was not > analyzed. > > Note that keyframe analysis is implemented only for AVC currently, but > even that is skipped if the frame is detected as non-conformant before > enough indications of whether or not it is a keyframe are detected. > Neither AVC-DV nor HEVC AnnexB analyses do any actual keyframe analysis, > since such was either skipped or not yet implemented previously, > respectively. > > A subsequent CL will use the newly reported MSE MP4 video bitstream > converter's keyframe analysis results, if that analysis was done, in > reporting to chrome://media-internals when the bitstream keyframe-ness > mismatches that of the mp4 container for a coded frame. > > BUG= 860420 , 584384 > > Cq-Include-Trybots: luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel > Change-Id: I169c7774070ad232c86658bcd7803160323993ad > Reviewed-on: https://chromium-review.googlesource.com/1144456 > Commit-Queue: Matthew Wolenetz <wolenetz@chromium.org> > Reviewed-by: Sergey Volk <servolk@chromium.org> > Reviewed-by: Dan Sanders <sandersd@chromium.org> > Cr-Commit-Position: refs/heads/master@{#577717} Bug: 860420 , 584384 Change-Id: I8bbb3abc2079b53a14db8babbf04d5bad9e94767 Cq-Include-Trybots: luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel Reviewed-on: https://chromium-review.googlesource.com/1150247 Reviewed-by: Dan Sanders <sandersd@chromium.org> Commit-Queue: Matthew Wolenetz <wolenetz@chromium.org> Cr-Commit-Position: refs/heads/master@{#578116} [modify] https://crrev.com/a9b7a993e82e84c44dd16f332e55da569ab016c9/media/filters/ffmpeg_demuxer_unittest.cc [modify] https://crrev.com/a9b7a993e82e84c44dd16f332e55da569ab016c9/media/formats/mp4/avc.cc [modify] https://crrev.com/a9b7a993e82e84c44dd16f332e55da569ab016c9/media/formats/mp4/avc.h [modify] https://crrev.com/a9b7a993e82e84c44dd16f332e55da569ab016c9/media/formats/mp4/avc_unittest.cc [modify] https://crrev.com/a9b7a993e82e84c44dd16f332e55da569ab016c9/media/formats/mp4/bitstream_converter.cc [modify] https://crrev.com/a9b7a993e82e84c44dd16f332e55da569ab016c9/media/formats/mp4/bitstream_converter.h [modify] https://crrev.com/a9b7a993e82e84c44dd16f332e55da569ab016c9/media/formats/mp4/hevc.cc [modify] https://crrev.com/a9b7a993e82e84c44dd16f332e55da569ab016c9/media/formats/mp4/hevc.h [modify] https://crrev.com/a9b7a993e82e84c44dd16f332e55da569ab016c9/media/formats/mp4/mp4_stream_parser.cc
,
Jul 26
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/ae47fd35e9ca08850dc9a3585644616358d49364 commit ae47fd35e9ca08850dc9a3585644616358d49364 Author: Matt Wolenetz <wolenetz@chromium.org> Date: Thu Jul 26 02:16:45 2018 Reland "MSE: Log when MP4 keyframe-ness mismatches the compressed media" This is a reland of 61697367404dc4c7033e389556f01a652f2b6516, which had been reverted in 9a4096bed9f7071637393d67a2d2fe9f554bfd45 as part of reverting 568cc6a9efbcc2cbfbc33d5442dbaa7eaf5ae6ae which had broken some builders. That CL has now been fixed and relanded (a9b7a993e82e84c44dd16f332e55da569ab016c9). This reland is the same as the original. Original change's description: > MSE: Log when MP4 keyframe-ness mismatches the compressed media > > Logs to chrome://media-internals when the container's keyframe metadata > for a frame mismatches the contents of that frame (if the latter was > determined by the respective bitstream converter's Analyze() > implementation, of whom only AVC currently does any keyframe analysis). > > BUG= 860420 , 584384 > TEST=Repro crbug 860420 shows logs, new MP4StreamParserTest.AVC_Key* and media > > Change-Id: Ic87eab8726ad76b36b569df08ab2212b267008c2 > Reviewed-on: https://chromium-review.googlesource.com/1149153 > Reviewed-by: Dan Sanders <sandersd@chromium.org> > Commit-Queue: Matthew Wolenetz <wolenetz@chromium.org> > Cr-Commit-Position: refs/heads/master@{#577751} TBR=sandersd@chromium.org Bug: 860420 , 584384 Change-Id: I5a00819883e71ca9fd8c1a211956bdca66047b54 Reviewed-on: https://chromium-review.googlesource.com/1150541 Reviewed-by: Matthew Wolenetz <wolenetz@chromium.org> Reviewed-by: Dan Sanders <sandersd@chromium.org> Commit-Queue: Matthew Wolenetz <wolenetz@chromium.org> Cr-Commit-Position: refs/heads/master@{#578171} [modify] https://crrev.com/ae47fd35e9ca08850dc9a3585644616358d49364/media/formats/mp4/mp4_stream_parser.cc [modify] https://crrev.com/ae47fd35e9ca08850dc9a3585644616358d49364/media/formats/mp4/mp4_stream_parser.h [modify] https://crrev.com/ae47fd35e9ca08850dc9a3585644616358d49364/media/formats/mp4/mp4_stream_parser_unittest.cc [modify] https://crrev.com/ae47fd35e9ca08850dc9a3585644616358d49364/media/test/data/README [add] https://crrev.com/ae47fd35e9ca08850dc9a3585644616358d49364/media/test/data/bear-640x360-v-2frames-keyframe-is-non-sync-sample_frag.mp4 [add] https://crrev.com/ae47fd35e9ca08850dc9a3585644616358d49364/media/test/data/bear-640x360-v-2frames-nonkeyframe-is-sync-sample_frag.mp4 [add] https://crrev.com/ae47fd35e9ca08850dc9a3585644616358d49364/media/test/data/bear-640x360-v-2frames_frag.mp4
,
Aug 31
|
||||||||||
►
Sign in to add a comment |
||||||||||
Comment 1 by krajshree@chromium.org
, Jul 5