Issue metadata
Sign in to add a comment
|
Media Source Extensions no longer decoding individual frames accurately
Reported by
dustin.k...@gmail.com,
Nov 17 2017
|
||||||||||||||||||||
Issue descriptionUserAgent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.79 Safari/537.36 Example URL: https://jsfiddle.net/u3enjLzz/ Steps to reproduce the problem: 1. Click this link - https://jsfiddle.net/u3enjLzz/ 2. Attempt to index frames using the [ ] keys 3. Note that it doesn't work What is the expected behavior? I rely on frame accurate decoding using MSE for https://www.panomoments.com - This was working just fine from Chrome versions 51.x until 61.x but I just updated to Chrome 62.0.3202.97 and it no longer functions as it used to. While my use of MSE is a little unorthodox, this was working just fine for over a year. I also tested this with Hardware-accelerated video decode = Disabled and it behaves in the same broken way. Both OSX and Windows 10 behave the same way. It seems like this broke sometime in 62.x Any idea what has changed? Is this a bug or something that changed in the design? Thanks, Dustin What went wrong? Not sure. Did this work before? Yes 61.0.3163.79 Is it a problem with Flash or HTML5? N/A Does this work in other browsers? N/A Chrome version: 62.0.3202.97 Channel: n/a OS Version: 10.0 Flash Version: Contents of chrome://gpu: Note: To properly save this page, select the "Webpage, Complete" option in the Save File dialog. Graphics Feature Status Canvas: Hardware accelerated CheckerImaging: Disabled Flash: Hardware accelerated Flash Stage3D: Hardware accelerated Flash Stage3D Baseline profile: Hardware accelerated Compositing: Hardware accelerated Multiple Raster Threads: Enabled Native GpuMemoryBuffers: Software only. Hardware acceleration disabled Rasterization: Hardware accelerated Video Decode: Hardware accelerated Video Encode: Hardware accelerated WebGL: Hardware accelerated WebGL2: Hardware accelerated Driver Bug Workarounds clear_uniforms_before_first_program_use decode_encode_srgb_for_generatemipmap disable_delayed_copy_nv12 disable_discard_framebuffer disable_dxgi_zero_copy_video disable_framebuffer_cmaa exit_on_context_lost force_cube_complete scalarize_vec_and_mat_constructor_args texsubimage_faster_than_teximage Problems Detected Some drivers are unable to reset the D3D device in the GPU process sandbox Applied Workarounds: exit_on_context_lost TexSubImage is faster for full uploads on ANGLE Applied Workarounds: texsubimage_faster_than_teximage Clear uniforms before first program use on all platforms: 124764, 349137 Applied Workarounds: clear_uniforms_before_first_program_use Always rewrite vec/mat constructors to be consistent: 398694 Applied Workarounds: scalarize_vec_and_mat_constructor_args ANGLE crash on glReadPixels from incomplete cube map texture: 518889 Applied Workarounds: force_cube_complete Framebuffer discarding can hurt performance on non-tilers: 570897 Applied Workarounds: disable_discard_framebuffer Limited enabling of Chromium GL_INTEL_framebuffer_CMAA: 535198 Applied Workarounds: disable_framebuffer_cmaa Zero-copy NV12 video displays incorrect colors on NVIDIA drivers.: 635319 Applied Workarounds: disable_dxgi_zero_copy_video Disable KHR_blend_equation_advanced until cc shaders are updated: 661715 Decode and Encode before generateMipmap for srgb format textures on Windows: 634519 Applied Workarounds: decode_encode_srgb_for_generatemipmap Delayed copy NV12 displays incorrect colors on NVIDIA drivers.: 728670 Applied Workarounds: disable_delayed_copy_nv12 Native GpuMemoryBuffers have been disabled, either via about:flags or command line. Disabled Features: native_gpu_memory_buffers Checker-imaging has been disabled via finch trial or the command line. Disabled Features: checker_imaging Version Information Data exported 11/17/2017, 5:06:53 PM Chrome version Chrome/61.0.3163.79 Operating system Windows NT 10.0.16299 Software rendering list version 13.10 Driver bug list version 10.28 ANGLE commit id 0d2ecb4ea992 2D graphics backend Skia/61 0eefc0552cfb5ac077560b7c2630c5bd475ea585- Command Line "C:\Program Files (x86)\Google\Chrome\Application\chrome.exe" --flag-switches-begin --flag-switches-end Driver Information Initialization time 173 In-process GPU false Passthrough Command Decoder false Supports overlays false Sandboxed false GPU0 VENDOR = 0x10de, DEVICE= 0x1c20 *ACTIVE* GPU1 VENDOR = 0x8086, DEVICE= 0x591b Optimus false Optimus false AMD switchable false Desktop compositing Aero Glass Diagonal Monitor Size of \\.\DISPLAY1 13.2" Diagonal Monitor Size of \\.\DISPLAY4 24.0" Driver vendor NVIDIA Driver version 22.21.13.8541 Driver date 8-21-2017 Pixel shader version 5.0 Vertex shader version 5.0 Max. MSAA samples 8 Machine model name Machine model version GL_VENDOR Google Inc. GL_RENDERER ANGLE (NVIDIA GeForce GTX 1060 Direct3D11 vs_5_0 ps_5_0) GL_VERSION OpenGL ES 3.0 (ANGLE 2.1.0.0d2ecb4ea992) GL_EXTENSIONS GL_ANGLE_client_arrays GL_ANGLE_depth_texture GL_ANGLE_framebuffer_blit GL_ANGLE_framebuffer_multisample GL_ANGLE_instanced_arrays GL_ANGLE_lossy_etc_decode GL_ANGLE_pack_reverse_row_order GL_ANGLE_program_cache_control GL_ANGLE_request_extension GL_ANGLE_robust_client_memory GL_ANGLE_robust_resource_initialization GL_ANGLE_texture_compression_dxt3 GL_ANGLE_texture_compression_dxt5 GL_ANGLE_texture_usage GL_ANGLE_translated_shader_source GL_CHROMIUM_bind_generates_resource GL_CHROMIUM_bind_uniform_location GL_CHROMIUM_color_buffer_float_rgb GL_CHROMIUM_color_buffer_float_rgba GL_CHROMIUM_copy_compressed_texture GL_CHROMIUM_copy_texture GL_CHROMIUM_sync_query GL_EXT_blend_minmax GL_EXT_color_buffer_float GL_EXT_color_buffer_half_float GL_EXT_debug_marker GL_EXT_discard_framebuffer GL_EXT_disjoint_timer_query GL_EXT_draw_buffers GL_EXT_frag_depth GL_EXT_map_buffer_range GL_EXT_occlusion_query_boolean GL_EXT_read_format_bgra GL_EXT_robustness GL_EXT_sRGB GL_EXT_shader_texture_lod GL_EXT_texture_compression_dxt1 GL_EXT_texture_compression_s3tc_srgb GL_EXT_texture_filter_anisotropic GL_EXT_texture_format_BGRA8888 GL_EXT_texture_norm16 GL_EXT_texture_rg GL_EXT_texture_storage GL_EXT_unpack_subimage GL_KHR_debug GL_NV_EGL_stream_consumer_external GL_NV_fence GL_NV_pack_subimage GL_NV_pixel_buffer_object GL_OES_EGL_image GL_OES_EGL_image_external GL_OES_EGL_image_external_essl3 GL_OES_compressed_ETC1_RGB8_texture GL_OES_depth32 GL_OES_element_index_uint GL_OES_get_program_binary GL_OES_mapbuffer GL_OES_packed_depth_stencil GL_OES_rgb8_rgba8 GL_OES_standard_derivatives GL_OES_surfaceless_context GL_OES_texture_float GL_OES_texture_float_linear GL_OES_texture_half_float GL_OES_texture_half_float_linear GL_OES_texture_npot GL_OES_vertex_array_object Disabled Extensions GL_KHR_blend_equation_advanced GL_KHR_blend_equation_advanced_coherent Window system binding vendor Google Inc. (adapter LUID: 0000000000012a32) Window system binding version 1.4 (ANGLE 2.1.0.0d2ecb4ea992) Window system binding extensions EGL_EXT_create_context_robustness EGL_ANGLE_d3d_share_handle_client_buffer EGL_ANGLE_d3d_texture_client_buffer EGL_ANGLE_surface_d3d_texture_2d_share_handle EGL_ANGLE_query_surface_pointer EGL_ANGLE_window_fixed_size EGL_ANGLE_keyed_mutex EGL_ANGLE_surface_orientation EGL_ANGLE_direct_composition EGL_NV_post_sub_buffer EGL_KHR_create_context EGL_EXT_device_query EGL_KHR_image EGL_KHR_image_base EGL_KHR_gl_texture_2D_image EGL_KHR_gl_texture_cubemap_image EGL_KHR_gl_renderbuffer_image EGL_KHR_get_all_proc_addresses EGL_KHR_stream EGL_KHR_stream_consumer_gltexture EGL_NV_stream_consumer_gltexture_yuv EGL_ANGLE_flexible_surface_compatibility EGL_ANGLE_stream_producer_d3d_texture_nv12 EGL_ANGLE_create_context_webgl_compatibility EGL_CHROMIUM_create_context_bind_generates_resource EGL_CHROMIUM_sync_control EGL_EXT_pixel_format_float EGL_KHR_surfaceless_context EGL_ANGLE_display_texture_share_group EGL_ANGLE_create_context_client_arrays EGL_ANGLE_program_cache_control Direct rendering Yes Reset notification strategy 0x8252 GPU process crash count 0 Compositor Information Tile Update Mode One-copy Partial Raster Enabled GpuMemoryBuffers Status ATC Software only ATCIA Software only DXT1 Software only DXT5 Software only ETC1 Software only R_8 Software only R_16 Software only RG_88 Software only BGR_565 Software only RGBA_4444 Software only RGBX_8888 Software only RGBA_8888 Software only BGRX_8888 Software only BGRA_8888 Software only RGBA_F16 Software only YVU_420 Software only YUV_420_BIPLANAR Software only UYVY_422 Software only Diagnostics ... loading ... Log Messages [15504:11428:1117/165658.579:WARNING:angle_platform_impl.cc(41)] : rx::HLSLCompiler::compileToBinary(224): C:\fakepath(69,11-35): warning X3571: pow(f, e) will not work for negative f, use abs(f) or conditionally handle negative values if you expect them C:\fakepath(91,11-35): warning X3571: pow(f, e) will not work for negative f, use abs(f) or conditionally handle negative values if you expect them [15504:2716:1117/165658.605:WARNING:angle_platform_impl.cc(41)] : rx::HLSLCompiler::compileToBinary(224): C:\fakepath(52,8-56): warning X3571: pow(f, e) will not work for negative f, use abs(f) or conditionally handle negative values if you expect them C:\fakepath(63,9-41): warning X3571: pow(f, e) will not work for negative f, use abs(f) or conditionally handle negative values if you expect them [15504:14192:1117/165708.221:ERROR:gles2_cmd_decoder.cc(9798)] : [.Offscreen-For-WebGL-0000025702F820C0]RENDER WARNING: texture bound to texture unit 0 is not renderable. It maybe non-power-of-2 and have incompatible texture filtering. [15504:11428:1117/165709.551:WARNING:angle_platform_impl.cc(41)] : rx::HLSLCompiler::compileToBinary(224): C:\fakepath(49,8-56): warning X3571: pow(f, e) will not work for negative f, use abs(f) or conditionally handle negative values if you expect them C:\fakepath(60,9-41): warning X3571: pow(f, e) will not work for negative f, use abs(f) or conditionally handle negative values if you expect them [15504:14192:1117/165727.782:ERROR:mf_helpers.cc(14)] : Error in dxva_video_decode_accelerator_win.cc on line 1523 [15504:14192:1117/165727.782:ERROR:mf_helpers.cc(14)] : Error in dxva_video_decode_accelerator_win.cc on line 632 [15504:14192:1117/165727.782:ERROR:gpu_video_decode_accelerator.cc(381)] : HW video decode not available for profile vp8 [15504:11428:1117/170238.451:WARNING:angle_platform_impl.cc(41)] : rx::HLSLCompiler::compileToBinary(224): C:\fakepath(31,8-56): warning X3571: pow(f, e) will not work for negative f, use abs(f) or conditionally handle negative values if you expect them C:\fakepath(42,9-41): warning X3571: pow(f, e) will not work for negative f, use abs(f) or conditionally handle negative values if you expect them [15504:2716:1117/170238.461:WARNING:angle_platform_impl.cc(41)] : rx::HLSLCompiler::compileToBinary(224): C:\fakepath(49,8-56): warning X3571: pow(f, e) will not work for negative f, use abs(f) or conditionally handle negative values if you expect them C:\fakepath(60,9-41): warning X3571: pow(f, e) will not work for negative f, use abs(f) or conditionally handle negative values if you expect them [15504:11428:1117/170303.792:WARNING:angle_platform_impl.cc(41)] : rx::HLSLCompiler::compileToBinary(224): C:\fakepath(57,8-56): warning X3571: pow(f, e) will not work for negative f, use abs(f) or conditionally handle negative values if you expect them C:\fakepath(68,9-41): warning X3571: pow(f, e) will not work for negative f, use abs(f) or conditionally handle negative values if you expect them [15504:14192:1117/170638.147:ERROR:gles2_cmd_decoder.cc(9798)] : [.Offscreen-For-WebGL-00000150DBE8F100]RENDER WARNING: texture bound to texture unit 0 is not renderable. It maybe non-power-of-2 and have incompatible texture filtering. GpuProcessHostUIShim: The GPU process exited normally. Everything is okay.
,
Nov 18 2017
And note that you won't be able to replicate this on PanoMoments.com as I've already disabled MSE decoding as a temp workaround. Only use https://jsfiddle.net/u3enjLzz/ for testing.
,
Nov 18 2017
This broke because we now require at least 2 frames to resume from underflow even for low delay. We assume that if you're underflowing you we need more data for a smooth playback. I think we can improve this slightly to help this case and issue 767878 , by relying on the AverageDuration() value provided from DecoderStream. I'll see if I can put a small patch together for this next week.
,
Nov 18 2017
Thanks for the background info Dale. Do you think the potential patch would allow for a 0 to infinite delay between subsequent frame appends to the buffer? In my use-case the decoder may be stalled for .1 seconds, 10 minutes, or 4 hours.
,
Nov 21 2017
Not really any client side workaround for this in your use case. You would have to ensure you keep more frames in the queue to prevent underflow, but I think that's impossible for you since you allow going backwards. https://chromium-review.googlesource.com/c/chromium/src/+/780267 fixes the original issue while returning low delay to 1 frame of latency.
,
Nov 21 2017
Ok cool, I'll make sure to test 780267 as soon as it hits Canary.
,
Nov 29 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/21ece16c7d9e0d06786579e7d86a7993a706f656 commit 21ece16c7d9e0d06786579e7d86a7993a706f656 Author: Dale Curtis <dalecurtis@chromium.org> Date: Wed Nov 29 02:10:55 2017 Use video frame duration metadata instead of waiting for 2 frames. http://crrev.com/493874 changed the low delay case to wait for two frames since we need to be sure we don't resume until we actually have valid frames since the first frame was valid forever without duration information. It turns out we actually do have duration infromation hanging off the DecoderStream, so use this information and set it as the FRAME_DURATION metadata key for each VideoFrame. We can then have the algorithm use this information for the estimated end time of frames when only a single frame is present in the queue. Note: The video-canvas layout test was showing the wrong frame; I manually extracted the frames and verified that before we were showing the "7" frame when pts=2.0s == "6" frame. I've updated the test expectations appropriately. BUG= 786576 , 767878 ,709302 TEST=new unittest, old unittests pass w/o modification, manual test of 4k60 vp9 low latency content doesn't exhibit multiple stalls after returning to the foreground when the video track has been disabled, manual test with https://jsfiddle.net/u3enjLzz/ Cq-Include-Trybots: master.tryserver.chromium.android:android_optional_gpu_tests_rel;master.tryserver.chromium.linux:linux_optional_gpu_tests_rel;master.tryserver.chromium.mac:mac_optional_gpu_tests_rel;master.tryserver.chromium.win:win_optional_gpu_tests_rel Change-Id: I963903d6a173038a2d534db4e040c8f4774825b5 Reviewed-on: https://chromium-review.googlesource.com/780267 Commit-Queue: Dale Curtis <dalecurtis@chromium.org> Reviewed-by: Matthew Wolenetz <wolenetz@chromium.org> Cr-Commit-Position: refs/heads/master@{#519961} [modify] https://crrev.com/21ece16c7d9e0d06786579e7d86a7993a706f656/media/filters/video_renderer_algorithm.cc [modify] https://crrev.com/21ece16c7d9e0d06786579e7d86a7993a706f656/media/filters/video_renderer_algorithm.h [modify] https://crrev.com/21ece16c7d9e0d06786579e7d86a7993a706f656/media/filters/video_renderer_algorithm_unittest.cc [modify] https://crrev.com/21ece16c7d9e0d06786579e7d86a7993a706f656/media/renderers/video_renderer_impl.cc [modify] https://crrev.com/21ece16c7d9e0d06786579e7d86a7993a706f656/media/renderers/video_renderer_impl.h [modify] https://crrev.com/21ece16c7d9e0d06786579e7d86a7993a706f656/third_party/WebKit/LayoutTests/media/video-canvas.html [delete] https://crrev.com/afedd691d118f767302c309ffa7b337ae07ca13e/third_party/WebKit/LayoutTests/platform/linux/media/video-canvas-expected.txt
,
Nov 29 2017
Fixed, should be available in 64.0.3281.0 and higher.
,
Nov 30 2017
Looks great in Canary! |
|||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||
Comment 1 by dustin.k...@gmail.com
, Nov 18 2017