New issue
Advanced search Search tips

Issue 786576 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Nov 2017
Components:
EstimatedDays: ----
NextAction: ----
OS: Windows
Pri: 2
Type: Bug-Regression



Sign in to add a comment

Media Source Extensions no longer decoding individual frames accurately

Reported by dustin.k...@gmail.com, Nov 17 2017

Issue description

UserAgent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.79 Safari/537.36

Example URL:
https://jsfiddle.net/u3enjLzz/

Steps to reproduce the problem:
1. Click this link - https://jsfiddle.net/u3enjLzz/
2. Attempt to index frames using the [ ] keys
3. Note that it doesn't work

What is the expected behavior?
I rely on frame accurate decoding using MSE for https://www.panomoments.com - This was working just fine from Chrome versions 51.x until 61.x but I just updated to Chrome 62.0.3202.97 and it no longer functions as it used to. While my use of MSE is a little unorthodox, this was working just fine for over a year. 

I also tested this with Hardware-accelerated video decode = Disabled and it behaves in the same broken way. Both OSX and Windows 10 behave the same way. It seems like this broke sometime in 62.x

Any idea what has changed? Is this a bug or something that changed in the design?

Thanks,
Dustin

What went wrong?
Not sure.

Did this work before? Yes 61.0.3163.79

Is it a problem with Flash or HTML5? N/A

Does this work in other browsers? N/A

Chrome version: 62.0.3202.97  Channel: n/a
OS Version: 10.0
Flash Version: 

Contents of chrome://gpu: 
Note: To properly save this page, select the "Webpage, Complete" option in the Save File dialog.
Graphics Feature Status
Canvas: Hardware accelerated
CheckerImaging: Disabled
Flash: Hardware accelerated
Flash Stage3D: Hardware accelerated
Flash Stage3D Baseline profile: Hardware accelerated
Compositing: Hardware accelerated
Multiple Raster Threads: Enabled
Native GpuMemoryBuffers: Software only. Hardware acceleration disabled
Rasterization: Hardware accelerated
Video Decode: Hardware accelerated
Video Encode: Hardware accelerated
WebGL: Hardware accelerated
WebGL2: Hardware accelerated
Driver Bug Workarounds
clear_uniforms_before_first_program_use
decode_encode_srgb_for_generatemipmap
disable_delayed_copy_nv12
disable_discard_framebuffer
disable_dxgi_zero_copy_video
disable_framebuffer_cmaa
exit_on_context_lost
force_cube_complete
scalarize_vec_and_mat_constructor_args
texsubimage_faster_than_teximage
Problems Detected
Some drivers are unable to reset the D3D device in the GPU process sandbox
Applied Workarounds: exit_on_context_lost
TexSubImage is faster for full uploads on ANGLE
Applied Workarounds: texsubimage_faster_than_teximage
Clear uniforms before first program use on all platforms: 124764, 349137
Applied Workarounds: clear_uniforms_before_first_program_use
Always rewrite vec/mat constructors to be consistent: 398694
Applied Workarounds: scalarize_vec_and_mat_constructor_args
ANGLE crash on glReadPixels from incomplete cube map texture: 518889
Applied Workarounds: force_cube_complete
Framebuffer discarding can hurt performance on non-tilers: 570897
Applied Workarounds: disable_discard_framebuffer
Limited enabling of Chromium GL_INTEL_framebuffer_CMAA: 535198
Applied Workarounds: disable_framebuffer_cmaa
Zero-copy NV12 video displays incorrect colors on NVIDIA drivers.: 635319
Applied Workarounds: disable_dxgi_zero_copy_video
Disable KHR_blend_equation_advanced until cc shaders are updated: 661715
Decode and Encode before generateMipmap for srgb format textures on Windows: 634519
Applied Workarounds: decode_encode_srgb_for_generatemipmap
Delayed copy NV12 displays incorrect colors on NVIDIA drivers.: 728670
Applied Workarounds: disable_delayed_copy_nv12
Native GpuMemoryBuffers have been disabled, either via about:flags or command line.
Disabled Features: native_gpu_memory_buffers
Checker-imaging has been disabled via finch trial or the command line.
Disabled Features: checker_imaging
Version Information
Data exported	11/17/2017, 5:06:53 PM
Chrome version	Chrome/61.0.3163.79
Operating system	Windows NT 10.0.16299
Software rendering list version	13.10
Driver bug list version	10.28
ANGLE commit id	0d2ecb4ea992
2D graphics backend	Skia/61 0eefc0552cfb5ac077560b7c2630c5bd475ea585-
Command Line	"C:\Program Files (x86)\Google\Chrome\Application\chrome.exe" --flag-switches-begin --flag-switches-end
Driver Information
Initialization time	173
In-process GPU	false
Passthrough Command Decoder	false
Supports overlays	false
Sandboxed	false
GPU0	VENDOR = 0x10de, DEVICE= 0x1c20 *ACTIVE*
GPU1	VENDOR = 0x8086, DEVICE= 0x591b
Optimus	false
Optimus	false
AMD switchable	false
Desktop compositing	Aero Glass
Diagonal Monitor Size of \\.\DISPLAY1	13.2"
Diagonal Monitor Size of \\.\DISPLAY4	24.0"
Driver vendor	NVIDIA
Driver version	22.21.13.8541
Driver date	8-21-2017
Pixel shader version	5.0
Vertex shader version	5.0
Max. MSAA samples	8
Machine model name	
Machine model version	
GL_VENDOR	Google Inc.
GL_RENDERER	ANGLE (NVIDIA GeForce GTX 1060 Direct3D11 vs_5_0 ps_5_0)
GL_VERSION	OpenGL ES 3.0 (ANGLE 2.1.0.0d2ecb4ea992)
GL_EXTENSIONS	GL_ANGLE_client_arrays GL_ANGLE_depth_texture GL_ANGLE_framebuffer_blit GL_ANGLE_framebuffer_multisample GL_ANGLE_instanced_arrays GL_ANGLE_lossy_etc_decode GL_ANGLE_pack_reverse_row_order GL_ANGLE_program_cache_control GL_ANGLE_request_extension GL_ANGLE_robust_client_memory GL_ANGLE_robust_resource_initialization GL_ANGLE_texture_compression_dxt3 GL_ANGLE_texture_compression_dxt5 GL_ANGLE_texture_usage GL_ANGLE_translated_shader_source GL_CHROMIUM_bind_generates_resource GL_CHROMIUM_bind_uniform_location GL_CHROMIUM_color_buffer_float_rgb GL_CHROMIUM_color_buffer_float_rgba GL_CHROMIUM_copy_compressed_texture GL_CHROMIUM_copy_texture GL_CHROMIUM_sync_query GL_EXT_blend_minmax GL_EXT_color_buffer_float GL_EXT_color_buffer_half_float GL_EXT_debug_marker GL_EXT_discard_framebuffer GL_EXT_disjoint_timer_query GL_EXT_draw_buffers GL_EXT_frag_depth GL_EXT_map_buffer_range GL_EXT_occlusion_query_boolean GL_EXT_read_format_bgra GL_EXT_robustness GL_EXT_sRGB GL_EXT_shader_texture_lod GL_EXT_texture_compression_dxt1 GL_EXT_texture_compression_s3tc_srgb GL_EXT_texture_filter_anisotropic GL_EXT_texture_format_BGRA8888 GL_EXT_texture_norm16 GL_EXT_texture_rg GL_EXT_texture_storage GL_EXT_unpack_subimage GL_KHR_debug GL_NV_EGL_stream_consumer_external GL_NV_fence GL_NV_pack_subimage GL_NV_pixel_buffer_object GL_OES_EGL_image GL_OES_EGL_image_external GL_OES_EGL_image_external_essl3 GL_OES_compressed_ETC1_RGB8_texture GL_OES_depth32 GL_OES_element_index_uint GL_OES_get_program_binary GL_OES_mapbuffer GL_OES_packed_depth_stencil GL_OES_rgb8_rgba8 GL_OES_standard_derivatives GL_OES_surfaceless_context GL_OES_texture_float GL_OES_texture_float_linear GL_OES_texture_half_float GL_OES_texture_half_float_linear GL_OES_texture_npot GL_OES_vertex_array_object
Disabled Extensions	GL_KHR_blend_equation_advanced GL_KHR_blend_equation_advanced_coherent
Window system binding vendor	Google Inc. (adapter LUID: 0000000000012a32)
Window system binding version	1.4 (ANGLE 2.1.0.0d2ecb4ea992)
Window system binding extensions	EGL_EXT_create_context_robustness EGL_ANGLE_d3d_share_handle_client_buffer EGL_ANGLE_d3d_texture_client_buffer EGL_ANGLE_surface_d3d_texture_2d_share_handle EGL_ANGLE_query_surface_pointer EGL_ANGLE_window_fixed_size EGL_ANGLE_keyed_mutex EGL_ANGLE_surface_orientation EGL_ANGLE_direct_composition EGL_NV_post_sub_buffer EGL_KHR_create_context EGL_EXT_device_query EGL_KHR_image EGL_KHR_image_base EGL_KHR_gl_texture_2D_image EGL_KHR_gl_texture_cubemap_image EGL_KHR_gl_renderbuffer_image EGL_KHR_get_all_proc_addresses EGL_KHR_stream EGL_KHR_stream_consumer_gltexture EGL_NV_stream_consumer_gltexture_yuv EGL_ANGLE_flexible_surface_compatibility EGL_ANGLE_stream_producer_d3d_texture_nv12 EGL_ANGLE_create_context_webgl_compatibility EGL_CHROMIUM_create_context_bind_generates_resource EGL_CHROMIUM_sync_control EGL_EXT_pixel_format_float EGL_KHR_surfaceless_context EGL_ANGLE_display_texture_share_group EGL_ANGLE_create_context_client_arrays EGL_ANGLE_program_cache_control
Direct rendering	Yes
Reset notification strategy	0x8252
GPU process crash count	0
Compositor Information
Tile Update Mode	One-copy
Partial Raster	Enabled
GpuMemoryBuffers Status
ATC	Software only
ATCIA	Software only
DXT1	Software only
DXT5	Software only
ETC1	Software only
R_8	Software only
R_16	Software only
RG_88	Software only
BGR_565	Software only
RGBA_4444	Software only
RGBX_8888	Software only
RGBA_8888	Software only
BGRX_8888	Software only
BGRA_8888	Software only
RGBA_F16	Software only
YVU_420	Software only
YUV_420_BIPLANAR	Software only
UYVY_422	Software only
Diagnostics
... loading ...
Log Messages
[15504:11428:1117/165658.579:WARNING:angle_platform_impl.cc(41)] : rx::HLSLCompiler::compileToBinary(224): C:\fakepath(69,11-35): warning X3571: pow(f, e) will not work for negative f, use abs(f) or conditionally handle negative values if you expect them C:\fakepath(91,11-35): warning X3571: pow(f, e) will not work for negative f, use abs(f) or conditionally handle negative values if you expect them
[15504:2716:1117/165658.605:WARNING:angle_platform_impl.cc(41)] : rx::HLSLCompiler::compileToBinary(224): C:\fakepath(52,8-56): warning X3571: pow(f, e) will not work for negative f, use abs(f) or conditionally handle negative values if you expect them C:\fakepath(63,9-41): warning X3571: pow(f, e) will not work for negative f, use abs(f) or conditionally handle negative values if you expect them
[15504:14192:1117/165708.221:ERROR:gles2_cmd_decoder.cc(9798)] : [.Offscreen-For-WebGL-0000025702F820C0]RENDER WARNING: texture bound to texture unit 0 is not renderable. It maybe non-power-of-2 and have incompatible texture filtering.
[15504:11428:1117/165709.551:WARNING:angle_platform_impl.cc(41)] : rx::HLSLCompiler::compileToBinary(224): C:\fakepath(49,8-56): warning X3571: pow(f, e) will not work for negative f, use abs(f) or conditionally handle negative values if you expect them C:\fakepath(60,9-41): warning X3571: pow(f, e) will not work for negative f, use abs(f) or conditionally handle negative values if you expect them
[15504:14192:1117/165727.782:ERROR:mf_helpers.cc(14)] : Error in dxva_video_decode_accelerator_win.cc on line 1523
[15504:14192:1117/165727.782:ERROR:mf_helpers.cc(14)] : Error in dxva_video_decode_accelerator_win.cc on line 632
[15504:14192:1117/165727.782:ERROR:gpu_video_decode_accelerator.cc(381)] : HW video decode not available for profile vp8
[15504:11428:1117/170238.451:WARNING:angle_platform_impl.cc(41)] : rx::HLSLCompiler::compileToBinary(224): C:\fakepath(31,8-56): warning X3571: pow(f, e) will not work for negative f, use abs(f) or conditionally handle negative values if you expect them C:\fakepath(42,9-41): warning X3571: pow(f, e) will not work for negative f, use abs(f) or conditionally handle negative values if you expect them
[15504:2716:1117/170238.461:WARNING:angle_platform_impl.cc(41)] : rx::HLSLCompiler::compileToBinary(224): C:\fakepath(49,8-56): warning X3571: pow(f, e) will not work for negative f, use abs(f) or conditionally handle negative values if you expect them C:\fakepath(60,9-41): warning X3571: pow(f, e) will not work for negative f, use abs(f) or conditionally handle negative values if you expect them
[15504:11428:1117/170303.792:WARNING:angle_platform_impl.cc(41)] : rx::HLSLCompiler::compileToBinary(224): C:\fakepath(57,8-56): warning X3571: pow(f, e) will not work for negative f, use abs(f) or conditionally handle negative values if you expect them C:\fakepath(68,9-41): warning X3571: pow(f, e) will not work for negative f, use abs(f) or conditionally handle negative values if you expect them
[15504:14192:1117/170638.147:ERROR:gles2_cmd_decoder.cc(9798)] : [.Offscreen-For-WebGL-00000150DBE8F100]RENDER WARNING: texture bound to texture unit 0 is not renderable. It maybe non-power-of-2 and have incompatible texture filtering.
GpuProcessHostUIShim: The GPU process exited normally. Everything is okay.
 
Interestingly, I am not able to replicate this behavior when testing on Android Chrome and on Windows Chrome when I enable the Mobile testing in the Dev console.
And note that you won't be able to replicate this on PanoMoments.com as I've already disabled MSE decoding as a temp workaround. Only use https://jsfiddle.net/u3enjLzz/ for testing.
Owner: dalecur...@chromium.org
Status: Assigned (was: Unconfirmed)
This broke because we now require at least 2 frames to resume from underflow even for low delay. We assume that if you're underflowing you we need more data for a smooth playback.

I think we can improve this slightly to help this case and  issue 767878 , by relying on the AverageDuration() value provided from DecoderStream. I'll see if I can put a small patch together for this next week.

Comment 4 Deleted

Thanks for the background info Dale. Do you think the potential patch would allow for a 0 to infinite delay between subsequent frame appends to the buffer? In my use-case the decoder may be stalled for .1 seconds, 10 minutes, or 4 hours.
Not really any client side workaround for this in your use case. You would have to ensure you keep more frames in the queue to prevent underflow, but I think that's impossible for you since you allow going backwards.

https://chromium-review.googlesource.com/c/chromium/src/+/780267 fixes the original issue while returning low delay to 1 frame of latency.
Ok cool, I'll make sure to test 780267 as soon as it hits Canary.
Project Member

Comment 8 by bugdroid1@chromium.org, Nov 29 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/21ece16c7d9e0d06786579e7d86a7993a706f656

commit 21ece16c7d9e0d06786579e7d86a7993a706f656
Author: Dale Curtis <dalecurtis@chromium.org>
Date: Wed Nov 29 02:10:55 2017

Use video frame duration metadata instead of waiting for 2 frames.

http://crrev.com/493874 changed the low delay case to wait for two
frames since we need to be sure we don't resume until we actually
have valid frames since the first frame was valid forever without
duration information.

It turns out we actually do have duration infromation hanging off
the DecoderStream, so use this information and set it as the
FRAME_DURATION metadata key for each VideoFrame.

We can then have the algorithm use this information for the
estimated end time of frames when only a single frame is present
in the queue.

Note: The video-canvas layout test was showing the wrong frame;
I manually extracted the frames and verified that before we were
showing the "7" frame when pts=2.0s == "6" frame. I've updated
the test expectations appropriately.

BUG= 786576 , 767878 ,709302
TEST=new unittest, old unittests pass w/o modification, manual
test of 4k60 vp9 low latency content doesn't exhibit multiple
stalls after returning to the foreground when the video track
has been disabled, manual test with https://jsfiddle.net/u3enjLzz/

Cq-Include-Trybots: master.tryserver.chromium.android:android_optional_gpu_tests_rel;master.tryserver.chromium.linux:linux_optional_gpu_tests_rel;master.tryserver.chromium.mac:mac_optional_gpu_tests_rel;master.tryserver.chromium.win:win_optional_gpu_tests_rel
Change-Id: I963903d6a173038a2d534db4e040c8f4774825b5
Reviewed-on: https://chromium-review.googlesource.com/780267
Commit-Queue: Dale Curtis <dalecurtis@chromium.org>
Reviewed-by: Matthew Wolenetz <wolenetz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#519961}
[modify] https://crrev.com/21ece16c7d9e0d06786579e7d86a7993a706f656/media/filters/video_renderer_algorithm.cc
[modify] https://crrev.com/21ece16c7d9e0d06786579e7d86a7993a706f656/media/filters/video_renderer_algorithm.h
[modify] https://crrev.com/21ece16c7d9e0d06786579e7d86a7993a706f656/media/filters/video_renderer_algorithm_unittest.cc
[modify] https://crrev.com/21ece16c7d9e0d06786579e7d86a7993a706f656/media/renderers/video_renderer_impl.cc
[modify] https://crrev.com/21ece16c7d9e0d06786579e7d86a7993a706f656/media/renderers/video_renderer_impl.h
[modify] https://crrev.com/21ece16c7d9e0d06786579e7d86a7993a706f656/third_party/WebKit/LayoutTests/media/video-canvas.html
[delete] https://crrev.com/afedd691d118f767302c309ffa7b337ae07ca13e/third_party/WebKit/LayoutTests/platform/linux/media/video-canvas-expected.txt

Labels: M-64
Status: Fixed (was: Assigned)
Fixed, should be available in 64.0.3281.0 and higher.
Looks great in Canary!

Sign in to add a comment