New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 740007 link

Starred by 3 users

Issue metadata

Status: WontFix
Owner: ----
Closed: Dec 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 2
Type: Bug



Sign in to add a comment

MediaRecorder framerate drop

Reported by cgfarm...@gmail.com, Jul 7 2017

Issue description

UserAgent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36

Example URL:
provide privately

Steps to reproduce the problem:
1. Setup web page with navigator.getUserMedia to access video stream.
2. Set video stream to draw on canvas element from video src.
3. Use canvas element as texture in WebGL Renderer.
4. Canvas capture stream at 30 fps of renderer + audio. 
5. Use MediaRecorder to capture video / audio streams.
5. FPS stays steady on desktop chrome / FPS tanks on Android Chrome

What is the expected behavior?
FPS does not drop to below 30 while recording or stays steady.  Stays steady around 60 on Desktop.

What went wrong?
This does not seem to be a problem of the video buffer piling up its something to do with the recorder as I still noticed the drop after 
I was no longer adding data to the videoBuffer and the FPS still was badly effected on mobile. 

Did this work before? N/A 

Is it a problem with Flash or HTML5? HTML5

Does this work in other browsers? N/A

Chrome version: 59.0.3071.115  Channel: stable
OS Version: OS X 10.12.1
Flash Version: 

Contents of chrome://gpu: 
Graphics Feature Status
Canvas: Hardware accelerated
Flash: Hardware accelerated
Flash Stage3D: Hardware accelerated
Flash Stage3D Baseline profile: Hardware accelerated
Compositing: Hardware accelerated
Multiple Raster Threads: Enabled
Native GpuMemoryBuffers: Hardware accelerated
Rasterization: Hardware accelerated
Video Decode: Hardware accelerated
Video Encode: Hardware accelerated
WebGL: Hardware accelerated
WebGL2: Hardware accelerated
Driver Bug Workarounds
add_and_true_to_loop_condition
adjust_src_dst_region_for_blitframebuffer
decode_encode_srgb_for_generatemipmap
disable_framebuffer_cmaa
disable_multimonitor_multisampling
disable_webgl_rgb_multisampling_usage
emulate_abs_int_function
get_frag_data_info_bug
init_two_cube_map_levels_before_copyteximage
msaa_is_slow
pack_parameters_workaround_with_pack_buffer
rebind_transform_feedback_before_resume
regenerate_struct_names
remove_invariant_and_centroid_for_essl3
reset_base_mipmap_level_before_texstorage
rewrite_texelfetchoffset_to_texelfetch
scalarize_vec_and_mat_constructor_args
set_zero_level_before_generating_mipmap
unfold_short_circuit_as_ternary_operation
unpack_alignment_workaround_with_unpack_buffer
unpack_image_height_workaround_with_unpack_buffer
use_intermediary_for_copy_texture_image
use_shadowed_tex_level_params
use_unused_standard_shared_blocks
avoid_stencil_buffers
Problems Detected
Multisampling is buggy on OSX when multiple monitors are connected: 237931
Applied Workarounds: disable_multimonitor_multisampling
Unfold short circuit on Mac OS X: 307751
Applied Workarounds: unfold_short_circuit_as_ternary_operation
Always rewrite vec/mat constructors to be consistent: 398694
Applied Workarounds: scalarize_vec_and_mat_constructor_args
Mac drivers handle struct scopes incorrectly: 403957
Applied Workarounds: regenerate_struct_names
On Intel GPUs MSAA performance is not acceptable for GPU rasterization: 527565
Applied Workarounds: msaa_is_slow
glGenerateMipmap fails if the zero texture level is not set on some Mac drivers: 560499
Applied Workarounds: set_zero_level_before_generating_mipmap
Pack parameters work incorrectly with pack buffer bound: 563714
Applied Workarounds: pack_parameters_workaround_with_pack_buffer
Alignment works incorrectly with unpack buffer bound: 563714
Applied Workarounds: unpack_alignment_workaround_with_unpack_buffer
copyTexImage2D fails when reading from IOSurface on multiple GPU types.: 581777
Applied Workarounds: use_intermediary_for_copy_texture_image
Multisample renderbuffers with format GL_RGB8 have performance issues on Intel GPUs.: 607130
Applied Workarounds: disable_webgl_rgb_multisampling_usage
Mac Drivers store texture level parameters on int16_t that overflow: 610153
Applied Workarounds: use_shadowed_tex_level_params
Limited enabling of Chromium GL_INTEL_framebuffer_CMAA: 535198
Applied Workarounds: disable_framebuffer_cmaa
glGetFragData{Location|Index} works incorrectly on Max: 638340
Applied Workarounds: get_frag_data_info_bug
glResumeTransformFeedback works incorrectly on Intel GPUs: 638514
Applied Workarounds: rebind_transform_feedback_before_resume
glTexStorage* are buggy when base mipmap level is not 0: 640506
Applied Workarounds: reset_base_mipmap_level_before_texstorage
Result of abs(i) where i is an integer in vertex shader is wrong: 642227
Applied Workarounds: emulate_abs_int_function
Rewrite texelFetchOffset to texelFetch for Intel Mac: 642605
Applied Workarounds: rewrite_texelfetchoffset_to_texelfetch
Rewrite condition in for and while loops for Intel Mac: 644669
Applied Workarounds: add_and_true_to_loop_condition
Decode and encode before generateMipmap for srgb format textures on macosx: 634519
Applied Workarounds: decode_encode_srgb_for_generatemipmap
Init first two levels before CopyTexImage2D for cube map texture on Intel Mac 10.12: 648197
Applied Workarounds: init_two_cube_map_levels_before_copyteximage
Insert statements to reference all members in unused std140/shared blocks on Mac: 618464
Applied Workarounds: use_unused_standard_shared_blocks
Tex(Sub)Image3D performs incorrectly when uploading from unpack buffer with GL_UNPACK_IMAGE_HEIGHT greater than zero on Intel Macs: 654258
Applied Workarounds: unpack_image_height_workaround_with_unpack_buffer
adjust src/dst region if blitting pixels outside read framebuffer on Mac: 644740
Applied Workarounds: adjust_src_dst_region_for_blitframebuffer
Mac driver GL 4.1 requires invariant and centroid to match between shaders: 639760, 641129
Applied Workarounds: remove_invariant_and_centroid_for_essl3
Disable KHR_blend_equation_advanced until cc shaders are updated: 661715
Certain Apple devices leak stencil buffers: 713854
Applied Workarounds: avoid_stencil_buffers
Version Information
Data exported	7/6/2017, 9:59:56 PM
Chrome version	Chrome/59.0.3071.115
Operating system	Mac OS X 10.12.1
Software rendering list version	13.8
Driver bug list version	10.102
ANGLE commit id	a9042d3c1952
2D graphics backend	Skia/59 ef6f9c65527412ec4057ea0551f2e051beb94d32
Command Line Args	Chrome.app/Contents/MacOS/Google Chrome -psn_0_5952941 --flag-switches-begin --flag-switches-end
Driver Information
Initialization time	45
In-process GPU	false
Passthrough Command Decoder	false
Supports overlays	false
Sandboxed	true
GPU0	VENDOR = 0x8086, DEVICE= 0x162b *ACTIVE*
Optimus	false
Optimus	false
AMD switchable	false
Driver vendor	
Driver version	10.20.23
Driver date	
Pixel shader version	4.10
Vertex shader version	4.10
Max. MSAA samples	8
Machine model name	MacBookPro
Machine model version	12.1
GL_VENDOR	Intel Inc.
GL_RENDERER	Intel(R) Iris(TM) Graphics 6100
GL_VERSION	4.1 INTEL-10.20.23
GL_EXTENSIONS	GL_ARB_blend_func_extended GL_ARB_draw_buffers_blend GL_ARB_draw_indirect GL_ARB_ES2_compatibility GL_ARB_explicit_attrib_location GL_ARB_gpu_shader_fp64 GL_ARB_gpu_shader5 GL_ARB_instanced_arrays GL_ARB_internalformat_query GL_ARB_occlusion_query2 GL_ARB_sample_shading GL_ARB_sampler_objects GL_ARB_separate_shader_objects GL_ARB_shader_bit_encoding GL_ARB_shader_subroutine GL_ARB_shading_language_include GL_ARB_tessellation_shader GL_ARB_texture_buffer_object_rgb32 GL_ARB_texture_cube_map_array GL_ARB_texture_gather GL_ARB_texture_query_lod GL_ARB_texture_rgb10_a2ui GL_ARB_texture_storage GL_ARB_texture_swizzle GL_ARB_timer_query GL_ARB_transform_feedback2 GL_ARB_transform_feedback3 GL_ARB_vertex_attrib_64bit GL_ARB_vertex_type_2_10_10_10_rev GL_ARB_viewport_array GL_EXT_debug_label GL_EXT_debug_marker GL_EXT_framebuffer_multisample_blit_scaled GL_EXT_texture_compression_s3tc GL_EXT_texture_filter_anisotropic GL_EXT_texture_sRGB_decode GL_APPLE_client_storage GL_APPLE_container_object_shareable GL_APPLE_flush_render GL_APPLE_object_purgeable GL_APPLE_rgb_422 GL_APPLE_row_bytes GL_APPLE_texture_range GL_ATI_texture_mirror_once GL_NV_texture_barrier
Disabled Extensions	GL_KHR_blend_equation_advanced GL_KHR_blend_equation_advanced_coherent
Window system binding vendor	
Window system binding version	
Window system binding extensions	
Direct rendering	Yes
Reset notification strategy	0x0000
GPU process crash count	0
Compositor Information
Tile Update Mode	Zero-copy
Partial Raster	Enabled
GpuMemoryBuffers Status
ATC	Software only
ATCIA	Software only
DXT1	Software only
DXT5	Software only
ETC1	Software only
R_8	GPU_READ_CPU_READ_WRITE, GPU_READ_CPU_READ_WRITE_PERSISTENT
RG_88	Software only
BGR_565	Software only
RGBA_4444	Software only
RGBX_8888	Software only
RGBA_8888	GPU_READ, SCANOUT, SCANOUT_CPU_READ_WRITE
BGRX_8888	GPU_READ, SCANOUT, SCANOUT_CPU_READ_WRITE
BGRA_8888	GPU_READ, SCANOUT, SCANOUT_CPU_READ_WRITE, GPU_READ_CPU_READ_WRITE, GPU_READ_CPU_READ_WRITE_PERSISTENT
RGBA_F16	GPU_READ_CPU_READ_WRITE, GPU_READ_CPU_READ_WRITE_PERSISTENT
YVU_420	Software only
YUV_420_BIPLANAR	GPU_READ_CPU_READ_WRITE, GPU_READ_CPU_READ_WRITE_PERSISTENT
UYVY_422	GPU_READ_CPU_READ_WRITE, GPU_READ_CPU_READ_WRITE_PERSISTENT
Log Messages
[68314:49363:0705/193916.732672:ERROR:vt_video_decode_accelerator_mac.cc(655)] : Illegal attempt to decode without IDR. Discarding decode requests until next IDR.

What about an API similar to the Android MediaRecorder that has an off main thread callback with the data?

 I realize this might be a resources issue but seeing how the camera performs otherwise on Android I believe its a threading issue.
 

Comment 1 Deleted

The FPS never seem to recover to pre MediaRecorder states when it is done recording as well. Im not sure how to destroy the artifacts other than stopping the output and capture streams.
Cc: brajkumar@chromium.org
Labels: Needs-Feedback
Could you please provide a sample test case file to check this issue from Chrome-TE end?

Thanks!

Comment 4 by cgfarm...@gmail.com, Jul 10 2017

Attached is an APK with a ChromeClient embedded which exhibits the behavior.

1. Start MediaRecorder and FPS drops.
2. FPS never recovers.

I also emailed you a website with that exhibits the same behavior on a Pixel XL in Chrome.





app-debug.apk.zip
2.2 MB Download
Project Member

Comment 5 by sheriffbot@chromium.org, Jul 10 2017

Labels: -Needs-Feedback
Thank you for providing more feedback. Adding requester "brajkumar@chromium.org" to the cc list and removing "Needs-Feedback" label.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
Labels: -OS-Mac OS-Android
Adding Android OS label for further triaging of the issue.

Thanks!!
Components: -Internals>Media Blink>GetUserMedia
Components: -Blink>GetUserMedia Blink>MediaRecording
Ping for triaging.
Is this still valid?
Yes
Cc: niklase@chromium.org
Labels: Needs-Feedback
cgfarmer4@, anything that indicates that this is not just a CPU resource problem?
Its definitely a resource issue. But I think its just due to the way the pipeline is architected. 

I understand there are performance implications to using JavaScript and not building a native app but this seems like its something that could be fixed via an alternatively designed api.




Project Member

Comment 14 by sheriffbot@chromium.org, Oct 30 2017

Labels: -Needs-Feedback
Thank you for providing more feedback. Adding requester "niklase@chromium.org" to the cc list and removing "Needs-Feedback" label.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
Status: WontFix (was: Unconfirmed)
I still think this is just a CPU issue, wiring up HW H264 support would probably help here, but that would be a separate feature.

Sign in to add a comment