I should add, it broke in Chrome 65 on both Windows and MacOS.

jlangeway@, can you please attach screenshots (Actual Vs Expected) if possible? some reason i couldn't see the difference b/n M64 and other latest channels.

Thank you!

jlangeway, could you also provide the contents of chrome://gpu on the affected system(s)? Thanks.

Actually, I was able to reproduce a difference between Chrome 65 and Safari on my MacBook Air (see attachments). I didn't test Chrome 64.

I'll see if I can do a bisect on my machine since it may reproduce the issue.

Deleted: about-gpu-macbookair-chrome66.txt 12.2 KB

about-gpu-macbookair-chrome66.txt 12.2 KB View Download

Deleted: b835363_safari.png 253 KB

	b835363_safari.png 253 KB View Download

Deleted: b835363_chrome65.png 247 KB

	b835363_chrome65.png 247 KB View Download

Able to reproduce the difference between 64 and 65. Bisecting.

Bisected to this ANGLE roll:
https://chromium.googlesource.com/chromium/src/+/1670e2ff12ad1b5666929a76ed8210229d24d916
Consisting of this commit:
https://chromium.googlesource.com/angle/angle.git/+/ea78d2bb7a07660301ed407d4618f008b9f4395f

Olli, please take a look, thanks!

kainino@, thank you so much for the bisect. Do you think this need to be considered for a stable re-spin? since this exists in previous M65 stable too.

No, it's not so urgent that it should be considered for stable respin. We will probably target 67.

cool, thank you!

I think y'all have it in hand, thank you!, just in case, please find attached screen shots of the example page in Chrome version 64 vs Chrome version 66.

The contents of chrome://gpu/ on my machine follows.

Graphics Feature Status
Canvas: Hardware accelerated
CheckerImaging: Disabled
Flash: Hardware accelerated
Flash Stage3D: Hardware accelerated
Flash Stage3D Baseline profile: Hardware accelerated
Compositing: Hardware accelerated
Multiple Raster Threads: Enabled
Native GpuMemoryBuffers: Hardware accelerated
Rasterization: Unavailable
Surface Synchronization: Enabled
Video Decode: Hardware accelerated
WebGL: Hardware accelerated
WebGL2: Hardware accelerated
Driver Bug Workarounds
adjust_src_dst_region_for_blitframebuffer
avoid_stencil_buffers
decode_encode_srgb_for_generatemipmap
depth_stencil_renderbuffer_resize_emulation
disable_framebuffer_cmaa
dont_use_loops_to_initialize_variables
get_frag_data_info_bug
needs_offscreen_buffer_workaround
pack_parameters_workaround_with_pack_buffer
regenerate_struct_names
remove_invariant_and_centroid_for_essl3
scalarize_vec_and_mat_constructor_args
set_zero_level_before_generating_mipmap
unfold_short_circuit_as_ternary_operation
unpack_alignment_workaround_with_unpack_buffer
use_intermediary_for_copy_texture_image
use_unused_standard_shared_blocks
Problems Detected
Macs with NVidia GPUs experience rendering issues on High Sierra: 773705
Disabled Features: gpu_rasterization
Work around a bug in offscreen buffers on NVIDIA GPUs on Macs: 89557
Applied Workarounds: needs_offscreen_buffer_workaround
Unfold short circuit on Mac OS X: 307751
Applied Workarounds: unfold_short_circuit_as_ternary_operation
Always rewrite vec/mat constructors to be consistent: 398694
Applied Workarounds: scalarize_vec_and_mat_constructor_args
Mac drivers handle struct scopes incorrectly: 403957
Applied Workarounds: regenerate_struct_names
glGenerateMipmap fails if the zero texture level is not set on some Mac drivers: 560499
Applied Workarounds: set_zero_level_before_generating_mipmap
Pack parameters work incorrectly with pack buffer bound: 563714
Applied Workarounds: pack_parameters_workaround_with_pack_buffer
Alignment works incorrectly with unpack buffer bound: 563714
Applied Workarounds: unpack_alignment_workaround_with_unpack_buffer
copyTexImage2D fails when reading from IOSurface on multiple GPU types.: 581777
Applied Workarounds: use_intermediary_for_copy_texture_image
Use GL_INTEL_framebuffer_CMAA on ChromeOS: 535198
Applied Workarounds: disable_framebuffer_cmaa
glGetFragData{Location|Index} works incorrectly on Max: 638340
Applied Workarounds: get_frag_data_info_bug
Decode and encode before generateMipmap for srgb format textures on macosx: 634519
Applied Workarounds: decode_encode_srgb_for_generatemipmap
Insert statements to reference all members in unused std140/shared blocks on Mac: 618464
Applied Workarounds: use_unused_standard_shared_blocks
adjust src/dst region if blitting pixels outside read framebuffer on Mac: 644740
Applied Workarounds: adjust_src_dst_region_for_blitframebuffer
Mac driver GL 4.1 requires invariant and centroid to match between shaders: 639760, 641129
Applied Workarounds: remove_invariant_and_centroid_for_essl3
Disable KHR_blend_equation_advanced until cc shaders are updated: 661715
Applied Workarounds: disable(GL_KHR_blend_equation_advanced), disable(GL_KHR_blend_equation_advanced_coherent)
Certain Apple devices leak stencil buffers: 713854
Applied Workarounds: avoid_stencil_buffers
Depth/stencil renderbuffers can't be resized on NVIDIA on early macOS 10.13: 775202
Applied Workarounds: depth_stencil_renderbuffer_resize_emulation
Don't expose disjoint_timer_query extensions to WebGL: 808744
Shader variable initialization in a loop caused perf regression on Mac Intel.: 809422
Applied Workarounds: dont_use_loops_to_initialize_variables
Checker-imaging has been disabled via finch trial or the command line.
Disabled Features: checker_imaging
Version Information
Data exported	2018-04-20T22:37:49.586Z
Chrome version	Chrome/66.0.3359.117
Operating system	Mac OS X 10.13.3
Software rendering list URL	https://chromium.googlesource.com/chromium/src/+/7f59c28e25283df88e0c6ac8d8b2551d8c6ad93b/gpu/config/software_rendering_list.json
Driver bug list URL	https://chromium.googlesource.com/chromium/src/+/7f59c28e25283df88e0c6ac8d8b2551d8c6ad93b/gpu/config/gpu_driver_bug_list.json
ANGLE commit id	22c768fbda54
2D graphics backend	Skia/66 773868fdade5f9f0e7697e6d09c9bd80aaa9b402-
Command Line	/Applications/Google Chrome.app/Contents/MacOS/Google Chrome --flag-switches-begin --flag-switches-end
Driver Information
Initialization time	45
In-process GPU	false
Passthrough Command Decoder	false
Direct Composition	false
Supports overlays	false
Sandboxed	true
GPU0	VENDOR = 0x10de, DEVICE= 0x0fe9 *ACTIVE*
GPU1	VENDOR = 0x8086, DEVICE= 0x0d26
Optimus	true
Optimus	true
AMD switchable	false
Driver vendor	
Driver version	10.28.10 355.11.10.10.20.111
Driver date	
Pixel shader version	4.10
Vertex shader version	4.10
Max. MSAA samples	8
Machine model name	MacBookPro
Machine model version	11.3
GL_VENDOR	NVIDIA Corporation
GL_RENDERER	NVIDIA GeForce GT 750M OpenGL Engine
GL_VERSION	4.1 NVIDIA-10.28.10 355.11.10.10.20.111
GL_EXTENSIONS	GL_ARB_blend_func_extended GL_ARB_draw_buffers_blend GL_ARB_draw_indirect GL_ARB_ES2_compatibility GL_ARB_explicit_attrib_location GL_ARB_gpu_shader_fp64 GL_ARB_gpu_shader5 GL_ARB_instanced_arrays GL_ARB_internalformat_query GL_ARB_occlusion_query2 GL_ARB_sample_shading GL_ARB_sampler_objects GL_ARB_separate_shader_objects GL_ARB_shader_bit_encoding GL_ARB_shader_subroutine GL_ARB_shading_language_include GL_ARB_tessellation_shader GL_ARB_texture_buffer_object_rgb32 GL_ARB_texture_cube_map_array GL_ARB_texture_gather GL_ARB_texture_query_lod GL_ARB_texture_rgb10_a2ui GL_ARB_texture_storage GL_ARB_texture_swizzle GL_ARB_timer_query GL_ARB_transform_feedback2 GL_ARB_transform_feedback3 GL_ARB_vertex_attrib_64bit GL_ARB_vertex_type_2_10_10_10_rev GL_ARB_viewport_array GL_EXT_debug_label GL_EXT_debug_marker GL_EXT_depth_bounds_test GL_EXT_framebuffer_multisample_blit_scaled GL_EXT_texture_compression_s3tc GL_EXT_texture_filter_anisotropic GL_EXT_texture_mirror_clamp GL_EXT_texture_sRGB_decode GL_APPLE_client_storage GL_APPLE_container_object_shareable GL_APPLE_flush_render GL_APPLE_object_purgeable GL_APPLE_rgb_422 GL_APPLE_row_bytes GL_APPLE_texture_range GL_ATI_texture_mirror_once GL_NV_texture_barrier
Disabled Extensions	GL_KHR_blend_equation_advanced GL_KHR_blend_equation_advanced_coherent
Disabled WebGL Extensions	EXT_disjoint_timer_query EXT_disjoint_timer_query_webgl2
Window system binding vendor	
Window system binding version	
Window system binding extensions	
Direct rendering	Yes
Reset notification strategy	0x0000
GPU process crash count	0
Compositor Information
Tile Update Mode	Zero-copy
Partial Raster	Enabled
GpuMemoryBuffers Status
ATC	Software only
ATCIA	Software only
DXT1	Software only
DXT5	Software only
ETC1	Software only
R_8	GPU_READ, SCANOUT, SCANOUT_CPU_READ_WRITE, GPU_READ_CPU_READ_WRITE, GPU_READ_CPU_READ_WRITE_PERSISTENT
R_16	Software only
RG_88	Software only
BGR_565	Software only
RGBA_4444	Software only
RGBX_8888	Software only
RGBA_8888	GPU_READ, SCANOUT, SCANOUT_CPU_READ_WRITE, GPU_READ_CPU_READ_WRITE, GPU_READ_CPU_READ_WRITE_PERSISTENT
BGRX_8888	GPU_READ, SCANOUT, SCANOUT_CPU_READ_WRITE
BGRX_1010102	GPU_READ, SCANOUT, SCANOUT_CPU_READ_WRITE, GPU_READ_CPU_READ_WRITE, GPU_READ_CPU_READ_WRITE_PERSISTENT
RGBX_1010102	Software only
BGRA_8888	GPU_READ, SCANOUT, SCANOUT_CPU_READ_WRITE, GPU_READ_CPU_READ_WRITE, GPU_READ_CPU_READ_WRITE_PERSISTENT
RGBA_F16	GPU_READ, SCANOUT, SCANOUT_CPU_READ_WRITE, GPU_READ_CPU_READ_WRITE, GPU_READ_CPU_READ_WRITE_PERSISTENT
YVU_420	Software only
YUV_420_BIPLANAR	GPU_READ, SCANOUT, SCANOUT_CPU_READ_WRITE, GPU_READ_CPU_READ_WRITE, GPU_READ_CPU_READ_WRITE_PERSISTENT
UYVY_422	GPU_READ, SCANOUT, SCANOUT_CPU_READ_WRITE, GPU_READ_CPU_READ_WRITE, GPU_READ_CPU_READ_WRITE_PERSISTENT
Display(s) Information
Info	Display[722496016] bounds=0,0 2560x1440, workarea=0,23 2560x1359, scale=2, external
Color space information	{primaries:[[0.4410,0.3698,0.1534,],[0.2236,0.7122,0.0642,],[0.0060,0.0551,0.7638,],], transfer:0.0774*x + 0.0000 if x < 0.0404 else (0.9479*x + 0.0521)**2.4000 + 0.0000, matrix:RGB, range:FULL}
Bits per color component	8
Bits per pixel	24
Info	Display[722494479] bounds=2560,0 2560x1440, workarea=2560,23 2560x1417, scale=2, external
Color space information	{primaries:[[0.4410,0.3698,0.1534,],[0.2236,0.7122,0.0642,],[0.0060,0.0551,0.7638,],], transfer:0.0774*x + 0.0000 if x < 0.0404 else (0.9479*x + 0.0521)**2.4000 + 0.0000, matrix:RGB, range:FULL}
Bits per color component	8
Bits per pixel	24
Video Acceleration Information
Decode h264 baseline	up to 4096x2160 pixels
Decode h264 extended	up to 4096x2160 pixels
Decode h264 main	up to 4096x2160 pixels
Decode h264 high	up to 4096x2160 pixels
Encode h264 baseline	up to 4096x2160 pixels and/or 30.000 fps
Encode h264 main	up to 4096x2160 pixels and/or 30.000 fps
Encode h264 high	up to 4096x2160 pixels and/or 30.000 fps

Deleted: chrome64.png 3.4 MB

	chrome64.png 3.4 MB View Download

Deleted: Chrom66.png 3.2 MB

	Chrom66.png 3.2 MB View Download

-OS-Mac since it's apparently affecting Windows.

On Linux/NVIDIA, I'm actually unable to get the correct result even on a Chromium revision which passed on Mac/Intel. I don't know why.

Perhaps there is some undefined behavior in the shader itself?

I removed all unused variables from the shader code, including making dummy use of vertex attributes, and now it works on my local machine.

@kainino You've made my weekend. Thank you!

The real problem appears to me to be that Chrome silently fails when a shader has unused variables, but that is easy to work around and really for all I know is the intended behavior. It looks like the problem has nothing to do with stencils, we just had several unused variables in the shader doing the stencil.

@all Thank you! Thank you! Thank you!

Glad to help! Now that I think about it, I think we recently saw another issue that was related to unused variable elimination - but I think it was just an attribute or uniform location issue (bug in the application).

I'm not sure if there should have been an error or warning produced here - but generally the shader should work the same with extra unused variables in it.
jlangeway, I haven't looked at the shader - do you think this is still a browser bug?

If you could show us the change that you made to your shader (before and after), it would be really helpful. Thanks!

Yes, shaders should still work the same even with unused variables in them (unless the unused variables introduce variable location conflicts, going over max limits or such). This does sound like a possible bug in the unused variable elimination. I'd really appreciate the before/after shaders to be able to better debug this.

@oetu... I apologize, I did not get approval to share any actual shader code.

The shader that didn't work had:

1. A `vec4[5]` uniform we'll call `colorTable` which wasn't needed for the stenciling operation the shader was for.

2. A `float` vertex attribute we'll call `colorIndex` which also wasn't needed.

We had observed in Chrome 64 that the shader didn't work without a line in it to the effect of `vec4 color = colorTable[colorIndex];` where `color` later didn't get used. It was convenient at the time to have the same uniforms and attributes in both a stencil program and a color program, so we left that voodoo in. With Chrome 65, it stopped working.

To fix it, we removed the unused uniform and made a trivial use of the `colorIndex` attribute as in something like `gl_position.z += clamp(0.0, 1.0, 0.000000001 * colorIndex);`. We later changed to construct different `Cesium.VertexArray` objects for each program so they did not receive the same attributes and we dropped the trivial use of `colorIndex`.

Cesium makes it tempting to send extra uniforms and attributes with it's object oriented API. We know better now.

I'm going to lower the priority on this for now; probably not P1.

I'm not able to reproduce any incorrect behavior based on the information provided, using ANGLE's shader_translator utility. There is a possibility that this was an attribute or uniform location related bug in the application.

This could only be verified if we were given the full shader code, but I'm closing this as WontFix based on inability to reproduce.