New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 702336 link

Starred by 2 users

Issue metadata

Status: Fixed
Owner:
Closed: May 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Mac
Pri: 3
Type: Bug

Blocked on:
issue 756537



Sign in to add a comment

WebglConformance_deqp_functional_gles3_shaderoperator_common_functions flaky on Mac AMD

Project Member Reported by ynovikov@chromium.org, Mar 16 2017

Issue description

For example https://build.chromium.org/p/chromium.gpu.fyi/builders/Mac%20GPU%20ASAN%20Release/builds/2052

[135/170] gpu_tests.webgl_conformance_integration_test.WebGLConformanceIntegrationTest.WebglConformance_deqp_functional_gles3_shaderoperator_common_functions failed unexpectedly 321.7956s:
  
  Traceback (most recent call last):
    _RunGpuTest at content/test/gpu/gpu_tests/gpu_integration_test.py:73
      self.RunActualGpuTest(url, *args)
    RunActualGpuTest at content/test/gpu/gpu_tests/webgl_conformance_integration_test.py:203
      getattr(self, test_name)(test_path, *args[1:])
    _RunConformanceTest at content/test/gpu/gpu_tests/webgl_conformance_integration_test.py:217
      self._CheckTestCompletion()
    _CheckTestCompletion at content/test/gpu/gpu_tests/webgl_conformance_integration_test.py:211
      'webglTestHarness._finished', timeout=300)
    traced_function at third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py:52
      return func(*args, **kwargs)
    WaitForJavaScriptCondition at third_party/catapult/telemetry/telemetry/internal/actions/action_runner.py:252
      return self._tab.WaitForJavaScriptCondition(*args, **kwargs)
    traced_function at third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py:52
      return func(*args, **kwargs)
    WaitForJavaScriptCondition at third_party/catapult/telemetry/telemetry/internal/browser/web_contents.py:191
      return self._inspector_backend.WaitForJavaScriptCondition(*args, **kwargs)
    traced_function at third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py:52
      return func(*args, **kwargs)
    WaitForJavaScriptCondition at third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_backend.py:294
      e.message + '\n' + debug_message)
  TimeoutException: Timed out while waiting 300s for IsJavaScriptExpressionTrue.
  Console output:
  
  Locals:
    IsJavaScriptExpressionTrue : <function IsJavaScriptExpressionTrue at 0x10ea9df50>
    condition                  : 'webglTestHarness._finished'
    context_id                 : None
    debug_message              : 'Console output:\n'
    e                          : TimeoutException('Timed out while waiting 300s for IsJavaScriptExpressionTrue.',)
    kwargs                     : {}
    timeout                    : 300
  
  Found crashpad_database_util
  No minidump found via crashpad_database_util
  No minidump paths to symbolize
  Found crashpad_database_util
  No minidump found via crashpad_database_util
  Restarting browser due to unexpected test failure
  Starting browser, attempt 1 of 3
  Requested remote debugging port: 0
  Starting Chrome ['/b/s/w/irzdtm7n/out/Release/Chromium.app/Contents/MacOS/Chromium', '--disable-gpu-watchdog', '--test-type=gpu', '--disable-domain-blocking-for-3d-apis', '--disable-gesture-requirement-for-media-playback', '--disable-gpu-process-crash-limit', '--enable-experimental-canvas-features', '--js-flags=--expose-gc', '--enable-es3-apis', '--enable-logging=stderr', '--enable-net-benchmarking', '--metrics-recording-only', '--no-default-browser-check', '--no-first-run', '--enable-gpu-benchmarking', '--disable-background-networking', '--proxy-server=socks://localhost:58091', '--ignore-certificate-errors', '--disable-component-extensions-with-background-pages', '--disable-default-apps', '--disable-search-geolocation-disclosure', '--remote-debugging-port=0', '--enable-crash-reporter-for-testing', '--window-size=1280,1024', '--user-data-dir=/b/s/w/itD3GsGW/tmp4Opw1q', 'about:blank']
  Discovered ephemeral port 58527
  OS: mac sierra
  Model: MacBookPro
  GPU device 0: VENDOR = 0x1002 (ATI), DEVICE = 0x6821
  GPU device 1: VENDOR = 0x8086 (Intel), DEVICE = 0xd26
  GPU Attributes:
    amd_switchable      : True
    basic_info_state    : 1
    context_info_state  : 1
    direct_rendering    : True
    driver_date         : 
    driver_vendor       : 
    driver_version      : 10.22.25
    encrypted_only      : False
    gl_extensions       : GL_ARB_blend_func_extended GL_ARB_draw_buffers_blend GL_ARB_draw_indirect GL_ARB_ES2_compatibility GL_ARB_explicit_attrib_location GL_ARB_gpu_shader_fp64 GL_ARB_gpu_shader5 GL_ARB_instanced_arrays GL_ARB_internalformat_query GL_ARB_occlusion_query2 GL_ARB_sample_shading GL_ARB_sampler_objects GL_ARB_separate_shader_objects GL_ARB_shader_bit_encoding GL_ARB_shader_subroutine GL_ARB_shading_language_include GL_ARB_tessellation_shader GL_ARB_texture_buffer_object_rgb32 GL_ARB_texture_cube_map_array GL_ARB_texture_gather GL_ARB_texture_query_lod GL_ARB_texture_rgb10_a2ui GL_ARB_texture_storage GL_ARB_texture_swizzle GL_ARB_timer_query GL_ARB_transform_feedback2 GL_ARB_transform_feedback3 GL_ARB_vertex_attrib_64bit GL_ARB_vertex_type_2_10_10_10_rev GL_ARB_viewport_array GL_EXT_debug_label GL_EXT_debug_marker GL_EXT_framebuffer_multisample_blit_scaled GL_EXT_texture_compression_s3tc GL_EXT_texture_filter_anisotropic GL_EXT_texture_sRGB_decode GL_APPLE_client_storage GL_APPLE_container_object_shareable GL_APPLE_flush_render GL_APPLE_object_purgeable GL_APPLE_rgb_422 GL_APPLE_row_bytes GL_APPLE_texture_range GL_ATI_texture_mirror_once GL_NV_texture_barrier
    gl_renderer         : Intel Iris Pro OpenGL Engine
    gl_reset_notification_strategy: 0
    gl_vendor           : Intel Inc.
    gl_version          : 4.1 INTEL-10.22.25
    gl_ws_extensions    : 
    gl_ws_vendor        : 
    gl_ws_version       : 
    in_process_gpu      : False
    initialization_time : 0.057081
    jpeg_decode_accelerator_supported: False
    max_framerate_denominator: 1
    max_framerate_numerator: 30
    max_msaa_samples    : 8
    max_resolution_height: 2160
    max_resolution_width: 4096
    min_resolution_height: 16
    min_resolution_width: 16
    optimus             : False
    passthrough_cmd_decoder: False
    pixel_shader_version: 4.10
    process_crash_count : 0
    profile             : 0
    sandboxed           : True
    software_rendering  : False
    vertex_shader_version: 4.10
    video_decode_accelerator_flags: 0
  Feature Status:
    2d_canvas           : enabled
    flash_3d            : enabled
    flash_stage3d       : enabled
    flash_stage3d_baseline: enabled
    gpu_compositing     : enabled
    multiple_raster_threads: enabled_on
    native_gpu_memory_buffers: enabled
    rasterization       : enabled
    video_decode        : enabled
    video_encode        : enabled
    vpx_decode          : enabled
    webgl               : enabled
    webgl2              : enabled
  Driver Bug Workarounds:
    add_and_true_to_loop_condition
    adjust_src_dst_region_for_blitframebuffer
    decode_encode_srgb_for_generatemipmap
    disable_framebuffer_cmaa
    disable_multimonitor_multisampling
    disable_webgl_rgb_multisampling_usage
    emulate_abs_int_function
    get_frag_data_info_bug
    init_two_cube_map_levels_before_copyteximage
    msaa_is_slow
    pack_parameters_workaround_with_pack_buffer
    rebind_transform_feedback_before_resume
    regenerate_struct_names
    remove_invariant_and_centroid_for_essl3
    reset_base_mipmap_level_before_texstorage
    rewrite_texelfetchoffset_to_texelfetch
    scalarize_vec_and_mat_constructor_args
    set_zero_level_before_generating_mipmap
    unfold_short_circuit_as_ternary_operation
    unpack_alignment_workaround_with_unpack_buffer
    unpack_image_height_workaround_with_unpack_buffer
    use_intermediary_for_copy_texture_image
    use_unused_standard_shared_blocks
  Started browser successfully.
  Traceback (most recent call last):
    File "/b/s/w/irzdtm7n/third_party/catapult/telemetry/telemetry/testing/serially_executed_browser_test_case.py", line 194, in <lambda>
      return lambda self: based_method(self, *args)
    File "/b/s/w/irzdtm7n/content/test/gpu/gpu_tests/gpu_integration_test.py", line 73, in _RunGpuTest
      self.RunActualGpuTest(url, *args)
    File "/b/s/w/irzdtm7n/content/test/gpu/gpu_tests/webgl_conformance_integration_test.py", line 203, in RunActualGpuTest
      getattr(self, test_name)(test_path, *args[1:])
    File "/b/s/w/irzdtm7n/content/test/gpu/gpu_tests/webgl_conformance_integration_test.py", line 217, in _RunConformanceTest
      self._CheckTestCompletion()
    File "/b/s/w/irzdtm7n/content/test/gpu/gpu_tests/webgl_conformance_integration_test.py", line 211, in _CheckTestCompletion
      'webglTestHarness._finished', timeout=300)
    File "/b/s/w/irzdtm7n/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 52, in traced_function
      return func(*args, **kwargs)
    File "/b/s/w/irzdtm7n/third_party/catapult/telemetry/telemetry/internal/actions/action_runner.py", line 252, in WaitForJavaScriptCondition
      return self._tab.WaitForJavaScriptCondition(*args, **kwargs)
    File "/b/s/w/irzdtm7n/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 52, in traced_function
      return func(*args, **kwargs)
    File "/b/s/w/irzdtm7n/third_party/catapult/telemetry/telemetry/internal/browser/web_contents.py", line 191, in WaitForJavaScriptCondition
      return self._inspector_backend.WaitForJavaScriptCondition(*args, **kwargs)
    File "/b/s/w/irzdtm7n/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 52, in traced_function
      return func(*args, **kwargs)
    File "/b/s/w/irzdtm7n/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_backend.py", line 294, in WaitForJavaScriptCondition
      e.message + '\n' + debug_message)
  TimeoutException: Timed out while waiting 300s for IsJavaScriptExpressionTrue.
  Console output:
  
[25482:49683:0315/123515.929712:WARNING:simple_synchronous_entry.cc(1054)] Could not open platform files for entry.
 
Summary: WebglConformance_deqp_functional_gles3_shaderoperator_common_functions flaky on Mac AMD (was: WebglConformance_deqp_functional_gles3_shaderoperator_common_functions flaky on Mac GPU ASAN Release)
Note that this has flaked in "webgl2_conformance_tests on ATI GPU on Mac Retina" step.

Looks like this test has a history of failing on Mac AMD
https://codereview.chromium.org/2122493002
But, no bug reports associated with it.

Going to mark as Flaky and see what happens.
Project Member

Comment 2 by bugdroid1@chromium.org, Mar 17 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/373cfc2e3bd3da59db2ed3356d2022a0c092c740

commit 373cfc2e3bd3da59db2ed3356d2022a0c092c740
Author: ynovikov <ynovikov@chromium.org>
Date: Fri Mar 17 02:21:36 2017

Suppress some flaky WebGL and WebGL2 tests

conformance/glsl/misc/glsl-long-variable-names.html
conformance/textures/image_bitmap_from_video/tex-2d-rgb-rgb-unsigned_byte.html
on NVIDIA Shield

deqp/functional/gles3/shaderoperator/common_functions.html
deqp/functional/gles3/shaderindexing/mat_01.html
on Mac AMD

BUG=636648,  701929 ,  702336 
TBR=kbr@chromium.org
CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel;master.tryserver.chromium.linux:linux_optional_gpu_tests_rel;master.tryserver.chromium.mac:mac_optional_gpu_tests_rel;master.tryserver.chromium.win:win_optional_gpu_tests_rel

Review-Url: https://codereview.chromium.org/2754853004
Cr-Commit-Position: refs/heads/master@{#457659}

[modify] https://crrev.com/373cfc2e3bd3da59db2ed3356d2022a0c092c740/content/test/gpu/gpu_tests/webgl2_conformance_expectations.py
[modify] https://crrev.com/373cfc2e3bd3da59db2ed3356d2022a0c092c740/content/test/gpu/gpu_tests/webgl_conformance_expectations.py

Cc: kbr@chromium.org zmo@chromium.org kainino@chromium.org
Status: Available (was: Untriaged)

Comment 4 by kbr@chromium.org, Jul 24 2017

Cc: piman@chromium.org cwallez@chromium.org
 Issue 748136  has been merged into this issue.
Project Member

Comment 5 by bugdroid1@chromium.org, Jul 24 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/ae0474c3546441ef731b5d79908f733c9e5b586d

commit ae0474c3546441ef731b5d79908f733c9e5b586d
Author: Kenneth Russell <kbr@chromium.org>
Date: Mon Jul 24 22:15:32 2017

Suppress shaderoperator_common_functions flakes on Mac/NVIDIA.

BUG= 702336 
TBR=zmo@chromium.org
NOTRY=true

Cq-Include-Trybots: master.tryserver.chromium.android:android_optional_gpu_tests_rel;master.tryserver.chromium.linux:linux_optional_gpu_tests_rel;master.tryserver.chromium.mac:mac_optional_gpu_tests_rel;master.tryserver.chromium.win:win_optional_gpu_tests_rel
Change-Id: I8effde179586e867f18773dd363f1ceb96914f97
Reviewed-on: https://chromium-review.googlesource.com/583444
Commit-Queue: Kenneth Russell <kbr@chromium.org>
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Cr-Commit-Position: refs/heads/master@{#489094}
[modify] https://crrev.com/ae0474c3546441ef731b5d79908f733c9e5b586d/content/test/gpu/gpu_tests/webgl2_conformance_expectations.py

Project Member

Comment 6 by bugdroid1@chromium.org, Jul 26 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/5ad9ff6c130597b324fe94e4decc5a9c64b07218

commit 5ad9ff6c130597b324fe94e4decc5a9c64b07218
Author: Kenneth Russell <kbr@chromium.org>
Date: Wed Jul 26 00:38:39 2017

Mark shaderoperator_common_functions failing on Mac NVIDIA.

The retries aren't working on this configuration. Mark it flaky again
on Mac/AMD.

BUG= 702336 
TBR=zmo@chromium.org
NOTRY=true

Cq-Include-Trybots: master.tryserver.chromium.android:android_optional_gpu_tests_rel;master.tryserver.chromium.linux:linux_optional_gpu_tests_rel;master.tryserver.chromium.mac:mac_optional_gpu_tests_rel;master.tryserver.chromium.win:win_optional_gpu_tests_rel
Change-Id: Ie163baa0e9e802253eec65ede94362155a6dd537
Reviewed-on: https://chromium-review.googlesource.com/585831
Commit-Queue: Kenneth Russell <kbr@chromium.org>
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Cr-Commit-Position: refs/heads/master@{#489499}
[modify] https://crrev.com/5ad9ff6c130597b324fe94e4decc5a9c64b07218/content/test/gpu/gpu_tests/webgl2_conformance_expectations.py

Project Member

Comment 7 by bugdroid1@chromium.org, Aug 7 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/f0a3d267bb925a8334981dfe55e3820ddc3c7410

commit f0a3d267bb925a8334981dfe55e3820ddc3c7410
Author: Eric Karl <ericrk@chromium.org>
Date: Mon Aug 07 23:35:22 2017

Mark shaderoperator common functions test as failing on Mac AMD/Intel

AMD was already marked flaky, and timing-wise was close to the 300s
timeout. It appears to have been pushed over the timeout, and is now
consistently failing. Intel wasn't marked flaky, but was actually flaky
already (1/10 of the time). It now appears to fail 1/3 of the time.

Marking AMD as failing, Intel as flaky.

Bug:  702336 
Cq-Include-Trybots: master.tryserver.chromium.android:android_optional_gpu_tests_rel;master.tryserver.chromium.linux:linux_optional_gpu_tests_rel;master.tryserver.chromium.mac:mac_optional_gpu_tests_rel;master.tryserver.chromium.win:win_optional_gpu_tests_rel
Change-Id: I19a7cb40f4dfc8793b384978284e7632c9095918
Reviewed-on: https://chromium-review.googlesource.com/604468
Reviewed-by: Zhenyao Mo <zmo@chromium.org>
Commit-Queue: Eric Karl <ericrk@chromium.org>
Cr-Commit-Position: refs/heads/master@{#492450}
[modify] https://crrev.com/f0a3d267bb925a8334981dfe55e3820ddc3c7410/content/test/gpu/gpu_tests/webgl2_conformance_expectations.py

Project Member

Comment 8 by bugdroid1@chromium.org, Aug 10 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/938e2fc0adf875e3d8b9a64d47213a6b2d09d751

commit 938e2fc0adf875e3d8b9a64d47213a6b2d09d751
Author: Eric Karl <ericrk@chromium.org>
Date: Thu Aug 10 00:45:26 2017

Increase Asan webgl conformance test timeout

We had disabled/marked-flaky a number of mac tests due to
timeouts on the Asan bot. This change increases the timeout
limit on Asan configs, and removes the failures/flakes from
expectations.

Bug:  702336 
Cq-Include-Trybots: master.tryserver.chromium.android:android_optional_gpu_tests_rel;master.tryserver.chromium.linux:linux_optional_gpu_tests_rel;master.tryserver.chromium.mac:mac_optional_gpu_tests_rel;master.tryserver.chromium.win:win_optional_gpu_tests_rel
Change-Id: Icc35980295c7ee014d19de975d818aa8ebae4869
Reviewed-on: https://chromium-review.googlesource.com/608740
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Commit-Queue: Eric Karl <ericrk@chromium.org>
Cr-Commit-Position: refs/heads/master@{#493194}
[modify] https://crrev.com/938e2fc0adf875e3d8b9a64d47213a6b2d09d751/content/test/gpu/gpu_tests/webgl2_conformance_expectations.py
[modify] https://crrev.com/938e2fc0adf875e3d8b9a64d47213a6b2d09d751/content/test/gpu/gpu_tests/webgl_conformance_integration_test.py

Comment 9 by kbr@chromium.org, Aug 17 2017

Blockedon: 756537

Comment 10 by kbr@chromium.org, Aug 18 2017

Owner: ericrk@chromium.org
Status: Fixed (was: Available)
With the suppression added back in for Mac NVIDIA, this should no longer be flaky. Thanks ericrk@ for fixing the ASAN timeout, which was the main issue.

Labels: -Pri-3 Hotlist-PixelWrangler Pri-1
Status: Assigned (was: Fixed)
This is still flaking. Only seen on Intel as of yet:

https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Mac%20FYI%20Release%20%28Intel%29/3320

[106/135] gpu_tests.webgl_conformance_integration_test.WebGLConformanceIntegrationTest.WebglConformance_deqp_functional_gles3_shaderoperator_common_functions failed unexpectedly 968.6528s:

Dashboard link:

https://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=webgl2_conformance_tests&tests=gpu_tests.webgl_conformance_integration_test.WebGLConformanceIntegrationTest.WebglConformance_deqp_functional_gles3_shaderoperator_common_functions

Flakes on ANGLE CQ:

https://ci.chromium.org/p/chromium/builders/luci.chromium.try/mac_angle_rel_ng/1266
https://ci.chromium.org/p/chromium/builders/luci.chromium.try/mac_angle_rel_ng/1262

Eric can you investigate?
Labels: -Pri-1 -Hotlist-PixelWrangler Pri-3
Status: Fixed (was: Assigned)
Oh wow, this issue is way too old. Sorry, will file a new issue.

Sign in to add a comment