New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 884210 link

Starred by 2 users

Issue metadata

Status: ExternalDependency
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Windows
Pri: 1
Type: Bug-Regression

Blocking:
issue 849576
issue 884043



Sign in to add a comment

gl_FragCoord_001_to_003 causes failures in subsequent no-over-optimization-on-uniform-array-* tests during webgl2_conformance_gl_passthrough_tests on Win Intel

Project Member Reported by jmad...@chromium.org, Sep 14

Issue description

This seems to be failing reliably since 

https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Win10%20FYI%20Release%20%28Intel%20HD%20630%29/2041

Possibly related to "Expose KHR_parallel_shader_compile to WebGL" by jie.a.chen@intel.com ?

Regression range:

https://crrev.com/591306..591321

Also affecting ANGLE CQ.
 
Blocking: 884043
Interestingly, I saw reliable failures of WebglConformance_conformance_uniforms_no_over_optimization_on_uniform_array_07 in webgl2_conformance_gl_passthrough_tests on the try jobs for this WebGL conformance roll:

https://chromium-review.googlesource.com/c/chromium/src/+/1225605

https://ci.chromium.org/p/chromium/builders/luci.chromium.try/win_angle_rel_ng/2981
https://ci.chromium.org/p/chromium/builders/luci.chromium.try/win_angle_rel_ng/2982
https://ci.chromium.org/p/chromium/builders/luci.chromium.try/win_angle_rel_ng/2983

which was filed as  Issue 884043 .

The earliest of those failures happened at r591199 however which happened before https://chromium.googlesource.com/chromium/src/+/5ec934e8bc4037d70c87149f23a8cfeb074b4b14 .

Need to track down what's going on here. Could there be a state management bug in the OpenGL backend? Or a bug in Intel's driver?

Owner: kainino@chromium.org
Status: Assigned (was: Untriaged)
Assigning to current wrangler.
Project Member

Comment 4 by bugdroid1@chromium.org, Sep 14

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/ee176c8a7622bbbf17cd4ee85041d5d093fafb08

commit ee176c8a7622bbbf17cd4ee85041d5d093fafb08
Author: Jamie Madill <jmadill@chromium.org>
Date: Fri Sep 14 18:52:07 2018

Suppress failure in Win/Intel WebGL UBO test.

no-over-optimization-on-uniform-array-06 is failing consistently.

Bug: 884210
Cq-Include-Trybots: luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel
Change-Id: I2cbead8a26e4883b221be9f873f711ba3aeb84ef
Tbr: kbr@chromium.org
No-try: True
Reviewed-on: https://chromium-review.googlesource.com/1226308
Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Commit-Queue: Yuly Novikov <ynovikov@chromium.org>
Cr-Commit-Position: refs/heads/master@{#591415}
[modify] https://crrev.com/ee176c8a7622bbbf17cd4ee85041d5d093fafb08/content/test/gpu/gpu_tests/webgl2_conformance_expectations.py

Blocking: 849576
Labels: -Type-Bug -Hotlist-PixelWrangler Type-Bug-Regression
Owner: jie.a.c...@intel.com
Awesome, thanks Jamie for taking care of this. I'll watch the affected bots to make sure they go green again.

Assigning to jie.a.chen to look into the problem.
I found a same "ANGLE (Intel Intel(R) HD Graphics 630 OpenGL 4.5 core)" machine and run the case manually dozens of times, but failed to reproduce the error. I am not quite sure about the difference between the bots and my machine. Could someone help to update the bots with the latest Intel driver? Alternatively please tell me the specific driver version number like "24.20.100.6286", so that I will be able to revert to the same driver on my machine to reproduce.
For the "Expose KHR_parallel_shader_compile to WebGL", It's hard to for me speculate a potential cause. As the case doesn't turn on the extension, the code in the patch won't run at all. Besides, currently we have only enabled the parallel compilation for D3D backend in ANGLE. For Windows GL, we still compile synchronously as usual.
I think the driver version is 23.20.16.4877.
Take note this was failing only on webgl2_conformance_gl_passthrough_tests, so are you passing all the relevant arguments, i.e. :
run_gpu_integration_test.py webgl_conformance --show-stdout --browser=release --passthrough -v --extra-browser-args="--enable-logging=stderr --js-flags=--expose-gc --use-gl=angle --use-angle=gl --use-cmd-decoder=passthrough" --webgl-conformance-version=2.0.1
This failure may rely on the tests being run in a specific order. In  Issue 884043  WebglConformance_conformance_uniforms_no_over_optimization_on_uniform_array_07 became flaky (or started failing consistently) after the addition of a new test caused the tests to be redistributed among shards.

In order to replicate this behavior it'd be necessary to run exactly those tests in the same order. This can be done by adding the following command line arguments:

--total-shards=20 --shard-index=15

These flags are usually picked up from the environment here:

https://cs.chromium.org/chromium/src/testing/scripts/run_gpu_integration_test_as_googletest.py?q=run_gpu_int&sq=package:chromium&g=0&l=99

You can then probably start reducing the tests which run using the --test-filter argument. Note that you will probably also need the --filter-tests-after-sharding argument in this case:

https://cs.chromium.org/chromium/src/third_party/catapult/telemetry/telemetry/testing/run_browser_tests.py?q=run_browser_tests.py&sq=package:chromium&g=0&l=191

Thanks for the instructions. I was able to run the scripts. But after updating to the TOT of master, I have seen the websocket errors below:

INFO:root:DoNothingForwarder started between 127.0.0.1:54149 and 54149
ERROR:root:Unexpected error checking if ws://127.0.0.1:54149/devtools/browser/7a2017f4-4b03-4efd-be05-2fa11cdfb7ed is ready.
Traceback (most recent call last):
  File "D:\workspace\jie\chromium\src\third_party\catapult\telemetry\telemetry\internal\backends\chrome_inspector\devtools_client_backend.py", line 148, in _IsInspectorWebsocketReady
    ws.Connect(self.browser_target_url, timeout=10)
  File "D:\workspace\jie\chromium\src\third_party\catapult\telemetry\telemetry\internal\backends\chrome_inspector\inspector_websocket.py", line 84, in Connect
    skip_utf8_validation=True)
  File "D:\workspace\jie\chromium\src\third_party\catapult\telemetry\telemetry\internal\backends\chrome_inspector\websocket.py", line 25, in CreateConnection
    return _create_connection(*args, **kwargs)
  File "D:\workspace\jie\chromium\src\third_party\catapult\telemetry\third_party\websocket-client\websocket\_core.py", line 487, in create_connection
    websock.connect(url, **options)
  File "D:\workspace\jie\chromium\src\third_party\catapult\telemetry\third_party\websocket-client\websocket\_core.py", line 214, in connect
    self.handshake_response = handshake(self.sock, *addrs, **options)
  File "D:\workspace\jie\chromium\src\third_party\catapult\telemetry\third_party\websocket-client\websocket\_handshake.py", line 65, in handshake
    status, resp = _get_resp_headers(sock)
  File "D:\workspace\jie\chromium\src\third_party\catapult\telemetry\third_party\websocket-client\websocket\_handshake.py", line 122, in _get_resp_headers
    raise WebSocketBadStatusException("Handshake status %d", status)

Have you ever seen such errors? Could this be a new bug in master?
Components: Tests>Telemetry
I've never seen that error. The bots are all testing ToT so I'd be surprised if something was broken at that level. Do you have some sort of firewall running on your workstations that might be interfering?

Oops! It was my proxy environmental variables. I can reproduce the error now, both on "UHD Graphics 630" and "HD Graphics 630" with the latest Intel driver.
The affecting case was "gl_FragCoord_001_to_003.html". "no-over-optimization-on-uniform-array-06.html" fails when running after it. It happens even on the latest stable chrome. I have reported it to the Intel graphics driver team.


In that case I think we need to mark gl_FragCoord_001_to_003.html as Skip on affected platform(s) on our bots.

If there was another test that was suppressed for the same reason, it should be unsuppressed.
It can also affect no-over-optimization-on-uniform-array-07.html and no-over-optimization-on-uniform-array-16.html as well per my test.
Project Member

Comment 15 by bugdroid1@chromium.org, Sep 20

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/6d96c420d33adc4d874c026f050767e64aad99d2

commit 6d96c420d33adc4d874c026f050767e64aad99d2
Author: jchen10 <jie.a.chen@intel.com>
Date: Thu Sep 20 15:58:25 2018

Skip gl_FragCoord in Win/Intel/GL WebGL test.

It causes no-over-optimization-on-uniform-array cases failing
consistently.

TBR=kbr@chromium.org

Bug: 884210
Cq-Include-Trybots: luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel
Change-Id: I5e61cdcfd793fb3a7916b9c25acca535d61771fd
Reviewed-on: https://chromium-review.googlesource.com/1235327
Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Commit-Queue: Yuly Novikov <ynovikov@chromium.org>
Cr-Commit-Position: refs/heads/master@{#592820}
[modify] https://crrev.com/6d96c420d33adc4d874c026f050767e64aad99d2/content/test/gpu/gpu_tests/webgl2_conformance_expectations.py

 Issue 884043  has been merged into this issue.
Summary: gl_FragCoord_001_to_003 causes failures in subsequent no-over-optimization-on-uniform-array-* tests during webgl2_conformance_gl_passthrough_tests on Win Intel (was: WebGL Intel Windows test failure in WebglConformance_conformance_uniforms_no_over_optimization_on_uniform_array_06)
Status: ExternalDependency (was: Assigned)

Comment 19 by benhenry@google.com, Jan 16 (6 days ago)

Components: Test>Telemetry

Comment 20 by benhenry@google.com, Jan 16 (6 days ago)

Components: -Tests>Telemetry

Sign in to add a comment