Flaky WebGL Nexus 5X and 9 timeout failures |
||||||||
Issue descriptionThese failures are during webgl_conformance_gles_passthrough_tests: WebglConformance_conformance_glsl_functions_glsl_function_dot https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Android%20FYI%20Release%20%28Nexus%205X%29/12050 https://ci.chromium.org/p/chromium/builders/luci.chromium.try/android-marshmallow-arm64-rel/144595 WebglConformance_conformance_glsl_constructors_glsl_construct_ivec4 https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Android%20FYI%20Release%20%28Nexus%205X%29/12057 WebglConformance_conformance_glsl_constructors_glsl_construct_vec_mat_corner_cases https://ci.chromium.org/p/chromium/builders/luci.chromium.try/android-marshmallow-arm64-rel/144303 These failures are during webgl_conformance_tests: WebglConformance_conformance_glsl_functions_glsl_function_dot https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Android%20FYI%20Release%20%28Nexus%205X%29/12059 https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Android%20FYI%20Release%20%28Nexus%205X%29/12044 https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Android%20FYI%20Release%20%28Nexus%205X%29/12039 https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Android%20FYI%20Release%20%28Nexus%205X%29/12025 WebglConformance_conformance_glsl_constructors_glsl_construct_vec_mat_corner_cases https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Android%20FYI%20Release%20%28Nexus%205X%29/11996 Looks like the failures are not specific to ANGLE, so going to mark Flaky on all Nexus 5X. WebglConformance_conformance_glsl_constructors_glsl_construct_mat3 in issue 845411 looks similar to the above. What worries me is that in the logs it looks like we are trying to get additional data (stack trace) regarding the failure, but that also seems to fail. Going to open a separate bug about that. Timed out. Dumping threads. ******************************************************************************** Stack dump for thread 'TimeoutThread-1-for-MainThread' ******************************************************************************** File: "/b/swarming/w/ir/.swarming_module/lib/python2.7/threading.py", line 778, in __bootstrap raise File: "/b/swarming/w/ir/.swarming_module/lib/python2.7/threading.py", line 855, in __bootstrap_inner pass File: "/b/swarming/w/ir/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 95, in run self._exc_info = sys.exc_info() File: "/b/swarming/w/ir/third_party/catapult/devil/devil/utils/timeout_retry.py", line 151, in <lambda> child_thread = reraiser_thread.ReraiserThread(lambda: func(*args, **kwargs), File: "/b/swarming/w/ir/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl return f(*args, **kwargs) File: "/b/swarming/w/ir/third_party/catapult/devil/devil/android/app_ui.py", line 203, in _GetRootUiNode self._device.ReadFile(dtemp.name, force_pull=True)) File: "/b/swarming/w/ir/third_party/catapult/devil/devil/android/decorators.py", line 51, in timeout_retry_wrapper return impl() File: "/b/swarming/w/ir/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl return f(*args, **kwargs) File: "/b/swarming/w/ir/third_party/catapult/devil/devil/android/device_utils.py", line 1161, in RunShellCommand output = handle_large_output(cmd, large_output) File: "/b/swarming/w/ir/third_party/catapult/devil/devil/android/device_utils.py", line 1130, in handle_large_output return handle_large_command(cmd) File: "/b/swarming/w/ir/third_party/catapult/devil/devil/android/device_utils.py", line 1112, in handle_large_command return handle_check_return(cmd) File: "/b/swarming/w/ir/third_party/catapult/devil/devil/android/device_utils.py", line 1103, in handle_check_return return run(cmd) File: "/b/swarming/w/ir/third_party/catapult/devil/devil/android/device_utils.py", line 1099, in run return self.adb.Shell(cmd) File: "/b/swarming/w/ir/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 533, in Shell output = self._RunDeviceAdbCmd(args, timeout, retries, check_error=False) File: "/b/swarming/w/ir/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 316, in _RunDeviceAdbCmd check_error=check_error) File: "/b/swarming/w/ir/third_party/catapult/devil/devil/android/decorators.py", line 51, in timeout_retry_wrapper return impl() File: "/b/swarming/w/ir/third_party/catapult/devil/devil/android/decorators.py", line 47, in impl return f(*args, **kwargs) File: "/b/swarming/w/ir/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 277, in _RunAdbCmd timeout, env=cls._ADB_ENV) File: "/b/swarming/w/ir/third_party/catapult/devil/devil/utils/cmd_helper.py", line 415, in GetCmdStatusAndOutputWithTimeout for data in _IterProcessStdout(process, timeout=timeout): File: "/b/swarming/w/ir/third_party/catapult/devil/devil/utils/cmd_helper.py", line 277, in _IterProcessStdoutFcntl [child_fd], [], [], iter_aware_poll_interval) ********************************************************************************
,
Dec 5
,
Dec 5
I'd upgrade this from P3 but due to Issue monorail:4643 that's not working right now.
,
Dec 5
,
Dec 7
WebglConformance_conformance_glsl_constructors_glsl_construct_ivec4 is Flaky on Nexus 9 as well https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Android%20FYI%20Release%20%28Nexus%209%29/11036 https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Android%20FYI%20Release%20%28Nexus%209%29/11045 https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Android%20FYI%20Release%20%28Nexus%209%29/11051 https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Android%20FYI%20Release%20%28Nexus%209%29/11054 https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Android%20FYI%20Release%20%28Nexus%209%29/11056 https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Android%20FYI%20Release%20%28Nexus%209%29/11059
,
Dec 7
Issue 912583 has been merged into this issue.
,
Dec 7
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/8f16182c71d3511c8dee8704c8da500abe302a60 commit 8f16182c71d3511c8dee8704c8da500abe302a60 Author: Yuly Novikov <ynovikov@chromium.org> Date: Fri Dec 07 18:19:17 2018 Suppress couple WebGL Flaky tests conformance/glsl/constructors/glsl-construct-ivec4.html on Android Nvidia conformance/glsl/constructors/glsl-construct-mat2.html on Nexus 5X TBR=kbr@chromium.org Bug: 912161 Change-Id: I9d772e5defe1f76fbb1235a1af924ae2a7a7e059 Reviewed-on: https://chromium-review.googlesource.com/c/1367932 Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Yuly Novikov <ynovikov@chromium.org> Cr-Commit-Position: refs/heads/master@{#614743} [modify] https://crrev.com/8f16182c71d3511c8dee8704c8da500abe302a60/content/test/gpu/gpu_tests/webgl_conformance_expectations.py
,
Dec 7
,
Dec 10
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/d587561ae75deb5e1f310cca81e59ab5970e7e3f commit d587561ae75deb5e1f310cca81e59ab5970e7e3f Author: Joshua Peraza <jperaza@chromium.org> Date: Mon Dec 10 20:29:20 2018 android: Restore crash signal handlers for child processes The original tombstone producing signal handlers are still used by tests. This can be revisited later: https://bugs.chromium.org/p/chromium/issues/detail?id=912306 Bug: 911678, 912166, 912161 Change-Id: I55c0627bc431c6d61872a60d47423fd99567028f Reviewed-on: https://chromium-review.googlesource.com/c/1361597 Commit-Queue: Joshua Peraza <jperaza@chromium.org> Reviewed-by: Mark Mentovai <mark@chromium.org> Reviewed-by: Kenneth Russell <kbr@chromium.org> Cr-Commit-Position: refs/heads/master@{#615230} [modify] https://crrev.com/d587561ae75deb5e1f310cca81e59ab5970e7e3f/components/crash/content/app/crashpad_linux.cc
,
Dec 11
We're still seeing flaky timeouts of random WebGL conformance tests: WebglConformance_conformance_glsl_implicit_add_ivec3_vec3_vert https://ci.chromium.org/p/chromium/builders/luci.chromium.try/android-marshmallow-arm64-rel/149070 https://chromium-swarm.appspot.com/task?id=41b7c000e7e83d10&refresh=10&show_raw=1 WebglConformance_conformance_glsl_functions_glsl_function_min_gentype https://ci.chromium.org/p/chromium/builders/luci.chromium.try/android-marshmallow-arm64-rel/149062 https://chromium-swarm.appspot.com/task?id=41b7a71551a0a110&refresh=10&show_raw=1 Going to suppress these, but there seems to be some deeper issue going on. The GPU process shouldn't be hanging or crashing while running these tests, and Telemetry shouldn't be hanging and timing out while trying to get a stack trace.
,
Dec 11
,
Dec 11
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/1187c4bfab2116d136e37fe9f4b7e8e15ce5a819 commit 1187c4bfab2116d136e37fe9f4b7e8e15ce5a819 Author: Kenneth Russell <kbr@chromium.org> Date: Tue Dec 11 23:30:39 2018 Suppress two WebGL conformance tests flaking on Android. conformance/glsl/functions/glsl-function-min-gentype.html conformance/glsl/implicit/add_ivec3_vec3.vert.html Flaking on Nexus 5X. Tbr: ynovikov@chromium.org Tbr: crouleau@chromium.org Tbr: perezju@chromium.org Bug: 912161 Change-Id: I62854de75db2ba50524cbc8f4150080be87ea47a Reviewed-on: https://chromium-review.googlesource.com/c/1372473 Reviewed-by: Kenneth Russell <kbr@chromium.org> Commit-Queue: Kenneth Russell <kbr@chromium.org> Cr-Commit-Position: refs/heads/master@{#615720} [modify] https://crrev.com/1187c4bfab2116d136e37fe9f4b7e8e15ce5a819/content/test/gpu/gpu_tests/webgl_conformance_expectations.py
,
Dec 12
|
||||||||
►
Sign in to add a comment |
||||||||
Comment 1 by bugdroid1@chromium.org
, Dec 5