WebGL blocking/unblocking context_lost_tests fail on Android FYI Release (NVIDIA Shield TV) and (Nexus 9) |
|||||||||||||||||
Issue descriptionContextLost_WebGLBlockedAfterJSNavigation and ContextLost_WebGLUnblockedAfterUserInitiatedReload Always fail on Android FYI Release (NVIDIA Shield TV) since https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Android%20FYI%20Release%20%28NVIDIA%20Shield%20TV%29/1471 Flaky on Android FYI Release (Nexus 9) since https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Android%20FYI%20Release%20%28Nexus%209%29/2010 and https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Android%20FYI%20Release%20%28Nexus%209%29/2015 Suspecting https://chromium-review.googlesource.com/522938 Going to update expectations.
,
Apr 13 2018
Looks like we may need a try/catch around the closing of the about:gpucrash tab, after all.
Traceback (most recent call last):
_RunGpuTest at content/test/gpu/gpu_tests/gpu_integration_test.py:132
self.RunActualGpuTest(url, *args)
RunActualGpuTest at content/test/gpu/gpu_tests/context_lost_integration_test.py:93
getattr(self, test_name)(test_path)
_ContextLost_WebGLBlockedAfterJSNavigation at content/test/gpu/gpu_tests/context_lost_integration_test.py:301
gpucrash_tab.Close()
traced_function at third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py:52
return func(*args, **kwargs)
Close at third_party/catapult/telemetry/telemetry/internal/browser/tab.py:104
self._tab_list_backend.CloseTab(self.id)
CloseTab at third_party/catapult/telemetry/telemetry/internal/backends/chrome/tab_list_backend.py:60
response = self._browser_backend.devtools_client.CloseTab(tab_id, timeout)
CloseTab at third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/devtools_client_backend.py:333
timeout=timeout)
Request at third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/devtools_http.py:88
response = self._conn.getresponse()
getresponse at .swarming_module/lib/python2.7/httplib.py:1121
response.begin()
begin at .swarming_module/lib/python2.7/httplib.py:438
version, status, reason = self._read_status()
_read_status at .swarming_module/lib/python2.7/httplib.py:402
raise BadStatusLine(line)
DevToolsClientConnectionError: ''
,
Apr 13 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/6233e939ef5951618d8b1c941930781ce39f11d1 commit 6233e939ef5951618d8b1c941930781ce39f11d1 Author: Yuly Novikov <ynovikov@chromium.org> Date: Fri Apr 13 23:03:53 2018 Mark couple context_lost_tests fail ContextLost_WebGLBlockedAfterJSNavigation ContextLost_WebGLUnblockedAfterUserInitiatedReload on Android Nvidia (Nexus 9 and Shield TV) BUG= 832886 TBR=kbr@chromium.org Cq-Include-Trybots: luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel Change-Id: I0b945fe5c02824ae92524d7592d899a9f1bac504 Reviewed-on: https://chromium-review.googlesource.com/1012982 Commit-Queue: Yuly Novikov <ynovikov@chromium.org> Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Cr-Commit-Position: refs/heads/master@{#550772} [modify] https://crrev.com/6233e939ef5951618d8b1c941930781ce39f11d1/content/test/gpu/gpu_tests/context_lost_expectations.py
,
Apr 16 2018
Issue 832974 has been merged into this issue.
,
Apr 17 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/6233e939ef5951618d8b1c941930781ce39f11d1 commit 6233e939ef5951618d8b1c941930781ce39f11d1 Author: Yuly Novikov <ynovikov@chromium.org> Date: Fri Apr 13 23:03:53 2018 Mark couple context_lost_tests fail ContextLost_WebGLBlockedAfterJSNavigation ContextLost_WebGLUnblockedAfterUserInitiatedReload on Android Nvidia (Nexus 9 and Shield TV) BUG= 832886 TBR=kbr@chromium.org Cq-Include-Trybots: luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel Change-Id: I0b945fe5c02824ae92524d7592d899a9f1bac504 Reviewed-on: https://chromium-review.googlesource.com/1012982 Commit-Queue: Yuly Novikov <ynovikov@chromium.org> Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Cr-Commit-Position: refs/heads/master@{#550772} [modify] https://crrev.com/6233e939ef5951618d8b1c941930781ce39f11d1/content/test/gpu/gpu_tests/context_lost_expectations.py
,
Apr 21 2018
Issue 834896 has been merged into this issue.
,
Apr 21 2018
Upgrading priority. This is happening on the CQ too: https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNygAELEgVGbGFrZSJ1Z3B1X3Rlc3RzLmNvbnRleHRfbG9zdF9pbnRlZ3JhdGlvbl90ZXN0LkNvbnRleHRMb3N0SW50ZWdyYXRpb25UZXN0LkNvbnRleHRMb3N0X1dlYkdMVW5ibG9ja2VkQWZ0ZXJVc2VySW5pdGlhdGVkUmVsb2FkDA
,
Apr 21 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/ebd02a7a495999c710b82c638d0408bd01fcd587 commit ebd02a7a495999c710b82c638d0408bd01fcd587 Author: Kenneth Russell <kbr@chromium.org> Date: Sat Apr 21 20:15:51 2018 Attempt to fix context_lost test flakes. Add try/except around closing of about:gpucrash tab. This is still causing exceptions in the DevTools backend. Removed failure exceptions for Nexus 9 and Shield TV, since the same failures were being seen on android_n5x_swarming_rel, and this should address both. If not, failure expectations will be re-introduced. Tbr: ynovikov@chromium.org Bug: 832886 Cq-Include-Trybots: luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel Change-Id: I138925c7ab8fd46f47946683a939a6cfc5c75653 Reviewed-on: https://chromium-review.googlesource.com/1023335 Reviewed-by: Kenneth Russell <kbr@chromium.org> Commit-Queue: Kenneth Russell <kbr@chromium.org> Cr-Commit-Position: refs/heads/master@{#552592} [modify] https://crrev.com/ebd02a7a495999c710b82c638d0408bd01fcd587/content/test/gpu/gpu_tests/context_lost_expectations.py [modify] https://crrev.com/ebd02a7a495999c710b82c638d0408bd01fcd587/content/test/gpu/gpu_tests/context_lost_integration_test.py
,
Apr 23 2018
Detected 3 new flakes for test/step "gpu_tests.context_lost_integration_test.ContextLostIntegrationTest.ContextLost_WebGLUnblockedAfterUserInitiatedReload". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNygAELEgVGbGFrZSJ1Z3B1X3Rlc3RzLmNvbnRleHRfbG9zdF9pbnRlZ3JhdGlvbl90ZXN0LkNvbnRleHRMb3N0SW50ZWdyYXRpb25UZXN0LkNvbnRleHRMb3N0X1dlYkdMVW5ibG9ja2VkQWZ0ZXJVc2VySW5pdGlhdGVkUmVsb2FkDA. This message was posted automatically by the chromium-try-flakes app. Since flakiness is ongoing, the issue was moved back into Sheriff Bug Queue (unless already there).
,
Apr 24 2018
,
Apr 24 2018
Detected 3 new flakes for test/step "gpu_tests.context_lost_integration_test.ContextLostIntegrationTest.ContextLost_WebGLUnblockedAfterUserInitiatedReload". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNygAELEgVGbGFrZSJ1Z3B1X3Rlc3RzLmNvbnRleHRfbG9zdF9pbnRlZ3JhdGlvbl90ZXN0LkNvbnRleHRMb3N0SW50ZWdyYXRpb25UZXN0LkNvbnRleHRMb3N0X1dlYkdMVW5ibG9ja2VkQWZ0ZXJVc2VySW5pdGlhdGVkUmVsb2FkDA. This message was posted automatically by the chromium-try-flakes app. Since flakiness is ongoing, the issue was moved back into Sheriff Bug Queue (unless already there).
,
Apr 24 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/b6077ecacd144c279175b2db08eb9c6ccd6f4f8f commit b6077ecacd144c279175b2db08eb9c6ccd6f4f8f Author: Kenneth Russell <kbr@chromium.org> Date: Tue Apr 24 22:42:02 2018 Re-suppress new context lost tests on Nexus 9 and Shield TV. Suppress the following tests again: ContextLost_WebGLBlockedAfterJSNavigation ContextLost_WebGLUnblockedAfterUserInitiatedReload They're still failing reliably on these two bots. Also restart the browser after running these two tests; this was forgotten, and appears necessary for any of these tests which tear down the GPU process. Tbr: ynovikov@chromium.org Tbr: sugoi@chromium.org No-Try: true Bug: 832886 Cq-Include-Trybots: luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel Change-Id: I5918e7b731023b8efb8e1202f8c9b57e1089e1bd Reviewed-on: https://chromium-review.googlesource.com/1025112 Commit-Queue: Yuly Novikov <ynovikov@chromium.org> Reviewed-by: Kenneth Russell <kbr@chromium.org> Reviewed-by: Alexis Hétu <sugoi@chromium.org> Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Cr-Commit-Position: refs/heads/master@{#553330} [modify] https://crrev.com/b6077ecacd144c279175b2db08eb9c6ccd6f4f8f/content/test/gpu/gpu_tests/context_lost_expectations.py [modify] https://crrev.com/b6077ecacd144c279175b2db08eb9c6ccd6f4f8f/content/test/gpu/gpu_tests/context_lost_integration_test.py
,
Apr 25 2018
Removing sheriff label since the tests were suppressed
,
Apr 26 2018
Detected 4 new flakes for test/step "gpu_tests.context_lost_integration_test.ContextLostIntegrationTest.ContextLost_WebGLUnblockedAfterUserInitiatedReload". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNygAELEgVGbGFrZSJ1Z3B1X3Rlc3RzLmNvbnRleHRfbG9zdF9pbnRlZ3JhdGlvbl90ZXN0LkNvbnRleHRMb3N0SW50ZWdyYXRpb25UZXN0LkNvbnRleHRMb3N0X1dlYkdMVW5ibG9ja2VkQWZ0ZXJVc2VySW5pdGlhdGVkUmVsb2FkDA. This message was posted automatically by the chromium-try-flakes app. Since flakiness is ongoing, the issue was moved back into Sheriff Bug Queue (unless already there).
,
Apr 26 2018
Looks like this is still flaking on Nexus 5 bots. I'm not sure why the change referred to by #12 didn't suppress that failure. Kenneth, can you take a look?
,
Apr 26 2018
Change in #12 was only for Nexus9 and Shield TV. I'll mark those tests flaky on other Android.
,
Apr 26 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/74e04301a05a4e3254192735802e1ffa0b160f91 commit 74e04301a05a4e3254192735802e1ffa0b160f91 Author: Yuly Novikov <ynovikov@chromium.org> Date: Thu Apr 26 22:32:26 2018 Mark couple context_lost_tests Flaky on Android Bug: 832886 Cq-Include-Trybots: luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel Change-Id: If566aa3fd7d00c6310b1919909f9a536a375627a Tbr: kbr@chromium.org No-try: true Reviewed-on: https://chromium-review.googlesource.com/1030900 Commit-Queue: Yuly Novikov <ynovikov@chromium.org> Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Cr-Commit-Position: refs/heads/master@{#554174} [modify] https://crrev.com/74e04301a05a4e3254192735802e1ffa0b160f91/content/test/gpu/gpu_tests/context_lost_expectations.py
,
Apr 26 2018
,
Apr 28 2018
,
Apr 28 2018
The flakes of this test on Android are caused by Issue 837884 . Since the flakes have been suppressed I'm downgrading this to P2, which is the priority used for the other bug.
,
Jun 2 2018
vikassoni@ picked up Issue 837884 and it turns out it was implicitly fixed by a code refactoring in another bug. Vikas will remove the failure expectations; thanks in advance.
,
Jun 2 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/f76b1477c25acbf5d5242164d84106f8b9a21693 commit f76b1477c25acbf5d5242164d84106f8b9a21693 Author: Vikas Soni <vikassoni@chromium.org> Date: Sat Jun 02 01:50:17 2018 Remove Android exceptions for WebGL [un]blocking context_lost tests. The assertion failure has been refactored away. Removes these two: ContextLost_WebGLBlockedAfterJSNavigation ContextLost_WebGLUnblockedAfterUserInitiatedReload Bug: 832886 Cq-Include-Trybots: luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel Change-Id: I95f885ef3e22babb3387079105fc9e3f49e21370 Reviewed-on: https://chromium-review.googlesource.com/1083951 Reviewed-by: Kenneth Russell <kbr@chromium.org> Commit-Queue: Kenneth Russell <kbr@chromium.org> Cr-Commit-Position: refs/heads/master@{#563911} [modify] https://crrev.com/f76b1477c25acbf5d5242164d84106f8b9a21693/content/test/gpu/gpu_tests/context_lost_expectations.py
,
Jun 18 2018
still looks flaky https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Android%20FYI%20Release%20%28NVIDIA%20Shield%20TV%29/3977 ContextLost_WebGLBlockedAfterJSNavigation https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Android%20FYI%20Release%20%28NVIDIA%20Shield%20TV%29/3978 ContextLost_WebGLUnblockedAfterUserInitiatedReload 3979 is green
,
Jul 24
Taking this back from Vikas. Still seeing flakes of these tests, for example: https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Android%20FYI%20Release%20%28NVIDIA%20Shield%20TV%29/5302 https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Android%20FYI%20Release%20%28NVIDIA%20Shield%20TV%29/5292 https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Android%20FYI%20Release%20%28NVIDIA%20Shield%20TV%29/5274 https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Android%20FYI%20Release%20%28NVIDIA%20Shield%20TV%29/5273 The current failure mode is as follows: Traceback (most recent call last): _RunGpuTest at content/test/gpu/gpu_tests/gpu_integration_test.py:132 self.RunActualGpuTest(url, *args) RunActualGpuTest at content/test/gpu/gpu_tests/context_lost_integration_test.py:102 getattr(self, test_name)(test_path) _ContextLost_WebGLBlockedAfterJSNavigation at content/test/gpu/gpu_tests/context_lost_integration_test.py:299 tab.Activate() traced_function at third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py:52 return func(*args, **kwargs) Activate at third_party/catapult/telemetry/telemetry/internal/browser/tab.py:85 self._tab_list_backend.ActivateTab(self.id) ActivateTab at third_party/catapult/telemetry/telemetry/internal/backends/chrome/tab_list_backend.py:87 timeout=5) WaitFor at third_party/catapult/common/py_utils/py_utils/__init__.py:148 (timeout, GetConditionString())) TimeoutException: Timed out while waiting 5s for lambda: tab_id == self._browser_backend.browser.foreground_tab.id,. The mechanism used by the context_lost_tests to crash the GPU process is to create a new tab, navigate it to chrome://gpucrash, and then switch back to the original tab. The creation of the new tab and switch back to the original tab seems flaky. To address this will try adding a new entry point to the gpu_benchmarking_extension which provokes a GPU process crash, and use that from the tests.
,
Aug 24
,
Aug 24
After the fix in Issue 858907 the flakiness of this test is gone. On these two bots: https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Android%20FYI%20Release%20%28Nexus%209%29?limit=200 https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Android%20FYI%20Release%20%28NVIDIA%20Shield%20TV%29?limit=200 there's only one failure each of context_lost tests in the last 200 builds, and it was an infrastructure-level failure. Closing as fixed.
,
Aug 31
|
|||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||
Comment 1 by kbr@chromium.org
, Apr 13 2018Labels: -Type-Bug-Regression Type-Bug
Owner: kbr@chromium.org
Status: Assigned (was: Untriaged)
Summary: WebGL blocking/unblocking context_lost_tests fail on Android FYI Release (NVIDIA Shield TV) and (Nexus 9) (was: Couple context_lost_tests fail on Android FYI Release (NVIDIA Shield TV) and (Nexus 9))