page_test refactor broke webgl_conformance_tests |
||||||
Issue descriptionhttps://build.chromium.org/p/chromium.gpu.fyi/builders/Linux%20Release%20%28NVIDIA%20GeForce%20730%29/builds/946 WebglConformance.deqp_functional_gles3_negativeshaderapi failed with: FLAKY TEST FAILURE, retrying: WebglConformance.deqp_functional_gles3_negativeshaderapi FLAKY TEST FAILURE, retrying: WebglConformance.deqp_functional_gles3_negativeshaderapi Traceback (most recent call last): File "/tmp/isolated_runIh5YbA/third_party/catapult/telemetry/telemetry/internal/story_runner.py", line 84, in _RunStoryAndProcessErrorIfNeeded state.RunStory(results) File "/tmp/isolated_runIh5YbA/content/test/gpu/gpu_tests/gpu_test_base.py", line 122, in RunStory RunStoryWithRetries(DesktopGpuSharedPageState, self, results) File "/tmp/isolated_runIh5YbA/content/test/gpu/gpu_tests/gpu_test_base.py", line 95, in RunStoryWithRetries super(cls, shared_page_state).RunStory(results) File "/tmp/isolated_runIh5YbA/third_party/catapult/telemetry/telemetry/page/shared_page_state.py", line 326, in RunStory self._current_page, self._current_tab, results) File "/tmp/isolated_runIh5YbA/content/test/gpu/gpu_tests/webgl_conformance.py", line 87, in ValidateAndMeasurePage raise page_test.Failure(_WebGLTestMessages(tab)) AttributeError: 'module' object has no attribute 'Failure' [ FAILED ] WebglConformance.deqp_functional_gles3_negativeshaderapi (1567 ms) The test flaked, but the AttributeError indicates that there's insufficient testing of Telemetry rolls into Chromium. This was probably broken by https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+/254b0aa0b9c710ff983d3256433543f58ac2db15 . Ned, could you please fix this? Please add any tests you find necessary to src/content/test/gpu/gpu_tests/gpu_test_base_unittest.py to catch breakage like this going forward.
,
Apr 26 2016
https://build.chromium.org/p/tryserver.chromium.linux/builders/chromium_presubmit/builds/173187 was the failing CQ job. stdout attached.
,
Apr 26 2016
I've stopped the Catapult auto-rolls at https://catapult-roll.skia.org/ and am rolling it back to a version that works with the GPU tests.
,
Apr 26 2016
,
Apr 26 2016
I reverted the most recent catapult roll in https://codereview.chromium.org/1921223002/ and that allowed a WebGL failure suppression https://codereview.chromium.org/1917093003/ to be submitted. Downgrading to P2. Must be fixed before Catapult rolls can resume.
,
Apr 26 2016
I found out what went wrong here: * content/test/gpu/run_unittests.py is still passing with this breaking change because there is no unittest coverage that touch page_test.Failure * pylint in content/test/gpu/PRESUBMIT.py is failing because it detects that telemetry.page.page_test has no Failure member. So this comes back to the bug of gpu PRESUBMIT is not kicked off upon catapult roll.
,
Apr 26 2016
I think I will cheat & add a gpu unittest that kick off content/test/gpu presubmit. Chatted witk kbr@ offline and he seems ok with this.
,
Apr 26 2016
,
Apr 26 2016
I verify manually that with https://codereview.chromium.org/1917173002, the gpu gpu PRESUBMIT is no longer failing. I enabled the catapult autoroll again. File a different bug about how to run PRESUBMIT of projects depending on catapult/ upon catapult roll.
,
Apr 26 2016
Issue 606855 is about triggering the PRESUBMIT of projects depending on catapult/ upon catapult roll.
,
Apr 26 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/26ef26441a318284b93d7522239aa2ae29e38597 commit 26ef26441a318284b93d7522239aa2ae29e38597 Author: catapult-deps-roller <catapult-deps-roller@chromium.org> Date: Tue Apr 26 20:22:02 2016 Roll src/third_party/catapult/ 89ac5c3cc..7744caa3a (9 commits). https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/89ac5c3cccf7..7744caa3a0f0 $ git log 89ac5c3cc..7744caa3a --date=short --no-merges --format='%ad %ae %s' BUG=605972, 606052 , 606706 TBR=catapult-sheriff@chromium.org Review URL: https://codereview.chromium.org/1923583002 Cr-Commit-Position: refs/heads/master@{#389872} [modify] https://crrev.com/26ef26441a318284b93d7522239aa2ae29e38597/DEPS
,
Apr 27 2016
,
Apr 27 2016
|
||||||
►
Sign in to add a comment |
||||||
Comment 1 by kbr@chromium.org
, Apr 26 2016