New issue
Advanced search Search tips

Issue 606706 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Apr 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: All
Pri: 2
Type: Bug

Blocking:
issue 606854



Sign in to add a comment

page_test refactor broke webgl_conformance_tests

Project Member Reported by kbr@chromium.org, Apr 26 2016

Issue description

https://build.chromium.org/p/chromium.gpu.fyi/builders/Linux%20Release%20%28NVIDIA%20GeForce%20730%29/builds/946

WebglConformance.deqp_functional_gles3_negativeshaderapi failed with:

FLAKY TEST FAILURE, retrying: WebglConformance.deqp_functional_gles3_negativeshaderapi
FLAKY TEST FAILURE, retrying: WebglConformance.deqp_functional_gles3_negativeshaderapi
Traceback (most recent call last):
  File "/tmp/isolated_runIh5YbA/third_party/catapult/telemetry/telemetry/internal/story_runner.py", line 84, in _RunStoryAndProcessErrorIfNeeded
    state.RunStory(results)
  File "/tmp/isolated_runIh5YbA/content/test/gpu/gpu_tests/gpu_test_base.py", line 122, in RunStory
    RunStoryWithRetries(DesktopGpuSharedPageState, self, results)
  File "/tmp/isolated_runIh5YbA/content/test/gpu/gpu_tests/gpu_test_base.py", line 95, in RunStoryWithRetries
    super(cls, shared_page_state).RunStory(results)
  File "/tmp/isolated_runIh5YbA/third_party/catapult/telemetry/telemetry/page/shared_page_state.py", line 326, in RunStory
    self._current_page, self._current_tab, results)
  File "/tmp/isolated_runIh5YbA/content/test/gpu/gpu_tests/webgl_conformance.py", line 87, in ValidateAndMeasurePage
    raise page_test.Failure(_WebGLTestMessages(tab))
AttributeError: 'module' object has no attribute 'Failure'

[  FAILED  ] WebglConformance.deqp_functional_gles3_negativeshaderapi (1567 ms)


The test flaked, but the AttributeError indicates that there's insufficient testing of Telemetry rolls into Chromium. This was probably broken by https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+/254b0aa0b9c710ff983d3256433543f58ac2db15 .

Ned, could you please fix this? Please add any tests you find necessary to src/content/test/gpu/gpu_tests/gpu_test_base_unittest.py to catch breakage like this going forward.

 
stdout.txt
1.1 MB View Download

Comment 1 by kbr@chromium.org, Apr 26 2016

This also broke the unittests in content/test/gpu, breaking the ability to commit changes to src/content/test/gpu/ . https://codereview.chromium.org/1917093003/ was just kicked out of the CQ because of this:

** Presubmit Messages **
--tbr was specified, skipping OWNERS check

Pylint (48 files using ['--disable=all', '--enable=cyclic-import']) (2.08s)

('run_content_test_gpu_unittests',) (22.11s)

** Presubmit ERRORS **
Pylint (48 files using ['--disable=cyclic-import'] on 8 cores) (2.44s) failed
************* Module gpu_tests.gpu_rasterization
E: 45,12: Module 'telemetry.page.page_test' has no 'Failure' member (no-member)


Ned, please change the Catapult auto-roll scripts to run src/content/test/gpu/run_unittests.py .

Comment 2 by kbr@chromium.org, Apr 26 2016

https://build.chromium.org/p/tryserver.chromium.linux/builders/chromium_presubmit/builds/173187 was the failing CQ job. stdout attached.

stdout.txt
13.0 KB View Download

Comment 3 by kbr@chromium.org, Apr 26 2016

I've stopped the Catapult auto-rolls at https://catapult-roll.skia.org/ and am rolling it back to a version that works with the GPU tests.

Comment 4 by kbr@chromium.org, Apr 26 2016

Cc: sullivan@chromium.org

Comment 5 by kbr@chromium.org, Apr 26 2016

Labels: -Pri-1 Pri-2
I reverted the most recent catapult roll in https://codereview.chromium.org/1921223002/ and that allowed a WebGL failure suppression https://codereview.chromium.org/1917093003/ to be submitted.

Downgrading to P2. Must be fixed before Catapult rolls can resume.

I found out what went wrong here:
* content/test/gpu/run_unittests.py is still passing with this breaking change because there is no unittest coverage that touch page_test.Failure
* pylint in content/test/gpu/PRESUBMIT.py is failing because it detects that telemetry.page.page_test has no Failure member.

So this comes back to the bug of gpu PRESUBMIT is not kicked off upon catapult roll. 
I think I will cheat & add a gpu unittest that kick off content/test/gpu presubmit. Chatted witk kbr@ offline and he seems ok with this.
Cc: jbudorick@chromium.org
I verify manually that with https://codereview.chromium.org/1917173002, the gpu gpu PRESUBMIT is no longer failing. I enabled the catapult autoroll again.

File a different bug about how to run PRESUBMIT of projects depending on catapult/ upon catapult roll.
 Issue 606855  is about triggering the PRESUBMIT of projects depending on catapult/ upon catapult roll.
Project Member

Comment 11 by bugdroid1@chromium.org, Apr 26 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/26ef26441a318284b93d7522239aa2ae29e38597

commit 26ef26441a318284b93d7522239aa2ae29e38597
Author: catapult-deps-roller <catapult-deps-roller@chromium.org>
Date: Tue Apr 26 20:22:02 2016

Roll src/third_party/catapult/ 89ac5c3cc..7744caa3a (9 commits).

https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/89ac5c3cccf7..7744caa3a0f0

$ git log 89ac5c3cc..7744caa3a --date=short --no-merges --format='%ad %ae %s'

BUG=605972, 606052 , 606706 

TBR=catapult-sheriff@chromium.org

Review URL: https://codereview.chromium.org/1923583002

Cr-Commit-Position: refs/heads/master@{#389872}

[modify] https://crrev.com/26ef26441a318284b93d7522239aa2ae29e38597/DEPS

Status: Fixed (was: Assigned)

Comment 13 by kbr@chromium.org, Apr 27 2016

Blocking: 606854

Sign in to add a comment