New issue
Advanced search Search tips

Issue 647465 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Nov 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug



Sign in to add a comment

graphics_dEQP test case not found in bvt-cq

Project Member Reported by dchan@google.com, Sep 15 2016

Issue description


lots of platfrom failed graphics_dEQP  tests in bvt-cq https://wmatrix.googleplex.com/testrun/unfiltered?test_ids=348685444,348684579,348684805,348684930,348685049,348695312

result is TEST_NA due to 'No test cases found for filter: dEQP-GLES31.info!'

job status page https://ubercautotest.corp.google.com/afe/#tab_id=view_job&object_id=76981038

info log:
09/14 14:53:49.529 INFO |     graphics_dEQP:0467| ChromeOS BOARD = banjo
09/14 14:53:49.530 INFO |     graphics_dEQP:0468| ChromeOS CPU family = x86_64
09/14 14:53:49.530 INFO |     graphics_dEQP:0469| ChromeOS GPU family = baytrail
09/14 14:53:49.531 INFO |     graphics_dEQP:0473| dEQP test filter = dEQP-GLES31.info
09/14 14:53:50.368 INFO |     graphics_dEQP:0514| Running each test individually.
09/14 14:53:50.369 INFO |     graphics_dEQP:0517| Test results:
09/14 14:53:50.370 INFO |     graphics_dEQP:0518| {}
09/14 14:53:50.370 INFO |     graphics_dEQP:0539| Completed all tests. Saw 0 tests, 0 passes and 0 failures.
09/14 14:53:50.381 WARNI|              test:0606| Autotest caught exception when running test:
Traceback (most recent call last):
  File "/usr/local/autotest/common_lib/test.py", line 600, in _exec
    _call_test_function(self.execute, *p_args, **p_dargs)
  File "/usr/local/autotest/common_lib/test.py", line 804, in _call_test_function
    return func(*args, **dargs)
  File "/usr/local/autotest/common_lib/test.py", line 461, in execute
    dargs)
  File "/usr/local/autotest/common_lib/test.py", line 347, in _call_run_once_with_retry
    postprocess_profiled_run, args, dargs)
  File "/usr/local/autotest/common_lib/test.py", line 376, in _call_run_once
    self.run_once(*args, **dargs)
  File "/usr/local/autotest/tests/graphics_dEQP/graphics_dEQP.py", line 544, in run_once
    self._filter)
TestNAError: No test cases found for filter: dEQP-GLES31.info!
 

Comment 1 by dchan@google.com, Sep 15 2016

for full view see https://wmatrix.googleplex.com/unfiltered?suites=bvt-cq&days_back=14&releases=55

this happen on banjo, butterfly, candy, enguarde, expresso, falco_li, glimmer, gnawty, kip, leon, link, lumpy, mccloud, monroe, ninja, nyan_big, orco, panther, parrot, parrot_ivb, peppy, quawks, stumpy, sumo, swanky, winky, zako

Comment 2 by ihf@chromium.org, Sep 16 2016

Status: Assigned (was: Untriaged)
That is just a warning. Working as intended. Of course it would be nice if it would not confuse people.
Project Member

Comment 3 by sheriffbot@chromium.org, Sep 16 2016

Labels: Hotlist-Google
Labels: bvttriage
Instead of NA, we could restrict the control file to only run on relevant boards.

On which boards do you expect this to pass?

Comment 6 by ihf@chromium.org, Oct 27 2016

I will change it to a pass.

Due to some recent events I became convinced that TestNA is dangerous, as the lab uses it for situations which are actually infra internal failures (generally a situation where the lab can't find machines with the right provisioning labels). Labels/dependencies are dangerous too as they complicate such failures , I want to stay clear of them.

I will change all graphics tests to run everywhere (already doing this) and return only Pass or Fail. No TestError, TestWarn or TestNA as these are ambiguous and map poorly to other Google infrastructure such as sponge.
Project Member

Comment 7 by bugdroid1@chromium.org, Nov 1 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/619ee2569463359d155539539f5c0d7b2515ccd6

commit 619ee2569463359d155539539f5c0d7b2515ccd6
Author: Ilja H. Friedel <ihf@chromium.org>
Date: Sat Oct 29 02:08:49 2016

autotest graphics_*: unify TestStatus.

The goal is for graphics tests to only pass or fail. As much as
possible do not use TestError, TestWarn or TestNA etc. Having
only Pass/Fail will make reading on external dashboards (wmatrix,
sponge) less ambiguous. Also prepend all TestFail messages with
"Failed:" to make reading/understanding messages easier.

BUG= chromium:647465 
TEST=pylint

Change-Id: I92160c9def35b72e22af8c19af6275932a63d00b
Reviewed-on: https://chromium-review.googlesource.com/404953
Commit-Ready: Ilja H. Friedel <ihf@chromium.org>
Tested-by: Ilja H. Friedel <ihf@chromium.org>
Reviewed-by: Dominik Behr <dbehr@chromium.org>

[modify] https://crrev.com/619ee2569463359d155539539f5c0d7b2515ccd6/client/site_tests/graphics_GLMark2/graphics_GLMark2.py
[modify] https://crrev.com/619ee2569463359d155539539f5c0d7b2515ccd6/client/site_tests/graphics_LibDRM/graphics_LibDRM.py
[modify] https://crrev.com/619ee2569463359d155539539f5c0d7b2515ccd6/client/site_tests/graphics_GpuReset/graphics_GpuReset.py
[modify] https://crrev.com/619ee2569463359d155539539f5c0d7b2515ccd6/client/site_tests/graphics_WebGLPerformance/graphics_WebGLPerformance.py
[modify] https://crrev.com/619ee2569463359d155539539f5c0d7b2515ccd6/client/site_tests/graphics_WebGLClear/graphics_WebGLClear.py
[modify] https://crrev.com/619ee2569463359d155539539f5c0d7b2515ccd6/client/site_tests/graphics_Idle/graphics_Idle.py
[modify] https://crrev.com/619ee2569463359d155539539f5c0d7b2515ccd6/client/site_tests/graphics_Gbm/graphics_Gbm.py
[modify] https://crrev.com/619ee2569463359d155539539f5c0d7b2515ccd6/client/site_tests/graphics_WebGLManyPlanetsDeep/graphics_WebGLManyPlanetsDeep.py
[modify] https://crrev.com/619ee2569463359d155539539f5c0d7b2515ccd6/client/site_tests/graphics_VTSwitch/graphics_VTSwitch.py
[modify] https://crrev.com/619ee2569463359d155539539f5c0d7b2515ccd6/client/site_tests/graphics_WebGLAquarium/graphics_WebGLAquarium.py
[modify] https://crrev.com/619ee2569463359d155539539f5c0d7b2515ccd6/client/site_tests/graphics_Gralloc/graphics_Gralloc.py
[modify] https://crrev.com/619ee2569463359d155539539f5c0d7b2515ccd6/client/site_tests/graphics_PerfControl/graphics_PerfControl.py
[modify] https://crrev.com/619ee2569463359d155539539f5c0d7b2515ccd6/client/site_tests/graphics_dEQP/graphics_dEQP.py
[modify] https://crrev.com/619ee2569463359d155539539f5c0d7b2515ccd6/client/site_tests/graphics_GLBench/graphics_GLBench.py
[modify] https://crrev.com/619ee2569463359d155539539f5c0d7b2515ccd6/client/site_tests/graphics_KernelMemory/graphics_KernelMemory.py
[modify] https://crrev.com/619ee2569463359d155539539f5c0d7b2515ccd6/client/site_tests/graphics_SanAngeles/graphics_SanAngeles.py
[modify] https://crrev.com/619ee2569463359d155539539f5c0d7b2515ccd6/client/site_tests/graphics_GLAPICheck/graphics_GLAPICheck.py

Project Member

Comment 8 by bugdroid1@chromium.org, Nov 1 2016

Labels: merge-merged-release-R55-8872.B
The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/525ff31ed9c3afceeb99212756bf540c0f06a051

commit 525ff31ed9c3afceeb99212756bf540c0f06a051
Author: Ilja H. Friedel <ihf@chromium.org>
Date: Sat Oct 29 02:08:49 2016

autotest graphics_*: unify TestStatus.

The goal is for graphics tests to only pass or fail. As much as
possible do not use TestError, TestWarn or TestNA etc. Having
only Pass/Fail will make reading on external dashboards (wmatrix,
sponge) less ambiguous. Also prepend all TestFail messages with
"Failed:" to make reading/understanding messages easier.

BUG= chromium:647465 
TEST=pylint

Change-Id: I92160c9def35b72e22af8c19af6275932a63d00b
Reviewed-on: https://chromium-review.googlesource.com/404953
Commit-Ready: Ilja H. Friedel <ihf@chromium.org>
Tested-by: Ilja H. Friedel <ihf@chromium.org>
Reviewed-by: Dominik Behr <dbehr@chromium.org>
(cherry picked from commit 619ee2569463359d155539539f5c0d7b2515ccd6)
Reviewed-on: https://chromium-review.googlesource.com/405834
Reviewed-by: Ilja H. Friedel <ihf@chromium.org>

[modify] https://crrev.com/525ff31ed9c3afceeb99212756bf540c0f06a051/client/site_tests/graphics_GLMark2/graphics_GLMark2.py
[modify] https://crrev.com/525ff31ed9c3afceeb99212756bf540c0f06a051/client/site_tests/graphics_LibDRM/graphics_LibDRM.py
[modify] https://crrev.com/525ff31ed9c3afceeb99212756bf540c0f06a051/client/site_tests/graphics_GpuReset/graphics_GpuReset.py
[modify] https://crrev.com/525ff31ed9c3afceeb99212756bf540c0f06a051/client/site_tests/graphics_WebGLPerformance/graphics_WebGLPerformance.py
[modify] https://crrev.com/525ff31ed9c3afceeb99212756bf540c0f06a051/client/site_tests/graphics_WebGLClear/graphics_WebGLClear.py
[modify] https://crrev.com/525ff31ed9c3afceeb99212756bf540c0f06a051/client/site_tests/graphics_Idle/graphics_Idle.py
[modify] https://crrev.com/525ff31ed9c3afceeb99212756bf540c0f06a051/client/site_tests/graphics_Gbm/graphics_Gbm.py
[modify] https://crrev.com/525ff31ed9c3afceeb99212756bf540c0f06a051/client/site_tests/graphics_WebGLManyPlanetsDeep/graphics_WebGLManyPlanetsDeep.py
[modify] https://crrev.com/525ff31ed9c3afceeb99212756bf540c0f06a051/client/site_tests/graphics_VTSwitch/graphics_VTSwitch.py
[modify] https://crrev.com/525ff31ed9c3afceeb99212756bf540c0f06a051/client/site_tests/graphics_WebGLAquarium/graphics_WebGLAquarium.py
[modify] https://crrev.com/525ff31ed9c3afceeb99212756bf540c0f06a051/client/site_tests/graphics_Gralloc/graphics_Gralloc.py
[modify] https://crrev.com/525ff31ed9c3afceeb99212756bf540c0f06a051/client/site_tests/graphics_PerfControl/graphics_PerfControl.py
[modify] https://crrev.com/525ff31ed9c3afceeb99212756bf540c0f06a051/client/site_tests/graphics_dEQP/graphics_dEQP.py
[modify] https://crrev.com/525ff31ed9c3afceeb99212756bf540c0f06a051/client/site_tests/graphics_GLBench/graphics_GLBench.py
[modify] https://crrev.com/525ff31ed9c3afceeb99212756bf540c0f06a051/client/site_tests/graphics_KernelMemory/graphics_KernelMemory.py
[modify] https://crrev.com/525ff31ed9c3afceeb99212756bf540c0f06a051/client/site_tests/graphics_SanAngeles/graphics_SanAngeles.py
[modify] https://crrev.com/525ff31ed9c3afceeb99212756bf540c0f06a051/client/site_tests/graphics_GLAPICheck/graphics_GLAPICheck.py

Project Member

Comment 9 by bugdroid1@chromium.org, Nov 1 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/525ff31ed9c3afceeb99212756bf540c0f06a051

commit 525ff31ed9c3afceeb99212756bf540c0f06a051
Author: Ilja H. Friedel <ihf@chromium.org>
Date: Sat Oct 29 02:08:49 2016

autotest graphics_*: unify TestStatus.

The goal is for graphics tests to only pass or fail. As much as
possible do not use TestError, TestWarn or TestNA etc. Having
only Pass/Fail will make reading on external dashboards (wmatrix,
sponge) less ambiguous. Also prepend all TestFail messages with
"Failed:" to make reading/understanding messages easier.

BUG= chromium:647465 
TEST=pylint

Change-Id: I92160c9def35b72e22af8c19af6275932a63d00b
Reviewed-on: https://chromium-review.googlesource.com/404953
Commit-Ready: Ilja H. Friedel <ihf@chromium.org>
Tested-by: Ilja H. Friedel <ihf@chromium.org>
Reviewed-by: Dominik Behr <dbehr@chromium.org>
(cherry picked from commit 619ee2569463359d155539539f5c0d7b2515ccd6)
Reviewed-on: https://chromium-review.googlesource.com/405834
Reviewed-by: Ilja H. Friedel <ihf@chromium.org>

[modify] https://crrev.com/525ff31ed9c3afceeb99212756bf540c0f06a051/client/site_tests/graphics_GLMark2/graphics_GLMark2.py
[modify] https://crrev.com/525ff31ed9c3afceeb99212756bf540c0f06a051/client/site_tests/graphics_LibDRM/graphics_LibDRM.py
[modify] https://crrev.com/525ff31ed9c3afceeb99212756bf540c0f06a051/client/site_tests/graphics_GpuReset/graphics_GpuReset.py
[modify] https://crrev.com/525ff31ed9c3afceeb99212756bf540c0f06a051/client/site_tests/graphics_WebGLPerformance/graphics_WebGLPerformance.py
[modify] https://crrev.com/525ff31ed9c3afceeb99212756bf540c0f06a051/client/site_tests/graphics_WebGLClear/graphics_WebGLClear.py
[modify] https://crrev.com/525ff31ed9c3afceeb99212756bf540c0f06a051/client/site_tests/graphics_Idle/graphics_Idle.py
[modify] https://crrev.com/525ff31ed9c3afceeb99212756bf540c0f06a051/client/site_tests/graphics_Gbm/graphics_Gbm.py
[modify] https://crrev.com/525ff31ed9c3afceeb99212756bf540c0f06a051/client/site_tests/graphics_WebGLManyPlanetsDeep/graphics_WebGLManyPlanetsDeep.py
[modify] https://crrev.com/525ff31ed9c3afceeb99212756bf540c0f06a051/client/site_tests/graphics_VTSwitch/graphics_VTSwitch.py
[modify] https://crrev.com/525ff31ed9c3afceeb99212756bf540c0f06a051/client/site_tests/graphics_WebGLAquarium/graphics_WebGLAquarium.py
[modify] https://crrev.com/525ff31ed9c3afceeb99212756bf540c0f06a051/client/site_tests/graphics_Gralloc/graphics_Gralloc.py
[modify] https://crrev.com/525ff31ed9c3afceeb99212756bf540c0f06a051/client/site_tests/graphics_PerfControl/graphics_PerfControl.py
[modify] https://crrev.com/525ff31ed9c3afceeb99212756bf540c0f06a051/client/site_tests/graphics_dEQP/graphics_dEQP.py
[modify] https://crrev.com/525ff31ed9c3afceeb99212756bf540c0f06a051/client/site_tests/graphics_GLBench/graphics_GLBench.py
[modify] https://crrev.com/525ff31ed9c3afceeb99212756bf540c0f06a051/client/site_tests/graphics_KernelMemory/graphics_KernelMemory.py
[modify] https://crrev.com/525ff31ed9c3afceeb99212756bf540c0f06a051/client/site_tests/graphics_SanAngeles/graphics_SanAngeles.py
[modify] https://crrev.com/525ff31ed9c3afceeb99212756bf540c0f06a051/client/site_tests/graphics_GLAPICheck/graphics_GLAPICheck.py

Comment 10 by ihf@chromium.org, Nov 1 2016

Status: Fixed (was: Assigned)
There is a merge conflict to M54 so I leave it at merging to M55.

Sign in to add a comment