graphics_dEQP remove also the test before the not passing test |
|||
Issue descriptionIn graphics_dEQP we _load_not_passing_cases and avoid running them, as tests known to cause problems should be avoided. One problem with this is that sometimes tests cause problems to the next following test and make it fail. So instead of just removing the failing test we should also remove the test preceding it. Otherwise what happens is that we keep removing good tests that were caused to fail by the bad one, but never remove the actual failure cause.
,
May 8 2017
Is it possible for us to still run the failed cases in normal mode as a separate group alongside the passed tests in hasty mode? As this helps us the identify which failed cases is a false negative.
,
May 9 2017
There are two use cases for running dEQP: bringup of new architectures and for checking regressions. I don't think bringup will be important anymore, so all we care for is effectively and reliably see if some change caused our tests to start failing. And for that it would be enough to run 99.99% of all tests to be very effective. A different thought: One thing that CTS does when running dEQP is it allows to rerun failing tests a few times. This takes care of some but not necessarily all failures. Maybe we should do that? I don't know yet.
,
May 25 2017
For gles2-master kevin, peach_*, veyron_* boards now are passing after the re-run patch https://wmatrix.googleplex.com/unfiltered?hide_missing=True&releases=tot&tests=graphics_dEQP.gles2-master&days_back=3 only nyan_*, daisy_* are failing for gles2 right now.
,
Jun 23 2017
Based on wmatrix for the past 100 days. GLES2: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&releases=tot&tests=graphics_dEQP.gles2-master&days_back=100 hasty master mail-t604: 3 3 tegra: 2 2 GLES3: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&releases=tot&tests=graphics_dEQP.gles3-master&days_back=100 hasty master broadwell: flaky flaky haswell: 6 6 mali-t604: 12+47=59 59 mali-t628: 12 12 mali-t760: 12 12 mali-t860: 1+12=13 13 rogue: 1 1 sandybridge: 2+8=10 10 tegra: 12+15+3+37=67 67 I think now the hasty file is as reliable as the master-list to detect the failure of the tests.
,
Jan 22 2018
|
|||
►
Sign in to add a comment |
|||
Comment 1 by ihf@chromium.org
, May 4 2017