New issue
Advanced search Search tips

Issue 918669 link

Starred by 2 users

Issue metadata

Status: Assigned
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug



Sign in to add a comment

Simplify logic for determining whether to ignore passes or failures:

Project Member Reported by erikc...@chromium.org, Jan 2

Issue description

When running tests in 'with patch', we want to ignore failures if there is a single success.

When running tests in 'without patch', we want to ignore successes if there is a single failure.

This works, but only with some very convoluted logic.

For GTests, the relevant logic is at:

https://cs.chromium.org/chromium/build/scripts/slave/recipe_modules/test_utils/api.py?type=cs&q=scripts/slave/recipe_modules/test_utils/api.py&sq=package:chromium&g=0&l=221

https://cs.chromium.org/chromium/build/scripts/slave/recipe_modules/chromium_tests/steps.py?type=cs&q=scripts/slave/recipe_modules/chromium_tests/steps.py&sq=package:chromium&g=0&l=355

https://cs.chromium.org/chromium/build/scripts/slave/recipe_modules/chromium_tests/steps.py?type=cs&q=scripts/slave/recipe_modules/chromium_tests/steps.py&sq=package:chromium&g=0&l=1572

https://cs.chromium.org/chromium/build/scripts/slave/recipe_modules/test_utils/util.py?type=cs&q=scripts/slave/recipe_modules/test_utils/util.py&sq=package:chromium&g=0&l=263

The line that causes us to ignore all but the last test in an iteration is:
https://cs.chromium.org/chromium/build/scripts/slave/recipe_modules/test_utils/util.py?type=cs&q=scripts/slave/recipe_modules/test_utils/util.py&sq=package:chromium&g=0&l=259

This logic happens to work because in 'with patch', retries are reported in a single iteration, but for 'without patch', retries are reported in multiple iterations. 

We should make this explicit. self.passes and self.failures should present exactly what they suggest [every pass and failure], and we should add some custom logic to run_tests_with_patch to cause us to look for a single success, and etc. for 'without patch', 'retry with patch'.

 
Labels: Infra-Platform-Test
Components: Infra>Client>Chrome
Status: Available (was: Untriaged)
Moving into our component, as this is chromium recipe code.
Owner: erikc...@chromium.org
Status: Assigned (was: Available)
I've got a CL that starts to implement this.

Sign in to add a comment