New issue
Advanced search Search tips

Issue 896239 link

Starred by 1 user
Status: Assigned
Owner:
erikc...@chromium.org
Cc:
martiniss@chromium.org
erikc...@chromium.org
jbudorick@chromium.org
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Bug



Sign in to add a comment

Better messaging for CQ runs that pass because test failures are also present on ToT

Project Member Reported by tikuta@chromium.org, Oct 17 
Why this CL is marked as success?
https://ci.chromium.org/p/chromium/builders/luci.chromium.try/linux_chromium_rel_ng/211380

I think CL failed in non-experimental with patch step should be blocked.

Comment 1 by tikuta@chromium.org, Oct 17

Description: Show this description

Comment 2 by erikc...@chromium.org, Oct 17

Owner: erikc...@chromium.org
Status: Assigned (was: Untriaged)
We should have better messaging.

The following webkit_layout_test failed 4 times in a row in 'with patch':
 virtual/user-activation-v2/fast/events/touch/gesture/touch-gesture-scroll-input-field.html

During 'retry without patch', it ppassed 7 times and failed 3 times. This suggests that it's flaky on ToT, which is why we let the CL pass the CQ.

Comment 3 by erikc...@chromium.org, Oct 17 

The reason it failed 4 times in a row is because the --reset-content-shell logic doesn't apply to retries that occur when running all layout tests.

Comment 4 by jbudorick@chromium.org, Oct 17 

Better messaging here sgtm.

Comment 5 by erikc...@chromium.org, Oct 18

Summary: Better messaging for CQ runs that pass because test failures are also present on ToT (was: linux_chromium_rel_ng become success for broken CL)
Project Member

Comment 6 by bugdroid1@chromium.org, Oct 29 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/adc156a25664491d7683863185b72a10518c9dbd

commit adc156a25664491d7683863185b72a10518c9dbd
Author: erikchen <erikchen@chromium.org>
Date: Mon Oct 29 16:24:09 2018

Better messaging for 'without patch' and 'retry with patch' steps.

This CL provides more descriptive messages for why test failures do or do not
cause the build to pass.

Change-Id: Ic025ba9590dbfec26e13c0d07dd90c6a6bad2c63
Bug: 896239
Reviewed-on: https://chromium-review.googlesource.com/c/1296836
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Erik Chen <erikchen@chromium.org>

[modify] https://crrev.com/adc156a25664491d7683863185b72a10518c9dbd/scripts/slave/recipes/blink_downstream.expected/full_client_v8_fyi_V8_Blink_Mac_fail.json
[modify] https://crrev.com/adc156a25664491d7683863185b72a10518c9dbd/scripts/slave/recipe_modules/chromium_tests/tests/api/run_tests_on_tryserver.expected/enable_retry_with_patch_succeed_after_deapply.json
[modify] https://crrev.com/adc156a25664491d7683863185b72a10518c9dbd/scripts/slave/recipes/chromium_trybot.expected/swarmed_webkit_tests_unexpected_error.json
[modify] https://crrev.com/adc156a25664491d7683863185b72a10518c9dbd/scripts/slave/recipes/blink_downstream.expected/full_client_v8_fyi_V8_Blink_Win_fail.json
[modify] https://crrev.com/adc156a25664491d7683863185b72a10518c9dbd/scripts/slave/recipes/chromium_trybot.expected/swarming_trigger_failure.json
[modify] https://crrev.com/adc156a25664491d7683863185b72a10518c9dbd/scripts/slave/recipes/blink_downstream.expected/minimal_pass_continues.json
[modify] https://crrev.com/adc156a25664491d7683863185b72a10518c9dbd/scripts/slave/recipes/blink_downstream.expected/full_client_v8_fyi_V8_Blink_Linux_64__dbg__fail.json
[modify] https://crrev.com/adc156a25664491d7683863185b72a10518c9dbd/scripts/slave/README.recipes.md
[modify] https://crrev.com/adc156a25664491d7683863185b72a10518c9dbd/scripts/slave/recipes/blink_downstream.expected/full_client_v8_fyi_V8_Blink_Linux_64___future_fail.json
[modify] https://crrev.com/adc156a25664491d7683863185b72a10518c9dbd/scripts/slave/recipes/chromium_trybot.expected/swarming_test_failure.json
[modify] https://crrev.com/adc156a25664491d7683863185b72a10518c9dbd/scripts/slave/recipe_modules/test_utils/api.py
[modify] https://crrev.com/adc156a25664491d7683863185b72a10518c9dbd/scripts/slave/recipes/blink_downstream.expected/full_client_v8_fyi_V8_Blink_Linux_64_fail.json
[modify] https://crrev.com/adc156a25664491d7683863185b72a10518c9dbd/scripts/slave/recipes/chromium_trybot.expected/dynamic_swarmed_isolated_script_test_failure_no_result_json.json
[modify] https://crrev.com/adc156a25664491d7683863185b72a10518c9dbd/scripts/slave/recipes/chromium_trybot.expected/invalid_results.json
[modify] https://crrev.com/adc156a25664491d7683863185b72a10518c9dbd/scripts/slave/recipes/blink_downstream.expected/too_many_failures_for_retcode.json
[modify] https://crrev.com/adc156a25664491d7683863185b72a10518c9dbd/scripts/slave/recipes/chromium_trybot.expected/dynamic_isolated_script_test_on_trybot_failing.json
[modify] https://crrev.com/adc156a25664491d7683863185b72a10518c9dbd/scripts/slave/recipes/chromium_trybot.expected/swarmed_webkit_tests_interrupted.json
[modify] https://crrev.com/adc156a25664491d7683863185b72a10518c9dbd/scripts/slave/recipe_modules/chromium_tests/tests/api/run_tests_on_tryserver.expected/basic.json
[modify] https://crrev.com/adc156a25664491d7683863185b72a10518c9dbd/scripts/slave/recipe_modules/chromium_tests/tests/api/run_tests_on_tryserver.py
[modify] https://crrev.com/adc156a25664491d7683863185b72a10518c9dbd/scripts/slave/recipes/chromium_trybot.expected/swarmed_layout_tests_too_many_failures_for_retcode.json
►

Sign in to add a comment