Switch TestLauncher retries to be opt-in, via command-line |
|||
Issue descriptionThe TestLauncher currently retries each test up to three times, by default, if they fail on the bots, or when running a whole suite, without filters, locally. We would like to replace the default retry allowance with explicit opt-in by each test suite which depends on retries (typically complex & integration suites). (See the proposal at https://docs.google.com/document/d/1_u2c4xrwkQMevHUs_1t0kYn5j6VHcqoKe8_rQkNNjFg/edit)
,
Oct 3
As an OWNER of //ash, I would be OK with ash_unittests dropping its retry limit from 3 to 1, but not 0.
,
Oct 3
Re #2: What is your rationale for wanting a single retry per-test for that suite (I'm assuming ash_unittests)?
,
Oct 21
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/12d2a52b7c6ac8a9ffaa7ddf6f01d5e944539024 commit 12d2a52b7c6ac8a9ffaa7ddf6f01d5e944539024 Author: Wez <wez@chromium.org> Date: Sun Oct 21 03:51:29 2018 Disable HeapTest.ShrinkVector test, which leaks state. This test leaks process-global GCPhase state, potentially crashing tests which run later in the same process, e.g several IncrementalMarkingTest suite tests expect GCPhase to be None when they start executing. Bug: 897400, 879806 Change-Id: I8071fded78038b9c77a4cb6d3a5e1b258a4bbd49 Reviewed-on: https://chromium-review.googlesource.com/c/1292895 Commit-Queue: Wez <wez@chromium.org> Reviewed-by: Kentaro Hara <haraken@chromium.org> Cr-Commit-Position: refs/heads/master@{#601417} [modify] https://crrev.com/12d2a52b7c6ac8a9ffaa7ddf6f01d5e944539024/third_party/blink/renderer/platform/heap/heap_test.cc
,
Oct 22
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/be3c82ec7012add86fe73c1d9b6684b7700db673 commit be3c82ec7012add86fe73c1d9b6684b7700db673 Author: Wez <wez@chromium.org> Date: Mon Oct 22 17:53:13 2018 Disable GLInProcessCommandBufferTest.CreateImage test. This test leaks process-global state, causing potentially causing later tests (e.g. WebGPUInProcessCommandBufferTest.Dummy) run in the same batch process to unexpectedly fail. Bug: 897456 , 879806 Change-Id: Ied7facc983f7f62e6954e3d3ad28572010b09ae5 Reviewed-on: https://chromium-review.googlesource.com/c/1293030 Reviewed-by: Eric Karl <ericrk@chromium.org> Commit-Queue: Wez <wez@chromium.org> Cr-Commit-Position: refs/heads/master@{#601640} [modify] https://crrev.com/be3c82ec7012add86fe73c1d9b6684b7700db673/gpu/ipc/client/gpu_in_process_context_tests.cc
,
Oct 22
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/384e09bd3497286d570fae503fe32fb2e3d97021 commit 384e09bd3497286d570fae503fe32fb2e3d97021 Author: Wez <wez@chromium.org> Date: Mon Oct 22 17:56:49 2018 Disable flaky ThreadConditionTest.WaitReportsBlockingCall test. This test often flakes on its first run in a batch, with each call expectation satisfied more than once. Bug: 897550 , 879806 Change-Id: I2b7b130e5fd2fc8abf6aa15fbfd417041da0420f Reviewed-on: https://chromium-review.googlesource.com/c/1293178 Reviewed-by: Kentaro Hara <haraken@chromium.org> Commit-Queue: Wez <wez@chromium.org> Cr-Commit-Position: refs/heads/master@{#601645} [modify] https://crrev.com/384e09bd3497286d570fae503fe32fb2e3d97021/third_party/blink/renderer/platform/wtf/threading_primitives_test.cc
,
Oct 30
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/c0e835b7079576ed199b2373ed41a4dba9a8de6a commit c0e835b7079576ed199b2373ed41a4dba9a8de6a Author: Wez <wez@chromium.org> Date: Tue Oct 30 00:38:41 2018 Add support for appending list values in mixins. Introduce a new $mixin_append key to mixins. Values in the $mixin_append dictionary must be lists, and will be appended to any existing items in the corresponding key of the test specification. Bug: 879806 Change-Id: Ibd51c679df1944bcc9f34a4acba47ee39a5f871d Reviewed-on: https://chromium-review.googlesource.com/c/1306352 Commit-Queue: Wez <wez@chromium.org> Reviewed-by: Stephen Martinis <martiniss@chromium.org> Cr-Commit-Position: refs/heads/master@{#603705} [modify] https://crrev.com/c0e835b7079576ed199b2373ed41a4dba9a8de6a/testing/buildbot/generate_buildbot_json.py [modify] https://crrev.com/c0e835b7079576ed199b2373ed41a4dba9a8de6a/testing/buildbot/generate_buildbot_json_unittest.py
,
Nov 5
,
Jan 9
WontFix, in favour of Erik's grand plans for TestLauncher retries. |
|||
►
Sign in to add a comment |
|||
Comment 1 by w...@chromium.org
, Oct 2