Support --gtest_filter, --gtest_repeat, --test-launcher-retry-limit in run-webkit-tests for FindIt |
||||||||||
Issue descriptionTo allow FindIt to run the Blink layout tests, the run-webkit-tests command needs to support these 3 parameters: - [ ] --gtest_filter - [x] --gtest_repeat - [ ] --test-launcher-retry-limit Open questions: - [x] We've already added support for 2 gtest parameters as env variables (https://crrev.com/2350853003), can we also accept these as env variables or must we support them as command line arguments? [yes, must support cmd line]
,
Sep 27 2016
To match with base/test/launcher-based gtest (https://cs.chromium.org/chromium/src/base/test/launcher/), they have to be command line arguments.
,
Sep 27 2016
,
Sep 27 2016
,
Mar 30 2017
,
Apr 4 2017
,
Jun 20 2017
Hey Shuotao, are there any docs I can reference for these flags? I'm pretty certain I know what --gtest_repeat needs to do but am less clear on the others. Also what is the relative priority of this? Is running run-webkit-tests in FindIt blocked on moving all platforms to swarming?
,
Jun 21 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/68444af30e59f730704616740707f77c90b27850 commit 68444af30e59f730704616740707f77c90b27850 Author: Jeff Carpenter <jeffcarp@chromium.org> Date: Wed Jun 21 01:31:49 2017 Add support for --gtest_repeat to run-webkit-tests The context for this change is to allow FindIt, which only speaks GTest, to run run-webkit-tests. Bug: 650747 Change-Id: Ic4930cda309395f1a19a619a701140909f6f296c Reviewed-on: https://chromium-review.googlesource.com/541757 Commit-Queue: Jeff Carpenter <jeffcarp@chromium.org> Reviewed-by: Quinten Yearsley <qyearsley@chromium.org> Cr-Commit-Position: refs/heads/master@{#481064} [modify] https://crrev.com/68444af30e59f730704616740707f77c90b27850/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py [modify] https://crrev.com/68444af30e59f730704616740707f77c90b27850/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests_unittest.py
,
Jun 21 2017
,
Jan 29 2018
,
Jan 30 2018
One more flag: --gtest_also_run_disabled_tests CL up: https://crrev.com/c/893683
,
Jan 31 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/01637921aef09b8582164ebabdeb317f9bc43208 commit 01637921aef09b8582164ebabdeb317f9bc43208 Author: Robert Ma <robertma@chromium.org> Date: Wed Jan 31 20:49:45 2018 [r-w-t] Add --gtest_also_run_disabled_tests as an alias for --skipped=ignore When both options are given, --gtest_also_run_disabled_tests takes precedence. Also add a TODO to change the execution order of --gtest_repeat (--repeat-each). Bug: 650747 Change-Id: I6159ccc3975a979570517a54dab5461234b81f4e Reviewed-on: https://chromium-review.googlesource.com/893683 Commit-Queue: Robert Ma <robertma@chromium.org> Reviewed-by: Quinten Yearsley <qyearsley@chromium.org> Cr-Commit-Position: refs/heads/master@{#533395} [modify] https://crrev.com/01637921aef09b8582164ebabdeb317f9bc43208/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/controllers/manager.py [modify] https://crrev.com/01637921aef09b8582164ebabdeb317f9bc43208/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py [modify] https://crrev.com/01637921aef09b8582164ebabdeb317f9bc43208/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests_unittest.py
,
Feb 2 2018
Two CLs up for the two remaining flags: - [.] --gtest_filter (CL: https://crrev.com/c/900125) - [x] --gtest_repeat - [.] --test-launcher-retry-limit (CL: https://crrev.com/c/895812) - [x] --gtest_also_run_disabled_tests
,
Feb 2 2018
,
Feb 13 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/be1a84101cd8b278dc57b1c4ac24ec89ee298cba commit be1a84101cd8b278dc57b1c4ac24ec89ee298cba Author: Robert Ma <robertma@chromium.org> Date: Tue Feb 13 08:24:09 2018 [r-w-t] Add --test-launcher-retry-limit and remove --retry-failures --test-launcher-retry-limit is added as an alias of --num-retries. Besides, the legacy --retry-failures flag is removed as no one is using it any more. --no-retry-failures is kept for now as a logical shorthand for --num-retries=0. The behaviours (both w/ and w/o the flags) are not changed. Furthermore, improve the unit tests by fixing a few wrong assertions of filesystem (they are always true), removing some unrelated flags in a few tests, and splitting the test for --clobber-old-results out of the test for retrying. Bug: 650747 Change-Id: I1617e1eee36339eb7df4d55d832a0e54af204c0c Reviewed-on: https://chromium-review.googlesource.com/895812 Commit-Queue: Robert Ma <robertma@chromium.org> Reviewed-by: Quinten Yearsley <qyearsley@chromium.org> Cr-Commit-Position: refs/heads/master@{#536291} [modify] https://crrev.com/be1a84101cd8b278dc57b1c4ac24ec89ee298cba/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/controllers/manager.py [modify] https://crrev.com/be1a84101cd8b278dc57b1c4ac24ec89ee298cba/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py [modify] https://crrev.com/be1a84101cd8b278dc57b1c4ac24ec89ee298cba/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests_unittest.py
,
Feb 13 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/2dde4abe519fdf6d4531aaa1a8e8e8b0b022b797 commit 2dde4abe519fdf6d4531aaa1a8e8e8b0b022b797 Author: Robert Ma <robertma@chromium.org> Date: Tue Feb 13 08:33:13 2018 [r-w-t] Add --gtest_filter for FindIt The argument takes a colon(:)-separated list of tests to run. Wildcards are NOT supported (as FindIt does not use wildcards). In fact, the code simply splits the list by colons and adds them as positional arguments. Bug: 650747 Change-Id: I9c4f863f17d15eec869649a0d7cb2d9c37932ea6 Reviewed-on: https://chromium-review.googlesource.com/900125 Commit-Queue: Robert Ma <robertma@chromium.org> Reviewed-by: Quinten Yearsley <qyearsley@chromium.org> Cr-Commit-Position: refs/heads/master@{#536292} [modify] https://crrev.com/2dde4abe519fdf6d4531aaa1a8e8e8b0b022b797/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py [modify] https://crrev.com/2dde4abe519fdf6d4531aaa1a8e8e8b0b022b797/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests_unittest.py
,
Feb 13 2018
chanli@ all done!
,
Apr 26 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/5653652498e1275cda59da9d06ef4b58e173da5b commit 5653652498e1275cda59da9d06ef4b58e173da5b Author: Robert Ma <robertma@chromium.org> Date: Thu Apr 26 15:49:19 2018 [blinkpy] Fix the semantics of --gtest_repeat in r-w-t gtest's --gtest_repeat runs all the tests in iterations, i.e. its execution order looks like ABCABCABC. Hence, the correct alias for this flag in r-w-t should be --iterations instead of --repeat-each (the latter repeats each test individually, like AAABBBCCC). Bug: 650747 Change-Id: I9abe19b89951d7b4b655c7a3118ae8d1c2654daf Reviewed-on: https://chromium-review.googlesource.com/1029180 Reviewed-by: Chan Li <chanli@chromium.org> Reviewed-by: Quinten Yearsley <qyearsley@chromium.org> Commit-Queue: Robert Ma <robertma@chromium.org> Cr-Commit-Position: refs/heads/master@{#554038} [modify] https://crrev.com/5653652498e1275cda59da9d06ef4b58e173da5b/third_party/blink/tools/blinkpy/web_tests/controllers/manager.py [modify] https://crrev.com/5653652498e1275cda59da9d06ef4b58e173da5b/third_party/blink/tools/blinkpy/web_tests/run_webkit_tests.py [modify] https://crrev.com/5653652498e1275cda59da9d06ef4b58e173da5b/third_party/blink/tools/blinkpy/web_tests/run_webkit_tests_unittest.py
,
Oct 10
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/6e58bdb9bc67b99300645edbd41d71437701d9b0 commit 6e58bdb9bc67b99300645edbd41d71437701d9b0 Author: Robert Ma <robertma@chromium.org> Date: Wed Oct 10 21:18:40 2018 Reland "[blinkpy] Fix the semantics of --gtest_repeat in r-w-t" This is a reland of 5653652498e1275cda59da9d06ef4b58e173da5b It was accidentally reverted (perhaps due to a bad merge) in https://crrev.com/c/1029430 . Original change's description: > [blinkpy] Fix the semantics of --gtest_repeat in r-w-t > > gtest's --gtest_repeat runs all the tests in iterations, i.e. its > execution order looks like ABCABCABC. Hence, the correct alias for this > flag in r-w-t should be --iterations instead of --repeat-each (the > latter repeats each test individually, like AAABBBCCC). > > Bug: 650747 > Change-Id: I9abe19b89951d7b4b655c7a3118ae8d1c2654daf > Reviewed-on: https://chromium-review.googlesource.com/1029180 > Reviewed-by: Chan Li <chanli@chromium.org> > Reviewed-by: Quinten Yearsley <qyearsley@chromium.org> > Commit-Queue: Robert Ma <robertma@chromium.org> > Cr-Commit-Position: refs/heads/master@{#554038} Bug: 650747 Change-Id: Ia41ad977605007727bfed7f3f1394b545588426c Reviewed-on: https://chromium-review.googlesource.com/c/1274109 Reviewed-by: Chan Li <chanli@chromium.org> Commit-Queue: Robert Ma <robertma@chromium.org> Cr-Commit-Position: refs/heads/master@{#598511} [modify] https://crrev.com/6e58bdb9bc67b99300645edbd41d71437701d9b0/third_party/blink/tools/blinkpy/web_tests/controllers/manager.py [modify] https://crrev.com/6e58bdb9bc67b99300645edbd41d71437701d9b0/third_party/blink/tools/blinkpy/web_tests/run_webkit_tests.py [modify] https://crrev.com/6e58bdb9bc67b99300645edbd41d71437701d9b0/third_party/blink/tools/blinkpy/web_tests/run_webkit_tests_unittest.py |
||||||||||
►
Sign in to add a comment |
||||||||||
Comment 1 by jeffcarp@chromium.org
, Sep 27 2016