Move run-webkit-tests to src-side runtests.py, or stop using runtests.py. |
|||||||
Issue descriptionWhen chatting to Pawel about https://codereview.chromium.org/1911673003, he said; -------------------------- I recommend a different approach My recommendation would be: why don't you switch to src-side runtest.py, or even ditch runtest.py entirely? You're moving in a very slow territory though. build-side changes are hard to test and risky whereas with src-side, you could just test on trybots as the person who's #2 in number of commits in build repo, I think I have the experience to make that recommendation -------------------------- This sounds like something we should investigate and understand if we are making the right choice here. Pawel, can you elaborate a little more why we would want to spend the effort here?
,
Apr 21 2016
I think it is a good idea to move as much src-side as we can.
,
May 3 2016
Is this bug about the run_webkit_tests.py in chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/? Or is there another run_webkit_tests?
,
May 3 2016
This is about the "webkit_tests" step in the build process. See this example run https://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_rel_ng/builds/215344
,
May 7 2016
As a result of work done while debugging bug 506498, I figured out how hard this would be, and the answer is "not very". If you were to revert https://codereview.chromium.org/1936943002 (i.e., reland the underlying changes) and then make run-webkit-tests smart enough to launch an xvfb instance on linux you could call it directly. Or you could just invoke run-webkit-tests via //testing/xvfb.py. Either way, you at least get to delete the layout_test_wrapper.py which is 271 lines of dead code, bugs, and confusion, and you can cut out 1-2 layers of python indirection.
,
Jun 27 2016
,
Mar 24 2017
In recipe_modules/chromium_tests/steps.py, runtest is used here: https://cs.chromium.org/chromium/build/scripts/slave/recipe_modules/chromium_tests/steps.py?l=2214 This is related to bug 695700 (stop using layout_test_wrapper.py). This issue doesn't apply to the layout test using swarming, which currently use src/testing/scripts/run_isolated_script_test.py and not runtest.py.
,
Apr 13 2017
,
Jan 2 2018
qyearsley@, issue 711515 is now fixed, does that unblock this issue? (Doing triage of Blink>Infra P2 issues with no activity in 60+ days.)
,
Jan 2 2018
As far as I know, this issue is unblocked but still not quite finished. Now, layout tests are run on swarming on all platforms except Android (thanks to Tim and Dirk). The BlinkTest class in recipe_modules/chromium_tests/steps.py still hasn't been removed because it's still used for Android, and runtest.py is still used in that case. Things that will solve this issue: - Change android to not use runtest.py by simplifying that code in BlinkTest. Maybe this would just work? - Run layout tests on swarming on Android. Dirk, what's still required for layout tests running on swarming on Android?
,
Jan 24 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/tools/build/+/7d41ee128093531c0b8fa07f0c6770be9b586e35 commit 7d41ee128093531c0b8fa07f0c6770be9b586e35 Author: Quinten Yearsley <qyearsley@chromium.org> Date: Wed Jan 24 22:06:05 2018 chromium_tests BlinkTest: Invoke rwt without runtest.py. This would change the way that run-webkit-tests is invoked on Android so that runtest.py is not used. I don't remebmer any reason why runtest.py might be necessary besides xvfb, so maybe we can stop using it here. Bug: 605496 Change-Id: I3e29e659f23872e88ba4b51fa907be8bc3cc30c3 Reviewed-on: https://chromium-review.googlesource.com/846266 Reviewed-by: Dirk Pranke <dpranke@chromium.org> Commit-Queue: Quinten Yearsley <qyearsley@chromium.org> [modify] https://crrev.com/7d41ee128093531c0b8fa07f0c6770be9b586e35/scripts/slave/recipe_modules/chromium_tests/tests/steps/blink_test.expected/android.json [modify] https://crrev.com/7d41ee128093531c0b8fa07f0c6770be9b586e35/scripts/slave/recipes/blink_downstream.expected/full_client_v8_fyi_V8_Blink_Mac_fail.json [modify] https://crrev.com/7d41ee128093531c0b8fa07f0c6770be9b586e35/scripts/slave/recipes/blink_downstream.expected/webkit_tests_interrupted.json [modify] https://crrev.com/7d41ee128093531c0b8fa07f0c6770be9b586e35/scripts/slave/recipes/blink_downstream.expected/full_client_v8_fyi_V8_Blink_Linux_64_fail.json [modify] https://crrev.com/7d41ee128093531c0b8fa07f0c6770be9b586e35/scripts/slave/recipes/blink_downstream.expected/full_client_v8_fyi_V8_Blink_Linux_64__dbg__pass.json [modify] https://crrev.com/7d41ee128093531c0b8fa07f0c6770be9b586e35/scripts/slave/recipe_modules/chromium_tests/tests/steps/blink_test.expected/big.json [modify] https://crrev.com/7d41ee128093531c0b8fa07f0c6770be9b586e35/scripts/slave/recipes/blink_downstream.expected/full_client_v8_fyi_V8_Blink_Linux_64_pass.json [modify] https://crrev.com/7d41ee128093531c0b8fa07f0c6770be9b586e35/scripts/slave/recipes/blink_downstream.expected/full_client_v8_fyi_V8_Blink_Linux_64__dbg__fail.json [modify] https://crrev.com/7d41ee128093531c0b8fa07f0c6770be9b586e35/scripts/slave/recipes/blink_downstream.expected/full_client_v8_fyi_V8_Blink_Linux_64___future_fail.json [modify] https://crrev.com/7d41ee128093531c0b8fa07f0c6770be9b586e35/scripts/slave/recipes/blink_downstream.expected/minimal_pass_continues.json [modify] https://crrev.com/7d41ee128093531c0b8fa07f0c6770be9b586e35/scripts/slave/recipes/blink_downstream.expected/webkit_tests_unexpected_error.json [modify] https://crrev.com/7d41ee128093531c0b8fa07f0c6770be9b586e35/scripts/slave/recipe_modules/chromium_tests/steps.py [modify] https://crrev.com/7d41ee128093531c0b8fa07f0c6770be9b586e35/scripts/slave/recipe_modules/chromium_tests/tests/steps/blink_test.expected/unexpected_flakes.json [modify] https://crrev.com/7d41ee128093531c0b8fa07f0c6770be9b586e35/scripts/slave/recipes/blink_downstream.expected/too_many_failures_for_retcode.json [modify] https://crrev.com/7d41ee128093531c0b8fa07f0c6770be9b586e35/scripts/slave/recipes/blink_downstream.expected/full_client_v8_fyi_V8_Blink_Linux_64___future_pass.json [modify] https://crrev.com/7d41ee128093531c0b8fa07f0c6770be9b586e35/scripts/slave/recipes/blink_downstream.expected/full_client_v8_fyi_V8_Blink_Mac_pass.json
,
Jan 25 2018
Now run-webkit-tests no longer uses runtest.py on the bots. |
|||||||
►
Sign in to add a comment |
|||||||
Comment 1 by tansell@chromium.org
, Apr 21 2016