Issue metadata
Sign in to add a comment
|
Make a w3c test "auto roll bot" to update W3C tests add expectations and baselines. |
||||||||||||||||||||||||
Issue descriptionCurrently, deps updater's --auto-update flag, updates w3c deps and commits if none of the try-job results failed. As an add-on, we would like this script to update TestExpectations with expectations lines for the failing tests from the try-job results and still commit. related to bug: 625254
,
Jul 27 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/a3e332718498531c05a7ac348fe790fb7e7b026b commit a3e332718498531c05a7ac348fe790fb7e7b026b Author: dcampb <dcampb@google.com> Date: Wed Jul 27 00:54:35 2016 Call update-w3c-test-expectations in deps_updater.py BUG= 629275 , 625253 Review-Url: https://codereview.chromium.org/2176223002 Cr-Commit-Position: refs/heads/master@{#407995} [modify] https://crrev.com/a3e332718498531c05a7ac348fe790fb7e7b026b/third_party/WebKit/Tools/Scripts/update-w3c-deps [modify] https://crrev.com/a3e332718498531c05a7ac348fe790fb7e7b026b/third_party/WebKit/Tools/Scripts/webkitpy/w3c/deps_updater.py [modify] https://crrev.com/a3e332718498531c05a7ac348fe790fb7e7b026b/third_party/WebKit/Tools/Scripts/webkitpy/w3c/deps_updater_unittest.py
,
Jul 27 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/80e6a17957c49b648085c652168db83dea2c727f commit 80e6a17957c49b648085c652168db83dea2c727f Author: dcampb <dcampb@google.com> Date: Wed Jul 27 18:36:26 2016 Removed line used for testing in deps_updater.py BUG= 629275 , Review-Url: https://codereview.chromium.org/2185823004 Cr-Commit-Position: refs/heads/master@{#408197} [modify] https://crrev.com/80e6a17957c49b648085c652168db83dea2c727f/third_party/WebKit/Tools/Scripts/webkitpy/w3c/deps_updater.py
,
Aug 3 2016
,
Aug 4 2016
,
Aug 5 2016
,
Aug 12 2016
This is mostly done, but we haven't quite yet got successful runs; will mark as done then. Current blocker is issue 637348 .
,
Aug 13 2016
,
Aug 17 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/c6c68eeb5dd4ccf9fa0e7bf97c5999d31d4fd9cc commit c6c68eeb5dd4ccf9fa0e7bf97c5999d31d4fd9cc Author: qyearsley <qyearsley@chromium.org> Date: Wed Aug 17 18:18:00 2016 In deps_updater.py: After updating, set commit and close on CQ failure. Currently, w3c-test-autoroller: 1. Uploads a CL. 2. Triggers try jobs on Blink try bots. 3. Waits for try job results (which are failing). 4. Modifies TestExpectations and commits. 5. Uploads another patch. 6. Calls `git cl land -f --auth-refresh-token-json ...`. I think that when landing CLs, w3c-test-autoroller commits should go through the CQ, and if the CQ fails even after updating TestExpectations etc., then the CL should be closed. BUG= 629275 Review-Url: https://codereview.chromium.org/2248913003 Cr-Commit-Position: refs/heads/master@{#412589} [modify] https://crrev.com/c6c68eeb5dd4ccf9fa0e7bf97c5999d31d4fd9cc/third_party/WebKit/Tools/Scripts/webkitpy/w3c/deps_updater.py
,
Aug 17 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/f4d25a5140a064d50a5c348acf7697219b2be3da commit f4d25a5140a064d50a5c348acf7697219b2be3da Author: qyearsley <qyearsley@chromium.org> Date: Wed Aug 17 20:00:24 2016 In deps_updater.py, only use Rietveld refresh token if given. In general, when `update-w3c-deps --auto-import` is run, it should only use the Rietveld authentication when given. This change makes it so that if the script is run on a user's workstation, then that user's credentials may be used. BUG= 629275 Review-Url: https://codereview.chromium.org/2249633005 Cr-Commit-Position: refs/heads/master@{#412619} [modify] https://crrev.com/f4d25a5140a064d50a5c348acf7697219b2be3da/third_party/WebKit/Tools/Scripts/webkitpy/w3c/deps_updater.py
,
Aug 24 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/c0ec8559a02e430a346d991b8e75d35eeb7088fa commit c0ec8559a02e430a346d991b8e75d35eeb7088fa Author: qyearsley <qyearsley@chromium.org> Date: Wed Aug 24 17:24:38 2016 Fix git cl upload invocation in deps_updater.py. This is a follow-up fix to http://crrev.com/2265603003. In this CL: - Add missing comma in arg list for git cl upload. This is causing recent builds to fail because "--rietveld" and "-m" are concatenated together, e.g. https://build.chromium.org/p/chromium.infra.cron/builders/w3c-test-autoroller/builds/6540 - Remove now-unused argument auth_refresh_token_json of do_auto_update. BUG= 629275 Review-Url: https://codereview.chromium.org/2268183004 Cr-Commit-Position: refs/heads/master@{#414097} [modify] https://crrev.com/c0ec8559a02e430a346d991b8e75d35eeb7088fa/third_party/WebKit/Tools/Scripts/webkitpy/w3c/deps_updater.py
,
Aug 25 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/0f77323cdf30b116f8055b11ef890d6a835b2044 commit 0f77323cdf30b116f8055b11ef890d6a835b2044 Author: qyearsley <qyearsley@chromium.org> Date: Thu Aug 25 18:58:45 2016 In update-w3c-test-expectations, handle failure to fetch results. Specifically, when no layout tests are fetched for a particular build, log a warning and continue instead of raising an exception and aborting. I noticed this error when running the script on my workstation. This CL also adds several TODO notes. BUG= 629275 Review-Url: https://codereview.chromium.org/2277303003 Cr-Commit-Position: refs/heads/master@{#414497} [modify] https://crrev.com/0f77323cdf30b116f8055b11ef890d6a835b2044/third_party/WebKit/Tools/Scripts/webkitpy/common/net/buildbot_mock.py [modify] https://crrev.com/0f77323cdf30b116f8055b11ef890d6a835b2044/third_party/WebKit/Tools/Scripts/webkitpy/common/net/layouttestresults.py [modify] https://crrev.com/0f77323cdf30b116f8055b11ef890d6a835b2044/third_party/WebKit/Tools/Scripts/webkitpy/w3c/update_w3c_test_expectations.py [modify] https://crrev.com/0f77323cdf30b116f8055b11ef890d6a835b2044/third_party/WebKit/Tools/Scripts/webkitpy/w3c/update_w3c_test_expectations_unittest.py
,
Aug 30 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/b6cc4b6ed72adabac176e3acc5a41c048de65e8a commit b6cc4b6ed72adabac176e3acc5a41c048de65e8a Author: qyearsley <qyearsley@chromium.org> Date: Tue Aug 30 03:45:40 2016 In webkitpy/common/net/git_cl.py, print rather than log "waiting" message. Reason: In the recent w3c-test-autoroller jobs (e.g. https://build.chromium.org/p/chromium.infra.cron/builders/w3c-test-autoroller/builds/6930), the update wpt step is timing out after 40 minutes with no output after print "Triggering try jobs". I believe that this started happening after http://crrev.com/2265603003, which made it so that when waiting for try job results it uses log.info rather than print. BUG= 629275 Review-Url: https://codereview.chromium.org/2285343002 Cr-Commit-Position: refs/heads/master@{#415055} [modify] https://crrev.com/b6cc4b6ed72adabac176e3acc5a41c048de65e8a/third_party/WebKit/Tools/Scripts/webkitpy/common/net/git_cl.py
,
Aug 31 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/9a34bc3cf231cec8694e70ad61ec57a1f4986a21 commit 9a34bc3cf231cec8694e70ad61ec57a1f4986a21 Author: qyearsley <qyearsley@chromium.org> Date: Wed Aug 31 22:35:41 2016 In update-w3c-test-expectations, only rebaseline newly-changed tests. Reason: When downloading expectations for newly imported/updated w3c tests, we only want to download baselines for tests that are modified in the CL, because w3c test update CLs only modify tests (not Blink source code). This should prevent downloading baselines for other tests that failed on the try bots on some platform. BUG= 629275 Review-Url: https://codereview.chromium.org/2291153003 Cr-Commit-Position: refs/heads/master@{#415787} [modify] https://crrev.com/9a34bc3cf231cec8694e70ad61ec57a1f4986a21/third_party/WebKit/Tools/Scripts/webkitpy/w3c/update_w3c_test_expectations.py
,
Sep 1 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/051c4fd1f1b9b3fa5e1dc8387b2c0e69d84726a4 commit 051c4fd1f1b9b3fa5e1dc8387b2c0e69d84726a4 Author: qyearsley <qyearsley@chromium.org> Date: Thu Sep 01 22:23:08 2016 Update list of resource files to copy from web-platform-tests. Context: There is a list of files from web-platform-tests that are put in LayoutTests/resources. Because these are copied from web-platform-tests, the copy in LayoutTests/resources is expected to be identical to the one in the imported wpt directory, and this is enforced by a PRESUBMIT. This CL updates both the list in the PRESUBMIT, and the list of files that are automatically copied over during the import process. Associating this with bug 629275 since the w3c test autoroller is currently getting stuck trying to import when idlharness.js is updated but it's not copied over so it's failing the presubmit. BUG= 629275 Review-Url: https://codereview.chromium.org/2302653002 Cr-Commit-Position: refs/heads/master@{#416083} [modify] https://crrev.com/051c4fd1f1b9b3fa5e1dc8387b2c0e69d84726a4/third_party/WebKit/LayoutTests/PRESUBMIT.py [modify] https://crrev.com/051c4fd1f1b9b3fa5e1dc8387b2c0e69d84726a4/third_party/WebKit/Tools/Scripts/webkitpy/w3c/deps_updater.py
,
Sep 2 2016
,
Sep 2 2016
,
Sep 2 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/905922aa31995068d026594cd86a7c8aa95578d7 commit 905922aa31995068d026594cd86a7c8aa95578d7 Author: qyearsley <qyearsley@chromium.org> Date: Fri Sep 02 17:16:06 2016 Fix invocation of git_cl.run() in DepsUpdater.write_test_expectations. This isn't caught by the unit test because it's not tested; it's questionable whether it's worth adding a unit test for that function as it is now, since that method just invokes some external commands. This issue was caught by looking at the latest failures on w3c-test-autoroller. BUG= 629275 Review-Url: https://codereview.chromium.org/2304053002 Cr-Commit-Position: refs/heads/master@{#416288} [modify] https://crrev.com/905922aa31995068d026594cd86a7c8aa95578d7/third_party/WebKit/Tools/Scripts/webkitpy/w3c/deps_updater.py
,
Sep 8 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/7d9d7a102b1eb03414e366e540b27c680a480ca3 commit 7d9d7a102b1eb03414e366e540b27c680a480ca3 Author: qyearsley <qyearsley@chromium.org> Date: Thu Sep 08 23:43:24 2016 Fix "git cl set-commit" line in deps_updater.py. After running update-w3c-test-expectations, w3c-test-autoroller is now failing to run `git cl set-commit`. For example: https://build.chromium.org/p/chromium.infra.cron/builders/w3c-test-autoroller/builds/7496/steps/update%20wpt/logs/stdio After this, I'm expecting that it should update expectations/baselines, then try to commit, and then likely fail to pass the commit queue and abort. BUG= 629275 Review-Url: https://codereview.chromium.org/2319763002 Cr-Commit-Position: refs/heads/master@{#417437} [modify] https://crrev.com/7d9d7a102b1eb03414e366e540b27c680a480ca3/third_party/WebKit/Tools/Scripts/webkitpy/w3c/deps_updater.py
,
Sep 8 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/7d9d7a102b1eb03414e366e540b27c680a480ca3 commit 7d9d7a102b1eb03414e366e540b27c680a480ca3 Author: qyearsley <qyearsley@chromium.org> Date: Thu Sep 08 23:43:24 2016 Fix "git cl set-commit" line in deps_updater.py. After running update-w3c-test-expectations, w3c-test-autoroller is now failing to run `git cl set-commit`. For example: https://build.chromium.org/p/chromium.infra.cron/builders/w3c-test-autoroller/builds/7496/steps/update%20wpt/logs/stdio After this, I'm expecting that it should update expectations/baselines, then try to commit, and then likely fail to pass the commit queue and abort. BUG= 629275 Review-Url: https://codereview.chromium.org/2319763002 Cr-Commit-Position: refs/heads/master@{#417437} [modify] https://crrev.com/7d9d7a102b1eb03414e366e540b27c680a480ca3/third_party/WebKit/Tools/Scripts/webkitpy/w3c/deps_updater.py
,
Sep 9 2016
,
Sep 9 2016
|
|||||||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||||||
Comment 1 by sheriffbot@chromium.org
, Jul 19 2016