wpt-import documentation needs to be updated |
|||||
Issue description
I'm trying to import some new tests suite manually, but I'm getting an error.
I've removed the skip lines at LayoutTests/W3CImportExpectations,
then I run Tools/Scripts/wpt-import and I got the following error:
Traceback (most recent call last):
File "Tools/Scripts/wpt-import", line 24, in <module>
main()
File "Tools/Scripts/wpt-import", line 17, in main
host.exit(importer.main())
File "/home/rego/checkout/chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/w3c/test_importer.py", line 117, in main
commits = self.apply_exportable_commits_locally(local_wpt)
File "/home/rego/checkout/chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/w3c/test_importer.py", line 312, in apply_exportable_commits_locally
commits = self.exportable_but_not_exported_commits(local_wpt)
File "/home/rego/checkout/chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/w3c/test_importer.py", line 343, in exportable_but_not_exported_commits
self.host, local_wpt, self.wpt_github, require_clean=False, verify_merged_pr=True)
File "/home/rego/checkout/chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/w3c/chromium_exportable_commits.py", line 47, in exportable_commits_over_last_n_commits
return _exportable_commits_since(start_commit, host, local_wpt, wpt_github, require_clean, verify_merged_pr)
File "/home/rego/checkout/chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/w3c/chromium_exportable_commits.py", line 73, in _exportable_commits_since
state, error = get_commit_export_state(commit, local_wpt, wpt_github, verify_merged_pr)
File "/home/rego/checkout/chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/w3c/chromium_exportable_commits.py", line 110, in get_commit_export_state
if _is_commit_exported(chromium_commit, local_wpt, wpt_github, verify_merged_pr):
File "/home/rego/checkout/chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/w3c/chromium_exportable_commits.py", line 132, in _is_commit_exported
if not wpt_github.is_pr_merged(pull_request.number):
File "/home/rego/checkout/chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/w3c/wpt_github.py", line 263, in is_pr_merged
response = self.request(path, method='GET')
File "/home/rego/checkout/chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/w3c/wpt_github.py", line 70, in request
headers=headers
File "/home/rego/checkout/chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/common/net/web.py", line 49, in request
return opener.open(request)
File "/usr/lib/python2.7/urllib2.py", line 435, in open
response = meth(req, response)
File "/usr/lib/python2.7/urllib2.py", line 548, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib/python2.7/urllib2.py", line 473, in error
return self._call_chain(*args)
File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain
result = func(*args)
File "/usr/lib/python2.7/urllib2.py", line 556, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 403: Forbidden
Even if I try to import a specific revision:
Tools/Scripts/wpt-import -r 0fe9f012a8ee5503b728a379705a6c5286ba1e9a
I'm getting the same error.
BTW, it'd be nice to update the documentation at:
https://chromium.googlesource.com/chromium/src/+/master/docs/testing/web_platform_tests.md#enabling-import-for-a-new-directory
It still mentions the "--allow-local-commits" flag which seems no longer present.
,
Feb 26 2018
(and I can reproduce the issue here if I don't use credentials, so marking Available)
,
Feb 26 2018
Yes, using a GitHub token fixes the issue. I've been talking with @foolip on IRC and it seems the documentation needs to be updated: * --allow-local-commits is gone * Using credentials is mandatory Thanks!
,
Feb 26 2018
,
Feb 26 2018
I think it'd also be helpful if we support reading GH_USER and GH_TOKEN from the environment and not just a JSON file. (Because the scripts in https://github.com/foolip/ecosystem-infra-stats do.)
,
Feb 26 2018
That's also easy to do. https://chromium-review.googlesource.com/c/chromium/src/+/937241 updates the documentation, there's a discussion in https://bit.ly/ecosystem-infra-rotation about whether to make passing credentials mandatory vs just warning when they're not used, and we can also independently read the values from the environment in a separate CL.
,
Feb 26 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/c089b56989b8db1b22e0aa2cfb337c5145d1ac68 commit c089b56989b8db1b22e0aa2cfb337c5145d1ac68 Author: Raphael Kubo da Costa <raphael.kubo.da.costa@intel.com> Date: Mon Feb 26 13:20:09 2018 web_platform_tests.md: Sync documentation with reality. --allow-local-commits was removed in 4a301c5 ("Remove the --allow-local-commits flag from wpt-import") so stop recommending its usage. Additionally, given the amount of export commits we have in web-platform-tests it is likely that unauthenticated runs will fail, so explain how to use the --credentials-json option in both wpt-export and wpt-import. Bug: 816390 Change-Id: I72d7bc6dcb80d0db372e9dac1176f68d1b812bcb Reviewed-on: https://chromium-review.googlesource.com/937241 Reviewed-by: Philip Jägenstedt <foolip@chromium.org> Commit-Queue: Raphael Kubo da Costa <raphael.kubo.da.costa@intel.com> Cr-Commit-Position: refs/heads/master@{#539116} [modify] https://crrev.com/c089b56989b8db1b22e0aa2cfb337c5145d1ac68/docs/testing/web_platform_tests.md
,
Feb 26 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/ac0fd25534bf32f69738125f01a23646d28fd6ad commit ac0fd25534bf32f69738125f01a23646d28fd6ad Author: Raphael Kubo da Costa <raphael.kubo.da.costa@intel.com> Date: Mon Feb 26 16:50:29 2018 WPT: Allow setting GitHub credentials as env variables, warn when they are not set A fresh WPT import checkout is very likely to fail when no GitHub credentials are used due to the number of API requests we make vs. GitHub's rate limits for unauthenticated requests. Make things more obvious by: * Allowing the credentials to be specified as environment variables in addition to a JSON file (when both are set, the JSON file specified in the command-line has precedence). * Printing a warning in test_importer.py when no credentials are set and pointing to the documentation in //docs/testing. * Expanding the error message in test_exporter.py to point to the same location. Bug: 816390 Change-Id: I5508e024e8a13e6f597f027cd157d7abbe9b2fe1 Reviewed-on: https://chromium-review.googlesource.com/937462 Reviewed-by: Philip Jägenstedt <foolip@chromium.org> Commit-Queue: Raphael Kubo da Costa <raphael.kubo.da.costa@intel.com> Cr-Commit-Position: refs/heads/master@{#539164} [modify] https://crrev.com/ac0fd25534bf32f69738125f01a23646d28fd6ad/docs/testing/web_platform_tests.md [modify] https://crrev.com/ac0fd25534bf32f69738125f01a23646d28fd6ad/third_party/WebKit/Tools/Scripts/webkitpy/w3c/common.py [modify] https://crrev.com/ac0fd25534bf32f69738125f01a23646d28fd6ad/third_party/WebKit/Tools/Scripts/webkitpy/w3c/common_unittest.py [modify] https://crrev.com/ac0fd25534bf32f69738125f01a23646d28fd6ad/third_party/WebKit/Tools/Scripts/webkitpy/w3c/test_exporter.py [modify] https://crrev.com/ac0fd25534bf32f69738125f01a23646d28fd6ad/third_party/WebKit/Tools/Scripts/webkitpy/w3c/test_importer.py
,
Feb 26 2018
|
|||||
►
Sign in to add a comment |
|||||
Comment 1 by raphael....@intel.com
, Feb 26 2018