New issue
Advanced search Search tips

Issue 847202 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: May 2018
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Bug



Sign in to add a comment

wpt sync: HTTP errors when talking to GitHub

Project Member Reported by raphael....@intel.com, May 28 2018

Issue description

In e.g. https://ci.chromium.org/buildbot/chromium.infra.cron/wpt-exporter/83173 we had errors that looked like the following while trying to create a pull request for https://chromium-review.googlesource.com/c/chromium/src/+/1072471:

Traceback (most recent call last):
  File "/b/rr/tmpbIgCFY/w/src/third_party/blink/tools/wpt_export.py", line 28, in <module>
    main()
  File "/b/rr/tmpbIgCFY/w/src/third_party/blink/tools/wpt_export.py", line 20, in main
    success = exporter.main()
  File "/b/rr/tmpbIgCFY/w/src/third_party/blink/tools/blinkpy/w3c/test_exporter.py", line 75, in main
    self.process_gerrit_cls(open_gerrit_cls)
  File "/b/rr/tmpbIgCFY/w/src/third_party/blink/tools/blinkpy/w3c/test_exporter.py", line 106, in process_gerrit_cls
    self.process_gerrit_cl(cl)
  File "/b/rr/tmpbIgCFY/w/src/third_party/blink/tools/blinkpy/w3c/test_exporter.py", line 131, in process_gerrit_cl
    self.create_or_update_pr_from_inflight_cl(cl)
  File "/b/rr/tmpbIgCFY/w/src/third_party/blink/tools/blinkpy/w3c/test_exporter.py", line 274, in create_or_update_pr_from_inflight_cl
    commit, provisional=True, pr_footer=footer, pr_branch_name=branch_name)
  File "/b/rr/tmpbIgCFY/w/src/third_party/blink/tools/blinkpy/w3c/test_exporter.py", line 333, in create_or_update_pr_from_commit
    self.local_wpt.create_branch_with_patch(pr_branch_name, message, patch, author, force_push=updating)
  File "/b/rr/tmpbIgCFY/w/src/third_party/blink/tools/blinkpy/w3c/local_wpt.py", line 103, in create_branch_with_patch
    self.run(['git', 'push', 'origin', branch_name])
  File "/b/rr/tmpbIgCFY/w/src/third_party/blink/tools/blinkpy/w3c/local_wpt.py", line 50, in run
    return self.host.executive.run_command(command, cwd=self.path, **kwargs)
  File "/b/rr/tmpbIgCFY/w/src/third_party/blink/tools/blinkpy/common/system/executive.py", line 337, in run_command
    (error_handler or self.default_error_handler)(script_error)
  File "/b/rr/tmpbIgCFY/w/src/third_party/blink/tools/blinkpy/common/system/executive.py", line 244, in default_error_handler
    raise error
blinkpy.common.system.executive.ScriptError: Failed to run "['git', 'push', 'origin', 'chromium-export-cl-1072471']" exit_code: 1 cwd: /tmp/wpt
output: Last 500 characters of output:
m-tests.git
 ! [rejected]              chromium-export-cl-1072471 -> chromium-export-cl-1072471 (non-fast-forward)
error: failed to push some refs to 'https://fd33bd6a41f84861545b1a1cb173d1a380b11c9e@github.com/w3c/web-platform-tests.git'

The exporter is now failing a bit earlier -- e.g. https://ci.chromium.org/buildbot/chromium.infra.cron/wpt-exporter/83197

2018-05-28 05:22:16,519 - Searching for exportable in-flight CLs.
2018-05-28 05:22:20,746 - Found Gerrit in-flight CL: "webrtc wpt: remove test_state_change_event" https://chromium-review.googlesource.com/1074696
Traceback (most recent call last):
  File "/b/rr/tmpexvMds/w/src/third_party/blink/tools/wpt_export.py", line 28, in <module>
    main()
  File "/b/rr/tmpexvMds/w/src/third_party/blink/tools/wpt_export.py", line 20, in main
    success = exporter.main()
  File "/b/rr/tmpexvMds/w/src/third_party/blink/tools/blinkpy/w3c/test_exporter.py", line 75, in main
    self.process_gerrit_cls(open_gerrit_cls)
  File "/b/rr/tmpexvMds/w/src/third_party/blink/tools/blinkpy/w3c/test_exporter.py", line 106, in process_gerrit_cls
    self.process_gerrit_cl(cl)
  File "/b/rr/tmpexvMds/w/src/third_party/blink/tools/blinkpy/w3c/test_exporter.py", line 115, in process_gerrit_cl
    pull_request = self.wpt_github.pr_with_change_id(cl.change_id)
  File "/b/rr/tmpexvMds/w/src/third_party/blink/tools/blinkpy/w3c/wpt_github.py", line 331, in pr_with_change_id
    for pull_request in self.all_pull_requests():
  File "/b/rr/tmpexvMds/w/src/third_party/blink/tools/blinkpy/common/memoized.py", line 45, in __call__
    result = self._function(*args)
  File "/b/rr/tmpexvMds/w/src/third_party/blink/tools/blinkpy/w3c/wpt_github.py", line 216, in all_pull_requests
    response = self.request(path, method='GET')
  File "/b/rr/tmpexvMds/w/src/third_party/blink/tools/blinkpy/w3c/wpt_github.py", line 70, in request
    headers=headers
  File "/b/rr/tmpexvMds/w/src/third_party/blink/tools/blinkpy/common/net/web.py", line 49, in request
    return opener.open(request)
  File "/b/cipd_path_tools/lib/python2.7/urllib2.py", line 435, in open
    response = meth(req, response)
  File "/b/cipd_path_tools/lib/python2.7/urllib2.py", line 548, in http_response
    'http', request, response, code, msg, hdrs)
  File "/b/cipd_path_tools/lib/python2.7/urllib2.py", line 473, in error
    return self._call_chain(*args)
  File "/b/cipd_path_tools/lib/python2.7/urllib2.py", line 407, in _call_chain
    result = func(*args)
  File "/b/cipd_path_tools/lib/python2.7/urllib2.py", line 556, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 422: Unprocessable Entity

According to https://developer.github.com/v3/ HTTP 422 is sent when one or more invalid fields are sent to GitHub's API endpoint.
 
Status: Started (was: Available)
Summary: wpt syncc: HTTP errors when talking to GitHub (was: wpt export: HTTP errors when talking to GitHub)
"Invalid fields" don't really make sense...

I suspect this has something to do with the WPT repo move. Let me fix all the repo names, URLs, etc. in the code base.

P.S. This is also affecting the importer.
Summary: wpt sync: HTTP errors when talking to GitHub (was: wpt syncc: HTTP errors when talking to GitHub)
Root cause: although we verified GitHub would redirect most of the API requests to the new repo, it doesn't automatically rewrite the "repo:" parameters in search requests, so we need to manually fix those.
Project Member

Comment 3 by bugdroid1@chromium.org, May 28 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/1245fdd1b1b7b516075cda5864491e2e3c0c5c13

commit 1245fdd1b1b7b516075cda5864491e2e3c0c5c13
Author: Robert Ma <robertma@chromium.org>
Date: Mon May 28 18:29:32 2018

[WPT] Change the path of the GitHub WPT repo

The repo has been moved to web-platform-tests/wpt.
Migration doc: https://bit.ly/wpt-transition

TBR=qyearsley

Bug:  847202 
Change-Id: I4f9026b23533f87ef42850bd5bca540695c17932
Reviewed-on: https://chromium-review.googlesource.com/1075296
Reviewed-by: Robert Ma <robertma@chromium.org>
Commit-Queue: Robert Ma <robertma@chromium.org>
Cr-Commit-Position: refs/heads/master@{#562288}
[modify] https://crrev.com/1245fdd1b1b7b516075cda5864491e2e3c0c5c13/third_party/blink/tools/blinkpy/w3c/common.py
[modify] https://crrev.com/1245fdd1b1b7b516075cda5864491e2e3c0c5c13/third_party/blink/tools/blinkpy/w3c/import_notifier.py
[modify] https://crrev.com/1245fdd1b1b7b516075cda5864491e2e3c0c5c13/third_party/blink/tools/blinkpy/w3c/import_notifier_unittest.py
[modify] https://crrev.com/1245fdd1b1b7b516075cda5864491e2e3c0c5c13/third_party/blink/tools/blinkpy/w3c/local_wpt.py
[modify] https://crrev.com/1245fdd1b1b7b516075cda5864491e2e3c0c5c13/third_party/blink/tools/blinkpy/w3c/local_wpt_unittest.py
[modify] https://crrev.com/1245fdd1b1b7b516075cda5864491e2e3c0c5c13/third_party/blink/tools/blinkpy/w3c/test_exporter_unittest.py
[modify] https://crrev.com/1245fdd1b1b7b516075cda5864491e2e3c0c5c13/third_party/blink/tools/blinkpy/w3c/test_importer.py
[modify] https://crrev.com/1245fdd1b1b7b516075cda5864491e2e3c0c5c13/third_party/blink/tools/blinkpy/w3c/wpt_manifest.py

Status: Fixed (was: Started)
Cc: -robertma@chromium.org
Owner: robertma@chromium.org

Sign in to add a comment