New issue
Advanced search Search tips

Issue 762844 link

Starred by 0 users

Issue metadata

Status: Fixed
Owner:
Closed: Sep 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug



Sign in to add a comment

[WPT import] Network error while waiting for CQ jobs

Project Member Reported by raphael....@intel.com, Sep 7 2017

Issue description

From https://luci-milo.appspot.com/buildbot/chromium.infra.cron/wpt-importer/1383 and https://luci-logdog.appspot.com/v/?s=chromium%2Fbb%2Fchromium.infra.cron%2Fwpt-importer%2F1383%2F%2B%2Frecipes%2Fsteps%2FImport_changes_from_WPT_to_Chromium%2F0%2Fstdout:

2017-09-07 00:19:43,505 - Triggering CQ try jobs.
Waiting for try jobs, timeout: 10800 seconds.
Waiting for try jobs. 121 seconds passed.
Traceback (most recent call last):
  File "/mnt/data/b/rr/tmpCFXgVV/w/src/third_party/WebKit/Tools/Scripts/wpt-import", line 25, in <module>
    main()
  File "/mnt/data/b/rr/tmpCFXgVV/w/src/third_party/WebKit/Tools/Scripts/wpt-import", line 18, in main
    host.exit(importer.main())
  File "/mnt/data/b/rr/tmpCFXgVV/w/src/third_party/WebKit/Tools/Scripts/webkitpy/w3c/test_importer.py", line 132, in main
    if not self.run_commit_queue_for_cl():
  File "/mnt/data/b/rr/tmpCFXgVV/w/src/third_party/WebKit/Tools/Scripts/webkitpy/w3c/test_importer.py", line 174, in run_commit_queue_for_cl
    timeout_seconds=TIMEOUT_SECONDS)
  File "/mnt/data/b/rr/tmpCFXgVV/w/src/third_party/WebKit/Tools/Scripts/webkitpy/common/net/git_cl.py", line 88, in wait_for_try_jobs
    message=' for try jobs')
  File "/mnt/data/b/rr/tmpCFXgVV/w/src/third_party/WebKit/Tools/Scripts/webkitpy/common/net/git_cl.py", line 108, in _wait_for
    value = poll_function()
  File "/mnt/data/b/rr/tmpCFXgVV/w/src/third_party/WebKit/Tools/Scripts/webkitpy/common/net/git_cl.py", line 78, in finished_try_job_results_or_none
    results = self.try_job_results()
  File "/mnt/data/b/rr/tmpCFXgVV/w/src/third_party/WebKit/Tools/Scripts/webkitpy/common/net/git_cl.py", line 149, in try_job_results
    build_to_status[self._build(result)] = self._try_job_status(result)
  File "/mnt/data/b/rr/tmpCFXgVV/w/src/third_party/WebKit/Tools/Scripts/webkitpy/common/net/git_cl.py", line 179, in _build
    assert match, '%s did not match expected format' % url
AssertionError: https://ci.chromium.org/swarming/task/38740befcd9c0010?server=chromium-swarm.appspot.com did not match expected format

This looks like a transient error, but it's probably a good idea to investigate what went wrong if possible.
 

Comment 1 by leon....@intel.com, Sep 7 2017

My findings until now:
The one url
https://ci.chromium.org/swarming/task/38740befcd9c0010?server=chromium-swarm.appspot.com
failed to match 
'match = re.match(r'.*/task/([0-9a-f]+)/?$', url)'
at
https://cs.chromium.org/chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/common/net/git_cl.py?rcl=21e78fbe27c34227e4c4fb7c627907506639b6f5&l=178

Maybe we could just loose the regular expression check there? As format of the CI result urls may change sometimes?
Labels: Hotlist-GoodFirstBug
You're right - I think we should just change that regex.

Background: in the past all try jobs had "build numbers". Now some jobs are swarming tasks, which don't have numbers, but they do have IDs, which are sha-1 hashes.

At some point in the future, we'll be tracking jobs with IDs rather than numbers, and layout test results will likely be stored in Google Storage directories with swarming hashes instead of build numbers.

Comment 3 by leon....@intel.com, Sep 9 2017

Cc: -leon....@intel.com
Owner: leon....@intel.com
Status: Started (was: Available)
Project Member

Comment 4 by bugdroid1@chromium.org, Sep 10 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/befdb2547302be64417138865af087be42941c33

commit befdb2547302be64417138865af087be42941c33
Author: Han Leon <leon.han@intel.com>
Date: Sun Sep 10 02:56:29 2017

[Infra] Loosen the regexp check against result urls of try jobs

WebKit/Tools/Scripts/webkitpy/common/net/git_cl.py invokes the
depot_tools/git_cl.py tool to communicate with Buildbucket, which schedules
and tracks the try jobs. The response from Buildbucket has the build URL.
Now, build URLs can also be like:
  https://ci.chromium.org/swarming/task/38740befcd9c0010?server=chromium-swarm.appspot.com
ci.chromium.org is the new web interface for Chromium continuous
integration job status, project name "Milo"
(https://github.com/luci/luci-go/tree/master/milo).

Only the first 2 urls are considered valid before by the regexp
check in git_cl.py, and now this CL loosens the check to accept
the 3rd one, too.
  https://luci-milo.appspot.com/swarming/task/38740befcd9c0010
  https://luci-milo.appspot.com/swarming/task/38740befcd9c0010/
  https://ci.chromium.org/swarming/task/38740befcd9c0010?server=chromium-swarm.appspot.com

Original error details:
From https://luci-milo.appspot.com/buildbot/chromium.infra.cron/wpt-importer/1383 and
https://luci-logdog.appspot.com/v/?s=chromium%2Fbb%2Fchromium.infra.cron%2Fwpt-importer%2F1383%2F%2B%2Frecipes%2Fsteps%2FImport_changes_from_WPT_to_Chromium%2F0%2Fstdout:
2017-09-07 00:19:43,505 - Triggering CQ try jobs.
Waiting for try jobs, timeout: 10800 seconds.
Waiting for try jobs. 121 seconds passed.
Traceback (most recent call last):
  File "/mnt/data/b/rr/tmpCFXgVV/w/src/third_party/WebKit/Tools/Scripts/wpt-import", line 25, in <module>
    main()
  File "/mnt/data/b/rr/tmpCFXgVV/w/src/third_party/WebKit/Tools/Scripts/wpt-import", line 18, in main
    host.exit(importer.main())
  File "/mnt/data/b/rr/tmpCFXgVV/w/src/third_party/WebKit/Tools/Scripts/webkitpy/w3c/test_importer.py", line 132, in main
    if not self.run_commit_queue_for_cl():
  File "/mnt/data/b/rr/tmpCFXgVV/w/src/third_party/WebKit/Tools/Scripts/webkitpy/w3c/test_importer.py", line 174, in run_commit_queue_for_cl
    timeout_seconds=TIMEOUT_SECONDS)
  File "/mnt/data/b/rr/tmpCFXgVV/w/src/third_party/WebKit/Tools/Scripts/webkitpy/common/net/git_cl.py", line 88, in wait_for_try_jobs
    message=' for try jobs')
  File "/mnt/data/b/rr/tmpCFXgVV/w/src/third_party/WebKit/Tools/Scripts/webkitpy/common/net/git_cl.py", line 108, in _wait_for
    value = poll_function()
  File "/mnt/data/b/rr/tmpCFXgVV/w/src/third_party/WebKit/Tools/Scripts/webkitpy/common/net/git_cl.py", line 78, in finished_try_job_results_or_none
    results = self.try_job_results()
  File "/mnt/data/b/rr/tmpCFXgVV/w/src/third_party/WebKit/Tools/Scripts/webkitpy/common/net/git_cl.py", line 149, in try_job_results
    build_to_status[self._build(result)] = self._try_job_status(result)
  File "/mnt/data/b/rr/tmpCFXgVV/w/src/third_party/WebKit/Tools/Scripts/webkitpy/common/net/git_cl.py", line 179, in _build
    assert match, '%s did not match expected format' % url
AssertionError: https://ci.chromium.org/swarming/task/38740befcd9c0010?server=chromium-swarm.appspot.com did not match expected format

BUG= 762844 

Change-Id: Ie02a351fbe92d78dbd44d28a289011e18ec653c1
Reviewed-on: https://chromium-review.googlesource.com/656606
Commit-Queue: Han Leon <leon.han@intel.com>
Reviewed-by: Quinten Yearsley <qyearsley@chromium.org>
Cr-Commit-Position: refs/heads/master@{#500807}
[modify] https://crrev.com/befdb2547302be64417138865af087be42941c33/third_party/WebKit/Tools/Scripts/webkitpy/common/net/git_cl.py
[modify] https://crrev.com/befdb2547302be64417138865af087be42941c33/third_party/WebKit/Tools/Scripts/webkitpy/common/net/git_cl_unittest.py

Comment 5 by leon....@intel.com, Sep 12 2017

Status: Fixed (was: Started)

Sign in to add a comment