New issue
Advanced search Search tips

Issue 745099 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Jul 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug



Sign in to add a comment

[WPT import] Importer didn't update expectations and didn't trigger CQ, and aborted.

Project Member Reported by leon....@intel.com, Jul 18 2017

Issue description

The auto import CL https://chromium-review.googlesource.com/c/575489/ failed because of https://luci-logdog.appspot.com/v/?s=chromium%2Fbb%2Fchromium.infra.cron%2Fwpt-importer%2F332%2F%2B%2Frecipes%2Fsteps%2FImport_changes_from_WPT_to_Chromium%2F0%2Fstdout

Detailed logs:
Triggering try jobs.
Waiting for try jobs (timeout: 10800 seconds).
Waiting. 121 seconds passed.
Waiting. 362 seconds passed.
Waiting. 604 seconds passed.
Waiting. 845 seconds passed.
Waiting. 1087 seconds passed.
Waiting. 1329 seconds passed.
Waiting. 1571 seconds passed.
Waiting. 1813 seconds passed.
Waiting. 2061 seconds passed.
Waiting. 2302 seconds passed.
Waiting. 2546 seconds passed.
Waiting. 2788 seconds passed.
Waiting. 3030 seconds passed.
Waiting. 3272 seconds passed.
Waiting. 3513 seconds passed.
Waiting. 3754 seconds passed.
Waiting. 3995 seconds passed.
Waiting. 4236 seconds passed.
Waiting. 4478 seconds passed.
Waiting. 4719 seconds passed.
Waiting. 4961 seconds passed.
Waiting. 5203 seconds passed.
Waiting. 5444 seconds passed.
Waiting. 5686 seconds passed.
Adding test expectations lines to LayoutTests/TestExpectations.
All jobs finished.
No results for build Build(builder_name=u'android_blink_rel', build_number=2968)
Tests to rebaseline:
Lines to write to TestExpectations:
Waiting for try jobs (timeout: 10800 seconds).
All jobs finished.
CQ appears to have failed; aborting.
step returned non-zero exit code: 1

 
Cc: -qyears...@chromium.org
Labels: -Pri-3 Pri-2
Owner: qyears...@chromium.org
Status: Assigned (was: Untriaged)
Summary: [WPT import] Importer didn't update expectations and didn't trigger CQ, and aborted. (was: [WPT import] No results for build Build(builder_name=u'android_blink_rel', build_number=2968))
The failure on android is not too unusual; android fails sometimes but doesn't usually affect import because newly imported tests aren't actually run on android. (Only a subset of tests are run on android, which only includes less than 50 web-platform-tests.

So, ideally, what I *think* should have happened should have been:

Triggering try jobs.
Waiting for try jobs (timeout: 10800 seconds).
Waiting. 121 seconds passed.
...
Waiting. 5686 seconds passed.
Adding test expectations lines to LayoutTests/TestExpectations.
All jobs finished.
No results for build Build(builder_name=u'android_blink_rel', build_number=2968)
Tests to rebaseline:
==> [Some list of tests, maybe the tests that failed on win10_blink_rel?]
Lines to write to TestExpectations:
==> [Some list of lines to write?]
==> Triggering CQ dry run.
Waiting for try jobs (timeout: 10800 seconds).
Waiting. 121 seconds passed.
...
All jobs finished.
==> CQ appears to have [failed/passed].
==> [Either land CL or abort.]


That is:
 1. It should have updated the test expectations/baselines, or at least printed a better message if there was nothing to update.
 2. It should have triggered the CQ and waited for CQ jobs to finish.
Project Member

Comment 3 by bugdroid1@chromium.org, Jul 18 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/e2ee51149ca9feef6ae60ab61c9b9efa5da79955

commit e2ee51149ca9feef6ae60ab61c9b9efa5da79955
Author: Quinten Yearsley <qyearsley@google.com>
Date: Tue Jul 18 23:13:57 2017

Import wpt@40e04eba01c2e35800a70261059495ebbde33309 (manual)

Using wpt-import in Chromium fcfcaa9965c21f1652f8616cc51ec1caa4603961.

Reason for manual import:
 1. Trying to see why wpt-update-expectations didn't update expectations
    for tests that failed on Windows in recent jobs.
 2. Also avoiding clobbering https://github.com/w3c/web-platform-tests/pull/6504

No-Export: true
Bug:  745099 
Change-Id: I48628d41aeaa7d452425330643020b1a2157b05a
Reviewed-on: https://chromium-review.googlesource.com/576267
Reviewed-by: Jeff Carpenter <jeffcarp@chromium.org>
Commit-Queue: Quinten Yearsley <qyearsley@chromium.org>
Cr-Commit-Position: refs/heads/master@{#487655}
[modify] https://crrev.com/e2ee51149ca9feef6ae60ab61c9b9efa5da79955/third_party/WebKit/LayoutTests/TestExpectations
[modify] https://crrev.com/e2ee51149ca9feef6ae60ab61c9b9efa5da79955/third_party/WebKit/LayoutTests/external/WPT_BASE_MANIFEST.json
[add] https://crrev.com/e2ee51149ca9feef6ae60ab61c9b9efa5da79955/third_party/WebKit/LayoutTests/external/wpt/XMLHttpRequest/allow-lists-starting-with-comma.htm
[add] https://crrev.com/e2ee51149ca9feef6ae60ab61c9b9efa5da79955/third_party/WebKit/LayoutTests/external/wpt/XMLHttpRequest/resources/access-control-allow-lists.py
[add] https://crrev.com/e2ee51149ca9feef6ae60ab61c9b9efa5da79955/third_party/WebKit/LayoutTests/external/wpt/mediacapture-fromelement/capture.html
[add] https://crrev.com/e2ee51149ca9feef6ae60ab61c9b9efa5da79955/third_party/WebKit/LayoutTests/external/wpt/mediacapture-fromelement/creation.html
[add] https://crrev.com/e2ee51149ca9feef6ae60ab61c9b9efa5da79955/third_party/WebKit/LayoutTests/external/wpt/mediacapture-fromelement/ended.html
[add] https://crrev.com/e2ee51149ca9feef6ae60ab61c9b9efa5da79955/third_party/WebKit/LayoutTests/external/wpt/mediacapture-fromelement/webm/test-a-128k-44100Hz-1ch.webm
[add] https://crrev.com/e2ee51149ca9feef6ae60ab61c9b9efa5da79955/third_party/WebKit/LayoutTests/external/wpt/mediacapture-fromelement/webm/test-av-384k-44100Hz-1ch-320x240-30fps-10kfr.webm
[add] https://crrev.com/e2ee51149ca9feef6ae60ab61c9b9efa5da79955/third_party/WebKit/LayoutTests/external/wpt/mediacapture-fromelement/webm/test-v-128k-320x240-24fps-8kfr.webm

Status: Fixed (was: Assigned)
Haven't seen this again in the last few days, and now the log messages are improved; will reopen if I see this again.

Sign in to add a comment