Issue metadata
Sign in to add a comment
|
WPT Import: Tests already in TestExpectations can end up being duplicated |
||||||||||||||||||||||||
Issue descriptionThis has been causing WPT imports to fail for the past few days. From https://ci.chromium.org/buildbot/chromium.infra.cron/wpt-importer/8733, for example: 2017-12-27 01:44:12,290 - All jobs finished. 2017-12-27 01:44:12,291 - Adding test expectations lines to LayoutTests/TestExpectations. 2017-12-27 01:44:23,554 - Tests to rebaseline: 2017-12-27 01:44:23,555 - external/wpt/websockets/opening-handshake/005.html?wss 2017-12-27 01:44:42,301 - Lines to write to TestExpectations: 2017-12-27 01:44:42,302 - crbug.com/626703 external/wpt/websockets/keeping-connection-open/001.html?wss [ Timeout ] 2017-12-27 01:44:42,302 - crbug.com/626703 external/wpt/websockets/constructor/014.html?wss [ Timeout ] 2017-12-27 01:44:43,269 - Generating MANIFEST.json 2017-12-27 01:44:48,940 - Committing changes. Traceback (most recent call last): File "/mnt/data/b/rr/tmp9ekTxk/w/src/third_party/WebKit/Tools/Scripts/wpt-import", line 24, in <module> main() File "/mnt/data/b/rr/tmp9ekTxk/w/src/third_party/WebKit/Tools/Scripts/wpt-import", line 17, in main host.exit(importer.main()) File "/mnt/data/b/rr/tmp9ekTxk/w/src/third_party/WebKit/Tools/Scripts/webkitpy/w3c/test_importer.py", line 162, in main if not self.update_expectations_for_cl(): File "/mnt/data/b/rr/tmp9ekTxk/w/src/third_party/WebKit/Tools/Scripts/webkitpy/w3c/test_importer.py", line 208, in update_expectations_for_cl self._upload_patchset(message) File "/mnt/data/b/rr/tmp9ekTxk/w/src/third_party/WebKit/Tools/Scripts/webkitpy/w3c/test_importer.py", line 435, in _upload_patchset self.git_cl.run(['upload', '-f', '-t', message, '--gerrit']) File "/mnt/data/b/rr/tmp9ekTxk/w/src/third_party/WebKit/Tools/Scripts/webkitpy/common/net/git_cl.py", line 60, in run return self._host.executive.run_command(command, cwd=self._cwd) File "/mnt/data/b/rr/tmp9ekTxk/w/src/third_party/WebKit/Tools/Scripts/webkitpy/common/system/executive.py", line 337, in run_command (error_handler or self.default_error_handler)(script_error) File "/mnt/data/b/rr/tmp9ekTxk/w/src/third_party/WebKit/Tools/Scripts/webkitpy/common/system/executive.py", line 244, in default_error_handler raise error webkitpy.common.system.executive.ScriptError: Failed to run "['git', 'cl', 'upload', '-f', '-t', 'Update test expectations and baselines.', '--gerrit']" exit_code: 1 output: Last 500 characters of output: ml?wss third_party/WebKit/LayoutTests/TestExpectations:2782 Duplicate or ambiguous entry lines third_party/WebKit/LayoutTests/TestExpectations:1931 and third_party/WebKit/LayoutTests/TestExpectations:2782. external/wpt/websockets/keeping-connection-open/001.html?wss Lint failed. external/wpt/websockets/keeping-connection-open/001.html?wss has an entry in TestExpectations that current looks like this: crbug.com/709227 external/wpt/websockets/keeping-connection-open/001.html?wss [ Failure ] whereas wpt_expectations_updater.py is trying to add a new entry that looks like this: crbug.com/626703 external/wpt/websockets/keeping-connection-open/001.html?wss [ Timeout ] instead of updating the existing TestExpecations entry.
,
Dec 27 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/8259896f3b644a5ce426e8367138c55a05bf9aec commit 8259896f3b644a5ce426e8367138c55a05bf9aec Author: Raphael Kubo da Costa <raphael.kubo.da.costa@intel.com> Date: Wed Dec 27 12:01:00 2017 Manually mark a few external/wpt/websockets tests as timing out. Recent WPT imports (post commit 3975ca81, "Fix some websockets tests") are failing because the WPT code is attempting to add new TestExpectation entries for some tests that are already there. For now, unblock imports by manually expecting external/wpt/websockets/constructor/014.html?wss and crbug.com/709227 external/wpt/websockets/keeping-connection-open/001.html?wss to both fail and time out. TBR=robertma@chromium.org Bug: 797743 Change-Id: I5c6b381a30eea4428472c77253cf5291c65d71dc No-Try: True Reviewed-on: https://chromium-review.googlesource.com/844321 Reviewed-by: Raphael Kubo da Costa <raphael.kubo.da.costa@intel.com> Commit-Queue: Raphael Kubo da Costa <raphael.kubo.da.costa@intel.com> Cr-Commit-Position: refs/heads/master@{#526216} [modify] https://crrev.com/8259896f3b644a5ce426e8367138c55a05bf9aec/third_party/WebKit/LayoutTests/TestExpectations
,
Dec 27 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/a9fd9eb2ade18dbd028703a7f6bcb862923c444a commit a9fd9eb2ade18dbd028703a7f6bcb862923c444a Author: Raphael Kubo da Costa <raphael.kubo.da.costa@intel.com> Date: Wed Dec 27 18:42:57 2017 Accept failures in external/wpt/websockets/opening-handshake/005.html?wss Recent WPT imports (post commit 3975ca81, "Fix some websockets tests") are failing because the WPT code is attempting to mark the test above as timing out only on Mac10.3 even though the test is actually failing on Linux (Mac10.3 is not even being tested in the trybots). For now, unblock imports by expecting failures in the test above so that the WPT code (hopefully) does not try to update its corresponding TestExpectations line. TBR=robertma@chromium.org Bug: 797743 Change-Id: I2bbf5a24b13219e9d2880a7ce1afe36f042348e7 No-Try: True Reviewed-on: https://chromium-review.googlesource.com/844816 Reviewed-by: Raphael Kubo da Costa <raphael.kubo.da.costa@intel.com> Commit-Queue: Raphael Kubo da Costa <raphael.kubo.da.costa@intel.com> Cr-Commit-Position: refs/heads/master@{#526233} [modify] https://crrev.com/a9fd9eb2ade18dbd028703a7f6bcb862923c444a/third_party/WebKit/LayoutTests/TestExpectations
,
Dec 28 2017
Regarding comment #3: I'm not totally sure why a failure in the linux_trusty_blink_rel trybot would cause the [Mac10.3] specifier to be added to TestExpectations.
,
Jan 4 2018
An example of the weird [Mac10.13] specifier can be found at https://ci.chromium.org/buildbot/chromium.infra.cron/wpt-importer/8736 Note that the log says: 2017-12-27 10:25:19,749 - Adding test expectations lines to LayoutTests/TestExpectations. 2017-12-27 10:25:26,749 - No results for build Build(builder_name=u'android_blink_rel', build_number=5051) 2017-12-27 10:25:28,191 - Tests to rebaseline: 2017-12-27 10:25:28,192 - external/wpt/websockets/opening-handshake/005.html?wss 2017-12-27 10:25:47,843 - No lines to write to TestExpectations. which means the expectation line was modified by rebaseline-cl, not added by wpt_expectations_updater. The latter case happens when a test cannot be rebaselined. I wasn't actually aware that rebaseline{-cl} might also modify TestExpectations. There might be a bug. Nevertheless, the changes should be logged. I also have a feeling that this is related to issue 799076 .
,
Jan 9 2018
The lingering [ Mac10.13 ] specifier was caused by issue 800557 . The original issue in the top description, however, is a serious problem I didn't know before. Filed a new bug and merging this issue there. |
|||||||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||||||
Comment 1 by raphael....@intel.com
, Dec 27 2017