New issue
Advanced search Search tips

Issue 711493 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner:
Closed: Sep 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug



Sign in to add a comment

Some wpt/XMLHttpRequest layout tests fail in non-deterministic ways

Project Member Reported by qyears...@chromium.org, Apr 13 2017

Issue description

Example:
https://storage.googleapis.com/chromium-layout-test-archives/mac10_11_blink_rel/2515/layout-test-results/results.html

Tests:
external/wpt/XMLHttpRequest/responsexml-document-properties.htm
external/wpt/XMLHttpRequest/send-authentication-prompt-2-manual.htm
 

Comment 1 by ricea@chromium.org, Apr 14 2017

Owner: qyears...@chromium.org
Status: Assigned (was: Unconfirmed)
external/wpt/XMLHttpRequest/responsexml-document-properties.htm looks like it depends on events taking a certain amount of time, so it might just be ordinary flaky.

external/wpt/XMLHttpRequest/send-authentication-prompt-2-manual.htm is a manual test and should not be running on bots!

Assigning to qyearsley to get to the bottom of why manual tests are running on bots. Feel free to send it back to me.
Cc: qyears...@chromium.org
Owner: ricea@chromium.org
For why manual tests are now being run on bots: There are some manual tests that we actually do want to run (where actions on the page are automated using JS injected by stuff in LayoutTests/external/wpt_automation).

So, we import and try to run manual tests. For manual tests that are not automated, we add expectations (either Skip, Timeout, or Fail) in TestExpectations. A bit more relevant context in  bug 708241 .

So, passing it back to decide what to do :-D
Right now there are failure expectations: https://cs.chromium.org/chromium/src/third_party/WebKit/LayoutTests/TestExpectations?l=2735.
And, there are more failure expectations at: https://cs.chromium.org/chromium/src/third_party/WebKit/LayoutTests/TestExpectations?l=1885

Maybe for now we just want to reorganize those expectations so that it's clear what all of the XMLHttpRequest failures are, which tests have problems we intend to fix. For any tests that we don't care about and don't want to run (like maybe that -manual.html test), they could have [ WontFix ] lines added to NeverFixTests.

All recent results for all XMLHttpRequest tests on all platforms (page takes a while to load): https://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=webkit_tests&tests=external/wpt/XMLHttpRequest

Comment 3 by ricea@chromium.org, Apr 17 2017

Owner: yhirano@chromium.org
Sending to our wpt triage expert, yhirano.
Status: WontFix (was: Assigned)
OK, the manual test looks manual.
The other test looks testing something related to DOM - so I filed a new bug 765116.

I'm closing this bug as WontFix.
Project Member

Comment 5 by bugdroid1@chromium.org, Sep 15 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/f0df391184a1d84e44ca60159e9a83faab5d49d0

commit f0df391184a1d84e44ca60159e9a83faab5d49d0
Author: Yutaka Hirano <yhirano@chromium.org>
Date: Fri Sep 15 01:45:29 2017

Change bug for responsexml-document-properties.htm in TestExpectation

This CL also adds a comment for send-authentication-prompt-2-manual.htm
expectation entry.

Bug: 765116,  711493 
Change-Id: I0ec3b1228a18d089f805debf8cff3aaa3b346410
No-Try: True
Reviewed-on: https://chromium-review.googlesource.com/666860
Commit-Queue: Yutaka Hirano <yhirano@chromium.org>
Reviewed-by: Adam Rice <ricea@chromium.org>
Cr-Commit-Position: refs/heads/master@{#502128}
[modify] https://crrev.com/f0df391184a1d84e44ca60159e9a83faab5d49d0/third_party/WebKit/LayoutTests/TestExpectations

Project Member

Comment 6 by bugdroid1@chromium.org, Jan 15 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/264cc0594f6a036e628726f1f2d3afb622f4c0e5

commit 264cc0594f6a036e628726f1f2d3afb622f4c0e5
Author: Robert Ma <robertma@chromium.org>
Date: Mon Jan 15 16:54:41 2018

Skip some manual WPT that slipped through

At f51689179bfbf2a29e2870e5d5081b51fa49a6a1, three manual Web Platform
Tests were imported without being skipped in expectations, and were
incorrectly rebaselined. The importer back then might have bugs skipping
manual tests, which is no longer an issue any more. The issue is found
when setting up a new platform (Mac 10.13).

Verified these are the only cases by grepping "LayoutBlock" among all
WPT baselines. ("LayoutBlock" is Blink's internal layout dump, which
should not appear in outputs of WPT at all.)

Also triaged TestExpectations and moved some manual tests into
NeverFixTests. And moved a line for pointerevents test to its
friends, with the more appropriate bug link.

Bug:  774301 , 626703,  711493 ,  706118 , 613672
Change-Id: I8316a38c61f175e3e7ae2bc786133ad82a4598be
Reviewed-on: https://chromium-review.googlesource.com/865656
Reviewed-by: Quinten Yearsley <qyearsley@chromium.org>
Commit-Queue: Robert Ma <robertma@chromium.org>
Cr-Commit-Position: refs/heads/master@{#529300}
[modify] https://crrev.com/264cc0594f6a036e628726f1f2d3afb622f4c0e5/third_party/WebKit/LayoutTests/NeverFixTests
[modify] https://crrev.com/264cc0594f6a036e628726f1f2d3afb622f4c0e5/third_party/WebKit/LayoutTests/TestExpectations
[delete] https://crrev.com/7337e34e0939dea633e9d7f50635cbb9f456b350/third_party/WebKit/LayoutTests/platform/linux/external/wpt/html-media-capture/capture_audio_cancel-manual-expected.txt
[delete] https://crrev.com/7337e34e0939dea633e9d7f50635cbb9f456b350/third_party/WebKit/LayoutTests/platform/linux/external/wpt/html-media-capture/capture_image_cancel-manual-expected.txt
[delete] https://crrev.com/7337e34e0939dea633e9d7f50635cbb9f456b350/third_party/WebKit/LayoutTests/platform/linux/external/wpt/html-media-capture/capture_video_cancel-manual-expected.txt
[delete] https://crrev.com/7337e34e0939dea633e9d7f50635cbb9f456b350/third_party/WebKit/LayoutTests/platform/mac-mac10.10/external/wpt/html-media-capture/capture_audio_cancel-manual-expected.txt
[delete] https://crrev.com/7337e34e0939dea633e9d7f50635cbb9f456b350/third_party/WebKit/LayoutTests/platform/mac-mac10.10/external/wpt/html-media-capture/capture_image_cancel-manual-expected.txt
[delete] https://crrev.com/7337e34e0939dea633e9d7f50635cbb9f456b350/third_party/WebKit/LayoutTests/platform/mac-mac10.10/external/wpt/html-media-capture/capture_video_cancel-manual-expected.txt
[delete] https://crrev.com/7337e34e0939dea633e9d7f50635cbb9f456b350/third_party/WebKit/LayoutTests/platform/mac-mac10.11/external/wpt/html-media-capture/capture_audio_cancel-manual-expected.txt
[delete] https://crrev.com/7337e34e0939dea633e9d7f50635cbb9f456b350/third_party/WebKit/LayoutTests/platform/mac-mac10.11/external/wpt/html-media-capture/capture_image_cancel-manual-expected.txt
[delete] https://crrev.com/7337e34e0939dea633e9d7f50635cbb9f456b350/third_party/WebKit/LayoutTests/platform/mac-mac10.11/external/wpt/html-media-capture/capture_video_cancel-manual-expected.txt
[delete] https://crrev.com/7337e34e0939dea633e9d7f50635cbb9f456b350/third_party/WebKit/LayoutTests/platform/mac/external/wpt/html-media-capture/capture_audio_cancel-manual-expected.txt
[delete] https://crrev.com/7337e34e0939dea633e9d7f50635cbb9f456b350/third_party/WebKit/LayoutTests/platform/mac/external/wpt/html-media-capture/capture_image_cancel-manual-expected.txt
[delete] https://crrev.com/7337e34e0939dea633e9d7f50635cbb9f456b350/third_party/WebKit/LayoutTests/platform/mac/external/wpt/html-media-capture/capture_video_cancel-manual-expected.txt
[delete] https://crrev.com/7337e34e0939dea633e9d7f50635cbb9f456b350/third_party/WebKit/LayoutTests/platform/win/external/wpt/html-media-capture/capture_audio_cancel-manual-expected.txt
[delete] https://crrev.com/7337e34e0939dea633e9d7f50635cbb9f456b350/third_party/WebKit/LayoutTests/platform/win/external/wpt/html-media-capture/capture_image_cancel-manual-expected.txt
[delete] https://crrev.com/7337e34e0939dea633e9d7f50635cbb9f456b350/third_party/WebKit/LayoutTests/platform/win/external/wpt/html-media-capture/capture_video_cancel-manual-expected.txt

Sign in to add a comment