New issue
Advanced search Search tips

Issue 803200 link

Starred by 1 user
Status: Fixed
Owner:
ricea@chromium.org
Closed: Sep 4
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug



Sign in to add a comment

external/wpt/websockets/cookies/006.html?wss fails with non-deterministic outputs

Project Member Reported by robertma@chromium.org, Jan 17 2018 
The output contains both a timestamp and a Math.random(), so the test cannot be rebaselined.
https://cs.chromium.org/chromium/src/third_party/WebKit/LayoutTests/external/wpt/websockets/cookies/006.html

Example output:
FAIL WebSockets: setting Secure cookie with document.cookie, checking ws request assert_regexp_match: expected object "/ws_test_1516216745283.0.9507825479042662=test/" but got "(none)"

Comment 1 by ricea@chromium.org, Jan 18 2018

Owner: ricea@chromium.org
Status: Assigned (was: Untriaged)
No-one really supports the wpt WebSocket tests. Because the major browser are more compatible with each other than they are with the tests, there's little motivation for anyone to work on them. I have fixed some things from time-to-time but there's still a lot wrong with them and it's hard to justify the time investment.

Having said that, it should be pretty to make failure text deterministic here, so I will add it to my pile.

Comment 2 by ricea@chromium.org, Jan 18 2018 

I was scratching my head as to why this test was failing in the first place. But then I remembered that Chrome won't let you set a secure cookie from Javascript on an insecure origin.

I'm not sure whether Chrome's behaviour is standard. If it is, then the test expectation can simply be fixed.
Project Member

Comment 3 by bugdroid1@chromium.org, Jan 18 2018 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/901dfc8c50721bc2fa7a4513b21e685894655539

commit 901dfc8c50721bc2fa7a4513b21e685894655539
Author: Robert Ma <robertma@chromium.org>
Date: Thu Jan 18 21:53:19 2018

Rebaseline tests with query strings in the names

Apart from a few timed out tests, one test has non-deterministic output, and
the other is pass-failure flaky.

Bug: 800570, 709227,  803200 , 803558
Change-Id: I86d5a14f26fcab6b862cd2e1b2108c10db4ee94e
Reviewed-on: https://chromium-review.googlesource.com/871138
Commit-Queue: Robert Ma <robertma@chromium.org>
Reviewed-by: Quinten Yearsley <qyearsley@chromium.org>
Cr-Commit-Position: refs/heads/master@{#530284}
[modify] https://crrev.com/901dfc8c50721bc2fa7a4513b21e685894655539/third_party/WebKit/LayoutTests/TestExpectations
[add] https://crrev.com/901dfc8c50721bc2fa7a4513b21e685894655539/third_party/WebKit/LayoutTests/external/wpt/html/syntax/parsing/html5lib_menuitem-element_run_type=uri-expected.txt
[add] https://crrev.com/901dfc8c50721bc2fa7a4513b21e685894655539/third_party/WebKit/LayoutTests/external/wpt/html/syntax/parsing/html5lib_menuitem-element_run_type=write-expected.txt
[add] https://crrev.com/901dfc8c50721bc2fa7a4513b21e685894655539/third_party/WebKit/LayoutTests/external/wpt/html/syntax/parsing/html5lib_menuitem-element_run_type=write_single-expected.txt
[add] https://crrev.com/901dfc8c50721bc2fa7a4513b21e685894655539/third_party/WebKit/LayoutTests/external/wpt/html/syntax/parsing/html5lib_template_run_type=uri-expected.txt
[add] https://crrev.com/901dfc8c50721bc2fa7a4513b21e685894655539/third_party/WebKit/LayoutTests/external/wpt/html/syntax/parsing/html5lib_template_run_type=write-expected.txt
[add] https://crrev.com/901dfc8c50721bc2fa7a4513b21e685894655539/third_party/WebKit/LayoutTests/external/wpt/html/syntax/parsing/html5lib_template_run_type=write_single-expected.txt
[add] https://crrev.com/901dfc8c50721bc2fa7a4513b21e685894655539/third_party/WebKit/LayoutTests/external/wpt/html/syntax/parsing/html5lib_tests11_run_type=uri-expected.txt
[add] https://crrev.com/901dfc8c50721bc2fa7a4513b21e685894655539/third_party/WebKit/LayoutTests/external/wpt/html/syntax/parsing/html5lib_tests11_run_type=write-expected.txt
[add] https://crrev.com/901dfc8c50721bc2fa7a4513b21e685894655539/third_party/WebKit/LayoutTests/external/wpt/html/syntax/parsing/html5lib_tests11_run_type=write_single-expected.txt
[add] https://crrev.com/901dfc8c50721bc2fa7a4513b21e685894655539/third_party/WebKit/LayoutTests/external/wpt/html/syntax/parsing/html5lib_tests25_run_type=uri-expected.txt
[add] https://crrev.com/901dfc8c50721bc2fa7a4513b21e685894655539/third_party/WebKit/LayoutTests/external/wpt/html/syntax/parsing/html5lib_tests25_run_type=write-expected.txt
[add] https://crrev.com/901dfc8c50721bc2fa7a4513b21e685894655539/third_party/WebKit/LayoutTests/external/wpt/html/syntax/parsing/html5lib_tests25_run_type=write_single-expected.txt
[add] https://crrev.com/901dfc8c50721bc2fa7a4513b21e685894655539/third_party/WebKit/LayoutTests/external/wpt/html/syntax/parsing/html5lib_webkit02_run_type=uri-expected.txt
[add] https://crrev.com/901dfc8c50721bc2fa7a4513b21e685894655539/third_party/WebKit/LayoutTests/external/wpt/html/syntax/parsing/html5lib_webkit02_run_type=write-expected.txt
[add] https://crrev.com/901dfc8c50721bc2fa7a4513b21e685894655539/third_party/WebKit/LayoutTests/external/wpt/html/syntax/parsing/html5lib_webkit02_run_type=write_single-expected.txt
[add] https://crrev.com/901dfc8c50721bc2fa7a4513b21e685894655539/third_party/WebKit/LayoutTests/external/wpt/websockets/constructor/002-expected.txt
[add] https://crrev.com/901dfc8c50721bc2fa7a4513b21e685894655539/third_party/WebKit/LayoutTests/external/wpt/websockets/constructor/002_wss-expected.txt
[add] https://crrev.com/901dfc8c50721bc2fa7a4513b21e685894655539/third_party/WebKit/LayoutTests/external/wpt/websockets/cookies/007-expected.txt
[add] https://crrev.com/901dfc8c50721bc2fa7a4513b21e685894655539/third_party/WebKit/LayoutTests/external/wpt/websockets/cookies/007_wss-expected.txt
[add] https://crrev.com/901dfc8c50721bc2fa7a4513b21e685894655539/third_party/WebKit/LayoutTests/external/wpt/websockets/interfaces/WebSocket/close/close-connecting-expected.txt
[add] https://crrev.com/901dfc8c50721bc2fa7a4513b21e685894655539/third_party/WebKit/LayoutTests/external/wpt/websockets/interfaces/WebSocket/close/close-connecting_wss-expected.txt
[add] https://crrev.com/901dfc8c50721bc2fa7a4513b21e685894655539/third_party/WebKit/LayoutTests/external/wpt/websockets/interfaces/WebSocket/close/close-nested-expected.txt
[add] https://crrev.com/901dfc8c50721bc2fa7a4513b21e685894655539/third_party/WebKit/LayoutTests/external/wpt/websockets/interfaces/WebSocket/close/close-nested_wss-expected.txt
[add] https://crrev.com/901dfc8c50721bc2fa7a4513b21e685894655539/third_party/WebKit/LayoutTests/external/wpt/websockets/interfaces/WebSocket/readyState/003-expected.txt
[add] https://crrev.com/901dfc8c50721bc2fa7a4513b21e685894655539/third_party/WebKit/LayoutTests/external/wpt/websockets/interfaces/WebSocket/readyState/003_wss-expected.txt
[add] https://crrev.com/901dfc8c50721bc2fa7a4513b21e685894655539/third_party/WebKit/LayoutTests/external/wpt/websockets/unload-a-document/002-expected.txt
[add] https://crrev.com/901dfc8c50721bc2fa7a4513b21e685894655539/third_party/WebKit/LayoutTests/external/wpt/websockets/unload-a-document/002_wss-expected.txt

Comment 4 by ricea@chromium.org, Sep 4

Status: Fixed (was: Assigned)
Project Member

Comment 5 by bugdroid1@chromium.org, Sep 4 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/5428f3ffcf4cee621c6a5b91faec1215703c755f

commit 5428f3ffcf4cee621c6a5b91faec1215703c755f
Author: Adam Rice <ricea@chromium.org>
Date: Tue Sep 04 08:02:04 2018

[WebSocket] Fix cookies/006.html?wss wpt

The websockets/cookies/006.html?wss test didn't work on any modern
browser because you can't set a secure cookie from JavaScript on an
insecure page. See
https://wpt.fyi/results/websockets/cookies/006.html%3Fwss.

Change the expectations to never see the cookie set.

Add a new test, 006.https.html, which runs over https so that the cookie
can be set. Unfortunately, due to mixed content checks, this can't test
the insecure "ws:" scheme, so it still doesn't cover all the cases the
original test intended to.

BUG= 803200 

Change-Id: I7e280ebc781c693529588a68cd5eef5332b730fa
Reviewed-on: https://chromium-review.googlesource.com/1203495
Reviewed-by: Yutaka Hirano <yhirano@chromium.org>
Commit-Queue: Adam Rice <ricea@chromium.org>
Cr-Commit-Position: refs/heads/master@{#588469}
[modify] https://crrev.com/5428f3ffcf4cee621c6a5b91faec1215703c755f/third_party/WebKit/LayoutTests/TestExpectations
[modify] https://crrev.com/5428f3ffcf4cee621c6a5b91faec1215703c755f/third_party/WebKit/LayoutTests/external/wpt/websockets/cookies/006.html
[add] https://crrev.com/5428f3ffcf4cee621c6a5b91faec1215703c755f/third_party/WebKit/LayoutTests/external/wpt/websockets/cookies/006.https.html
Project Member

Comment 6 by bugdroid1@chromium.org, Sep 4 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/5428f3ffcf4cee621c6a5b91faec1215703c755f

commit 5428f3ffcf4cee621c6a5b91faec1215703c755f
Author: Adam Rice <ricea@chromium.org>
Date: Tue Sep 04 08:02:04 2018

[WebSocket] Fix cookies/006.html?wss wpt

The websockets/cookies/006.html?wss test didn't work on any modern
browser because you can't set a secure cookie from JavaScript on an
insecure page. See
https://wpt.fyi/results/websockets/cookies/006.html%3Fwss.

Change the expectations to never see the cookie set.

Add a new test, 006.https.html, which runs over https so that the cookie
can be set. Unfortunately, due to mixed content checks, this can't test
the insecure "ws:" scheme, so it still doesn't cover all the cases the
original test intended to.

BUG= 803200 

Change-Id: I7e280ebc781c693529588a68cd5eef5332b730fa
Reviewed-on: https://chromium-review.googlesource.com/1203495
Reviewed-by: Yutaka Hirano <yhirano@chromium.org>
Commit-Queue: Adam Rice <ricea@chromium.org>
Cr-Commit-Position: refs/heads/master@{#588469}
[modify] https://crrev.com/5428f3ffcf4cee621c6a5b91faec1215703c755f/third_party/WebKit/LayoutTests/TestExpectations
[modify] https://crrev.com/5428f3ffcf4cee621c6a5b91faec1215703c755f/third_party/WebKit/LayoutTests/external/wpt/websockets/cookies/006.html
[add] https://crrev.com/5428f3ffcf4cee621c6a5b91faec1215703c755f/third_party/WebKit/LayoutTests/external/wpt/websockets/cookies/006.https.html
Project Member

Comment 7 by bugdroid1@chromium.org, Sep 4 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/5428f3ffcf4cee621c6a5b91faec1215703c755f

commit 5428f3ffcf4cee621c6a5b91faec1215703c755f
Author: Adam Rice <ricea@chromium.org>
Date: Tue Sep 04 08:02:04 2018

[WebSocket] Fix cookies/006.html?wss wpt

The websockets/cookies/006.html?wss test didn't work on any modern
browser because you can't set a secure cookie from JavaScript on an
insecure page. See
https://wpt.fyi/results/websockets/cookies/006.html%3Fwss.

Change the expectations to never see the cookie set.

Add a new test, 006.https.html, which runs over https so that the cookie
can be set. Unfortunately, due to mixed content checks, this can't test
the insecure "ws:" scheme, so it still doesn't cover all the cases the
original test intended to.

BUG= 803200 

Change-Id: I7e280ebc781c693529588a68cd5eef5332b730fa
Reviewed-on: https://chromium-review.googlesource.com/1203495
Reviewed-by: Yutaka Hirano <yhirano@chromium.org>
Commit-Queue: Adam Rice <ricea@chromium.org>
Cr-Commit-Position: refs/heads/master@{#588469}
[modify] https://crrev.com/5428f3ffcf4cee621c6a5b91faec1215703c755f/third_party/WebKit/LayoutTests/TestExpectations
[modify] https://crrev.com/5428f3ffcf4cee621c6a5b91faec1215703c755f/third_party/WebKit/LayoutTests/external/wpt/websockets/cookies/006.html
[add] https://crrev.com/5428f3ffcf4cee621c6a5b91faec1215703c755f/third_party/WebKit/LayoutTests/external/wpt/websockets/cookies/006.https.html
►

Sign in to add a comment