New issue
Advanced search Search tips

Issue 602886 link

Starred by 3 users

Issue metadata

Status: WontFix
Owner: ----
Closed: Jul 18
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: All
Pri: 3
Type: Feature



Sign in to add a comment

testharness.js tests should fail on uncaught exceptions before testharness.js is loaded.

Project Member Reported by alancutter@chromium.org, Apr 13 2016

Issue description

If exceptions are thrown before the testharness.js script is loaded in a LayoutTest the test passes. These should instead cause the test to fail.
 

Comment 2 by sshru...@google.com, Apr 18 2016

Components: -Blink>ToolsTest Blink>Infra
Blink>ToolsTest renamed to Blink>Infra
Update: Uploaded a patch some time ago to make run-webkit-tests treat console warnings and errors as failures.
https://codereview.chromium.org/1885703003

Ran into problems with expected warnings that included nonces that differed with each run.
Cc: alancutter@chromium.org
Owner: ----
Status: Available (was: Assigned)
Summary: testharness.js tests don't fail on uncaught exceptions. (was: testharness.js tests don't fail on uncaught exceptions)
Hi Alan -- so this is still an issue, but it's not as easy to fix as originally hoped, since there a lots of tests that have console error messages before loading testharness.js, right?

And if we want to fix this, then we might first want to change all those tests that have console errors before testharness.js is loaded?
Summary: testharness.js tests don't fail on uncaught exceptions before testharness.js is loaded. (was: testharness.js tests don't fail on uncaught exceptions.)
There are many tests where console error are intentional. For most of them we can just make expectation files, others have randomised output due to nonce tokens in the URL and are much harder to write expectations for. I should have kept a record of which tests they were, from memory I think they were in the http directory.

My guess is that progress on this patch is blocked on making run-webkit-tests use a seeded random function.
Labels: -Type-Bug Type-Feature
Summary: testharness.js tests should fail on uncaught exceptions before testharness.js is loaded. (was: testharness.js tests don't fail on uncaught exceptions before testharness.js is loaded.)

Comment 8 by suzyh@chromium.org, Jun 13 2017

Cc: -suzyh@chromium.org
Project Member

Comment 9 by sheriffbot@chromium.org, Jun 13 2018

Labels: Hotlist-Recharge-Cold
Status: Untriaged (was: Available)
This issue has been Available for over a year. If it's no longer important or seems unlikely to be fixed, please consider closing it out. If it is important, please re-triage the issue.

Sorry for the inconvenience if the bug really should have been left as Available.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
Components: Blink>Infra>Ecosystem
Status: WontFix (was: Untriaged)
I created https://jsbin.com/yisiwol/edit?html,output to test this, and unsurprisingly found that the test passed.

It's important that the behavior when running tests is as similar as possible in upstream wpt and in blink, so we'd want to discuss this in upstream first. alancutter@, if you still think this is important, can you file an issue at https://github.com/web-platform-tests/wpt/issues? Thanks!

Sign in to add a comment