New issue
Advanced search Search tips

Issue 868374 link

Starred by 2 users

Issue metadata

Status: Assigned
Owner: ----
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug



Sign in to add a comment

run_web_tests.py --reset-results doesn't work with wpt tests with ? in filename/path

Project Member Reported by foolip@chromium.org, Jul 27

Issue description

We have idlharness tests like these in dom/ and html/:
https://wpt.fyi/results/dom?sha=c54b451185
https://wpt.fyi/results/html/dom?sha=c54b451185

Trying to change one of these tests in Chromium and updating the baselines locally is very hard.

`run_web_tests.py --reset-results external/wpt/dom/interfaces.html` creates a external/wpt/dom/interfaces-expected.txt instead of the interfaces_exclude=Node-expected.txt currently there, but the file contents is the same, so one can move it into place.

`run_web_tests.py --reset-results 'external/wpt/html/dom/interfaces.https.html?exclude=(Document|Window|HTML.*)'` is different, it creates a file called 'interfaces.https.html?exclude=(Document|Window|HTML-expected.txt' with ? and a (, instead of the 'external/wpt/html/dom/interfaces.https_exclude=(Document_Window_HTML._)-expected.txt' which is already there.

I haven't found a way that works well.
 
rebaseline-cl should work. Can you try that?

As for --reset-results, there might be a bug in the code and/or command line escaping problem in the invocation.
I'll use rebaseline-cl, but didn't want to wait for the bots when I knew which tests I wanted to update and could do so in a matter of minutes. Ended up taking more :)
Components: Blink>Infra
Status: Available (was: Untriaged)
I agree rebaseline-cl is sometimes an overkill for small changes that developers know for sure would behave the same across platforms.

I suppose some of the issues I've fixed in rebaseline-cl also exist in reset-results in some shape or form. I'll need to find some time to look into it. P3 for now.
Owner: nedngu...@google.com
Status: Assigned (was: Available)
This seems like a simple bug, so I gonna take it to learn more about the framework
Owner: ----
No longer work on this

Sign in to add a comment