Allow off-by-one-or-two errors for some pixel layout tests |
||||
Issue descriptionWe're doing lots of churn of the pixel layout test expectations because of off-by-one errors. Any time that we change the math anywhere in the graphics pipeline ever-so-slightly, we end up with bazillion diffs like the attached "linux.png" and "win.png". The diff (if you can see it) is in "diff.png", and is entirely <1 pixel value. We also end up having lots of different expectations for different platforms because of this (the attached example is from virtual/exotic-color-space/images/color-profile-border-radius.html). This is easy enough to add as an argument to the image_diff utility, but having a mechanism wherein tests can specify to use this argument is a bit more complicated.
,
Jun 9 2017
It should be straightforward enough to have the test_runner set a bit that propagates into when we dump the image out with the mime-type headers to stdout, and then read that out of stdout from inside webkitpy and pass it to image_diff.
,
Jun 9 2017
That sounds reasonable!
,
May 8 2018
,
Jun 6 2018
Marking available.
,
Jan 11
Available, but no owner or component? Please find a component, as no one will ever find this without one. |
||||
►
Sign in to add a comment |
||||
Comment 1 by qyears...@chromium.org
, Jun 9 2017I guess from the test writers' perspective, maybe the most obvious way specify it would be by calling a method on the testRunner global in the test, e.g. something like: <script> ... if (window.testRunner) { testRunner.setAllowPixelDiff(2); testRunner.dumpAsTextWithPixelResults(); testRunner.waitUntilDone(); } </script> And then the question is, is there a nice way to change test_runner and the webkitpy code so that we get the right behavior when Port.diff_image is called? What do you think?