Glitchy image artifacts on HTML5 canvas game
Reported by
b...@subeta.net,
Jul 22 2017
|
||||||||||
Issue descriptionUserAgent: Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36 OPR/46.0.2597.46 Example URL: https://subeta.net/games/subeku/subeku.html Steps to reproduce the problem: 1. Go to game page. 2. Click one of the difficulty buttons to start the game, wait for the screen to load. What is the expected behavior? The numbers shown on the board are drawn from an image file - they should be rendered correctly. What went wrong? The numbers are rendered with "glitchy" and discolored artifacts around the outline of each number. See the two screenshots attached. "subeku_working.png" is a screenshot of the game in Firefox, where it works as intended. Does it occur on multiple sites: N/A Is it a problem with a plugin? No Did this work before? N/A Does this work in other browsers? Yes Chrome version: 59.0.3071.115 Channel: stable OS Version: 6.2 (Windows 8) Flash Version: Shockwave Flash 26.0 r0 I am using Opera 46.0.2597.46 (PGO) 64-bit for Windows. The game is written with ImpactJS.
,
Jul 22 2017
I can repro this on Mac, on Chrome 61.0.3163.5. It doesn't repro on 59.0.3071.115. But on Linux I see broken rendering even on 59.0.3071.115.
,
Jul 24 2017
Able to reprodcue the issue using #61.0.3159.5 on Win 10, Mac 10.12.5, Linux Ubuntu 14.04 as well. Below is the bisect info ========================= 61.0.3143.0 - Good Build 61.0.3144.0 - Bad build Unable to provide the tool bisect info as getting error message "doesn't have enough builds to invoke". Hence providing manual CL. CHANGELOG URL ============= https://chromium.googlesource.com/chromium/src/+log/61.0.3143.0..61.0.3144.0?pretty=fuller&n=10000 Unable to find the suspcet from above log. Hence ccing the canvas related dev from the log. @Linux: Able to reproduce the issue using #59.0.3071.115 on Linux Ubuntu 14.04. Observing the similar behavior since M50 and < M50 builds too. Please find the attached screenshot of Linux from M50. Issue is seen in M62 as well. Could someone from Dev team please look into this issue. Thanks!!
,
Jul 24 2017
,
Jul 25 2017
Downloaded Chrome canary (original report was on Opera) and was able to reproduce. Output of chrome://version says: Google Chrome 62.0.3165.0 (Official Build) canary (64-bit) (cohort: 64-Bit) Revision ebd65246aab3b4b1cb5bd462388344195420078d-refs/heads/master@{#488876} OS Windows JavaScript V8 6.2.3 Flash 26.0.0.143 C:\Users\cheryl\AppData\Local\Google\Chrome Screenshot attached.
,
Jul 25 2017
Using archived chromium win64 builds, I was able to reduce the regression range to this: https://chromium.googlesource.com/chromium/src/+log/8b4643f67dc1c6b142a152aa2b76c771287e3a22..06083fbf5d49b0dce17d6b86cb82b79dd3305005 Suspecting: https://chromium.googlesource.com/chromium/src/+/1894d423068be735560e0171922c833c4091b5d1 I suspect color correct rendering and canvas alpha-blending are not playing nice.
,
Jul 25 2017
Confirmed! If you manually disable color-correct rendering on the chrome://flags page, the problem goes away.
,
Jul 25 2017
+brianosman, +mtklein Still digging in to this, but I'm going to guess that this is related to our problems with color conversion of images with premultiplied alpha. I'm still investigating the first instance of this in issue 738517 .
,
Jul 25 2017
I was thinking the same thing. The artifacts really look like arithmetic overflows in the compositing math, probably caused by premul color values greater than alpha.
,
Jul 25 2017
,
Jul 25 2017
but@subeta, can you attach the numbers.png file that is being used by this game? I made SkImage::makeColorSpace do nothing, and that had no effect -- the images still look wrong. So I think that part of the pipeline is off of the hook. The very strange thing is that, when I remove all color conversions everywhere, the problem persists. It appears that the "do color conversion at decode time, while the image is still in un-premultiplied format" seems to be what's allowing this image to be correctly loaded. In particular, if I force a conversion at https://cs.chromium.org/chromium/src/third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp?rcl=f470dd6dfd2e88e40e344e5bd914c90c999ddbe4&l=371 then things work. That code all comes from Matt's r426858. I really want the numbers.png file, so I can see if I can reproduce this behavior outside of the <canvas> element. Perhaps it's a pathological image or an image with a pathological space?
,
Jul 25 2017
Looks like http://img.subeta.net/games/subeku/numbers-static.png (and http://img.subeta.net/games/subeku/numbers.png) are the images you want?
,
Jul 25 2017
At first glance these look like pretty vanilla sRGB to me:
~/skia (clean|✔) $ ninja -C out colorspaceinfo; and out/colorspaceinfo --input ~/Downloads/numbers-static.png
ninja: Entering directory `out'
[1/1] Regenerating ninja files
[246/246] link colorspaceinfo
SkCodec would naturally decode as colorType=RGBA_8888
Color Profile Description: "sRGB"
XYZ/TRC color space
/Users/mtklein/Downloads/numbers-static.png
R G B
X 0.436 0.385 0.143
Y 0.223 0.717 0.061
Z 0.014 0.097 0.714
Area of Gamut: 0.083
Transfer Function: sRGB
~/skia (clean|…) $ ninja -C out colorspaceinfo; and out/colorspaceinfo --input ~/Downloads/numbers.webp
ninja: Entering directory `out'
ninja: no work to do.
SkCodec would naturally decode as colorType=RGBA_8888
Color Profile Description: "sRGB"
XYZ/TRC color space
/Users/mtklein/Downloads/numbers.webp
R G B
X 0.436 0.385 0.143
Y 0.223 0.717 0.061
Z 0.014 0.097 0.714
Area of Gamut: 0.083
Transfer Function: sRGB
,
Jul 25 2017
Even just opening http://img.subeta.net/games/subeku/numbers-static.png directly in Chrome Canary (62.0.3166.0) shows something _very_ broken compared to Stable (59.0.3071.115) on my desktop, especially at non-100% zoom levels. I think that rules out all sorts of things and probably puts the focus on changes to the PNG decoder? I am also experiencing an odd behavior where Chrome seems to be saving http://img.subeta.net/games/subeku/numbers.png as a .webp. Beware. curl -O http://img.subeta.net/games/subeku/numbers.png seems to get the original .png.
,
Jul 25 2017
(The color-correct rendering flag does not have any effect on this in Canary for me. It's always broken, whether set to Default, Enabled, or Disabled.)
,
Jul 25 2017
I've found this patch that forces us to go down the color transform path, and it works around the issue. So something in skipping this path is causing the problem.
Re #14, if color correct rendering is disabled, then we wouldn't do this transform if your monitor and the image had the same color space. So, depending on the profile that is being used, this could happen before using color correct rendering.
Now trying to isolate down to the image that causes the bug, and peek at its pixels.
diff --git a/third_party/WebKit/Source/platform/image-decoders/webp/WEBPImageDecoder.cpp b/third_party/WebKit/Source/platform/image-decoders/webp/WEBPImageDecoder.cpp
index 1d4fad0d7b04..38b972bf637a 100644
--- a/third_party/WebKit/Source/platform/image-decoders/webp/WEBPImageDecoder.cpp
+++ b/third_party/WebKit/Source/platform/image-decoders/webp/WEBPImageDecoder.cpp
@@ -318,6 +318,16 @@ void WEBPImageDecoder::ApplyPostProcessing(size_t frame_index) {
// and linear blending. Can we find a way to perform the
// premultiplication and blending in a linear space?
SkColorSpaceXform* xform = ColorTransform();
+
+ bool work_around_the_bug = true;
+ if (work_around_the_bug) {
+ std::unique_ptr<SkColorSpaceXform> xform1 =
+ SkColorSpaceXform::New(
+ ColorSpaceForSkImages().get(),
+ SkColorSpace::MakeSRGB().get());
+ xform = xfrom1.get();
+ }
+
if (xform) {
const SkColorSpaceXform::ColorFormat kSrcFormat =
SkColorSpaceXform::kBGRA_8888_ColorFormat;
,
Jul 25 2017
I believe my monitor is explicitly set to sRGB, exactly matching those PNGs. Does that makes sense? See attached screenshot.
,
Jul 25 2017
Re #17 -- yes, that makes sense.
Actually, the color transform is a red herring -- the following diff which just reads-then-writes the values back to the image seems to be "fixing" it. The read-then-write does an unpremultiply at read and premultiply at write ... so I suspect that when I dig into the raw pixels coming out of the decoder, they will have invalid premultiplied values.
(mops sweat off forehead that this was related to color correct rendering feature)
diff --git a/third_party/WebKit/Source/platform/image-decoders/webp/WEBPImageDecoder.cpp b/third_party/WebKit/Source/platform/image-decoders/webp/WEBPImageDecoder.cpp
index 1d4fad0d7b04..5336d0f8bb1f 100644
--- a/third_party/WebKit/Source/platform/image-decoders/webp/WEBPImageDecoder.cpp
+++ b/third_party/WebKit/Source/platform/image-decoders/webp/WEBPImageDecoder.cpp
@@ -336,6 +336,17 @@ void WEBPImageDecoder::ApplyPostProcessing(size_t frame_index) {
pixel[3]);
}
}
+ } else {
+ for (int y = decoded_height_; y < decoded_height; ++y) {
+ const int canvas_y = top + y;
+ uint8_t* row = reinterpret_cast<uint8_t*>(buffer.GetAddr(left, canvas_y));
+ uint8_t* pixel = row;
+ for (int x = 0; x < width; ++x, pixel += 4) {
+ const int canvas_x = left + x;
+ buffer.SetRGBA(canvas_x, canvas_y, pixel[2], pixel[1], pixel[0],
+ pixel[3]);
+ }
+ }
}
// During the decoding of the current frame, we may have set some pixels to be
,
Jul 25 2017
The underlying bug is that the image for the WebP image for the image does not have the ALPHA_FLAG in its WEBP_FF_FORMAT_FLAGS, but it's definitely supposed to be treated as transparent. I also find it all very odd that we're loading a Png as a WebP ... is there something I don't know about there? Anyway, we think that the numbers image is opaque, and we tell it to get decoded as though it were unpremultiplied, even if the output format is expected to be premultiplied (since the two coincide for opaque content). We used to happen to do the premultiply at color transform time, which made this bug often not appear. The easy fix is to just specify that we're going to decode as premul based on if the image is supposed to be premul. But it's also concerning that this image is lying about whether or not it has alpha data.
,
Jul 25 2017
Yeah, I feel like you might argue we could scan the image to see if an image were overly conservative, saying it had alpha when they were all 0xff, but not mentioning alpha when the image actually does have transparent pixels seems like a big omission on the image's part. I too remain very puzzled about whether and which of these images are .pngs or .webps. ~ $ curl -O http://img.subeta.net/games/subeku/numbers-static.png ~ $ curl -O http://img.subeta.net/games/subeku/numbers.png ~ $ file *.png numbers-static.png: PNG image data, 500 x 50, 8-bit/color RGBA, non-interlaced numbers.png: PNG image data, 700 x 1950, 8-bit/color RGBA, non-interlaced But, when I open those URLs in Chrome and hit ⌘S, the dialog box prompts me to save both as .webp, and they are really .webp, and smaller for what it's worth. ~ $ file *.webp numbers-static.webp: RIFF (little-endian) data, Web/P image numbers.webp: RIFF (little-endian) data, Web/P image ~ $ du -sh *.png *.webp 32K numbers-static.png 192K numbers.png 28K numbers-static.webp 76K numbers.webp Are we converting images to .webp in Chrome?
,
Jul 25 2017
+scroggo, vmpstr > Are we converting images to .webp in Chrome? That's what I'm wondering. Whoever is creating this .webp seems to be creating a broken version of it. The quick fix would be to do WEBP_CSP_MODE mode = outputMode(premultiply_alpha_); instead of WEBP_CSP_MODE mode = outputMode(format_flags_ & ALPHA_FLAG); but that seems like weakening the spec. Attaching the busted webp
,
Jul 25 2017
(now attaching it for real)
,
Jul 25 2017
Is it possible that http://img.subeta.net/ serves Chrome a .webp when Chrome requests the .png because they know Chrome can handle it?
,
Jul 25 2017
I think that's it. The problematic encoder is probably on the website's end. We just recorded the fetch of numbers.png in Dev Tools. It showed we downloaded 73.9KB, which matches numbers.webp (75048 bytes) and not numbers.png (195850 bytes). In the headers, we're seeing Content-Type: image/webp, and Cf-Polished: origFormat=png.
,
Jul 25 2017
Does anyone think that we should add the fix from #21 (always decode as premultiplied)? Or should we just WontFix?
,
Jul 25 2017
Personally I think it's probably best for the most people in the long term if we encourage well-formed image files on the web (i.e. WontFix), but I would support always premultiplying if that's your inclination. If we do make this change, it might be worth a quick CPU profile to know how much we'd be burning on the well-formed majority of .webps. I bet it's peanuts compared to the rest of the decode, but I have nothing concrete to back that hunch up.
,
Jul 26 2017
Hmmm, it's PNG if I load the image directly curl -k -s http://img.subeta.net/games/subeku/numbers.png | od -c | head -1 0000000 211 P N G \r \n 032 \n \0 \0 \0 \r I H D R but it's WEBP when loaded / saved in Chrome. I suspect an chrome extension or corp proxy is converting the PNG to WEBP, see issue 535061 .
,
Jul 26 2017
I don't think this is a corp proxy or other extension. External users would not be affected by corp proxy, and I don't have any other interesting extensions installed. Here's the headers from the request Chrome makes: Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8 Accept-Encoding:gzip, deflate Accept-Language:en-US,en;q=0.8 Cache-Control:no-cache Connection:keep-alive Cookie:__cfduid=d07f393f69914fb0b9e0e63179eb9e2111501027901 Host:img.subeta.net Pragma:no-cache Upgrade-Insecure-Requests:1 User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3159.5 Safari/537.36 I believe that says, we're cool if you want to give us a .webp instead. And here's the response headers, Cache-Control:public, max-age=31536000 Cf-Bgj:imgq:85 CF-Cache-Status:HIT Cf-Polished:origFmt=png, origSize=235356 CF-RAY:384324fbe0ef569f-IAD Connection:keep-alive Content-Disposition:inline; filename="numbers.webp" Content-Length:75048 Content-Type:image/webp Date:Wed, 26 Jul 2017 00:11:56 GMT Etag:"138842f0d73261c6df9d212b6de1d30f" Expires:Thu, 26 Jul 2018 00:11:56 GMT Last-Modified:Sat, 11 Jan 2014 09:27:48 GMT Server:cloudflare-nginx Vary:Accept X-Amz-Id-2:fmaXs8ePZpwPKgPI9jOYROeWgcnwyoDnHnqrcinBzJc+dRCBryafH9dnN+mo6htA0yemD31MlI8= X-Amz-Request-Id:ACB97EA9F95CA83B X-Amz-Version-Id:null I get qualitatively the same results with corp proxy on and off, in incognito, and on my civilian Windows computer at home. I suspect we've got to loop in Cloudflare to fix this.
,
Jul 26 2017
Thank you (this did sound suspiciously like issue 535061 to me). Request headers say we'll accept WEBP, as you noted. Agree, need to loop in Cloudflare here.
,
Jul 26 2017
Hey everyone, thank you for the investigation. I can confirm our server does not host WEBP files, nor do we do any such conversion/encoding ourselves. So it makes sense to me that the WEBP conversion is happening when it gets to Cloudflare.
,
Jul 27 2017
I'll be following up with the engineering team internally to see what's up with how were encoding WebP.
,
Jul 30 2017
IIRC, the alpha bit for all images with the opposite of what it was supposed to be.
,
Aug 8 2017
FYI, we're going to be pushing 61 (with this change) to Stable soon (couple weeks).
,
Aug 10 2017
Closing as WontFix. I was on the fence WRT whether or not to work around this in Chrome, but I think that is the wrong thing to do. |
||||||||||
►
Sign in to add a comment |
||||||||||
Comment 1 by b...@subeta.net
, Jul 22 2017