New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 747640 link

Starred by 4 users

Issue metadata

Status: WontFix
Owner:
Closed: Aug 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Linux , Windows , Mac
Pri: 1
Type: Bug-Regression

Blocking:
issue 713889



Sign in to add a comment

Glitchy image artifacts on HTML5 canvas game

Reported by b...@subeta.net, Jul 22 2017

Issue description

UserAgent: Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36 OPR/46.0.2597.46

Example URL:
https://subeta.net/games/subeku/subeku.html

Steps to reproduce the problem:
1. Go to game page.
2. Click one of the difficulty buttons to start the game, wait for the screen to load.

What is the expected behavior?
The numbers shown on the board are drawn from an image file - they should be rendered correctly.

What went wrong?
The numbers are rendered with "glitchy" and discolored artifacts around the outline of each number. See the two screenshots attached. "subeku_working.png" is a screenshot of the game in Firefox, where it works as intended.

Does it occur on multiple sites: N/A

Is it a problem with a plugin? No 

Did this work before? N/A 

Does this work in other browsers? Yes

Chrome version: 59.0.3071.115  Channel: stable
OS Version: 6.2 (Windows 8)
Flash Version: Shockwave Flash 26.0 r0

I am using Opera 46.0.2597.46 (PGO) 64-bit for Windows. 

The game is written with ImpactJS.
 
subeku_working.png
543 KB View Download
subeku_glitch.PNG
446 KB View Download

Comment 1 by b...@subeta.net, Jul 22 2017

Oops, missed a question. Yes, this worked before. I am not sure which version it last worked on, but we began receiving reports of this bug about a week ago.

Comment 2 by ajuma@chromium.org, Jul 22 2017

Components: Blink>Canvas
Labels: Needs-Bisect OS-Linux OS-Mac
Status: Untriaged (was: Unconfirmed)
I can repro this on Mac, on Chrome 61.0.3163.5. It doesn't repro on 59.0.3071.115.

But on Linux I see broken rendering even on 59.0.3071.115.
Cc: leon....@intel.com sandeepk...@techmahindra.com
Labels: -Pri-2 -Type-Compat -Needs-Bisect M-62 hasbisect Pri-1 Type-Bug-Regression
Able to reprodcue the issue using #61.0.3159.5 on Win 10, Mac 10.12.5, Linux Ubuntu 14.04 as well.

Below is the bisect info
=========================

61.0.3143.0 -  Good Build
61.0.3144.0 -  Bad build 

Unable to provide the tool bisect info as getting error message "doesn't have enough builds to invoke". Hence providing manual CL.

CHANGELOG URL
=============

https://chromium.googlesource.com/chromium/src/+log/61.0.3143.0..61.0.3144.0?pretty=fuller&n=10000

Unable to find the suspcet from above log.  Hence ccing the canvas related dev from the log.

@Linux: Able to reproduce the issue using #59.0.3071.115 on Linux Ubuntu 14.04.

Observing the similar behavior since M50 and < M50 builds too. Please find the attached screenshot of Linux from M50.

Issue is seen in M62 as well.

Could someone from Dev team please look into this issue.

Thanks!!
Screenshot from 2017-07-24 12_22_56.png
460 KB View Download
Cc: -sandeepk...@techmahindra.com sandeepkumars@chromium.org

Comment 5 by b...@subeta.net, Jul 25 2017

Downloaded Chrome canary (original report was on Opera) and was able to reproduce.

Output of chrome://version says:

Google Chrome	62.0.3165.0 (Official Build) canary (64-bit) (cohort: 64-Bit)
Revision	ebd65246aab3b4b1cb5bd462388344195420078d-refs/heads/master@{#488876}
OS	Windows
JavaScript	V8 6.2.3
Flash	26.0.0.143 C:\Users\cheryl\AppData\Local\Google\Chrome 

Screenshot attached.
subeku_chrome.PNG
677 KB View Download

Comment 6 by junov@chromium.org, Jul 25 2017

Cc: zakerinasab@chromium.org
Owner: ccameron@chromium.org
Using archived chromium win64 builds, I was able to reduce the regression range to this:

https://chromium.googlesource.com/chromium/src/+log/8b4643f67dc1c6b142a152aa2b76c771287e3a22..06083fbf5d49b0dce17d6b86cb82b79dd3305005

Suspecting:
https://chromium.googlesource.com/chromium/src/+/1894d423068be735560e0171922c833c4091b5d1

I suspect color correct rendering and canvas alpha-blending are not playing nice.


Comment 7 by junov@chromium.org, Jul 25 2017

Confirmed! If you manually disable color-correct rendering on the chrome://flags page, the problem goes away.
Cc: brianosman@chromium.org mtklein@chromium.org
+brianosman, +mtklein

Still digging in to this, but I'm going to guess that this is related to our problems with color conversion of images with premultiplied alpha.

I'm still investigating the first instance of this in  issue 738517 .

Comment 9 by junov@chromium.org, Jul 25 2017

I was thinking the same thing.  The artifacts really look like arithmetic overflows in the compositing math, probably caused by premul color values greater than alpha.


Comment 10 by junov@chromium.org, Jul 25 2017

Blocking: 713889
but@subeta, can you attach the numbers.png file that is being used by this game?

I made SkImage::makeColorSpace do nothing, and that had no effect -- the images still look wrong. So I think that part of the pipeline is off of the hook.

The very strange thing is that, when I remove all color conversions everywhere, the problem persists.

It appears that the "do color conversion at decode time, while the image is still in un-premultiplied format" seems to be what's allowing this image to be correctly loaded. In particular, if I force a conversion at
https://cs.chromium.org/chromium/src/third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp?rcl=f470dd6dfd2e88e40e344e5bd914c90c999ddbe4&l=371
then things work.

That code all comes from Matt's r426858.

I really want the numbers.png file, so I can see if I can reproduce this behavior outside of the <canvas> element. Perhaps it's a pathological image or an image with a pathological space?
At first glance these look like pretty vanilla sRGB to me:

~/skia (clean|✔) $ ninja -C out colorspaceinfo; and out/colorspaceinfo --input ~/Downloads/numbers-static.png
ninja: Entering directory `out'
[1/1] Regenerating ninja files
[246/246] link colorspaceinfo
SkCodec would naturally decode as colorType=RGBA_8888
Color Profile Description: "sRGB"
XYZ/TRC color space
/Users/mtklein/Downloads/numbers-static.png
       R     G     B
X  0.436 0.385 0.143
Y  0.223 0.717 0.061
Z  0.014 0.097 0.714
Area of Gamut: 0.083

Transfer Function: sRGB

~/skia (clean|…) $ ninja -C out colorspaceinfo; and out/colorspaceinfo --input ~/Downloads/numbers.webp
ninja: Entering directory `out'
ninja: no work to do.
SkCodec would naturally decode as colorType=RGBA_8888
Color Profile Description: "sRGB"
XYZ/TRC color space
/Users/mtklein/Downloads/numbers.webp
       R     G     B
X  0.436 0.385 0.143
Y  0.223 0.717 0.061
Z  0.014 0.097 0.714
Area of Gamut: 0.083

Transfer Function: sRGB

Even just opening http://img.subeta.net/games/subeku/numbers-static.png directly in Chrome Canary (62.0.3166.0) shows something _very_ broken compared to Stable (59.0.3071.115) on my desktop, especially at non-100% zoom levels.

I think that rules out all sorts of things and probably puts the focus on changes to the PNG decoder?

I am also experiencing an odd behavior where Chrome seems to be saving http://img.subeta.net/games/subeku/numbers.png as a .webp.  Beware.  curl -O http://img.subeta.net/games/subeku/numbers.png seems to get the original .png.
(The color-correct rendering flag does not have any effect on this in Canary for me.  It's always broken, whether set to Default, Enabled, or Disabled.)
I've found this patch that forces us to go down the color transform path, and it works around the issue. So something in skipping this path is causing the problem.

Re #14, if color correct rendering is disabled, then we wouldn't do this transform if your monitor and the image had the same color space. So, depending on the profile that is being used, this could happen before using color correct rendering.

Now trying to isolate down to the image that causes the bug, and peek at its pixels.


diff --git a/third_party/WebKit/Source/platform/image-decoders/webp/WEBPImageDecoder.cpp b/third_party/WebKit/Source/platform/image-decoders/webp/WEBPImageDecoder.cpp
index 1d4fad0d7b04..38b972bf637a 100644
--- a/third_party/WebKit/Source/platform/image-decoders/webp/WEBPImageDecoder.cpp
+++ b/third_party/WebKit/Source/platform/image-decoders/webp/WEBPImageDecoder.cpp
@@ -318,6 +318,16 @@ void WEBPImageDecoder::ApplyPostProcessing(size_t frame_index) {
   // and linear blending.  Can we find a way to perform the
   // premultiplication and blending in a linear space?
   SkColorSpaceXform* xform = ColorTransform();
+
+  bool work_around_the_bug = true;
+  if (work_around_the_bug) {
+    std::unique_ptr<SkColorSpaceXform> xform1 = 
+        SkColorSpaceXform::New(
+            ColorSpaceForSkImages().get(),
+            SkColorSpace::MakeSRGB().get());
+    xform = xfrom1.get();
+  }
+
   if (xform) {
     const SkColorSpaceXform::ColorFormat kSrcFormat =
         SkColorSpaceXform::kBGRA_8888_ColorFormat;
I believe my monitor is explicitly set to sRGB, exactly matching those PNGs.  Does that makes sense?  See attached screenshot.
Screen Shot 2017-07-25 at 4.50.21 PM.png
85.4 KB View Download
Re #17 -- yes, that makes sense.

Actually, the color transform is a red herring -- the following diff which just reads-then-writes the values back to the image seems to be "fixing" it. The read-then-write does an unpremultiply at read and premultiply at write ... so I suspect that when I dig into the raw pixels coming out of the decoder, they will have invalid premultiplied values.

(mops sweat off forehead that this was related to color correct rendering feature)

diff --git a/third_party/WebKit/Source/platform/image-decoders/webp/WEBPImageDecoder.cpp b/third_party/WebKit/Source/platform/image-decoders/webp/WEBPImageDecoder.cpp
index 1d4fad0d7b04..5336d0f8bb1f 100644
--- a/third_party/WebKit/Source/platform/image-decoders/webp/WEBPImageDecoder.cpp
+++ b/third_party/WebKit/Source/platform/image-decoders/webp/WEBPImageDecoder.cpp
@@ -336,6 +336,17 @@ void WEBPImageDecoder::ApplyPostProcessing(size_t frame_index) {
                        pixel[3]);
       }
     }
+  } else {
+    for (int y = decoded_height_; y < decoded_height; ++y) {
+      const int canvas_y = top + y;
+      uint8_t* row = reinterpret_cast<uint8_t*>(buffer.GetAddr(left, canvas_y));
+      uint8_t* pixel = row;
+      for (int x = 0; x < width; ++x, pixel += 4) {
+        const int canvas_x = left + x;
+        buffer.SetRGBA(canvas_x, canvas_y, pixel[2], pixel[1], pixel[0],
+                       pixel[3]);
+      }
+    }
   }
 
   // During the decoding of the current frame, we may have set some pixels to be
The underlying bug is that the image for the WebP image for the image does not have the ALPHA_FLAG in its WEBP_FF_FORMAT_FLAGS, but it's definitely supposed to be treated as transparent.

I also find it all very odd that we're loading a Png as a WebP ... is there something I don't know about there?

Anyway, we think that the numbers image is opaque, and we tell it to get decoded as though it were unpremultiplied, even if the output format is expected to be premultiplied (since the two coincide for opaque content).

We used to happen to do the premultiply at color transform time, which made this bug often not appear.

The easy fix is to just specify that we're going to decode as premul based on if the image is supposed to be premul.

But it's also concerning that this image is lying about whether or not it has alpha data.
Yeah, I feel like you might argue we could scan the image to see if an image were overly conservative, saying it had alpha when they were all 0xff, but not mentioning alpha when the image actually does have transparent pixels seems like a big omission on the image's part.

I too remain very puzzled about whether and which of these images are .pngs or .webps.

  ~ $ curl -O http://img.subeta.net/games/subeku/numbers-static.png
  ~ $ curl -O http://img.subeta.net/games/subeku/numbers.png
  ~ $ file *.png
    numbers-static.png: PNG image data, 500 x 50, 8-bit/color RGBA, non-interlaced
    numbers.png:        PNG image data, 700 x 1950, 8-bit/color RGBA, non-interlaced

But, when I open those URLs in Chrome and hit ⌘S, the dialog box prompts me to save both as .webp, and they are really .webp, and smaller for what it's worth.
  ~ $ file *.webp
    numbers-static.webp: RIFF (little-endian) data, Web/P image
    numbers.webp:        RIFF (little-endian) data, Web/P image
  ~ $ du -sh *.png *.webp
     32K	numbers-static.png
    192K	numbers.png
     28K	numbers-static.webp
     76K	numbers.webp

Are we converting images to .webp in Chrome?
Cc: vmp...@chromium.org scroggo@chromium.org
+scroggo, vmpstr

> Are we converting images to .webp in Chrome?

That's what I'm wondering.

Whoever is creating this .webp seems to be creating a broken version of it. The quick fix would be to do
  WEBP_CSP_MODE mode = outputMode(premultiply_alpha_);
instead of
  WEBP_CSP_MODE mode = outputMode(format_flags_ & ALPHA_FLAG);
but that seems like weakening the spec.

Attaching the busted webp
(now attaching it for real)
numbers-static.webp
27.5 KB View Download
Is it possible that http://img.subeta.net/ serves Chrome a .webp when Chrome requests the .png because they know Chrome can handle it?
I think that's it.  The problematic encoder is probably on the website's end.

We just recorded the fetch of numbers.png in Dev Tools.  It showed we downloaded 73.9KB, which matches numbers.webp (75048 bytes) and not numbers.png (195850 bytes).

In the headers, we're seeing Content-Type: image/webp, and Cf-Polished: origFormat=png.
Status: Assigned (was: Untriaged)
Does anyone think that we should add the fix from #21 (always decode as premultiplied)?

Or should we just WontFix?
Personally I think it's probably best for the most people in the long term if we encourage well-formed image files on the web (i.e. WontFix), but I would support always premultiplying if that's your inclination.

If we do make this change, it might be worth a quick CPU profile to know how much we'd be burning on the well-formed majority of .webps.  I bet it's peanuts compared to the rest of the decode, but I have nothing concrete to back that hunch up.

Comment 27 by noel@chromium.org, Jul 26 2017

Hmmm, it's PNG if I load the image directly

curl -k -s http://img.subeta.net/games/subeku/numbers.png | od -c | head -1
0000000  211   P   N   G  \r  \n 032  \n  \0  \0  \0  \r   I   H   D   R

but it's WEBP when loaded / saved in Chrome.  I suspect an chrome extension or corp proxy is converting the PNG to WEBP, see  issue 535061 .

I don't think this is a corp proxy or other extension.  External users would not be affected by corp proxy, and I don't have any other interesting extensions installed.

Here's the headers from the request Chrome makes:

Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Encoding:gzip, deflate
Accept-Language:en-US,en;q=0.8
Cache-Control:no-cache
Connection:keep-alive
Cookie:__cfduid=d07f393f69914fb0b9e0e63179eb9e2111501027901
Host:img.subeta.net
Pragma:no-cache
Upgrade-Insecure-Requests:1
User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3159.5 Safari/537.36

I believe that says, we're cool if you want to give us a .webp instead.

And here's the response headers,

Cache-Control:public, max-age=31536000
Cf-Bgj:imgq:85
CF-Cache-Status:HIT
Cf-Polished:origFmt=png, origSize=235356
CF-RAY:384324fbe0ef569f-IAD
Connection:keep-alive
Content-Disposition:inline; filename="numbers.webp"
Content-Length:75048
Content-Type:image/webp
Date:Wed, 26 Jul 2017 00:11:56 GMT
Etag:"138842f0d73261c6df9d212b6de1d30f"
Expires:Thu, 26 Jul 2018 00:11:56 GMT
Last-Modified:Sat, 11 Jan 2014 09:27:48 GMT
Server:cloudflare-nginx
Vary:Accept
X-Amz-Id-2:fmaXs8ePZpwPKgPI9jOYROeWgcnwyoDnHnqrcinBzJc+dRCBryafH9dnN+mo6htA0yemD31MlI8=
X-Amz-Request-Id:ACB97EA9F95CA83B
X-Amz-Version-Id:null

I get qualitatively the same results with corp proxy on and off, in incognito, and on my civilian Windows computer at home.

I suspect we've got to loop in Cloudflare to fix this.

Comment 29 by noel@chromium.org, Jul 26 2017

Thank you (this did sound suspiciously like  issue 535061  to me).  Request headers say we'll accept WEBP, as you noted.  Agree, need to loop in Cloudflare here.

Comment 30 by b...@subeta.net, Jul 26 2017

Hey everyone, thank you for the investigation. I can confirm our server does not host WEBP files, nor do we do any such conversion/encoding ourselves. So it makes sense to me that the WEBP conversion is happening when it gets to Cloudflare.

Comment 31 Deleted

I'll be following up with the engineering team internally to see what's up with how were encoding WebP.
IIRC, the alpha bit for all images with the opposite of what it was supposed to be.
FYI, we're going to be pushing 61 (with this change) to Stable soon (couple weeks).

Status: WontFix (was: Assigned)
Closing as WontFix.

I was on the fence WRT whether or not to work around this in Chrome, but I think that is the wrong thing to do.

Sign in to add a comment