xlai@: please take a look.

Apparently it's due to WebGL: https://bugs.chromium.org/p/chromium/issues/detail?id=86040

The site is working as intended. Google map is webgl canvas; to save an image from webgl canvas, you need to set preserveDrawingBuffer to be "true" when initializing the webgl rendering context. Apparently this cannot be done externally on an existing canvas like Google map. This stackoverflow discussion (http://stackoverflow.com/questions/15558418/how-do-you-save-an-image-from-a-three-js-canvas) explains the situation. The corehtml5canvas example mentioned is a 2d rendering context so this requirement doesn't apply.

But "Save image as..." (from the context menu*) does work on Google Maps for some reason... So I'm thinking it is indeed technically possible to get the image, even from WebGL canvases. So why not do it? Many people would surely need it. I for one am working on a Chrome extension that would require this functionality.

*You have to enable the context menu on a site like Google Maps. Either with a browser extension or by running some JavaScript on the console.

pinging kbr@ who understands this area more: After the drawing buffer passes its contents to the compositor do we still retain a read-only shared reference to it? I am assuming that is what DrawingBuffers's "FrontBuffer" is?

xlai@: Have you tried changing the implementation to make it capture the FrontBuffer instead of the BackBuffer?

junov@: That works. I'll upload the patch.

The compositor does indeed hold on to a reference to the previously-rendered front buffer for WebGL-rendered canvas, even if preserveDrawingBuffer=false. However, this is not exposed to pure JavaScript; only to the "Save Image As..." functionality. All JavaScript-level operations for capturing WebGL-rendered canvases' contents must obey the preserveDrawingBuffer rules.

There should already be tests for "Save image as...". If not, let's write more.

@kbr: So are you saying that making toBlob/toDataURL capture the front buffer would *not* be okay?

If those functions capture the BackBuffer, that means there is only a tiny window for capturing the canvas contents: after finishing drawing the frame, and before the buffer swap (compositor commit).  This makes it really hard for a Chrome extension to capture content.

Looking at the specification for the event loop processing model (https://html.spec.whatwg.org/multipage/webappapis.html#event-loop), I don't think there is a hook that would guarantee the right timing. the last step before the graphics update is to run the animation frame callbacks.  If an extension were to wrap its call to toBlob inside a requestAnimationFrame callback, that would guarantee the right timing only when it is known that the WebGL canvas content has a pending update. And if the page itself uses rAF for its updates (very likely), then we'd have a race between the to rAFs. :-(

The webGL spec is quite clear: "Before the drawing buffer is presented for compositing the implementation shall ensure that all rendering operations have been flushed to the drawing buffer. By default, after compositing the contents of the drawing buffer shall be cleared to their default values, as shown in the table above. This default behavior can be changed by setting the preserveDrawingBuffer attribute of the WebGLContextAttributes object. If this flag is true, the contents of the drawing buffer shall be preserved until the author either clears or overwrites them. If this flag is false, attempting to perform operations using this context as a source image after the rendering function has returned can lead to undefined behavior. This includes readPixels or toDataURL calls, using this context as the source image of another context's texImage2D or drawImage call, or creating an ImageBitmap [HTML] from this context's canvas."

Fixing this would require a feature request against the specification.

Correct. The WebGL spec is very clear in this area and does not allow changing the behavior of Canvas.toBlob() when preserveDrawingBuffer=false.

I'm not sure what the extension intends to do, but one thing it could do is intercept calls to HTMLCanvasElement.getContext() and ensure that for all calls for the 'webgl' context, make sure the context creation attributes contain { preserveDrawingBuffer: true }.

I think this bug should be marked WontFix after more thought.

I think that could work. So I would need to inject a content script that overwrites HTMLCanvasElement.prototype.getContext with a custom function that ensures preserveDrawingBuffer is always set to true. Of course, the content script needs to escape the isolated world, otherwise it won't work. In case someone reading this finds it helpful, the techniques for doing that are listed here: http://stackoverflow.com/questions/9515704/building-a-chrome-extension-inject-code-in-a-page-using-a-content-script

My extension is for saving "background images". They can come from many different sources, like the background-image CSS property, or an <img> element that's behind other elements, or a <canvas>, to name a few examples.

To get the desired image, the user either 
1) right-clicks on the image and clicks the extension's context menu entry, or 
2) clicks the extension's browser action button and then clicks the image

So the extension will be invoked relatively rarely.

What worries me is performance. The spec says that setting preserveDrawingBuffer to true "can cause significant performance loss on some platforms" and that "whenever possible this flag should remain false and other techniques used.".

Sounds pretty severe. It would seem wasteful to cause large performance issues when the user just wants to invoke the extension every now and then. "Can cause" and "on some platforms" gives me some hope.

There's a StackOverFlow answer from 3½ years ago that says this:
"You should be fine in most desktop browsers, where the copy doesn't actually have to be made, and those do make up the vast majority of WebGL capable browsers...but only for now."

But I don't know how accurate it is. Can you guys comment on this?

I wouldn't worry about performance of an interactive extension. The performance warning is intended for content that intends to run at 60 FPS on mobile devices.

Just so I'm being clear: I would have to turn on preserveDrawingBuffer on ALL canvases on ALL sites, since it can't be turned on afterwards, and since the extension can't possibly know when and where the user will want to extract images from a canvas.

I understood from the spec that when preserveDrawingBuffer=false, then toDataURL() and toBlob() "can lead to undefined behavior". 
Not that they're not allowed to return the correct image. Currently that "undefined behavior" in Chrome is to return a blank image. 

So on platforms where it doesn't incur a performance penalty, toBlob() could return the correct image. And on other platforms, it could just return a blank image.

And when preserveDrawingBuffer=true, then toBlob() MUST return the correct image, no exceptions.

The "undefined behavior" is whether the browser's compositor has run, and consumed the most recently produced frame from the WebGL-rendered canvas, since JavaScript returned control to the browser's main loop. There is a race condition here. toBlob(), etc. will return either the most recently rendered frame, or the well-defined blank frame.

If preserveDrawingBuffer=true, then toBlob(), etc. are defined as returning the most recently rendered frame.

I'm closing this as WontFix. There is no bug on the browser's side.

Okay.

I went with the suggested approach of overwriting HTMLCanvasElement.prototype.getContext. It does work. I've attached the relevant code in case anyone reading this finds it useful.

Deleted: manifest.json 271 bytes

manifest.json 271 bytes View Download

Deleted: content_script.js 241 bytes

content_script.js 241 bytes View Download

Deleted: overwrite.js 439 bytes

overwrite.js 439 bytes View Download