The WebGL working group discussed this and the agreed-upon path among all of the WebGL implementers is to try using sandboxed iframes for this purpose. iframes provide a clear boundary between the embedder and the embedded content. They provide a container element with a fixed size, so layout can be persistent across updates of the WebGL texture. They also provide the opportunity to do things like "incognito mode". In Chrome, with the out-of-process iframe effort, these iframes can also be hosted in completely separate processes.

The basic idea, then, is:

- Add a new sandbox attribute to sandboxed iframes which:
-- Forces it to implicitly be incognito mode.
-- Disables fetches of resources from anything but the containing document's origin.

These are severe restrictions, but unfortunately are needed for the WebGL use case. Any data that can be uploaded to a WebGL texture can be deduced via side-channel timing attacks. For this reason, all of the data that is loaded in the iframe must be same-origin with respect to the containing document.

At that point the iframe can be sized appropriately and its contents uploaded repeatedly to WebGL.

It's also necessary to add APIs to dispatch mouse and keyboard events into these sandboxed iframes, because if they aren't interactive their utility is much lower.

From my perspective, incognito, same-origin iframes that can render to a texture would be a huge win. The main point here is to be able to generate textures from HTML/CSS, which enables many many developers and unlocks a world of content.

And yes, touch/mouse/keyboard event emulation is also important. Any proposals on how to do this? 

How much work is this to achieve, and any estimate for how long this can take?

Event re-dispatching should probably be exposed as a couple of methods that are only legal to call on these specially sandboxed iframes, and only legal to call during event dispatching of a real event of similar type (similar to requiring user interaction to go full-screen, for example).

This is a large chunk of work and I don't have a good estimate for how long it will take. Probably a month to get a first prototype in place. It might be possible to get the first pixels on the screen more quickly.

In in-person discussions the following points were made by esprehn, ojan, bajones and others:

- The out-of-process iframe infrastructure wouldn't add more security to this (in the current state of things) and has other downsides (high memory cost) so shouldn't be used for experimentation.

- It would be useful to expose this as some sort of auto-updating ImageBitmap, rather than just a WebGL texture, so that it could be used identically for 2D canvas and WebGL.

- It'll be necessary to install a SubresourceFilter to enforce that all loading requests from the iframe are same-origin with respect to the containing document, for safety. This is very important, to have acceptable security for this experimental feature even behind a flag.

- The DOM of the iframe should not be accessible.

- A new "sandboxed" iframe attribute should be created describing some of the new constraints. Not sure what the name should be. "sandboxed-for-canvas-rendering"? This should turn on the same-origin restriction, avoid rendering visited link state, and probably other properties.

zmo@ was interested in working on this. Assigning.

I'm not convinced ImageBitmap is the right vehicle for this.  ImageBitmaps are immutable by design, which enables all sorts of shortcuts and optimizations. Perhaps this could be implemented as a special kind of canvas rendering context?

Also, another idea fserb@ is exploring is to set-up a 2D canvas that renders directly into a WebGL texture, or a way to transfer a 2D canvas backing into a WebGL texture.  Either way, the idea is to avoid having to make a texture upload or a texture copy.  This would greatly accelerate WebGL apps that use 2D canvas for text labels, for example.  If I am not mistaken, this is how Google Maps renders street names.

Understood that manipulating ImageBitmap in this way might not be a good idea. We were just trying to make this mechanism work well for both 2D Canvas and WebGL. We can certainly design the transfer mechanism together.

 Issue 682972  has been merged into this issue.