Project: chromium Issues People Development process History Sign in
New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.
Starred by 28 users
Status: Fixed
Owner:
Closed: Apr 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: All
Pri: 2
Type: Launch-OWP
Launch-Accessibility: ----
Launch-Legal: ----
Launch-M-Approved: ----
Launch-M-Target: ----
Launch-Privacy: ----
Launch-Security: ----
Launch-Status: Approved
Launch-Test: ----
Launch-UI: ----
Product-Review: ----


Sign in to add a comment
Implement ImageBitmap, the createImageBitmap function, and ImageBitmap support in Canvas2D and WebGL
Project Member Reported by gman@chromium.org, Jun 13 2013 Back to list
Implement ImageBitmap from the w3 spec.

Change description:
Allow async decoding of images from various sources of data.

Changes to API surface:
New root object. ImageBitmap

Spec: 
http://www.w3.org/html/wg/drafts/html/master/webappapis.html#imagebitmap

Support in other browsers: current (expected)
Internet Explorer: Unknown
Firefox: Unknown
Safari: Unknown

 
Comment 1 by gman@chromium.org, Jun 13 2013
Labels: -Type-Launch -Cr-Blink-WebGL -Launch-DesignDoc-No -Launch-Accessibility-No -Launch-QA-No -Launch-UI-No -Launch-Strings-No -Launch-Security-No -Launch-Privacy-No -Launch-Legal-No -Launch-Marketing-No -Launch-Conops-No -Launch-SRE-No -Launch-AppExt-No OWP-Type-NewAPI OWP-Design-No OWP-Standards-OfficialSpec OWP-Format-Prefix OWP-LaunchBug-Needed OWP-Documentation-Needed Type-Launch-OWP OWP-TargetChannel-Stable Pri-2 OS-All
Labels: Cr-Blink-Canvas Cr-Blink-WebGL
tagging WebGL and Canvas not because its explicitly tied to these APIs but just because it's relevant to them
Blockedon: chromium:166658
Summary: Implement ImageBitmap, the createImageBitmap function, and ImageBitmap support in Canvas2D and WebGL (was: Implement ImageBitmap)
Note this also implies the new createImageBitmap function, and in order for it to be useful we'll also need to implement support in WebGL and Canvas2D.

The two blocking bugs together cover all of these implementations.
Comment 4 by junov@chromium.org, Sep 11 2013
Blockedon: chromium:258082 chromium:260933
FWIW, Image bitmap factories have already been implemented.  Support in 2D canvas has also been implemented but there are still a few tweaks that would be good to have before shipping. I am adding those to the "blocked on" list.
Blockedon: chromium:438986
Cc: senorblanco@chromium.org
Comment 7 by junov@chromium.org, Sep 15 2015
Owner: xidac...@chromium.org
Status: Assigned
Comment 8 by junov@chromium.org, Sep 15 2015
Blockedon: chromium:532142
Blockedon: chromium:538253
Blockedon: chromium:555958
Blocking: chromium:563816
Blockedon: chromium:563984
Blockedon: chromium:569779
Blockedon: chromium:576280
Blockedon: chromium:579539
Project Member Comment 16 by bugdroid1@chromium.org, Jan 25 2016
The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/aa3e16a6a6cdec7d99a1741ebad75618909ccca7

commit aa3e16a6a6cdec7d99a1741ebad75618909ccca7
Author: xidachen <xidachen@chromium.org>
Date: Mon Jan 25 22:52:21 2016

Ship ImageBitmap and createImageBitmap

Now that we have 3 LGTMs for ImageBitmap and createImageBitmap to ship:
https://groups.google.com/a/chromium.org/d/topic/blink-dev/jRzvehX9U1U/discussion,
let's ship it and expose it to larger audience!

This CL removes the Runtime flag in ImageBitmapFactories.idl and updates
some global interface layout tests results.

BUG= 249384 

Review URL: https://codereview.chromium.org/1636633002

Cr-Commit-Position: refs/heads/master@{#371340}

[modify] http://crrev.com/aa3e16a6a6cdec7d99a1741ebad75618909ccca7/third_party/WebKit/LayoutTests/virtual/stable/http/tests/serviceworker/webexposed/global-interface-listing-service-worker-expected.txt
[modify] http://crrev.com/aa3e16a6a6cdec7d99a1741ebad75618909ccca7/third_party/WebKit/LayoutTests/virtual/stable/webexposed/global-interface-listing-dedicated-worker-expected.txt
[modify] http://crrev.com/aa3e16a6a6cdec7d99a1741ebad75618909ccca7/third_party/WebKit/LayoutTests/virtual/stable/webexposed/global-interface-listing-expected.txt
[modify] http://crrev.com/aa3e16a6a6cdec7d99a1741ebad75618909ccca7/third_party/WebKit/LayoutTests/virtual/stable/webexposed/global-interface-listing-shared-worker-expected.txt
[modify] http://crrev.com/aa3e16a6a6cdec7d99a1741ebad75618909ccca7/third_party/WebKit/Source/core/imagebitmap/ImageBitmapFactories.idl

I did some testing of this feature with WebGL with a very large image (8980 x 1540), and it didn't make things any faster...

When I just use a regular image as a texture, texImage2D took a total of 157ms, including an Image Decode task of 130ms. This is a long time to block rendering, especially since I'm using this for WebVR.

When I decode the image with createImageBitmap in a worker and pass the resulting ImageBitmap back to the main thread, it's actually worse. Timeline shows a separate Image Decode task of 102ms followed by texImage2D for 205ms. In total, it's all quite a bit longer.

BTW, I'm using three.js with Chrome 50.0.2640.0 canary (64-bit) on OSX 10.11.3, with NVIDIA GeForce GT 750M 2048 MB.

Any idea why this is the case? Am I doing something wrong?
You are doing everything correctly. The problem here is that ImageBitmap supporting WebGL is not fully implemented yet. Please refer to a separate bug for that: https://code.google.com/p/chromium/issues/detail?id=249382

So what happened is that createImageBitmap decodes the source image, but when the created ImageBitmap is passed to WebGL, texImage2D re-decodes it.

Once the ImageBitmap supporting WebGL is fully implemented, texImage2D won't need to decode the ImageBitmap, which should make texImage2D much faster.

Hope that makes sense.
That is very helpful. Thank you. I'll follow that other bug and test again as it progresses.
Nice to see what is happening here.
Can anyone continue to explain why no pixel data is exposed?
Suppose I want to do some processing after decoding, how to do it?
Cc: xidac...@chromium.org
 Issue 584536  has been merged into this issue.
@#20: The pixel data is deliberately not exposed and is immutable in order to allow for many optimization that would not otherwise be possible. The main reason for this interface to exist is for fast copy-less data flows, and for that the object needs to be opaque. 

To access the raw image data you need to get the image into an ImageData object. Right now, there is not direct way to transfer an ImageBitmap object into an ImageData, you have to transit through a canvas. For example, use drawImage to draw the ImageBitmap into a 2D canvas, then use getImageData to extract an ImageData object from the canvas.  This flow will most likely be improved in the future by adding more direct interfaces that require fewer (or none) intermediate copied of the image data.
There's a WICG spec proposal here to cover getting ImageData asynchronously: http://wicg.github.io/img-conversion/
Status: Started
@xidachen
I have noticed your commit recently to avoid decode twice.

But still a few problems from my perspective
1) The createImageBitmap promise get resolved only after the encoded data is loaded and the size is available, not after decoding, right?
2) Us the result of 1), when you use texImage2D to upload a ImageBitmap, JS code could be something like this:
var image = new image(); image.src..
createImageBimtap(image).then(
function(imageBitmap) {
gl.texImage2D(.., imageBitmap...)
}
);
The gl.texImage2D will still cause a image decoder to decode image, for there the decoded pixel data is not available yet. That will block the render main thread for a while, and the performance is not as good as we expected.

Here is a question:
Why not do decoding before the promise get resolved in a decoding thread? That will make things faster
Comment 27 by a...@scirra.com, Mar 15 2016
Note if comment 26 is right and createImageBitmap resolves before the image is actually fully decoded, that would explain the synchronous decode seen in  issue 592590 .
@johnson.lin:
1). As you can see in the spec here: 
https://html.spec.whatwg.org/#imagebitmapfactories
The createImageBitmap promise is always resolved after image-decoding. I have attached an example and a screenshot from dev-tools. In this example, the image is 8000*8000 pixels, so it takes about 1.2 seconds for image decoding, but you can see that the image decoding happens on the workerpool, not the main thread.

2). The gl.texImage2D will not cause an image decoder to decode it again. It takes a 2D array representing the ImageBitmap's color data.

Hope that helps.
createImageBitmap.html
552 bytes View Download
screenshot.png
107 KB View Download
@xidachen
Thanks,
It sees your recent commit  https://codereview.chromium.org/1781613002
fix. Will have a test
The performance did get improved after  https://codereview.chromium.org/1781613002
#22 #23
Looking forward to the delivery of offscreen canvas, + any further steps toward 
http://wicg.github.io/img-conversion/

would be also great. We also want direct way to retrieve raw pixel data, JFYI (as an end-user)
Labels: Launch-Status-Approved
Status: Fixed
Sign in to add a comment