New issue
Advanced search Search tips
Starred by 36 users

Issue metadata

Status: ExternalDependency
Owner: ----
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Linux , Windows , Mac
Pri: 2
Type: Feature



Sign in to add a comment
link

Issue 669501: Arithmetic coded JPEG support

Reported by evgeny.v...@gmail.com, Nov 29 2016

Issue description

UserAgent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:51.0) Gecko/20100101 Firefox/51.0

Steps to reproduce the problem:
Open http://filmicgames.com/Images/Patents/bedroom_arithmetic.jpg

What is the expected behavior?
It have to be displayed

What went wrong?
Blank page is displayed

Did this work before? No 

Does this work in other browsers? N/A

Chrome version: 54.0.2840.99  Channel: n/a
OS Version: 6.1
Flash Version: 

As arithmetic coding patents for JPEG have now expired, it would be nice to support arithmetic coding in the JPEG. It will allow recompress all existing JPEG files using arithmetic coding losslessly, as the result size of all JPEG files will be reduced by 10% and even more.
 

Comment 1 by schenney@chromium.org, Nov 29 2016

Components: -Blink>Image Internals>Images>Codecs
Labels: -Type-Bug Type-Feature
Thanks for the info.

Comment 2 by ligim...@chromium.org, Nov 29 2016

Cc: ligim...@chromium.org
Labels: Needs-Triage-M54 M-54

Comment 3 by evgeny.v...@gmail.com, Nov 30 2016

Some additional info:
- Arithmetic coding is a part of the JPEG standard. Initially, it wasn't implemented  by decoders due patents. But now there is no such limitation.
- libjpeg supports arithmetic coding since version 7.x (2009): https://en.wikipedia.org/wiki/Libjpeg#Summary
- libjpeg-turbo and mozjpeg also support arithmetic coding.

As an example I've downloaded random images from the Google Play and tried to recompress it losslessly using mozjpeg's arithmetic coding. I've used such commands:

jpegtran -optimize -outfile demo1_huffman_opt.jpg demo1_huffman_orig.jpg
jpegtran -optimize -arithmetic -outfile demo1_arithmetic.jpg demo1_huffman_orig.jpg
jpegtran -optimize -outfile demo2_huffman_opt.jpg demo2_huffman_orig.jpg
jpegtran -optimize -arithmetic -outfile demo2_arithmetic.jpg demo2_huffman_orig.jpg

The results:

demo1_huffman_orig.jpg 879922 bytes (an original file from the Google Play)
demo1_huffman_opt.jpg  848015 bytes (-4%, optimized losslessly, huffman coded)
demo1_arithmetic.jpg   711015 bytes (-19%, optimized losslessly, arithmetic coded)

demo2_huffman_orig.jpg 967354 bytes (an original file from the Google Play)
demo2_huffman_opt.jpg  925276 bytes (-4%,  optimized losslessly, huffman coded)
demo2_arithmetic.jpg   843037 bytes (-13%, optimized losslessly, arithmetic coded)

So, it would be nice if arithmetic coding will be supported. It will be possible to automatically recompress losslessly all images without worrying that quality will be worse, just because it will be the same, but 10-20% less in size.
demo1_huffman_orig.jpg
859 KB View Download
demo1_huffman_opt.jpg
828 KB View Download
demo1_arithmetic.jpg
694 KB View Download
demo2_huffman_orig.jpg
944 KB View Download
demo2_huffman_opt.jpg
903 KB View Download
demo2_arithmetic.jpg
823 KB View Download

Comment 4 by durga.behera@chromium.org, Nov 30 2016

Labels: -M-54 M-57 OS-Linux OS-Mac
Status: Untriaged (was: Unconfirmed)
Able to reproduce the issue on Mac 10.11.6,Ubuntu 14.04 and Win 10 using stable 54.0.2840.98/99/100 and canary 57.0.2937.0.

Note: FireFox too behave the same way as chrome.

Marking this as untriaged to get addressed further.

Comment 5 by 3bl3ga...@gmail.com, Nov 30 2016

Arithmetic decoding works for me on Arch Linux. Tested in Chromium 54.0.2840.100 (64-bit).

But there seems to be couple of bugs:
1) for the first time only ~1/5 of images is displayed (http://i.imgur.com/r1Si2XX.png) but after refresh is fully apperas (http://i.imgur.com/cgTzGAF.jpg)
2) arithmetic-encoded images from Comment 3 are not displayed, console shows a warning "Resource interpreted as Document but transferred with MIME type image/jpeg". Although if saved locally as smth.jpg and opened as file:// it appears correctly.

Comment 6 by koc...@gmail.com, Dec 3 2016

Arithmetic decoding works for me on Android 5.1.1. Tested in Chrome 54.0.2840.85
Screenshot_2016-12-03-12-29-22.png
828 KB View Download

Comment 7 by pkasting@chromium.org, Dec 6 2016

Cc: pkasting@chromium.org
What would be the costs to Chrome of doing this?

* We're built on libjpeg-turbo, which you said includes support for this.  Is such support under a compile-time switch?  If yes, how much does enabling it increase the binary size?  Would we need to write any additional code in the wrapper around libjpeg-turbo, or would support be "transparent"?

* Has the relevant code here been security-audited?  Image codecs are a key security surface, and we need to be cautious in how we add more here.

* Do authors need some kind of accept header or similar indication that Chrome supports arithmetic coding in order to safely compress images in this format?

Basically, if additional binary size cost is very low, there is minimal additional security risk, and we don't need to provide more signals to authors about this, then we'll be more positive about taking this.  If these factors are untrue, we need to think more carefully.

Comment 8 by d...@virtualgl.org, Dec 7 2016

Mr. libjpeg-turbo here.  I can answer some of these questions:

* Arithmetic coding is a compile-time switch in libjpeg-turbo that increases the binary size only a miniscule amount (620541 bytes vs. 599328 bytes for the libjpeg API shared library on 64-bit Linux, at least in my testing.  Actual mileage may vary.)  The libjpeg-turbo build system has been enabling arithmetic coding by default since v1.1.0.  Support is transparent.  The arithmetic-coded JPEGs should "Just Work" (TM) if that feature is enabled in libjpeg-turbo.

* Yes, the code has been audited:  https://wiki.mozilla.org/MOSS/Secure_Open_Source/Completed#libjpeg-turbo, and it continues to receive attention from security-minded people (for instance, I have fixed several UBSan warnings in the code within the last year, which were reported by the community.)

* The accept header question is a good one, and I'm not sure.  Speaking as someone who has done his share of web development, I'd be very hesitant to use arithmetic-codec JPEGs in a web site until I was confident that the vast majority of browsers out there supported them.  That means minimally Mozilla, Google, Microsoft, and Apple need to get on the train, but probably others as well (Opera.)  We'd also need the authors of the most popular web development tools (particularly Adobe) to get on board.  PhotoShop, for instance, currently doesn't support arithmetic coding either.  I think this is a situation in which someone is going to have to make the first move in order to get the train rolling, and it makes sense that the open source browsers make that first move.  Browser support is only Step 1, because there will still be older browsers that don't support the format.  Web developers would have to phase it in over a period of years.  I do think that some kind of tag to let web developers know that arithmetic coding is supported would be helpful during that phase-in period.

Apparently the arithmetic codec doesn't currently support suspension (https://github.com/libjpeg-turbo/libjpeg-turbo/issues/120), which I'm given to understand is necessary for browser support.  If that's true, then that issue blocks this one.  It's unfortunately not something I could fix without someone paying for the labor (contact me offline for an estimate) or contributing code.  Also, the arithmetic codec currently decodes much more slowly than the Huffman codec, which brings the overall decode performance down by about 6x (vs. baseline JPEG.)  That means that you may see some performance improvement on really slow connections, but the results on faster connections are likely to be mixed.

I would personally like to see arithmetic coding be adopted more widely, but I also really think the codec needs some improvements (support for suspending data sources, better performance, etc.) before it can be as good of a solution as the current Huffman codecs in libjpeg-turbo.

Comment 9 by d...@virtualgl.org, Dec 7 2016

I should also say that, since arithmetic coding is enabled by default in libjpeg-turbo, most vendor-supplied versions of it on Linux/Unix already have this feature.  Thus any libjpeg-dependent software shipped with those Linux/Unix distributions (ImageMagick, Konqueror, and probably even the vendor-supplied builds of Firefox) has the feature as well.

Comment 10 by pkasting@chromium.org, Dec 7 2016

Status: ExternalDependency (was: Untriaged)
@8: That's super-helpful.  Thanks.

Reading the linked bug, I definitely agree that supporting suspension is a blocker for officially marking this supported.  Probably improving decode performance to be closer to the Huffman codec (if doing so is possible) would be a blocker too, as image decode perf is very important to us, and it likely wouldn't make sense to ship something that pays a 6x decode perf penalty for ~15% compression improvement.

I'm marking this as ExternalDependency for now, as further action on our part would be gated on addressing the above.

Assuming that were addressed:
* The notes about binary size and security auditing are encouraging.  I suspect that binary size cost is something we'd be willing to take.
* The accept-header question is one we'd probably look at in the same way we've looked at WebP and APNG while adding support for them.  For WebP we added a header.  For APNG I'm not sure what the current decision is.

I would summarize this as "if suspension were added and performance improved, it would be reasonable for us to at least consider adding support for these images, but I don't guarantee we'd decide to do so; and Chromium isn't going to pay for the above work to be done".

Comment 11 by evgeny.v...@gmail.com, Dec 7 2016

I've tested the libjpeg-turbo v1.5.1 performance using tjbench.

The results:
demo2_huffman_orig.jpg (1789 x 2560): 38.6 fps
demo2_huffman_opt.jpg  (1789 x 2560): 11.9 fps
demo2_arithmetic.jpg   (1789 x 2560): 5.5 fps

demo2_huffman_opt.jpg is supported by all browsers. So, supporting arithmetic coding only 2x slower than the "worst" case in the current supported JPEG subset.

Also I've compared it with WebP (v0.5.1):

cwebp -q 85 src.png -o demo2_q85.webp
dwebp demo2_q85.webp -v -o dst.png
cwebp -q 95 src.png -o demo2_q95.webp
dwebp demo2_q95.webp -v -o dst.png

The result:

demo2_q85.webp         (1789 x 2560): 9.7 fps
demo2_q95.webp         (1789 x 2560): 6.6 fps

So, the performance of the decoding of the arithmetic coded JPEGs is not so bad.

Comment 12 by tobefrei...@gmail.com, Oct 22 2017

GIMP 9 supports opening and saving of arithmetic coded JPEGs; the current development version allows you to test it.

I naively, happily created such a JPEG and uploaded it to Wikimedia Commons, where it failed to be displayed by my Chrome on Win10 correctly.

https://upload.wikimedia.org/wikipedia/de/archive/e/e6/20171022154923%21Mars_Terra.jpg
Mars_Terra.jpg
159 KB View Download

Comment 13 by ewoutter...@gmail.com, Jul 26 2018

Still fails to display in Chrome 68 (Win 10 64-bit). Any chance we will see support added?

Comment 14 by oscarnar...@outlook.com, Feb 16 (3 days ago)

Any update on this?

Sign in to add a comment