Investigate brotli for JS source string compression |
||||
Issue descriptionPer https://quixdb.github.io/squash-benchmark/unstable/, brotli can be both faster and smaller than zlib.
,
Nov 29
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/5d61d204beeee2263f7170efcdfbc8535ce1a5e6 commit 5d61d204beeee2263f7170efcdfbc8535ce1a5e6 Author: Benoît Lizé <lizeb@chromium.org> Date: Thu Nov 29 15:19:02 2018 tools/android: Improve compression_benchmark. Instead of compressing only the start of the data to report smaller chunks compression ratios, compress all the data chunk by chunk. This is meant to accurately represent the impact of larger block sizes on compression ratios, highlighting the impact of compressing whole scripts rather than individual pages. On Pixel (1st generation): Gzip Size = 4096 Compression Compression ratio = 2.2817 Throughput = 21.2864MB/s Latency (size = 4096) = 192.423us 4096,21.2864,192.423 Decompression Throughput = 143.899MB/s Latency (size = 4096) = 28.4645us 4096,143.899,28.4645 Size = 8192 Compression Compression ratio = 2.44109 Throughput = 22.9597MB/s Latency (size = 8192) = 356.799us 8192,22.9597,356.799 Decompression Throughput = 175.248MB/s Latency (size = 8192) = 46.7452us 8192,175.248,46.7452 Size = 16384 Compression Compression ratio = 2.56997 Throughput = 22.1722MB/s Latency (size = 16384) = 738.942us 16384,22.1722,738.942 Decompression Throughput = 206.567MB/s Latency (size = 16384) = 79.3158us 16384,206.567,79.3158 Size = 32768 Compression Compression ratio = 2.67076 Throughput = 19.6633MB/s Latency (size = 32768) = 1666.46us 32768,19.6633,1666.46 Decompression Throughput = 229.605MB/s Latency (size = 32768) = 142.715us 32768,229.605,142.715 Size = 65536 Compression Compression ratio = 2.74245 Throughput = 17.4527MB/s Latency (size = 65536) = 3755.06us 65536,17.4527,3755.06 [...] Bug: 907489 Change-Id: I71654f91980ad10f1b43fcb774bfb3808bffdaf4 Reviewed-on: https://chromium-review.googlesource.com/c/1352175 Reviewed-by: Egor Pasko <pasko@chromium.org> Commit-Queue: Benoit L <lizeb@chromium.org> Cr-Commit-Position: refs/heads/master@{#612198} [modify] https://crrev.com/5d61d204beeee2263f7170efcdfbc8535ce1a5e6/tools/android/io_benchmark/compression_benchmark.cc
,
Nov 30
Brotli on JS Strings
====================
Getting the data
----------------
Data to compress is taken from HTTP archive websites, using the latest mobile crawl.
More specifically:
Taking all JS source files from a random selection of pages in HTTP archive that have some non-trivial amount of JS:
SELECT page, url
FROM `httparchive.requests.2018_11_15_mobile`
WHERE
page IN (
SELECT url
FROM `httparchive.summary_pages.2018_11_15_mobile`
WHERE bytesJS > 100000
ORDER BY RAND()
LIMIT 100)
AND url LIKE '%.js';
This gives >1k scripts, which are then fetched and concatenated:
$ urls=$(cat ~/Downloads/query-results | tail +2 | cut -d',' -f2-)
$ for url in $urls; do curl $url >> ~/js-data.txt; done
$ ls -lh ~/js-data.txt
-rw-r----- 1 lizeb primarygroup 113M Nov 30 11:45 /usr/local/google/home/lizeb/js-data.txt
Running tests
-------------
Tests are run on Linux (Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz, aka Broadwell) and on a first generation pixel (Qualcomm Technologies, Inc MSM8996pro, aka Snapdrgaon 821).
$ out/Release-desktop/compression_benchmark ~/js-data.txt 2>&1 | tee compression-results-linux.txt
Full results attached.
,
Dec 1
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/c1120e9b2177c679288a3bdccecaba357f8c3bf8 commit c1120e9b2177c679288a3bdccecaba357f8c3bf8 Author: Benoît Lizé <lizeb@chromium.org> Date: Sat Dec 01 02:02:06 2018 tools/android: Benchmark brotli compression on JS. Adds support for brotli compression to the compression_benchmark test tool. Results below are taken from a sample of JS files found on the web (see the linked bug). Overall, at comparable compression ratios, brotli is slower than zlib for decompression but faster at compression, on Android and Linux. Tested on Pixel (Android) and Xeon "broadwell" (linux). For instance, for a 32kiB chunk size: - Linux Zlib Size = 32768 Compression Compression ratio = 2.97379 Throughput = 35.7633MB/s Latency (size = 32768) = 916.246us Decompression Throughput = 329.092MB/s Latency (size = 32768) = 99.5711us Brotli Size = 32768 Compression Compression ratio = 2.81762 Throughput = 52.4092MB/s Latency (size = 32768) = 625.234us Decompression Throughput = 159.92MB/s Latency (size = 32768) = 204.903us - Android Zlib Size = 32768 Compression Compression ratio = 2.97379 Throughput = 18.7666MB/s Latency (size = 32768) = 1746.08us Decompression Throughput = 204.837MB/s Latency (size = 32768) = 159.971us Brotli Size = 32768 Compression Compression ratio = 2.81762 Throughput = 30.1437MB/s Latency (size = 32768) = 1087.06us Decompression Throughput = 85.4071MB/s Latency (size = 32768) = 383.668us Bug: 907489 Change-Id: Ia6b0877fa6964098bd99e8d2dea3ff09e316d910 Reviewed-on: https://chromium-review.googlesource.com/c/1356549 Commit-Queue: Kenichi Ishibashi <bashi@chromium.org> Reviewed-by: Kenichi Ishibashi <bashi@chromium.org> Reviewed-by: Egor Pasko <pasko@chromium.org> Cr-Commit-Position: refs/heads/master@{#612903} [modify] https://crrev.com/c1120e9b2177c679288a3bdccecaba357f8c3bf8/tools/android/io_benchmark/BUILD.gn [modify] https://crrev.com/c1120e9b2177c679288a3bdccecaba357f8c3bf8/tools/android/io_benchmark/DEPS [modify] https://crrev.com/c1120e9b2177c679288a3bdccecaba357f8c3bf8/tools/android/io_benchmark/compression_benchmark.cc
,
Dec 3
Full results here: https://colab.corp.google.com/drive/1W-JieQZNX3uH2UhgyC7KjF3HWKOnNrt- HTML attached. tl;dr: With a comparable compression ratio, brotli is faster to compress, and significantly slower to decompress than zlib, both on Linux and Android.
,
Dec 3
,
Dec 3
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/ed6dc213c9a0e79b389afa552e92dfc7295745f0 commit ed6dc213c9a0e79b389afa552e92dfc7295745f0 Author: Benoît Lizé <lizeb@chromium.org> Date: Mon Dec 03 13:07:05 2018 tools/android: Better formatting in compression_benchmark. Better CSV output. Bug: 907489 Change-Id: Ibe6be70beffacdabc570c4bdc2a8868c0e9ab41c Reviewed-on: https://chromium-review.googlesource.com/c/1358432 Commit-Queue: Egor Pasko <pasko@chromium.org> Reviewed-by: Egor Pasko <pasko@chromium.org> Cr-Commit-Position: refs/heads/master@{#613065} [modify] https://crrev.com/ed6dc213c9a0e79b389afa552e92dfc7295745f0/tools/android/io_benchmark/compression_benchmark.cc
,
Dec 12
,
Jan 9
|
||||
►
Sign in to add a comment |
||||
Comment 1 by bugdroid1@chromium.org
, Nov 27