MerkleTreeLeaf compression benchmark

results.md

Single MerkleTreeLeaf

Algorithm	level	Compression
zstd	3	566 B => 417 B (73.67%)
lz4	3	566 B => 438 B (77.39%)

Single MerkleTreeLeaf with dictionary

Dictionary created with zstd --train -r leafs/.

Algorithm	level	Compression
zstd	3	566 B => 128 B (22.61%)
lz4	3	566 B => 135 B (23.85%)

Bulk

32768 MerkleTreeLeafs (concatenated bytes). Original file size 20021421 bytes (19.1 MiB)

$ compresto benchmark-many leafs

Algorithm	level	Compression	Compression speed	Decompression speed
lz4	1	4.6 MiB (24.0%, 4.17x)	759.4 MB/s	15302.0 MB/s
lz4	2	4.6 MiB (24.0%, 4.17x)	782.1 MB/s	15453.1 MB/s
lz4	3	4.5 MiB (23.4%, 4.27x)	623.7 MB/s	15639.7 MB/s
lz4	4	4.5 MiB (23.3%, 4.29x)	579.4 MB/s	16007.5 MB/s
lz4	5	4.5 MiB (23.3%, 4.29x)	558.4 MB/s	17411.8 MB/s
lz4	6	4.5 MiB (23.3%, 4.29x)	524.4 MB/s	16042.8 MB/s
lz4	7	4.5 MiB (23.3%, 4.29x)	495.9 MB/s	16622.8 MB/s
lz4	8	4.5 MiB (23.3%, 4.29x)	484.8 MB/s	15374.5 MB/s
lz4	9	4.5 MiB (23.3%, 4.29x)	469.4 MB/s	16189.3 MB/s
lzav	0	4.6 MiB (23.9%, 4.18x)	2113.0 MB/s	10056.0 MB/s
lzav	1	4.5 MiB (23.6%, 4.24x)	520.4 MB/s	9913.2 MB/s
snappy	0	5.1 MiB (26.9%, 3.72x)	3359.8 MB/s	7509.2 MB/s
zstd	1	4.3 MiB (22.4%, 4.46x)	1081.3 MB/s	6359.4 MB/s
zstd	2	4.3 MiB (22.3%, 4.48x)	1081.6 MB/s	6234.8 MB/s
zstd	3	4.2 MiB (22.2%, 4.50x)	889.5 MB/s	5887.5 MB/s
zstd	4	4.2 MiB (22.1%, 4.52x)	429.9 MB/s	6221.3 MB/s
zstd	5	4.2 MiB (22.1%, 4.52x)	397.0 MB/s	6475.1 MB/s
zstd	6	4.2 MiB (22.1%, 4.52x)	362.4 MB/s	6842.5 MB/s
zstd	7	4.2 MiB (22.1%, 4.52x)	318.2 MB/s	6696.8 MB/s
zstd	8	4.2 MiB (22.1%, 4.52x)	317.2 MB/s	6892.1 MB/s
zstd	9	4.2 MiB (22.1%, 4.52x)	124.8 MB/s	6821.1 MB/s
zstd	10	4.2 MiB (22.1%, 4.52x)	111.1 MB/s	6923.4 MB/s
zstd	11	4.2 MiB (22.1%, 4.52x)	29.7 MB/s	6895.6 MB/s
zstd	12	4.2 MiB (22.1%, 4.52x)	27.3 MB/s	5977.1 MB/s
brotli	1	4.5 MiB (23.5%, 4.26x)	534.1 MB/s	756.1 MB/s
brotli	2	4.5 MiB (23.3%, 4.29x)	351.7 MB/s	772.3 MB/s
brotli	3	4.3 MiB (22.7%, 4.41x)	271.2 MB/s	775.8 MB/s
brotli	4	4.3 MiB (22.5%, 4.44x)	196.3 MB/s	733.8 MB/s
brotli	5	4.2 MiB (22.0%, 4.55x)	158.5 MB/s	794.1 MB/s
brotli	6	4.2 MiB (22.0%, 4.55x)	96.6 MB/s	792.3 MB/s
brotli	7	4.2 MiB (22.0%, 4.55x)	43.9 MB/s	797.8 MB/s
brotli	8	4.2 MiB (22.0%, 4.55x)	121.1 MB/s	801.7 MB/s

Bulk with dictionary

Algorithm	level	Compression	Compression speed	Decompression speed
lz4	1	4.6 MiB (24.0%, 4.17x)	748.1 MB/s	16127.9 MB/s
lz4	2	4.6 MiB (24.0%, 4.17x)	811.2 MB/s	17027.4 MB/s
lz4	3	4.5 MiB (23.4%, 4.27x)	629.4 MB/s	16857.8 MB/s
lz4	4	4.5 MiB (23.3%, 4.29x)	589.8 MB/s	16174.0 MB/s
lz4	5	4.5 MiB (23.3%, 4.29x)	541.0 MB/s	15598.1 MB/s
lz4	6	4.5 MiB (23.3%, 4.29x)	515.3 MB/s	16468.9 MB/s
lz4	7	4.5 MiB (23.3%, 4.29x)	478.9 MB/s	16177.3 MB/s
lz4	8	4.5 MiB (23.3%, 4.29x)	482.2 MB/s	16020.3 MB/s
lz4	9	4.5 MiB (23.3%, 4.29x)	450.3 MB/s	16152.3 MB/s
lzav	0	4.6 MiB (23.9%, 4.18x)	2133.1 MB/s	10421.3 MB/s
lzav	1	4.5 MiB (23.6%, 4.24x)	502.8 MB/s	9630.3 MB/s
snappy	0	5.1 MiB (26.9%, 3.72x)	3267.1 MB/s	7619.3 MB/s
zstd	1	3.5 MiB (18.4%, 5.43x)	886.0 MB/s	9688.0 MB/s
zstd	2	3.5 MiB (18.4%, 5.43x)	724.7 MB/s	8451.1 MB/s
zstd	3	3.5 MiB (18.5%, 5.41x)	683.0 MB/s	9166.6 MB/s
zstd	4	3.5 MiB (18.5%, 5.41x)	675.5 MB/s	9804.4 MB/s
zstd	5	3.5 MiB (18.4%, 5.43x)	349.0 MB/s	9695.4 MB/s
zstd	6	3.5 MiB (18.3%, 5.46x)	306.1 MB/s	8536.7 MB/s
zstd	7	3.5 MiB (18.3%, 5.46x)	280.5 MB/s	8417.2 MB/s
zstd	8	3.5 MiB (18.3%, 5.46x)	258.6 MB/s	8635.8 MB/s
zstd	9	3.5 MiB (18.3%, 5.46x)	218.2 MB/s	8547.3 MB/s
zstd	10	3.5 MiB (18.3%, 5.46x)	190.3 MB/s	7423.9 MB/s
zstd	11	3.5 MiB (18.3%, 5.46x)	139.1 MB/s	7903.8 MB/s
zstd	12	3.5 MiB (18.3%, 5.46x)	130.2 MB/s	8160.6 MB/s
brotli	1	4.5 MiB (23.5%, 4.26x)	535.6 MB/s	747.9 MB/s
brotli	2	4.5 MiB (23.3%, 4.29x)	353.8 MB/s	758.6 MB/s
brotli	3	4.3 MiB (22.7%, 4.41x)	271.1 MB/s	772.1 MB/s
brotli	4	4.3 MiB (22.5%, 4.44x)	197.7 MB/s	731.6 MB/s
brotli	5	3.5 MiB (18.2%, 5.49x)	145.8 MB/s	1070.5 MB/s
brotli	6	3.5 MiB (18.2%, 5.49x)	90.5 MB/s	1073.2 MB/s
brotli	7	3.5 MiB (18.2%, 5.49x)	40.7 MB/s	1060.4 MB/s
brotli	8	3.5 MiB (18.2%, 5.49x)	124.7 MB/s	1051.1 MB/s