lz4: Ultra-fast LZ4 with 770 MB/s compression and 4950 MB/s decompression

LZ4 is a lossless data compression algorithm that provides compression speeds of over 500 MB/s per core. Decompression speeds reach several GB/s per core, approaching RAM speed limits on multicore systems. This makes it suitable for environments requiring data processing significantly faster than existing methods.

Compression speed can be dynamically adjusted via acceleration factors, allowing for a trade-off between compression ratio and speed. For higher compression ratios, the LZ4_HC version can be used, though all versions maintain the same decompression speed. Additionally, predefined dictionaries can be used to significantly improve compression performance for small files.

Benchmark results show that LZ4's default settings achieve a compression speed of 770 MB/s and a decompression speed of 4950 MB/s. This outperforms other popular compression algorithms such as Snappy, Zstandard, and zlib in both compression and decompression speeds. In particular, the decompression speed is overwhelmingly faster than competing products.

The library is released under the BSD 2-Clause license and includes a C-based reference implementation along with ports to various languages such as Java, Python, and Ruby. It is highly valuable in development environments where speed is a priority, such as real-time data streaming, log processing, or large-scale database backups.