Skip to end of metadata
Go to start of metadata

For a long time now, ZFS has allowed for transparent compression of datasets using a number of user-selectable algorithms, one of the most frequently used ones being LZJB. With the integration of issue #3035, Illumos now also includes support for LZ4, an extremely fast BSD-licensed algorithm designed and written by Yann Collet. LZ4's high performance and features make it a good potential replacement for LZJB in the future.

LZ4 Performance

A rough comparative analysis of LZ4 performance vs. LZJB:

  • Approximately 50% faster compression when operating on compressible data.
  • Approximately 80% faster on decompression.
  • Over three times faster on compression of incompressible data.
  • Higher compression ratio (up to 10% on the larger block sizes).
  • Performance on modern CPUs often exceeds 500 MB/s on compression and over 1.5 GB/s on decompression and incompressible data (per single CPU core).

See detailed numbers for a small benchmark run. The integrated version of LZ4 corresponds to the "lz4 (r85)" rows ("lz4hc" is a different high-compression variant of LZ4). These numbers were obtained using a development build on an HP MicroServer with a 1.3 GHz dual-core 64-bit Athlon II processor with 8GB of RAM. The dataset used was the standard Silesia Corpus test set. The testing zpool was backed by a ramdisk so as to avoid hitting any IO subsystem bottlenecks and show pure LZ4 on ZFS throughput.

How To Use It

You will need an Illumos build published after January 17th, 2013. If the feature isn't available in your distro yet, you can try and build Illumos yourself, the source and build instructions are easily available.

LZ4 uses the new ZFS Features Flags feature to facilitate on-disk format management. If you would like to enable LZ4 on a pool which doesn't have feature flags enabled, upgrade the pool first using the "zpool upgrade <poolname>" command. Please note that this will make your pool unimportable on systems which lack support for feature flags (e.g. Solaris 10, Oracle Solaris 11, etc.). See zpool-features(5) for more information on how feature flags work in ZFS.

Since LZ4 compression introduces new blocks into your pool compressed using a non-backwards-compatible format, pools which use LZ4 cannot be imported on systems which lack support for the LZ4 feature. To guard against this, ZFS will refuse to enable LZ4 compression on any dataset unless the feature@lz4_compress pool property is set to enabled. To enable it, simply issue the "zpool set feature@lz4_compress=enabled <poolname>" command. From this moment on, you can set compression on any dataset on the pool to LZ4 using the familiar ZFS set command: "zfs set compression=lz4 <dataset>". As soon as this is done, the lz4_compress feature on the pool will click over into the active position and you will need LZ4 support on the machine to import the pool again.

Allow me to stress again: enabling the lz4_compress feature on the pool and setting compression=lz4 on ANY dataset on the pool activates the feature, rendering your pool unimportable on machines without LZ4 support. This operation is irreversible, so think twice before switching to LZ4 on your production machines.

LZ4 On Root Pools

This integration work includes a patch to GRUB's stage2 ZFS module to allow you to boot LZ4-compressed pools. When enabling LZ4 on your existing root pool, FIRST update GRUB and make sure you can boot with it prior to enabling LZ4 on your root pool. If you don't update GRUB but only your ZFS kernel modules, and you enable LZ4 straight away, you might get this message when GRUB loads:

missing feature for read 'org.illumos:lz4_compress'

This essentially means that GRUB can't read your pool and you can't boot from it. At this point, the only thing you can do is fire up a Live ISO with LZ4 support (so that you can import your unbootable root pool) and update GRUB's stage2 from there. Needless to say, it's better to avoid this, so always remember to update GRUB first and only then enable LZ4 on your root pool.

Why Use LZ4?

There is a myriad of arguments for and against enabling transparent compression on your ZFS datasets and we're not going to repeat them all here. If you are already using compression, this feature will improve your performance essentially across the board. Therefore, the points below might convince people who have traditionally avoided compressing datasets for one reason or another to give LZ4 a try:

  • The extremely high per-core throughput of LZ4 means that database admins can lower the impact that compression has on transaction latency (at 1.5 GB/s decompression of an 8k disk block adds only about 5 microseconds to each storage access, about an order of magnitude faster than even the fastest flash devices available today).

  • The high throughput coupled with a 2x reduction in data volume might also benefit storage systems by reducing congestion on IO buses. Storage clusters typically reach saturation in their IO subsystems much sooner than on their CPUs. Using high-speed compression can help you shift the balance away from your IO subsystem by making your mostly idle CPUs share partly in the storage chores.

  • Another particular thing to note is that LZ4's performance on incompressible data is very high. It achieves this by incorporating an "early abort" mechanism which will trigger if LZ4 can't meet the expected minimum compression ratio (12.5% on ZFS). This feature is particularly useful to storage administrators who would like to use compression as an "enable and forget about it" feature. Enabling LZ4 across the board only uses compression where appropriate, while making sure that the algorithm won't work hard on incompressible data (e.g. multimedia) and reads don't incur a latency penalty due to transparent decompression.

How significant the above performance improvements depends largely on your dataset composition. If you have a reasonable mix of compressible data, then the gains might be significant. On the other hand, if you're dealing mostly with incompressible stuff (e.g. multimedia streaming), LZ4 isn't going to help you much at all.

Credits

The greatest credit should most certainly go to Yann Collet for having created such a good algorithm and sharing it with the world. The algorithm was integrated into Illumos by Sašo Kiselkov.

Labels: