lrzip: compress large files more

compress_complrzip or Long Range ZIP is an archiver with better compression for large files (> 100MB). It uses more memory to achieve this. lrzip extends the first pass long distance redundancy reduction done in rzip by scaling it according to the available memory. The compression can be LZMA, ZPAQ, LZO, GZIP or BZIP2, However, in our tests we got the best results with ZPAQ.

lrzip works per file. It comes with lrztar as a wrapper to compress and archive directories. The amount of memory needed for best (max level) compression results is roughly the same as the file being compressed. It can use swap memory if available.

Installation

To install lrzip on Ubuntu (along with wrappers), run:

$ sudo apt-get install lrzip

Usage

To compress a directory, run:

$ lrztar somedir

To compress a file, run:

$ lrzip somefile

For best compression size:

$ lrztar -z somedir

For fastest compression (size will be more):

$ lrztar -l somedir

To decompress an archive directory:

$ lrzuntar somedir.tar.lrz

To decompress a compressed file:

$ lrunzip somefile.lrz

Tests

We ran benchmarks for lrzip on an i5 machine with 5GB RAM against 7z and xz compression, all at max compression. The test file was an MPEG video file of size 101.5MB.

$ lrztar -z video/
$ 7zr a -mx=9 video.7z video/
$ tar -cf - video/ | xz -9 -c - > video.tar.xz

lzrtar took more time to complete but showed around 3MB gain:

$ ll -Sr video.*
-rw-rw-r-- 1 neo neo  87885283 Jan 22 20:28 video.tar.lrz
-rw-rw-r-- 1 neo neo  90402799 Jan 22 20:32 video.7z
-rw-rw-r-- 1 neo neo  90416952 Jan 22 20:31 video.tar.xz
-rw-rw-r-- 1 neo neo 106464764 Dec 30 02:18 video.mpg //original file

However, the compression of a 346MB optimized MKV video file showed better results with xz:

$ ll -Sr video.*
-rw-rw-r-- 1 neo neo 360458148 Jan 23 14:33 video.tar.xz
-rw-rw-r-- 1 neo neo 361279556 Jan 23 14:38 video.tar.lrz
-rw-rw-r-- 1 neo neo 361789063 Jan 15 12:13 video.mkv //original file
-rw-rw-r-- 1 neo neo 363510640 Jan 23 14:35 video.7z

Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s