Think of a situation where you have multiple disks. Each time you want to store large files on any of them (or you have a growing collection) you may need to check which disk has enough space. It’s a question of keeping your stuff organized too. You may want to use different volumes for Classical, Pop and Rock. What if the volume with Classical music doesn’t have any space for the next Beethoven collection you received?
RAID could be a solution. However, RAID 0 will stripe data across multiple disks and may not be able to recover any data if one of the disks fail. RAID 1 solves this problem but you need to have one mirror disk for each disk and the available space per mirror set is only equal to the capacity of the smallest disk you got. Last but not the least, RAID configuration is tricky.
mhddfs could rescue you from the situation. It is a simple FUSE filesystem module to combine multiple disks. The utility is a cmdline one and is agnostic of the filesystem on each volume as long as the OS understands them.
mhddfs creates a virtual drive having space equal to the sum of all the participating volumes. It has a mlimit option (default 4GB) which determines whether the disk is low on space. When a new file is added, mhddfs copies it to the first participating disk if it has enough free space. If mlimit is already reached it tries the second one and so on. If all the disks have reached mlimit the file is copied to the disk having maximum free space.
If a file copy fails midway due to some reason, mlimit tries to copy it (including already written data) once again to the disk having maximum free space. However, you don’t need to bother because mhddfs gives you a single virtual volume and takes care of all these internally.
mhddfs does not span the data of a single file between multiple disks and hence it’s easy to “disconnect” the disks. You can copy all the data into a single large enough physical volume or you can unmount the mhddfs mountpoint and get back your data split among the disks.
To install mhddfs on Ubuntu, run:
$ sudo apt-get install mhddfs
We will start with 3 physical volumes:
$ df -h Filesystem Size Used Avail Use% Mounted on ... /dev/sda1 80G 50G 30G 63% /mnt/hdd1 /dev/sdb1 40G 35G 5G 88% /mnt/hdd2 /dev/sdc1 60G 10G 50G 17% /mnt/hdd3
To create a large virtual volume from multiple smaller physical volumes, run:
$ sudo mkdir /mnt/virtual $ sudo mhddfs /mnt/hdd1,/mnt/hdd2,/mnt/hdd3 /mnt/virtual -o allow_other
The allow_other option allows non-root users to read and write the virtual disk.
Check the status of disks:
$ df -h Filesystem Size Used Avail Use% Mounted on ... /dev/sda1 80G 50G 30G 63% /mnt/hdd1 /dev/sdb1 40G 35G 5G 88% /mnt/hdd2 /dev/sdc1 60G 10G 50G 17% /mnt/hdd3 mhddfs 180G 95G 85G 53% /mnt/virtual
To make this arrangement permanent, add the following to /etc/fstab:
mhddfs#/mnt/hdd1,/mnt/hdd2,/mnt/hdd3 /mnt/virtual fuse defaults,allow_other 0 0
To disconnect the volumes, run:
$ sudo umount /mnt/virtual
Man page: mhddfs