I’ve been looking at cloning ( or somehow maintaining synced ) file-systems between multiple servers, and after looking at other solutions ( regular rsyncs, inotify etc ), a clustered file-system seems to be the best solution. GlusterFS looks to be a popular Open Source based solution, however there doesn’t seem to be a complete walk-through for setting it up on Gentoo ( there is a guide for Fedora 13 which I have based some of this guide from, however it seems to be based on an older version of GlusterFS ), so I’ve decided to write one specifically for Gentoo and GlusterFS 3.2.
For this tutorial, I am only going to document setting up 2 servers, but adding more servers during the initial setup should just require adding the additional servers in the same way as the first two are described ( an extra hosts entry, extra ‘disks’ listed on the volume creation commands ).
In this example, we have two servers, server1 and server2. If there is no DNS setup to do this already, the first step is to add these entries on each machine, replacing the server IPs with the correct ones:
192.168.1.100 server1 192.168.1.101 server2
Next, simply install and start sys-cluster/glusterfs on each machine:
emerge sys-cluster/glusterfs /etc/init.d/glusterd start
A Glusterfs daemon should now be listening on both servers. Next we need to probe the second server from the first ( which fixes the ‘Host server2 not a friend’ error message which I had before I ran this ), simply run this on server1:
gluster peer probe server2
You can now confirm that the machines can see each other using the gluster peer status command ( you can do this on both machines and they should report each other ):
# gluster peer status Number of Peers: 1 Hostname: server2 Uuid: 963aee89-fca3-4a5a-90ae-ada89f8a277d State: Peer in Cluster (Connected)
Now we have GlusterFS up and running, we can create a directory to store the data on each machine:
mkdir -p /export/store1
Now we can actually create the volume for the data to sit on. For this example I have set replica to be 2, so the data will exist on all machines ( this command only needs to be ran on server1 ):
gluster volume create test-volume replica 2 transport tcp server1:/export/store1 server2:/export/store1
This should return successfully, now we can start the volume ( again, only run this on server1 ):
gluster volume start test-volume
We now have a GlusterFS volume up and running, but before we can access the file-system, we need to mount it. First create a mount-point on both servers:
Then ensure GlusterFS runs on boot ( on both servers ):
rc-update add glusterd default
And add a row to fstab on server1 ( so the file-system is mounted on boot ):
server1:/test-volume /mnt/glusterfs glusterfs auto,rw,allow_other,default_permissions,max_read=131072 0 0
server2: server2:/test-volume /mnt/glusterfs glusterfs auto,rw,allow_other,default_permissions,max_read=131072 0 0
On both servers we can now mount the file-system manually, and should be able to add/remove/copy etc files on one machine and have it appear on the other:
Assuming there were no errors reporting, the disks should now stay in sync ( it seems that even when one machine is disabled, and files are added to the disk , the files are copied to the other machine when it returns. ), and you have a solution for serving or sharing multiple files across multiple servers.