Linux: Synchronize Local and Remote Directories With Rsync
If you manage multiple Linux servers, you might find yourself in a situation where you have two machines with directories that need to remain in sync with one another. This can be used as a backup solution or maybe you have different servers assigned to different users but have that one directory housing data for which all users need to have access.
There are several use cases for such a setup and one tool to make setting it up quite simple. That tool is rsync.
Rsync stands for remote sync and has been around for a long, long time. I’ve used rsync as a reliable (scriptable) backup solution on Linux but have also found instances where it was necessary that local and remote directories are always in sync with one another.
Nearly every Linux distribution ships with rsync, so there’s nothing to install; it’s just a matter of configuring everything. Of course, if you find yourself with a distribution that doesn’t include rsync, I’ll show you how to take care of that.
Let’s get those directories in sync.
What You’ll Need
To make this work, you’ll need two Linux machines and a user with sudo privileges. Both machines should be configured with static IP addresses, so you don’t have to worry about either address changing, which would cause the sync to fail.
With that in mind, let’s get to work.
Installing Rsync
Just in case you run into a situation where rsync isn’t installed, here’s how you can do it on both Debian and Fedora-based distributions:
- Debian –
sudo apt-get install rsync -y - Fedora –
sudo dnf install rsync -y
Once rsync is installed, make sure it’s started and enabled with the command:
sudo systemctl enable --now rsync
Rsync is now ready for configuration.
Configuring the Destination
Let’s say we have two different Linux servers:
- 192.168.1.115 — source
- 192.168.1.116 — destination
The first thing we’ll do is create a directory that will house the synced data. Let’s do that on the destination server with the command:
sudo mkdir /data
Still, on the destination server, we need to create an rsync configuration file. Create a new file to house the configuration with the command:
sudo nano /etc/rsyncd.conf
In that file, paste the following content:
Where DIRECTORY is our newly created/data directory (or whatever you named it) and SOURCE_IP is the IP address of the source machine.
Let’s break this down.
- [backup] – this is the title of the block that will be used when syncing from the source.
- path
- path – the full path to the directory that will house the synced data.
- hosts allow – the IP address of the source machine.
- hosts deny – this instructs rsync that all other IP addresses are to be denied.
- list – determines if the module is listed when a client requests.
- uid – the user ID which can be set to root, nobody, or a Linux user.
- gid – the group ID
- read only – defines if clients can upload files or not.
Make sure to change DIRECTORY and SOURCE_IP. Once you’ve done that, save and close the file.
Test the Connection
What we’re going to do now is test to make sure rsync works as expected. On the source machine (the machine housing the directory that will be synced with /data on the destination server), run the command:
rsync -avz DIRECTORY DESTINATION_IP::backup
Let’s break that command down:
- rsync – the main command.
- -avz – options for archive, verbose output, and compress.
- DIRECTORY – the directory on the source machine that is to be synced with the destination directory.
- DESTINATION_IP – the IP address of the destination machine.
- backup – the name of the job listed in the rsyncd.conf file.
After running this command, everything in the source directory should now be found in the destination directory. If that’s the case, you’ve successfully configured rsync. The only issue at the moment is that it’s currently capable of syncing manually. Let’s automate that process.
Automating Rsync
Linux has a very powerful tool for automation called cron. With cron, you can set up jobs that run at just about any interval you need. Let’s say you want those folders to be synced every five minutes. The first thing we must do is create a bash script that will be run by cron.
Create the bash script with the command:
nano rsync.sh
In that file, add the following:
#! /bin/bash rsync -avz DIRECTORY DESTINATION_IP::backup
Where DIRECTORY to the full path of the directory and DESTINATION_IP is the IP address for the remote machine.
Once you’ve taken care of that, give the file executable permission with:
chmod u+x rsync.sh
Now, we create our cronjob. Open the cronjob editor with:
crontab -e
At the bottom of the file, add the following:
Make sure to change /path/to/rsync.sh to the exact path of the rsync.sh bash script. The > /dev/null 2>&1 section silences the output of the rsync command, otherwise, it would error out when the cronjob attempts to run.
Save and close the file.
The rysnc.sh script will now start running every five minutes, to ensure the destination directory is in sync with the source directory.
Two-Way Syncing
Let’s say you need both source and destination to be in sync with one another. To do that, it’s essentially the same, the only difference is you have to run rsync twice (which can be added to the rsync.sh script). We’ll do that with the -au options like so:
- Source to destination – rsync -avz DIRECTORY DESTINATION_IP::backup
- Destination to source – rsync -avz DESTINATION_IP::backup DIRECTORY
For this, our rsync.sh script will be:
Where DIRECTORY to the full path of the directory and DESTINATION_IP is the IP address for the remote machine.
Now, both local and remote directories will be in sync every five minutes (or whatever interval you’ve set for your cron job).
And that’s how you can use rsync to synchronize two Linux directories on your local network. Yes, there are easier ways to do this, but using rsync and cron is a quick and flexible method that will never fail you.
