Dev Notes

Software Development Resources by David Egan.

Secure rsync Between Servers


Backup, Linux, Sysadmin, rsync
David Egan

This method allows for automatic incremental backups between servers.

Backup Server Pulls from Production Server

Either server could initiate the sync - but if Production initiates, it would need write access to the Backup server. It is probably safest to give the Backup server read access to the Production server, so this guide assumes that the Backup machine initiates the sync.

The files to be synced have been built by a local automatic rsync-based incremental backup script that runs on the Production server. This is because:

  • There is enough space on the Production machine (approx 8G out of 40G is in use)
  • Having an incremental backup on the Production server allows for quick restores in the event of non-catastrophic failure
  • Having backups on the production server is “belt and braces” - backups are stored in multiple locations
  • The script has root access and can build a complete set of backup files: /var/www/html/, Apache config files, MySQL config files, Database files (built by mysqldump)

This method uses the rrsync perl script (restricted rsync) to restrict rsync to a subdirectory declared in .ssh/authorized_keys. This works really well since our local incremental backup has written all the files we need to a single backup directory.

The Incremental Backup that is being synced is described here.

Set up a Backup User on the Production Server

Set up a user without sudo privileges. This user is only going to be used to store backups - which provides another level of security.

The backup user home directory will receive incremental backups, and hold the SSH public key to allow the remote backup server to connect - and that’s about it.

Set up an SSH Key Pair on the Backup Server

Because the backup script is designed to be triggered automatically by a cron job, the user that will be running the rsync command on the production server will be root.

It is therefore important to transfer the public SSH key for the root user to the backup target machine.

In addition, the SSH key should NOT have a passphrase, so that connection can be made automatically. We will also limit the actions that can be carried out by this SSH connection.

# Switch to root
su root

# Generate an SSH key pair, WITHOUT a passphrase
ssh-keygen -t rsa

# copy the public key to the backupuser account on the target Production machine
ssh-copy-id -p 1234 backupuser@123.456.789.0

Failure to Copy Public Key

If the public key doesn’t copy across, you’ll see an error message like this:

/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
Permission denied (publickey).

This means that the remote server has refused the connection.

In my case, this is because I usually set up servers such that access is via SSH key only - with password authentication disabled.

To copy a public SSH key to a remote server, password SSH login must be enabled on the remote server

To enable password login:

  • SSH into the remote server
  • Configure SSH: sudo nano /etc/ssh/sshd_config
  • Edit to allow password authentication: PasswordAuthentication yes
  • Save & exit, then reload the SSH server: sudo reload ssh

Once the key has been copied across (which will require password entry for the relevant user), test SSH access (as prompted). If it works, you can safely disable password login on the remote machine.

Limit Actions for this SSH Connection to Restricted rsync

Using an SSH key without a passphrase makes it possible to automate remote tasks - a user-entered passphrase is not required for key decryption.

Automation of backup tasks is essential - if backup relies on human intervention, sooner or later it will get missed or messed up. However, using SSH keys without a passphrase is a security risk - is someone had access to the backup server, they would be easily able to access the remote Production server.

To prevent this, SSH allows restriction of the commands that can be executed by a specific set of keys. This is accomplished by editing the /home/backupuser/.ssh/authorized_keys file for the backup user on the Production server.

A script called rrsync (which stands for restricted rsync) is provided with rsync specifically to ease the restricting keys to be used only for rsync via .ssh/authorized_keys.

On Ubuntu, the script is located here: /usr/share/doc/rsync/scripts/rrsync.gz. The script needs to be unzipped and installed under user/local/bin:

# Copy the archive rrsync script to /usr/local/bin
sudo cp /usr/share/doc/rsync/scripts/rrsync.gz  /usr/local/bin/

# Unzip the script
sudo gzip -d   /usr/local/bin/rrsync.gz

# Give it correct permissions
sudo chmod 755 /usr/local/bin/rrsync

Once rrsync is in place, we can lock down access for our SSH key by amending the /home/backupuser/.ssh/authorized_keys file.

Open the public key for the backup user on the Production Server. This will constitute a single line of text:

sudo nano /home/backupuser/.ssh/authorized_keys

Prepend the following:

command="/usr/local/bin/rrsync -ro /home/path-to/backup/",no-agent-forwarding,no-port-forwarding,no-pty,no-user-rc,no-X11-forwarding
  • The command="/usr/local/bin/rrsync ..." restricts access of that particular public key - only the given command can be executed
  • The ...-ro /home/path-to/backup/" gives read-only access to the specified directory
  • The no-* options further restrict what actions can be carried out with the public key.
  • Note that the full path is need to reference rrsync

If you try and SSH into the Production machine from the Backup server you should now see a message like this:

PTY allocation request failed on channel 0
/usr/local/bin/rrsync: Not invoked via sshd
Use 'command="/usr/local/bin/rrsync [-ro] SUBDIR"' in front of lines in /home/backupuser/.ssh/authorized_keys
Connection to 123.45.67.89 closed.

Command rsync from Backup Server

Note: After the above amendment to the authorized_keys file, the final path from the Backup servers point of view is now “/”:

Sample rsync command from the Backup server:

rsync --log-file=$HOME/.rsyncd.log --progress -az -H -e "ssh -p 1234" backupuser@123.456.789.0:/ ~/backup-target-directory

This specifies:

  • -a: archive mode; equals -rlptgoD (no -H,-A,-X)
  • -z: Compress file data during the transfer
  • -H: Maintain hardlinks (important due to our incremental backup)
  • -e “ssh -p 1234”: connect via SSH on port 1234
  • See rsync man

Resources

comments powered by Disqus