Scheduled Rsync
Running rsync via cron has been around a long time. Ideally, you use public keys and limit the account. You do it something like this.
- On the source
- Configure SSHD to handle user keys
- Create a control script to restrict users to rsync
- Add an account specific to backups
- Generate user keys and limit to the control script
- On the destination
- Copy the private key
- Create a script and cronjob
Source
# Add a central location for keys and have sshd look there. Notice the
# '%u'. It's substituted with user ID at login to match the correct filename
sudo mkdir /etc/ssh/authorized_keys
echo "AuthorizedKeysFile /etc/ssh/authorized_keys/%u.pub" > /etc/ssh/sshd_config.d/authorized_users.conf
systemctl restart ssh.service
# Create the script logic that makes sure it's an rsync command. You can modify this to allow other cmds as needed.
sudo tee /etc/ssh/authorized_keys/checkssh.sh << "EOF"
#!/bin/bash
if [ -n "$SSH_ORIGINAL_COMMAND" ]; then
if [[ "$SSH_ORIGINAL_COMMAND" =~ ^rsync\ ]]; then
echo $SSH_ORIGINAL_COMMAND | systemd-cat -t rsync
exec $SSH_ORIGINAL_COMMAND
else
echo DENIED $SSH_ORIGINAL_COMMAND | systemd-cat -t rsync
fi
fi
EOF
chmod +x /etc/ssh/authorized_keys/checkssh.sh
# Add the user account and create keys for them
THE_USER="backup-account-1"
sudo adduser --no-create-home --home /nonexistent --disabled-password --gecos "" ${THE_USER}
ssh-keygen -f /etc/ssh/authorized_keys/${THE_USER} -q -N "" -C "${THE_USER}"
# Add the key stipulations that invoke the script and limit ssh options.
# command="/etc/ssh/authorized_keys/checkssh.sh\",no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty
THE_COMMAND="\
command=\
\"/etc/ssh/authorized_keys/checkssh.sh\",\
no-port-forwarding,\
no-X11-forwarding,\
no-agent-forwarding,\
no-pty "
# Insert the command in front of the user's key - the whole file remains a single line
sed -i "1s|^|$THE_COMMAND|" /etc/ssh/authorized_keys/${THE_USER}.pub
# Finally, copy the account's private key to the remote location
scp /etc/ssh/authorized_keys/${THE_USER} [email protected]:
Destination
It’s usually best to create a script that uses rsync and call that from cron. Preferably one that doesn’t step on itself for long running syncs. Like this:
vi ~/schedule-rsync
#!/bin/bash
THE_USER="backup-account-1"
THE_KEY="~/backup-account-1" # If you move the key, make sure to adjust this
SCRIPT_NAME=$(basename "$0")
PIDOF=$(pidof -x $SCRIPT_NAME)
for PID in $PIDOF; do
if [ $PID != $$ ]; then
echo "[$(date)] : $SCRIPT_NAME : Process is already running with PID $PID"
exit 1
fi
done
rsync \
--archive \
--bwlimit=5m \
--delete \
--delete-excluded \
--exclude .DS* \
--exclude ._* \
--human-readable \
--inplace \
--itemize-changes \
--no-group \
--no-owner \
--no-perms \
--progress \
--recursive \
--rsh "ssh -i ${THE_KEY}" \
--verbose \
--stats \
${THE_USER}@some.server.org\
:/mnt/pool01/folder.1 \
:/mnt/pool01/folder.2 \
:/mnt/pool01/folder.2 \
/mnt/pool02/
Then, call it from a file in the cron drop folder.
echo "0 1 * * * /home/$USER/schedule-rsync >> /home/$USER/rsync-video.log 2>&1" > /etc/cron.d/schedule-rsync
Notes
Why not use rrsync?
The rrsync script is similar to the script we use, but is distributed and maintained as part of the rsync package. It’s arguably a better choice. I like the checkssh.sh approach as it’s more flexible, allows for things other than rsync, and doesn’t force relative paths. But if you’re only doing rsync, consider using rrsync like this;
THE_COMMAND="\
command=\
\"rrsync -ro /mnt/pool01\",\
no-port-forwarding,\
no-X11-forwarding,\
no-agent-forwarding,\
no-pty "
In your client’s rsync command, make the paths relative to path rrsync expects above.
rsync \
...
...
${THE_USER}@some.server.org\
:folder.1 \
:folder.2 \
:folder.2 \
/mnt/pool02/
If you see the client-side error message:
rrsync error: option -L has been disabled on this server
You discovered that following symlinks has been disabled by default in rrsync. You can enable with an edit to the script.
sudo sed -i 's/KLk//' /usr/bin/rrsync
# This changes
# short_disabled_subdir = 'KLk'
to
# short_disabled_subdir = ''
Troubleshooting
Sources
https://peterbabic.dev/blog/transfer-files-between-servers-using-rrsync/ http://gergap.de/restrict-ssh-to-rsync.html https://superuser.com/questions/641275/make-linux-server-allow-rsync-scp-sftp-but-not-a-terminal-login
Feedback
Was this page helpful?
Glad to hear it! Please tell us how we can improve.
Sorry to hear that. Please tell us how we can improve.