This is my quick and dirty How To rsync script using SSH!
I have created the below script which will use rsync to backup a single local directory to a single directory on a remote server. If you have not used rsync then I need to point out that this script will remove any files on the remote server if they are not located on the local server. This means that if you delete or move a file from the local directory rsync will do the same for the remote backup directory. rsync has some very cool features and can be setup for a variety of different backup needs.
In addition I have created a log file of the rsync process. If there is an error you will be able to diagnose the problem by reviewing the log. I have also programmed the rsycn script to remove the log file after 7 days, you can easily increase or decrease the number of days required.
Please also note that the below rsycn script assumes that you have already installed your public and private keys on the local and remote computers/servers. If you do not know how to do this then check out my How To for: Secure Copy – without a password (pass on password with SCP).
#!/bin/bash ########################################## # CREATED BY BRENDAN SKOREYKO # # WEB: www.skrakes.com # # PURPOSE: RSYNC SINGLE DIRECTORY # # FILENAME: rsync-remote.bsh # ########################################## TIMESTAMP=`date +%Y%m%d_%H%M` # YYYYMMDD_HHMM (TIMESTAMP) DELDATE=`date +%Y%m%d --date="7 days ago"` # YYYYMMDD (DELDATE) LOGDIR=/home/<user-name>/log LOG=/home/<user-name>/log/${TIMESTAMP}_rsync_htdocs.log REMOTE_DEST_HOST=<myserver.com> ## PLACE THE DNS OR IP ADDRESS OF THE REMOTE BACKUP SERVER HERE REMOTE_DEST_PORT=<port-number, if applicable> ## I CHANGED MY DEFAULT SSH PORT FOR SECURITY REASONS, AT LEAST ON VERY SENSITIVE SERVERS. I ALSO DISABLE ROOT LOGIN!!! REMOTE_DEST_USR=<user-name> ## REMOTE USER NAME # SOURCE DIRECTORIE ON LOCAL SERVER SRC_DIR="/home/<user-name>/Music" # DESTINATION DIRECTORIE ON REMOTE SERVER DEST_DIR="/backup/<user-name>/" # CREATE FUNCTION TO REMOVE LOG FILES OLDER THAN X NUMBER OF DAYS function RemoveLogs { for i in $( ls $LOGDIR ); do fname=`echo $i | grep -E '^[0-9]{8}\_[0-9]{4}\_rsync\.log$' | cut -d'_' -f1` if [[ $fname != '' && $fname < $DELDATE ]]; then echo REMOVE $i >> $LOG rm -rfv $LOGDIR/$i >> $LOG else echo DO NOT REMOVE $i >> $LOG fi done } # BEGIN RSYNC echo -e "Starting rsync to $REMOTE_DEST_HOST...\n" >> $LOG 2>&1 # RECURSIVE RSYNC rsync -varz -e "ssh -p $REMOTE_DEST_PORT" --delete $SRC_DIR [email protected]$REMOTE_DEST_HOST:$DEST_DIR >> $LOG 2>&1 # END RSYNC echo -e "\nEnding rsync to $REMOTE_DEST_HOST..." >> $LOG 2>&1 # CALL REMOVELOGS FUNCTION AND CLEANUP OLD LOG FILES RemoveLogs
If you have any questions, comments or concerns please post them! I would like to point out that this is a “quick and dirty rsync script to backup a single directory to a remote server.”
Hi Brendan,
can you tell me why the log files contain these lines
++++
Ending rsync to …
DO NOT REMOVE 20120904_1002_rsync_htdocs.log
DO NOT REMOVE 20120904_1008_rsync_htdocs.log
++++
dont understand why. Thanks
Hi Carsten,
The script I created is logging the files which are not to be removed and those that will be removed based on the deletion date. You can change this of course but I simply pre-appended the “DO NOT REMOVE” for logging purposes. If something were to go wrong with the script then I can review the log and see where something was or was not removed and this will help me trouble shoot the issue.
Does that make sense?
Thank you,
Brendan
Hi,
yes that makes sense. Insert a simple line of text to distinguish between operations.
I now use my rsync script to copy files from SD Cards for a timelapse project. copy first, rename and then delete the sd card.
I found rsync more “verbose” than “cp”.
thanks and have a good weekend.
carsten