rsync script using ssh

This is my quick and dirty How To rsync script using SSH!

I have created the below script which will use rsync to backup a single local directory to a single directory on a remote server. If you have not used rsync then I need to point out that this script will remove any files on the remote server if they are not located on the local server. This means that if you delete or move a file from the local directory rsync will do the same for the remote backup directory. rsync has some very cool features and can be setup for a variety of different backup needs.

In addition I have created a log file of the rsync process. If there is an error you will be able to diagnose the problem by reviewing the log. I have also programmed the rsycn script to remove the log file after 7 days, you can easily increase or decrease the number of days required.

Please also note that the below rsycn script assumes that you have already installed your public and private keys on the local and remote computers/servers. If you do not know how to do this then check out my How To for: Secure Copy – without a password (pass on password with SCP).

#!/bin/bash
##########################################
# CREATED BY BRENDAN SKOREYKO            #
# WEB: www.skrakes.com                   #
# PURPOSE: RSYNC SINGLE DIRECTORY        #
# FILENAME: rsync-remote.bsh             #
##########################################

TIMESTAMP=`date +%Y%m%d_%H%M`
# YYYYMMDD_HHMM (TIMESTAMP)
DELDATE=`date +%Y%m%d --date="7 days ago"`
# YYYYMMDD (DELDATE)

LOGDIR=/home/<user-name>/log
LOG=/home/<user-name>/log/${TIMESTAMP}_rsync_htdocs.log

REMOTE_DEST_HOST=<myserver.com>
## PLACE THE DNS OR IP ADDRESS OF THE REMOTE BACKUP SERVER HERE
REMOTE_DEST_PORT=<port-number, if applicable>
## I CHANGED MY DEFAULT SSH PORT FOR SECURITY REASONS, AT LEAST ON VERY SENSITIVE SERVERS. I ALSO DISABLE ROOT LOGIN!!!
REMOTE_DEST_USR=<user-name>
## REMOTE USER NAME

# SOURCE DIRECTORIE ON LOCAL SERVER
SRC_DIR="/home/<user-name>/Music"

# DESTINATION DIRECTORIE ON REMOTE SERVER
DEST_DIR="/backup/<user-name>/"

# CREATE FUNCTION TO REMOVE LOG FILES OLDER THAN X NUMBER OF DAYS
function RemoveLogs
{
for i in $( ls $LOGDIR ); do
fname=`echo $i | grep -E '^[0-9]{8}\_[0-9]{4}\_rsync\.log$' | cut -d'_' -f1`
if [[ $fname != '' && $fname < $DELDATE ]]; then
echo REMOVE $i >> $LOG
rm -rfv $LOGDIR/$i >> $LOG
else
echo DO NOT REMOVE $i >> $LOG
fi
done
}

# BEGIN RSYNC
echo -e "Starting rsync to $REMOTE_DEST_HOST...\n" >> $LOG 2>&1
# RECURSIVE RSYNC
rsync -varz -e "ssh -p $REMOTE_DEST_PORT" --delete $SRC_DIR $REMOTE_DEST_USR@$REMOTE_DEST_HOST:$DEST_DIR >> $LOG 2>&1
# END RSYNC
echo -e "\nEnding rsync to $REMOTE_DEST_HOST..." >> $LOG 2>&1
# CALL REMOVELOGS FUNCTION AND CLEANUP OLD LOG FILES
RemoveLogs

If you have any questions, comments or concerns please post them! I would like to point out that this is a “quick and dirty rsync script to backup a single directory to a remote server.”

3 thoughts on “rsync script using ssh

  1. Carsten

    Hi Brendan,

    can you tell me why the log files contain these lines
    ++++

    Ending rsync to …
    DO NOT REMOVE 20120904_1002_rsync_htdocs.log
    DO NOT REMOVE 20120904_1008_rsync_htdocs.log

    ++++
    dont understand why. Thanks

  2. Brendan Skoreyko

    Hi Carsten,

    The script I created is logging the files which are not to be removed and those that will be removed based on the deletion date. You can change this of course but I simply pre-appended the “DO NOT REMOVE” for logging purposes. If something were to go wrong with the script then I can review the log and see where something was or was not removed and this will help me trouble shoot the issue.

    Does that make sense?

    Thank you,
    Brendan

  3. Carsten

    Hi,

    yes that makes sense. Insert a simple line of text to distinguish between operations.

    I now use my rsync script to copy files from SD Cards for a timelapse project. copy first, rename and then delete the sd card.
    I found rsync more “verbose” than “cp”.

    thanks and have a good weekend.
    carsten

Leave a Reply

Your email address will not be published. Required fields are marked *

*