Category Archives: Linux

General Discussion threads regarding Linux/UNIX

rsync script using ssh

This is my quick and dirty How To rsync script using SSH!

I have created the below script which will use rsync to backup a single local directory to a single directory on a remote server. If you have not used rsync then I need to point out that this script will remove any files on the remote server if they are not located on the local server. This means that if you delete or move a file from the local directory rsync will do the same for the remote backup directory. rsync has some very cool features and can be setup for a variety of different backup needs.

In addition I have created a log file of the rsync process. If there is an error you will be able to diagnose the problem by reviewing the log. I have also programmed the rsycn script to remove the log file after 7 days, you can easily increase or decrease the number of days required.

Please also note that the below rsycn script assumes that you have already installed your public and private keys on the local and remote computers/servers. If you do not know how to do this then check out my How To for: Secure Copy – without a password (pass on password with SCP).

#!/bin/bash
##########################################
# CREATED BY BRENDAN SKOREYKO            #
# WEB: www.skrakes.com                   #
# PURPOSE: RSYNC SINGLE DIRECTORY        #
# FILENAME: rsync-remote.bsh             #
##########################################

TIMESTAMP=`date +%Y%m%d_%H%M`
# YYYYMMDD_HHMM (TIMESTAMP)
DELDATE=`date +%Y%m%d --date="7 days ago"`
# YYYYMMDD (DELDATE)

LOGDIR=/home/<user-name>/log
LOG=/home/<user-name>/log/${TIMESTAMP}_rsync_htdocs.log

REMOTE_DEST_HOST=<myserver.com>
## PLACE THE DNS OR IP ADDRESS OF THE REMOTE BACKUP SERVER HERE
REMOTE_DEST_PORT=<port-number, if applicable>
## I CHANGED MY DEFAULT SSH PORT FOR SECURITY REASONS, AT LEAST ON VERY SENSITIVE SERVERS. I ALSO DISABLE ROOT LOGIN!!!
REMOTE_DEST_USR=<user-name>
## REMOTE USER NAME

# SOURCE DIRECTORIE ON LOCAL SERVER
SRC_DIR="/home/<user-name>/Music"

# DESTINATION DIRECTORIE ON REMOTE SERVER
DEST_DIR="/backup/<user-name>/"

# CREATE FUNCTION TO REMOVE LOG FILES OLDER THAN X NUMBER OF DAYS
function RemoveLogs
{
for i in $( ls $LOGDIR ); do
fname=`echo $i | grep -E '^[0-9]{8}\_[0-9]{4}\_rsync\.log$' | cut -d'_' -f1`
if [[ $fname != '' && $fname < $DELDATE ]]; then
echo REMOVE $i >> $LOG
rm -rfv $LOGDIR/$i >> $LOG
else
echo DO NOT REMOVE $i >> $LOG
fi
done
}

# BEGIN RSYNC
echo -e "Starting rsync to $REMOTE_DEST_HOST...\n" >> $LOG 2>&1
# RECURSIVE RSYNC
rsync -varz -e "ssh -p $REMOTE_DEST_PORT" --delete $SRC_DIR [email protected]$REMOTE_DEST_HOST:$DEST_DIR >> $LOG 2>&1
# END RSYNC
echo -e "\nEnding rsync to $REMOTE_DEST_HOST..." >> $LOG 2>&1
# CALL REMOVELOGS FUNCTION AND CLEANUP OLD LOG FILES
RemoveLogs

If you have any questions, comments or concerns please post them! I would like to point out that this is a “quick and dirty rsync script to backup a single directory to a remote server.”

Parallels Plesk Migration 9 to 10

So I had some fun trying to migrate from Parallels Plesk 9.5.x to 10.0.x (UNIX based system). It took me some time but after finding the documentation the process was actually very smooth, however unlike other new releases I found this documentation much harder to find.

Anyhow I had to move my Parallels Plesk 9.5.x VDS (Virtual Dedicated Server) to my new Parallels Plesk 10.0.x VDS (Virtual Dedicated Server). I bought my VDS systems through Go Daddy and renew yearly. Go Daddy does not allow root access to their VDS systems so doing a migration from the Parallels Plesk Control Panel was not an option – although Go Daddy has a KB article which allows you to modify the SUDO (super users list – UNIX based systems) which didn’t work for me.

Thanks to GOOGLE and after several hours I found some documentation which instructs a user to convert a Plesk 9.xdata backup to Plesk 10.0.x compatible backup: Parallels Plesk KB Article. Once I read this article the process went very smooth. The reason I had problems until stumbling across this article was that there is/was no documentation at the time telling me I could not restore a Plesk 9.x backup to a Plesk 10.x backup.

Hard Drive Diagnostics in Linux

So this morning I had to deal with a server which has been consistently crashing once every week. Each crash indicated that there is something wrong with the file system – no I am not a programmer and nor am I about to spend a ton of time looking into the crash, for reasons I don’t want to get into here.

Anyway the OS that is being used is Fedora Core 2, which is no longer supported.  Now even though I think the OS is the problem I am going to scan the hard drive to make sure its working properly, if the drive isn’t function properly then this could be the root cause of the problem. So there are two great options for testing the hard drive. The first is SMART, luckily the hard drive is SMART capable. The second is badblocks, a program that is called E/2/3/4/fsprogs (you can download the program from sourceforge if it is not installed).

If you do not know what type of hard drive you have in your system you can run the following command:

fdisk -l

Because the hard drive in this server is SMART compatible I can run the following command which will output drive information for a particular drive: (PLEASE NOTE THAT THIS EXAMPLE WAS PERFORMED ON A RED HAT BASED SYSTEM)

smartctl -a /dev/HDD_DEVICE

This command will run a LONG test of your hard drive:

smartctl -t long /dev/HDD_DEVICE

Now to view the status of your test run this command(please note that the time it takes to test your hard drive will very depending on the capacity):

smartctl -l selftest /dev/HDD_DEVICE

If you don’t have a SMART capable hard drive you can use the E/2/3/4/fsprogs to check your hard drive. By running the following command you will initiate a scan of the hard drive:

badblocks -n -v /dev/HDD_DEVICE

Here is what I would run if I were you:

badblocks -n /dev/HDD_DEVICE -o outputfile &

This will output any bad blocks to the outputfile (you name it whatever you want) and the “&” runs the program in the background. For more information on E/2/3/4/fsprogs click the link!

Alternatively you can also use GOOGLE to help solve your problems! I also used this wonderful site to help me with hardware diagnostics.

How To: Bash Backup Script

Alright so here is my simple version of a bash backup script… Feel free to comment, if I miss something or you think something should be added please let me know. For this example I assume that you have the proper user access and have some minimal command line experience!

First you should know that I am using several basic commands like “cp” to copy files and “tar” to compress files. You will also need to know how to use command line text editors, I am using “emacs” but “vi,”  “view” or whatever text editor you use will work.

Any line beginning with the number sign # and has text colored in green indicates a comment. Please note that I do not take responsibility for any issues you may cause when using my backup method, I do recommend testing the backup method on a directory that does not have important contents (so create a test directory with test files and try backing it up first). However I highly doubt the method will cause problems unless you severely screw up on my instructions…

[ad#google-468×60]

1) The first thing we need to do is make a backup directory on the Linux box:

## Change to the / (root) directory (again I am assuming you have proper access to your system - as root)
cd /

## make your backup directory and create a folder for your backups
mkdir -p /backup/my-backup

2) Now we need to create the script:

# Change to the home directory (remember you are root so you may have to change permissions later)
cd ~

## now lets create the bash file for our backup script using emacs
emacs my-backup.bsh

## at the start/top of the file please add the following, note the number sign which looks like a comment, this allows your system check the file and run it as bash script
#!/bin/bash

## save the bash file and change its permissions so that the script can be executed
chmod +x my-backup.bsh

## now go back into the backup bash file and start the code
emacs my-backup.bsh

3) Now we write the script:

#!/bin/bash
# This is my backup file - created by <your-name-here>

# here I am setting a time stamp variable which I like to use for logging
TIMESTAMP=`date +%Y%m%d.%H%M`

# here I am setting up the backup directory as a variable
DEST_DIR="/backup/my-backup"

# here I am setting up the directory in which I want to backup, again another variable
SRC_DIR="/home/<user-name-here>/Documents"

# let's create a variable for the backup file name file
FNAME="MyBackup"

# let's create a variable for the log file, let's also name the log file with the filename and timestamp it
LOG="/home/<user-name-here>/log/$FNAME-$TIMESTAMP.log"

# start the backup, create a log file which will record any messages run by this script
echo -e "Starting backup of <user-name-here> $SRC_DIR directory" >> ${LOG}

# compress the directory and files, direct the tar.gz file to your destination directory
tar -vczf ${DEST_DIR}/${FNAME}-${TIMESTAMP}.tar.gz ${SRC_DIR} >> ${LOG}

# end the backup, append to log file created by this script
echo -e "Ending backup of <user-name-here> $SRC_DIR" >> ${LOG}

The backup is now complete, we have created a compressed copy of the /home/<user-name>/Documents directory in the /backup/my-backup directory. If there were any errors with backup process the log file we created in /home/<user-name>/log will tell us what the error may be. Normally you don’t have to create variables to do a backup but if you wanted to backup another directory all you need to do is duplicate this script and change the SOURCE DIRECTORY, DESTINATION DIRECTORY, LOG-FILE (location) and FILE-NAME to the additional backup directory.

Advanced Settings: This backup script is a great way to backup your files but you should really consider moving your files off the same drive (computer) to an alternate location. If you do not have an off-site location perhaps you have another local Linux box or NAS device which will allow you to preform backups via scp (secure copy) or rsync. Finally the last thing you should do is set this script to run on a regular basis. To do this you need to create a cronjob using your systems crontab.

See my other How To’s:

[ad#google-468×60]

mutt command line attachment

Yesterday I was working on sending email with mutt from the command line. Today I want to show you how to send an attachment with mutt at the command line. Here is how to send an attachment using mutt from the command line:

# mutt command line attachment, using the switch -a
echo "message here" | mutt -s "subject" [email protected] -a /path-to-attachment

Now I attach my files a little differently. In most of my scripts I create a variable for logging, with mutt I can do two things, output the log directly to the message of the email and attach it as well. To send the contents of a file within the message do the following:

# mutt command line attachment, using the switch -a and cat command
cat /path-to-file.txt | mutt -s "subject" [email protected] -a /path-to-attachment

My code usually looks like the following (where “$LOGFILE” is predefined earlier in my script):

cat $LOGFILE | mutt -s "subject" [email protected] -a $LOGFILE

Your message body no contains the contents of the “path-to-file.txt” and has the file attached, a little redundant but some mobile phones have issues with different file extensions.