Backing up Cron and Unix Command Lines

Today I have been mostly messing with UNIX command lines. This post details the work of part of the day.

I have a cron job set up which automatically backs up the crontab. Crontab is a thing which executes schedules commands.

WinXP has a scheduler, as does OSX. In linux you can get to the file by typing 'crontab -e' (you will probably be in vi, so you may want to look up how to use vi!)

This is the line in crontab:

15 19 * * * 
crontab -l 
> ~/backup.cron/cronbackup.$(date +\%m).$(date +\%d)

This, as with other commands, is actually on one line, I have split it for clarity. That says, once a day put a copy of the cron into a file labelled cronbackup.08.24 (for today).

There will be lots of files created, but I am not that bothered as each one is small. However, I need to delete the files after some time. I need this litterbug to have an accompanying litterpicker

The brute force method is to schedule a cron job to delete the files from X months earlier, for example:

15 19 1 9 * rm ~/backup.cron/cronbackup.06*

Fairly easy to do, but it would need 12 lines in the cron, and editing it to maintain a different duration of log would be a pain.

I decided to try and be clever. Surely, thought I, it would be possible to get the machine to automagically subtract X months from the date?

The first thing was to try and do some sums on the date.

d=$(date +\%m); expr $d / 10 ; 
expr \( $d - 1 \) % 10 + 1

This outputs something like:


for August.

This one may not look much, but it is necessary in order to keep the leading zero. The next step is to bolt things back together:

d=$(date +\%m); 
a=$(expr $d / 10) ; 
b=$(expr \( $d - 1 \) % 10 + 1) ; 
echo $a$b

For August this outputs:


as required.

The next step is to start manipulating the date.

c=2 ; 
d=$(date +\%m); d=$(expr \( $d - $c \)) ; 
a=$(expr $d / 10) ;
b=$(expr \( $d - 1 \) % 10 + 1) ; 
echo $a$b

This adds two things, first the value of the correction. In this case the correction is 2, then the value of d is changed by 2. So, for august, we get 6

Unfortunately there is a bug, as demonstrated by changing c to 9. The result is 0-1, a correct result, but not desired. Ideally the result should be 11.

One solution is to put in an if statement which says that if d is less than 1, add 12. Unfortunately I have yet to get this to work on a command line.

I came up with this. The inequality yields a 1 if the result is out of range, zero if it is okay as is. This is multiplied by 12, and then added on to the number we first thought of:

c=$(expr \( $d \< 1 \) \* 12) ; 
d=$(expr \( $d + $c \));

and the expression becomes:

c=2 ; 
d=$(date +\%m); d=$(expr \( $d - $c \)); 
c=$(expr \( $d \< 1 \) \* 12) ; 
d=$(expr \( $d + $c \)); a=$(expr $d / 10) ; 
b=$(expr \( $d - 1 \) % 10 + 1) ; 
echo $a$b

Adding the timing info for crontab, my final expression is:

20 20 1 * * c=2 ; 
d=$(date +\%m); 
d=$(expr \( $d - $c \)); 
c=$(expr \( $d \< 1 \) \* 12) ; 
d=$(expr \( $d + $c \)); 
a=$(expr $d / 10) ; 
b=$(expr \( $d - 1 \) % 10 + 1) ; 
rm ~/backup.cron/cronbackup.$a$b* > /dev/null 2>&1

The final dev/null thing suppresses errors for the first few months before the backups exist, or if I delete them manually.

This solution is a bit complex, and it may have I bug which I have not seen (if it does, please comment), but I can change the backup retention time with a single change - the initial value of c.

Crontab Fun

I have a crontab behind the scenes on this site which does all sorts of things. It automatically changes the graphics on certain dates, it rebuilds the site when I've set something to 'futurepost', it grabs links from and massages them into a form where they can be included on this site. For the uninitiated, a crontab is a scheduler (winxp has a similar thing buried in the control panels).

It allows you to specify that certain tasks run at certain times.

To edit the crontab, one types crontab -e

To list it, one types crontab -l

To remove it, one types crontab -r

Note that e and r are neighbours on the keyboard. Can you guess what I did this morning? Can you?

Of course, being on Linux there was no 'are you sure?' - 'are you really, really sure?' - 'Okay, if you insist, but I'll keep a copy in the trash for you'. No, none of that. It was GONE.

Disaster, especially as I'd spent a deal of time (as some helpful people will testify) putting all sorts of cleverness in there.

Fortunately, I am in the habit of typing crontab -l prior to any editing, and was able to reconstruct it (I hope I got it all!)

I now have a new line in my crontab: 0 19 * * * crontab -l > ~/backup.cron/cronbackup

This takes the output of crontab -l and writes it to a file once a day.

Why didn't I do that before? Always the way with backups.....

Backing up MySQL

The contents of this post have been superceded

Using Movable type, the site is based on a MySQL database, and it's important to back this up.

Therefore I've put the following commands into crontab - I think they're okay (if anyone spots errors, please let me know!)

The first line will dump the sql database into a file at 1:08am. This time was arbitrarily chosen

The remaining lines make daily copies of the latest backup, about an hour before making the new latest backup.

In this fashion there will always be a backup every day for the last week. I'll also make weekly backups for the last month and a monthly backup for the past year using the same principle. I've shown the line which will create the monthly backup.

Of course, you should replace anything (in brackets) with appropriate values - localhost may not be right for you either. Passwords are stored in ~/.my.cnf - I will store more variables here and quietly change this entry to make the command better.

Modified thanks to Aquarion and links to manuals - some mods likely to occur in the future. The $(date +\%B) would be $(date +%B) on the command line, the backslash is so that cron doesn't treat % as a new line.

8 1 * * * mysqldump -h localhost -u (username) (database name) | gzip -c > ~/(backup directory)/latest/backup-mt.sql.gz
50 23 * * * cp ~/(backup directory)/latest/backup-mt.sql.gz ~/(backup directory)/daily/$(date +\%A).sql.gz
55 23 28 * * cp ~/(backup directory)/latest/backup-mt.sql.gz ~/(backup directory)/monthly/$(date +\%B).sql.gz

Contents of ~/.my.cnf:

password = "(password)"

The size of the backup directory should be limited, as old backups are overwritten, and I can't forget to make a backup. In principle(!) I could lose 24 hours worth of updates at most.

I should also look into smiling sweetly at someone I trust who would be willing to let me ftp the latest backup to them each day. As I trust nobody that much, and don't quite know if anything sensitive is visible in the SQLdump, I'd have to work out how to encrypt it first.