Backup

Backup is a tedious thing. If you don’t do it regularly, it will come and bite you in the ass. If you do it all the time, nothing happens (mostly). That’s probably why there are far more stories of situations where a backup could have save something, but didn’t when it wasn’t followed through.

So far I didn’t come in a situation, where I’ve lost any data. I still remember that there was one day, where I though that happened, but I was wrong. I just assumed I deleted the wrong directory. That was the day where I started doing backups.

It’s a boring task if you do it on a small scale at home. Easy to forget. One goal for the new year of 2019 is to be better with this and get it done more often.

Automation is the key here (as with so much other).

Since I write myself writing the same commands on my hosts over and over again, it’s time to centralize this and to make some decisions.

In my private environment I have to handle three types of data:

1. Private data of all kind

2. Music

3. Movies

This is also the order of priorities for my data. Luckily, the size of the backup is the opposite.

The private data I’ll update once every month.

The music I’ll backup once every three months.

The movies I’ll backup once every six months.

I use duplicity for doing this task for me.

Based on the time period it will create a new full backup if required and otherwise perform an incremental backup. This short code example should be enough to explain how it works:

$ timestamp=$(date +%Y%m%d-%H%M%S)
$ duplicity \
    incremental \
    --full-if-older-than 1M \
    ./backup/logs/$(timestamp).backup.log \
    --no-encryption \
    ${backup_source} file:///mnt/usb/backup/${timestamp}"

I should be mentioned, that when using the parameter –exlude <pattern>, the pattern must be part of the source ($(backup_source)). So when taking a backup of some directory /path/a and you are lazy and want to script the exlude pattern in a variable only once (like –exclude /path/a/no_backup –exclude /path/b): this will fail, since /path/b is not path of the source. The error message is not very clear about this (some python gibberish on the first glance) and will not give you a hint of the cause.