In part 1, we learned how to make simple automated unencrypted and encrypted backups. In this article, I will show you how to fine-tune your file selection, and how to backup your encryption keys.
rsync Include and Exclude Files
rsync supports all kinds of complex ways to build lists of files that you want to copy, and lists of files that you want to exclude from copying. man rsync details five ways to select files:
--exclude=PATTERN exclude files matching PATTERN --exclude-from=FILE read exclude patterns from FILE --include=PATTERN don't exclude files matching PATTERN --include-from=FILE read include patterns from FILE --files-from=FILE read list of source-file names from FILE
Include rules copy everything by default. It seems they should exclude everything by default and copy only the files that you list, but it doesn’t work that way, and you have to use them in combination with exclude rules. Exclude rules exclude nothing by default, but exclude only the files you list. Include rules drive me nuts, so I don’t use them.
The two simplest methods use the –files-from and –exclude-from options. Put your list of files in a text file and then call this file in your backup command. Use the –files-from option when your list of files to copy is smaller than the number of files you don’t want to copy. –files-from does not support pattern matching; it is just a plain list.
Use –exclude-from when your exclude list is shorter than your include list. This supports pattern matching, so you can use regular expressions.
This example include file lists subdirectories and one file from the 1mybooks directory, and the entire blog directory. Filepaths are relative to the root directory, which is ~/Documents/:
1mybooks/newbook/ 1mybooks/oldbook/ 1mybooks/hacks.pdf blog/
This example backup command use the -a (archive) option, which preserves your file metadata, including permissions, file ownerships, and timestamps. -r (recursive) is normally included in the -a option, but the –files-from option does not recurse. -v adds verbosity.
$ rsync -arv --files-from=include.txt ~/Documents/ carla@backup:/home/carla/backupfiles
You have to specify the target directory, and trailing slashes have no effect.
–exclude-from supports pattern matching. In this example, logs/2015/* will not copy any subdirectories or files after 2015/. sketchbook/sketch* will not copy any files that start with “sketch”. .* means do not copy dotfiles. games/ and Videos/ are completely excluded:
.* games/ downloads/ logs/2015/* sketchbook/sketch* Videos/
Use it like just like the include example:
$ rsync -arv --exclude-from=exclude.txt ~/Documents carla@backup:/home/carla/backupfiles
Now you must mind your trailing slashes, as we learned in part 1. A trailing slash on the source directory copies only the contents, and omitting it copies the directory and contents. It makes no difference on the target directory.
duplicity File Selection
duplicity supports similar file selection conventions to rsync, with includes and excludes and pattern matching. The simplest backup command names a single directory, as we learned in part 1. When you want to backup more than one file or directory, it gets more complicated. No, sorry, you can’t just make a normal plain file list. This example excludes two subdirectories in the ~/foo directory, and by default includes all the others:
duplicity --encrypt-key 088D5F09 --exclude ~/foo/dir1 --exclude ~/foo/dir2 ~/foo scp://carla@backupserver/somefiles
To include files, list the files you want and then exclude the root directory of your backup:
duplicity --encrypt-key 088D5F09 --include ~/foo/dir3 --include ~/foo/filename --exclude '**' ~/foo scp://carla@backupserver/somefiles
Note that duplicity defaults to not allowing multiple different backups to the same backup directory. You can override this with the –allow-source-mismatch option, although I don’t recommend it. It’s cheap and easy to give each backup archive its own directory.
You can put your file list in a file and then call it with the –include-filelist or –exclude-filelist option. This example includes files to backup and excludes all the others in your root backup directory:
+ /home/carla/dir1/ + /home/carla/dir2 - **
Call your include file like this example:
$ duplicity --encrypt-key 088D5F09 --include-filelist filelist.txt /home/carla/ scp://carla@backupserver/somefiles
This example lists files to exclude:
- /home/carla/dir1 - /home/carla/dir2 - /home/carla/filename
You don’t need to tell it to include all the other files because that is the default. Then, run your backup command:
$ duplicity --encrypt-key 088D5F09 --exclude-filelist filelist.txt /home/carla/ scp://carla@backupserver/somefiles
duplicity has eleventy-fourteen options. Please study man duplicity to learn about full and incremental backups, pattern matching for uber-nerdy file selection, and the correct syntax for FTP and local backups. Yes, it’s a bit of a rough read, but that is the authoritative source, and believe me it is faster in the long run than wading through web searches.
Backup Keys
The quickest method is to copy your .ssh and .gnupg directories to a safe location. I recommend using an SD card rather than a USB key — because SD cards are more reliable — and locking it somewhere safe. You could also store these on a safe cloud service such as SpiderOak. Then to restore them, you can just copy them back into place. There are several other cool nerdy ways to back up encryption keys, which I’ll discuss in a future article.
Advance your career in Linux System Administration! Check out the Essentials of System Administration course from The Linux Foundation.
