backup

This is my personal backup tool on top of Rsync.

What it does for me:

• Configure different rsync calls in a human readable fashion through the config.ini file.
• Maintain the base parameters I wish to include for all rsync calls in a central place (the rsync section).
• It let's me define config blocks for each rsync call (module), in which I supply the local path and specific parameters. The path can make use of a shared base path you can set in the rsync config block.
• Using the transfer direction option, I can configure per module whether I wish to pull from the host, or push to it.
• It can easily be configured to log to file, but I have it configured to log to stdout when run manually, and to journald otherwise (see Scheduling with journal logging).
  • To make it log to file, simply provide a base option --log-file <FILEPATH> in the rsync section.
• When backing up manually, it allows intuitive usage by simply stating the module name(s) you wish to back up, and that's it.
• Transparant, readable code, as opposed to shell scripts which quickly turn ugly and less robust.

Installation

Currently, I haven't released this on PyPi and I don't know if I'm going to.

Install

For now, I just install directly from the source code:

# pip3 install git+https://github.com/bartkl/backup.git#egg=Backup

Uninstall

To uninstall, simply use pip again:

# pip3 uninstall backup

Configuration

The script will look for a config.ini file as follows:

• If the environment variable $BACKUP_CONFIG_DIR is defined, it looks there.
• Otherwise, it looks for that file in ~/.config/backup.

The config file contains:

• A mandatory rsync section. Here global Rsync related options can be defined.

  Fields:

  • host: The Rsync daemon host to backup to (Required).
  • base opts: Whitespace separated string of Rsync options that will be used for all modules.

• One or more module sections. These are module-specific config blocks to configure how to call rsync for a certain module. These sections have their names prefixed, like module: books.

  Fields:

  • path: The path of the source files to be synced (Required).
  • transfer direction: Either push or pull. Determines whether files for this module are sent to or received from the configured host (Required).
  • opts: Whitespace separated string of Rsync options that will be used (on top of the base options above) for this module specifically.

You can define your own fields for re-use later in the file as well. This is particularly useful when repeating something often, like a base path.

See the example below which demonstrates what's been layed out here. It's a valid config.

Example

[rsync]
host = me@host
password file = /home/me/.config/backup/rsync_password
source base path = /media/
base opts =
    --itemize-changes
    --verbose
    --archive
    --update
    --delete
    --password-file=${password file}

[module: stuff]
transfer direction = pull
path = ${rsync:source base path}/data/stuff
opts = --partial --inplace

[module: links]
transfer direction = push
path = ${rsync:source base path}/data/.torrents
opts = --whole-file --no-links

Scheduling with journal logging

Use systemd-cat to take in the output from the script and journal it. I myself have it scheduled (somewhat) like so:

# m h  dom mon dow   command
03  04   *   *   *   /usr/local/bin/backup books libraries 2>&1 | systemd-cat -t backup