|
@@ -54,338 +54,15 @@ href="https://asciinema.org/a/203761" target="_blank">screencast</a>.
|
|
<script src="https://asciinema.org/a/203761.js" id="asciicast-203761" async></script>
|
|
<script src="https://asciinema.org/a/203761.js" id="asciicast-203761" async></script>
|
|
|
|
|
|
|
|
|
|
-## Installation
|
|
|
|
|
|
+## How-to guides
|
|
|
|
|
|
-To get up and running, first [install
|
|
|
|
-Borg](https://borgbackup.readthedocs.io/en/latest/installation.html), at
|
|
|
|
-least version 1.1.
|
|
|
|
-
|
|
|
|
-Then, run the following command to download and install borgmatic:
|
|
|
|
-
|
|
|
|
-```bash
|
|
|
|
-sudo pip3 install --upgrade borgmatic
|
|
|
|
-```
|
|
|
|
-
|
|
|
|
-Note that your pip binary may have a different name than "pip3". Make sure
|
|
|
|
-you're using Python 3, as borgmatic does not support Python 2.
|
|
|
|
-
|
|
|
|
-
|
|
|
|
-### Other ways to install
|
|
|
|
-
|
|
|
|
- * [Docker image](https://hub.docker.com/r/monachus/borgmatic/)
|
|
|
|
- * [Another Docker image](https://hub.docker.com/r/b3vis/borgmatic/)
|
|
|
|
- * [Debian](https://tracker.debian.org/pkg/borgmatic)
|
|
|
|
- * [Ubuntu](https://launchpad.net/ubuntu/+source/borgmatic)
|
|
|
|
- * [Fedora](https://bodhi.fedoraproject.org/updates/?search=borgmatic)
|
|
|
|
- * [Arch Linux](https://aur.archlinux.org/packages/borgmatic/)
|
|
|
|
- * [OpenBSD](http://ports.su/sysutils/borgmatic)
|
|
|
|
- * [openSUSE](https://software.opensuse.org/package/borgmatic)
|
|
|
|
-
|
|
|
|
-<br><br>
|
|
|
|
-
|
|
|
|
-
|
|
|
|
-## Configuration
|
|
|
|
-
|
|
|
|
-After you install borgmatic, generate a sample configuration file:
|
|
|
|
-
|
|
|
|
-```bash
|
|
|
|
-sudo generate-borgmatic-config
|
|
|
|
-```
|
|
|
|
-
|
|
|
|
-If that command is not found, then it may be installed in a location that's
|
|
|
|
-not in your system `PATH`. Try looking in `/usr/local/bin/`.
|
|
|
|
-
|
|
|
|
-This generates a sample configuration file at /etc/borgmatic/config.yaml (by
|
|
|
|
-default). You should edit the file to suit your needs, as the values are
|
|
|
|
-representative. All fields are optional except where indicated, so feel free
|
|
|
|
-to ignore anything you don't need.
|
|
|
|
-
|
|
|
|
-You can also have a look at the [full configuration
|
|
|
|
-schema](https://projects.torsion.org/witten/borgmatic/src/master/borgmatic/config/schema.yaml)
|
|
|
|
-for the authoritative set of all configuration options. This is handy if
|
|
|
|
-borgmatic has added new options since you originally created your
|
|
|
|
-configuration file.
|
|
|
|
-
|
|
|
|
-
|
|
|
|
-### Encryption
|
|
|
|
-
|
|
|
|
-Note that if you plan to run borgmatic on a schedule with cron, and you
|
|
|
|
-encrypt your Borg repository with a passphrase instead of a key file, you'll
|
|
|
|
-either need to set the borgmatic `encryption_passphrase` configuration
|
|
|
|
-variable or set the `BORG_PASSPHRASE` environment variable. See the
|
|
|
|
-[repository encryption
|
|
|
|
-section](https://borgbackup.readthedocs.io/en/latest/quickstart.html#repository-encryption)
|
|
|
|
-of the Quick Start for more info.
|
|
|
|
-
|
|
|
|
-Alternatively, the passphrase can be specified programatically by setting
|
|
|
|
-either the borgmatic `encryption_passcommand` configuration variable or the
|
|
|
|
-`BORG_PASSCOMMAND` environment variable. See the [Borg Security
|
|
|
|
-FAQ](http://borgbackup.readthedocs.io/en/stable/faq.html#how-can-i-specify-the-encryption-passphrase-programmatically)
|
|
|
|
-for more info.
|
|
|
|
-
|
|
|
|
-
|
|
|
|
-## Usage
|
|
|
|
-
|
|
|
|
-### Initialization
|
|
|
|
-
|
|
|
|
-Before you can create backups with borgmatic, you first need to initialize a
|
|
|
|
-Borg repository so you have a destination for your backup archives. (But skip
|
|
|
|
-this step if you already have a Borg repository.) To create a repository, run
|
|
|
|
-a command like the following:
|
|
|
|
-
|
|
|
|
-```bash
|
|
|
|
-borgmatic --init --encryption repokey
|
|
|
|
-```
|
|
|
|
-
|
|
|
|
-This uses the borgmatic configuration file you created above to determine
|
|
|
|
-which local or remote repository to create, and encrypts it with the
|
|
|
|
-encryption passphrase specified there if one is provided. Read about [Borg
|
|
|
|
-encryption
|
|
|
|
-modes](https://borgbackup.readthedocs.io/en/latest/usage/init.html#encryption-modes)
|
|
|
|
-for the menu of available encryption modes.
|
|
|
|
-
|
|
|
|
-Also, optionally check out the [Borg Quick
|
|
|
|
-Start](https://borgbackup.readthedocs.org/en/latest/quickstart.html) for more
|
|
|
|
-background about repository initialization.
|
|
|
|
-
|
|
|
|
-Note that borgmatic skips repository initialization if the repository already
|
|
|
|
-exists. This supports use cases like ensuring a repository exists prior to
|
|
|
|
-performing a backup.
|
|
|
|
-
|
|
|
|
-If the repository is on a remote host, make sure that your local user has
|
|
|
|
-key-based SSH access to the desired user account on the remote host.
|
|
|
|
-
|
|
|
|
-
|
|
|
|
-### Backups
|
|
|
|
-
|
|
|
|
-You can run borgmatic and start a backup by invoking it without arguments:
|
|
|
|
-
|
|
|
|
-```bash
|
|
|
|
-borgmatic
|
|
|
|
-```
|
|
|
|
-
|
|
|
|
-This will also prune any old backups as per the configured retention policy,
|
|
|
|
-and check backups for consistency problems due to things like file damage.
|
|
|
|
-
|
|
|
|
-If you'd like to see the available command-line arguments, view the help:
|
|
|
|
-
|
|
|
|
-```bash
|
|
|
|
-borgmatic --help
|
|
|
|
-```
|
|
|
|
-
|
|
|
|
-Note that borgmatic prunes archives *before* creating an archive, so as to
|
|
|
|
-free up space for archiving. This means that when a borgmatic run finishes,
|
|
|
|
-there may still be prune-able archives. Not to worry, as they will get cleaned
|
|
|
|
-up at the start of the next run.
|
|
|
|
-
|
|
|
|
-
|
|
|
|
-### Verbosity
|
|
|
|
-
|
|
|
|
-By default, the backup will proceed silently except in the case of errors. But
|
|
|
|
-if you'd like to to get additional information about the progress of the
|
|
|
|
-backup as it proceeds, use the verbosity option:
|
|
|
|
-
|
|
|
|
-```bash
|
|
|
|
-borgmatic --verbosity 1
|
|
|
|
-```
|
|
|
|
-
|
|
|
|
-Or, for even more progress spew:
|
|
|
|
-
|
|
|
|
-```bash
|
|
|
|
-borgmatic --verbosity 2
|
|
|
|
-```
|
|
|
|
-
|
|
|
|
-### À la carte
|
|
|
|
-
|
|
|
|
-If you want to run borgmatic with only pruning, creating, or checking enabled,
|
|
|
|
-the following optional flags are available:
|
|
|
|
-
|
|
|
|
-```bash
|
|
|
|
-borgmatic --prune
|
|
|
|
-borgmatic --create
|
|
|
|
-borgmatic --check
|
|
|
|
-```
|
|
|
|
-
|
|
|
|
-You can run with only one of these flags provided, or you can mix and match
|
|
|
|
-any number of them. This supports use cases like running consistency checks
|
|
|
|
-from a different cron job with a different frequency, or running pruning with
|
|
|
|
-a different verbosity level.
|
|
|
|
-
|
|
|
|
-Additionally, borgmatic provides convenient flags for Borg's
|
|
|
|
-[list](https://borgbackup.readthedocs.io/en/stable/usage/list.html) and
|
|
|
|
-[info](https://borgbackup.readthedocs.io/en/stable/usage/info.html)
|
|
|
|
-functionality:
|
|
|
|
-
|
|
|
|
-
|
|
|
|
-```bash
|
|
|
|
-borgmatic --list
|
|
|
|
-borgmatic --info
|
|
|
|
-```
|
|
|
|
-
|
|
|
|
-You can include an optional `--json` flag with `--create`, `--list`, or
|
|
|
|
-`--info` to get the output formatted as JSON.
|
|
|
|
-
|
|
|
|
-
|
|
|
|
-## Autopilot
|
|
|
|
-
|
|
|
|
-If you want to run borgmatic automatically, say once a day, the you can
|
|
|
|
-configure a job runner to invoke it periodically.
|
|
|
|
-
|
|
|
|
-### cron
|
|
|
|
-
|
|
|
|
-If you're using cron, download the [sample cron
|
|
|
|
-file](https://projects.torsion.org/witten/borgmatic/src/master/sample/cron/borgmatic).
|
|
|
|
-Then, from the directory where you downloaded it:
|
|
|
|
-
|
|
|
|
-```bash
|
|
|
|
-sudo mv borgmatic /etc/cron.d/borgmatic
|
|
|
|
-sudo chmod +x /etc/cron.d/borgmatic
|
|
|
|
-```
|
|
|
|
-
|
|
|
|
-You can modify the cron file if you'd like to run borgmatic more or less frequently.
|
|
|
|
-
|
|
|
|
-### systemd
|
|
|
|
-
|
|
|
|
-If you're using systemd instead of cron to run jobs, download the [sample
|
|
|
|
-systemd service
|
|
|
|
-file](https://projects.torsion.org/witten/borgmatic/src/master/sample/systemd/borgmatic.service)
|
|
|
|
-and the [sample systemd timer
|
|
|
|
-file](https://projects.torsion.org/witten/borgmatic/src/master/sample/systemd/borgmatic.timer).
|
|
|
|
-Then, from the directory where you downloaded them:
|
|
|
|
-
|
|
|
|
-```bash
|
|
|
|
-sudo mv borgmatic.service borgmatic.timer /etc/systemd/system/
|
|
|
|
-sudo systemctl enable borgmatic.timer
|
|
|
|
-sudo systemctl start borgmatic.timer
|
|
|
|
-```
|
|
|
|
-
|
|
|
|
-Feel free to modify the timer file based on how frequently you'd like
|
|
|
|
-borgmatic to run.
|
|
|
|
-
|
|
|
|
-
|
|
|
|
-## Advanced configuration
|
|
|
|
-
|
|
|
|
-### Multiple configuration files
|
|
|
|
-
|
|
|
|
-A more advanced usage is to create multiple separate configuration files and
|
|
|
|
-place each one in an /etc/borgmatic.d directory. For instance:
|
|
|
|
-
|
|
|
|
-```bash
|
|
|
|
-sudo mkdir /etc/borgmatic.d
|
|
|
|
-sudo generate-borgmatic-config --destination /etc/borgmatic.d/app1.yaml
|
|
|
|
-sudo generate-borgmatic-config --destination /etc/borgmatic.d/app2.yaml
|
|
|
|
-```
|
|
|
|
-
|
|
|
|
-With this approach, you can have entirely different backup policies for
|
|
|
|
-different applications on your system. For instance, you may want one backup
|
|
|
|
-configuration for your database data directory, and a different configuration
|
|
|
|
-for your user home directories.
|
|
|
|
-
|
|
|
|
-When you set up multiple configuration files like this, borgmatic will run
|
|
|
|
-each one in turn from a single borgmatic invocation. This includes, by
|
|
|
|
-default, the traditional /etc/borgmatic/config.yaml as well.
|
|
|
|
-
|
|
|
|
-And if you need even more customizability, you can specify alternate
|
|
|
|
-configuration paths on the command-line with borgmatic's `--config` option.
|
|
|
|
-See `borgmatic --help` for more information.
|
|
|
|
-
|
|
|
|
-
|
|
|
|
-### Hooks
|
|
|
|
-
|
|
|
|
-If you find yourself performing prepraration tasks before your backup runs, or
|
|
|
|
-cleanup work afterwards, borgmatic hooks may be of interest. Hooks are
|
|
|
|
-shell commands that borgmatic executes for you at various points, and they're
|
|
|
|
-configured in the `hooks` section of your configuration file.
|
|
|
|
-
|
|
|
|
-For instance, you can specify `before_backup` hooks to dump a database to file
|
|
|
|
-before backing it up, and specify `after_backup` hooks to delete the temporary
|
|
|
|
-file afterwards.
|
|
|
|
-
|
|
|
|
-borgmatic hooks run once per configuration file. `before_backup` hooks run
|
|
|
|
-prior to backups of all repositories. `after_backup` hooks run afterwards, but
|
|
|
|
-not if an error occurs in a previous hook or in the backups themselves. And
|
|
|
|
-borgmatic runs `on_error` hooks if an error occurs.
|
|
|
|
-
|
|
|
|
-An important security note about hooks: borgmatic executes all hook commands
|
|
|
|
-with the user permissions of borgmatic itself. So to prevent potential shell
|
|
|
|
-injection or privilege escalation, do not forget to set secure permissions
|
|
|
|
-(`chmod 0700`) on borgmatic configuration files and scripts invoked by hooks.
|
|
|
|
-
|
|
|
|
-See the sample generated configuration file mentioned above for specifics
|
|
|
|
-about hook configuration syntax.
|
|
|
|
-
|
|
|
|
-
|
|
|
|
-## Upgrading
|
|
|
|
-
|
|
|
|
-In general, all you should need to do to upgrade borgmatic is run the
|
|
|
|
-following:
|
|
|
|
-
|
|
|
|
-```bash
|
|
|
|
-sudo pip3 install --upgrade borgmatic
|
|
|
|
-```
|
|
|
|
-
|
|
|
|
-See below about special cases.
|
|
|
|
-
|
|
|
|
-
|
|
|
|
-### Upgrading from borgmatic 1.0.x
|
|
|
|
-
|
|
|
|
-borgmatic changed its configuration file format in version 1.1.0 from
|
|
|
|
-INI-style to YAML. This better supports validation, and has a more natural way
|
|
|
|
-to express lists of values. To upgrade your existing configuration, first
|
|
|
|
-upgrade to the new version of borgmatic.
|
|
|
|
-
|
|
|
|
-As of version 1.1.0, borgmatic no longer supports Python 2. If you were
|
|
|
|
-already running borgmatic with Python 3, then you can upgrade borgmatic
|
|
|
|
-in-place:
|
|
|
|
-
|
|
|
|
-```bash
|
|
|
|
-sudo pip3 install --upgrade borgmatic
|
|
|
|
-```
|
|
|
|
-
|
|
|
|
-But if you were running borgmatic with Python 2, uninstall and reinstall instead:
|
|
|
|
-
|
|
|
|
-```bash
|
|
|
|
-sudo pip uninstall borgmatic
|
|
|
|
-sudo pip3 install borgmatic
|
|
|
|
-```
|
|
|
|
-
|
|
|
|
-The pip binary names for different versions of Python can differ, so the above
|
|
|
|
-commands may need some tweaking to work on your machine.
|
|
|
|
-
|
|
|
|
-
|
|
|
|
-Once borgmatic is upgraded, run:
|
|
|
|
-
|
|
|
|
-```bash
|
|
|
|
-sudo upgrade-borgmatic-config
|
|
|
|
-```
|
|
|
|
-
|
|
|
|
-That will generate a new YAML configuration file at /etc/borgmatic/config.yaml
|
|
|
|
-(by default) using the values from both your existing configuration and
|
|
|
|
-excludes files. The new version of borgmatic will consume the YAML
|
|
|
|
-configuration file instead of the old one.
|
|
|
|
-
|
|
|
|
-
|
|
|
|
-### Upgrading from atticmatic
|
|
|
|
-
|
|
|
|
-You can ignore this section if you're not an atticmatic user (the former name
|
|
|
|
-of borgmatic).
|
|
|
|
-
|
|
|
|
-borgmatic only supports Borg now and no longer supports Attic. So if you're
|
|
|
|
-an Attic user, consider switching to Borg. See the [Borg upgrade
|
|
|
|
-command](https://borgbackup.readthedocs.io/en/stable/usage.html#borg-upgrade)
|
|
|
|
-for more information. Then, follow the instructions above about setting up
|
|
|
|
-your borgmatic configuration files.
|
|
|
|
-
|
|
|
|
-If you were already using Borg with atticmatic, then you can upgrade
|
|
|
|
-from atticmatic to borgmatic by running the following commands:
|
|
|
|
-
|
|
|
|
-```bash
|
|
|
|
-sudo pip3 uninstall atticmatic
|
|
|
|
-sudo pip3 install borgmatic
|
|
|
|
-```
|
|
|
|
-
|
|
|
|
-That's it! borgmatic will continue using your /etc/borgmatic configuration
|
|
|
|
-files.
|
|
|
|
|
|
+ * [Set up backups with borgmatic](docs/how-to/set-up-backups.md) \<-- *Start here!*
|
|
|
|
+ * [Make per-application backups](docs/how-to/make-per-application-backups.md)
|
|
|
|
+ * [Deal with very large backups](docs/how-to/deal-with-very-large-backups.md)
|
|
|
|
+ * [Inspect your backups](docs/how-to/inspect-your-backups.md)
|
|
|
|
+ * [Run preparation steps before backups](docs/how-to/run-preparation-steps-before-backups.md)
|
|
|
|
+ * [Upgrade borgmatic](docs/how-to/upgrade.md)
|
|
|
|
+ * [Develop on borgmatic](docs/how-to/develop-on-borgmatic.md)
|
|
|
|
|
|
|
|
|
|
## Support and contributing
|
|
## Support and contributing
|
|
@@ -409,95 +86,10 @@ or open an [issue](https://projects.torsion.org/witten/borgmatic/issues) first
|
|
to discuss your idea. We also accept Pull Requests on GitHub, if that's more
|
|
to discuss your idea. We also accept Pull Requests on GitHub, if that's more
|
|
your thing. In general, contributions are very welcome. We don't bite!
|
|
your thing. In general, contributions are very welcome. We don't bite!
|
|
|
|
|
|
|
|
+Also, please check out the [borgmatic development
|
|
|
|
+how-to](docs/how-to/develop-on-borgmatic.md) for info on cloning source code,
|
|
|
|
+running tests, etc.
|
|
|
|
|
|
-### Code style
|
|
|
|
-
|
|
|
|
-Start with [PEP 8](https://www.python.org/dev/peps/pep-0008/). But then, apply
|
|
|
|
-the following deviations from it:
|
|
|
|
-
|
|
|
|
- * For strings, prefer single quotes over double quotes.
|
|
|
|
- * Limit all lines to a maximum of 100 characters.
|
|
|
|
- * Use trailing commas within multiline values or argument lists.
|
|
|
|
- * For multiline constructs, put opening and closing delimeters on lines
|
|
|
|
- separate from their contents.
|
|
|
|
- * Within multiline constructs, use standard four-space indentation. Don't align
|
|
|
|
- indentation with an opening delimeter.
|
|
|
|
-
|
|
|
|
-borgmatic code uses the [Black](https://black.readthedocs.io/en/stable/) code
|
|
|
|
-formatter and [Flake8](http://flake8.pycqa.org/en/latest/) code checker, so
|
|
|
|
-certain code style requirements will be enforced when running automated tests.
|
|
|
|
-See the Black and Flake8 documentation for more information.
|
|
|
|
-
|
|
|
|
-
|
|
|
|
-### Development
|
|
|
|
-
|
|
|
|
-To get set up to hack on borgmatic, first clone master via HTTPS or SSH:
|
|
|
|
-
|
|
|
|
-```bash
|
|
|
|
-git clone https://projects.torsion.org/witten/borgmatic.git
|
|
|
|
-```
|
|
|
|
-
|
|
|
|
-Or:
|
|
|
|
-
|
|
|
|
-```bash
|
|
|
|
-git clone ssh://git@projects.torsion.org:3022/witten/borgmatic.git
|
|
|
|
-```
|
|
|
|
-
|
|
|
|
-Then, install borgmatic
|
|
|
|
-"[editable](https://pip.pypa.io/en/stable/reference/pip_install/#editable-installs)"
|
|
|
|
-so that you can run borgmatic commands while you're hacking on them to
|
|
|
|
-make sure your changes work.
|
|
|
|
-
|
|
|
|
-```bash
|
|
|
|
-cd borgmatic/
|
|
|
|
-pip3 install --editable --user .
|
|
|
|
-```
|
|
|
|
-
|
|
|
|
-Note that this will typically install the borgmatic commands into
|
|
|
|
-`~/.local/bin`, which may or may not be on your PATH. There are other ways to
|
|
|
|
-install borgmatic editable as well, for instance into the system Python
|
|
|
|
-install (so without `--user`, as root), or even into a
|
|
|
|
-[virtualenv](https://virtualenv.pypa.io/en/stable/). How or where you install
|
|
|
|
-borgmatic is up to you, but generally an editable install makes development
|
|
|
|
-and testing easier.
|
|
|
|
-
|
|
|
|
-
|
|
|
|
-### Running tests
|
|
|
|
-
|
|
|
|
-Assuming you've cloned the borgmatic source code as described above, and
|
|
|
|
-you're in the `borgmatic/` working copy, install tox, which is used for
|
|
|
|
-setting up testing environments:
|
|
|
|
-
|
|
|
|
-```bash
|
|
|
|
-sudo pip3 install tox
|
|
|
|
-```
|
|
|
|
-
|
|
|
|
-Finally, to actually run tests, run:
|
|
|
|
-
|
|
|
|
-```bash
|
|
|
|
-cd borgmatic
|
|
|
|
-tox
|
|
|
|
-```
|
|
|
|
-
|
|
|
|
-If when running tests, you get an error from the
|
|
|
|
-[Black](https://black.readthedocs.io/en/stable/) code formatter about files
|
|
|
|
-that would be reformatted, you can ask Black to format them for you via the
|
|
|
|
-following:
|
|
|
|
-
|
|
|
|
-```bash
|
|
|
|
-tox -e black
|
|
|
|
-```
|
|
|
|
-
|
|
|
|
-### End-to-end tests
|
|
|
|
-
|
|
|
|
-borgmatic additionally includes some end-to-end tests that integration test
|
|
|
|
-with Borg for a few representative scenarios. These tests don't run by default
|
|
|
|
-because they're relatively slow and depend on Borg. If you would like to run
|
|
|
|
-them:
|
|
|
|
-
|
|
|
|
-```bash
|
|
|
|
-tox -e end-to-end
|
|
|
|
-```
|
|
|
|
|
|
|
|
## Troubleshooting
|
|
## Troubleshooting
|
|
|
|
|