update man pages / usage docs

docs/man/borg-compact.1

@@ -0,0 +1,87 @@
+.\" Man page generated from reStructuredText.
+.
+.TH BORG-COMPACT 1 "2018-07-14" "" "borg backup tool"
+.SH NAME
+borg-compact \- compact segment files in the repository
+.
+.nr rst2man-indent-level 0
+.
+.de1 rstReportMargin
+\\$1 \\n[an-margin]
+level \\n[rst2man-indent-level]
+level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
+-
+\\n[rst2man-indent0]
+\\n[rst2man-indent1]
+\\n[rst2man-indent2]
+..
+.de1 INDENT
+.\" .rstReportMargin pre:
+. RS \\$1
+. nr rst2man-indent\\n[rst2man-indent-level] \\n[an-margin]
+. nr rst2man-indent-level +1
+.\" .rstReportMargin post:
+..
+.de UNINDENT
+. RE
+.\" indent \\n[an-margin]
+.\" old: \\n[rst2man-indent\\n[rst2man-indent-level]]
+.nr rst2man-indent-level -1
+.\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
+.in \\n[rst2man-indent\\n[rst2man-indent-level]]u
+..
+.SH SYNOPSIS
+.sp
+borg [common options] compact [options] REPOSITORY
+.SH DESCRIPTION
+.sp
+This command frees repository space by compacting segments.
+.sp
+Use this regularly to avoid running out of space \- you do not need to use this
+after each borg command though.
+.sp
+borg compact does not need a key, so it is possible to invoke it from the
+client or also from the server.
+.sp
+Depending on the amount of segments that need compaction, it may take a while.
+.sp
+See \fIseparate_compaction\fP in Additional Notes for more details.
+.SH OPTIONS
+.sp
+See \fIborg\-common(1)\fP for common options of Borg commands.
+.SS arguments
+.INDENT 0.0
+.TP
+.B REPOSITORY
+repository to compact
+.UNINDENT
+.SS optional arguments
+.INDENT 0.0
+.TP
+.B \-\-cleanup\-commits
+cleanup commit\-only 17\-byte segment files
+.UNINDENT
+.SH EXAMPLES
+.INDENT 0.0
+.INDENT 3.5
+.sp
+.nf
+.ft C
+# compact segments and free repo disk space
+$ borg compact /path/to/repo
+
+# same as above plus clean up 17byte commit\-only segments,
+# use this one time after upgrading borg (server) to 1.2+
+# to clean up the tiny segments files created by borg 1.1:
+$ borg compact \-\-cleanup\-commits /path/to/repo
+.ft P
+.fi
+.UNINDENT
+.UNINDENT
+.SH SEE ALSO
+.sp
+\fIborg\-common(1)\fP
+.SH AUTHOR
+The Borg Collective
+.\" Generated by docutils manpage writer.
+.

docs/man/borg-delete.1

@@ -1,6 +1,6 @@
 .\" Man page generated from reStructuredText.
 .
-.TH BORG-DELETE 1 "2017-11-25" "" "borg backup tool"
+.TH BORG-DELETE 1 "2018-07-14" "" "borg backup tool"
 .SH NAME
 borg-delete \- Delete an existing repository or archives
 .
@@ -36,13 +36,28 @@ borg [common options] delete [options] [TARGET] [ARCHIVE...]
 .SH DESCRIPTION
 .sp
 This command deletes an archive from the repository or the complete repository.
-Disk space is reclaimed accordingly. If you delete the complete repository, the
-local cache for it (if any) is also deleted.
+.sp
+Important: When deleting archives, repository disk space is \fBnot\fP freed until
+you run \fBborg compact\fP\&.
+.sp
+If you delete the complete repository, the local cache for it (if any) is
+also deleted. Alternatively, you can delete just the local cache with the
+\fB\-\-cache\-only\fP option.
 .sp
 When using \fB\-\-stats\fP, you will get some statistics about how much data was
 deleted \- the "Deleted data" deduplicated size there is most interesting as
 that is how much your repository will shrink.
 Please note that the "All archives" stats refer to the state after deletion.
+.sp
+You can delete multiple archives by specifying their common prefix, if they
+have one, using the \fB\-\-prefix PREFIX\fP option. You can also specify a shell
+pattern to match multiple archives using the \fB\-\-glob\-archives GLOB\fP option
+(for more info on these patterns, see \fBborg help patterns\fP). Note that these
+two options are mutually exclusive.
+.sp
+To avoid accidentally deleting archives, especially when using glob patterns,
+it might be helpful to use the \fB\-\-dry\-run\fP to test out the command without
+actually making any changes to the repository.
 .SH OPTIONS
 .sp
 See \fIborg\-common(1)\fP for common options of Borg commands.
@@ -58,6 +73,9 @@ archives to delete
 .SS optional arguments
 .INDENT 0.0
 .TP
+.B \-n\fP,\fB  \-\-dry\-run
+do not change repository
+.TP
 .B \-s\fP,\fB  \-\-stats
 print statistics for the deleted archive
 .TP
@@ -96,6 +114,17 @@ consider last N archives after other filters were applied
 .ft C
 # delete a single backup archive:
 $ borg delete /path/to/repo::Monday
+# actually free disk space:
+$ borg compact /path/to/repo
+
+# delete all archives whose names begin with the machine\(aqs hostname followed by "\-"
+$ borg delete \-\-prefix \(aq{hostname}\-\(aq /path/to/repo
+
+# delete all archives whose names contain "\-2012\-"
+$ borg delete \-\-glob\-archives \(aq*\-2012\-*\(aq /path/to/repo
+
+# see what would be deleted if delete was run without \-\-dry\-run
+$ borg delete \-v \-\-dry\-run \-a \(aq*\-May\-*\(aq /path/to/repo
 
 # delete the whole repository and the related local cache:
 $ borg delete /path/to/repo
@@ -110,7 +139,7 @@ Type \(aqYES\(aq if you understand this and want to continue: YES
 .UNINDENT
 .SH SEE ALSO
 .sp
-\fIborg\-common(1)\fP
+\fIborg\-common(1)\fP, \fIborg\-compact(1)\fP
 .SH AUTHOR
 The Borg Collective
 .\" Generated by docutils manpage writer.

docs/man/borg-prune.1

@@ -1,6 +1,6 @@
 .\" Man page generated from reStructuredText.
 .
-.TH BORG-PRUNE 1 "2017-11-25" "" "borg backup tool"
+.TH BORG-PRUNE 1 "2018-07-14" "" "borg backup tool"
 .SH NAME
 borg-prune \- Prune repository archives according to specified rules
 .
@@ -36,8 +36,12 @@ borg [common options] prune [options] [REPOSITORY]
 .SH DESCRIPTION
 .sp
 The prune command prunes a repository by deleting all archives not matching
-any of the specified retention options. This command is normally used by
-automated backup scripts wanting to keep a certain number of historic backups.
+any of the specified retention options.
+.sp
+Important: Repository disk space is \fBnot\fP freed until you run \fBborg compact\fP\&.
+.sp
+This command is normally used by automated backup scripts wanting to keep a
+certain number of historic backups.
 .sp
 Also, prune automatically removes checkpoint archives (incomplete archives left
 behind by interrupted backup runs) except if the checkpoint is the latest
@@ -162,6 +166,8 @@ $ borg prune \-v \-\-list \-\-dry\-run \-\-keep\-daily=7 \-\-keep\-weekly=4 /pat
 # Same as above but only apply to archive names starting with the hostname
 # of the machine followed by a "\-" character:
 $ borg prune \-v \-\-list \-\-keep\-daily=7 \-\-keep\-weekly=4 \-\-prefix=\(aq{hostname}\-\(aq /path/to/repo
+# actually free disk space:
+$ borg compact /path/to/repo
 
 # Keep 7 end of day, 4 additional end of week archives,
 # and an end of month archive for every month:
@@ -175,23 +181,10 @@ $ borg prune \-v \-\-list \-\-keep\-within=10d \-\-keep\-weekly=4 \-\-keep\-mont
 .UNINDENT
 .UNINDENT
 .sp
-There is also a visualized prune example in \fBdocs/misc/prune\-example.txt\fP:
-.IP "System Message: ERROR/3 (docs/virtmanpage.rst:, line 145)"
-Unknown directive type "highlight".
-.INDENT 0.0
-.INDENT 3.5
-.sp
-.nf
-.ft C
-\&.. highlight:: none
-
-.ft P
-.fi
-.UNINDENT
-.UNINDENT
+There is also a visualized prune example in \fBdocs/misc/prune\-example.txt\fP\&.
 .SH SEE ALSO
 .sp
-\fIborg\-common(1)\fP
+\fIborg\-common(1)\fP, \fIborg\-compact(1)\fP
 .SH AUTHOR
 The Borg Collective
 .\" Generated by docutils manpage writer.

docs/man/borg-recreate.1

@@ -1,6 +1,6 @@
 .\" Man page generated from reStructuredText.
 .
-.TH BORG-RECREATE 1 "2017-11-25" "" "borg backup tool"
+.TH BORG-RECREATE 1 "2018-07-14" "" "borg backup tool"
 .SH NAME
 borg-recreate \- Re-create archives
 .
@@ -39,6 +39,8 @@ Recreate the contents of existing archives.
 .sp
 This is an \fIexperimental\fP feature. Do \fInot\fP use this on your only backup.
 .sp
+Important: Repository disk space is \fBnot\fP freed until you run \fBborg compact\fP\&.
+.sp
 \fB\-\-exclude\fP, \fB\-\-exclude\-from\fP, \fB\-\-exclude\-if\-present\fP, \fB\-\-keep\-exclude\-tags\fP, and PATH
 have the exact same semantics as in "borg create". If PATHs are specified the
 resulting archive will only contain files from these PATHs.
@@ -67,10 +69,9 @@ archive that is built during the operation exists at the same time at
 .sp
 With \fB\-\-target\fP the original archive is not replaced, instead a new archive is created.
 .sp
-When rechunking space usage can be substantial, expect at least the entire
-deduplicated size of the archives using the previous chunker params.
-When recompressing expect approx. (throughput / checkpoint\-interval) in space usage,
-assuming all chunks are recompressed.
+When rechunking (or recompressing), space usage can be substantial \- expect
+at least the entire deduplicated size of the archives using the previous
+chunker (or compression) params.
 .sp
 If you recently ran borg check \-\-repair and it had to fix lost chunks with all\-zero
 replacement chunks, please first run another backup for the same data and re\-run
@@ -151,8 +152,8 @@ manually specify the archive creation date/time (UTC, yyyy\-mm\-ddThh:mm:ss form
 .BI \-C \ COMPRESSION\fP,\fB \ \-\-compression \ COMPRESSION
 select compression algorithm, see the output of the "borg help compression" command for details.
 .TP
-.B \-\-recompress
-recompress data chunks according to \fB\-\-compression\fP if \fIif\-different\fP\&. When \fIalways\fP, chunks that are already compressed that way are not skipped, but compressed again. Only the algorithm is considered for \fIif\-different\fP, not the compression level (if any).
+.BI \-\-recompress \ MODE
+recompress data chunks according to \fB\-\-compression\fP\&. MODE \fIif\-different\fP: recompress if current compression is with a different compression algorithm (the level is not considered). MODE \fIalways\fP: recompress even if current compression is with the same compression algorithm (use this to change the compression level). MODE \fInever\fP (default): do not recompress.
 .TP
 .BI \-\-chunker\-params \ PARAMS
 specify the chunker parameters (CHUNK_MIN_EXP, CHUNK_MAX_EXP, HASH_MASK_BITS, HASH_WINDOW_SIZE) or \fIdefault\fP to use the current defaults. default: 19,23,21,4095

docs/usage.rst

@@ -42,6 +42,7 @@ Usage
    usage/diff
    usage/delete
    usage/prune
+   usage/compact
    usage/info
    usage/mount
    usage/key

docs/usage/compact.rst

@@ -0,0 +1,15 @@
+.. include:: compact.rst.inc
+
+Examples
+~~~~~~~~
+::
+
+    # compact segments and free repo disk space
+    $ borg compact /path/to/repo
+
+    # same as above plus clean up 17byte commit-only segments,
+    # use this one time after upgrading borg (server) to 1.2+
+    # to clean up the tiny segments files created by borg 1.1:
+    $ borg compact --cleanup-commits /path/to/repo
+
+

docs/usage/compact.rst.inc

@@ -0,0 +1,63 @@
+.. IMPORTANT: this file is auto-generated from borg's built-in help, do not edit!
+
+.. _borg_compact:
+
+borg compact
+------------
+.. code-block:: none
+
+    borg [common options] compact [options] REPOSITORY
+
+.. only:: html
+
+    .. class:: borg-options-table
+
+    +-------------------------------------------------------+-----------------------+-------------------------------------------+
+    | **positional arguments**                                                                                                  |
+    +-------------------------------------------------------+-----------------------+-------------------------------------------+
+    |                                                       | ``REPOSITORY``        | repository to compact                     |
+    +-------------------------------------------------------+-----------------------+-------------------------------------------+
+    | **optional arguments**                                                                                                    |
+    +-------------------------------------------------------+-----------------------+-------------------------------------------+
+    |                                                       | ``--cleanup-commits`` | cleanup commit-only 17-byte segment files |
+    +-------------------------------------------------------+-----------------------+-------------------------------------------+
+    | .. class:: borg-common-opt-ref                                                                                            |
+    |                                                                                                                           |
+    | :ref:`common_options`                                                                                                     |
+    +-------------------------------------------------------+-----------------------+-------------------------------------------+
+
+    .. raw:: html
+
+        <script type='text/javascript'>
+        $(document).ready(function () {
+            $('.borg-options-table colgroup').remove();
+        })
+        </script>
+
+.. only:: latex
+
+    REPOSITORY
+        repository to compact
+
+
+    optional arguments
+        --cleanup-commits     cleanup commit-only 17-byte segment files
+
+
+    :ref:`common_options`
+        |
+
+Description
+~~~~~~~~~~~
+
+This command frees repository space by compacting segments.
+
+Use this regularly to avoid running out of space - you do not need to use this
+after each borg command though.
+
+borg compact does not need a key, so it is possible to invoke it from the
+client or also from the server.
+
+Depending on the amount of segments that need compaction, it may take a while.
+
+See :ref:`separate_compaction` in Additional Notes for more details.

docs/usage/delete.rst.inc

@@ -21,6 +21,8 @@ borg delete
     +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
     | **optional arguments**                                                                                                                                                                                                                                                       |
     +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                                             | ``-n``, ``--dry-run``                 | do not change repository                                                                                                                               |
+    +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
     |                                                                             | ``-s``, ``--stats``                   | print statistics for the deleted archive                                                                                                               |
     +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
     |                                                                             | ``--cache-only``                      | delete only the local cache for the given repository                                                                                                   |
@@ -63,6 +65,7 @@ borg delete
 
 
     optional arguments
+        -n, --dry-run    do not change repository
         -s, --stats     print statistics for the deleted archive
         --cache-only    delete only the local cache for the given repository
         --force         force deletion of corrupted archives, use ``--force --force`` in case ``--force`` does not work.
@@ -84,10 +87,25 @@ Description
 ~~~~~~~~~~~
 
 This command deletes an archive from the repository or the complete repository.
-Disk space is reclaimed accordingly. If you delete the complete repository, the
-local cache for it (if any) is also deleted.
+
+Important: When deleting archives, repository disk space is **not** freed until
+you run ``borg compact``.
+
+If you delete the complete repository, the local cache for it (if any) is
+also deleted. Alternatively, you can delete just the local cache with the
+``--cache-only`` option.
 
 When using ``--stats``, you will get some statistics about how much data was
 deleted - the "Deleted data" deduplicated size there is most interesting as
 that is how much your repository will shrink.
-Please note that the "All archives" stats refer to the state after deletion.
+Please note that the "All archives" stats refer to the state after deletion.
+
+You can delete multiple archives by specifying their common prefix, if they
+have one, using the ``--prefix PREFIX`` option. You can also specify a shell
+pattern to match multiple archives using the ``--glob-archives GLOB`` option
+(for more info on these patterns, see ``borg help patterns``). Note that these
+two options are mutually exclusive.
+
+To avoid accidentally deleting archives, especially when using glob patterns,
+it might be helpful to use the ``--dry-run`` to test out the command without
+actually making any changes to the repository.

docs/usage/prune.rst.inc

@@ -98,8 +98,12 @@ Description
 ~~~~~~~~~~~
 
 The prune command prunes a repository by deleting all archives not matching
-any of the specified retention options. This command is normally used by
-automated backup scripts wanting to keep a certain number of historic backups.
+any of the specified retention options.
+
+Important: Repository disk space is **not** freed until you run ``borg compact``.
+
+This command is normally used by automated backup scripts wanting to keep a
+certain number of historic backups.
 
 Also, prune automatically removes checkpoint archives (incomplete archives left
 behind by interrupted backup runs) except if the checkpoint is the latest

docs/usage/recreate.rst.inc

@@ -12,59 +12,59 @@ borg recreate
 
     .. class:: borg-options-table
 
-    +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
-    | **positional arguments**                                                                                                                                                                                                                                                                                                                                                        |
-    +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
-    |                                                       | ``REPOSITORY_OR_ARCHIVE``                         | repository/archive to recreate                                                                                                                                                                                                                                      |
-    +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
-    |                                                       | ``PATH``                                          | paths to recreate; patterns are supported                                                                                                                                                                                                                           |
-    +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
-    | **optional arguments**                                                                                                                                                                                                                                                                                                                                                          |
-    +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
-    |                                                       | ``--list``                                        | output verbose list of items (files, dirs, ...)                                                                                                                                                                                                                     |
-    +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
-    |                                                       | ``--filter STATUSCHARS``                          | only display items with the given status characters (listed in borg create --help)                                                                                                                                                                                  |
-    +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
-    |                                                       | ``-n``, ``--dry-run``                             | do not change anything                                                                                                                                                                                                                                              |
-    +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
-    |                                                       | ``-s``, ``--stats``                               | print statistics at end                                                                                                                                                                                                                                             |
-    +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
-    | .. class:: borg-common-opt-ref                                                                                                                                                                                                                                                                                                                                                  |
-    |                                                                                                                                                                                                                                                                                                                                                                                 |
-    | :ref:`common_options`                                                                                                                                                                                                                                                                                                                                                           |
-    +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
-    | **Exclusion options**                                                                                                                                                                                                                                                                                                                                                           |
-    +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
-    |                                                       | ``-e PATTERN``, ``--exclude PATTERN``             | exclude paths matching PATTERN                                                                                                                                                                                                                                      |
-    +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
-    |                                                       | ``--exclude-from EXCLUDEFILE``                    | read exclude patterns from EXCLUDEFILE, one per line                                                                                                                                                                                                                |
-    +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
-    |                                                       | ``--pattern PATTERN``                             | experimental: include/exclude paths matching PATTERN                                                                                                                                                                                                                |
-    +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
-    |                                                       | ``--patterns-from PATTERNFILE``                   | experimental: read include/exclude patterns from PATTERNFILE, one per line                                                                                                                                                                                          |
-    +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
-    |                                                       | ``--exclude-caches``                              | exclude directories that contain a CACHEDIR.TAG file (http://www.brynosaurus.com/cachedir/spec.html)                                                                                                                                                                |
-    +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
-    |                                                       | ``--exclude-if-present NAME``                     | exclude directories that are tagged by containing a filesystem object with the given NAME                                                                                                                                                                           |
-    +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
-    |                                                       | ``--keep-exclude-tags``, ``--keep-tag-files``     | if tag objects are specified with ``--exclude-if-present``, don't omit the tag objects themselves from the backup archive                                                                                                                                           |
-    +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
-    | **Archive options**                                                                                                                                                                                                                                                                                                                                                             |
-    +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
-    |                                                       | ``--target TARGET``                               | create a new archive with the name ARCHIVE, do not replace existing archive (only applies for a single archive)                                                                                                                                                     |
-    +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
-    |                                                       | ``-c SECONDS``, ``--checkpoint-interval SECONDS`` | write checkpoint every SECONDS seconds (Default: 1800)                                                                                                                                                                                                              |
-    +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
-    |                                                       | ``--comment COMMENT``                             | add a comment text to the archive                                                                                                                                                                                                                                   |
-    +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
-    |                                                       | ``--timestamp TIMESTAMP``                         | manually specify the archive creation date/time (UTC, yyyy-mm-ddThh:mm:ss format). alternatively, give a reference file/directory.                                                                                                                                  |
-    +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
-    |                                                       | ``-C COMPRESSION``, ``--compression COMPRESSION`` | select compression algorithm, see the output of the "borg help compression" command for details.                                                                                                                                                                    |
-    +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
-    |                                                       | ``--recompress``                                  | recompress data chunks according to ``--compression`` if `if-different`. When `always`, chunks that are already compressed that way are not skipped, but compressed again. Only the algorithm is considered for `if-different`, not the compression level (if any). |
-    +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
-    |                                                       | ``--chunker-params PARAMS``                       | specify the chunker parameters (CHUNK_MIN_EXP, CHUNK_MAX_EXP, HASH_MASK_BITS, HASH_WINDOW_SIZE) or `default` to use the current defaults. default: 19,23,21,4095                                                                                                    |
-    +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+    +-------------------------------------------------------+---------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+    | **positional arguments**                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
+    +-------------------------------------------------------+---------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                       | ``REPOSITORY_OR_ARCHIVE``                         | repository/archive to recreate                                                                                                                                                                                                                                                                                                                                             |
+    +-------------------------------------------------------+---------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                       | ``PATH``                                          | paths to recreate; patterns are supported                                                                                                                                                                                                                                                                                                                                  |
+    +-------------------------------------------------------+---------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+    | **optional arguments**                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
+    +-------------------------------------------------------+---------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                       | ``--list``                                        | output verbose list of items (files, dirs, ...)                                                                                                                                                                                                                                                                                                                            |
+    +-------------------------------------------------------+---------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                       | ``--filter STATUSCHARS``                          | only display items with the given status characters (listed in borg create --help)                                                                                                                                                                                                                                                                                         |
+    +-------------------------------------------------------+---------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                       | ``-n``, ``--dry-run``                             | do not change anything                                                                                                                                                                                                                                                                                                                                                     |
+    +-------------------------------------------------------+---------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                       | ``-s``, ``--stats``                               | print statistics at end                                                                                                                                                                                                                                                                                                                                                    |
+    +-------------------------------------------------------+---------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+    | .. class:: borg-common-opt-ref                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
+    |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
+    | :ref:`common_options`                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
+    +-------------------------------------------------------+---------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+    | **Exclusion options**                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
+    +-------------------------------------------------------+---------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                       | ``-e PATTERN``, ``--exclude PATTERN``             | exclude paths matching PATTERN                                                                                                                                                                                                                                                                                                                                             |
+    +-------------------------------------------------------+---------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                       | ``--exclude-from EXCLUDEFILE``                    | read exclude patterns from EXCLUDEFILE, one per line                                                                                                                                                                                                                                                                                                                       |
+    +-------------------------------------------------------+---------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                       | ``--pattern PATTERN``                             | experimental: include/exclude paths matching PATTERN                                                                                                                                                                                                                                                                                                                       |
+    +-------------------------------------------------------+---------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                       | ``--patterns-from PATTERNFILE``                   | experimental: read include/exclude patterns from PATTERNFILE, one per line                                                                                                                                                                                                                                                                                                 |
+    +-------------------------------------------------------+---------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                       | ``--exclude-caches``                              | exclude directories that contain a CACHEDIR.TAG file (http://www.brynosaurus.com/cachedir/spec.html)                                                                                                                                                                                                                                                                       |
+    +-------------------------------------------------------+---------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                       | ``--exclude-if-present NAME``                     | exclude directories that are tagged by containing a filesystem object with the given NAME                                                                                                                                                                                                                                                                                  |
+    +-------------------------------------------------------+---------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                       | ``--keep-exclude-tags``, ``--keep-tag-files``     | if tag objects are specified with ``--exclude-if-present``, don't omit the tag objects themselves from the backup archive                                                                                                                                                                                                                                                  |
+    +-------------------------------------------------------+---------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+    | **Archive options**                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
+    +-------------------------------------------------------+---------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                       | ``--target TARGET``                               | create a new archive with the name ARCHIVE, do not replace existing archive (only applies for a single archive)                                                                                                                                                                                                                                                            |
+    +-------------------------------------------------------+---------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                       | ``-c SECONDS``, ``--checkpoint-interval SECONDS`` | write checkpoint every SECONDS seconds (Default: 1800)                                                                                                                                                                                                                                                                                                                     |
+    +-------------------------------------------------------+---------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                       | ``--comment COMMENT``                             | add a comment text to the archive                                                                                                                                                                                                                                                                                                                                          |
+    +-------------------------------------------------------+---------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                       | ``--timestamp TIMESTAMP``                         | manually specify the archive creation date/time (UTC, yyyy-mm-ddThh:mm:ss format). alternatively, give a reference file/directory.                                                                                                                                                                                                                                         |
+    +-------------------------------------------------------+---------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                       | ``-C COMPRESSION``, ``--compression COMPRESSION`` | select compression algorithm, see the output of the "borg help compression" command for details.                                                                                                                                                                                                                                                                           |
+    +-------------------------------------------------------+---------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                       | ``--recompress MODE``                             | recompress data chunks according to ``--compression``. MODE `if-different`: recompress if current compression is with a different compression algorithm (the level is not considered). MODE `always`: recompress even if current compression is with the same compression algorithm (use this to change the compression level). MODE `never` (default): do not recompress. |
+    +-------------------------------------------------------+---------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                       | ``--chunker-params PARAMS``                       | specify the chunker parameters (CHUNK_MIN_EXP, CHUNK_MAX_EXP, HASH_MASK_BITS, HASH_WINDOW_SIZE) or `default` to use the current defaults. default: 19,23,21,4095                                                                                                                                                                                                           |
+    +-------------------------------------------------------+---------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 
     .. raw:: html
 
@@ -108,7 +108,7 @@ borg recreate
         --comment COMMENT                             add a comment text to the archive
         --timestamp TIMESTAMP                         manually specify the archive creation date/time (UTC, yyyy-mm-ddThh:mm:ss format). alternatively, give a reference file/directory.
         -C COMPRESSION, --compression COMPRESSION     select compression algorithm, see the output of the "borg help compression" command for details.
-        --recompress                                  recompress data chunks according to ``--compression`` if `if-different`. When `always`, chunks that are already compressed that way are not skipped, but compressed again. Only the algorithm is considered for `if-different`, not the compression level (if any).
+        --recompress MODE                             recompress data chunks according to ``--compression``. MODE `if-different`: recompress if current compression is with a different compression algorithm (the level is not considered). MODE `always`: recompress even if current compression is with the same compression algorithm (use this to change the compression level). MODE `never` (default): do not recompress.
         --chunker-params PARAMS                       specify the chunker parameters (CHUNK_MIN_EXP, CHUNK_MAX_EXP, HASH_MASK_BITS, HASH_WINDOW_SIZE) or `default` to use the current defaults. default: 19,23,21,4095
 
 
@@ -119,6 +119,8 @@ Recreate the contents of existing archives.
 
 This is an *experimental* feature. Do *not* use this on your only backup.
 
+Important: Repository disk space is **not** freed until you run ``borg compact``.
+
 ``--exclude``, ``--exclude-from``, ``--exclude-if-present``, ``--keep-exclude-tags``, and PATH
 have the exact same semantics as in "borg create". If PATHs are specified the
 resulting archive will only contain files from these PATHs.
@@ -147,10 +149,9 @@ archive that is built during the operation exists at the same time at
 
 With ``--target`` the original archive is not replaced, instead a new archive is created.
 
-When rechunking space usage can be substantial, expect at least the entire
-deduplicated size of the archives using the previous chunker params.
-When recompressing expect approx. (throughput / checkpoint-interval) in space usage,
-assuming all chunks are recompressed.
+When rechunking (or recompressing), space usage can be substantial - expect
+at least the entire deduplicated size of the archives using the previous
+chunker (or compression) params.
 
 If you recently ran borg check --repair and it had to fix lost chunks with all-zero
 replacement chunks, please first run another backup for the same data and re-run

setup_docs.py

@@ -273,6 +273,8 @@ class build_man(Command):
         'mount': ('umount', 'extract'),  # Would be cooler if these two were on the same page
         'umount': ('mount', ),
         'extract': ('mount', ),
+        'delete': ('compact', ),
+        'prune': ('compact', ),
     }
 
     rst_prelude = textwrap.dedent("""