build_usage (master)

also: add new borgfs.rst.inc

docs/usage/borgfs.rst.inc

@@ -0,0 +1,138 @@
+.. IMPORTANT: this file is auto-generated from borg's built-in help, do not edit!
+
+.. _borg_borgfs:
+
+borg borgfs
+-----------
+.. code-block:: none
+
+    borg [common options] borgfs [options] REPOSITORY_OR_ARCHIVE MOUNTPOINT [PATH...]
+
+.. only:: html
+
+    .. class:: borg-options-table
+
+    +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
+    | **positional arguments**                                                                                                                                                                                                                                                     |
+    +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                                             | ``REPOSITORY_OR_ARCHIVE``             | repository/archive to mount                                                                                                                            |
+    +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                                             | ``MOUNTPOINT``                        | where to mount filesystem                                                                                                                              |
+    +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                                             | ``PATH``                              | paths to extract; patterns are supported                                                                                                               |
+    +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
+    | **optional arguments**                                                                                                                                                                                                                                                       |
+    +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                                             | ``-V``, ``--version``                 | show version number and exit                                                                                                                           |
+    +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                                             | ``-f``, ``--foreground``              | stay in foreground, do not daemonize                                                                                                                   |
+    +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                                             | ``-o``                                | Extra mount options                                                                                                                                    |
+    +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
+    | .. class:: borg-common-opt-ref                                                                                                                                                                                                                                               |
+    |                                                                                                                                                                                                                                                                              |
+    | :ref:`common_options`                                                                                                                                                                                                                                                        |
+    +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
+    | **Archive filters** — Archive filters can be applied to repository targets.                                                                                                                                                                                                  |
+    +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                                             | ``-P PREFIX``, ``--prefix PREFIX``    | only consider archive names starting with this prefix.                                                                                                 |
+    +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                                             | ``-a GLOB``, ``--glob-archives GLOB`` | only consider archive names matching the glob. sh: rules apply, see "borg help patterns". ``--prefix`` and ``--glob-archives`` are mutually exclusive. |
+    +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                                             | ``--sort-by KEYS``                    | Comma-separated list of sorting keys; valid keys are: timestamp, name, id; default is: timestamp                                                       |
+    +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                                             | ``--first N``                         | consider first N archives after other filters were applied                                                                                             |
+    +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                                             | ``--last N``                          | consider last N archives after other filters were applied                                                                                              |
+    +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
+    | **Exclusion options**                                                                                                                                                                                                                                                        |
+    +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                                             | ``-e PATTERN``, ``--exclude PATTERN`` | exclude paths matching PATTERN                                                                                                                         |
+    +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                                             | ``--exclude-from EXCLUDEFILE``        | read exclude patterns from EXCLUDEFILE, one per line                                                                                                   |
+    +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                                             | ``--pattern PATTERN``                 | experimental: include/exclude paths matching PATTERN                                                                                                   |
+    +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                                             | ``--patterns-from PATTERNFILE``       | experimental: read include/exclude patterns from PATTERNFILE, one per line                                                                             |
+    +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                                             | ``--strip-components NUMBER``         | Remove the specified number of leading path elements. Paths with fewer elements will be silently skipped.                                              |
+    +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
+
+    .. raw:: html
+
+        <script type='text/javascript'>
+        $(document).ready(function () {
+            $('.borg-options-table colgroup').remove();
+        })
+        </script>
+
+.. only:: latex
+
+    REPOSITORY_OR_ARCHIVE
+        repository/archive to mount
+    MOUNTPOINT
+        where to mount filesystem
+    PATH
+        paths to extract; patterns are supported
+
+
+    optional arguments
+        -V, --version    show version number and exit
+        -f, --foreground    stay in foreground, do not daemonize
+        -o     Extra mount options
+
+
+    :ref:`common_options`
+        |
+
+    Archive filters
+        -P PREFIX, --prefix PREFIX        only consider archive names starting with this prefix.
+        -a GLOB, --glob-archives GLOB     only consider archive names matching the glob. sh: rules apply, see "borg help patterns". ``--prefix`` and ``--glob-archives`` are mutually exclusive.
+        --sort-by KEYS                    Comma-separated list of sorting keys; valid keys are: timestamp, name, id; default is: timestamp
+        --first N                         consider first N archives after other filters were applied
+        --last N                          consider last N archives after other filters were applied
+
+
+    Exclusion options
+        -e PATTERN, --exclude PATTERN     exclude paths matching PATTERN
+        --exclude-from EXCLUDEFILE        read exclude patterns from EXCLUDEFILE, one per line
+        --pattern PATTERN                 experimental: include/exclude paths matching PATTERN
+        --patterns-from PATTERNFILE       experimental: read include/exclude patterns from PATTERNFILE, one per line
+        --strip-components NUMBER         Remove the specified number of leading path elements. Paths with fewer elements will be silently skipped.
+
+
+Description
+~~~~~~~~~~~
+
+This command mounts an archive as a FUSE filesystem. This can be useful for
+browsing an archive or restoring individual files. Unless the ``--foreground``
+option is given the command will run in the background until the filesystem
+is ``umounted``.
+
+The command ``borgfs`` provides a wrapper for ``borg mount``. This can also be
+used in fstab entries:
+``/path/to/repo /mnt/point fuse.borgfs defaults,noauto 0 0``
+
+To allow a regular user to use fstab entries, add the ``user`` option:
+``/path/to/repo /mnt/point fuse.borgfs defaults,noauto,user 0 0``
+
+For mount options, see the fuse(8) manual page. Additional mount options
+supported by borg:
+
+- versions: when used with a repository mount, this gives a merged, versioned
+  view of the files in the archives. EXPERIMENTAL, layout may change in future.
+- allow_damaged_files: by default damaged files (where missing chunks were
+  replaced with runs of zeros by borg check ``--repair``) are not readable and
+  return EIO (I/O error). Set this option to read such files.
+
+The BORG_MOUNT_DATA_CACHE_ENTRIES environment variable is meant for advanced users
+to tweak the performance. It sets the number of cached data chunks; additional
+memory usage can be up to ~8 MiB times this number. The default is the number
+of CPU cores.
+
+When the daemonized process receives a signal or crashes, it does not unmount.
+Unmounting in these cases could cause an active rsync or similar process
+to unintentionally delete data.
+
+When running in the foreground ^C/SIGINT unmounts cleanly, but other
+signals or crashes do not.

docs/usage/change-passphrase.rst.inc

@@ -43,4 +43,9 @@ Description
 ~~~~~~~~~~~
 
 The key files used for repository encryption are optionally passphrase
-protected. This command can be used to change this passphrase.
+protected. This command can be used to change this passphrase.
+
+Please note that this command only changes the passphrase, but not any
+secret protected by it (like e.g. encryption/MAC keys or chunker seed).
+Thus, changing the passphrase after passphrase and borg key got compromised
+does not protect future (nor past) backups to the same repository.

docs/usage/common-options.rst.inc

@@ -10,7 +10,6 @@
 --lock-wait SECONDS    wait at most SECONDS for acquiring a repository/cache lock (default: 1).
 --show-version     show/log the borg version
 --show-rc          show/log the return code (rc)
---no-files-cache    do not load/update the file metadata cache used to detect unchanged files
 --umask M          set umask to M (local and remote, default: 0077)
 --remote-path PATH    use PATH as borg executable on the remote (default: "borg")
 --remote-ratelimit RATE    set remote network upload rate limit in kiByte/s (default: 0=unlimited)

docs/usage/create.rst.inc

@@ -31,7 +31,9 @@ borg create
     +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+
     |                                                       | ``--json``                                        | output stats as JSON. Implies ``--stats``.                                                                                                        |
     +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+
-    |                                                       | ``--no-cache-sync``                               | experimental: do not synchronize the cache. Implies ``--no-files-cache``.                                                                         |
+    |                                                       | ``--no-cache-sync``                               | experimental: do not synchronize the cache. Implies not using the files cache.                                                                    |
+    +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                       | ``--no-files-cache``                              | do not load/update the file metadata cache used to detect unchanged files                                                                         |
     +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+
     | .. class:: borg-common-opt-ref                                                                                                                                                                                                                                |
     |                                                                                                                                                                                                                                                               |
@@ -53,6 +55,8 @@ borg create
     +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+
     |                                                       | ``--keep-exclude-tags``, ``--keep-tag-files``     | if tag objects are specified with ``--exclude-if-present``, don't omit the tag objects themselves from the backup archive                         |
     +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                       | ``--exclude-nodump``                              | exclude files flagged NODUMP                                                                                                                      |
+    +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+
     | **Filesystem options**                                                                                                                                                                                                                                        |
     +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+
     |                                                       | ``-x``, ``--one-file-system``                     | stay in the same file system and do not store mount points of other file systems                                                                  |
@@ -63,8 +67,14 @@ borg create
     +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+
     |                                                       | ``--noctime``                                     | do not store ctime into archive                                                                                                                   |
     +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                       | ``--nobirthtime``                                 | do not store birthtime (creation date) into archive                                                                                               |
+    +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                       | ``--nobsdflags``                                  | do not read and store bsdflags (e.g. NODUMP, IMMUTABLE) into archive                                                                              |
+    +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+
     |                                                       | ``--ignore-inode``                                | ignore inode data in the file metadata cache used to detect unchanged files.                                                                      |
     +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                       | ``--files-cache MODE``                            | operate files cache in MODE. default: ctime,size,inode                                                                                            |
+    +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+
     |                                                       | ``--read-special``                                | open and read block and char device files as well as FIFOs as if they were regular files. Also follows symlinks pointing to these kinds of files. |
     +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+
     | **Archive options**                                                                                                                                                                                                                                           |
@@ -102,7 +112,8 @@ borg create
         --list          output verbose list of items (files, dirs, ...)
         --filter STATUSCHARS    only display items with the given status characters (see description)
         --json          output stats as JSON. Implies ``--stats``.
-        --no-cache-sync    experimental: do not synchronize the cache. Implies ``--no-files-cache``.
+        --no-cache-sync    experimental: do not synchronize the cache. Implies not using the files cache.
+        --no-files-cache    do not load/update the file metadata cache used to detect unchanged files
 
 
     :ref:`common_options`
@@ -116,6 +127,7 @@ borg create
         --exclude-caches                  exclude directories that contain a CACHEDIR.TAG file (http://www.brynosaurus.com/cachedir/spec.html)
         --exclude-if-present NAME         exclude directories that are tagged by containing a filesystem object with the given NAME
         --keep-exclude-tags, --keep-tag-files    if tag objects are specified with ``--exclude-if-present``, don't omit the tag objects themselves from the backup archive
+        --exclude-nodump                  exclude files flagged NODUMP
 
 
     Filesystem options
@@ -123,7 +135,10 @@ borg create
         --numeric-owner           only store numeric user and group identifiers
         --noatime                 do not store atime into archive
         --noctime                 do not store ctime into archive
+        --nobirthtime             do not store birthtime (creation date) into archive
+        --nobsdflags              do not read and store bsdflags (e.g. NODUMP, IMMUTABLE) into archive
         --ignore-inode            ignore inode data in the file metadata cache used to detect unchanged files.
+        --files-cache MODE        operate files cache in MODE. default: ctime,size,inode
         --read-special            open and read block and char device files as well as FIFOs as if they were regular files. Also follows symlinks pointing to these kinds of files.
 
 
@@ -157,13 +172,39 @@ In the archive name, you may use the following placeholders:
 {now}, {utcnow}, {fqdn}, {hostname}, {user} and some others.
 
 Backup speed is increased by not reprocessing files that are already part of
-existing archives and weren't modified. Normally, detecting file modifications
-will take inode information into consideration. This is problematic for files
-located on sshfs and similar network file systems which do not provide stable
-inode numbers, such files will always be considered modified. The
-``--ignore-inode`` flag can be used to prevent this and improve performance.
-This flag will reduce reliability of change detection however, with files
-considered unmodified as long as their size and modification time are unchanged.
+existing archives and weren't modified. The detection of unmodified files is
+done by comparing multiple file metadata values with previous values kept in
+the files cache.
+
+This comparison can operate in different modes as given by ``--files-cache``:
+
+- ctime,size,inode (default)
+- mtime,size,inode (default behaviour of borg versions older than 1.1.0rc4)
+- ctime,size (ignore the inode number)
+- mtime,size (ignore the inode number)
+- rechunk,ctime (all files are considered modified - rechunk, cache ctime)
+- rechunk,mtime (all files are considered modified - rechunk, cache mtime)
+- disabled (disable the files cache, all files considered modified - rechunk)
+
+inode number: better safety, but often unstable on network filesystems
+
+Normally, detecting file modifications will take inode information into
+consideration to improve the reliability of file change detection.
+This is problematic for files located on sshfs and similar network file
+systems which do not provide stable inode numbers, such files will always
+be considered modified. You can use modes without `inode` in this case to
+improve performance, but reliability of change detection might be reduced.
+
+ctime vs. mtime: safety vs. speed
+
+- ctime is a rather safe way to detect changes to a file (metadata and contents)
+  as it can not be set from userspace. But, a metadata-only change will already
+  update the ctime, so there might be some unnecessary chunking/hashing even
+  without content changes. Some filesystems do not support ctime (change time).
+- mtime usually works and only updates if file contents were changed. But mtime
+  can be arbitrarily set from userspace, e.g. to set mtime back to the same value
+  it had before a content change happened. This can be used maliciously as well as
+  well-meant, but in both cases mtime based cache modes can be problematic.
 
 The mount points of filesystems or filesystem snapshots should be the same for every
 creation of a new archive to ensure fast operation. This is because the file cache that
@@ -174,6 +215,12 @@ The ``--progress`` option shows (from left to right) Original, Compressed and De
 (O, C and D, respectively), then the Number of files (N) processed so far, followed by
 the currently processed path.
 
+When using ``--stats``, you will get some statistics about how much data was
+added - the "This Archive" deduplicated size there is most interesting as that is
+how much your repository will grow. Please note that the "All archives" stats refer to
+the state after creation. Also, the ``--stats`` and ``--dry-run`` options are mutually
+exclusive because the data is not actually compressed and deduplicated during a dry run.
+
 See the output of the "borg help patterns" command for more help on exclude patterns.
 See the output of the "borg help placeholders" command for more help on placeholders.
 

docs/usage/delete.rst.inc

@@ -6,7 +6,7 @@ borg delete
 -----------
 .. code-block:: none
 
-    borg [common options] delete [options] [TARGET]
+    borg [common options] delete [options] [TARGET] [ARCHIVE...]
 
 .. only:: html
 
@@ -17,6 +17,8 @@ borg delete
     +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
     |                                                                             | ``TARGET``                            | archive or repository to delete                                                                                                                        |
     +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                                             | ``ARCHIVE``                           | archives to delete                                                                                                                                     |
+    +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
     | **optional arguments**                                                                                                                                                                                                                                                       |
     +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
     |                                                                             | ``-s``, ``--stats``                   | print statistics for the deleted archive                                                                                                               |
@@ -56,6 +58,8 @@ borg delete
 
     TARGET
         archive or repository to delete
+    ARCHIVE
+        archives to delete
 
 
     optional arguments
@@ -81,4 +85,9 @@ Description
 
 This command deletes an archive from the repository or the complete repository.
 Disk space is reclaimed accordingly. If you delete the complete repository, the
-local cache for it (if any) is also deleted.
+local cache for it (if any) is also deleted.
+
+When using ``--stats``, you will get some statistics about how much data was
+deleted - the "Deleted data" deduplicated size there is most interesting as
+that is how much your repository will shrink.
+Please note that the "All archives" stats refer to the state after deletion.

docs/usage/diff.rst.inc

@@ -12,43 +12,37 @@ borg diff
 
     .. class:: borg-options-table
 
-    +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+
-    | **positional arguments**                                                                                                                                                                                                          |
-    +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+
-    |                                                       | ``REPO_ARCHIVE1``                             | repository location and ARCHIVE1 name                                                                                     |
-    +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+
-    |                                                       | ``ARCHIVE2``                                  | ARCHIVE2 name (no repository location allowed)                                                                            |
-    +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+
-    |                                                       | ``PATH``                                      | paths of items inside the archives to compare; patterns are supported                                                     |
-    +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+
-    | **optional arguments**                                                                                                                                                                                                            |
-    +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+
-    |                                                       | ``--numeric-owner``                           | only consider numeric user and group identifiers                                                                          |
-    +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+
-    |                                                       | ``--same-chunker-params``                     | Override check of chunker parameters.                                                                                     |
-    +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+
-    |                                                       | ``--sort``                                    | Sort the output lines by file path.                                                                                       |
-    +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+
-    | .. class:: borg-common-opt-ref                                                                                                                                                                                                    |
-    |                                                                                                                                                                                                                                   |
-    | :ref:`common_options`                                                                                                                                                                                                             |
-    +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+
-    | **Exclusion options**                                                                                                                                                                                                             |
-    +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+
-    |                                                       | ``-e PATTERN``, ``--exclude PATTERN``         | exclude paths matching PATTERN                                                                                            |
-    +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+
-    |                                                       | ``--exclude-from EXCLUDEFILE``                | read exclude patterns from EXCLUDEFILE, one per line                                                                      |
-    +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+
-    |                                                       | ``--pattern PATTERN``                         | experimental: include/exclude paths matching PATTERN                                                                      |
-    +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+
-    |                                                       | ``--patterns-from PATTERNFILE``               | experimental: read include/exclude patterns from PATTERNFILE, one per line                                                |
-    +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+
-    |                                                       | ``--exclude-caches``                          | exclude directories that contain a CACHEDIR.TAG file (http://www.brynosaurus.com/cachedir/spec.html)                      |
-    +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+
-    |                                                       | ``--exclude-if-present NAME``                 | exclude directories that are tagged by containing a filesystem object with the given NAME                                 |
-    +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+
-    |                                                       | ``--keep-exclude-tags``, ``--keep-tag-files`` | if tag objects are specified with ``--exclude-if-present``, don't omit the tag objects themselves from the backup archive |
-    +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+
+    +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+
+    | **positional arguments**                                                                                                                                                   |
+    +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+
+    |                                                       | ``REPO_ARCHIVE1``                     | repository location and ARCHIVE1 name                                      |
+    +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+
+    |                                                       | ``ARCHIVE2``                          | ARCHIVE2 name (no repository location allowed)                             |
+    +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+
+    |                                                       | ``PATH``                              | paths of items inside the archives to compare; patterns are supported      |
+    +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+
+    | **optional arguments**                                                                                                                                                     |
+    +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+
+    |                                                       | ``--numeric-owner``                   | only consider numeric user and group identifiers                           |
+    +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+
+    |                                                       | ``--same-chunker-params``             | Override check of chunker parameters.                                      |
+    +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+
+    |                                                       | ``--sort``                            | Sort the output lines by file path.                                        |
+    +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+
+    | .. class:: borg-common-opt-ref                                                                                                                                             |
+    |                                                                                                                                                                            |
+    | :ref:`common_options`                                                                                                                                                      |
+    +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+
+    | **Exclusion options**                                                                                                                                                      |
+    +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+
+    |                                                       | ``-e PATTERN``, ``--exclude PATTERN`` | exclude paths matching PATTERN                                             |
+    +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+
+    |                                                       | ``--exclude-from EXCLUDEFILE``        | read exclude patterns from EXCLUDEFILE, one per line                       |
+    +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+
+    |                                                       | ``--pattern PATTERN``                 | experimental: include/exclude paths matching PATTERN                       |
+    +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+
+    |                                                       | ``--patterns-from PATTERNFILE``       | experimental: read include/exclude patterns from PATTERNFILE, one per line |
+    +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+
 
     .. raw:: html
 
@@ -82,9 +76,6 @@ borg diff
         --exclude-from EXCLUDEFILE        read exclude patterns from EXCLUDEFILE, one per line
         --pattern PATTERN                 experimental: include/exclude paths matching PATTERN
         --patterns-from PATTERNFILE       experimental: read include/exclude patterns from PATTERNFILE, one per line
-        --exclude-caches                  exclude directories that contain a CACHEDIR.TAG file (http://www.brynosaurus.com/cachedir/spec.html)
-        --exclude-if-present NAME         exclude directories that are tagged by containing a filesystem object with the given NAME
-        --keep-exclude-tags, --keep-tag-files    if tag objects are specified with ``--exclude-if-present``, don't omit the tag objects themselves from the backup archive
 
 
 Description

docs/usage/extract.rst.inc

@@ -27,6 +27,8 @@ borg extract
     +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------------+
     |                                                       | ``--numeric-owner``                   | only obey numeric user and group identifiers                                                              |
     +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------------+
+    |                                                       | ``--nobsdflags``                      | do not extract/set bsdflags (e.g. NODUMP, IMMUTABLE)                                                      |
+    +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------------+
     |                                                       | ``--stdout``                          | write all extracted data to stdout                                                                        |
     +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------------+
     |                                                       | ``--sparse``                          | create holes in output sparse file from all-zero chunks                                                   |
@@ -68,6 +70,7 @@ borg extract
         --list            output verbose list of items (files, dirs, ...)
         -n, --dry-run     do not actually change any files
         --numeric-owner    only obey numeric user and group identifiers
+        --nobsdflags      do not extract/set bsdflags (e.g. NODUMP, IMMUTABLE)
         --stdout          write all extracted data to stdout
         --sparse          create holes in output sparse file from all-zero chunks
 

docs/usage/help.rst.inc

@@ -124,10 +124,12 @@ Examples::
     may specify the backup roots (starting points) and patterns for inclusion/exclusion.
     A root path starts with the prefix `R`, followed by a path (a plain path, not a
     file pattern). An include rule starts with the prefix +, an exclude rule starts
-    with the prefix -, both followed by a pattern.
+    with the prefix -, an exclude-norecurse rule starts with !, all followed by a pattern.
     Inclusion patterns are useful to include paths that are contained in an excluded
     path. The first matching pattern is used so if an include pattern matches before
-    an exclude pattern, the file is backed up.
+    an exclude pattern, the file is backed up. If an exclude-norecurse pattern matches
+    a directory, it won't recurse into it and won't discover any potential matches for
+    include rules below that directory.
 
     Note that the default pattern style for ``--pattern`` and ``--patterns-from`` is
     shell style (`sh:`), so those patterns behave similar to rsync include/exclude
@@ -169,11 +171,11 @@ placeholders:
 
 {now}
     The current local date and time, by default in ISO-8601 format.
-    You can also supply your own `format string <https://docs.python.org/3.4/library/datetime.html#strftime-and-strptime-behavior>`_, e.g. {now:%Y-%m-%d_%H:%M:%S}
+    You can also supply your own `format string <https://docs.python.org/3.5/library/datetime.html#strftime-and-strptime-behavior>`_, e.g. {now:%Y-%m-%d_%H:%M:%S}
 
 {utcnow}
     The current UTC date and time, by default in ISO-8601 format.
-    You can also supply your own `format string <https://docs.python.org/3.4/library/datetime.html#strftime-and-strptime-behavior>`_, e.g. {utcnow:%Y-%m-%d_%H:%M:%S}
+    You can also supply your own `format string <https://docs.python.org/3.5/library/datetime.html#strftime-and-strptime-behavior>`_, e.g. {utcnow:%Y-%m-%d_%H:%M:%S}
 
 {user}
     The user name (or UID, if no name is available) of the user running borg.

docs/usage/info.rst.inc

@@ -77,6 +77,11 @@ up to the deduplicated size of the repository ("all archives"), because the two
 are meaning different things:
 
 This archive / deduplicated size = amount of data stored ONLY for this archive
-                                 = unique chunks of this archive.
+= unique chunks of this archive.
 All archives / deduplicated size = amount of data stored in the repo
-                                 = all chunks in the repository.
+= all chunks in the repository.
+
+Borg archives can only contain a limited amount of file metadata.
+The size of an archive relative to this limit depends on a number of factors,
+mainly the number of files, the lengths of paths and other metadata stored for files.
+This is shown as *utilization of maximum supported archive size*.

docs/usage/key_change-passphrase.rst.inc

@@ -43,4 +43,9 @@ Description
 ~~~~~~~~~~~
 
 The key files used for repository encryption are optionally passphrase
-protected. This command can be used to change this passphrase.
+protected. This command can be used to change this passphrase.
+
+Please note that this command only changes the passphrase, but not any
+secret protected by it (like e.g. encryption/MAC keys or chunker seed).
+Thus, changing the passphrase after passphrase and borg key got compromised
+does not protect future (nor past) backups to the same repository.

docs/usage/key_export.rst.inc

@@ -59,11 +59,15 @@ Description
 
 If repository encryption is used, the repository is inaccessible
 without the key. This command allows to backup this essential key.
+Note that the backup produced does not include the passphrase itself
+(i.e. the exported key stays encrypted). In order to regain access to a
+repository, one needs both the exported key and the original passphrase.
 
-There are two backup formats. The normal backup format is suitable for
+There are three backup formats. The normal backup format is suitable for
 digital storage as a file. The ``--paper`` backup format is optimized
 for printing and typing in while importing, with per line checks to
-reduce problems with manual input.
+reduce problems with manual input. The ``--qr-html`` creates a printable
+HTML template with a QR code and a copy of the ``--paper``-formatted key.
 
 For repositories using keyfile encryption the key is saved locally
 on the system that is capable of doing backups. To guard against loss

docs/usage/list.rst.inc

@@ -23,7 +23,7 @@ borg list
     +-----------------------------------------------------------------------------+-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
     |                                                                             | ``--short``                                   | only print file/directory names, nothing else                                                                                                                                                                                                                            |
     +-----------------------------------------------------------------------------+-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
-    |                                                                             | ``--format FORMAT``, ``--list-format FORMAT`` | specify format for file listing (default: "{mode} {user:6} {group:6} {size:8d} {isomtime} {path}{extra}{NL}")                                                                                                                                                            |
+    |                                                                             | ``--format FORMAT``, ``--list-format FORMAT`` | specify format for file listing (default: "{mode} {user:6} {group:6} {size:8d} {mtime} {path}{extra}{NL}")                                                                                                                                                               |
     +-----------------------------------------------------------------------------+-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
     |                                                                             | ``--json``                                    | Only valid for listing repository contents. Format output as JSON. The form of ``--format`` is ignored, but keys used in it are added to the JSON output. Some keys are always present. Note: JSON can only represent text. A "barchive" key is therefore not available. |
     +-----------------------------------------------------------------------------+-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
@@ -55,12 +55,6 @@ borg list
     +-----------------------------------------------------------------------------+-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
     |                                                                             | ``--patterns-from PATTERNFILE``               | experimental: read include/exclude patterns from PATTERNFILE, one per line                                                                                                                                                                                               |
     +-----------------------------------------------------------------------------+-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
-    |                                                                             | ``--exclude-caches``                          | exclude directories that contain a CACHEDIR.TAG file (http://www.brynosaurus.com/cachedir/spec.html)                                                                                                                                                                     |
-    +-----------------------------------------------------------------------------+-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
-    |                                                                             | ``--exclude-if-present NAME``                 | exclude directories that are tagged by containing a filesystem object with the given NAME                                                                                                                                                                                |
-    +-----------------------------------------------------------------------------+-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
-    |                                                                             | ``--keep-exclude-tags``, ``--keep-tag-files`` | if tag objects are specified with ``--exclude-if-present``, don't omit the tag objects themselves from the backup archive                                                                                                                                                |
-    +-----------------------------------------------------------------------------+-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 
     .. raw:: html
 
@@ -80,7 +74,7 @@ borg list
 
     optional arguments
         --short     only print file/directory names, nothing else
-        --format FORMAT, --list-format FORMAT    specify format for file listing (default: "{mode} {user:6} {group:6} {size:8d} {isomtime} {path}{extra}{NL}")
+        --format FORMAT, --list-format FORMAT    specify format for file listing (default: "{mode} {user:6} {group:6} {size:8d} {mtime} {path}{extra}{NL}")
         --json      Only valid for listing repository contents. Format output as JSON. The form of ``--format`` is ignored, but keys used in it are added to the JSON output. Some keys are always present. Note: JSON can only represent text. A "barchive" key is therefore not available.
         --json-lines    Only valid for listing archive contents. Format output as JSON Lines. The form of ``--format`` is ignored, but keys used in it are added to the JSON output. Some keys are always present. Note: JSON can only represent text. A "bpath" key is therefore not available.
 
@@ -101,9 +95,6 @@ borg list
         --exclude-from EXCLUDEFILE        read exclude patterns from EXCLUDEFILE, one per line
         --pattern PATTERN                 experimental: include/exclude paths matching PATTERN
         --patterns-from PATTERNFILE       experimental: read include/exclude patterns from PATTERNFILE, one per line
-        --exclude-caches                  exclude directories that contain a CACHEDIR.TAG file (http://www.brynosaurus.com/cachedir/spec.html)
-        --exclude-if-present NAME         exclude directories that are tagged by containing a filesystem object with the given NAME
-        --keep-exclude-tags, --keep-tag-files    if tag objects are specified with ``--exclude-if-present``, don't omit the tag objects themselves from the backup archive
 
 
 Description
@@ -127,15 +118,15 @@ The following keys are available for ``--format``:
 
 Keys for listing repository archives:
 
-- name: archive name interpreted as text (might be missing non-text characters, see barchive)
 - archive: archive name interpreted as text (might be missing non-text characters, see barchive)
+- name: alias of "archive"
 - barchive: verbatim archive name, can contain any character except NUL
 - comment: archive comment interpreted as text (might be missing non-text characters, see bcomment)
 - bcomment: verbatim archive comment, can contain any character except NUL
 - id: internal ID of the archive
 
-- time: time (start) of creation of the archive
 - start: time (start) of creation of the archive
+- time: alias of "start"
 - end: time (end) of creation of the archive
 
 

docs/usage/mount.rst.inc

@@ -6,7 +6,7 @@ borg mount
 ----------
 .. code-block:: none
 
-    borg [common options] mount [options] REPOSITORY_OR_ARCHIVE MOUNTPOINT
+    borg [common options] mount [options] REPOSITORY_OR_ARCHIVE MOUNTPOINT [PATH...]
 
 .. only:: html
 
@@ -19,6 +19,8 @@ borg mount
     +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
     |                                                                             | ``MOUNTPOINT``                        | where to mount filesystem                                                                                                                              |
     +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                                             | ``PATH``                              | paths to extract; patterns are supported                                                                                                               |
+    +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
     | **optional arguments**                                                                                                                                                                                                                                                       |
     +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
     |                                                                             | ``-f``, ``--foreground``              | stay in foreground, do not daemonize                                                                                                                   |
@@ -41,6 +43,18 @@ borg mount
     +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
     |                                                                             | ``--last N``                          | consider last N archives after other filters were applied                                                                                              |
     +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
+    | **Exclusion options**                                                                                                                                                                                                                                                        |
+    +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                                             | ``-e PATTERN``, ``--exclude PATTERN`` | exclude paths matching PATTERN                                                                                                                         |
+    +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                                             | ``--exclude-from EXCLUDEFILE``        | read exclude patterns from EXCLUDEFILE, one per line                                                                                                   |
+    +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                                             | ``--pattern PATTERN``                 | experimental: include/exclude paths matching PATTERN                                                                                                   |
+    +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                                             | ``--patterns-from PATTERNFILE``       | experimental: read include/exclude patterns from PATTERNFILE, one per line                                                                             |
+    +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                                             | ``--strip-components NUMBER``         | Remove the specified number of leading path elements. Paths with fewer elements will be silently skipped.                                              |
+    +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
 
     .. raw:: html
 
@@ -56,6 +70,8 @@ borg mount
         repository/archive to mount
     MOUNTPOINT
         where to mount filesystem
+    PATH
+        paths to extract; patterns are supported
 
 
     optional arguments
@@ -74,6 +90,14 @@ borg mount
         --last N                          consider last N archives after other filters were applied
 
 
+    Exclusion options
+        -e PATTERN, --exclude PATTERN     exclude paths matching PATTERN
+        --exclude-from EXCLUDEFILE        read exclude patterns from EXCLUDEFILE, one per line
+        --pattern PATTERN                 experimental: include/exclude paths matching PATTERN
+        --patterns-from PATTERNFILE       experimental: read include/exclude patterns from PATTERNFILE, one per line
+        --strip-components NUMBER         Remove the specified number of leading path elements. Paths with fewer elements will be silently skipped.
+
+
 Description
 ~~~~~~~~~~~
 

docs/usage/prune.rst.inc

@@ -134,4 +134,9 @@ negative number of archives to keep means that there is no limit.
 
 The ``--keep-last N`` option is doing the same as ``--keep-secondly N`` (and it will
 keep the last N archives under the assumption that you do not create more than one
-backup archive in the same second).
+backup archive in the same second).
+
+When using ``--stats``, you will get some statistics about how much data was
+deleted - the "Deleted data" deduplicated size there is most interesting as
+that is how much your repository will shrink.
+Please note that the "All archives" stats refer to the state after pruning.

docs/usage/recreate.rst.inc

@@ -150,4 +150,15 @@ With ``--target`` the original archive is not replaced, instead a new archive is
 When rechunking space usage can be substantial, expect at least the entire
 deduplicated size of the archives using the previous chunker params.
 When recompressing expect approx. (throughput / checkpoint-interval) in space usage,
-assuming all chunks are recompressed.
+assuming all chunks are recompressed.
+
+If you recently ran borg check --repair and it had to fix lost chunks with all-zero
+replacement chunks, please first run another backup for the same data and re-run
+borg check --repair afterwards to heal any archives that had lost chunks which are
+still generated from the input data.
+
+Important: running borg recreate to re-chunk will remove the chunks_healthy
+metadata of all items with replacement chunks, so healing will not be possible
+any more after re-chunking (it is also unlikely it would ever work: due to the
+change of chunking parameters, the missing chunk likely will never be seen again
+even if you still have the data that produced it).

docs/usage/upgrade.rst.inc

@@ -130,13 +130,13 @@ make sure the cache files are also removed:
 
     borg delete borg
 
-Unless ``--inplace`` is specified, the upgrade process first
-creates a backup copy of the repository, in
-REPOSITORY.before-upgrade-DATETIME, using hardlinks. This takes
-longer than in place upgrades, but is much safer and gives
-progress information (as opposed to ``cp -al``). Once you are
-satisfied with the conversion, you can safely destroy the
-backup copy.
+Unless ``--inplace`` is specified, the upgrade process first creates a backup
+copy of the repository, in REPOSITORY.before-upgrade-DATETIME, using hardlinks.
+This requires that the repository and its parent directory reside on same
+filesystem so the hardlink copy can work.
+This takes longer than in place upgrades, but is much safer and gives
+progress information (as opposed to ``cp -al``). Once you are satisfied
+with the conversion, you can safely destroy the backup copy.
 
 WARNING: Running the upgrade in place will make the current
 copy unusable with older version, with no way of going back

docs/usage/with-lock.rst.inc

@@ -60,5 +60,6 @@ code as borg's return code.
 .. note::
 
     If you copy a repository with the lock held, the lock will be present in
-    the copy, obviously. Thus, before using borg on the copy, you need to
-    use "borg break-lock" on it.
+    the copy. Thus, before using borg on the copy from a different host,
+    you need to use "borg break-lock" on the copied repository, because
+    Borg is cautious and does not automatically remove stale locks made by a different host.