Merge pull request #2974 from borgbackup/rel110rc2

Release 1.1.0rc2

docs/changes.rst

@@ -131,8 +131,8 @@ The best check that everything is ok is to run a dry-run extraction::
 Changelog
 =========
 
-Version 1.1.0rc2 (not released yet)
-------------------------------------
+Version 1.1.0rc2 (2017-08-28)
+-----------------------------
 
 Compatibility notes:
 
@@ -143,6 +143,42 @@ Compatibility notes:
   and the default is now "mtime".
   "isomtime" is now a real ISO-8601 format ("T" between date and time, not a space).
 
+New features:
+
+- None.
+
+Fixes:
+
+- list: fix weird mixup of mtime/isomtime
+- create --timestamp: set start time, #2957
+- ignore corrupt files cache, #2939
+- migrate locks to child PID when daemonize is used
+- fix exitcode of borg serve, #2910
+- only compare contents when chunker params match, #2899
+- umount: try fusermount, then try umount, #2863
+
+Other changes:
+
+- JSON: use a more standard ISO 8601 datetime format, #2376
+- cache: write_archive_index: truncate_and_unlink on error, #2628
+- detect non-upgraded Attic repositories, #1933
+- delete various nogil and threading related lines
+- coala / pylint related improvements
+- docs:
+
+  - renew asciinema/screencasts, #669
+  - create: document exclusion through nodump, #2949
+  - minor formatting fixes
+  - tar: tarpipe example
+  - improve "with-lock" and "info" docs, #2869
+  - detail how to use macOS/GNOME/KDE keyrings for repo passwords, #392
+- travis: only short-circuit docs-only changes for pull requests
+- vagrant:
+
+  - netbsd: bash is already installed
+  - fix netbsd version in PKG_PATH
+  - add exe location to PATH when we build an exe
+
 
 Version 1.1.0rc1 (2017-07-24)
 -----------------------------

docs/man/borg-benchmark-crud.1

@@ -1,6 +1,6 @@
 .\" Man page generated from reStructuredText.
 .
-.TH BORG-BENCHMARK-CRUD 1 "2017-06-18" "" "borg backup tool"
+.TH BORG-BENCHMARK-CRUD 1 "2017-08-27" "" "borg backup tool"
 .SH NAME
 borg-benchmark-crud \- Benchmark Create, Read, Update, Delete for archives.
 .
@@ -32,7 +32,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 ..
 .SH SYNOPSIS
 .sp
-borg [common options] benchmark crud <options> REPO PATH
+borg [common options] benchmark crud [options] REPO PATH
 .SH DESCRIPTION
 .sp
 This command benchmarks borg CRUD (create, read, update, delete) operations.

docs/man/borg-benchmark.1

@@ -1,6 +1,6 @@
 .\" Man page generated from reStructuredText.
 .
-.TH BORG-BENCHMARK 1 "2017-06-18" "" "borg backup tool"
+.TH BORG-BENCHMARK 1 "2017-08-27" "" "borg backup tool"
 .SH NAME
 borg-benchmark \- benchmark command
 .

docs/man/borg-break-lock.1

@@ -1,6 +1,6 @@
 .\" Man page generated from reStructuredText.
 .
-.TH BORG-BREAK-LOCK 1 "2017-06-18" "" "borg backup tool"
+.TH BORG-BREAK-LOCK 1 "2017-08-27" "" "borg backup tool"
 .SH NAME
 borg-break-lock \- Break the repository lock (e.g. in case it was left by a dead borg.
 .
@@ -32,7 +32,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 ..
 .SH SYNOPSIS
 .sp
-borg [common options] break\-lock <options> REPOSITORY
+borg [common options] break\-lock [options] [REPOSITORY]
 .SH DESCRIPTION
 .sp
 This command breaks the repository and cache locks.

docs/man/borg-change-passphrase.1

@@ -1,6 +1,6 @@
 .\" Man page generated from reStructuredText.
 .
-.TH BORG-CHANGE-PASSPHRASE 1 "2017-06-18" "" "borg backup tool"
+.TH BORG-CHANGE-PASSPHRASE 1 "2017-08-27" "" "borg backup tool"
 .SH NAME
 borg-change-passphrase \- Change repository key file passphrase
 .
@@ -32,7 +32,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 ..
 .SH SYNOPSIS
 .sp
-borg [common options] change\-passphrase <options> REPOSITORY
+borg [common options] change\-passphrase [options] [REPOSITORY]
 .SH DESCRIPTION
 .sp
 The key files used for repository encryption are optionally passphrase

docs/man/borg-check.1

@@ -1,6 +1,6 @@
 .\" Man page generated from reStructuredText.
 .
-.TH BORG-CHECK 1 "2017-06-18" "" "borg backup tool"
+.TH BORG-CHECK 1 "2017-08-27" "" "borg backup tool"
 .SH NAME
 borg-check \- Check repository consistency
 .
@@ -32,7 +32,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 ..
 .SH SYNOPSIS
 .sp
-borg [common options] check <options> REPOSITORY_OR_ARCHIVE
+borg [common options] check [options] [REPOSITORY_OR_ARCHIVE]
 .SH DESCRIPTION
 .sp
 The check command verifies the consistency of a repository and the corresponding archives.
@@ -121,16 +121,16 @@ attempt to repair any inconsistencies found
 .B \-\-save\-space
 work slower, but using less space
 .UNINDENT
-.SS filters
+.SS Archive filters
 .INDENT 0.0
 .TP
-.B \-P\fP,\fB  \-\-prefix
+.BI \-P \ PREFIX\fP,\fB \ \-\-prefix \ PREFIX
 only consider archive names starting with this prefix.
 .TP
-.B \-a\fP,\fB  \-\-glob\-archives
+.BI \-a \ GLOB\fP,\fB \ \-\-glob\-archives \ GLOB
 only consider archive names matching the glob. sh: rules apply, see "borg help patterns". \fB\-\-prefix\fP and \fB\-\-glob\-archives\fP are mutually exclusive.
 .TP
-.B \-\-sort\-by
+.BI \-\-sort\-by \ KEYS
 Comma\-separated list of sorting keys; valid keys are: timestamp, name, id; default is: timestamp
 .TP
 .BI \-\-first \ N

docs/man/borg-common.1

@@ -1,6 +1,6 @@
 .\" Man page generated from reStructuredText.
 .
-.TH BORG-COMMON 1 "2017-06-18" "" "borg backup tool"
+.TH BORG-COMMON 1 "2017-08-27" "" "borg backup tool"
 .SH NAME
 borg-common \- Common options of Borg commands
 .
@@ -60,8 +60,8 @@ show progress information
 .B \-\-log\-json
 Output one JSON object per log line instead of formatted text.
 .TP
-.BI \-\-lock\-wait \ N
-wait for the lock, but max. N seconds (default: 1).
+.BI \-\-lock\-wait \ SECONDS
+wait at most SECONDS for acquiring a repository/cache lock (default: 1).
 .TP
 .B \-\-show\-version
 show/log the borg version
@@ -78,7 +78,7 @@ set umask to M (local and remote, default: 0077)
 .BI \-\-remote\-path \ PATH
 use PATH as borg executable on the remote (default: "borg")
 .TP
-.BI \-\-remote\-ratelimit \ rate
+.BI \-\-remote\-ratelimit \ RATE
 set remote network upload rate limit in kiByte/s (default: 0=unlimited)
 .TP
 .B \-\-consider\-part\-files

docs/man/borg-compression.1

@@ -1,6 +1,6 @@
 .\" Man page generated from reStructuredText.
 .
-.TH BORG-COMPRESSION 1 "2017-06-18" "" "borg backup tool"
+.TH BORG-COMPRESSION 1 "2017-08-27" "" "borg backup tool"
 .SH NAME
 borg-compression \- Details regarding compression
 .

docs/man/borg-create.1

@@ -1,6 +1,6 @@
 .\" Man page generated from reStructuredText.
 .
-.TH BORG-CREATE 1 "2017-06-18" "" "borg backup tool"
+.TH BORG-CREATE 1 "2017-08-27" "" "borg backup tool"
 .SH NAME
 borg-create \- Create new archive
 .
@@ -32,7 +32,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 ..
 .SH SYNOPSIS
 .sp
-borg [common options] create <options> ARCHIVE PATH
+borg [common options] create [options] ARCHIVE [PATH...]
 .SH DESCRIPTION
 .sp
 This command creates a backup archive containing all files found while recursively
@@ -53,10 +53,14 @@ checkpoints and treated in special ways.
 In the archive name, you may use the following placeholders:
 {now}, {utcnow}, {fqdn}, {hostname}, {user} and some others.
 .sp
-To speed up pulling backups over sshfs and similar network file systems which do
-not provide correct inode information the \fB\-\-ignore\-inode\fP flag can be used. This
-potentially decreases reliability of change detection, while avoiding always reading
-all files on these file systems.
+Backup speed is increased by not reprocessing files that are already part of
+existing archives and weren\(aqt modified. Normally, detecting file modifications
+will take inode information into consideration. This is problematic for files
+located on sshfs and similar network file systems which do not provide stable
+inode numbers, such files will always be considered modified. The
+\fB\-\-ignore\-inode\fP flag can be used to prevent this and improve performance.
+This flag will reduce reliability of change detection however, with files
+considered unmodified as long as their size and modification time are unchanged.
 .sp
 The mount points of filesystems or filesystem snapshots should be the same for every
 creation of a new archive to ensure fast operation. This is because the file cache that
@@ -94,13 +98,13 @@ print statistics for the created archive
 output verbose list of items (files, dirs, ...)
 .TP
 .BI \-\-filter \ STATUSCHARS
-only display items with the given status characters
+only display items with the given status characters (see description)
 .TP
 .B \-\-json
-output stats as JSON (implies \-\-stats)
+output stats as JSON. Implies \fB\-\-stats\fP\&.
 .TP
 .B \-\-no\-cache\-sync
-experimental: do not synchronize the cache. Implies \-\-no\-files\-cache.
+experimental: do not synchronize the cache. Implies \fB\-\-no\-files\-cache\fP\&.
 .UNINDENT
 .SS Exclusion options
 .INDENT 0.0
@@ -111,6 +115,12 @@ exclude paths matching PATTERN
 .BI \-\-exclude\-from \ EXCLUDEFILE
 read exclude patterns from EXCLUDEFILE, one per line
 .TP
+.BI \-\-pattern \ PATTERN
+experimental: include/exclude paths matching PATTERN
+.TP
+.BI \-\-patterns\-from \ PATTERNFILE
+experimental: read include/exclude patterns from PATTERNFILE, one per line
+.TP
 .B \-\-exclude\-caches
 exclude directories that contain a CACHEDIR.TAG file (\fI\%http://www.brynosaurus.com/cachedir/spec.html\fP)
 .TP
@@ -118,13 +128,7 @@ exclude directories that contain a CACHEDIR.TAG file (\fI\%http://www.brynosauru
 exclude directories that are tagged by containing a filesystem object with the given NAME
 .TP
 .B \-\-keep\-exclude\-tags\fP,\fB  \-\-keep\-tag\-files
-if tag objects are specified with \-\-exclude\-if\-present, don\(aqt omit the tag objects themselves from the backup archive
-.TP
-.BI \-\-pattern \ PATTERN
-experimental: include/exclude paths matching PATTERN
-.TP
-.BI \-\-patterns\-from \ PATTERNFILE
-experimental: read include/exclude patterns from PATTERNFILE, one per line
+if tag objects are specified with \fB\-\-exclude\-if\-present\fP, don\(aqt omit the tag objects themselves from the backup archive
 .UNINDENT
 .SS Filesystem options
 .INDENT 0.0
@@ -154,7 +158,7 @@ open and read block and char device files as well as FIFOs as if they were regul
 add a comment text to the archive
 .TP
 .BI \-\-timestamp \ TIMESTAMP
-manually specify the archive creation date/time (UTC, yyyy\-mm\-ddThh:mm:ss format). alternatively, give a reference file/directory.
+manually specify the archive creation date/time (UTC, yyyy\-mm\-ddThh:mm:ss format). Alternatively, give a reference file/directory.
 .TP
 .BI \-c \ SECONDS\fP,\fB \ \-\-checkpoint\-interval \ SECONDS
 write checkpoint every SECONDS seconds (Default: 1800)
@@ -250,6 +254,9 @@ all of its contents will be omitted from the backup.  If, however, you wish to
 only include the objects specified by \fB\-\-exclude\-if\-present\fP in your backup,
 and not include any other contents of the containing folder, this can be enabled
 through using the \fB\-\-keep\-exclude\-tags\fP option.
+.sp
+Borg respects the nodump flag. Files flagged nodump will be marked as excluded (x)
+in \fB\-\-list\fP output.
 .SS Item flags
 .sp
 \fB\-\-list\fP outputs a list of all files, directories and other

docs/man/borg-delete.1

@@ -1,6 +1,6 @@
 .\" Man page generated from reStructuredText.
 .
-.TH BORG-DELETE 1 "2017-06-18" "" "borg backup tool"
+.TH BORG-DELETE 1 "2017-08-27" "" "borg backup tool"
 .SH NAME
 borg-delete \- Delete an existing repository or archives
 .
@@ -32,7 +32,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 ..
 .SH SYNOPSIS
 .sp
-borg [common options] delete <options> TARGET
+borg [common options] delete [options] [TARGET]
 .SH DESCRIPTION
 .sp
 This command deletes an archive from the repository or the complete repository.
@@ -53,25 +53,25 @@ archive or repository to delete
 .B \-s\fP,\fB  \-\-stats
 print statistics for the deleted archive
 .TP
-.B \-c\fP,\fB  \-\-cache\-only
+.B \-\-cache\-only
 delete only the local cache for the given repository
 .TP
 .B \-\-force
-force deletion of corrupted archives, use \-\-force \-\-force in case \-\-force does not work.
+force deletion of corrupted archives, use \fB\-\-force \-\-force\fP in case \fB\-\-force\fP does not work.
 .TP
 .B \-\-save\-space
 work slower, but using less space
 .UNINDENT
-.SS filters
+.SS Archive filters
 .INDENT 0.0
 .TP
-.B \-P\fP,\fB  \-\-prefix
+.BI \-P \ PREFIX\fP,\fB \ \-\-prefix \ PREFIX
 only consider archive names starting with this prefix.
 .TP
-.B \-a\fP,\fB  \-\-glob\-archives
+.BI \-a \ GLOB\fP,\fB \ \-\-glob\-archives \ GLOB
 only consider archive names matching the glob. sh: rules apply, see "borg help patterns". \fB\-\-prefix\fP and \fB\-\-glob\-archives\fP are mutually exclusive.
 .TP
-.B \-\-sort\-by
+.BI \-\-sort\-by \ KEYS
 Comma\-separated list of sorting keys; valid keys are: timestamp, name, id; default is: timestamp
 .TP
 .BI \-\-first \ N

docs/man/borg-diff.1

@@ -1,6 +1,6 @@
 .\" Man page generated from reStructuredText.
 .
-.TH BORG-DIFF 1 "2017-06-18" "" "borg backup tool"
+.TH BORG-DIFF 1 "2017-08-27" "" "borg backup tool"
 .SH NAME
 borg-diff \- Diff contents of two archives
 .
@@ -32,7 +32,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 ..
 .SH SYNOPSIS
 .sp
-borg [common options] diff <options> REPO_ARCHIVE1 ARCHIVE2 PATH
+borg [common options] diff [options] REPO_ARCHIVE1 ARCHIVE2 [PATH...]
 .SH DESCRIPTION
 .sp
 This command finds differences (file contents, user/group/mode) between archives.
@@ -87,6 +87,12 @@ exclude paths matching PATTERN
 .BI \-\-exclude\-from \ EXCLUDEFILE
 read exclude patterns from EXCLUDEFILE, one per line
 .TP
+.BI \-\-pattern \ PATTERN
+experimental: include/exclude paths matching PATTERN
+.TP
+.BI \-\-patterns\-from \ PATTERNFILE
+experimental: read include/exclude patterns from PATTERNFILE, one per line
+.TP
 .B \-\-exclude\-caches
 exclude directories that contain a CACHEDIR.TAG file (\fI\%http://www.brynosaurus.com/cachedir/spec.html\fP)
 .TP
@@ -94,13 +100,7 @@ exclude directories that contain a CACHEDIR.TAG file (\fI\%http://www.brynosauru
 exclude directories that are tagged by containing a filesystem object with the given NAME
 .TP
 .B \-\-keep\-exclude\-tags\fP,\fB  \-\-keep\-tag\-files
-if tag objects are specified with \-\-exclude\-if\-present, don\(aqt omit the tag objects themselves from the backup archive
-.TP
-.BI \-\-pattern \ PATTERN
-experimental: include/exclude paths matching PATTERN
-.TP
-.BI \-\-patterns\-from \ PATTERNFILE
-experimental: read include/exclude patterns from PATTERNFILE, one per line
+if tag objects are specified with \fB\-\-exclude\-if\-present\fP, don\(aqt omit the tag objects themselves from the backup archive
 .UNINDENT
 .SH EXAMPLES
 .INDENT 0.0

docs/man/borg-export-tar.1

@@ -1,6 +1,6 @@
 .\" Man page generated from reStructuredText.
 .
-.TH BORG-EXPORT-TAR 1 "2017-06-18" "" "borg backup tool"
+.TH BORG-EXPORT-TAR 1 "2017-08-27" "" "borg backup tool"
 .SH NAME
 borg-export-tar \- Export archive contents as a tarball
 .
@@ -32,7 +32,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 ..
 .SH SYNOPSIS
 .sp
-borg [common options] export\-tar <options> ARCHIVE FILE PATH
+borg [common options] export\-tar [options] ARCHIVE FILE [PATH...]
 .SH DESCRIPTION
 .sp
 This command creates a tarball from an archive.
@@ -95,6 +95,9 @@ filter program to pipe data through
 .TP
 .B \-\-list
 output verbose list of items (files, dirs, ...)
+.UNINDENT
+.SS Exclusion options
+.INDENT 0.0
 .TP
 .BI \-e \ PATTERN\fP,\fB \ \-\-exclude \ PATTERN
 exclude paths matching PATTERN
@@ -109,7 +112,7 @@ experimental: include/exclude paths matching PATTERN
 experimental: read include/exclude patterns from PATTERNFILE, one per line
 .TP
 .BI \-\-strip\-components \ NUMBER
-Remove the specified number of leading path elements. Pathnames with fewer elements will be silently skipped.
+Remove the specified number of leading path elements. Paths with fewer elements will be silently skipped.
 .UNINDENT
 .SH EXAMPLES
 .INDENT 0.0
@@ -129,6 +132,9 @@ $ borg export\-tar testrepo::linux \-\-tar\-filter="gzip \-9" Monday.tar.gz
 # export a gzipped tar, but instead of storing it on disk,
 # upload it to a remote site using curl.
 $ borg export\-tar ... \-\-tar\-filter="gzip" \- | curl \-\-data\-binary @\- https://somewhere/to/POST
+
+# remote extraction via "tarpipe"
+$ borg export\-tar /path/to/repo::Monday \- | ssh somewhere "cd extracted; tar x"
 .ft P
 .fi
 .UNINDENT

docs/man/borg-extract.1

@@ -1,6 +1,6 @@
 .\" Man page generated from reStructuredText.
 .
-.TH BORG-EXTRACT 1 "2017-06-18" "" "borg backup tool"
+.TH BORG-EXTRACT 1 "2017-08-27" "" "borg backup tool"
 .SH NAME
 borg-extract \- Extract archive contents
 .
@@ -32,7 +32,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 ..
 .SH SYNOPSIS
 .sp
-borg [common options] extract <options> ARCHIVE PATH
+borg [common options] extract [options] ARCHIVE [PATH...]
 .SH DESCRIPTION
 .sp
 This command extracts the contents of an archive. By default the entire
@@ -48,6 +48,14 @@ decrypting, decompressing.
 .sp
 \fB\-\-progress\fP can be slower than no progress display, since it makes one additional
 pass over the archive metadata.
+.sp
+\fBNOTE:\fP
+.INDENT 0.0
+.INDENT 3.5
+Currently, extract always writes into the current working directory ("."),
+so make sure you \fBcd\fP to the right place before calling \fBborg extract\fP\&.
+.UNINDENT
+.UNINDENT
 .SH OPTIONS
 .sp
 See \fIborg\-common(1)\fP for common options of Borg commands.
@@ -69,6 +77,18 @@ output verbose list of items (files, dirs, ...)
 .B \-n\fP,\fB  \-\-dry\-run
 do not actually change any files
 .TP
+.B \-\-numeric\-owner
+only obey numeric user and group identifiers
+.TP
+.B \-\-stdout
+write all extracted data to stdout
+.TP
+.B \-\-sparse
+create holes in output sparse file from all\-zero chunks
+.UNINDENT
+.SS Exclusion options
+.INDENT 0.0
+.TP
 .BI \-e \ PATTERN\fP,\fB \ \-\-exclude \ PATTERN
 exclude paths matching PATTERN
 .TP
@@ -81,17 +101,8 @@ experimental: include/exclude paths matching PATTERN
 .BI \-\-patterns\-from \ PATTERNFILE
 experimental: read include/exclude patterns from PATTERNFILE, one per line
 .TP
-.B \-\-numeric\-owner
-only obey numeric user and group identifiers
-.TP
 .BI \-\-strip\-components \ NUMBER
-Remove the specified number of leading path elements. Pathnames with fewer elements will be silently skipped.
-.TP
-.B \-\-stdout
-write all extracted data to stdout
-.TP
-.B \-\-sparse
-create holes in output sparse file from all\-zero chunks
+Remove the specified number of leading path elements. Paths with fewer elements will be silently skipped.
 .UNINDENT
 .SH EXAMPLES
 .INDENT 0.0
@@ -120,14 +131,6 @@ $ borg extract \-\-stdout /path/to/repo::my\-sdx | dd of=/dev/sdx bs=10M
 .fi
 .UNINDENT
 .UNINDENT
-.sp
-\fBNOTE:\fP
-.INDENT 0.0
-.INDENT 3.5
-Currently, extract always writes into the current working directory ("."),
-so make sure you \fBcd\fP to the right place before calling \fBborg extract\fP\&.
-.UNINDENT
-.UNINDENT
 .SH SEE ALSO
 .sp
 \fIborg\-common(1)\fP, \fIborg\-mount(1)\fP

docs/man/borg-info.1

@@ -1,6 +1,6 @@
 .\" Man page generated from reStructuredText.
 .
-.TH BORG-INFO 1 "2017-06-18" "" "borg backup tool"
+.TH BORG-INFO 1 "2017-08-27" "" "borg backup tool"
 .SH NAME
 borg-info \- Show archive details such as disk space used
 .
@@ -32,7 +32,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 ..
 .SH SYNOPSIS
 .sp
-borg [common options] info <options> REPOSITORY_OR_ARCHIVE
+borg [common options] info [options] [REPOSITORY_OR_ARCHIVE]
 .SH DESCRIPTION
 .sp
 This command displays detailed information about the specified archive or repository.
@@ -40,14 +40,16 @@ This command displays detailed information about the specified archive or reposi
 Please note that the deduplicated sizes of the individual archives do not add
 up to the deduplicated size of the repository ("all archives"), because the two
 are meaning different things:
-.INDENT 0.0
-.TP
-.B This archive / deduplicated size = amount of data stored ONLY for this archive
+.sp
+This archive / deduplicated size = amount of data stored ONLY for this archive
 = unique chunks of this archive.
-.TP
-.B All archives / deduplicated size = amount of data stored in the repo
+All archives / deduplicated size = amount of data stored in the repo
 = all chunks in the repository.
-.UNINDENT
+.sp
+Borg archives can only contain a limited amount of file metadata.
+The size of an archive relative to this limit depends on a number of factors,
+mainly the number of files, the lengths of paths and other metadata stored for files.
+This is shown as \fIutilization of maximum supported archive size\fP\&.
 .SH OPTIONS
 .sp
 See \fIborg\-common(1)\fP for common options of Borg commands.
@@ -63,16 +65,16 @@ archive or repository to display information about
 .B \-\-json
 format output as JSON
 .UNINDENT
-.SS filters
+.SS Archive filters
 .INDENT 0.0
 .TP
-.B \-P\fP,\fB  \-\-prefix
+.BI \-P \ PREFIX\fP,\fB \ \-\-prefix \ PREFIX
 only consider archive names starting with this prefix.
 .TP
-.B \-a\fP,\fB  \-\-glob\-archives
+.BI \-a \ GLOB\fP,\fB \ \-\-glob\-archives \ GLOB
 only consider archive names matching the glob. sh: rules apply, see "borg help patterns". \fB\-\-prefix\fP and \fB\-\-glob\-archives\fP are mutually exclusive.
 .TP
-.B \-\-sort\-by
+.BI \-\-sort\-by \ KEYS
 Comma\-separated list of sorting keys; valid keys are: timestamp, name, id; default is: timestamp
 .TP
 .BI \-\-first \ N
@@ -87,22 +89,58 @@ consider last N archives after other filters were applied
 .sp
 .nf
 .ft C
-$ borg info /path/to/repo::root\-2016\-02\-15
-Name: root\-2016\-02\-15
-Fingerprint: 57c827621f21b000a8d363c1e163cc55983822b3afff3a96df595077a660be50
+$ borg info /path/to/repo::2017\-06\-29T11:00\-srv
+Archive name: 2017\-06\-29T11:00\-srv
+Archive fingerprint: b2f1beac2bd553b34e06358afa45a3c1689320d39163890c5bbbd49125f00fe5
+Comment:
 Hostname: myhostname
 Username: root
-Time (start): Mon, 2016\-02\-15 19:36:29
-Time (end):   Mon, 2016\-02\-15 19:39:26
-Command line: /usr/local/bin/borg create \-\-list \-C zlib,6 /path/to/repo::root\-2016\-02\-15 / \-\-one\-file\-system
-Number of files: 38100
+Time (start): Thu, 2017\-06\-29 11:03:07
+Time (end): Thu, 2017\-06\-29 11:03:13
+Duration: 5.66 seconds
+Number of files: 17037
+Command line: /usr/sbin/borg create /path/to/repo::2017\-06\-29T11:00\-srv /srv
+Utilization of max. archive size: 0%
+\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
+                       Original size      Compressed size    Deduplicated size
+This archive:               12.53 GB             12.49 GB              1.62 kB
+All archives:              121.82 TB            112.41 TB            215.42 GB
+
+                       Unique chunks         Total chunks
+Chunk index:                 1015213            626934122
+
+$ borg info /path/to/repo \-\-last 1
+Archive name: 2017\-06\-29T11:00\-srv
+Archive fingerprint: b2f1beac2bd553b34e06358afa45a3c1689320d39163890c5bbbd49125f00fe5
+Comment:
+Hostname: myhostname
+Username: root
+Time (start): Thu, 2017\-06\-29 11:03:07
+Time (end): Thu, 2017\-06\-29 11:03:13
+Duration: 5.66 seconds
+Number of files: 17037
+Command line: /usr/sbin/borg create /path/to/repo::2017\-06\-29T11:00\-srv /srv
+Utilization of max. archive size: 0%
+\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
+                       Original size      Compressed size    Deduplicated size
+This archive:               12.53 GB             12.49 GB              1.62 kB
+All archives:              121.82 TB            112.41 TB            215.42 GB
+
+                       Unique chunks         Total chunks
+Chunk index:                 1015213            626934122
 
+$ borg info /path/to/repo
+Repository ID: d857ce5788c51272c61535062e89eac4e8ef5a884ffbe976e0af9d8765dedfa5
+Location: /path/to/repo
+Encrypted: Yes (repokey)
+Cache: /root/.cache/borg/d857ce5788c51272c61535062e89eac4e8ef5a884ffbe976e0af9d8765dedfa5
+Security dir: /root/.config/borg/security/d857ce5788c51272c61535062e89eac4e8ef5a884ffbe976e0af9d8765dedfa5
+\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
                        Original size      Compressed size    Deduplicated size
-This archive:                1.33 GB            613.25 MB            571.64 MB
-All archives:                1.63 GB            853.66 MB            584.12 MB
+All archives:              121.82 TB            112.41 TB            215.42 GB
 
                        Unique chunks         Total chunks
-Chunk index:                   36858                48844
+Chunk index:                 1015213            626934122
 .ft P
 .fi
 .UNINDENT

docs/man/borg-init.1

@@ -1,6 +1,6 @@
 .\" Man page generated from reStructuredText.
 .
-.TH BORG-INIT 1 "2017-06-18" "" "borg backup tool"
+.TH BORG-INIT 1 "2017-08-27" "" "borg backup tool"
 .SH NAME
 borg-init \- Initialize an empty repository
 .
@@ -32,7 +32,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 ..
 .SH SYNOPSIS
 .sp
-borg [common options] init <options> REPOSITORY
+borg [common options] init [options] [REPOSITORY]
 .SH DESCRIPTION
 .sp
 This command initializes an empty repository. A repository is a filesystem
@@ -178,13 +178,13 @@ repository to create
 .SS optional arguments
 .INDENT 0.0
 .TP
-.B \-e\fP,\fB  \-\-encryption
+.BI \-e \ MODE\fP,\fB \ \-\-encryption \ MODE
 select encryption key mode \fB(required)\fP
 .TP
 .B \-\-append\-only
 create an append\-only mode repository
 .TP
-.B \-\-storage\-quota
+.BI \-\-storage\-quota \ QUOTA
 Set storage quota of the new repository (e.g. 5G, 1.5T). Default: no quota.
 .UNINDENT
 .SH EXAMPLES

docs/man/borg-key-change-passphrase.1

@@ -1,6 +1,6 @@
 .\" Man page generated from reStructuredText.
 .
-.TH BORG-KEY-CHANGE-PASSPHRASE 1 "2017-06-18" "" "borg backup tool"
+.TH BORG-KEY-CHANGE-PASSPHRASE 1 "2017-08-27" "" "borg backup tool"
 .SH NAME
 borg-key-change-passphrase \- Change repository key file passphrase
 .
@@ -32,7 +32,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 ..
 .SH SYNOPSIS
 .sp
-borg [common options] key change\-passphrase <options> REPOSITORY
+borg [common options] key change\-passphrase [options] [REPOSITORY]
 .SH DESCRIPTION
 .sp
 The key files used for repository encryption are optionally passphrase

docs/man/borg-key-export.1

@@ -1,6 +1,6 @@
 .\" Man page generated from reStructuredText.
 .
-.TH BORG-KEY-EXPORT 1 "2017-06-18" "" "borg backup tool"
+.TH BORG-KEY-EXPORT 1 "2017-08-27" "" "borg backup tool"
 .SH NAME
 borg-key-export \- Export the repository key for backup
 .
@@ -32,7 +32,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 ..
 .SH SYNOPSIS
 .sp
-borg [common options] key export <options> REPOSITORY PATH
+borg [common options] key export [options] [REPOSITORY] [PATH]
 .SH DESCRIPTION
 .sp
 If repository encryption is used, the repository is inaccessible

docs/man/borg-key-import.1

@@ -1,6 +1,6 @@
 .\" Man page generated from reStructuredText.
 .
-.TH BORG-KEY-IMPORT 1 "2017-06-18" "" "borg backup tool"
+.TH BORG-KEY-IMPORT 1 "2017-08-27" "" "borg backup tool"
 .SH NAME
 borg-key-import \- Import the repository key from backup
 .
@@ -32,7 +32,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 ..
 .SH SYNOPSIS
 .sp
-borg [common options] key import <options> REPOSITORY PATH
+borg [common options] key import [options] [REPOSITORY] [PATH]
 .SH DESCRIPTION
 .sp
 This command allows to restore a key previously backed up with the
@@ -50,7 +50,7 @@ REPOSITORY
 .INDENT 0.0
 .TP
 .B PATH
-path to the backup
+path to the backup (\(aq\-\(aq to read from stdin)
 .UNINDENT
 .SS optional arguments
 .INDENT 0.0

docs/man/borg-key-migrate-to-repokey.1

@@ -1,6 +1,6 @@
 .\" Man page generated from reStructuredText.
 .
-.TH BORG-KEY-MIGRATE-TO-REPOKEY 1 "2017-06-18" "" "borg backup tool"
+.TH BORG-KEY-MIGRATE-TO-REPOKEY 1 "2017-08-27" "" "borg backup tool"
 .SH NAME
 borg-key-migrate-to-repokey \- Migrate passphrase -> repokey
 .
@@ -32,7 +32,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 ..
 .SH SYNOPSIS
 .sp
-borg [common options] key migrate\-to\-repokey <options> REPOSITORY
+borg [common options] key migrate\-to\-repokey [options] [REPOSITORY]
 .SH DESCRIPTION
 .sp
 This command migrates a repository from passphrase mode (removed in Borg 1.0)

docs/man/borg-key.1

@@ -1,6 +1,6 @@
 .\" Man page generated from reStructuredText.
 .
-.TH BORG-KEY 1 "2017-06-18" "" "borg backup tool"
+.TH BORG-KEY 1 "2017-08-27" "" "borg backup tool"
 .SH NAME
 borg-key \- Manage a keyfile or repokey of a repository
 .

docs/man/borg-list.1

@@ -1,6 +1,6 @@
 .\" Man page generated from reStructuredText.
 .
-.TH BORG-LIST 1 "2017-06-18" "" "borg backup tool"
+.TH BORG-LIST 1 "2017-08-27" "" "borg backup tool"
 .SH NAME
 borg-list \- List archive or repository contents
 .
@@ -32,7 +32,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 ..
 .SH SYNOPSIS
 .sp
-borg [common options] list <options> REPOSITORY_OR_ARCHIVE PATH
+borg [common options] list [options] [REPOSITORY_OR_ARCHIVE] [PATH...]
 .SH DESCRIPTION
 .sp
 This command lists the contents of a repository or an archive.
@@ -56,9 +56,8 @@ paths to list; patterns are supported
 .B \-\-short
 only print file/directory names, nothing else
 .TP
-.B \-\-format\fP,\fB  \-\-list\-format
-specify format for file listing
-(default: "{mode} {user:6} {group:6} {size:8d} {isomtime} {path}{extra}{NL}")
+.BI \-\-format \ FORMAT\fP,\fB \ \-\-list\-format \ FORMAT
+specify format for file listing (default: "{mode} {user:6} {group:6} {size:8d} {mtime} {path}{extra}{NL}")
 .TP
 .B \-\-json
 Only valid for listing repository contents. Format output as JSON. The form of \fB\-\-format\fP is ignored, but keys used in it are added to the JSON output. Some keys are always present. Note: JSON can only represent text. A "barchive" key is therefore not available.
@@ -66,16 +65,16 @@ Only valid for listing repository contents. Format output as JSON. The form of \
 .B \-\-json\-lines
 Only valid for listing archive contents. Format output as JSON Lines. The form of \fB\-\-format\fP is ignored, but keys used in it are added to the JSON output. Some keys are always present. Note: JSON can only represent text. A "bpath" key is therefore not available.
 .UNINDENT
-.SS filters
+.SS Archive filters
 .INDENT 0.0
 .TP
-.B \-P\fP,\fB  \-\-prefix
+.BI \-P \ PREFIX\fP,\fB \ \-\-prefix \ PREFIX
 only consider archive names starting with this prefix.
 .TP
-.B \-a\fP,\fB  \-\-glob\-archives
+.BI \-a \ GLOB\fP,\fB \ \-\-glob\-archives \ GLOB
 only consider archive names matching the glob. sh: rules apply, see "borg help patterns". \fB\-\-prefix\fP and \fB\-\-glob\-archives\fP are mutually exclusive.
 .TP
-.B \-\-sort\-by
+.BI \-\-sort\-by \ KEYS
 Comma\-separated list of sorting keys; valid keys are: timestamp, name, id; default is: timestamp
 .TP
 .BI \-\-first \ N
@@ -93,6 +92,12 @@ exclude paths matching PATTERN
 .BI \-\-exclude\-from \ EXCLUDEFILE
 read exclude patterns from EXCLUDEFILE, one per line
 .TP
+.BI \-\-pattern \ PATTERN
+experimental: include/exclude paths matching PATTERN
+.TP
+.BI \-\-patterns\-from \ PATTERNFILE
+experimental: read include/exclude patterns from PATTERNFILE, one per line
+.TP
 .B \-\-exclude\-caches
 exclude directories that contain a CACHEDIR.TAG file (\fI\%http://www.brynosaurus.com/cachedir/spec.html\fP)
 .TP
@@ -100,13 +105,7 @@ exclude directories that contain a CACHEDIR.TAG file (\fI\%http://www.brynosauru
 exclude directories that are tagged by containing a filesystem object with the given NAME
 .TP
 .B \-\-keep\-exclude\-tags\fP,\fB  \-\-keep\-tag\-files
-if tag objects are specified with \-\-exclude\-if\-present, don\(aqt omit the tag objects themselves from the backup archive
-.TP
-.BI \-\-pattern \ PATTERN
-experimental: include/exclude paths matching PATTERN
-.TP
-.BI \-\-patterns\-from \ PATTERNFILE
-experimental: read include/exclude patterns from PATTERNFILE, one per line
+if tag objects are specified with \fB\-\-exclude\-if\-present\fP, don\(aqt omit the tag objects themselves from the backup archive
 .UNINDENT
 .SH EXAMPLES
 .INDENT 0.0
@@ -162,13 +161,23 @@ LF
 Keys for listing repository archives:
 .INDENT 0.0
 .IP \(bu 2
-archive, name: archive name interpreted as text (might be missing non\-text characters, see barchive)
+name: archive name interpreted as text (might be missing non\-text characters, see barchive)
+.IP \(bu 2
+archive: archive name interpreted as text (might be missing non\-text characters, see barchive)
 .IP \(bu 2
 barchive: verbatim archive name, can contain any character except NUL
 .IP \(bu 2
-time: time of creation of the archive
+comment: archive comment interpreted as text (might be missing non\-text characters, see bcomment)
+.IP \(bu 2
+bcomment: verbatim archive comment, can contain any character except NUL
 .IP \(bu 2
 id: internal ID of the archive
+.IP \(bu 2
+time: time (start) of creation of the archive
+.IP \(bu 2
+start: time (start) of creation of the archive
+.IP \(bu 2
+end: time (end) of creation of the archive
 .UNINDENT
 .sp
 Keys for listing archive files:

docs/man/borg-mount.1

@@ -1,6 +1,6 @@
 .\" Man page generated from reStructuredText.
 .
-.TH BORG-MOUNT 1 "2017-06-18" "" "borg backup tool"
+.TH BORG-MOUNT 1 "2017-08-27" "" "borg backup tool"
 .SH NAME
 borg-mount \- Mount archive or an entire repository as a FUSE filesystem
 .
@@ -32,7 +32,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 ..
 .SH SYNOPSIS
 .sp
-borg [common options] mount <options> REPOSITORY_OR_ARCHIVE MOUNTPOINT
+borg [common options] mount [options] REPOSITORY_OR_ARCHIVE MOUNTPOINT
 .SH DESCRIPTION
 .sp
 This command mounts an archive as a FUSE filesystem. This can be useful for
@@ -91,16 +91,16 @@ stay in foreground, do not daemonize
 .B \-o
 Extra mount options
 .UNINDENT
-.SS filters
+.SS Archive filters
 .INDENT 0.0
 .TP
-.B \-P\fP,\fB  \-\-prefix
+.BI \-P \ PREFIX\fP,\fB \ \-\-prefix \ PREFIX
 only consider archive names starting with this prefix.
 .TP
-.B \-a\fP,\fB  \-\-glob\-archives
+.BI \-a \ GLOB\fP,\fB \ \-\-glob\-archives \ GLOB
 only consider archive names matching the glob. sh: rules apply, see "borg help patterns". \fB\-\-prefix\fP and \fB\-\-glob\-archives\fP are mutually exclusive.
 .TP
-.B \-\-sort\-by
+.BI \-\-sort\-by \ KEYS
 Comma\-separated list of sorting keys; valid keys are: timestamp, name, id; default is: timestamp
 .TP
 .BI \-\-first \ N

docs/man/borg-patterns.1

@@ -1,6 +1,6 @@
 .\" Man page generated from reStructuredText.
 .
-.TH BORG-PATTERNS 1 "2017-06-18" "" "borg backup tool"
+.TH BORG-PATTERNS 1 "2017-08-27" "" "borg backup tool"
 .SH NAME
 borg-patterns \- Details regarding patterns
 .
@@ -59,7 +59,7 @@ matching is attempted.  Thus, if a given pattern ends in a path
 separator, a \(aq*\(aq is appended before matching is attempted.
 .TP
 .B Shell\-style patterns, selector \fIsh:\fP
-This is the default style for \-\-pattern and \-\-patterns\-from.
+This is the default style for \fB\-\-pattern\fP and \fB\-\-patterns\-from\fP\&.
 Like fnmatch patterns these are similar to shell patterns. The difference
 is that the pattern may include \fI**/\fP for matching zero or more directory
 levels, \fI*\fP for matching zero or more arbitrary characters with the

docs/man/borg-placeholders.1

@@ -1,6 +1,6 @@
 .\" Man page generated from reStructuredText.
 .
-.TH BORG-PLACEHOLDERS 1 "2017-06-18" "" "borg backup tool"
+.TH BORG-PLACEHOLDERS 1 "2017-08-27" "" "borg backup tool"
 .SH NAME
 borg-placeholders \- Details regarding placeholders
 .

docs/man/borg-prune.1

@@ -1,6 +1,6 @@
 .\" Man page generated from reStructuredText.
 .
-.TH BORG-PRUNE 1 "2017-06-18" "" "borg backup tool"
+.TH BORG-PRUNE 1 "2017-08-27" "" "borg backup tool"
 .SH NAME
 borg-prune \- Prune repository archives according to specified rules
 .
@@ -32,7 +32,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 ..
 .SH SYNOPSIS
 .sp
-borg [common options] prune <options> REPOSITORY
+borg [common options] prune [options] [REPOSITORY]
 .SH DESCRIPTION
 .sp
 The prune command prunes a repository by deleting all archives not matching
@@ -97,7 +97,7 @@ print statistics for the deleted archive
 .B \-\-list
 output verbose list of archives it keeps/prunes
 .TP
-.BI \-\-keep\-within \ WITHIN
+.BI \-\-keep\-within \ INTERVAL
 keep all archives within this time interval
 .TP
 .B \-\-keep\-last\fP,\fB  \-\-keep\-secondly
@@ -124,13 +124,13 @@ number of yearly archives to keep
 .B \-\-save\-space
 work slower, but using less space
 .UNINDENT
-.SS filters
+.SS Archive filters
 .INDENT 0.0
 .TP
-.B \-P\fP,\fB  \-\-prefix
+.BI \-P \ PREFIX\fP,\fB \ \-\-prefix \ PREFIX
 only consider archive names starting with this prefix.
 .TP
-.B \-a\fP,\fB  \-\-glob\-archives
+.BI \-a \ GLOB\fP,\fB \ \-\-glob\-archives \ GLOB
 only consider archive names matching the glob. sh: rules apply, see "borg help patterns". \fB\-\-prefix\fP and \fB\-\-glob\-archives\fP are mutually exclusive.
 .UNINDENT
 .SH EXAMPLES
@@ -145,8 +145,6 @@ prefix "foo" if you do not also want to match "foobar".
 .sp
 It is strongly recommended to always run \fBprune \-v \-\-list \-\-dry\-run ...\fP
 first so you will see what it would do without it actually doing anything.
-.sp
-There is also a visualized prune example in \fBdocs/misc/prune\-example.txt\fP\&.
 .INDENT 0.0
 .INDENT 3.5
 .sp
@@ -167,6 +165,21 @@ $ borg prune \-v \-\-list \-\-keep\-daily=7 \-\-keep\-weekly=4 \-\-keep\-monthly
 # Keep all backups in the last 10 days, 4 additional end of week archives,
 # and an end of month archive for every month:
 $ borg prune \-v \-\-list \-\-keep\-within=10d \-\-keep\-weekly=4 \-\-keep\-monthly=\-1 /path/to/repo
+.ft P
+.fi
+.UNINDENT
+.UNINDENT
+.sp
+There is also a visualized prune example in \fBdocs/misc/prune\-example.txt\fP:
+.IP "System Message: ERROR/3 (docs/virtmanpage.rst:, line 140)"
+Unknown directive type "highlight".
+.INDENT 0.0
+.INDENT 3.5
+.sp
+.nf
+.ft C
+\&.. highlight:: none
+
 .ft P
 .fi
 .UNINDENT

docs/man/borg-recreate.1

@@ -1,6 +1,6 @@
 .\" Man page generated from reStructuredText.
 .
-.TH BORG-RECREATE 1 "2017-06-18" "" "borg backup tool"
+.TH BORG-RECREATE 1 "2017-08-27" "" "borg backup tool"
 .SH NAME
 borg-recreate \- Re-create archives
 .
@@ -32,7 +32,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 ..
 .SH SYNOPSIS
 .sp
-borg [common options] recreate <options> REPOSITORY_OR_ARCHIVE PATH
+borg [common options] recreate [options] [REPOSITORY_OR_ARCHIVE] [PATH...]
 .SH DESCRIPTION
 .sp
 Recreate the contents of existing archives.
@@ -90,7 +90,7 @@ paths to recreate; patterns are supported
 output verbose list of items (files, dirs, ...)
 .TP
 .BI \-\-filter \ STATUSCHARS
-only display items with the given status characters
+only display items with the given status characters (listed in borg create \-\-help)
 .TP
 .B \-n\fP,\fB  \-\-dry\-run
 do not change anything
@@ -107,6 +107,12 @@ exclude paths matching PATTERN
 .BI \-\-exclude\-from \ EXCLUDEFILE
 read exclude patterns from EXCLUDEFILE, one per line
 .TP
+.BI \-\-pattern \ PATTERN
+experimental: include/exclude paths matching PATTERN
+.TP
+.BI \-\-patterns\-from \ PATTERNFILE
+experimental: read include/exclude patterns from PATTERNFILE, one per line
+.TP
 .B \-\-exclude\-caches
 exclude directories that contain a CACHEDIR.TAG file (\fI\%http://www.brynosaurus.com/cachedir/spec.html\fP)
 .TP
@@ -115,12 +121,6 @@ exclude directories that are tagged by containing a filesystem object with the g
 .TP
 .B \-\-keep\-exclude\-tags\fP,\fB  \-\-keep\-tag\-files
 if tag objects are specified with \fB\-\-exclude\-if\-present\fP, don\(aqt omit the tag objects themselves from the backup archive
-.TP
-.BI \-\-pattern \ PATTERN
-experimental: include/exclude paths matching PATTERN
-.TP
-.BI \-\-patterns\-from \ PATTERNFILE
-experimental: read include/exclude patterns from PATTERNFILE, one per line
 .UNINDENT
 .SS Archive options
 .INDENT 0.0
@@ -152,19 +152,20 @@ specify the chunker parameters (CHUNK_MIN_EXP, CHUNK_MAX_EXP, HASH_MASK_BITS, HA
 .sp
 .nf
 .ft C
-# Make old (Attic / Borg 0.xx) archives deduplicate with Borg 1.x archives
-# Archives created with Borg 1.1+ and the default chunker params are skipped (archive ID stays the same)
+# Make old (Attic / Borg 0.xx) archives deduplicate with Borg 1.x archives.
+# Archives created with Borg 1.1+ and the default chunker params are skipped
+# (archive ID stays the same).
 $ borg recreate /mnt/backup \-\-chunker\-params default \-\-progress
 
 # Create a backup with little but fast compression
 $ borg create /mnt/backup::archive /some/files \-\-compression lz4
-# Then compress it \- this might take longer, but the backup has already completed, so no inconsistencies
-# from a long\-running backup job.
+# Then compress it \- this might take longer, but the backup has already completed,
+# so no inconsistencies from a long\-running backup job.
 $ borg recreate /mnt/backup::archive \-\-recompress \-\-compression zlib,9
 
-# Remove unwanted files from all archives in a repository
-$ borg recreate /mnt/backup \-e /home/icke/Pictures/drunk_photos
-
+# Remove unwanted files from all archives in a repository.
+# Note the relative path for the \-\-exclude option \- archives only contain relative paths.
+$ borg recreate /mnt/backup \-\-exclude home/icke/Pictures/drunk_photos
 
 # Change archive comment
 $ borg create \-\-comment "This is a comment" /mnt/backup::archivename ~

docs/man/borg-rename.1

@@ -1,6 +1,6 @@
 .\" Man page generated from reStructuredText.
 .
-.TH BORG-RENAME 1 "2017-06-18" "" "borg backup tool"
+.TH BORG-RENAME 1 "2017-08-27" "" "borg backup tool"
 .SH NAME
 borg-rename \- Rename an existing archive
 .
@@ -32,7 +32,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 ..
 .SH SYNOPSIS
 .sp
-borg [common options] rename <options> ARCHIVE NEWNAME
+borg [common options] rename [options] ARCHIVE NEWNAME
 .SH DESCRIPTION
 .sp
 This command renames an archive in the repository.

docs/man/borg-serve.1

@@ -1,6 +1,6 @@
 .\" Man page generated from reStructuredText.
 .
-.TH BORG-SERVE 1 "2017-06-18" "" "borg backup tool"
+.TH BORG-SERVE 1 "2017-08-27" "" "borg backup tool"
 .SH NAME
 borg-serve \- Start in server mode. This command is usually not used manually.
 .
@@ -32,7 +32,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 ..
 .SH SYNOPSIS
 .sp
-borg [common options] serve <options>
+borg [common options] serve [options]
 .SH DESCRIPTION
 .sp
 This command starts a repository server process. This command is usually not used manually.
@@ -46,12 +46,12 @@ See \fIborg\-common(1)\fP for common options of Borg commands.
 restrict repository access to PATH. Can be specified multiple times to allow the client access to several directories. Access to all sub\-directories is granted implicitly; PATH doesn\(aqt need to directly point to a repository.
 .TP
 .BI \-\-restrict\-to\-repository \ PATH
-restrict repository access. Only the repository located at PATH (no sub\-directories are considered) is accessible. Can be specified multiple times to allow the client access to several repositories. Unlike \-\-restrict\-to\-path sub\-directories are not accessible; PATH needs to directly point at a repository location. PATH may be an empty directory or the last element of PATH may not exist, in which case the client may initialize a repository there.
+restrict repository access. Only the repository located at PATH (no sub\-directories are considered) is accessible. Can be specified multiple times to allow the client access to several repositories. Unlike \fB\-\-restrict\-to\-path\fP sub\-directories are not accessible; PATH needs to directly point at a repository location. PATH may be an empty directory or the last element of PATH may not exist, in which case the client may initialize a repository there.
 .TP
 .B \-\-append\-only
 only allow appending to repository segment files
 .TP
-.B \-\-storage\-quota
+.BI \-\-storage\-quota \ QUOTA
 Override storage quota of the repository (e.g. 5G, 1.5T). When a new repository is initialized, sets the storage quota on the new repository as well. Default: no quota.
 .UNINDENT
 .SH EXAMPLES
@@ -78,7 +78,7 @@ locations like \fB/etc/environment\fP or in the forced command itself (example b
 # Use key options to disable unneeded and potentially dangerous SSH functionality.
 # This will help to secure an automated remote backup system.
 $ cat ~/.ssh/authorized_keys
-command="borg serve \-\-restrict\-to\-path /path/to/repo",no\-pty,no\-agent\-forwarding,no\-port\-forwarding,no\-X11\-forwarding,no\-user\-rc ssh\-rsa AAAAB3[...]
+command="borg serve \-\-restrict\-to\-path /path/to/repo",restrict ssh\-rsa AAAAB3[...]
 
 # Set a BORG_XXX environment variable on the "borg serve" side
 $ cat ~/.ssh/authorized_keys
@@ -87,6 +87,21 @@ command="export BORG_XXX=value; borg serve [...]",restrict ssh\-rsa [...]
 .fi
 .UNINDENT
 .UNINDENT
+.sp
+\fBNOTE:\fP
+.INDENT 0.0
+.INDENT 3.5
+The examples above use the \fBrestrict\fP directive. This does automatically
+block potential dangerous ssh features, even when they are added in a future
+update. Thus, this option should be preferred.
+.sp
+If you\(aqre using openssh\-server < 7.2, however, you have to explicitly specify
+the ssh features to restrict and cannot simply use the restrict option as it
+has been introduced in v7.2. We recommend to use
+\fBno\-port\-forwarding,no\-X11\-forwarding,no\-pty,no\-agent\-forwarding,no\-user\-rc\fP
+in this case.
+.UNINDENT
+.UNINDENT
 .SH SEE ALSO
 .sp
 \fIborg\-common(1)\fP

docs/man/borg-umount.1

@@ -1,6 +1,6 @@
 .\" Man page generated from reStructuredText.
 .
-.TH BORG-UMOUNT 1 "2017-06-18" "" "borg backup tool"
+.TH BORG-UMOUNT 1 "2017-08-27" "" "borg backup tool"
 .SH NAME
 borg-umount \- un-mount the FUSE filesystem
 .
@@ -32,7 +32,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 ..
 .SH SYNOPSIS
 .sp
-borg [common options] umount <options> MOUNTPOINT
+borg [common options] umount [options] MOUNTPOINT
 .SH DESCRIPTION
 .sp
 This command un\-mounts a FUSE filesystem that was mounted with \fBborg mount\fP\&.
@@ -49,31 +49,39 @@ See \fIborg\-common(1)\fP for common options of Borg commands.
 mountpoint of the filesystem to umount
 .UNINDENT
 .SH EXAMPLES
-.SS borg mount
 .INDENT 0.0
 .INDENT 3.5
 .sp
 .nf
 .ft C
+# Mounting the repository shows all archives.
+# Archives are loaded lazily, expect some delay when navigating to an archive
+# for the first time.
+$ borg mount /path/to/repo /tmp/mymountpoint
+$ ls /tmp/mymountpoint
+root\-2016\-02\-14 root\-2016\-02\-15
+$ borg umount /tmp/mymountpoint
+
+# Mounting a specific archive is possible as well.
 $ borg mount /path/to/repo::root\-2016\-02\-15 /tmp/mymountpoint
 $ ls /tmp/mymountpoint
-bin  boot  etc      home  lib  lib64  lost+found  media  mnt  opt  root  sbin  srv  tmp  usr  var
+bin  boot  etc      home  lib  lib64  lost+found  media  mnt  opt
+root  sbin  srv  tmp  usr  var
 $ borg umount /tmp/mymountpoint
-.ft P
-.fi
-.UNINDENT
-.UNINDENT
-.INDENT 0.0
-.INDENT 3.5
-.sp
-.nf
-.ft C
+
+# The experimental "versions view" merges all archives in the repository
+# and provides a versioned view on files.
 $ borg mount \-o versions /path/to/repo /tmp/mymountpoint
 $ ls \-l /tmp/mymountpoint/home/user/doc.txt/
 total 24
-\-rw\-rw\-r\-\- 1 user group 12357 Aug 26 21:19 doc.txt.cda00bc9
-\-rw\-rw\-r\-\- 1 user group 12204 Aug 26 21:04 doc.txt.fa760f28
-$ fusermount \-u /tmp/mymountpoint
+\-rw\-rw\-r\-\- 1 user group 12357 Aug 26 21:19 doc.cda00bc9.txt
+\-rw\-rw\-r\-\- 1 user group 12204 Aug 26 21:04 doc.fa760f28.txt
+$ borg umount /tmp/mymountpoint
+
+# Archive filters are supported.
+# These are especially handy for the "versions view",
+# which does not support lazy processing of archives.
+$ borg mount \-o versions \-\-glob\-archives \(aq*\-my\-home\(aq \-\-last 10 /path/to/repo /tmp/mymountpoint
 .ft P
 .fi
 .UNINDENT

docs/man/borg-upgrade.1

@@ -1,6 +1,6 @@
 .\" Man page generated from reStructuredText.
 .
-.TH BORG-UPGRADE 1 "2017-06-18" "" "borg backup tool"
+.TH BORG-UPGRADE 1 "2017-08-27" "" "borg backup tool"
 .SH NAME
 borg-upgrade \- upgrade a repository from a previous version
 .
@@ -32,7 +32,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 ..
 .SH SYNOPSIS
 .sp
-borg [common options] upgrade <options> REPOSITORY
+borg [common options] upgrade [options] [REPOSITORY]
 .SH DESCRIPTION
 .sp
 Upgrade an existing, local Borg repository.
@@ -106,7 +106,7 @@ borg delete borg
 .sp
 Unless \fB\-\-inplace\fP is specified, the upgrade process first
 creates a backup copy of the repository, in
-REPOSITORY.upgrade\-DATETIME, using hardlinks. This takes
+REPOSITORY.before\-upgrade\-DATETIME, using hardlinks. This takes
 longer than in place upgrades, but is much safer and gives
 progress information (as opposed to \fBcp \-al\fP). Once you are
 satisfied with the conversion, you can safely destroy the
@@ -133,17 +133,16 @@ path to the repository to be upgraded
 do not change repository
 .TP
 .B \-\-inplace
-rewrite repository in place, with no chance of going back to older
-versions of the repository.
+rewrite repository in place, with no chance of going back to older versions of the repository.
 .TP
 .B \-\-force
 Force upgrade
 .TP
 .B \-\-tam
-Enable manifest authentication (in key and cache) (Borg 1.0.9 and later)
+Enable manifest authentication (in key and cache) (Borg 1.0.9 and later).
 .TP
 .B \-\-disable\-tam
-Disable manifest authentication (in key and cache)
+Disable manifest authentication (in key and cache).
 .UNINDENT
 .SH EXAMPLES
 .INDENT 0.0
@@ -153,7 +152,7 @@ Disable manifest authentication (in key and cache)
 .ft C
 # Upgrade the borg repository to the most recent version.
 $ borg upgrade \-v /path/to/repo
-making a hardlink copy in /path/to/repo.upgrade\-2016\-02\-15\-20:51:55
+making a hardlink copy in /path/to/repo.before\-upgrade\-2016\-02\-15\-20:51:55
 opening attic repository with borg and converting
 no key file found for repository
 converting repo index /path/to/repo/index.0

docs/man/borg-with-lock.1

@@ -1,6 +1,6 @@
 .\" Man page generated from reStructuredText.
 .
-.TH BORG-WITH-LOCK 1 "2017-06-18" "" "borg backup tool"
+.TH BORG-WITH-LOCK 1 "2017-08-27" "" "borg backup tool"
 .SH NAME
 borg-with-lock \- run a user specified command with the repository lock held
 .
@@ -32,7 +32,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 ..
 .SH SYNOPSIS
 .sp
-borg [common options] with\-lock <options> REPOSITORY COMMAND ARGS
+borg [common options] with\-lock [options] REPOSITORY COMMAND [ARGS...]
 .SH DESCRIPTION
 .sp
 This command runs a user\-specified command while the repository lock is held.
@@ -41,11 +41,15 @@ It will first try to acquire the lock (make sure that no other operation is
 running in the repo), then execute the given command as a subprocess and wait
 for its termination, release the lock and return the user command\(aqs return
 code as borg\(aqs return code.
+.sp
+\fBNOTE:\fP
 .INDENT 0.0
-.TP
-.B Note: if you copy a repository with the lock held, the lock will be present in
-the copy, obviously. Thus, before using borg on the copy, you need to
-use "borg break\-lock" on it.
+.INDENT 3.5
+If you copy a repository with the lock held, the lock will be present in
+the copy. Thus, before using borg on the copy from a different host,
+you need to use "borg break\-lock" on the copied repository, because
+Borg is cautious and does not automatically remove stale locks made by a different host.
+.UNINDENT
 .UNINDENT
 .SH OPTIONS
 .sp

docs/man/borg.1

@@ -472,6 +472,21 @@ know a list of affected hardware.
 If you are suspicious whether your Borg repository is still consistent
 and readable after one of the failures mentioned above occurred, run
 \fBborg check \-\-verify\-data\fP to make sure it is consistent.
+Requirements for Borg repository file systems.INDENT 0.0
+.IP \(bu 2
+Long file names
+.IP \(bu 2
+At least three directory levels with short names
+.IP \(bu 2
+Typically, file sizes up to a few hundred MB.
+Large repositories may require large files (>2 GB).
+.IP \(bu 2
+Up to 1000 files per directory (10000 for repositories initialized with Borg 1.0)
+.IP \(bu 2
+mkdir(2) should be atomic, since it is used for locking
+.IP \(bu 2
+Hardlinks are needed for \fIborg_upgrade\fP \fB\-\-inplace\fP
+.UNINDENT
 .SS Units
 .sp
 To display quantities, Borg takes care of respecting the
@@ -574,6 +589,167 @@ operations used for transaction support also go over the connection. If
 you backup multiple sources to one target repository, additional traffic
 happens for cache resynchronization.
 .UNINDENT
+.SS Support for file metadata
+.sp
+Besides regular file and directory structures, Borg can preserve
+.INDENT 0.0
+.IP \(bu 2
+symlinks (stored as symlink, the symlink is not followed)
+.IP \(bu 2
+special files:
+.INDENT 2.0
+.IP \(bu 2
+character and block device files (restored via mknod)
+.IP \(bu 2
+FIFOs ("named pipes")
+.IP \(bu 2
+special file \fIcontents\fP can be backed up in \fB\-\-read\-special\fP mode.
+By default the metadata to create them with mknod(2), mkfifo(2) etc. is stored.
+.UNINDENT
+.IP \(bu 2
+hardlinked regular files, devices, FIFOs (considering all items in the same archive)
+.IP \(bu 2
+timestamps in nanosecond precision: mtime, atime, ctime
+.IP \(bu 2
+permissions:
+.INDENT 2.0
+.IP \(bu 2
+IDs of owning user and owning group
+.IP \(bu 2
+names of owning user and owning group (if the IDs can be resolved)
+.IP \(bu 2
+Unix Mode/Permissions (u/g/o permissions, suid, sgid, sticky)
+.UNINDENT
+.UNINDENT
+.sp
+On some platforms additional features are supported:
+.\" Yes/No's are grouped by reason/mechanism/reference.
+.
+.TS
+center;
+|l|l|l|l|.
+_
+T{
+Platform
+T}	T{
+ACLs
+[5]
+T}	T{
+xattr
+[6]
+T}	T{
+Flags
+[7]
+T}
+_
+T{
+Linux
+T}	T{
+Yes
+T}	T{
+Yes
+T}	T{
+Yes [1]
+T}
+_
+T{
+Mac OS X
+T}	T{
+Yes
+T}	T{
+Yes
+T}	T{
+Yes (all)
+T}
+_
+T{
+FreeBSD
+T}	T{
+Yes
+T}	T{
+Yes
+T}
+_
+T{
+OpenBSD
+T}	T{
+n/a
+T}	T{
+n/a
+T}
+_
+T{
+NetBSD
+T}	T{
+n/a
+T}	T{
+No [2]
+T}
+_
+T{
+Solaris 11
+T}	T{
+No [3]
+T}	T{
+n/a
+T}
+_
+T{
+OpenIndiana
+T}
+_
+T{
+Windows (cygwin)
+T}	T{
+No [4]
+T}	T{
+No
+T}	T{
+No
+T}
+_
+.TE
+.sp
+Other Unix\-like operating systems may work as well, but have not been tested at all.
+.sp
+Note that most of the platform\-dependent features also depend on the file system.
+For example, ntfs\-3g on Linux isn\(aqt able to convey NTFS ACLs.
+.IP [1] 5
+Only "nodump", "immutable", "compressed" and "append" are supported.
+Feature request 
+.nf
+:issue:\(ga618\(ga
+.fi
+ for more flags.
+.IP "System Message: ERROR/3 (docs/usage_general.rst.inc:, line 410)"
+Unknown interpreted text role "issue".
+.IP [2] 5
+Feature request 
+.nf
+:issue:\(ga1332\(ga
+.fi
+
+.IP "System Message: ERROR/3 (docs/usage_general.rst.inc:, line 412)"
+Unknown interpreted text role "issue".
+.IP [3] 5
+Feature request 
+.nf
+:issue:\(ga1337\(ga
+.fi
+
+.IP "System Message: ERROR/3 (docs/usage_general.rst.inc:, line 413)"
+Unknown interpreted text role "issue".
+.IP [4] 5
+Cygwin tries to map NTFS ACLs to permissions with varying degress of success.
+.IP [5] 5
+The native access control list mechanism of the OS. This normally limits access to
+non\-native ACLs. For example, NTFS ACLs aren\(aqt completely accessible on Linux with ntfs\-3g.
+.IP [6] 5
+extended attributes; key\-value pairs attached to a file, mainly used by the OS.
+This includes resource forks on Mac OS X.
+.IP [7] 5
+aka \fIBSD flags\fP\&. The Linux set of flags [1] is portable across platforms.
+The BSDs define additional flags.
 .SH SEE ALSO
 .sp
 \fIborg\-common(1)\fP for common command line options

docs/usage/create.rst.inc

@@ -193,6 +193,9 @@ only include the objects specified by ``--exclude-if-present`` in your backup,
 and not include any other contents of the containing folder, this can be enabled
 through using the ``--keep-exclude-tags`` option.
 
+Borg respects the nodump flag. Files flagged nodump will be marked as excluded (x)
+in ``--list`` output.
+
 Item flags
 ++++++++++
 

docs/usage/info.rst.inc

@@ -77,6 +77,11 @@ up to the deduplicated size of the repository ("all archives"), because the two
 are meaning different things:
 
 This archive / deduplicated size = amount of data stored ONLY for this archive
-                                 = unique chunks of this archive.
+= unique chunks of this archive.
 All archives / deduplicated size = amount of data stored in the repo
-                                 = all chunks in the repository.
+= all chunks in the repository.
+
+Borg archives can only contain a limited amount of file metadata.
+The size of an archive relative to this limit depends on a number of factors,
+mainly the number of files, the lengths of paths and other metadata stored for files.
+This is shown as *utilization of maximum supported archive size*.

docs/usage/list.rst.inc

@@ -23,7 +23,7 @@ borg list
     +-----------------------------------------------------------------------------+-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
     |                                                                             | ``--short``                                   | only print file/directory names, nothing else                                                                                                                                                                                                                            |
     +-----------------------------------------------------------------------------+-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
-    |                                                                             | ``--format FORMAT``, ``--list-format FORMAT`` | specify format for file listing (default: "{mode} {user:6} {group:6} {size:8d} {isomtime} {path}{extra}{NL}")                                                                                                                                                            |
+    |                                                                             | ``--format FORMAT``, ``--list-format FORMAT`` | specify format for file listing (default: "{mode} {user:6} {group:6} {size:8d} {mtime} {path}{extra}{NL}")                                                                                                                                                               |
     +-----------------------------------------------------------------------------+-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
     |                                                                             | ``--json``                                    | Only valid for listing repository contents. Format output as JSON. The form of ``--format`` is ignored, but keys used in it are added to the JSON output. Some keys are always present. Note: JSON can only represent text. A "barchive" key is therefore not available. |
     +-----------------------------------------------------------------------------+-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
@@ -80,7 +80,7 @@ borg list
 
     optional arguments
         --short     only print file/directory names, nothing else
-        --format FORMAT, --list-format FORMAT    specify format for file listing (default: "{mode} {user:6} {group:6} {size:8d} {isomtime} {path}{extra}{NL}")
+        --format FORMAT, --list-format FORMAT    specify format for file listing (default: "{mode} {user:6} {group:6} {size:8d} {mtime} {path}{extra}{NL}")
         --json      Only valid for listing repository contents. Format output as JSON. The form of ``--format`` is ignored, but keys used in it are added to the JSON output. Some keys are always present. Note: JSON can only represent text. A "barchive" key is therefore not available.
         --json-lines    Only valid for listing archive contents. Format output as JSON Lines. The form of ``--format`` is ignored, but keys used in it are added to the JSON output. Some keys are always present. Note: JSON can only represent text. A "bpath" key is therefore not available.
 

docs/usage/with-lock.rst.inc

@@ -60,5 +60,6 @@ code as borg's return code.
 .. note::
 
     If you copy a repository with the lock held, the lock will be present in
-    the copy, obviously. Thus, before using borg on the copy, you need to
-    use "borg break-lock" on it.
+    the copy. Thus, before using borg on the copy from a different host,
+    you need to use "borg break-lock" on the copied repository, because
+    Borg is cautious and does not automatically remove stale locks made by a different host.