Merge pull request #7646 from ThomasWaldmann/rel200b6

Release 2.0.0b6

docs/changes.rst

@@ -12,8 +12,8 @@ This section provides information about security and corruption issues.
 Change Log 2.x
 ==============
 
-Version 2.0.0b6 (not released yet)
-----------------------------------
+Version 2.0.0b6 (2023-06-11)
+----------------------------
 
 Please note:
 
@@ -118,6 +118,7 @@ New features:
 - diff: include changes in ctime and mtime, #7248
 - diff: sort JSON output alphabetically
 - diff --content-only: option added to ignore metadata changes
+- diff: add --format option, #4634
 - import-tar --ignore-zeros: new option to support importing concatenated tars, #7432
 - debug id-hash / parse-obj / format-obj: new debug commands, #7406
 - transfer --compression=C --recompress=M: recompress while transferring, #7529
@@ -173,6 +174,7 @@ Other changes:
 - borg.logger: add logging debugging functionality
 - add function to clear empty directories at end of compact process
 - unify scanning and listing of segment dirs / segment files, #7597
+- replace `LRUCache` internals with `OrderedDict`
 - docs:
 
   - add installation instructions for Windows
@@ -202,6 +204,7 @@ Other changes:
   - tox: package = editable-legacy, #7580
   - tox under fakeroot: fix finding setup_docs, #7391
   - check buzhash chunksize distribution, #7586
+  - use debian/bookworm64 box
 
 
 Version 2.0.0b5 (2023-02-27)

docs/deployment/central-backup-server.rst

@@ -1,5 +1,6 @@
 .. include:: ../global.rst.inc
 .. highlight:: none
+.. _central-backup-server:
 
 Central repository server with Ansible or Salt
 ==============================================

docs/deployment/pull-backup.rst

@@ -470,13 +470,13 @@ Make sure to use port ``8022`` and ``localhost`` for the repository as this inst
 remote forwarded ssh connection.
 
 SSH Keys
-~~~~~~~~
+--------
 
 If you want to automate backups when using this method, the ssh ``known_hosts`` and ``authorized_keys`` need to be set up
 to allow connections.
 
 Security Considerations
-~~~~~~~~~~~~~~~~~~~~~~~
+-----------------------
 
 Opening up SSH access this way can pose a security risk as it effectively opens remote access to your
 backup server on the client even if it is located outside of your company network.
@@ -496,7 +496,7 @@ All the additional security considerations for borg should be applied, see :ref:
 hints.
 
 More information
-~~~~~~~~~~~~~~~~
+----------------
 
 See `remote forwarding`_ and the `ssh man page`_ for more information about remote forwarding.
 

docs/faq.rst

@@ -25,7 +25,7 @@ running on the remote side, so *every* operation needs to go over the network,
 which is slower.
 
 Can I back up from multiple servers into a single repository?
-------------------------------------------------------------
+-------------------------------------------------------------
 
 In order for the deduplication used by Borg to work, it
 needs to keep a local cache containing checksums of all file
@@ -162,7 +162,7 @@ file is being processed.
 
 
 How can I back up huge file(s) over a unstable connection?
----------------------------------------------------------
+----------------------------------------------------------
 
 Yes. For more details, see :ref:`checkpoints_parts`.
 
@@ -744,7 +744,7 @@ This has some pros and cons, though:
 The long term plan to improve this is called "borgception", see :issue:`474`.
 
 Can I back up my root partition (/) with Borg?
----------------------------------------------
+----------------------------------------------
 
 Backing up your entire root partition works just fine, but remember to
 exclude directories that make no sense to back up, such as /dev, /proc,

docs/man/borg-benchmark-cpu.1

@@ -27,7 +27,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .in \\n[rst2man-indent\\n[rst2man-indent-level]]u
 ..
-.TH "BORG-BENCHMARK-CPU" 1 "2023-02-26" "" "borg backup tool"
+.TH "BORG-BENCHMARK-CPU" 1 "2023-06-11" "" "borg backup tool"
 .SH NAME
 borg-benchmark-cpu \- Benchmark CPU bound operations.
 .SH SYNOPSIS

docs/man/borg-benchmark-crud.1

@@ -27,7 +27,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .in \\n[rst2man-indent\\n[rst2man-indent-level]]u
 ..
-.TH "BORG-BENCHMARK-CRUD" 1 "2023-02-26" "" "borg backup tool"
+.TH "BORG-BENCHMARK-CRUD" 1 "2023-06-11" "" "borg backup tool"
 .SH NAME
 borg-benchmark-crud \- Benchmark Create, Read, Update, Delete for archives.
 .SH SYNOPSIS

docs/man/borg-benchmark.1

@@ -27,7 +27,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .in \\n[rst2man-indent\\n[rst2man-indent-level]]u
 ..
-.TH "BORG-BENCHMARK" 1 "2023-02-26" "" "borg backup tool"
+.TH "BORG-BENCHMARK" 1 "2023-06-11" "" "borg backup tool"
 .SH NAME
 borg-benchmark \- benchmark command
 .SH SYNOPSIS

docs/man/borg-break-lock.1

@@ -27,7 +27,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .in \\n[rst2man-indent\\n[rst2man-indent-level]]u
 ..
-.TH "BORG-BREAK-LOCK" 1 "2023-02-26" "" "borg backup tool"
+.TH "BORG-BREAK-LOCK" 1 "2023-06-11" "" "borg backup tool"
 .SH NAME
 borg-break-lock \- Break the repository lock (e.g. in case it was left by a dead borg.
 .SH SYNOPSIS

docs/man/borg-check.1

@@ -27,7 +27,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .in \\n[rst2man-indent\\n[rst2man-indent-level]]u
 ..
-.TH "BORG-CHECK" 1 "2023-02-26" "" "borg backup tool"
+.TH "BORG-CHECK" 1 "2023-06-11" "" "borg backup tool"
 .SH NAME
 borg-check \- Check repository consistency
 .SH SYNOPSIS

docs/man/borg-common.1

@@ -27,7 +27,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .in \\n[rst2man-indent\\n[rst2man-indent-level]]u
 ..
-.TH "BORG-COMMON" 1 "2023-02-26" "" "borg backup tool"
+.TH "BORG-COMMON" 1 "2023-06-11" "" "borg backup tool"
 .SH NAME
 borg-common \- Common options of Borg commands
 .SH SYNOPSIS
@@ -93,6 +93,9 @@ Write execution profile in Borg format into FILE. For local use a Python\-compat
 .BI \-\-rsh \ RSH
 Use this command to connect to the \(aqborg serve\(aq process (default: \(aqssh\(aq)
 .TP
+.BI \-\-socket \ PATH
+Use UNIX DOMAIN (IPC) socket at PATH for client/server communication with socket: protocol.
+.TP
 .BI \-r \ REPO\fR,\fB \ \-\-repo \ REPO
 repository to use
 .UNINDENT

docs/man/borg-compact.1

@@ -27,7 +27,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .in \\n[rst2man-indent\\n[rst2man-indent-level]]u
 ..
-.TH "BORG-COMPACT" 1 "2023-02-26" "" "borg backup tool"
+.TH "BORG-COMPACT" 1 "2023-06-11" "" "borg backup tool"
 .SH NAME
 borg-compact \- compact segment files in the repository
 .SH SYNOPSIS

docs/man/borg-compression.1

@@ -27,7 +27,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .in \\n[rst2man-indent\\n[rst2man-indent-level]]u
 ..
-.TH "BORG-COMPRESSION" 1 "2023-02-26" "" "borg backup tool"
+.TH "BORG-COMPRESSION" 1 "2023-06-11" "" "borg backup tool"
 .SH NAME
 borg-compression \- Details regarding compression
 .SH DESCRIPTION
@@ -82,29 +82,70 @@ being any valid compression specifier.
 .TP
 .B obfuscate,SPEC,C[,L]
 Use compressed\-size obfuscation to make fingerprinting attacks based on
-the observable stored chunk size more difficult.
-Note:
-\- you must combine this with encryption or it won\(aqt make any sense.
-\- your repo size will be bigger, of course.
+the observable stored chunk size more difficult. Note:
+.INDENT 7.0
+.IP \(bu 2
+You must combine this with encryption, or it won\(aqt make any sense.
+.IP \(bu 2
+Your repo size will be bigger, of course.
+.IP \(bu 2
+A chunk is limited by the constant \fBMAX_DATA_SIZE\fP (cur. ~20MiB).
+.UNINDENT
+.sp
+The SPEC value determines how the size obfuscation works:
 .sp
-The SPEC value will determine how the size obfuscation will work:
+\fIRelative random reciprocal size variation\fP (multiplicative)
 .sp
-Relative random reciprocal size variation:
 Size will increase by a factor, relative to the compressed data size.
-Smaller factors are often used, larger factors rarely.
-1: factor 0.01 .. 100.0
-2: factor 0.1 .. 1000.0
-3: factor 1.0 .. 10000.0
-4: factor 10.0 .. 100000.0
-5: factor 100.0 .. 1000000.0
-6: factor 1000.0 .. 10000000.0
+Smaller factors are used often, larger factors rarely.
+.sp
+Available factors:
+.INDENT 7.0
+.INDENT 3.5
+.sp
+.nf
+.ft C
+1:     0.01 ..        100
+2:     0.1  ..      1,000
+3:     1    ..     10,000
+4:    10    ..    100,000
+5:   100    ..  1,000,000
+6: 1,000    .. 10,000,000
+.ft P
+.fi
+.UNINDENT
+.UNINDENT
+.sp
+Example probabilities for SPEC \fB1\fP:
+.INDENT 7.0
+.INDENT 3.5
+.sp
+.nf
+.ft C
+90   %  0.01 ..   0.1
+ 9   %  0.1  ..   1
+ 0.9 %  1    ..  10
+ 0.09% 10    .. 100
+.ft P
+.fi
+.UNINDENT
+.UNINDENT
+.sp
+\fIRandomly sized padding up to the given size\fP (additive)
+.INDENT 7.0
+.INDENT 3.5
 .sp
-Add a randomly sized padding up to the given size:
-110: 1kiB
+.nf
+.ft C
+110: 1kiB (2 ^ (SPEC \- 100))
 \&...
 120: 1MiB
 \&...
 123: 8MiB (max.)
+.ft P
+.fi
+.UNINDENT
+.UNINDENT
 .UNINDENT
 .sp
 Examples:
@@ -120,7 +161,7 @@ borg create \-\-compression zlib REPO::ARCHIVE data
 borg create \-\-compression zlib,1 REPO::ARCHIVE data
 borg create \-\-compression auto,lzma,6 REPO::ARCHIVE data
 borg create \-\-compression auto,lzma ...
-borg create \-\-compression obfuscate,3,none ...
+borg create \-\-compression obfuscate,110,none ...
 borg create \-\-compression obfuscate,3,auto,zstd,10 ...
 borg create \-\-compression obfuscate,2,zstd,6 ...
 .ft P

docs/man/borg-config.1

@@ -27,7 +27,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .in \\n[rst2man-indent\\n[rst2man-indent-level]]u
 ..
-.TH "BORG-CONFIG" 1 "2023-02-26" "" "borg backup tool"
+.TH "BORG-CONFIG" 1 "2023-06-11" "" "borg backup tool"
 .SH NAME
 borg-config \- get, set, and delete values in a repository or cache config file
 .SH SYNOPSIS

docs/man/borg-create.1

@@ -27,7 +27,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .in \\n[rst2man-indent\\n[rst2man-indent-level]]u
 ..
-.TH "BORG-CREATE" 1 "2023-02-26" "" "borg backup tool"
+.TH "BORG-CREATE" 1 "2023-06-11" "" "borg backup tool"
 .SH NAME
 borg-create \- Create new archive
 .SH SYNOPSIS
@@ -169,15 +169,15 @@ set mode to M in archive for stdin data (default: 0660)
 interpret PATH as command and store its stdout. See also section Reading from stdin below.
 .TP
 .B  \-\-paths\-from\-stdin
-read DELIM\-separated list of paths to back up from stdin. Will not recurse into directories.
+read DELIM\-separated list of paths to back up from stdin. All control is external: it will back up all files given \- no more, no less.
 .TP
 .B  \-\-paths\-from\-command
 interpret PATH as command and treat its output as \fB\-\-paths\-from\-stdin\fP
 .TP
 .BI \-\-paths\-delimiter \ DELIM
-set path delimiter for \fB\-\-paths\-from\-stdin\fP and \fB\-\-paths\-from\-command\fP (default: n)
+set path delimiter for \fB\-\-paths\-from\-stdin\fP and \fB\-\-paths\-from\-command\fP (default: \fB\en\fP)
 .UNINDENT
-.SS Exclusion options
+.SS Include/Exclude options
 .INDENT 0.0
 .TP
 .BI \-e \ PATTERN\fR,\fB \ \-\-exclude \ PATTERN
@@ -208,7 +208,7 @@ exclude files flagged NODUMP
 .INDENT 0.0
 .TP
 .B  \-x\fP,\fB  \-\-one\-file\-system
-stay in the same file system and do not store mount points of other file systems.  This might behave different from your expectations, see the docs.
+stay in the same file system and do not store mount points of other file systems \- this might behave different from your expectations, see the description below.
 .TP
 .B  \-\-numeric\-ids
 only store numeric user and group identifiers
@@ -368,13 +368,13 @@ through using the \fB\-\-keep\-exclude\-tags\fP option.
 The \fB\-x\fP or \fB\-\-one\-file\-system\fP option excludes directories, that are mountpoints (and everything in them).
 It detects mountpoints by comparing the device number from the output of \fBstat()\fP of the directory and its
 parent directory. Specifically, it excludes directories for which \fBstat()\fP reports a device number different
-from the device number of their parent. Be aware that in Linux (and possibly elsewhere) there are directories
-with device number different from their parent, which the kernel does not consider a mountpoint and also the
-other way around. Examples are bind mounts (possibly same device number, but always a mountpoint) and ALL
-subvolumes of a btrfs (different device number from parent but not necessarily a mountpoint). Therefore when
-using \fB\-\-one\-file\-system\fP, one should make doubly sure that the backup works as intended especially when using
-btrfs. This is even more important, if the btrfs layout was created by someone else, e.g. a distribution
-installer.
+from the device number of their parent.
+In general: be aware that there are directories with device number different from their parent, which the kernel
+does not consider a mountpoint and also the other way around.
+Linux examples for this are bind mounts (possibly same device number, but always a mountpoint) and ALL
+subvolumes of a btrfs (different device number from parent but not necessarily a mountpoint).
+macOS examples are the apfs mounts of a typical macOS installation.
+Therefore, when using \fB\-\-one\-file\-system\fP, you should double\-check that the backup works as intended.
 .SS Item flags
 .sp
 \fB\-\-list\fP outputs a list of all files, directories and other

docs/man/borg-delete.1

@@ -27,7 +27,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .in \\n[rst2man-indent\\n[rst2man-indent-level]]u
 ..
-.TH "BORG-DELETE" 1 "2023-02-26" "" "borg backup tool"
+.TH "BORG-DELETE" 1 "2023-06-11" "" "borg backup tool"
 .SH NAME
 borg-delete \- Delete archives
 .SH SYNOPSIS
@@ -70,9 +70,6 @@ consider checkpoint archives for deletion (default: not considered).
 .B  \-s\fP,\fB  \-\-stats
 print statistics for the deleted archive
 .TP
-.B  \-\-cache\-only
-delete only the local cache for the given repository
-.TP
 .B  \-\-force
 force deletion of corrupted archives, use \fB\-\-force \-\-force\fP in case \fB\-\-force\fP does not work.
 .TP
@@ -118,13 +115,13 @@ $ borg delete Monday
 $ borg compact
 
 # delete all archives whose names begin with the machine\(aqs hostname followed by \(dq\-\(dq
-$ borg delete \-a \(aq{hostname}\-*\(aq
+$ borg delete \-a \(aqsh:{hostname}\-*\(aq
 
 # delete all archives whose names contain \(dq\-2012\-\(dq
-$ borg delete \-a \(aq*\-2012\-*\(aq
+$ borg delete \-a \(aqsh:*\-2012\-*\(aq
 
 # see what would be deleted if delete was run without \-\-dry\-run
-$ borg delete \-\-list \-\-dry\-run \-a \(aq*\-May\-*\(aq
+$ borg delete \-\-list \-\-dry\-run \-a \(aqsh:*\-May\-*\(aq
 .ft P
 .fi
 .UNINDENT

docs/man/borg-diff.1

@@ -27,7 +27,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .in \\n[rst2man-indent\\n[rst2man-indent-level]]u
 ..
-.TH "BORG-DIFF" 1 "2023-02-26" "" "borg backup tool"
+.TH "BORG-DIFF" 1 "2023-06-11" "" "borg backup tool"
 .SH NAME
 borg-diff \- Diff contents of two archives
 .SH SYNOPSIS
@@ -35,20 +35,7 @@ borg-diff \- Diff contents of two archives
 borg [common options] diff [options] ARCHIVE1 ARCHIVE2 [PATH...]
 .SH DESCRIPTION
 .sp
-This command finds differences (file contents, user/group/mode) between archives.
-.sp
-A repository location and an archive name must be specified for REPO::ARCHIVE1.
-ARCHIVE2 is just another archive name in same repository (no repository location
-allowed).
-.sp
-For archives created with Borg 1.1 or newer diff automatically detects whether
-the archives are created with the same chunker params. If so, only chunk IDs
-are compared, which is very fast.
-.sp
-For archives prior to Borg 1.1 chunk contents are compared by default.
-If you did not create the archives with different chunker params,
-pass \fB\-\-same\-chunker\-params\fP\&.
-Note that the chunker params changed from Borg 0.xx to 1.0.
+This command finds differences (file contents, metadata) between ARCHIVE1 and ARCHIVE2.
 .sp
 For more help on include/exclude patterns, see the \fIborg_patterns\fP command output.
 .SH OPTIONS
@@ -78,10 +65,16 @@ Override check of chunker parameters.
 .B  \-\-sort
 Sort the output lines by file path.
 .TP
+.BI \-\-format \ FORMAT
+specify format for differences between archives (default: \(dq{change} {path}{NL}\(dq)
+.TP
 .B  \-\-json\-lines
 Format output as JSON Lines.
+.TP
+.B  \-\-content\-only
+Only compare differences in content (exclude metadata differences)
 .UNINDENT
-.SS Exclusion options
+.SS Include/Exclude options
 .INDENT 0.0
 .TP
 .BI \-e \ PATTERN\fR,\fB \ \-\-exclude \ PATTERN
@@ -117,6 +110,86 @@ $ borg diff archive1 archive2
 .fi
 .UNINDENT
 .UNINDENT
+.SH NOTES
+.SS The FORMAT specifier syntax
+.sp
+The \fB\-\-format\fP option uses python\(aqs \fI\%format string syntax\fP\&.
+.sp
+Examples:
+.INDENT 0.0
+.INDENT 3.5
+.sp
+.nf
+.ft C
+$ borg diff \-\-format \(aq{content:30} {path}{NL}\(aq ArchiveFoo ArchiveBar
+modified:  +4.1 kB  \-1.0 kB    file\-diff
+\&...
+
+# {VAR:<NUMBER} \- pad to NUMBER columns left\-aligned.
+# {VAR:>NUMBER} \- pad to NUMBER columns right\-aligned.
+$ borg diff \-\-format \(aq{content:>30} {path}{NL}\(aq ArchiveFoo ArchiveBar
+   modified:  +4.1 kB  \-1.0 kB file\-diff
+\&...
+.ft P
+.fi
+.UNINDENT
+.UNINDENT
+.sp
+The following keys are always available:
+.INDENT 0.0
+.IP \(bu 2
+NEWLINE: OS dependent line separator
+.IP \(bu 2
+NL: alias of NEWLINE
+.IP \(bu 2
+NUL: NUL character for creating print0 / xargs \-0 like output
+.IP \(bu 2
+SPACE: space character
+.IP \(bu 2
+TAB: tab character
+.IP \(bu 2
+CR: carriage return character
+.IP \(bu 2
+LF: line feed character
+.UNINDENT
+.sp
+Keys available only when showing differences between archives:
+.INDENT 0.0
+.IP \(bu 2
+path: archived file path
+.IP \(bu 2
+change: all available changes
+.IP \(bu 2
+content: file content change
+.IP \(bu 2
+mode: file mode change
+.IP \(bu 2
+type: file type change
+.IP \(bu 2
+owner: file owner (user/group) change
+.IP \(bu 2
+group: file group change
+.IP \(bu 2
+user: file user change
+.IP \(bu 2
+link: file link change
+.IP \(bu 2
+directory: file directory change
+.IP \(bu 2
+blkdev: file block device change
+.IP \(bu 2
+chrdev: file character device change
+.IP \(bu 2
+fifo: file fifo change
+.IP \(bu 2
+mtime: file modification time change
+.IP \(bu 2
+ctime: file change time change
+.IP \(bu 2
+isomtime: file modification time change (ISO 8601)
+.IP \(bu 2
+isoctime: file creation time change (ISO 8601)
+.UNINDENT
 .SH SEE ALSO
 .sp
 \fIborg\-common(1)\fP

docs/man/borg-export-tar.1

@@ -27,7 +27,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .in \\n[rst2man-indent\\n[rst2man-indent-level]]u
 ..
-.TH "BORG-EXPORT-TAR" 1 "2023-02-26" "" "borg backup tool"
+.TH "BORG-EXPORT-TAR" 1 "2023-06-11" "" "borg backup tool"
 .SH NAME
 borg-export-tar \- Export archive contents as a tarball
 .SH SYNOPSIS
@@ -136,7 +136,7 @@ output verbose list of items (files, dirs, ...)
 .BI \-\-tar\-format \ FMT
 select tar format: BORG, PAX or GNU
 .UNINDENT
-.SS Exclusion options
+.SS Include/Exclude options
 .INDENT 0.0
 .TP
 .BI \-e \ PATTERN\fR,\fB \ \-\-exclude \ PATTERN

docs/man/borg-extract.1

@@ -27,7 +27,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .in \\n[rst2man-indent\\n[rst2man-indent-level]]u
 ..
-.TH "BORG-EXTRACT" 1 "2023-02-26" "" "borg backup tool"
+.TH "BORG-EXTRACT" 1 "2023-06-11" "" "borg backup tool"
 .SH NAME
 borg-extract \- Extract archive contents
 .SH SYNOPSIS
@@ -98,8 +98,11 @@ write all extracted data to stdout
 .TP
 .B  \-\-sparse
 create holes in output sparse file from all\-zero chunks
+.TP
+.B  \-\-continue
+continue a previously interrupted extraction of same archive
 .UNINDENT
-.SS Exclusion options
+.SS Include/Exclude options
 .INDENT 0.0
 .TP
 .BI \-e \ PATTERN\fR,\fB \ \-\-exclude \ PATTERN

docs/man/borg-import-tar.1

@@ -27,7 +27,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .in \\n[rst2man-indent\\n[rst2man-indent-level]]u
 ..
-.TH "BORG-IMPORT-TAR" 1 "2023-02-26" "" "borg backup tool"
+.TH "BORG-IMPORT-TAR" 1 "2023-06-11" "" "borg backup tool"
 .SH NAME
 borg-import-tar \- Create a backup archive from a tarball
 .SH SYNOPSIS
@@ -80,6 +80,10 @@ UNIX V7 tar
 .IP \(bu 2
 SunOS tar with extended attributes
 .UNINDENT
+.sp
+To import multiple tarballs into a single archive, they can be simply
+concatenated (e.g. using \(dqcat\(dq) into a single file, and imported with an
+\fB\-\-ignore\-zeros\fP option to skip through the stop markers between them.
 .SH OPTIONS
 .sp
 See \fIborg\-common(1)\fP for common options of Borg commands.
@@ -109,6 +113,9 @@ only display items with the given status characters
 .TP
 .B  \-\-json
 output stats as JSON (implies \-\-stats)
+.TP
+.B  \-\-ignore\-zeros
+ignore zero\-filled blocks in the input tarball
 .UNINDENT
 .SS Archive options
 .INDENT 0.0

docs/man/borg-info.1

@@ -27,7 +27,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .in \\n[rst2man-indent\\n[rst2man-indent-level]]u
 ..
-.TH "BORG-INFO" 1 "2023-02-26" "" "borg backup tool"
+.TH "BORG-INFO" 1 "2023-06-11" "" "borg backup tool"
 .SH NAME
 borg-info \- Show archive details such as disk space used
 .SH SYNOPSIS

docs/man/borg-key-change-location.1

@@ -27,7 +27,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .in \\n[rst2man-indent\\n[rst2man-indent-level]]u
 ..
-.TH "BORG-KEY-CHANGE-LOCATION" 1 "2023-02-26" "" "borg backup tool"
+.TH "BORG-KEY-CHANGE-LOCATION" 1 "2023-06-11" "" "borg backup tool"
 .SH NAME
 borg-key-change-location \- Change repository key location
 .SH SYNOPSIS

docs/man/borg-key-change-passphrase.1

@@ -27,7 +27,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .in \\n[rst2man-indent\\n[rst2man-indent-level]]u
 ..
-.TH "BORG-KEY-CHANGE-PASSPHRASE" 1 "2023-02-26" "" "borg backup tool"
+.TH "BORG-KEY-CHANGE-PASSPHRASE" 1 "2023-06-11" "" "borg backup tool"
 .SH NAME
 borg-key-change-passphrase \- Change repository key file passphrase
 .SH SYNOPSIS

docs/man/borg-key-export.1

@@ -27,7 +27,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .in \\n[rst2man-indent\\n[rst2man-indent-level]]u
 ..
-.TH "BORG-KEY-EXPORT" 1 "2023-02-26" "" "borg backup tool"
+.TH "BORG-KEY-EXPORT" 1 "2023-06-11" "" "borg backup tool"
 .SH NAME
 borg-key-export \- Export the repository key for backup
 .SH SYNOPSIS

docs/man/borg-key-import.1

@@ -27,7 +27,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .in \\n[rst2man-indent\\n[rst2man-indent-level]]u
 ..
-.TH "BORG-KEY-IMPORT" 1 "2023-02-26" "" "borg backup tool"
+.TH "BORG-KEY-IMPORT" 1 "2023-06-11" "" "borg backup tool"
 .SH NAME
 borg-key-import \- Import the repository key from backup
 .SH SYNOPSIS

docs/man/borg-key.1

@@ -27,7 +27,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .in \\n[rst2man-indent\\n[rst2man-indent-level]]u
 ..
-.TH "BORG-KEY" 1 "2023-02-26" "" "borg backup tool"
+.TH "BORG-KEY" 1 "2023-06-11" "" "borg backup tool"
 .SH NAME
 borg-key \- Manage a keyfile or repokey of a repository
 .SH SYNOPSIS

docs/man/borg-list.1

@@ -27,7 +27,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .in \\n[rst2man-indent\\n[rst2man-indent-level]]u
 ..
-.TH "BORG-LIST" 1 "2023-02-26" "" "borg backup tool"
+.TH "BORG-LIST" 1 "2023-06-11" "" "borg backup tool"
 .SH NAME
 borg-list \- List archive contents
 .SH SYNOPSIS
@@ -62,7 +62,7 @@ specify format for file listing (default: \(dq{mode} {user:6} {group:6} {size:8}
 .B  \-\-json\-lines
 Format output as JSON Lines. The form of \fB\-\-format\fP is ignored, but keys used in it are added to the JSON output. Some keys are always present. Note: JSON can only represent text.
 .UNINDENT
-.SS Exclusion options
+.SS Include/Exclude options
 .INDENT 0.0
 .TP
 .BI \-e \ PATTERN\fR,\fB \ \-\-exclude \ PATTERN
@@ -152,29 +152,29 @@ NL: alias of NEWLINE
 .IP \(bu 2
 NUL: NUL character for creating print0 / xargs \-0 like output
 .IP \(bu 2
-SPACE
+SPACE: space character
 .IP \(bu 2
-TAB
+TAB: tab character
 .IP \(bu 2
-CR
+CR: carriage return character
 .IP \(bu 2
-LF
+LF: line feed character
 .UNINDENT
 .sp
 Keys available only when listing files in an archive:
 .INDENT 0.0
 .IP \(bu 2
-type
+type: file type (file, dir, symlink, ...)
 .IP \(bu 2
-mode
+mode: file mode (as in stat)
 .IP \(bu 2
-uid
+uid: user id of file owner
 .IP \(bu 2
-gid
+gid: group id of file owner
 .IP \(bu 2
-user
+user: user name of file owner
 .IP \(bu 2
-group
+group: group name of file owner
 .IP \(bu 2
 path: file path
 .IP \(bu 2
@@ -182,9 +182,9 @@ target: link target for symlinks
 .IP \(bu 2
 hlid: hard link identity (same if hardlinking same fs object)
 .IP \(bu 2
-flags
+flags: file flags
 .IP \(bu 2
-size
+size: file size
 .IP \(bu 2
 dsize: deduplicated size
 .IP \(bu 2
@@ -192,17 +192,17 @@ num_chunks: number of chunks in this file
 .IP \(bu 2
 unique_chunks: number of unique chunks in this file
 .IP \(bu 2
-mtime
+mtime: file modification time
 .IP \(bu 2
-ctime
+ctime: file change time
 .IP \(bu 2
-atime
+atime: file access time
 .IP \(bu 2
-isomtime
+isomtime: file modification time (ISO 8601 format)
 .IP \(bu 2
-isoctime
+isoctime: file change time (ISO 8601 format)
 .IP \(bu 2
-isoatime
+isoatime: file access time (ISO 8601 format)
 .IP \(bu 2
 blake2b
 .IP \(bu 2
@@ -230,9 +230,9 @@ sha512
 .IP \(bu 2
 xxh64: XXH64 checksum of this file (note: this is NOT a cryptographic hash!)
 .IP \(bu 2
-archiveid
+archiveid: internal ID of the archive
 .IP \(bu 2
-archivename
+archivename: name of the archive
 .IP \(bu 2
 extra: prepends {target} with \(dq \-> \(dq for soft links and \(dq link to \(dq for hard links
 .IP \(bu 2

docs/man/borg-match-archives.1

@@ -27,7 +27,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .in \\n[rst2man-indent\\n[rst2man-indent-level]]u
 ..
-.TH "BORG-MATCH-ARCHIVES" 1 "2023-02-26" "" "borg backup tool"
+.TH "BORG-MATCH-ARCHIVES" 1 "2023-06-11" "" "borg backup tool"
 .SH NAME
 borg-match-archives \- Details regarding match-archives
 .SH DESCRIPTION

docs/man/borg-mount.1

@@ -27,7 +27,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .in \\n[rst2man-indent\\n[rst2man-indent-level]]u
 ..
-.TH "BORG-MOUNT" 1 "2023-02-26" "" "borg backup tool"
+.TH "BORG-MOUNT" 1 "2023-06-11" "" "borg backup tool"
 .SH NAME
 borg-mount \- Mount archive or an entire repository as a FUSE filesystem
 .SH SYNOPSIS
@@ -72,16 +72,16 @@ manually. Unlike the \fBuid\fP and \fBgid\fP mount options which affect all file
 Additional mount options supported by borg:
 .INDENT 0.0
 .IP \(bu 2
-versions: when used with a repository mount, this gives a merged, versioned
+\fBversions\fP: when used with a repository mount, this gives a merged, versioned
 view of the files in the archives. EXPERIMENTAL, layout may change in future.
 .IP \(bu 2
-allow_damaged_files: by default damaged files (where missing chunks were
-replaced with runs of zeros by borg check \fB\-\-repair\fP) are not readable and
+\fBallow_damaged_files\fP: by default damaged files (where missing chunks were
+replaced with runs of zeros by \fBborg check \-\-repair\fP) are not readable and
 return EIO (I/O error). Set this option to read such files.
 .IP \(bu 2
-ignore_permissions: for security reasons the \(dqdefault_permissions\(dq mount
-option is internally enforced by borg. \(dqignore_permissions\(dq can be given to
-not enforce \(dqdefault_permissions\(dq.
+\fBignore_permissions\fP: for security reasons the \fBdefault_permissions\fP mount
+option is internally enforced by borg. \fBignore_permissions\fP can be given to
+not enforce \fBdefault_permissions\fP\&.
 .UNINDENT
 .sp
 The BORG_MOUNT_DATA_CACHE_ENTRIES environment variable is meant for advanced users
@@ -149,7 +149,7 @@ consider archives older than (now \- TIMESPAN), e.g. 7d oder 12m.
 .BI \-\-newer \ TIMESPAN
 consider archives newer than (now \- TIMESPAN), e.g. 7d or 12m.
 .UNINDENT
-.SS Exclusion options
+.SS Include/Exclude options
 .INDENT 0.0
 .TP
 .BI \-e \ PATTERN\fR,\fB \ \-\-exclude \ PATTERN

docs/man/borg-patterns.1

@@ -27,7 +27,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .in \\n[rst2man-indent\\n[rst2man-indent-level]]u
 ..
-.TH "BORG-PATTERNS" 1 "2023-02-26" "" "borg backup tool"
+.TH "BORG-PATTERNS" 1 "2023-06-11" "" "borg backup tool"
 .SH NAME
 borg-patterns \- Details regarding patterns
 .SH DESCRIPTION
@@ -56,15 +56,13 @@ confuse the root with the \fBPATH\fP argument of e.g. \fIborg extract\fP\&.
 .sp
 Starting with Borg 1.2, paths that are matched against patterns always
 appear relative. If you give \fB/absolute/\fP as root, the paths going
-into the matcher will look relative like \fBabsolute/.../file.ext\fP\&.
-If you give \fB\&../some/path\fP as root, the paths will look like
-\fBsome/path/.../file.ext\fP\&.
+into the matcher will start with \fBabsolute/\fP\&.
+If you give \fB\&../../relative\fP as root, the paths will be normalized
+as \fBrelative/\fP\&.
 .sp
-File patterns support five different styles. If followed by a colon \(aq:\(aq,
-the first two characters of a pattern are used as a style selector.
-Explicit style selection is necessary if a non\-default style is desired
-or when the desired pattern starts with two alphanumeric characters
-followed by a colon (i.e. \fBaa:something/*\fP).
+Borg supports different pattern styles. To define a non\-default
+style for a specific pattern, prefix it with two characters followed
+by a colon \(aq:\(aq (i.e. \fBfm:path/*\fP, \fBsh:path/**\fP).
 .INDENT 0.0
 .TP
 .B \fI\%Fnmatch\fP, selector \fBfm:\fP
@@ -88,7 +86,8 @@ This is the default style for \fB\-\-pattern\fP and \fB\-\-patterns\-from\fP\&.
 Like fnmatch patterns these are similar to shell patterns. The difference
 is that the pattern may include \fB**/\fP for matching zero or more directory
 levels, \fB*\fP for matching zero or more arbitrary characters with the
-exception of any path separator. A leading path separator is always removed.
+exception of any path separator, \fB{}\fP containing comma\-separated
+alternative patterns. A leading path separator is always removed.
 .TP
 .B \fI\%Regular expressions\fP, selector \fBre:\fP
 Unlike shell patterns, regular expressions are not required to match the full
@@ -222,6 +221,33 @@ before an exclude pattern, the file is backed up. Note that a no\-recurse
 exclude stops examination of subdirectories so that potential includes
 will not match \- use normal excludes for such use cases.
 .sp
+Example:
+.INDENT 0.0
+.INDENT 3.5
+.sp
+.nf
+.ft C
+# Define the recursion root
+R /
+# Exclude all iso files in any directory
+\- **/*.iso
+# Explicitly include all inside etc and root
++ etc/**
++ root/**
+# Exclude a specific directory under each user\(aqs home directories
+\- home/*/.cache
+# Explicitly include everything in /home
++ home/**
+# Explicitly exclude some directories without recursing into them
+! re:^(dev|proc|run|sys|tmp)
+# Exclude all other files and directories
+# that are not specifically included earlier.
+\- **
+.ft P
+.fi
+.UNINDENT
+.UNINDENT
+.sp
 \fBTip: You can easily test your patterns with \-\-dry\-run and  \-\-list\fP:
 .INDENT 0.0
 .INDENT 3.5

docs/man/borg-placeholders.1

@@ -27,7 +27,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .in \\n[rst2man-indent\\n[rst2man-indent-level]]u
 ..
-.TH "BORG-PLACEHOLDERS" 1 "2023-02-26" "" "borg backup tool"
+.TH "BORG-PLACEHOLDERS" 1 "2023-06-11" "" "borg backup tool"
 .SH NAME
 borg-placeholders \- Details regarding placeholders
 .SH DESCRIPTION

docs/man/borg-prune.1

@@ -27,7 +27,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .in \\n[rst2man-indent\\n[rst2man-indent-level]]u
 ..
-.TH "BORG-PRUNE" 1 "2023-02-26" "" "borg backup tool"
+.TH "BORG-PRUNE" 1 "2023-06-11" "" "borg backup tool"
 .SH NAME
 borg-prune \- Prune repository archives according to specified rules
 .SH SYNOPSIS
@@ -89,6 +89,10 @@ When using \fB\-\-stats\fP, you will get some statistics about how much data was
 deleted \- the \(dqDeleted data\(dq deduplicated size there is most interesting as
 that is how much your repository will shrink.
 Please note that the \(dqAll archives\(dq stats refer to the state after pruning.
+.sp
+You can influence how the \fB\-\-list\fP output is formatted by using the \fB\-\-short\fP
+option (less wide output) or by giving a custom format using \fB\-\-format\fP (see
+the \fBborg rlist\fP description for more details about the format string).
 .SH OPTIONS
 .sp
 See \fIborg\-common(1)\fP for common options of Borg commands.
@@ -107,6 +111,18 @@ print statistics for the deleted archive
 .B  \-\-list
 output verbose list of archives it keeps/prunes
 .TP
+.B  \-\-short
+use a less wide archive part format
+.TP
+.B  \-\-list\-pruned
+output verbose list of archives it prunes
+.TP
+.B  \-\-list\-kept
+output verbose list of archives it keeps
+.TP
+.BI \-\-format \ FORMAT
+specify format for the archive part (default: \(dq{archive:<36} {time} [{id}]\(dq)
+.TP
 .BI \-\-keep\-within \ INTERVAL
 keep all archives within this time interval
 .TP
@@ -158,7 +174,7 @@ Be careful, prune is a potentially dangerous command, it will remove backup
 archives.
 .sp
 The default of prune is to apply to \fBall archives in the repository\fP unless
-you restrict its operation to a subset of the archives using \fB\-a\fP / \fB\-\-glob\-archives\fP\&.
+you restrict its operation to a subset of the archives using \fB\-a\fP / \fB\-\-match\-archives\fP\&.
 When using \fB\-a\fP, be careful to choose a good pattern \- e.g. do not use a
 prefix \(dqfoo\(dq if you do not also want to match \(dqfoobar\(dq.
 .sp
@@ -175,7 +191,7 @@ $ borg prune \-v \-\-list \-\-dry\-run \-\-keep\-daily=7 \-\-keep\-weekly=4
 
 # Same as above but only apply to archive names starting with the hostname
 # of the machine followed by a \(dq\-\(dq character:
-$ borg prune \-v \-\-list \-\-keep\-daily=7 \-\-keep\-weekly=4 \-a \(aq{hostname}\-*\(aq
+$ borg prune \-v \-\-list \-\-keep\-daily=7 \-\-keep\-weekly=4 \-a \(aqsh:{hostname}\-*\(aq
 # actually free disk space:
 $ borg compact
 

docs/man/borg-rcompress.1

@@ -27,7 +27,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .in \\n[rst2man-indent\\n[rst2man-indent-level]]u
 ..
-.TH "BORG-RCOMPRESS" 1 "2023-02-26" "" "borg backup tool"
+.TH "BORG-RCOMPRESS" 1 "2023-06-11" "" "borg backup tool"
 .SH NAME
 borg-rcompress \- Repository (re-)compression
 .SH SYNOPSIS

docs/man/borg-rcreate.1

@@ -27,7 +27,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .in \\n[rst2man-indent\\n[rst2man-indent-level]]u
 ..
-.TH "BORG-RCREATE" 1 "2023-02-26" "" "borg backup tool"
+.TH "BORG-RCREATE" 1 "2023-06-11" "" "borg backup tool"
 .SH NAME
 borg-rcreate \- Create a new, empty repository
 .SH SYNOPSIS

docs/man/borg-rdelete.1

@@ -27,7 +27,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .in \\n[rst2man-indent\\n[rst2man-indent-level]]u
 ..
-.TH "BORG-RDELETE" 1 "2023-02-26" "" "borg backup tool"
+.TH "BORG-RDELETE" 1 "2023-06-11" "" "borg backup tool"
 .SH NAME
 borg-rdelete \- Delete a repository
 .SH SYNOPSIS

docs/man/borg-recreate.1

@@ -27,7 +27,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .in \\n[rst2man-indent\\n[rst2man-indent-level]]u
 ..
-.TH "BORG-RECREATE" 1 "2023-02-26" "" "borg backup tool"
+.TH "BORG-RECREATE" 1 "2023-06-11" "" "borg backup tool"
 .SH NAME
 borg-recreate \- Re-create archives
 .SH SYNOPSIS
@@ -108,7 +108,7 @@ do not change anything
 .B  \-s\fP,\fB  \-\-stats
 print statistics at end
 .UNINDENT
-.SS Exclusion options
+.SS Include/Exclude options
 .INDENT 0.0
 .TP
 .BI \-e \ PATTERN\fR,\fB \ \-\-exclude \ PATTERN

docs/man/borg-rename.1

@@ -27,7 +27,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .in \\n[rst2man-indent\\n[rst2man-indent-level]]u
 ..
-.TH "BORG-RENAME" 1 "2023-02-26" "" "borg backup tool"
+.TH "BORG-RENAME" 1 "2023-06-11" "" "borg backup tool"
 .SH NAME
 borg-rename \- Rename an existing archive
 .SH SYNOPSIS

docs/man/borg-rinfo.1

@@ -27,7 +27,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .in \\n[rst2man-indent\\n[rst2man-indent-level]]u
 ..
-.TH "BORG-RINFO" 1 "2023-02-26" "" "borg backup tool"
+.TH "BORG-RINFO" 1 "2023-06-11" "" "borg backup tool"
 .SH NAME
 borg-rinfo \- Show repository infos
 .SH SYNOPSIS

docs/man/borg-rlist.1

@@ -27,7 +27,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .in \\n[rst2man-indent\\n[rst2man-indent-level]]u
 ..
-.TH "BORG-RLIST" 1 "2023-02-26" "" "borg backup tool"
+.TH "BORG-RLIST" 1 "2023-06-11" "" "borg backup tool"
 .SH NAME
 borg-rlist \- List the archives contained in a repository
 .SH SYNOPSIS
@@ -133,13 +133,13 @@ NL: alias of NEWLINE
 .IP \(bu 2
 NUL: NUL character for creating print0 / xargs \-0 like output
 .IP \(bu 2
-SPACE
+SPACE: space character
 .IP \(bu 2
-TAB
+TAB: tab character
 .IP \(bu 2
-CR
+CR: carriage return character
 .IP \(bu 2
-LF
+LF: line feed character
 .UNINDENT
 .sp
 Keys available only when listing archives in a repository:
@@ -164,6 +164,10 @@ command_line: command line which was used to create the archive
 hostname: hostname of host on which this archive was created
 .IP \(bu 2
 username: username of user who created this archive
+.IP \(bu 2
+size: size of this archive (data plus metadata, not considering compression and deduplication)
+.IP \(bu 2
+nfiles: count of files in this archive
 .UNINDENT
 .SH SEE ALSO
 .sp

docs/man/borg-serve.1

@@ -27,7 +27,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .in \\n[rst2man-indent\\n[rst2man-indent-level]]u
 ..
-.TH "BORG-SERVE" 1 "2023-02-26" "" "borg backup tool"
+.TH "BORG-SERVE" 1 "2023-06-11" "" "borg backup tool"
 .SH NAME
 borg-serve \- Start in server mode. This command is usually not used manually.
 .SH SYNOPSIS
@@ -35,7 +35,19 @@ borg-serve \- Start in server mode. This command is usually not used manually.
 borg [common options] serve [options]
 .SH DESCRIPTION
 .sp
-This command starts a repository server process. This command is usually not used manually.
+This command starts a repository server process.
+.sp
+borg serve can currently support:
+.INDENT 0.0
+.IP \(bu 2
+Getting automatically started via ssh when the borg client uses a \fI\%ssh://\fP\&...
+remote repository. In this mode, \fIborg serve\fP will live until that ssh connection
+gets terminated.
+.IP \(bu 2
+Getting started by some other means (not by the borg client) as a long\-running socket
+server to be used for borg clients using a socket://... repository (see the \fI\-\-socket\fP
+option if you do not want to use the default path for the socket and pid file).
+.UNINDENT
 .SH OPTIONS
 .sp
 See \fIborg\-common(1)\fP for common options of Borg commands.

docs/man/borg-transfer.1

@@ -27,7 +27,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .in \\n[rst2man-indent\\n[rst2man-indent-level]]u
 ..
-.TH "BORG-TRANSFER" 1 "2023-02-26" "" "borg backup tool"
+.TH "BORG-TRANSFER" 1 "2023-06-11" "" "borg backup tool"
 .SH NAME
 borg-transfer \- archives transfer from other repository, optionally upgrade data format
 .SH SYNOPSIS
@@ -37,6 +37,14 @@ borg [common options] transfer [options]
 .sp
 This command transfers archives from one repository to another repository.
 Optionally, it can also upgrade the transferred data.
+Optionally, it can also recompress the transferred data.
+.sp
+It is easiest (and fastest) to give \fB\-\-compression=COMPRESSION \-\-recompress=never\fP using
+the same COMPRESSION mode as in the SRC_REPO \- borg will use that COMPRESSION for metadata (in
+any case) and keep data compressed \(dqas is\(dq (saves time as no data compression is needed).
+.sp
+If you want to globally change compression while transferring archives to the DST_REPO,
+give \fB\-\-compress=WANTED_COMPRESSION \-\-recompress=always\fP\&.
 .sp
 Suggested use for general purpose archive transfer (not repo upgrades):
 .INDENT 0.0
@@ -60,7 +68,7 @@ borg \-\-repo=DST_REPO transfer \-\-other\-repo=SRC_REPO \-\-dry\-run  # check!
 The default is to transfer all archives, including checkpoint archives.
 .sp
 You could use the misc. archive filter options to limit which archives it will
-transfer, e.g. using the \-a option. This is recommended for big
+transfer, e.g. using the \fB\-a\fP option. This is recommended for big
 repositories with multiple data sets to keep the runtime per invocation lower.
 .sp
 For repository upgrades (e.g. from a borg 1.2 repo to a related borg 2.0 repo), usage is
@@ -70,7 +78,13 @@ quite similar to the above:
 .sp
 .nf
 .ft C
-borg \-\-repo=DST_REPO transfer \-\-other\-repo=SRC_REPO \-\-upgrader=From12To20
+# fast: compress metadata with zstd,3, but keep data chunks compressed as they are:
+borg \-\-repo=DST_REPO transfer \-\-other\-repo=SRC_REPO \-\-upgrader=From12To20 \e
+     \-\-compress=zstd,3 \-\-recompress=never
+
+# compress metadata and recompress data with zstd,3
+borg \-\-repo=DST_REPO transfer \-\-other\-repo=SRC_REPO \-\-upgrader=From12To20 \e
+     \-\-compress=zstd,3 \-\-recompress=always
 .ft P
 .fi
 .UNINDENT
@@ -89,6 +103,12 @@ transfer archives from the other repository
 .TP
 .BI \-\-upgrader \ UPGRADER
 use the upgrader to convert transferred data (default: no conversion)
+.TP
+.BI \-C \ COMPRESSION\fR,\fB \ \-\-compression \ COMPRESSION
+select compression algorithm, see the output of the \(dqborg help compression\(dq command for details.
+.TP
+.BI \-\-recompress \ MODE
+recompress data chunks according to \fIMODE\fP and \fB\-\-compression\fP\&. Possible modes are \fIalways\fP: recompress unconditionally; and \fInever\fP: do not recompress (faster: re\-uses compressed data chunks w/o change).If no MODE is given, \fIalways\fP will be used. Not passing \-\-recompress is equivalent to \(dq\-\-recompress never\(dq.
 .UNINDENT
 .SS Archive filters
 .INDENT 0.0

docs/man/borg-umount.1

@@ -27,7 +27,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .in \\n[rst2man-indent\\n[rst2man-indent-level]]u
 ..
-.TH "BORG-UMOUNT" 1 "2023-02-26" "" "borg backup tool"
+.TH "BORG-UMOUNT" 1 "2023-06-11" "" "borg backup tool"
 .SH NAME
 borg-umount \- un-mount the FUSE filesystem
 .SH SYNOPSIS
@@ -74,7 +74,7 @@ $ borg umount /tmp/mymountpoint
 # Archive filters are supported.
 # These are especially handy for the \(dqversions view\(dq,
 # which does not support lazy processing of archives.
-$ borg mount \-o versions \-\-glob\-archives \(aq*\-my\-home\(aq \-\-last 10 /tmp/mymountpoint
+$ borg mount \-o versions \-\-match\-archives \(aqsh:*\-my\-home\(aq \-\-last 10 /tmp/mymountpoint
 
 # Exclusion options are supported.
 # These can speed up mounting and lower memory needs significantly.

docs/man/borg-with-lock.1

@@ -27,7 +27,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .in \\n[rst2man-indent\\n[rst2man-indent-level]]u
 ..
-.TH "BORG-WITH-LOCK" 1 "2023-02-26" "" "borg backup tool"
+.TH "BORG-WITH-LOCK" 1 "2023-06-11" "" "borg backup tool"
 .SH NAME
 borg-with-lock \- run a user specified command with the repository lock held
 .SH SYNOPSIS

docs/man/borg.1

@@ -27,7 +27,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .in \\n[rst2man-indent\\n[rst2man-indent-level]]u
 ..
-.TH "BORG" 1 "2023-02-26" "" "borg backup tool"
+.TH "BORG" 1 "2023-06-11" "" "borg backup tool"
 .SH NAME
 borg \- deduplicating and encrypting backup tool
 .SH SYNOPSIS
@@ -168,8 +168,8 @@ $ borg \-r /path/to/repo delete \-a Monday
 .UNINDENT
 .UNINDENT
 .sp
-Please note the \fB\-a\fP option here (short for \fB\-\-glob\-archives\fP) which enables you
-to give a globbing pattern to delete multiple archives, like \fB\-a \(aqoldcrap\-*\(aq\fP\&.
+Please note the \fB\-a\fP option here (short for \fB\-\-match\-archives\fP) which enables you
+to give a pattern to delete multiple archives, like \fB\-a \(aqsh:oldcrap\-*\(aq\fP\&.
 You can also combine this with \fB\-\-first\fP, \fB\-\-last\fP and \fB\-\-sort\-by\fP\&.
 Be careful, always first use with \fB\-\-dry\-run\fP and \fB\-\-list\fP!
 .IP 8. 3
@@ -496,6 +496,19 @@ in WSL1 (Windows Subsystem for Linux 1).
 .UNINDENT
 .UNINDENT
 .TP
+.B Output formatting:
+.INDENT 7.0
+.TP
+.B BORG_LIST_FORMAT
+Giving the default value for \fBborg list \-\-format=X\fP\&.
+.TP
+.B BORG_RLIST_FORMAT
+Giving the default value for \fBborg rlist \-\-format=X\fP\&.
+.TP
+.B BORG_PRUNE_FORMAT
+Giving the default value for \fBborg prune \-\-format=X\fP\&.
+.UNINDENT
+.TP
 .B Some automatic \(dqanswerers\(dq (if set, they automatically answer confirmation questions):
 .INDENT 7.0
 .TP
@@ -540,17 +553,44 @@ Defaults to \fB$BORG_BASE_DIR/.config/borg\fP\&. If \fBBORG_BASE_DIR\fP is not e
 This directory contains all borg configuration directories, see the FAQ
 for a security advisory about the data in this directory: \fIhome_config_borg\fP
 .TP
+.B BORG_DATA_DIR
+Defaults to \fB$BORG_BASE_DIR/.local/share/borg\fP\&. If \fBBORG_BASE_DIR\fP is not explicitly set while
+\fI\%XDG env var\fP \fBXDG_DATA_HOME\fP is set, then \fB$XDG_DATA_HOME/borg\fP is being used instead.
+This directory contains all borg data directories, see the FAQ
+for a security advisory about the data in this directory: \fIhome_data_borg\fP
+.TP
+.B BORG_RUNTIME_DIR
+Defaults to \fB$BORG_BASE_DIR/.cache/borg\fP\&. If \fBBORG_BASE_DIR\fP is not explicitly set while
+\fI\%XDG env var\fP \fBXDG_RUNTIME_DIR\fP is set, then \fB$XDG_RUNTIME_DIR/borg\fP is being used instead.
+This directory contains borg runtime files, like e.g. the socket file.
+.TP
 .B BORG_SECURITY_DIR
-Defaults to \fB$BORG_CONFIG_DIR/security\fP\&.
-This directory contains information borg uses to track its usage of NONCES (\(dqnumbers used
-once\(dq \- usually in encryption context) and other security relevant data.
+Defaults to \fB$BORG_DATA_DIR/security\fP\&.
+This directory contains security relevant data.
 .TP
 .B BORG_KEYS_DIR
 Defaults to \fB$BORG_CONFIG_DIR/keys\fP\&.
 This directory contains keys for encrypted repositories.
 .TP
 .B BORG_KEY_FILE
-When set, use the given filename as repository key file.
+When set, use the given path as repository key file. Please note that this is only
+for rather special applications that externally fully manage the key files:
+.INDENT 7.0
+.IP \(bu 2
+this setting only applies to the keyfile modes (not to the repokey modes).
+.IP \(bu 2
+using a full, absolute path to the key file is recommended.
+.IP \(bu 2
+all directories in the given path must exist.
+.IP \(bu 2
+this setting forces borg to use the key file at the given location.
+.IP \(bu 2
+the key file must either exist (for most commands) or will be created (\fBborg rcreate\fP).
+.IP \(bu 2
+you need to give a different path for different repositories.
+.IP \(bu 2
+you need to point to the correct key file matching the repository the command will operate on.
+.UNINDENT
 .TP
 .B TMPDIR
 This is where temporary files are stored (might need a lot of temporary space for some

docs/man/borgfs.1

@@ -27,7 +27,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .in \\n[rst2man-indent\\n[rst2man-indent-level]]u
 ..
-.TH "BORGFS" 1 "2023-02-26" "" "borg backup tool"
+.TH "BORGFS" 1 "2023-06-11" "" "borg backup tool"
 .SH NAME
 borgfs \- Mount archive or an entire repository as a FUSE filesystem
 .SH SYNOPSIS
@@ -93,7 +93,7 @@ consider archives older than (now \- TIMESPAN), e.g. 7d oder 12m.
 .BI \-\-newer \ TIMESPAN
 consider archives newer than (now \- TIMESPAN), e.g. 7d or 12m.
 .UNINDENT
-.SS Exclusion options
+.SS Include/Exclude options
 .INDENT 0.0
 .TP
 .BI \-e \ PATTERN\fR,\fB \ \-\-exclude \ PATTERN

docs/usage/common-options.rst.inc

@@ -18,4 +18,5 @@
 --upload-buffer UPLOAD_BUFFER    set network upload buffer size in MiB. (default: 0=no buffer)
 --debug-profile FILE     Write execution profile in Borg format into FILE. For local use a Python-compatible file can be generated by suffixing FILE with ".pyprof".
 --rsh RSH                Use this command to connect to the 'borg serve' process (default: 'ssh')
+--socket PATH            Use UNIX DOMAIN (IPC) socket at PATH for client/server communication with socket: protocol.
 -r REPO, --repo REPO     repository to use

docs/usage/create.rst.inc

@@ -43,17 +43,17 @@ borg create
     +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
     |                                                       | ``--content-from-command``                        | interpret PATH as command and store its stdout. See also section Reading from stdin below.                                                                                                        |
     +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
-    |                                                       | ``--paths-from-stdin``                            | read DELIM-separated list of paths to back up from stdin. Will not recurse into directories.                                                                                                      |
+    |                                                       | ``--paths-from-stdin``                            | read DELIM-separated list of paths to back up from stdin. All control is external: it will back up all files given - no more, no less.                                                            |
     +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
     |                                                       | ``--paths-from-command``                          | interpret PATH as command and treat its output as ``--paths-from-stdin``                                                                                                                          |
     +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
-    |                                                       | ``--paths-delimiter DELIM``                       | set path delimiter for ``--paths-from-stdin`` and ``--paths-from-command`` (default: \n)                                                                                                          |
+    |                                                       | ``--paths-delimiter DELIM``                       | set path delimiter for ``--paths-from-stdin`` and ``--paths-from-command`` (default: ``\n``)                                                                                                      |
     +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
     | .. class:: borg-common-opt-ref                                                                                                                                                                                                                                                                                |
     |                                                                                                                                                                                                                                                                                                               |
     | :ref:`common_options`                                                                                                                                                                                                                                                                                         |
     +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
-    | **Exclusion options**                                                                                                                                                                                                                                                                                         |
+    | **Include/Exclude options**                                                                                                                                                                                                                                                                                   |
     +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
     |                                                       | ``-e PATTERN``, ``--exclude PATTERN``             | exclude paths matching PATTERN                                                                                                                                                                    |
     +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
@@ -73,7 +73,7 @@ borg create
     +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
     | **Filesystem options**                                                                                                                                                                                                                                                                                        |
     +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
-    |                                                       | ``-x``, ``--one-file-system``                     | stay in the same file system and do not store mount points of other file systems.  This might behave different from your expectations, see the docs.                                              |
+    |                                                       | ``-x``, ``--one-file-system``                     | stay in the same file system and do not store mount points of other file systems - this might behave different from your expectations, see the description below.                                 |
     +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
     |                                                       | ``--numeric-ids``                                 | only store numeric user and group identifiers                                                                                                                                                     |
     +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
@@ -138,15 +138,15 @@ borg create
         --stdin-group GROUP    set group GROUP in archive for stdin data (default: do not store group/gid)
         --stdin-mode M    set mode to M in archive for stdin data (default: 0660)
         --content-from-command    interpret PATH as command and store its stdout. See also section Reading from stdin below.
-        --paths-from-stdin    read DELIM-separated list of paths to back up from stdin. Will not recurse into directories.
+        --paths-from-stdin    read DELIM-separated list of paths to back up from stdin. All control is external: it will back up all files given - no more, no less.
         --paths-from-command    interpret PATH as command and treat its output as ``--paths-from-stdin``
-        --paths-delimiter DELIM    set path delimiter for ``--paths-from-stdin`` and ``--paths-from-command`` (default: \n) 
+        --paths-delimiter DELIM    set path delimiter for ``--paths-from-stdin`` and ``--paths-from-command`` (default: ``\n``) 
 
 
     :ref:`common_options`
         |
 
-    Exclusion options
+    Include/Exclude options
         -e PATTERN, --exclude PATTERN     exclude paths matching PATTERN
         --exclude-from EXCLUDEFILE        read exclude patterns from EXCLUDEFILE, one per line
         --pattern PATTERN                 include/exclude paths matching PATTERN
@@ -158,7 +158,7 @@ borg create
 
 
     Filesystem options
-        -x, --one-file-system     stay in the same file system and do not store mount points of other file systems.  This might behave different from your expectations, see the docs.
+        -x, --one-file-system     stay in the same file system and do not store mount points of other file systems - this might behave different from your expectations, see the description below.
         --numeric-ids             only store numeric user and group identifiers
         --atime                   do store atime into archive
         --noctime                 do not store ctime into archive
@@ -276,13 +276,13 @@ through using the ``--keep-exclude-tags`` option.
 The ``-x`` or ``--one-file-system`` option excludes directories, that are mountpoints (and everything in them).
 It detects mountpoints by comparing the device number from the output of ``stat()`` of the directory and its
 parent directory. Specifically, it excludes directories for which ``stat()`` reports a device number different
-from the device number of their parent. Be aware that in Linux (and possibly elsewhere) there are directories
-with device number different from their parent, which the kernel does not consider a mountpoint and also the
-other way around. Examples are bind mounts (possibly same device number, but always a mountpoint) and ALL
-subvolumes of a btrfs (different device number from parent but not necessarily a mountpoint). Therefore when
-using ``--one-file-system``, one should make doubly sure that the backup works as intended especially when using
-btrfs. This is even more important, if the btrfs layout was created by someone else, e.g. a distribution
-installer.
+from the device number of their parent.
+In general: be aware that there are directories with device number different from their parent, which the kernel
+does not consider a mountpoint and also the other way around.
+Linux examples for this are bind mounts (possibly same device number, but always a mountpoint) and ALL
+subvolumes of a btrfs (different device number from parent but not necessarily a mountpoint).
+macOS examples are the apfs mounts of a typical macOS installation.
+Therefore, when using ``--one-file-system``, you should double-check that the backup works as intended.
 
 
 .. _list_item_flags:

docs/usage/delete.rst.inc

@@ -23,8 +23,6 @@ borg delete
     +-----------------------------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------+
     |                                                                             | ``-s``, ``--stats``                               | print statistics for the deleted archive                                                          |
     +-----------------------------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------+
-    |                                                                             | ``--cache-only``                                  | delete only the local cache for the given repository                                              |
-    +-----------------------------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------+
     |                                                                             | ``--force``                                       | force deletion of corrupted archives, use ``--force --force`` in case ``--force`` does not work.  |
     +-----------------------------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------+
     |                                                                             | ``-c SECONDS``, ``--checkpoint-interval SECONDS`` | write checkpoint every SECONDS seconds (Default: 1800)                                            |
@@ -69,7 +67,6 @@ borg delete
         --list          output verbose list of archives
         --consider-checkpoints    consider checkpoint archives for deletion (default: not considered).
         -s, --stats     print statistics for the deleted archive
-        --cache-only    delete only the local cache for the given repository
         --force         force deletion of corrupted archives, use ``--force --force`` in case ``--force`` does not work.
         -c SECONDS, --checkpoint-interval SECONDS    write checkpoint every SECONDS seconds (Default: 1800)
 

docs/usage/diff.rst.inc

@@ -12,39 +12,43 @@ borg diff
 
     .. class:: borg-options-table
 
-    +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------+
-    | **positional arguments**                                                                                                                                              |
-    +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------+
-    |                                                       | ``ARCHIVE1``                          | ARCHIVE1 name                                                         |
-    +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------+
-    |                                                       | ``ARCHIVE2``                          | ARCHIVE2 name                                                         |
-    +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------+
-    |                                                       | ``PATH``                              | paths of items inside the archives to compare; patterns are supported |
-    +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------+
-    | **options**                                                                                                                                                           |
-    +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------+
-    |                                                       | ``--numeric-ids``                     | only consider numeric user and group identifiers                      |
-    +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------+
-    |                                                       | ``--same-chunker-params``             | Override check of chunker parameters.                                 |
-    +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------+
-    |                                                       | ``--sort``                            | Sort the output lines by file path.                                   |
-    +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------+
-    |                                                       | ``--json-lines``                      | Format output as JSON Lines.                                          |
-    +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------+
-    | .. class:: borg-common-opt-ref                                                                                                                                        |
-    |                                                                                                                                                                       |
-    | :ref:`common_options`                                                                                                                                                 |
-    +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------+
-    | **Exclusion options**                                                                                                                                                 |
-    +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------+
-    |                                                       | ``-e PATTERN``, ``--exclude PATTERN`` | exclude paths matching PATTERN                                        |
-    +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------+
-    |                                                       | ``--exclude-from EXCLUDEFILE``        | read exclude patterns from EXCLUDEFILE, one per line                  |
-    +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------+
-    |                                                       | ``--pattern PATTERN``                 | include/exclude paths matching PATTERN                                |
-    +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------+
-    |                                                       | ``--patterns-from PATTERNFILE``       | read include/exclude patterns from PATTERNFILE, one per line          |
-    +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------+
+    +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------------+
+    | **positional arguments**                                                                                                                                                         |
+    +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------------+
+    |                                                       | ``ARCHIVE1``                          | ARCHIVE1 name                                                                    |
+    +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------------+
+    |                                                       | ``ARCHIVE2``                          | ARCHIVE2 name                                                                    |
+    +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------------+
+    |                                                       | ``PATH``                              | paths of items inside the archives to compare; patterns are supported            |
+    +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------------+
+    | **options**                                                                                                                                                                      |
+    +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------------+
+    |                                                       | ``--numeric-ids``                     | only consider numeric user and group identifiers                                 |
+    +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------------+
+    |                                                       | ``--same-chunker-params``             | Override check of chunker parameters.                                            |
+    +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------------+
+    |                                                       | ``--sort``                            | Sort the output lines by file path.                                              |
+    +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------------+
+    |                                                       | ``--format FORMAT``                   | specify format for differences between archives (default: "{change} {path}{NL}") |
+    +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------------+
+    |                                                       | ``--json-lines``                      | Format output as JSON Lines.                                                     |
+    +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------------+
+    |                                                       | ``--content-only``                    | Only compare differences in content (exclude metadata differences)               |
+    +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------------+
+    | .. class:: borg-common-opt-ref                                                                                                                                                   |
+    |                                                                                                                                                                                  |
+    | :ref:`common_options`                                                                                                                                                            |
+    +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------------+
+    | **Include/Exclude options**                                                                                                                                                      |
+    +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------------+
+    |                                                       | ``-e PATTERN``, ``--exclude PATTERN`` | exclude paths matching PATTERN                                                   |
+    +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------------+
+    |                                                       | ``--exclude-from EXCLUDEFILE``        | read exclude patterns from EXCLUDEFILE, one per line                             |
+    +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------------+
+    |                                                       | ``--pattern PATTERN``                 | include/exclude paths matching PATTERN                                           |
+    +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------------+
+    |                                                       | ``--patterns-from PATTERNFILE``       | read include/exclude patterns from PATTERNFILE, one per line                     |
+    +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------------+
 
     .. raw:: html
 
@@ -68,13 +72,15 @@ borg diff
         --numeric-ids    only consider numeric user and group identifiers
         --same-chunker-params    Override check of chunker parameters.
         --sort     Sort the output lines by file path.
+        --format FORMAT    specify format for differences between archives (default: "{change} {path}{NL}")
         --json-lines    Format output as JSON Lines. 
+        --content-only    Only compare differences in content (exclude metadata differences)
 
 
     :ref:`common_options`
         |
 
-    Exclusion options
+    Include/Exclude options
         -e PATTERN, --exclude PATTERN     exclude paths matching PATTERN
         --exclude-from EXCLUDEFILE        read exclude patterns from EXCLUDEFILE, one per line
         --pattern PATTERN                 include/exclude paths matching PATTERN
@@ -84,19 +90,61 @@ borg diff
 Description
 ~~~~~~~~~~~
 
-This command finds differences (file contents, user/group/mode) between archives.
+This command finds differences (file contents, metadata) between ARCHIVE1 and ARCHIVE2.
 
-A repository location and an archive name must be specified for REPO::ARCHIVE1.
-ARCHIVE2 is just another archive name in same repository (no repository location
-allowed).
+For more help on include/exclude patterns, see the :ref:`borg_patterns` command output.
 
-For archives created with Borg 1.1 or newer diff automatically detects whether
-the archives are created with the same chunker params. If so, only chunk IDs
-are compared, which is very fast.
+.. man NOTES
 
-For archives prior to Borg 1.1 chunk contents are compared by default.
-If you did not create the archives with different chunker params,
-pass ``--same-chunker-params``.
-Note that the chunker params changed from Borg 0.xx to 1.0.
+The FORMAT specifier syntax
++++++++++++++++++++++++++++
 
-For more help on include/exclude patterns, see the :ref:`borg_patterns` command output.
+The ``--format`` option uses python's `format string syntax
+<https://docs.python.org/3.9/library/string.html#formatstrings>`_.
+
+Examples:
+::
+
+    $ borg diff --format '{content:30} {path}{NL}' ArchiveFoo ArchiveBar
+    modified:  +4.1 kB  -1.0 kB    file-diff
+    ...
+
+    # {VAR:<NUMBER} - pad to NUMBER columns left-aligned.
+    # {VAR:>NUMBER} - pad to NUMBER columns right-aligned.
+    $ borg diff --format '{content:>30} {path}{NL}' ArchiveFoo ArchiveBar
+       modified:  +4.1 kB  -1.0 kB file-diff
+    ...
+
+The following keys are always available:
+
+- NEWLINE: OS dependent line separator
+- NL: alias of NEWLINE
+- NUL: NUL character for creating print0 / xargs -0 like output
+- SPACE: space character
+- TAB: tab character
+- CR: carriage return character
+- LF: line feed character
+
+
+Keys available only when showing differences between archives:
+
+- path: archived file path
+- change: all available changes
+
+- content: file content change
+- mode: file mode change
+- type: file type change
+- owner: file owner (user/group) change
+- group: file group change
+- user: file user change
+
+- link: file link change
+- directory: file directory change
+- blkdev: file block device change
+- chrdev: file character device change
+- fifo: file fifo change
+
+- mtime: file modification time change
+- ctime: file change time change
+- isomtime: file modification time change (ISO 8601)
+- isoctime: file creation time change (ISO 8601)

docs/usage/export-tar.rst.inc

@@ -33,7 +33,7 @@ borg export-tar
     |                                                                                                                                                                                                           |
     | :ref:`common_options`                                                                                                                                                                                     |
     +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------------+
-    | **Exclusion options**                                                                                                                                                                                     |
+    | **Include/Exclude options**                                                                                                                                                                               |
     +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------------+
     |                                                       | ``-e PATTERN``, ``--exclude PATTERN`` | exclude paths matching PATTERN                                                                            |
     +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------------+
@@ -73,7 +73,7 @@ borg export-tar
     :ref:`common_options`
         |
 
-    Exclusion options
+    Include/Exclude options
         -e PATTERN, --exclude PATTERN     exclude paths matching PATTERN
         --exclude-from EXCLUDEFILE        read exclude patterns from EXCLUDEFILE, one per line
         --pattern PATTERN                 include/exclude paths matching PATTERN

docs/usage/extract.rst.inc

@@ -37,11 +37,13 @@ borg extract
     +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------------+
     |                                                       | ``--sparse``                          | create holes in output sparse file from all-zero chunks                                                   |
     +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------------+
+    |                                                       | ``--continue``                        | continue a previously interrupted extraction of same archive                                              |
+    +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------------+
     | .. class:: borg-common-opt-ref                                                                                                                                                                            |
     |                                                                                                                                                                                                           |
     | :ref:`common_options`                                                                                                                                                                                     |
     +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------------+
-    | **Exclusion options**                                                                                                                                                                                     |
+    | **Include/Exclude options**                                                                                                                                                                               |
     +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------------+
     |                                                       | ``-e PATTERN``, ``--exclude PATTERN`` | exclude paths matching PATTERN                                                                            |
     +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------------+
@@ -79,12 +81,13 @@ borg extract
         --noxattrs        do not extract/set xattrs
         --stdout          write all extracted data to stdout
         --sparse          create holes in output sparse file from all-zero chunks
+        --continue        continue a previously interrupted extraction of same archive
 
 
     :ref:`common_options`
         |
 
-    Exclusion options
+    Include/Exclude options
         -e PATTERN, --exclude PATTERN     exclude paths matching PATTERN
         --exclude-from EXCLUDEFILE        read exclude patterns from EXCLUDEFILE, one per line
         --pattern PATTERN                 include/exclude paths matching PATTERN

docs/usage/help.rst.inc

@@ -24,15 +24,13 @@ confuse the root with the ``PATH`` argument of e.g. `borg extract`.
 
 Starting with Borg 1.2, paths that are matched against patterns always
 appear relative. If you give ``/absolute/`` as root, the paths going
-into the matcher will look relative like ``absolute/.../file.ext``.
-If you give ``../some/path`` as root, the paths will look like
-``some/path/.../file.ext``.
+into the matcher will start with ``absolute/``.
+If you give ``../../relative`` as root, the paths will be normalized
+as ``relative/``.
 
-File patterns support five different styles. If followed by a colon ':',
-the first two characters of a pattern are used as a style selector.
-Explicit style selection is necessary if a non-default style is desired
-or when the desired pattern starts with two alphanumeric characters
-followed by a colon (i.e. ``aa:something/*``).
+Borg supports different pattern styles. To define a non-default
+style for a specific pattern, prefix it with two characters followed
+by a colon ':' (i.e. ``fm:path/*``, ``sh:path/**``).
 
 `Fnmatch <https://docs.python.org/3/library/fnmatch.html>`_, selector ``fm:``
     This is the default style for ``--exclude`` and ``--exclude-from``.
@@ -55,7 +53,8 @@ Shell-style patterns, selector ``sh:``
     Like fnmatch patterns these are similar to shell patterns. The difference
     is that the pattern may include ``**/`` for matching zero or more directory
     levels, ``*`` for matching zero or more arbitrary characters with the
-    exception of any path separator. A leading path separator is always removed.
+    exception of any path separator, ``{}`` containing comma-separated
+    alternative patterns. A leading path separator is always removed.
 
 `Regular expressions <https://docs.python.org/3/library/re.html>`_, selector ``re:``
     Unlike shell patterns, regular expressions are not required to match the full
@@ -175,6 +174,25 @@ before an exclude pattern, the file is backed up. Note that a no-recurse
 exclude stops examination of subdirectories so that potential includes
 will not match - use normal excludes for such use cases.
 
+Example::
+
+    # Define the recursion root
+    R /
+    # Exclude all iso files in any directory
+    - **/*.iso
+    # Explicitly include all inside etc and root
+    + etc/**
+    + root/**
+    # Exclude a specific directory under each user's home directories
+    - home/*/.cache
+    # Explicitly include everything in /home
+    + home/**
+    # Explicitly exclude some directories without recursing into them
+    ! re:^(dev|proc|run|sys|tmp)
+    # Exclude all other files and directories
+    # that are not specifically included earlier.
+    - **
+
 **Tip: You can easily test your patterns with --dry-run and  --list**::
 
     $ borg create --dry-run --list --patterns-from patterns.txt archive
@@ -393,29 +411,44 @@ auto,C[,L]
 
 obfuscate,SPEC,C[,L]
     Use compressed-size obfuscation to make fingerprinting attacks based on
-    the observable stored chunk size more difficult.
-    Note:
-    - you must combine this with encryption or it won't make any sense.
-    - your repo size will be bigger, of course.
+    the observable stored chunk size more difficult. Note:
 
-    The SPEC value will determine how the size obfuscation will work:
+    - You must combine this with encryption, or it won't make any sense.
+    - Your repo size will be bigger, of course.
+    - A chunk is limited by the constant ``MAX_DATA_SIZE`` (cur. ~20MiB).
+
+    The SPEC value determines how the size obfuscation works:
+
+    *Relative random reciprocal size variation* (multiplicative)
 
-    Relative random reciprocal size variation:
     Size will increase by a factor, relative to the compressed data size.
-    Smaller factors are often used, larger factors rarely.
-    1: factor 0.01 .. 100.0
-    2: factor 0.1 .. 1000.0
-    3: factor 1.0 .. 10000.0
-    4: factor 10.0 .. 100000.0
-    5: factor 100.0 .. 1000000.0
-    6: factor 1000.0 .. 10000000.0
-
-    Add a randomly sized padding up to the given size:
-    110: 1kiB
-    ...
-    120: 1MiB
-    ...
-    123: 8MiB (max.)
+    Smaller factors are used often, larger factors rarely.
+
+    Available factors::
+
+      1:     0.01 ..        100
+      2:     0.1  ..      1,000
+      3:     1    ..     10,000
+      4:    10    ..    100,000
+      5:   100    ..  1,000,000
+      6: 1,000    .. 10,000,000
+
+    Example probabilities for SPEC ``1``::
+
+      90   %  0.01 ..   0.1
+       9   %  0.1  ..   1
+       0.9 %  1    ..  10
+       0.09% 10    .. 100
+
+    *Randomly sized padding up to the given size* (additive)
+
+    ::
+
+      110: 1kiB (2 ^ (SPEC - 100))
+      ...
+      120: 1MiB
+      ...
+      123: 8MiB (max.)
 
 Examples::
 
@@ -426,7 +459,7 @@ Examples::
     borg create --compression zlib,1 REPO::ARCHIVE data
     borg create --compression auto,lzma,6 REPO::ARCHIVE data
     borg create --compression auto,lzma ...
-    borg create --compression obfuscate,3,none ...
+    borg create --compression obfuscate,110,none ...
     borg create --compression obfuscate,3,auto,zstd,10 ...
     borg create --compression obfuscate,2,zstd,6 ...
 

docs/usage/import-tar.rst.inc

@@ -31,6 +31,8 @@ borg import-tar
     +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
     |                                                       | ``--json``                                        | output stats as JSON (implies --stats)                                                                                                                                                            |
     +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                       | ``--ignore-zeros``                                | ignore zero-filled blocks in the input tarball                                                                                                                                                    |
+    +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
     | .. class:: borg-common-opt-ref                                                                                                                                                                                                                                                                                |
     |                                                                                                                                                                                                                                                                                                               |
     | :ref:`common_options`                                                                                                                                                                                                                                                                                         |
@@ -72,6 +74,7 @@ borg import-tar
         --list          output verbose list of items (files, dirs, ...)
         --filter STATUSCHARS    only display items with the given status characters
         --json          output stats as JSON (implies --stats)
+        --ignore-zeros    ignore zero-filled blocks in the input tarball
 
 
     :ref:`common_options`
@@ -121,3 +124,7 @@ import-tar reads these tar formats:
 - POSIX.1-1988 (ustar)
 - UNIX V7 tar
 - SunOS tar with extended attributes
+
+To import multiple tarballs into a single archive, they can be simply
+concatenated (e.g. using "cat") into a single file, and imported with an
+``--ignore-zeros`` option to skip through the stop markers between them.

docs/usage/list.rst.inc

@@ -31,7 +31,7 @@ borg list
     |                                                                                                                                                                                                                                                                                       |
     | :ref:`common_options`                                                                                                                                                                                                                                                                 |
     +-------------------------------------------------------+---------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
-    | **Exclusion options**                                                                                                                                                                                                                                                                 |
+    | **Include/Exclude options**                                                                                                                                                                                                                                                           |
     +-------------------------------------------------------+---------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
     |                                                       | ``-e PATTERN``, ``--exclude PATTERN`` | exclude paths matching PATTERN                                                                                                                                                        |
     +-------------------------------------------------------+---------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
@@ -67,7 +67,7 @@ borg list
     :ref:`common_options`
         |
 
-    Exclusion options
+    Include/Exclude options
         -e PATTERN, --exclude PATTERN     exclude paths matching PATTERN
         --exclude-from EXCLUDEFILE        read exclude patterns from EXCLUDEFILE, one per line
         --pattern PATTERN                 include/exclude paths matching PATTERN
@@ -107,35 +107,36 @@ The following keys are always available:
 - NEWLINE: OS dependent line separator
 - NL: alias of NEWLINE
 - NUL: NUL character for creating print0 / xargs -0 like output
-- SPACE
-- TAB
-- CR
-- LF
+- SPACE: space character
+- TAB: tab character
+- CR: carriage return character
+- LF: line feed character
+
 
 Keys available only when listing files in an archive:
 
-- type
-- mode
-- uid
-- gid
-- user
-- group
+- type: file type (file, dir, symlink, ...)
+- mode: file mode (as in stat)
+- uid: user id of file owner
+- gid: group id of file owner
+- user: user name of file owner
+- group: group name of file owner
 - path: file path
 - target: link target for symlinks
 - hlid: hard link identity (same if hardlinking same fs object)
-- flags
+- flags: file flags
 
-- size
+- size: file size
 - dsize: deduplicated size
 - num_chunks: number of chunks in this file
 - unique_chunks: number of unique chunks in this file
 
-- mtime
-- ctime
-- atime
-- isomtime
-- isoctime
-- isoatime
+- mtime: file modification time
+- ctime: file change time
+- atime: file access time
+- isomtime: file modification time (ISO 8601 format)
+- isoctime: file change time (ISO 8601 format)
+- isoatime: file access time (ISO 8601 format)
 
 - blake2b
 - blake2s
@@ -151,8 +152,8 @@ Keys available only when listing files in an archive:
 - sha512
 - xxh64: XXH64 checksum of this file (note: this is NOT a cryptographic hash!)
 
-- archiveid
-- archivename
+- archiveid: internal ID of the archive
+- archivename: name of the archive
 - extra: prepends {target} with " -> " for soft links and " link to " for hard links
 
 - health: either "healthy" (file ok) or "broken" (if file has all-zero replacement chunks)

docs/usage/mount.rst.inc

@@ -51,7 +51,7 @@ borg mount
     +-----------------------------------------------------------------------------+----------------------------------------------+-----------------------------------------------------------------------------------------------------------+
     |                                                                             | ``--newer TIMESPAN``                         | consider archives newer than (now - TIMESPAN), e.g. 7d or 12m.                                            |
     +-----------------------------------------------------------------------------+----------------------------------------------+-----------------------------------------------------------------------------------------------------------+
-    | **Exclusion options**                                                                                                                                                                                                                  |
+    | **Include/Exclude options**                                                                                                                                                                                                            |
     +-----------------------------------------------------------------------------+----------------------------------------------+-----------------------------------------------------------------------------------------------------------+
     |                                                                             | ``-e PATTERN``, ``--exclude PATTERN``        | exclude paths matching PATTERN                                                                            |
     +-----------------------------------------------------------------------------+----------------------------------------------+-----------------------------------------------------------------------------------------------------------+
@@ -101,7 +101,7 @@ borg mount
         --newer TIMESPAN                         consider archives newer than (now - TIMESPAN), e.g. 7d or 12m.
 
 
-    Exclusion options
+    Include/Exclude options
         -e PATTERN, --exclude PATTERN     exclude paths matching PATTERN
         --exclude-from EXCLUDEFILE        read exclude patterns from EXCLUDEFILE, one per line
         --pattern PATTERN                 include/exclude paths matching PATTERN
@@ -148,14 +148,14 @@ manually. Unlike the ``uid`` and ``gid`` mount options which affect all files,
 
 Additional mount options supported by borg:
 
-- versions: when used with a repository mount, this gives a merged, versioned
+- ``versions``: when used with a repository mount, this gives a merged, versioned
   view of the files in the archives. EXPERIMENTAL, layout may change in future.
-- allow_damaged_files: by default damaged files (where missing chunks were
-  replaced with runs of zeros by borg check ``--repair``) are not readable and
+- ``allow_damaged_files``: by default damaged files (where missing chunks were
+  replaced with runs of zeros by ``borg check --repair``) are not readable and
   return EIO (I/O error). Set this option to read such files.
-- ignore_permissions: for security reasons the "default_permissions" mount
-  option is internally enforced by borg. "ignore_permissions" can be given to
-  not enforce "default_permissions".
+- ``ignore_permissions``: for security reasons the ``default_permissions`` mount
+  option is internally enforced by borg. ``ignore_permissions`` can be given to
+  not enforce ``default_permissions``.
 
 The BORG_MOUNT_DATA_CACHE_ENTRIES environment variable is meant for advanced users
 to tweak the performance. It sets the number of cached data chunks; additional

docs/usage/prune.rst.inc

@@ -23,6 +23,14 @@ borg prune
     +-----------------------------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------+
     |                                                                             | ``--list``                                        | output verbose list of archives it keeps/prunes                                                   |
     +-----------------------------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------+
+    |                                                                             | ``--short``                                       | use a less wide archive part format                                                               |
+    +-----------------------------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------+
+    |                                                                             | ``--list-pruned``                                 | output verbose list of archives it prunes                                                         |
+    +-----------------------------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------+
+    |                                                                             | ``--list-kept``                                   | output verbose list of archives it keeps                                                          |
+    +-----------------------------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------+
+    |                                                                             | ``--format FORMAT``                               | specify format for the archive part (default: "{archive:<36} {time} [{id}]")                      |
+    +-----------------------------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------+
     |                                                                             | ``--keep-within INTERVAL``                        | keep all archives within this time interval                                                       |
     +-----------------------------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------+
     |                                                                             | ``--keep-last``, ``--keep-secondly``              | number of secondly archives to keep                                                               |
@@ -75,6 +83,10 @@ borg prune
         --force               force pruning of corrupted archives, use ``--force --force`` in case ``--force`` does not work.
         -s, --stats           print statistics for the deleted archive
         --list                output verbose list of archives it keeps/prunes
+        --short               use a less wide archive part format
+        --list-pruned         output verbose list of archives it prunes
+        --list-kept           output verbose list of archives it keeps
+        --format FORMAT       specify format for the archive part (default: "{archive:<36} {time} [{id}]")
         --keep-within INTERVAL    keep all archives within this time interval
         --keep-last, --keep-secondly    number of secondly archives to keep
         --keep-minutely       number of minutely archives to keep
@@ -153,4 +165,8 @@ backup archive in the same second).
 When using ``--stats``, you will get some statistics about how much data was
 deleted - the "Deleted data" deduplicated size there is most interesting as
 that is how much your repository will shrink.
-Please note that the "All archives" stats refer to the state after pruning.
+Please note that the "All archives" stats refer to the state after pruning.
+
+You can influence how the ``--list`` output is formatted by using the ``--short``
+option (less wide output) or by giving a custom format using ``--format`` (see
+the ``borg rlist`` description for more details about the format string).

docs/usage/recreate.rst.inc

@@ -31,7 +31,7 @@ borg recreate
     |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
     | :ref:`common_options`                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
     +-----------------------------------------------------------------------------+---------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
-    | **Exclusion options**                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
+    | **Include/Exclude options**                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
     +-----------------------------------------------------------------------------+---------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
     |                                                                             | ``-e PATTERN``, ``--exclude PATTERN``             | exclude paths matching PATTERN                                                                                                                                                                                                                                                                                                                                                                                                                     |
     +-----------------------------------------------------------------------------+---------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
@@ -106,7 +106,7 @@ borg recreate
     :ref:`common_options`
         |
 
-    Exclusion options
+    Include/Exclude options
         -e PATTERN, --exclude PATTERN     exclude paths matching PATTERN
         --exclude-from EXCLUDEFILE        read exclude patterns from EXCLUDEFILE, one per line
         --pattern PATTERN                 include/exclude paths matching PATTERN

docs/usage/rlist.rst.inc

@@ -112,10 +112,11 @@ The following keys are always available:
 - NEWLINE: OS dependent line separator
 - NL: alias of NEWLINE
 - NUL: NUL character for creating print0 / xargs -0 like output
-- SPACE
-- TAB
-- CR
-- LF
+- SPACE: space character
+- TAB: tab character
+- CR: carriage return character
+- LF: line feed character
+
 
 Keys available only when listing archives in a repository:
 
@@ -131,3 +132,6 @@ Keys available only when listing archives in a repository:
 
 - hostname: hostname of host on which this archive was created
 - username: username of user who created this archive
+
+- size: size of this archive (data plus metadata, not considering compression and deduplication)
+- nfiles: count of files in this archive

docs/usage/serve.rst.inc

@@ -53,4 +53,14 @@ borg serve
 Description
 ~~~~~~~~~~~
 
-This command starts a repository server process. This command is usually not used manually.
+This command starts a repository server process.
+
+borg serve can currently support:
+
+- Getting automatically started via ssh when the borg client uses a ssh://...
+  remote repository. In this mode, `borg serve` will live until that ssh connection
+  gets terminated.
+
+- Getting started by some other means (not by the borg client) as a long-running socket
+  server to be used for borg clients using a socket://... repository (see the `--socket`
+  option if you do not want to use the default path for the socket and pid file).

docs/usage/transfer.rst.inc

@@ -12,37 +12,41 @@ borg transfer
 
     .. class:: borg-options-table
 
-    +-----------------------------------------------------------------------------+----------------------------------------------+---------------------------------------------------------------------------------------------------+
-    | **options**                                                                                                                                                                                                                    |
-    +-----------------------------------------------------------------------------+----------------------------------------------+---------------------------------------------------------------------------------------------------+
-    |                                                                             | ``-n``, ``--dry-run``                        | do not change repository, just check                                                              |
-    +-----------------------------------------------------------------------------+----------------------------------------------+---------------------------------------------------------------------------------------------------+
-    |                                                                             | ``--other-repo SRC_REPOSITORY``              | transfer archives from the other repository                                                       |
-    +-----------------------------------------------------------------------------+----------------------------------------------+---------------------------------------------------------------------------------------------------+
-    |                                                                             | ``--upgrader UPGRADER``                      | use the upgrader to convert transferred data (default: no conversion)                             |
-    +-----------------------------------------------------------------------------+----------------------------------------------+---------------------------------------------------------------------------------------------------+
-    | .. class:: borg-common-opt-ref                                                                                                                                                                                                 |
-    |                                                                                                                                                                                                                                |
-    | :ref:`common_options`                                                                                                                                                                                                          |
-    +-----------------------------------------------------------------------------+----------------------------------------------+---------------------------------------------------------------------------------------------------+
-    | **Archive filters** — Archive filters can be applied to repository targets.                                                                                                                                                    |
-    +-----------------------------------------------------------------------------+----------------------------------------------+---------------------------------------------------------------------------------------------------+
-    |                                                                             | ``-a PATTERN``, ``--match-archives PATTERN`` | only consider archive names matching the pattern. see "borg help match-archives".                 |
-    +-----------------------------------------------------------------------------+----------------------------------------------+---------------------------------------------------------------------------------------------------+
-    |                                                                             | ``--sort-by KEYS``                           | Comma-separated list of sorting keys; valid keys are: timestamp, name, id; default is: timestamp  |
-    +-----------------------------------------------------------------------------+----------------------------------------------+---------------------------------------------------------------------------------------------------+
-    |                                                                             | ``--first N``                                | consider first N archives after other filters were applied                                        |
-    +-----------------------------------------------------------------------------+----------------------------------------------+---------------------------------------------------------------------------------------------------+
-    |                                                                             | ``--last N``                                 | consider last N archives after other filters were applied                                         |
-    +-----------------------------------------------------------------------------+----------------------------------------------+---------------------------------------------------------------------------------------------------+
-    |                                                                             | ``--oldest TIMESPAN``                        | consider archives between the oldest archive's timestamp and (oldest + TIMESPAN), e.g. 7d or 12m. |
-    +-----------------------------------------------------------------------------+----------------------------------------------+---------------------------------------------------------------------------------------------------+
-    |                                                                             | ``--newest TIMESPAN``                        | consider archives between the newest archive's timestamp and (newest - TIMESPAN), e.g. 7d or 12m. |
-    +-----------------------------------------------------------------------------+----------------------------------------------+---------------------------------------------------------------------------------------------------+
-    |                                                                             | ``--older TIMESPAN``                         | consider archives older than (now - TIMESPAN), e.g. 7d oder 12m.                                  |
-    +-----------------------------------------------------------------------------+----------------------------------------------+---------------------------------------------------------------------------------------------------+
-    |                                                                             | ``--newer TIMESPAN``                         | consider archives newer than (now - TIMESPAN), e.g. 7d or 12m.                                    |
-    +-----------------------------------------------------------------------------+----------------------------------------------+---------------------------------------------------------------------------------------------------+
+    +-----------------------------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+    | **options**                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
+    +-----------------------------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                                             | ``-n``, ``--dry-run``                             | do not change repository, just check                                                                                                                                                                                                                                                                                      |
+    +-----------------------------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                                             | ``--other-repo SRC_REPOSITORY``                   | transfer archives from the other repository                                                                                                                                                                                                                                                                               |
+    +-----------------------------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                                             | ``--upgrader UPGRADER``                           | use the upgrader to convert transferred data (default: no conversion)                                                                                                                                                                                                                                                     |
+    +-----------------------------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                                             | ``-C COMPRESSION``, ``--compression COMPRESSION`` | select compression algorithm, see the output of the "borg help compression" command for details.                                                                                                                                                                                                                          |
+    +-----------------------------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                                             | ``--recompress MODE``                             | recompress data chunks according to `MODE` and ``--compression``. Possible modes are `always`: recompress unconditionally; and `never`: do not recompress (faster: re-uses compressed data chunks w/o change).If no MODE is given, `always` will be used. Not passing --recompress is equivalent to "--recompress never". |
+    +-----------------------------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+    | .. class:: borg-common-opt-ref                                                                                                                                                                                                                                                                                                                                                                                                                              |
+    |                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
+    | :ref:`common_options`                                                                                                                                                                                                                                                                                                                                                                                                                                       |
+    +-----------------------------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+    | **Archive filters** — Archive filters can be applied to repository targets.                                                                                                                                                                                                                                                                                                                                                                                 |
+    +-----------------------------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                                             | ``-a PATTERN``, ``--match-archives PATTERN``      | only consider archive names matching the pattern. see "borg help match-archives".                                                                                                                                                                                                                                         |
+    +-----------------------------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                                             | ``--sort-by KEYS``                                | Comma-separated list of sorting keys; valid keys are: timestamp, name, id; default is: timestamp                                                                                                                                                                                                                          |
+    +-----------------------------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                                             | ``--first N``                                     | consider first N archives after other filters were applied                                                                                                                                                                                                                                                                |
+    +-----------------------------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                                             | ``--last N``                                      | consider last N archives after other filters were applied                                                                                                                                                                                                                                                                 |
+    +-----------------------------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                                             | ``--oldest TIMESPAN``                             | consider archives between the oldest archive's timestamp and (oldest + TIMESPAN), e.g. 7d or 12m.                                                                                                                                                                                                                         |
+    +-----------------------------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                                             | ``--newest TIMESPAN``                             | consider archives between the newest archive's timestamp and (newest - TIMESPAN), e.g. 7d or 12m.                                                                                                                                                                                                                         |
+    +-----------------------------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                                             | ``--older TIMESPAN``                              | consider archives older than (now - TIMESPAN), e.g. 7d oder 12m.                                                                                                                                                                                                                                                          |
+    +-----------------------------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+    |                                                                             | ``--newer TIMESPAN``                              | consider archives newer than (now - TIMESPAN), e.g. 7d or 12m.                                                                                                                                                                                                                                                            |
+    +-----------------------------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 
     .. raw:: html
 
@@ -60,6 +64,8 @@ borg transfer
         -n, --dry-run     do not change repository, just check
         --other-repo SRC_REPOSITORY    transfer archives from the other repository
         --upgrader UPGRADER    use the upgrader to convert transferred data (default: no conversion)
+        -C COMPRESSION, --compression COMPRESSION    select compression algorithm, see the output of the "borg help compression" command for details.
+        --recompress MODE    recompress data chunks according to `MODE` and ``--compression``. Possible modes are `always`: recompress unconditionally; and `never`: do not recompress (faster: re-uses compressed data chunks w/o change).If no MODE is given, `always` will be used. Not passing --recompress is equivalent to "--recompress never".
 
 
     :ref:`common_options`
@@ -81,6 +87,14 @@ Description
 
 This command transfers archives from one repository to another repository.
 Optionally, it can also upgrade the transferred data.
+Optionally, it can also recompress the transferred data.
+
+It is easiest (and fastest) to give ``--compression=COMPRESSION --recompress=never`` using
+the same COMPRESSION mode as in the SRC_REPO - borg will use that COMPRESSION for metadata (in
+any case) and keep data compressed "as is" (saves time as no data compression is needed).
+
+If you want to globally change compression while transferring archives to the DST_REPO,
+give ``--compress=WANTED_COMPRESSION --recompress=always``.
 
 Suggested use for general purpose archive transfer (not repo upgrades)::
 
@@ -96,11 +110,17 @@ Suggested use for general purpose archive transfer (not repo upgrades)::
 The default is to transfer all archives, including checkpoint archives.
 
 You could use the misc. archive filter options to limit which archives it will
-transfer, e.g. using the -a option. This is recommended for big
+transfer, e.g. using the ``-a`` option. This is recommended for big
 repositories with multiple data sets to keep the runtime per invocation lower.
 
 For repository upgrades (e.g. from a borg 1.2 repo to a related borg 2.0 repo), usage is
 quite similar to the above::
 
-    borg --repo=DST_REPO transfer --other-repo=SRC_REPO --upgrader=From12To20
+    # fast: compress metadata with zstd,3, but keep data chunks compressed as they are:
+    borg --repo=DST_REPO transfer --other-repo=SRC_REPO --upgrader=From12To20 \
+         --compress=zstd,3 --recompress=never
+
+    # compress metadata and recompress data with zstd,3
+    borg --repo=DST_REPO transfer --other-repo=SRC_REPO --upgrader=From12To20 \
+         --compress=zstd,3 --recompress=always