updated 1.0.x changes up to last one, dissect 1.0.x vs 1.1.x

docs/changes.rst

@@ -2607,36 +2607,6 @@ Other changes:
   - added FAQ section about backing up root partition
 
 
-Version 1.0.10 (2017-02-13)
----------------------------
-
-Bug fixes:
-
-- Manifest timestamps are now monotonically increasing,
-  this fixes issues when the system clock jumps backwards
-  or is set inconsistently across computers accessing the same repository, #2115
-- Fixed testing regression in 1.0.10rc1 that lead to a hard dependency on
-  py.test >= 3.0, #2112
-
-New features:
-
-- "key export" can now generate a printable HTML page with both a QR code and
-  a human-readable "paperkey" representation (and custom text) through the
-  ``--qr-html`` option.
-
-  The same functionality is also available through `paperkey.html <paperkey.html>`_,
-  which is the same HTML page generated by ``--qr-html``. It works with existing
-  "key export" files and key files.
-
-Other changes:
-
-- docs:
-
-  - language clarification - "borg create --one-file-system" option does not respect
-    mount points, but considers different file systems instead, #2141
-- setup.py: build_api: sort file list for determinism
-
-
 Version 1.1.0b3 (2017-01-15)
 ----------------------------
 
@@ -2718,6 +2688,438 @@ Other changes:
   - remove all BORG_* env vars from the outer environment
 
 
+Version 1.1.0b2 (2016-10-01)
+----------------------------
+
+Bug fixes:
+
+- fix incorrect preservation of delete tags, leading to "object count mismatch"
+  on borg check, #1598. This only occurred with 1.1.0b1 (not with 1.0.x) and is
+  normally fixed by running another borg create/delete/prune.
+- fix broken --progress for double-cell paths (e.g. CJK), #1624
+- borg recreate: also catch SIGHUP
+- FUSE:
+
+  - fix hardlinks in versions view, #1599
+  - add parameter check to ItemCache.get to make potential failures more clear
+
+New features:
+
+- Archiver, RemoteRepository: add --remote-ratelimit (send data)
+- borg help compression, #1582
+- borg check: delete chunks with integrity errors, #1575, so they can be
+  "repaired" immediately and maybe healed later.
+- archives filters concept (refactoring/unifying older code)
+
+  - covers --first/--last/--prefix/--sort-by options
+  - currently used for borg list/info/delete
+
+Other changes:
+
+- borg check --verify-data slightly tuned (use get_many())
+- change {utcnow} and {now} to ISO-8601 format ("T" date/time separator)
+- repo check: log transaction IDs, improve object count mismatch diagnostic
+- Vagrantfile: use TW's fresh-bootloader pyinstaller branch
+- fix module names in api.rst
+- hashindex: bump api_version
+
+
+Version 1.1.0b1 (2016-08-28)
+----------------------------
+
+New features:
+
+- new commands:
+
+  - borg recreate: re-create existing archives, #787 #686 #630 #70, also see
+    #757, #770.
+
+    - selectively remove files/dirs from old archives
+    - re-compress data
+    - re-chunkify data, e.g. to have upgraded Attic / Borg 0.xx archives
+      deduplicate with Borg 1.x archives or to experiment with chunker-params.
+  - borg diff: show differences between archives
+  - borg with-lock: execute a command with the repository locked, #990
+- borg create:
+
+  - Flexible compression with pattern matching on path/filename,
+    and LZ4 heuristic for deciding compressibility, #810, #1007
+  - visit files in inode order (better speed, esp. for large directories and rotating disks)
+  - in-file checkpoints, #1217
+  - increased default checkpoint interval to 30 minutes (was 5 minutes), #896
+  - added uuid archive format tag, #1151
+  - save mountpoint directories with --one-file-system, makes system restore easier, #1033
+  - Linux: added support for some BSD flags, #1050
+  - add 'x' status for excluded paths, #814
+
+    - also means files excluded via UF_NODUMP, #1080
+- borg check:
+
+  - will not produce the "Checking segments" output unless new --progress option is passed, #824.
+  - --verify-data to verify data cryptographically on the client, #975
+- borg list, #751, #1179
+
+  - removed {formatkeys}, see "borg list --help"
+  - --list-format is deprecated, use --format instead
+  - --format now also applies to listing archives, not only archive contents, #1179
+  - now supports the usual [PATH [PATHS…]] syntax and excludes
+  - new keys: csize, num_chunks, unique_chunks, NUL
+  - supports guaranteed_available hashlib hashes
+    (to avoid varying functionality depending on environment),
+    which includes the SHA1 and SHA2 family as well as MD5
+- borg prune:
+
+  - to better visualize the "thinning out", we now list all archives in
+    reverse time order. rephrase and reorder help text.
+  - implement --keep-last N via --keep-secondly N, also --keep-minutely.
+    assuming that there is not more than 1 backup archive made in 1s,
+    --keep-last N and --keep-secondly N are equivalent, #537
+  - cleanup checkpoints except the latest, #1008
+- borg extract:
+
+  - added --progress, #1449
+  - Linux: limited support for BSD flags, #1050
+- borg info:
+
+  - output is now more similar to borg create --stats, #977
+- borg mount:
+
+  - provide "borgfs" wrapper for borg mount, enables usage via fstab, #743
+  - "versions" mount option - when used with a repository mount, this gives
+    a merged, versioned view of the files in all archives, #729
+- repository:
+
+  - added progress information to commit/compaction phase (often takes some time when deleting/pruning), #1519
+  - automatic recovery for some forms of repository inconsistency, #858
+  - check free space before going forward with a commit, #1336
+  - improved write performance (esp. for rotating media), #985
+
+    - new IO code for Linux
+    - raised default segment size to approx 512 MiB
+  - improved compaction performance, #1041
+  - reduced client CPU load and improved performance for remote repositories, #940
+
+- options that imply output (--show-rc, --show-version, --list, --stats,
+  --progress) don't need -v/--info to have that output displayed, #865
+- add archive comments (via borg (re)create --comment), #842
+- borg list/prune/delete: also output archive id, #731
+- --show-version: shows/logs the borg version, #725
+- added --debug-topic for granular debug logging, #1447
+- use atomic file writing/updating for configuration and key files, #1377
+- BORG_KEY_FILE environment variable, #1001
+- self-testing module, #970
+
+
+Bug fixes:
+
+- list: fixed default output being produced if --format is given with empty parameter, #1489
+- create: fixed overflowing progress line with CJK and similar characters, #1051
+- prune: fixed crash if --prefix resulted in no matches, #1029
+- init: clean up partial repo if passphrase input is aborted, #850
+- info: quote cmdline arguments that have spaces in them
+- fix hardlinks failing in some cases for extracting subtrees, #761
+
+Other changes:
+
+- replace stdlib hmac with OpenSSL, zero-copy decrypt (10-15% increase in
+  performance of hash-lists and extract).
+- improved chunker performance, #1021
+- open repository segment files in exclusive mode (fail-safe), #1134
+- improved error logging, #1440
+- Source:
+
+  - pass meta-data around, #765
+  - move some constants to new constants module
+  - better readability and fewer errors with namedtuples, #823
+  - moved source tree into src/ subdirectory, #1016
+  - made borg.platform a package, #1113
+  - removed dead crypto code, #1032
+  - improved and ported parts of the test suite to py.test, #912
+  - created data classes instead of passing dictionaries around, #981, #1158, #1161
+  - cleaned up imports, #1112
+- Docs:
+
+  - better help texts and sphinx reproduction of usage help:
+
+    - Group options
+    - Nicer list of options in Sphinx
+    - Deduplicate 'Common options' (including --help)
+  - chunker: added some insights by "Voltara", #903
+  - clarify what "deduplicated size" means
+  - fix / update / add package list entries
+  - added a SaltStack usage example, #956
+  - expanded FAQ
+  - new contributors in AUTHORS!
+- Tests:
+
+  - vagrant: add ubuntu/xenial 64bit - this box has still some issues
+  - ChunkBuffer: add test for leaving partial chunk in buffer, fixes #945
+
+
+Version 1.0.13 (2019-02-15)
+---------------------------
+
+Please note: this is very likely the last 1.0.x release, please upgrade to 1.1.x.
+
+Bug fixes:
+
+- security fix: configure FUSE with "default_permissions", #3903.
+  "default_permissions" is now enforced by borg by default to let the
+  kernel check uid/gid/mode based permissions.
+  "ignore_permissions" can be given to not enforce "default_permissions".
+- xattrs: fix borg exception handling on ENOSPC error, #3808.
+
+New features:
+
+- Read a passphrase from a file descriptor specified in the
+  BORG_PASSPHRASE_FD environment variable.
+
+Other changes:
+
+- acl platform code: fix acl set return type
+- xattr:
+
+  - add linux {list,get,set}xattr ctypes prototypes
+  - fix darwin flistxattr ctypes prototype
+- testing / travis-ci:
+
+  - fix the homebrew 1.9 issues on travis-ci, #4254
+  - travis OS X: use xcode 8.3 (not broken)
+  - tox.ini: lock requirements
+  - unbreak 1.0-maint on travis, fixes #4123
+- vagrant:
+
+  - misc. fixes
+  - FUSE for macOS: upgrade 3.7.1 to 3.8.3
+  - Python: upgrade 3.5.5 to 3.5.6
+- docs:
+
+  - Update installation instructions for macOS
+  - update release workflow using twine (docs, scripts), #4213
+
+Version 1.0.12 (2018-04-08)
+---------------------------
+
+Bug fixes:
+
+- repository: cleanup/write: invalidate cached FDs, tests
+- serve: fix exitcode, #2910
+- extract: set bsdflags last (include immutable flag), #3263
+- create --timestamp: set start time, #2957
+- create: show excluded dir with "x" for tagged dirs / caches, #3189
+- migrate locks to child PID when daemonize is used
+- Buffer: fix wrong thread-local storage use, #2951
+- fix detection of non-local path, #3108
+- fix LDLP restoration for subprocesses, #3077
+- fix subprocess environments (xattr module's fakeroot version check,
+  borg umount, BORG_PASSCOMMAND), #3050
+- remote: deal with partial lines, #2637
+- get rid of datetime.isoformat, use safe parse_timestamp to parse
+  timestamps, #2994
+- build: do .h file content checks in binary mode, fixes build issue for
+  non-ascii header files on pure-ascii locale platforms, #3544 #3639
+- remove platform.uname() call which caused library mismatch issues, #3732
+- add exception handler around deprecated platform.linux_distribution() call
+
+Other changes:
+
+- require msgpack-python >= 0.4.6 and < 0.5.0, see #3753
+- add parens for C preprocessor macro argument usages (did not cause
+  malfunction)
+- ignore corrupt files cache, #2939
+- replace "modulo" with "if" to check for wraparound in hashmap
+- keymanager: don't depend on optional readline module, #2980
+- exclude broken pytest 3.3.0 release
+- exclude broken Cython 0.27(.0) release, #3066
+- flake8: add some ignores
+- docs:
+
+  - create: document exclusion through nodump
+  - document good and problematic option placements, fix examples, #3356
+  - update docs about hardlinked symlinks limitation
+  - faq: we do not implement futile attempts of ETA / progress displays
+  - simplified rate limiting wrapper in FAQ
+  - twitter account @borgbackup, #2948
+  - add note about metadata dedup and --no[ac]time, #2518
+  - change-passphrase only changes the passphrase, #2990
+  - clarify encrypted key format for borg key export, #3296
+  - document sshfs rename workaround, #3315
+  - update release checklist about security fixes
+  - docs about how to verify a signed release, #3634
+  - chunk seed is generated per /repository/
+- vagrant:
+
+  - use FUSE for macOS 3.7.1 to build the macOS binary
+  - use python 3.5.5 to build the binaries
+  - add exe location to PATH when we build an exe
+  - use https pypi url for wheezy
+  - netbsd: bash is already installed
+  - netbsd: fix netbsd version in PKG_PATH
+  - use self-made FreeBSD 10.3 box, #3022
+  - backport fs_init (including related updates) from 1.1
+  - the boxcutter wheezy boxes are 404, use local ones
+- travis:
+
+  - don't perform full Travis build on docs-only changes, #2531
+  - only short-circuit docs-only changes for pull requests
+
+
+Version 1.0.11 (2017-07-21)
+---------------------------
+
+Bug fixes:
+
+- use limited unpacker for outer key (security precaution), #2174
+- fix paperkey import bug
+
+Other changes:
+
+- change --checkpoint-interval default from 600s to 1800s, #2841.
+  this improves efficiency for big repositories a lot.
+- docs: fix OpenSUSE command and add OpenSUSE section
+- tests: add tests for split_lstring and paperkey
+- vagrant:
+
+  - fix openbsd shell
+  - backport cpu/ram setup from master
+  - add stretch64 VM
+
+Version 1.0.11rc1 (2017-06-27)
+------------------------------
+
+Bug fixes:
+
+- performance: rebuild hashtable if we have too few empty buckets, #2246.
+  this fixes some sporadic, but severe performance breakdowns.
+- Archive: allocate zeros when needed, #2308
+  fixes huge memory usage of mount (8 MiB × number of archives)
+- IPv6 address support
+  also: Location: more informative exception when parsing fails
+- borg single-file binary: use pyinstaller v3.2.1, #2396
+  this fixes that the prelink cronjob on some distros kills the
+  borg binary by stripping away parts of it.
+- extract:
+
+  - warning for unextracted big extended attributes, #2258
+  - also create parent dir for device files, if needed.
+  - don't write to disk with --stdout, #2645
+- archive check: detect and fix missing all-zero replacement chunks, #2180
+- fix (de)compression exceptions, #2224 #2221
+- files cache: update inode number, #2226
+- borg rpc: use limited msgpack.Unpacker (security precaution), #2139
+- Manifest: use limited msgpack.Unpacker (security precaution), #2175
+- Location: accept //servername/share/path
+- fix ChunkIndex.__contains__ assertion  for big-endian archs (harmless)
+- create: handle BackupOSError on a per-path level in one spot
+- fix error msg, there is no --keep-last in borg 1.0.x, #2282
+- clamp (nano)second values to unproblematic range, #2304
+- fuse / borg mount:
+
+  - fix st_blocks to be an integer (not float) value
+  - fix negative uid/gid crash (they could come into archives e.g. when
+    backing up external drives under cygwin), #2674
+  - fix crash if empty (None) xattr is read
+  - do pre-mount checks before opening repository
+  - check llfuse is installed before asking for passphrase
+- borg rename: expand placeholders, #2386
+- borg serve: fix forced command lines containing BORG_ env vars
+- fix error msg, it is --keep-within, not --within
+- fix borg key/debug/benchmark crashing without subcommand, #2240
+- chunker: fix invalid use of types, don't do uint32_t >> 32
+- document follow_symlinks requirements, check libc, #2507
+
+New features:
+
+- added BORG_PASSCOMMAND environment variable, #2573
+- add minimal version of in repository mandatory feature flags, #2134
+
+  This should allow us to make sure older borg versions can be cleanly
+  prevented from doing operations that are no longer safe because of
+  repository format evolution. This allows more fine grained control than
+  just incrementing the manifest version. So for example a change that
+  still allows new archives to be created but would corrupt the repository
+  when an old version tries to delete an archive or check the repository
+  would add the new feature to the check and delete set but leave it out
+  of the write set.
+- borg delete --force --force to delete severely corrupted archives, #1975
+
+Other changes:
+
+- embrace y2038 issue to support 32bit platforms
+- be more clear that this is a "beyond repair" case, #2427
+- key file names: limit to 100 characters and remove colons from host name
+- upgrade FUSE for macOS to 3.5.8, #2346
+- split up parsing and filtering for --keep-within, better error message, #2610
+- docs:
+
+  - fix caskroom link, #2299
+  - address SSH batch mode, #2202 #2270
+  - improve remote-path description
+  - document snapshot usage, #2178
+  - document relative path usage, #1868
+  - one link per distro in the installation page
+  - development: new branching model in git repository
+  - kill api page
+  - added FAQ section about backing up root partition
+  - add bountysource badge, #2558
+  - create empty docs.txt reequirements, #2694
+  - README: how to help the project
+  - note -v/--verbose requirement on affected options, #2542
+  - document borg init behaviour via append-only borg serve, #2440
+  - be clear about what buzhash is used for (chunking) and want it is not
+    used for (deduplication)- also say already in the readme that we use a
+    cryptohash for dedupe, so people don't worry, #2390
+  - add hint about chunker params to borg upgrade docs, #2421
+  - clarify borg upgrade docs, #2436
+  - quickstart: delete problematic BORG_PASSPRHASE use, #2623
+  - faq: specify "using inline shell scripts"
+  - document pattern denial of service, #2624
+- tests:
+
+  - remove attic dependency of the tests, #2505
+  - travis:
+
+    - enhance travis setuptools_scm situation
+    - install fakeroot for Linux
+  - add test for borg delete --force
+  - enable remote tests on cygwin (the cygwin issue that caused these tests
+    to break was fixed in cygwin at least since cygwin 2.8, maybe even since
+    2.7.0).
+  - remove skipping the noatime tests on GNU/Hurd, #2710
+  - fix borg import issue, add comment, #2718
+  - include attic.tar.gz when installing the package
+    also: add include_package_data=True
+
+Version 1.0.10 (2017-02-13)
+---------------------------
+
+Bug fixes:
+
+- Manifest timestamps are now monotonically increasing,
+  this fixes issues when the system clock jumps backwards
+  or is set inconsistently across computers accessing the same repository, #2115
+- Fixed testing regression in 1.0.10rc1 that lead to a hard dependency on
+  py.test >= 3.0, #2112
+
+New features:
+
+- "key export" can now generate a printable HTML page with both a QR code and
+  a human-readable "paperkey" representation (and custom text) through the
+  ``--qr-html`` option.
+
+  The same functionality is also available through `paperkey.html <paperkey.html>`_,
+  which is the same HTML page generated by ``--qr-html``. It works with existing
+  "key export" files and key files.
+
+Other changes:
+
+- docs:
+
+  - language clarification - "borg create --one-file-system" option does not respect
+    mount points, but considers different file systems instead, #2141
+- setup.py: build_api: sort file list for determinism
+
+
 Version 1.0.10rc1 (2017-01-29)
 ------------------------------
 
@@ -2887,173 +3289,6 @@ Other changes:
   - add windows virtual machine with cygwin
   - Vagrantfile cleanup / code deduplication
 
-Version 1.1.0b2 (2016-10-01)
-----------------------------
-
-Bug fixes:
-
-- fix incorrect preservation of delete tags, leading to "object count mismatch"
-  on borg check, #1598. This only occurred with 1.1.0b1 (not with 1.0.x) and is
-  normally fixed by running another borg create/delete/prune.
-- fix broken --progress for double-cell paths (e.g. CJK), #1624
-- borg recreate: also catch SIGHUP
-- FUSE:
-
-  - fix hardlinks in versions view, #1599
-  - add parameter check to ItemCache.get to make potential failures more clear
-
-New features:
-
-- Archiver, RemoteRepository: add --remote-ratelimit (send data)
-- borg help compression, #1582
-- borg check: delete chunks with integrity errors, #1575, so they can be
-  "repaired" immediately and maybe healed later.
-- archives filters concept (refactoring/unifying older code)
-
-  - covers --first/--last/--prefix/--sort-by options
-  - currently used for borg list/info/delete
-
-Other changes:
-
-- borg check --verify-data slightly tuned (use get_many())
-- change {utcnow} and {now} to ISO-8601 format ("T" date/time separator)
-- repo check: log transaction IDs, improve object count mismatch diagnostic
-- Vagrantfile: use TW's fresh-bootloader pyinstaller branch
-- fix module names in api.rst
-- hashindex: bump api_version
-
-
-Version 1.1.0b1 (2016-08-28)
-----------------------------
-
-New features:
-
-- new commands:
-
-  - borg recreate: re-create existing archives, #787 #686 #630 #70, also see
-    #757, #770.
-
-    - selectively remove files/dirs from old archives
-    - re-compress data
-    - re-chunkify data, e.g. to have upgraded Attic / Borg 0.xx archives
-      deduplicate with Borg 1.x archives or to experiment with chunker-params.
-  - borg diff: show differences between archives
-  - borg with-lock: execute a command with the repository locked, #990
-- borg create:
-
-  - Flexible compression with pattern matching on path/filename,
-    and LZ4 heuristic for deciding compressibility, #810, #1007
-  - visit files in inode order (better speed, esp. for large directories and rotating disks)
-  - in-file checkpoints, #1217
-  - increased default checkpoint interval to 30 minutes (was 5 minutes), #896
-  - added uuid archive format tag, #1151
-  - save mountpoint directories with --one-file-system, makes system restore easier, #1033
-  - Linux: added support for some BSD flags, #1050
-  - add 'x' status for excluded paths, #814
-
-    - also means files excluded via UF_NODUMP, #1080
-- borg check:
-
-  - will not produce the "Checking segments" output unless new --progress option is passed, #824.
-  - --verify-data to verify data cryptographically on the client, #975
-- borg list, #751, #1179
-
-  - removed {formatkeys}, see "borg list --help"
-  - --list-format is deprecated, use --format instead
-  - --format now also applies to listing archives, not only archive contents, #1179
-  - now supports the usual [PATH [PATHS…]] syntax and excludes
-  - new keys: csize, num_chunks, unique_chunks, NUL
-  - supports guaranteed_available hashlib hashes
-    (to avoid varying functionality depending on environment),
-    which includes the SHA1 and SHA2 family as well as MD5
-- borg prune:
-
-  - to better visualize the "thinning out", we now list all archives in
-    reverse time order. rephrase and reorder help text.
-  - implement --keep-last N via --keep-secondly N, also --keep-minutely.
-    assuming that there is not more than 1 backup archive made in 1s,
-    --keep-last N and --keep-secondly N are equivalent, #537
-  - cleanup checkpoints except the latest, #1008
-- borg extract:
-
-  - added --progress, #1449
-  - Linux: limited support for BSD flags, #1050
-- borg info:
-
-  - output is now more similar to borg create --stats, #977
-- borg mount:
-
-  - provide "borgfs" wrapper for borg mount, enables usage via fstab, #743
-  - "versions" mount option - when used with a repository mount, this gives
-    a merged, versioned view of the files in all archives, #729
-- repository:
-
-  - added progress information to commit/compaction phase (often takes some time when deleting/pruning), #1519
-  - automatic recovery for some forms of repository inconsistency, #858
-  - check free space before going forward with a commit, #1336
-  - improved write performance (esp. for rotating media), #985
-
-    - new IO code for Linux
-    - raised default segment size to approx 512 MiB
-  - improved compaction performance, #1041
-  - reduced client CPU load and improved performance for remote repositories, #940
-
-- options that imply output (--show-rc, --show-version, --list, --stats,
-  --progress) don't need -v/--info to have that output displayed, #865
-- add archive comments (via borg (re)create --comment), #842
-- borg list/prune/delete: also output archive id, #731
-- --show-version: shows/logs the borg version, #725
-- added --debug-topic for granular debug logging, #1447
-- use atomic file writing/updating for configuration and key files, #1377
-- BORG_KEY_FILE environment variable, #1001
-- self-testing module, #970
-
-
-Bug fixes:
-
-- list: fixed default output being produced if --format is given with empty parameter, #1489
-- create: fixed overflowing progress line with CJK and similar characters, #1051
-- prune: fixed crash if --prefix resulted in no matches, #1029
-- init: clean up partial repo if passphrase input is aborted, #850
-- info: quote cmdline arguments that have spaces in them
-- fix hardlinks failing in some cases for extracting subtrees, #761
-
-Other changes:
-
-- replace stdlib hmac with OpenSSL, zero-copy decrypt (10-15% increase in
-  performance of hash-lists and extract).
-- improved chunker performance, #1021
-- open repository segment files in exclusive mode (fail-safe), #1134
-- improved error logging, #1440
-- Source:
-
-  - pass meta-data around, #765
-  - move some constants to new constants module
-  - better readability and fewer errors with namedtuples, #823
-  - moved source tree into src/ subdirectory, #1016
-  - made borg.platform a package, #1113
-  - removed dead crypto code, #1032
-  - improved and ported parts of the test suite to py.test, #912
-  - created data classes instead of passing dictionaries around, #981, #1158, #1161
-  - cleaned up imports, #1112
-- Docs:
-
-  - better help texts and sphinx reproduction of usage help:
-
-    - Group options
-    - Nicer list of options in Sphinx
-    - Deduplicate 'Common options' (including --help)
-  - chunker: added some insights by "Voltara", #903
-  - clarify what "deduplicated size" means
-  - fix / update / add package list entries
-  - added a SaltStack usage example, #956
-  - expanded FAQ
-  - new contributors in AUTHORS!
-- Tests:
-
-  - vagrant: add ubuntu/xenial 64bit - this box has still some issues
-  - ChunkBuffer: add test for leaving partial chunk in buffer, fixes #945
-
 
 Version 1.0.8 (2016-10-29)
 --------------------------