changes.rst 29 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571572573574575576577578579580581582583584585586587588589590591592593594595596597598599600601602603604605606607608609610611612613614615616617618619620621622623624625626627628629630631632633634635636637638639640641642643644645646647648649650651652653654655656657658659660661662663664665666667668669670671672673674675676677678679680681682683684685686687688689690691692693694695696697698699700701
  1. .. _important_notes:
  2. Important notes 2.x
  3. ===================
  4. This section provides information about security and corruption issues.
  5. (nothing to see here yet)
  6. .. _upgradenotes2:
  7. Upgrade Notes
  8. =============
  9. borg 1.2.x to borg 2.0
  10. ----------------------
  11. Compatibility notes:
  12. - this is a major "breaking" release that is not compatible with existing repos.
  13. We tried to put all the necessary "breaking" changes into this release, so we
  14. hopefully do not need another breaking release in the near future. The changes
  15. were necessary for improved security, improved speed, unblocking future
  16. improvements, getting rid of legacy crap / design limitations, having less and
  17. simpler code to maintain.
  18. You can use "borg transfer" to transfer archives from borg 1.1/1.2 repos to
  19. a new borg 2.0 repo, but it will need some time and space.
  20. Before using "borg transfer", you must have upgraded to borg >= 1.2.6 (or
  21. another borg version that was patched to fix CVE-2023-CVE-2023-36811) and
  22. you must have followed the upgrade instructions at top of the change log
  23. relating to manifest and archive TAMs (borg2 just requires these TAMs now).
  24. - command line syntax was changed, scripts and wrappers will need changes:
  25. - you will usually either export BORG_REPO=<MYREPO> into your environment or
  26. call borg like: "borg -r <MYREPO> <COMMAND>".
  27. in the docs, we usually omit "-r ..." for brevity.
  28. - the scp-style REPO syntax was removed, please use ssh://..., #6697
  29. - ssh:// URLs: removed support for /~otheruser/, #6855.
  30. If you used this, just replace it by: ssh://user@host:port/home/otheruser/
  31. - -P / --prefix option was removed, please use the similar -a / --match-archives.
  32. - the archive name is always given separately from the repository
  33. (differently than with borg 1.x you must not give repo::archive).
  34. - the archive name is either given as a positional parameter, like:
  35. - borg create myarchive2 /some/path
  36. - borg diff myarchive1 myarchive2
  37. - or, if the command makes sense for an arbitrary amount of archives, archives
  38. can be selected using a glob pattern, like:
  39. - borg delete -a 'sh:myarchive*'
  40. - borg recreate -a 'sh:myarchive*'
  41. - some borg 1.x commands that supported working on a repo AND on an archive
  42. were split into 2 commands, some others were renamed:
  43. - borg 2 repo commands:
  44. - borg rcreate # "repo create", was: borg init
  45. - borg rlist # "repo list"
  46. - borg rinfo # "repo info"
  47. - borg rdelete # "repo delete"
  48. - borg 2 archive commands:
  49. - borg create ARCHIVE ...
  50. - borg list ARCHIVE
  51. - borg extract ARCHIVE ...
  52. - borg diff ARCH1 ARCH2
  53. - borg rename OLDNAME NEWNAME
  54. - borg info -a ARCH_GLOB
  55. - borg delete -a ARCH_GLOB
  56. - borg recreate -a ARCH_GLOB ...
  57. - borg mount -a ARCH_GLOB mountpoint ...
  58. For more details, please consult the docs or --help option output.
  59. - create/recreate/import-tar --timestamp: defaults to local timezone
  60. now (was: UTC)
  61. - some deprecated options were removed:
  62. - removed --remote-ratelimit (use --upload-ratelimit)
  63. - removed --numeric-owner (use --numeric-ids)
  64. - removed --nobsdflags (use --noflags)
  65. - removed --noatime (default now, see also --atime)
  66. - removed --save-space option (does not change behaviour)
  67. - using --list together with --progress is now disallowed (except with --log-json), #7219
  68. - the --glob-archives option was renamed to --match-archives (the short option
  69. name -a is unchanged) and extended to support different pattern styles:
  70. - id: for identical string match (this is the new default!)
  71. - sh: for shell pattern / globbing match (this was used by --glob-archives)
  72. - re: for regular expression match
  73. So you might need to edit your scripts like e.g.::
  74. borg 1.x: --glob-archives 'myserver-*'
  75. borg 2.0: --match-archives 'sh:myserver-*'
  76. - use platformdirs 3.x.x instead of home-grown code. Due to that:
  77. - XDG_*_HOME is not honoured on macOS and on Windows.
  78. - BORG_BASE_DIR can still be used to enforce some base dir + .config/ or .cache/.
  79. - the default macOS config and cache dir will now be in ~/Library/Application Support/borg/.
  80. - create: different included/excluded status chars, #7321
  81. - dry-run: now uses "+" (was: "-") and "-" (was: "x") for included/excluded status
  82. - non-dry-run: now uses "-" (was: "x") for excluded files
  83. Option --filter=... might need an update, if you filter for the status chars
  84. that were changed.
  85. - borg is now more strict and disallows giving some options multiple times -
  86. if that makes no sense. Highlander options, see #6269. That might make scripts
  87. fail now that somehow "worked" before (but maybe didn't work as intended due to
  88. the contradicting options).
  89. .. _changelog:
  90. Change Log 2.x
  91. ==============
  92. Version 2.0.0b8 (not released yet)
  93. ----------------------------------
  94. Please note:
  95. This is a beta release, only for testing - do not use for production repos.
  96. For upgrade and compatibility hints, please also read the section "Upgrade Notes"
  97. above.
  98. New features:
  99. - implement "borg version" (shows client and server version), #7829
  100. Fixes:
  101. - docs: CVE-2023-36811 upgrade steps: consider checkpoint archives, #7802
  102. - check/compact: fix spurious reappearance of orphan chunks since borg 1.2, #6687 -
  103. this consists of 2 fixes:
  104. - for existing chunks: check --repair: recreate shadow index, #7897 #6687
  105. - for newly created chunks: update shadow index when doing a double-put, #7896 #5661
  106. If you have experienced issue #6687, you may want to run borg check --repair
  107. after upgrading to borg 1.2.7 to recreate the shadow index and get rid of the
  108. issue for existing chunks.
  109. - check: fix return code for index entry value discrepancies
  110. - LockRoster.modify: no KeyError if element was already gone, #7937
  111. - create --X-from-command: run subcommands with a clean environment, #7916
  112. - list --sort-by: support "archive" as alias of "name", #7873
  113. - fix rc and msg if arg parsing throws an exception, #7885
  114. - PATH: do not accept empty strings, #4221
  115. - fix invalid pattern argument error msg
  116. - zlib legacy decompress fixes, #7883
  117. Other changes:
  118. - replace archive/manifest TAMs by typed repo objects (ro_type), docs, #7670
  119. - crypto: use a one-step kdf for session keys, #7953
  120. - remove recreate --recompress option, use the more efficient repo-wide "rcompress".
  121. - include unistd.h in _chunker.c (fix for Python 3.13)
  122. - allow msgpack 1.0.7
  123. - allow platformdirs 4, #7950
  124. - use and require cython3
  125. - move conftest.py to src/borg/testsuite, #6386
  126. - use less setup.py, use pip and build
  127. - linux: use pkgconfig to find libacl
  128. - borg.logger: use same method params as python logging
  129. - create and use Brewfile, document "brew bundle" install (macOS)
  130. - blacken master branch
  131. - prevent CLI argument issues in scripts/glibc_check.py
  132. - pyproject.toml: exclude source files which have been compiled, #7828
  133. - sdist: dynamically compute readme (long_description)
  134. - init: better borg key export instructions
  135. - scripts/make.py: move clean, build_man, build_usage to there,
  136. so we do not need to invoke setup.py directly, update docs
  137. - vagrant:
  138. - use openssl 3.0 on macOS
  139. - add script for fetching borg binaries from VMs, #7989
  140. - use generic/openbsd7 box
  141. - netbsd: test on py311 only
  142. - remove debian 9 "stretch" box
  143. - use freebsd 14, #6871
  144. - use python 3.9.4 for tests, latest python 3.11.7 for binary builds
  145. - use pyinstaller 6.3.0
  146. - docs:
  147. - add typical PR workflow to development docs, #7495
  148. - improve docs for borg with-lock, add example #8024
  149. - create disk/partition sector backup by disk serial number
  150. - Add "check.rebuild_refcounts" message
  151. - not only attack/unsafe, can also be a fs issue, #7853
  152. - use virtualenv on Cygwin
  153. - readthedocs: also build offline docs, #7835
  154. - do not refer to setup.py installation method
  155. - how to run the testsuite using the dist package
  156. - requirements are defined in pyproject.toml
  157. Version 2.0.0b7 (2023-09-14)
  158. ----------------------------
  159. New features:
  160. - BORG_WORKAROUNDS=authenticated_no_key to extract from authenticated repos
  161. without having the borg key, #7700
  162. Fixes:
  163. - archive tam verify security fix, fixes CVE-2023-36811
  164. - remote logging/progress: use callback to send queued records, #7662
  165. - make_path_safe: remove test for backslashes, #7651
  166. - benchmark cpu: use sanitized path, #7654
  167. - create: do not try to read parent dir of recursion root, #7746
  168. Other changes:
  169. - always implicitly require archive TAMs (all archives have TAMs since borg 1.2.6)
  170. - always implicitly require manifest TAMs (manifests have TAMs since borg 1.0.9)
  171. - rlist: remove support for {tam} placeholder, archives are now always TAM-authenticated.
  172. - support / test on Python 3.12
  173. - allow msgpack 1.0.6 (which has py312 wheels), #7810
  174. - manifest: move item_keys into config dict (manifest.version == 2 now), #7710
  175. - replace "datetime.utcfromtimestamp" to avoid deprecation warnings with Python 3.12
  176. - properly normalise paths on Windows (forward slashes, integrate drive letter into path)
  177. - Docs:
  178. - move upgrade / compat. notes to own section, see #7546
  179. - fix borg delete examples, #7759
  180. - improve rcreate / related repos docs
  181. - automated-local.rst: use UUID for consistent udev rule
  182. - rewrite `borg check` docs, #7578
  183. - misc. other docs updates
  184. - Tests / CI / Vagrant:
  185. - major testsuite refactoring: a lot more tests now use pytest, #7626
  186. - freebsd: add some ACL tests, #7745
  187. - fix test_disk_full, #7617
  188. - fix failing test_get_runtime_dir test on OpenBSD, #7719
  189. - CI: run on ubuntu 22.04
  190. - CI: test building the docs
  191. - simplify flake8 config, fix some complaints
  192. - use pyinstaller 5.13.1 to build the borg binaries
  193. Version 2.0.0b6 (2023-06-11)
  194. ----------------------------
  195. New features:
  196. - diff: include changes in ctime and mtime, #7248
  197. - diff: sort JSON output alphabetically
  198. - diff --content-only: option added to ignore metadata changes
  199. - diff: add --format option, #4634
  200. - import-tar --ignore-zeros: new option to support importing concatenated tars, #7432
  201. - debug id-hash / parse-obj / format-obj: new debug commands, #7406
  202. - transfer --compression=C --recompress=M: recompress while transferring, #7529
  203. - extract --continue: continue a previously interrupted extraction, #1356
  204. - prune --list-kept/--list-pruned: only list the kept (or pruned) archives, #7511
  205. - prune --short/--format: enable users to format the list output, #3238
  206. - implement BORG_<CMD>_FORMAT env vars for prune, list, rlist, #5166
  207. - rlist: size and nfiles format keys
  208. - implement unix domain (ipc) socket support, #6183::
  209. borg serve --socket # server side (not started automatically!)
  210. borg -r socket:///path/to/repo ... # client side
  211. - add get_runtime_dir / BORG_RUNTIME_DIR (contains e.g. .sock and .pid file)
  212. - support shell-style alternatives, like: sh:image.{png,jpg}, #7602
  213. Fixes:
  214. - do not retry on permission errors (pointless)
  215. - transfer: verify chunks we get using assert_id, #7383
  216. - fix config/cache dir compatibility issues, #7445
  217. - xattrs: fix namespace processing on FreeBSD, #6997
  218. - ProgressIndicatorPercent: fix space computation for wide chars, #3027
  219. - delete: remove --cache-only option, #7440.
  220. for deleting the cache only, use: borg rdelete --cache-only
  221. - borg debug get-obj/put-obj: fixed chunk id
  222. - create: ignore empty paths, print warning, #5637
  223. - extract: support extraction of atime/mtime on win32
  224. - benchmark crud: use TemporaryDirectory below given path, #4706
  225. - Ensure that cli options specified with action=Highlander can only be set once, even
  226. if the set value is a default value. Add tests for action=Highlander, #7500, #6269.
  227. - Fix argparse error messages from misc. validators (being more specific).
  228. - put security infos into data dir, add BORG_DATA_DIR env var, #5760
  229. - setup.cfg: remove setup_requires (we have a pyproject.toml for that), #7574
  230. - do not crash for empty archives list in borg rlist date based matching, #7522
  231. - sanitize paths during archive creation and extraction, #7108 #7099
  232. - make sure we do not get backslashes into item paths
  233. Other changes:
  234. - allow msgpack 1.0.5 also
  235. - development.lock.txt: upgrade cython to 0.29.35, misc. other upgrades
  236. - clarify platformdirs requirements, #7393.
  237. 3.0.0 is only required for macOS due to breaking changes.
  238. 2.6.0 was the last breaking change for Linux/UNIX.
  239. - mount: improve mountpoint error msgs, see #7496
  240. - more Highlander options, #6269
  241. - Windows: simplify building (just use pip)
  242. - refactor toplevel exception handling, #6018
  243. - remove nonce management, related repo methods (not needed for borg2)
  244. - borg.remote: remove support for borg < 1.1.0
  245. ($LOG, logging setup, exceptions, rpc tuple data format, version)
  246. - new remote and progress logging, #7604
  247. - borg.logger: add logging debugging functionality
  248. - add function to clear empty directories at end of compact process
  249. - unify scanning and listing of segment dirs / segment files, #7597
  250. - replace `LRUCache` internals with `OrderedDict`
  251. - docs:
  252. - add installation instructions for Windows
  253. - improve --one-file-system help and docs (macOS APFS), #5618 #4876
  254. - BORG_KEY_FILE: clarify docs, #7444
  255. - installation: add link to OS dependencies, #7356
  256. - update FAQ about locale/unicode issues, #6999
  257. - improve mount options rendering, #7359
  258. - make timestamps in manual pages reproducible.
  259. - describe performing pull-backups via ssh remote forwarding
  260. - suggest to use forced command when using remote-forwarding via ssh
  261. - fix some -a / --match-archives docs issues
  262. - incl./excl. options header, clarify --path-from-stdin exclusive control
  263. - add note about MAX_DATA_SIZE
  264. - update security support docs
  265. - improve patterns help
  266. - CI / tests / vagrant:
  267. - added pre-commit for linting purposes, #7476
  268. - resolved mode bug and added sleep clause for darwin systems, #7470
  269. - "auto" compressor tests: do not assume zlib is better than lz4, #7363
  270. - add stretch64 VM with deps built from source
  271. - misc. other CI / test fixes and updates
  272. - vagrant: add lunar64 VM, fix packages_netbsd
  273. - avoid long ids in pytest output
  274. - tox: package = editable-legacy, #7580
  275. - tox under fakeroot: fix finding setup_docs, #7391
  276. - check buzhash chunksize distribution, #7586
  277. - use debian/bookworm64 box
  278. Version 2.0.0b5 (2023-02-27)
  279. ----------------------------
  280. New features:
  281. - create: implement retries for individual fs files
  282. (e.g. if a file changed while we read it, if a file had an OSError)
  283. - info: add used storage quota, #7121
  284. - transfer: support --progress
  285. - create/recreate/import-tar: add --checkpoint-volume option
  286. - support date-based matching for archive selection,
  287. add --newer/--older/--newest/--oldest options, #7062 #7296
  288. Fixes:
  289. - disallow --list with --progress, #7219
  290. - create: fix --list --dry-run output for directories, #7209
  291. - do no assume hardlink_master=True if not present, #7175
  292. - fix item_ptrs orphaned chunks of checkpoint archives
  293. - avoid orphan content chunks on BackupOSError, #6709
  294. - transfer: fix bug in obfuscated data upgrade code
  295. - fs.py: fix bug in f-string (thanks mypy!)
  296. - recreate: when --target is given, do not detect "nothing to do", #7254
  297. - locking (win32): deal with os.rmdir/listdir PermissionErrors
  298. - locking: thread id must be parsed as hex from lock file name
  299. - extract: fix mtime when ResourceFork xattr is set (macOS specific), #7234
  300. - recreate: without --chunker-params borg shall not rechunk, #7336
  301. - allow mixing --progress and --list in log-json mode
  302. - add "files changed while reading" to Statistics class, #7354
  303. - fixed keys determination in Statistics.__add__(), #7355
  304. Other changes:
  305. - use local time / local timezone to output timestamps, #7283
  306. - update development.lock.txt, including a setuptools security fix, #7227
  307. - remove --save-space option (does not change behaviour)
  308. - remove part files from final archive
  309. - remove --consider-part-files, related stats code, update docs
  310. - transfer: drop part files
  311. - check: show id of orphaned chunks
  312. - ArchiveItem.cmdline list-of-str -> .command_line str, #7246
  313. - Item: symlinks: rename .source to .target, #7245
  314. - Item: make user/group/uid/gid optional
  315. - create: do not store user/group for stdin data by default, #7249
  316. - extract: chown only if we have u/g info in archived item, #7249
  317. - export-tar: for items w/o uid/gid, default to 0/0, #7249
  318. - fix some uid/gid lookup code / tests for win32
  319. - cache.py: be less verbose during cache sync
  320. - update bash completion script commands and options, #7273
  321. - require and use platformdirs 3.x.x package, tests
  322. - better included/excluded status chars, docs, #7321
  323. - undef NDEBUG for chunker and hashindex (make assert() work)
  324. - assert_id: better be paranoid (add back same crypto code as in old borg), #7362
  325. - check --verify_data: always decompress and call assert_id(), #7362
  326. - make hashindex_compact simpler and probably faster, minor fixes, cleanups, more tests
  327. - hashindex minor fixes, refactor, tweaks, tests
  328. - pyinstaller: remove icon
  329. - validation / placeholders / JSON:
  330. - implement (text|binary)_to_json: key (text), key_b64 (base64(binary))
  331. - remove bpath, barchive, bcomment placeholders / JSON keys
  332. - archive metadata: make sure hostname and username have no surrogate escapes
  333. - text attributes (like archive name, comment): validate more strictly, #2290
  334. - transfer: validate archive names and comment before transfer
  335. - json output: use text_to_json (path, target), #6151
  336. - docs:
  337. - docs and comments consistency, readability and spelling fixes
  338. - fix --progress display description, #7180
  339. - document how borg deals with non-unicode bytes in JSON output
  340. - document another way to get UTF-8 encoding on stdin/stdout/stderr, #2273
  341. - pruning interprets timestamps in the local timezone where borg prune runs
  342. - shellpattern: add license, use copyright/license markup
  343. - key change-passphrase: fix --encryption value in examples
  344. - remove BORG_LIBB2_PREFIX (not used any more)
  345. - Installation: Update Fedora in distribution list, #7357
  346. - add .readthedocs.yaml (use py311, use non-shallow clone)
  347. - tests:
  348. - fix archiver tests on Windows, add running the tests to Windows CI
  349. - fix tox4 passenv issue, #7199
  350. - github actions updates (fix deprecation warnings)
  351. - add tests for borg transfer/upgrade
  352. - fix test hanging reading FIFO when `borg create` failed
  353. - mypy inspired fixes / updates
  354. - fix prune tests, prune in localtime
  355. - do not look up uid 0 / gid 0, but current process uid/gid
  356. - safe_unlink tests: use os.link to support win32 also
  357. - fix test_size_on_disk_accurate for large st_blksize, #7250
  358. - relaxed timestamp comparisons, use same_ts_ns
  359. - add test for extracted directory mtime
  360. - use "fail" chunker to test erroneous input file skipping
  361. Version 2.0.0b4 (2022-11-27)
  362. ----------------------------
  363. Fixes:
  364. - transfer/upgrade: fix borg < 1.2 chunker_params, #7079
  365. - transfer/upgrade: do not access Item._dict, #7077
  366. - transfer/upgrade: fix crash in borg transfer, #7156
  367. - archive.save(): always use metadata from stats, #7072
  368. - benchmark: fixed TypeError in compression benchmarks, #7075
  369. - fix repository.scan api minimum requirement
  370. - fix args.paths related argparsing, #6994
  371. Other changes:
  372. - tar_filter: recognize .tar.zst as zstd, #7093
  373. - adding performance statistics to borg create, #6991
  374. - docs: add rcompress to usage index
  375. - tests:
  376. - use github and MSYS2 for Windows CI, #7097
  377. - win32 and cygwin: test fixes / skip hanging test
  378. - vagrant / github CI: use python 3.11.0 / 3.10.8
  379. - vagrant:
  380. - upgrade pyinstaller to 5.6.2 (supports python 3.11)
  381. - use python 3.11 to build the borg binary
  382. Version 2.0.0b3 (2022-10-02)
  383. ----------------------------
  384. Fixes:
  385. - transfer: fix user/group == None crash with borg1 archives
  386. - compressors: avoid memoryview related TypeError
  387. - check: fix uninitialised variable if repo is completely empty, #7034
  388. - do not use version_tuple placeholder in setuptools_scm template, #7024
  389. - get_chunker: fix missing sparse=False argument, #7056
  390. New features:
  391. - rcompress: do a repo-wide (re)compression, #7037
  392. - implement pattern support for --match-archives, #6504
  393. - BORG_LOCK_WAIT=n env var to set default for --lock-wait option, #5279
  394. Other:
  395. - repository.scan: misc. fixes / improvements
  396. - metadata: differentiate between empty/zero and unknown, #6908
  397. - CI: test pyfuse3 with python 3.11
  398. - use more relative imports
  399. - make borg.testsuite.archiver a package, split archiver tests into many modules
  400. - support reading new, improved hashindex header format, #6960.
  401. added version number and num_empty to the HashHeader, fixed alignment.
  402. - vagrant: upgrade pyinstaller 4.10 -> 5.4.1, use python 3.9.14 for binary build
  403. - item.pyx: use more Cython (faster, uses less memory), #5763
  404. Version 2.0.0b2 (2022-09-10)
  405. ----------------------------
  406. Bug fixes:
  407. - xattrs / extended stat: improve exception handling, #6988
  408. - fix and refactor replace_placeholders, #6966
  409. New features:
  410. - support archive timestamps with utc offsets, adapt them when using
  411. borg transfer to transfer from borg 1.x repos (append +00:00 for UTC).
  412. - create/recreate/import-tar --timestamp: accept giving timezone via
  413. its utc offset. defaults to local timezone, if no utc offset is given.
  414. Other changes:
  415. - chunks: have separate encrypted metadata (ctype, clevel, csize, size)
  416. chunk = enc_meta_len16 + encrypted(msgpacked(meta)) + encrypted(compressed(data)).
  417. this breaks repo format compatibility, you need to create fresh repos!
  418. - repository api: flags support, #6982
  419. - OpenBSD only - statically link OpenSSL, #6474.
  420. Avoid conflicting with shared libcrypto from the base OS pulled in via dependencies.
  421. - restructured source code
  422. - update diagrams to odg format, #6928
  423. Version 2.0.0b1 (2022-08-08)
  424. ----------------------------
  425. New features:
  426. - massively increase archive metadata stream size limit, #1473.
  427. currently rather testing the code, scalability will improve later, see #6945.
  428. - rcreate --copy-crypt-key: copy crypt_key from key of other repo, #6710.
  429. default: create new, random authenticated encryption key.
  430. - prune/delete --checkpoint-interval=1800 and ctrl-c/SIGINT support, #6284
  431. Fixes:
  432. - ctrl-c must not kill important subprocesses, #6912
  433. - transfer: check whether ID hash method and chunker secret are same.
  434. add PlaintextKey and AuthenticatedKey support to uses_same_id_hash function.
  435. - check: try harder to create the key, #5719
  436. - SaveFile: use a custom mkstemp with mode support, #6933, #6400
  437. - make setuptools happy, #6874
  438. - fix misc. compiler warnings
  439. - list: fix {flags:<WIDTH>} formatting, #6081
  440. Other changes:
  441. - new crypto does not need to call ._assert_id(), update code and docs.
  442. https://github.com/borgbackup/borg/pull/6463#discussion_r925436156
  443. - check: --verify-data does not need to decompress with new crypto modes
  444. - Key: crypt_key instead of enc_key + enc_hmac_key, #6611
  445. - misc. docs updates and improvements
  446. - CI: test on macOS 12 without fuse / fuse tests
  447. - repository: add debug logging for issue #6687
  448. - _version.py: remove trailing blank, add LF at EOF (make pep8 checker happy)
  449. Version 2.0.0a4 (2022-07-17)
  450. ----------------------------
  451. New features:
  452. - recreate: consider level for recompression, #6698, #3622
  453. Other changes:
  454. - stop using libdeflate
  455. - CI: add mypy (if we add type hints, it can do type checking)
  456. - big changes to the source code:
  457. - split up archiver module, transform it into a package
  458. - use Black for automated code formatting
  459. - remove some legacy code
  460. - adapt/fix code for mypy
  461. - use language_level = 3str for cython (this will be the default in cython 3)
  462. - docs: document HardLinkManager and hlid, #2388
  463. Version 2.0.0a3 (2022-07-04)
  464. ----------------------------
  465. Fixes:
  466. - check repo version, accept old repos only for --other-repo (e.g. rcreate/transfer).
  467. v2 is the default repo version for borg 2.0. v1 repos must only be used in a
  468. read-only way, e.g. for --other-repo=V1_REPO with borg init and borg transfer!
  469. New features:
  470. - transfer: --upgrader=NoOp is the default.
  471. This is to support general-purpose transfer of archives between related borg2
  472. repos.
  473. - transfer: --upgrader=From12To20 must be used to transfer (and convert) archives
  474. from borg 1.2 repos to borg 2.0 repos.
  475. Other changes:
  476. - removed some deprecated options
  477. - removed -P (aka --prefix) option, #6806. The option -a (aka --glob-archives)
  478. can be used for same purpose and is more powerful, e.g.: -a 'PREFIX*'
  479. - rcreate: always use argon2 kdf for new repos, #6820
  480. - rcreate: remove legacy encryption modes for new repos, #6490
  481. Version 2.0.0a2 (2022-06-26)
  482. ----------------------------
  483. Changes:
  484. - split repo and archive name into separate args, #948
  485. - use -r or --repo or BORG_REPO env var to give the repository
  486. - use --other-repo or BORG_OTHER_REPO to give another repo (e.g. borg transfer)
  487. - use positional argument for archive name or `-a ARCH_GLOB`
  488. - remove support for scp-style repo specification, use ssh://...
  489. - simplify stats output: repo ops -> repo stats, archive ops -> archive stats
  490. - repository index: add payload size (==csize) and flags to NSIndex entries
  491. - repository index: set/query flags, iteration over flagged items (NSIndex)
  492. - repository: sync write file in get_fd
  493. - stats: deduplicated size now, was deduplicated compressed size in borg 1.x
  494. - remove csize support at most places in the code (chunks index, stats, get_size,
  495. Item.chunks)
  496. - replace problematic/ugly hardlink_master approach of borg 1.x by:
  497. - symmetric hlid (all hardlinks pointing to same inode have same hlid)
  498. - all archived hardlinked regular files have a chunks list
  499. - borg rcreate --other-repo=OTHER_REPO: reuse key material from OTHER_REPO, #6554.
  500. This is useful if you want to use borg transfer to transfer archives from an
  501. existing borg 1.1/1.2 repo. If the chunker secret and the id key and algorithm
  502. stay the same, the deduplication will also work between past and future backups.
  503. - borg transfer:
  504. - efficiently copy archives from a borg 1.1/1.2 repo to a new repo.
  505. uses deduplication and does not decompress/recompress file content data.
  506. - does some cleanups / fixes / conversions:
  507. - disallow None value for .user/group/chunks/chunks_healthy
  508. - cleanup msgpack related str/bytes mess, use new msgpack spec, #968
  509. - obfuscation: fix byte order for size, #6701
  510. - compression: use the 2 bytes for type and level, #6698
  511. - use version 2 for new archives
  512. - convert timestamps int/bigint -> msgpack.Timestamp, see #2323
  513. - all hardlinks have chunks, maybe chunks_healthy, hlid
  514. - remove the zlib type bytes hack
  515. - make sure items with chunks have precomputed size
  516. - removes the csize element from the tuples in the Item.chunks list
  517. - clean item of attic 0.13 'acl' bug remnants
  518. - crypto: see 1.3.0a1 log entry
  519. - removed "borg upgrade" command (not needed any more)
  520. - compact: removed --cleanup-commits option
  521. - docs: fixed quickstart and usage docs with new cli command syntax
  522. - docs: removed the parts talking about potential AES-CTR mode issues
  523. (we will not use that any more).
  524. Version 1.3.0a1 (2022-04-15)
  525. ----------------------------
  526. Although this was released as 1.3.0a1, it can be also seen as 2.0.0a1 as it was
  527. later decided to do breaking changes and thus the major release number had to
  528. be increased (thus, there will not be a 1.3.0 release, but 2.0.0).
  529. New features:
  530. - init: new --encryption=(repokey|keyfile)-[blake2-](aes-ocb|chacha20-poly1305)
  531. - New, better, faster crypto (see encryption-aead diagram in the docs), #6463.
  532. - New AEAD cipher suites: AES-OCB and CHACHA20-POLY1305.
  533. - Session keys are derived via HKDF from random session id and master key.
  534. - Nonces/MessageIVs are counters starting from 0 for each session.
  535. - AAD: chunk id, key type, messageIV, sessionID are now authenticated also.
  536. - Solves the potential AES-CTR mode counter management issues of the legacy crypto.
  537. - init: --key-algorithm=argon2 (new default KDF, older pbkdf2 also still available)
  538. borg key change-passphrase / change-location keeps the key algorithm unchanged.
  539. - key change-algorithm: to upgrade existing keys to argon2 or downgrade to pbkdf2.
  540. We recommend you to upgrade unless you have to keep the key compatible with older versions of borg.
  541. - key change-location: usable for repokey <-> keyfile location change
  542. - benchmark cpu: display benchmarks of cpu bound stuff
  543. - export-tar: new --tar-format=PAX (default: GNU)
  544. - import-tar/export-tar: can use PAX format for ctime and atime support
  545. - import-tar/export-tar: --tar-format=BORG: roundtrip ALL item metadata, #5830
  546. - repository: create and use version 2 repos only for now
  547. - repository: implement PUT2: header crc32, overall xxh64, #1704
  548. Other changes:
  549. - require python >= 3.9, #6315
  550. - simplify libs setup, #6482
  551. - unbundle most bundled 3rd party code, use libs, #6316
  552. - use libdeflate.crc32 (Linux and all others) or zlib.crc32 (macOS)
  553. - repository: code cleanups / simplifications
  554. - internal crypto api: speedups / cleanups / refactorings / modernisation
  555. - remove "borg upgrade" support for "attic backup" repos
  556. - remove PassphraseKey code and borg key migrate-to-repokey command
  557. - OpenBSD: build borg with OpenSSL (not: LibreSSL), #6474
  558. - remove support for LibreSSL, #6474
  559. - remove support for OpenSSL < 1.1.1