changes.rst 57 KB

1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162636465666768697071727374757677787980818283848586878889909192939495969798991001011021031041051061071081091101111121131141151161171181191201211221231241251261271281291301311321331341351361371381391401411421431441451461471481491501511521531541551561571581591601611621631641651661671681691701711721731741751761771781791801811821831841851861871881891901911921931941951961971981992002012022032042052062072082092102112122132142152162172182192202212222232242252262272282292302312322332342352362372382392402412422432442452462472482492502512522532542552562572582592602612622632642652662672682692702712722732742752762772782792802812822832842852862872882892902912922932942952962972982993003013023033043053063073083093103113123133143153163173183193203213223233243253263273283293303313323333343353363373383393403413423433443453463473483493503513523533543553563573583593603613623633643653663673683693703713723733743753763773783793803813823833843853863873883893903913923933943953963973983994004014024034044054064074084094104114124134144154164174184194204214224234244254264274284294304314324334344354364374384394404414424434444454464474484494504514524534544554564574584594604614624634644654664674684694704714724734744754764774784794804814824834844854864874884894904914924934944954964974984995005015025035045055065075085095105115125135145155165175185195205215225235245255265275285295305315325335345355365375385395405415425435445455465475485495505515525535545555565575585595605615625635645655665675685695705715725735745755765775785795805815825835845855865875885895905915925935945955965975985996006016026036046056066076086096106116126136146156166176186196206216226236246256266276286296306316326336346356366376386396406416426436446456466476486496506516526536546556566576586596606616626636646656666676686696706716726736746756766776786796806816826836846856866876886896906916926936946956966976986997007017027037047057067077087097107117127137147157167177187197207217227237247257267277287297307317327337347357367377387397407417427437447457467477487497507517527537547557567577587597607617627637647657667677687697707717727737747757767777787797807817827837847857867877887897907917927937947957967977987998008018028038048058068078088098108118128138148158168178188198208218228238248258268278288298308318328338348358368378388398408418428438448458468478488498508518528538548558568578588598608618628638648658668678688698708718728738748758768778788798808818828838848858868878888898908918928938948958968978988999009019029039049059069079089099109119129139149159169179189199209219229239249259269279289299309319329339349359369379389399409419429439449459469479489499509519529539549559569579589599609619629639649659669679689699709719729739749759769779789799809819829839849859869879889899909919929939949959969979989991000100110021003100410051006100710081009101010111012101310141015101610171018101910201021102210231024102510261027102810291030103110321033103410351036103710381039104010411042104310441045104610471048104910501051105210531054105510561057105810591060106110621063106410651066106710681069107010711072107310741075107610771078107910801081108210831084108510861087108810891090109110921093109410951096109710981099110011011102110311041105110611071108110911101111111211131114111511161117111811191120112111221123112411251126112711281129113011311132113311341135113611371138113911401141114211431144114511461147114811491150115111521153115411551156115711581159116011611162116311641165116611671168116911701171117211731174117511761177117811791180118111821183118411851186118711881189119011911192119311941195119611971198119912001201120212031204120512061207120812091210121112121213121412151216121712181219122012211222122312241225122612271228122912301231123212331234123512361237123812391240124112421243124412451246124712481249125012511252125312541255125612571258125912601261126212631264126512661267126812691270127112721273127412751276127712781279128012811282128312841285128612871288128912901291129212931294129512961297129812991300130113021303130413051306130713081309131013111312131313141315131613171318131913201321132213231324132513261327132813291330133113321333133413351336133713381339134013411342134313441345134613471348134913501351135213531354135513561357135813591360136113621363136413651366136713681369137013711372137313741375137613771378137913801381138213831384138513861387138813891390139113921393139413951396139713981399140014011402140314041405140614071408140914101411141214131414141514161417141814191420142114221423142414251426142714281429143014311432143314341435143614371438143914401441144214431444144514461447144814491450145114521453145414551456145714581459146014611462146314641465146614671468146914701471147214731474
  1. Changelog
  2. =========
  3. Important note about pre-1.0.4 potential repo corruption
  4. --------------------------------------------------------
  5. Some external errors (like network or disk I/O errors) could lead to
  6. corruption of the backup repository due to issue #1138.
  7. A sign that this happened is if "E" status was reported for a file that can
  8. not be explained by problems with the source file. If you still have logs from
  9. "borg create -v --list", you can check for "E" status.
  10. Here is what could cause corruption and what you can do now:
  11. 1) I/O errors (e.g. repo disk errors) while writing data to repo.
  12. This could lead to corrupted segment files.
  13. Fix::
  14. # check for corrupt chunks / segments:
  15. borg check -v --repository-only REPO
  16. # repair the repo:
  17. borg check -v --repository-only --repair REPO
  18. # make sure everything is fixed:
  19. borg check -v --repository-only REPO
  20. 2) Unreliable network / unreliable connection to the repo.
  21. This could lead to archive metadata corruption.
  22. Fix::
  23. # check for corrupt archives:
  24. borg check -v --archives-only REPO
  25. # delete the corrupt archives:
  26. borg delete --force REPO::CORRUPT_ARCHIVE
  27. # make sure everything is fixed:
  28. borg check -v --archives-only REPO
  29. 3) In case you want to do more intensive checking.
  30. The best check that everything is ok is to run a dry-run extraction::
  31. borg extract -v --dry-run REPO::ARCHIVE
  32. Version 1.1.0 (not released yet)
  33. --------------------------------
  34. New features:
  35. - borg check: will not produce the "Checking segments" output unless
  36. new --progress option is passed, #824.
  37. - options that imply output (--show-rc, --show-version, --list, --stats,
  38. --progress) don't need -v/--info to have that output displayed, #865
  39. - borg recreate: re-create existing archives, #787 #686 #630 #70, also see
  40. #757, #770.
  41. - selectively remove files/dirs from old archives
  42. - re-compress data
  43. - re-chunkify data, e.g. to have upgraded Attic / Borg 0.xx archives
  44. deduplicate with Borg 1.x archives or to experiment with chunker-params.
  45. - create: visit files in inode order (better speed, esp. for large directories
  46. and rotating disks)
  47. - borg diff: show differences between archives
  48. - borg list improved:
  49. - removed {formatkeys}, see "borg list --help"
  50. - --list-format is deprecated, use --format instead
  51. - now supports the usual [PATH [PATHS…]] syntax and excludes
  52. - new keys: csize, num_chunks, unique_chunks, NUL
  53. - supports guaranteed_available hashlib hashes
  54. (to avoid varying functionality depending on environment)
  55. - prune:
  56. - to better visualize the "thinning out", we now list all archives in
  57. reverse time order. rephrase and reorder help text.
  58. - implement --keep-last N via --keep-secondly N, also --keep-minutely.
  59. assuming that there is not more than 1 backup archive made in 1s,
  60. --keep-last N and --keep-secondly N are equivalent, #537
  61. - borg comment: add archive comments, #842
  62. - provide "borgfs" wrapper for borg mount, enables usage via fstab, #743
  63. - create: add 'x' status for excluded paths, #814
  64. - --show-version: shows/logs the borg version, #725
  65. - borg list/prune/delete: also output archive id, #731
  66. Bug fixes:
  67. - init: clean up partial repo if passphrase input is aborted, #850
  68. - info: quote cmdline arguments that have spaces in them
  69. - failing hashindex tests on netbsd, #804
  70. - fix links failing for extracting subtrees, #761
  71. Other changes:
  72. - replace stdlib hmac with OpenSSL, zero-copy decrypt (10-15% increase in
  73. performance of hash-lists and extract).
  74. - Source:
  75. - pass meta-data around, #765
  76. - move some constants to new constants module
  77. - better readability and less errors with namedtuples, #823
  78. - Docs:
  79. - better help texts and sphinx reproduction of usage help:
  80. - Group options
  81. - Nicer list of options in Sphinx
  82. - Deduplicate 'Common options' (including --help)
  83. - chunker: added some insights by "Voltara", #903
  84. - clarify what "deduplicated size" means
  85. - fix / update / add package list entries
  86. - new contributors in AUTHORS!
  87. - Tests:
  88. - vagrant: add ubuntu/xenial 64bit - this box has still some issues
  89. - ChunkBuffer: add test for leaving partial chunk in buffer, fixes #945
  90. Version 1.0.7 (not released yet)
  91. --------------------------------
  92. Security fixes:
  93. - fix security issue with remote repository access, #1428
  94. Version 1.0.7rc1 (2016-08-05)
  95. -----------------------------
  96. Bug fixes:
  97. - fix repo lock deadlocks (related to lock upgrade), #1220
  98. - catch unpacker exceptions, resync, #1351
  99. - fix borg break-lock ignoring BORG_REPO env var, #1324
  100. - files cache performance fixes (fixes unneccessary re-reading/chunking/
  101. hashing of unmodified files for some use cases):
  102. - fix unintended file cache eviction, #1430
  103. - implement BORG_FILES_CACHE_TTL, update FAQ, raise default TTL from 10
  104. to 20, #1338
  105. - FUSE:
  106. - cache partially read data chunks (performance), #965, #966
  107. - always create a root dir, #1125
  108. - use an OrderedDict for helptext, making the build reproducible, #1346
  109. - RemoteRepository init: always call close on exceptions, #1370 (cosmetic)
  110. - ignore stdout/stderr broken pipe errors (cosmetic), #1116
  111. New features:
  112. - better borg versions management support (useful esp. for borg servers
  113. wanting to offer multiple borg versions and for clients wanting to choose
  114. a specific server borg version), #1392:
  115. - add BORG_VERSION environment variable before executing "borg serve" via ssh
  116. - add new placeholder {borgversion}
  117. - substitute placeholders in --remote-path
  118. - borg init --append-only option (makes using the more secure append-only mode
  119. more convenient. when used remotely, this requires 1.0.7+ also on the borg
  120. server), #1291.
  121. Other changes:
  122. - Vagrantfile:
  123. - darwin64: upgrade to FUSE for macOS 3.4.1 (aka osxfuse), #1378
  124. - xenial64: use user "ubuntu", not "vagrant" (as usual), #1331
  125. - tests:
  126. - fix fuse tests on OS X, #1433
  127. - docs:
  128. - FAQ: add backup using stable filesystem names recommendation
  129. - FAQ about glibc compatibility added, #491, glibc-check improved
  130. - FAQ: 'A' unchanged file; remove ambiguous entry age sentence.
  131. - OS X: install pkg-config to build with FUSE support, fixes #1400
  132. - add notes about shell/sudo pitfalls with env. vars, #1380
  133. - added platform feature matrix
  134. - implement borg debug-dump-repo-objs
  135. Version 1.0.6 (2016-07-12)
  136. --------------------------
  137. Bug fixes:
  138. - Linux: handle multiple LD_PRELOAD entries correctly, #1314, #1111
  139. - Fix crash with unclear message if the libc is not found, #1314, #1111
  140. Other changes:
  141. - tests:
  142. - Fixed O_NOATIME tests for Solaris and GNU Hurd, #1315
  143. - Fixed sparse file tests for (file) systems not supporting it, #1310
  144. - docs:
  145. - Fixed syntax highlighting, #1313
  146. - misc docs: added data processing overview picture
  147. Version 1.0.6rc1 (2016-07-10)
  148. -----------------------------
  149. New features:
  150. - borg check --repair: heal damaged files if missing chunks re-appear (e.g. if
  151. the previously missing chunk was added again in a later backup archive),
  152. #148. (*) Also improved logging.
  153. Bug fixes:
  154. - sync_dir: silence fsync() failing with EINVAL, #1287
  155. Some network filesystems (like smbfs) don't support this and we use this in
  156. repository code.
  157. - borg mount (FUSE):
  158. - fix directories being shadowed when contained paths were also specified,
  159. #1295
  160. - raise I/O Error (EIO) on damaged files (unless -o allow_damaged_files is
  161. used), #1302. (*)
  162. - borg extract: warn if a damaged file is extracted, #1299. (*)
  163. - Added some missing return code checks (ChunkIndex._add, hashindex_resize).
  164. - borg check: fix/optimize initial hash table size, avoids resize of the table.
  165. Other changes:
  166. - tests:
  167. - add more FUSE tests, #1284
  168. - deduplicate fuse (u)mount code
  169. - fix borg binary test issues, #862
  170. - docs:
  171. - changelog: added release dates to older borg releases
  172. - fix some sphinx (docs generator) warnings, #881
  173. Notes:
  174. (*) Some features depend on information (chunks_healthy list) added to item
  175. metadata when a file with missing chunks was "repaired" using all-zero
  176. replacement chunks. The chunks_healthy list is generated since borg 1.0.4,
  177. thus borg can't recognize such "repaired" (but content-damaged) files if the
  178. repair was done with an older borg version.
  179. Version 1.0.5 (2016-07-07)
  180. --------------------------
  181. Bug fixes:
  182. - borg mount: fix FUSE crash in xattr code on Linux introduced in 1.0.4, #1282
  183. Other changes:
  184. - backport some FAQ entries from master branch
  185. - add release helper scripts
  186. - Vagrantfile:
  187. - centos6: no FUSE, don't build binary
  188. - add xz for redhat-like dists
  189. Version 1.0.4 (2016-07-07)
  190. --------------------------
  191. New features:
  192. - borg serve --append-only, #1168
  193. This was included because it was a simple change (append-only functionality
  194. was already present via repository config file) and makes better security now
  195. practically usable.
  196. - BORG_REMOTE_PATH environment variable, #1258
  197. This was included because it was a simple change (--remote-path cli option
  198. was already present) and makes borg much easier to use if you need it.
  199. - Repository: cleanup incomplete transaction on "no space left" condition.
  200. In many cases, this can avoid a 100% full repo filesystem (which is very
  201. problematic as borg always needs free space - even to delete archives).
  202. Bug fixes:
  203. - Fix wrong handling and reporting of OSErrors in borg create, #1138.
  204. This was a serious issue: in the context of "borg create", errors like
  205. repository I/O errors (e.g. disk I/O errors, ssh repo connection errors)
  206. were handled badly and did not lead to a crash (which would be good for this
  207. case, because the repo transaction would be incomplete and trigger a
  208. transaction rollback to clean up).
  209. Now, error handling for source files is cleanly separated from every other
  210. error handling, so only problematic input files are logged and skipped.
  211. - Implement fail-safe error handling for borg extract.
  212. Note that this isn't nearly as critical as the borg create error handling
  213. bug, since nothing is written to the repo. So this was "merely" misleading
  214. error reporting.
  215. - Add missing error handler in directory attr restore loop.
  216. - repo: make sure write data hits disk before the commit tag (#1236) and also
  217. sync the containing directory.
  218. - FUSE: getxattr fail must use errno.ENOATTR, #1126
  219. (fixes Mac OS X Finder malfunction: "zero bytes" file length, access denied)
  220. - borg check --repair: do not lose information about the good/original chunks.
  221. If we do not lose the original chunk IDs list when "repairing" a file
  222. (replacing missing chunks with all-zero chunks), we have a chance to "heal"
  223. the file back into its original state later, in case the chunks re-appear
  224. (e.g. in a fresh backup). Healing is not implemented yet, see #148.
  225. - fixes for --read-special mode:
  226. - ignore known files cache, #1241
  227. - fake regular file mode, #1214
  228. - improve symlinks handling, #1215
  229. - remove passphrase from subprocess environment, #1105
  230. - Ignore empty index file (will trigger index rebuild), #1195
  231. - add missing placeholder support for --prefix, #1027
  232. - improve exception handling for placeholder replacement
  233. - catch and format exceptions in arg parsing
  234. - helpers: fix "undefined name 'e'" in exception handler
  235. - better error handling for missing repo manifest, #1043
  236. - borg delete:
  237. - make it possible to delete a repo without manifest
  238. - borg delete --forced allows to delete corrupted archives, #1139
  239. - borg check:
  240. - make borg check work for empty repo
  241. - fix resync and msgpacked item qualifier, #1135
  242. - rebuild_manifest: fix crash if 'name' or 'time' key were missing.
  243. - better validation of item metadata dicts, #1130
  244. - better validation of archive metadata dicts
  245. - close the repo on exit - even if rollback did not work, #1197.
  246. This is rather cosmetic, it avoids repo closing in the destructor.
  247. - tests:
  248. - fix sparse file test, #1170
  249. - flake8: ignore new F405, #1185
  250. - catch "invalid argument" on cygwin, #257
  251. - fix sparseness assertion in test prep, #1264
  252. Other changes:
  253. - make borg build/work on OpenSSL 1.0 and 1.1, #1187
  254. - docs / help:
  255. - fix / clarify prune help, #1143
  256. - fix "patterns" help formatting
  257. - add missing docs / help about placeholders
  258. - resources: rename atticmatic to borgmatic
  259. - document sshd settings, #545
  260. - more details about checkpoints, add split trick, #1171
  261. - support docs: add freenode web chat link, #1175
  262. - add prune visualization / example, #723
  263. - add note that Fnmatch is default, #1247
  264. - make clear that lzma levels > 6 are a waste of cpu cycles
  265. - add a "do not edit" note to auto-generated files, #1250
  266. - update cygwin installation docs
  267. - repository interoperability with borg master (1.1dev) branch:
  268. - borg check: read item metadata keys from manifest, #1147
  269. - read v2 hints files, #1235
  270. - fix hints file "unknown version" error handling bug
  271. - tests: add tests for format_line
  272. - llfuse: update version requirement for freebsd
  273. - Vagrantfile:
  274. - use openbsd 5.9, #716
  275. - do not install llfuse on netbsd (broken)
  276. - update OSXfuse to version 3.3.3
  277. - use Python 3.5.2 to build the binaries
  278. - glibc compatibility checker: scripts/glibc_check.py
  279. - add .eggs to .gitignore
  280. Version 1.0.3 (2016-05-20)
  281. --------------------------
  282. Bug fixes:
  283. - prune: avoid that checkpoints are kept and completed archives are deleted in
  284. a prune run), #997
  285. - prune: fix commandline argument validation - some valid command lines were
  286. considered invalid (annoying, but harmless), #942
  287. - fix capabilities extraction on Linux (set xattrs last, after chown()), #1069
  288. - repository: fix commit tags being seen in data
  289. - when probing key files, do binary reads. avoids crash when non-borg binary
  290. files are located in borg's key files directory.
  291. - handle SIGTERM and make a clean exit - avoids orphan lock files.
  292. - repository cache: don't cache large objects (avoid using lots of temp. disk
  293. space), #1063
  294. Other changes:
  295. - Vagrantfile: OS X: update osxfuse / install lzma package, #933
  296. - setup.py: add check for platform_darwin.c
  297. - setup.py: on freebsd, use a llfuse release that builds ok
  298. - docs / help:
  299. - update readthedocs URLs, #991
  300. - add missing docs for "borg break-lock", #992
  301. - borg create help: add some words to about the archive name
  302. - borg create help: document format tags, #894
  303. Version 1.0.2 (2016-04-16)
  304. --------------------------
  305. Bug fixes:
  306. - fix malfunction and potential corruption on (nowadays rather rare) big-endian
  307. architectures or bi-endian archs in (rare) BE mode. #886, #889
  308. cache resync / index merge was malfunctioning due to this, potentially
  309. leading to data loss. borg info had cosmetic issues (displayed wrong values).
  310. note: all (widespread) little-endian archs (like x86/x64) or bi-endian archs
  311. in (widespread) LE mode (like ARMEL, MIPSEL, ...) were NOT affected.
  312. - add overflow and range checks for 1st (special) uint32 of the hashindex
  313. values, switch from int32 to uint32.
  314. - fix so that refcount will never overflow, but just stick to max. value after
  315. a overflow would have occured.
  316. - borg delete: fix --cache-only for broken caches, #874
  317. Makes --cache-only idempotent: it won't fail if the cache is already deleted.
  318. - fixed borg create --one-file-system erroneously traversing into other
  319. filesystems (if starting fs device number was 0), #873
  320. - workround a bug in Linux fadvise FADV_DONTNEED, #907
  321. Other changes:
  322. - better test coverage for hashindex, incl. overflow testing, checking correct
  323. computations so endianness issues would be discovered.
  324. - reproducible doc for ProgressIndicator*, make the build reproducible.
  325. - use latest llfuse for vagrant machines
  326. - docs:
  327. - use /path/to/repo in examples, fixes #901
  328. - fix confusing usage of "repo" as archive name (use "arch")
  329. Version 1.0.1 (2016-04-08)
  330. --------------------------
  331. New features:
  332. Usually there are no new features in a bugfix release, but these were added
  333. due to their high impact on security/safety/speed or because they are fixes
  334. also:
  335. - append-only mode for repositories, #809, #36 (see docs)
  336. - borg create: add --ignore-inode option to make borg detect unmodified files
  337. even if your filesystem does not have stable inode numbers (like sshfs and
  338. possibly CIFS).
  339. - add options --warning, --error, --critical for missing log levels, #826.
  340. it's not recommended to suppress warnings or errors, but the user may decide
  341. this on his own.
  342. note: --warning is not given to borg serve so a <= 1.0.0 borg will still
  343. work as server (it is not needed as it is the default).
  344. do not use --error or --critical when using a <= 1.0.0 borg server.
  345. Bug fixes:
  346. - fix silently skipping EIO, #748
  347. - add context manager for Repository (avoid orphan repository locks), #285
  348. - do not sleep for >60s while waiting for lock, #773
  349. - unpack file stats before passing to FUSE
  350. - fix build on illumos
  351. - don't try to backup doors or event ports (Solaris and derivates)
  352. - remove useless/misleading libc version display, #738
  353. - test suite: reset exit code of persistent archiver, #844
  354. - RemoteRepository: clean up pipe if remote open() fails
  355. - Remote: don't print tracebacks for Error exceptions handled downstream, #792
  356. - if BORG_PASSPHRASE is present but wrong, don't prompt for password, but fail
  357. instead, #791
  358. - ArchiveChecker: move "orphaned objects check skipped" to INFO log level, #826
  359. - fix capitalization, add ellipses, change log level to debug for 2 messages,
  360. #798
  361. Other changes:
  362. - update llfuse requirement, llfuse 1.0 works
  363. - update OS / dist packages on build machines, #717
  364. - prefer showing --info over -v in usage help, #859
  365. - docs:
  366. - fix cygwin requirements (gcc-g++)
  367. - document how to debug / file filesystem issues, #664
  368. - fix reproducible build of api docs
  369. - RTD theme: CSS !important overwrite, #727
  370. - Document logo font. Recreate logo png. Remove GIMP logo file.
  371. Version 1.0.0 (2016-03-05)
  372. --------------------------
  373. The major release number change (0.x -> 1.x) indicates bigger incompatible
  374. changes, please read the compatibility notes, adapt / test your scripts and
  375. check your backup logs.
  376. Compatibility notes:
  377. - drop support for python 3.2 and 3.3, require 3.4 or 3.5, #221 #65 #490
  378. note: we provide binaries that include python 3.5.1 and everything else
  379. needed. they are an option in case you are stuck with < 3.4 otherwise.
  380. - change encryption to be on by default (using "repokey" mode)
  381. - moved keyfile keys from ~/.borg/keys to ~/.config/borg/keys,
  382. you can either move them manually or run "borg upgrade <REPO>"
  383. - remove support for --encryption=passphrase,
  384. use borg migrate-to-repokey to switch to repokey mode, #97
  385. - remove deprecated --compression <number>,
  386. use --compression zlib,<number> instead
  387. in case of 0, you could also use --compression none
  388. - remove deprecated --hourly/daily/weekly/monthly/yearly
  389. use --keep-hourly/daily/weekly/monthly/yearly instead
  390. - remove deprecated --do-not-cross-mountpoints,
  391. use --one-file-system instead
  392. - disambiguate -p option, #563:
  393. - -p now is same as --progress
  394. - -P now is same as --prefix
  395. - remove deprecated "borg verify",
  396. use "borg extract --dry-run" instead
  397. - cleanup environment variable semantics, #355
  398. the environment variables used to be "yes sayers" when set, this was
  399. conceptually generalized to "automatic answerers" and they just give their
  400. value as answer (as if you typed in that value when being asked).
  401. See the "usage" / "Environment Variables" section of the docs for details.
  402. - change the builtin default for --chunker-params, create 2MiB chunks, #343
  403. --chunker-params new default: 19,23,21,4095 - old default: 10,23,16,4095
  404. one of the biggest issues with borg < 1.0 (and also attic) was that it had a
  405. default target chunk size of 64kiB, thus it created a lot of chunks and thus
  406. also a huge chunk management overhead (high RAM and disk usage).
  407. please note that the new default won't change the chunks that you already
  408. have in your repository. the new big chunks do not deduplicate with the old
  409. small chunks, so expect your repo to grow at least by the size of every
  410. changed file and in the worst case (e.g. if your files cache was lost / is
  411. not used) by the size of every file (minus any compression you might use).
  412. in case you want to immediately see a much lower resource usage (RAM / disk)
  413. for chunks management, it might be better to start with a new repo than
  414. continuing in the existing repo (with an existing repo, you'ld have to wait
  415. until all archives with small chunks got pruned to see a lower resource
  416. usage).
  417. if you used the old --chunker-params default value (or if you did not use
  418. --chunker-params option at all) and you'ld like to continue using small
  419. chunks (and you accept the huge resource usage that comes with that), just
  420. explicitly use borg create --chunker-params=10,23,16,4095.
  421. - archive timestamps: the 'time' timestamp now refers to archive creation
  422. start time (was: end time), the new 'time_end' timestamp refers to archive
  423. creation end time. This might affect prune if your backups take rather long.
  424. if you give a timestamp via cli this is stored into 'time', therefore it now
  425. needs to mean archive creation start time.
  426. New features:
  427. - implement password roundtrip, #695
  428. Bug fixes:
  429. - remote end does not need cache nor keys directories, do not create them, #701
  430. - added retry counter for passwords, #703
  431. Other changes:
  432. - fix compiler warnings, #697
  433. - docs:
  434. - update README.rst to new changelog location in docs/changes.rst
  435. - add Teemu to AUTHORS
  436. - changes.rst: fix old chunker params, #698
  437. - FAQ: how to limit bandwidth
  438. Version 1.0.0rc2 (2016-02-28)
  439. -----------------------------
  440. New features:
  441. - format options for location: user, pid, fqdn, hostname, now, utcnow, user
  442. - borg list --list-format
  443. - borg prune -v --list enables the keep/prune list output, #658
  444. Bug fixes:
  445. - fix _open_rb noatime handling, #657
  446. - add a simple archivename validator, #680
  447. - borg create --stats: show timestamps in localtime, use same labels/formatting
  448. as borg info, #651
  449. - llfuse compatibility fixes (now compatible with: 0.40, 0.41, 0.42)
  450. Other changes:
  451. - it is now possible to use "pip install borgbackup[fuse]" to automatically
  452. install the llfuse dependency using the correct version requirement
  453. for it. you still need to care about having installed the FUSE / build
  454. related OS package first, though, so that building llfuse can succeed.
  455. - Vagrant: drop Ubuntu Precise (12.04) - does not have Python >= 3.4
  456. - Vagrant: use pyinstaller v3.1.1 to build binaries
  457. - docs:
  458. - borg upgrade: add to docs that only LOCAL repos are supported
  459. - borg upgrade also handles borg 0.xx -> 1.0
  460. - use pip extras or requirements file to install llfuse
  461. - fix order in release process
  462. - updated usage docs and other minor / cosmetic fixes
  463. - verified borg examples in docs, #644
  464. - freebsd dependency installation and fuse configuration, #649
  465. - add example how to restore a raw device, #671
  466. - add a hint about the dev headers needed when installing from source
  467. - add examples for delete (and handle delete after list, before prune), #656
  468. - update example for borg create -v --stats (use iso datetime format), #663
  469. - added example to BORG_RSH docs
  470. - "connection closed by remote": add FAQ entry and point to issue #636
  471. Version 1.0.0rc1 (2016-02-07)
  472. -----------------------------
  473. New features:
  474. - borg migrate-to-repokey ("passphrase" -> "repokey" encryption key mode)
  475. - implement --short for borg list REPO, #611
  476. - implement --list for borg extract (consistency with borg create)
  477. - borg serve: overwrite client's --restrict-to-path with ssh forced command's
  478. option value (but keep everything else from the client commandline), #544
  479. - use $XDG_CONFIG_HOME/keys for keyfile keys (~/.config/borg/keys), #515
  480. - "borg upgrade" moves the keyfile keys to the new location
  481. - display both archive creation start and end time in "borg info", #627
  482. Bug fixes:
  483. - normalize trailing slashes for the repository path, #606
  484. - Cache: fix exception handling in __init__, release lock, #610
  485. Other changes:
  486. - suppress unneeded exception context (PEP 409), simpler tracebacks
  487. - removed special code needed to deal with imperfections / incompatibilities /
  488. missing stuff in py 3.2/3.3, simplify code that can be done simpler in 3.4
  489. - removed some version requirements that were kept on old versions because
  490. newer did not support py 3.2 any more
  491. - use some py 3.4+ stdlib code instead of own/openssl/pypi code:
  492. - use os.urandom instead of own cython openssl RAND_bytes wrapper, #493
  493. - use hashlib.pbkdf2_hmac from py stdlib instead of own openssl wrapper
  494. - use hmac.compare_digest instead of == operator (constant time comparison)
  495. - use stat.filemode instead of homegrown code
  496. - use "mock" library from stdlib, #145
  497. - remove borg.support (with non-broken argparse copy), it is ok in 3.4+, #358
  498. - Vagrant: copy CHANGES.rst as symlink, #592
  499. - cosmetic code cleanups, add flake8 to tox/travis, #4
  500. - docs / help:
  501. - make "borg -h" output prettier, #591
  502. - slightly rephrase prune help
  503. - add missing example for --list option of borg create
  504. - quote exclude line that includes an asterisk to prevent shell expansion
  505. - fix dead link to license
  506. - delete Ubuntu Vivid, it is not supported anymore (EOL)
  507. - OS X binary does not work for older OS X releases, #629
  508. - borg serve's special support for forced/original ssh commands, #544
  509. - misc. updates and fixes
  510. Version 0.30.0 (2016-01-23)
  511. ---------------------------
  512. Compatibility notes:
  513. - you may need to use -v (or --info) more often to actually see output emitted
  514. at INFO log level (because it is suppressed at the default WARNING log level).
  515. See the "general" section in the usage docs.
  516. - for borg create, you need --list (additionally to -v) to see the long file
  517. list (was needed so you can have e.g. --stats alone without the long list)
  518. - see below about BORG_DELETE_I_KNOW_WHAT_I_AM_DOING (was:
  519. BORG_CHECK_I_KNOW_WHAT_I_AM_DOING)
  520. Bug fixes:
  521. - fix crash when using borg create --dry-run --keep-tag-files, #570
  522. - make sure teardown with cleanup happens for Cache and RepositoryCache,
  523. avoiding leftover locks and TEMP dir contents, #285 (partially), #548
  524. - fix locking KeyError, partial fix for #502
  525. - log stats consistently, #526
  526. - add abbreviated weekday to timestamp format, fixes #496
  527. - strip whitespace when loading exclusions from file
  528. - unset LD_LIBRARY_PATH before invoking ssh, fixes strange OpenSSL library
  529. version warning when using the borg binary, #514
  530. - add some error handling/fallback for C library loading, #494
  531. - added BORG_DELETE_I_KNOW_WHAT_I_AM_DOING for check in "borg delete", #503
  532. - remove unused "repair" rpc method name
  533. New features:
  534. - borg create: implement exclusions using regular expression patterns.
  535. - borg create: implement inclusions using patterns.
  536. - borg extract: support patterns, #361
  537. - support different styles for patterns:
  538. - fnmatch (`fm:` prefix, default when omitted), like borg <= 0.29.
  539. - shell (`sh:` prefix) with `*` not matching directory separators and
  540. `**/` matching 0..n directories
  541. - path prefix (`pp:` prefix, for unifying borg create pp1 pp2 into the
  542. patterns system), semantics like in borg <= 0.29
  543. - regular expression (`re:`), new!
  544. - --progress option for borg upgrade (#291) and borg delete <archive>
  545. - update progress indication more often (e.g. for borg create within big
  546. files or for borg check repo), #500
  547. - finer chunker granularity for items metadata stream, #547, #487
  548. - borg create --list now used (additionally to -v) to enable the verbose
  549. file list output
  550. - display borg version below tracebacks, #532
  551. Other changes:
  552. - hashtable size (and thus: RAM and disk consumption) follows a growth policy:
  553. grows fast while small, grows slower when getting bigger, #527
  554. - Vagrantfile: use pyinstaller 3.1 to build binaries, freebsd sqlite3 fix,
  555. fixes #569
  556. - no separate binaries for centos6 any more because the generic linux binaries
  557. also work on centos6 (or in general: on systems with a slightly older glibc
  558. than debian7
  559. - dev environment: require virtualenv<14.0 so we get a py32 compatible pip
  560. - docs:
  561. - add space-saving chunks.archive.d trick to FAQ
  562. - important: clarify -v and log levels in usage -> general, please read!
  563. - sphinx configuration: create a simple man page from usage docs
  564. - add a repo server setup example
  565. - disable unneeded SSH features in authorized_keys examples for security.
  566. - borg prune only knows "--keep-within" and not "--within"
  567. - add gource video to resources docs, #507
  568. - add netbsd install instructions
  569. - authors: make it more clear what refers to borg and what to attic
  570. - document standalone binary requirements, #499
  571. - rephrase the mailing list section
  572. - development docs: run build_api and build_usage before tagging release
  573. - internals docs: hash table max. load factor is 0.75 now
  574. - markup, typo, grammar, phrasing, clarifications and other fixes.
  575. - add gcc gcc-c++ to redhat/fedora/corora install docs, fixes #583
  576. Version 0.29.0 (2015-12-13)
  577. ---------------------------
  578. Compatibility notes:
  579. - when upgrading to 0.29.0 you need to upgrade client as well as server
  580. installations due to the locking and commandline interface changes otherwise
  581. you'll get an error msg about a RPC protocol mismatch or a wrong commandline
  582. option.
  583. if you run a server that needs to support both old and new clients, it is
  584. suggested that you have a "borg-0.28.2" and a "borg-0.29.0" command.
  585. clients then can choose via e.g. "borg --remote-path=borg-0.29.0 ...".
  586. - the default waiting time for a lock changed from infinity to 1 second for a
  587. better interactive user experience. if the repo you want to access is
  588. currently locked, borg will now terminate after 1s with an error message.
  589. if you have scripts that shall wait for the lock for a longer time, use
  590. --lock-wait N (with N being the maximum wait time in seconds).
  591. Bug fixes:
  592. - hash table tuning (better chosen hashtable load factor 0.75 and prime initial
  593. size of 1031 gave ~1000x speedup in some scenarios)
  594. - avoid creation of an orphan lock for one case, #285
  595. - --keep-tag-files: fix file mode and multiple tag files in one directory, #432
  596. - fixes for "borg upgrade" (attic repo converter), #466
  597. - remove --progress isatty magic (and also --no-progress option) again, #476
  598. - borg init: display proper repo URL
  599. - fix format of umask in help pages, #463
  600. New features:
  601. - implement --lock-wait, support timeout for UpgradableLock, #210
  602. - implement borg break-lock command, #157
  603. - include system info below traceback, #324
  604. - sane remote logging, remote stderr, #461:
  605. - remote log output: intercept it and log it via local logging system,
  606. with "Remote: " prefixed to message. log remote tracebacks.
  607. - remote stderr: output it to local stderr with "Remote: " prefixed.
  608. - add --debug and --info (same as --verbose) to set the log level of the
  609. builtin logging configuration (which otherwise defaults to warning), #426
  610. note: there are few messages emitted at DEBUG level currently.
  611. - optionally configure logging via env var BORG_LOGGING_CONF
  612. - add --filter option for status characters: e.g. to show only the added
  613. or modified files (and also errors), use "borg create -v --filter=AME ...".
  614. - more progress indicators, #394
  615. - use ISO-8601 date and time format, #375
  616. - "borg check --prefix" to restrict archive checking to that name prefix, #206
  617. Other changes:
  618. - hashindex_add C implementation (speed up cache re-sync for new archives)
  619. - increase FUSE read_size to 1024 (speed up metadata operations)
  620. - check/delete/prune --save-space: free unused segments quickly, #239
  621. - increase rpc protocol version to 2 (see also Compatibility notes), #458
  622. - silence borg by default (via default log level WARNING)
  623. - get rid of C compiler warnings, #391
  624. - upgrade OS X FUSE to 3.0.9 on the OS X binary build system
  625. - use python 3.5.1 to build binaries
  626. - docs:
  627. - new mailing list borgbackup@python.org, #468
  628. - readthedocs: color and logo improvements
  629. - load coverage icons over SSL (avoids mixed content)
  630. - more precise binary installation steps
  631. - update release procedure docs about OS X FUSE
  632. - FAQ entry about unexpected 'A' status for unchanged file(s), #403
  633. - add docs about 'E' file status
  634. - add "borg upgrade" docs, #464
  635. - add developer docs about output and logging
  636. - clarify encryption, add note about client-side encryption
  637. - add resources section, with videos, talks, presentations, #149
  638. - Borg moved to Arch Linux [community]
  639. - fix wrong installation instructions for archlinux
  640. Version 0.28.2 (2015-11-15)
  641. ---------------------------
  642. New features:
  643. - borg create --exclude-if-present TAGFILE - exclude directories that have the
  644. given file from the backup. You can additionally give --keep-tag-files to
  645. preserve just the directory roots and the tag-files (but not backup other
  646. directory contents), #395, attic #128, attic #142
  647. Other changes:
  648. - do not create docs sources at build time (just have them in the repo),
  649. completely remove have_cython() hack, do not use the "mock" library at build
  650. time, #384
  651. - avoid hidden import, make it easier for PyInstaller, easier fix for #218
  652. - docs:
  653. - add description of item flags / status output, fixes #402
  654. - explain how to regenerate usage and API files (build_api or
  655. build_usage) and when to commit usage files directly into git, #384
  656. - minor install docs improvements
  657. Version 0.28.1 (2015-11-08)
  658. ---------------------------
  659. Bug fixes:
  660. - do not try to build api / usage docs for production install,
  661. fixes unexpected "mock" build dependency, #384
  662. Other changes:
  663. - avoid using msgpack.packb at import time
  664. - fix formatting issue in changes.rst
  665. - fix build on readthedocs
  666. Version 0.28.0 (2015-11-08)
  667. ---------------------------
  668. Compatibility notes:
  669. - changed return codes (exit codes), see docs. in short:
  670. old: 0 = ok, 1 = error. now: 0 = ok, 1 = warning, 2 = error
  671. New features:
  672. - refactor return codes (exit codes), fixes #61
  673. - add --show-rc option enable "terminating with X status, rc N" output, fixes 58, #351
  674. - borg create backups atime and ctime additionally to mtime, fixes #317
  675. - extract: support atime additionally to mtime
  676. - FUSE: support ctime and atime additionally to mtime
  677. - support borg --version
  678. - emit a warning if we have a slow msgpack installed
  679. - borg list --prefix=thishostname- REPO, fixes #205
  680. - Debug commands (do not use except if you know what you do: debug-get-obj,
  681. debug-put-obj, debug-delete-obj, debug-dump-archive-items.
  682. Bug fixes:
  683. - setup.py: fix bug related to BORG_LZ4_PREFIX processing
  684. - fix "check" for repos that have incomplete chunks, fixes #364
  685. - borg mount: fix unlocking of repository at umount time, fixes #331
  686. - fix reading files without touching their atime, #334
  687. - non-ascii ACL fixes for Linux, FreeBSD and OS X, #277
  688. - fix acl_use_local_uid_gid() and add a test for it, attic #359
  689. - borg upgrade: do not upgrade repositories in place by default, #299
  690. - fix cascading failure with the index conversion code, #269
  691. - borg check: implement 'cmdline' archive metadata value decoding, #311
  692. - fix RobustUnpacker, it missed some metadata keys (new atime and ctime keys
  693. were missing, but also bsdflags). add check for unknown metadata keys.
  694. - create from stdin: also save atime, ctime (cosmetic)
  695. - use default_notty=False for confirmations, fixes #345
  696. - vagrant: fix msgpack installation on centos, fixes #342
  697. - deal with unicode errors for symlinks in same way as for regular files and
  698. have a helpful warning message about how to fix wrong locale setup, fixes #382
  699. - add ACL keys the RobustUnpacker must know about
  700. Other changes:
  701. - improve file size displays, more flexible size formatters
  702. - explicitly commit to the units standard, #289
  703. - archiver: add E status (means that an error occurred when processing this
  704. (single) item
  705. - do binary releases via "github releases", closes #214
  706. - create: use -x and --one-file-system (was: --do-not-cross-mountpoints), #296
  707. - a lot of changes related to using "logging" module and screen output, #233
  708. - show progress display if on a tty, output more progress information, #303
  709. - factor out status output so it is consistent, fix surrogates removal,
  710. maybe fixes #309
  711. - move away from RawConfigParser to ConfigParser
  712. - archive checker: better error logging, give chunk_id and sequence numbers
  713. (can be used together with borg debug-dump-archive-items).
  714. - do not mention the deprecated passphrase mode
  715. - emit a deprecation warning for --compression N (giving a just a number)
  716. - misc .coverragerc fixes (and coverage measurement improvements), fixes #319
  717. - refactor confirmation code, reduce code duplication, add tests
  718. - prettier error messages, fixes #307, #57
  719. - tests:
  720. - add a test to find disk-full issues, #327
  721. - travis: also run tests on Python 3.5
  722. - travis: use tox -r so it rebuilds the tox environments
  723. - test the generated pyinstaller-based binary by archiver unit tests, #215
  724. - vagrant: tests: announce whether fakeroot is used or not
  725. - vagrant: add vagrant user to fuse group for debianoid systems also
  726. - vagrant: llfuse install on darwin needs pkgconfig installed
  727. - vagrant: use pyinstaller from develop branch, fixes #336
  728. - benchmarks: test create, extract, list, delete, info, check, help, fixes #146
  729. - benchmarks: test with both the binary and the python code
  730. - archiver tests: test with both the binary and the python code, fixes #215
  731. - make basic test more robust
  732. - docs:
  733. - moved docs to borgbackup.readthedocs.org, #155
  734. - a lot of fixes and improvements, use mobile-friendly RTD standard theme
  735. - use zlib,6 compression in some examples, fixes #275
  736. - add missing rename usage to docs, closes #279
  737. - include the help offered by borg help <topic> in the usage docs, fixes #293
  738. - include a list of major changes compared to attic into README, fixes #224
  739. - add OS X install instructions, #197
  740. - more details about the release process, #260
  741. - fix linux glibc requirement (binaries built on debian7 now)
  742. - build: move usage and API generation to setup.py
  743. - update docs about return codes, #61
  744. - remove api docs (too much breakage on rtd)
  745. - borgbackup install + basics presentation (asciinema)
  746. - describe the current style guide in documentation
  747. - add section about debug commands
  748. - warn about not running out of space
  749. - add example for rename
  750. - improve chunker params docs, fixes #362
  751. - minor development docs update
  752. Version 0.27.0 (2015-10-07)
  753. ---------------------------
  754. New features:
  755. - "borg upgrade" command - attic -> borg one time converter / migration, #21
  756. - temporary hack to avoid using lots of disk space for chunks.archive.d, #235:
  757. To use it: rm -rf chunks.archive.d ; touch chunks.archive.d
  758. - respect XDG_CACHE_HOME, attic #181
  759. - add support for arbitrary SSH commands, attic #99
  760. - borg delete --cache-only REPO (only delete cache, not REPO), attic #123
  761. Bug fixes:
  762. - use Debian 7 (wheezy) to build pyinstaller borgbackup binaries, fixes slow
  763. down observed when running the Centos6-built binary on Ubuntu, #222
  764. - do not crash on empty lock.roster, fixes #232
  765. - fix multiple issues with the cache config version check, #234
  766. - fix segment entry header size check, attic #352
  767. plus other error handling improvements / code deduplication there.
  768. - always give segment and offset in repo IntegrityErrors
  769. Other changes:
  770. - stop producing binary wheels, remove docs about it, #147
  771. - docs:
  772. - add warning about prune
  773. - generate usage include files only as needed
  774. - development docs: add Vagrant section
  775. - update / improve / reformat FAQ
  776. - hint to single-file pyinstaller binaries from README
  777. Version 0.26.1 (2015-09-28)
  778. ---------------------------
  779. This is a minor update, just docs and new pyinstaller binaries.
  780. - docs update about python and binary requirements
  781. - better docs for --read-special, fix #220
  782. - re-built the binaries, fix #218 and #213 (glibc version issue)
  783. - update web site about single-file pyinstaller binaries
  784. Note: if you did a python-based installation, there is no need to upgrade.
  785. Version 0.26.0 (2015-09-19)
  786. ---------------------------
  787. New features:
  788. - Faster cache sync (do all in one pass, remove tar/compression stuff), #163
  789. - BORG_REPO env var to specify the default repo, #168
  790. - read special files as if they were regular files, #79
  791. - implement borg create --dry-run, attic issue #267
  792. - Normalize paths before pattern matching on OS X, #143
  793. - support OpenBSD and NetBSD (except xattrs/ACLs)
  794. - support / run tests on Python 3.5
  795. Bug fixes:
  796. - borg mount repo: use absolute path, attic #200, attic #137
  797. - chunker: use off_t to get 64bit on 32bit platform, #178
  798. - initialize chunker fd to -1, so it's not equal to STDIN_FILENO (0)
  799. - fix reaction to "no" answer at delete repo prompt, #182
  800. - setup.py: detect lz4.h header file location
  801. - to support python < 3.2.4, add less buggy argparse lib from 3.2.6 (#194)
  802. - fix for obtaining ``char *`` from temporary Python value (old code causes
  803. a compile error on Mint 17.2)
  804. - llfuse 0.41 install troubles on some platforms, require < 0.41
  805. (UnicodeDecodeError exception due to non-ascii llfuse setup.py)
  806. - cython code: add some int types to get rid of unspecific python add /
  807. subtract operations (avoid ``undefined symbol FPE_``... error on some platforms)
  808. - fix verbose mode display of stdin backup
  809. - extract: warn if a include pattern never matched, fixes #209,
  810. implement counters for Include/ExcludePatterns
  811. - archive names with slashes are invalid, attic issue #180
  812. - chunker: add a check whether the POSIX_FADV_DONTNEED constant is defined -
  813. fixes building on OpenBSD.
  814. Other changes:
  815. - detect inconsistency / corruption / hash collision, #170
  816. - replace versioneer with setuptools_scm, #106
  817. - docs:
  818. - pkg-config is needed for llfuse installation
  819. - be more clear about pruning, attic issue #132
  820. - unit tests:
  821. - xattr: ignore security.selinux attribute showing up
  822. - ext3 seems to need a bit more space for a sparse file
  823. - do not test lzma level 9 compression (avoid MemoryError)
  824. - work around strange mtime granularity issue on netbsd, fixes #204
  825. - ignore st_rdev if file is not a block/char device, fixes #203
  826. - stay away from the setgid and sticky mode bits
  827. - use Vagrant to do easy cross-platform testing (#196), currently:
  828. - Debian 7 "wheezy" 32bit, Debian 8 "jessie" 64bit
  829. - Ubuntu 12.04 32bit, Ubuntu 14.04 64bit
  830. - Centos 7 64bit
  831. - FreeBSD 10.2 64bit
  832. - OpenBSD 5.7 64bit
  833. - NetBSD 6.1.5 64bit
  834. - Darwin (OS X Yosemite)
  835. Version 0.25.0 (2015-08-29)
  836. ---------------------------
  837. Compatibility notes:
  838. - lz4 compression library (liblz4) is a new requirement (#156)
  839. - the new compression code is very compatible: as long as you stay with zlib
  840. compression, older borg releases will still be able to read data from a
  841. repo/archive made with the new code (note: this is not the case for the
  842. default "none" compression, use "zlib,0" if you want a "no compression" mode
  843. that can be read by older borg). Also the new code is able to read repos and
  844. archives made with older borg versions (for all zlib levels 0..9).
  845. Deprecations:
  846. - --compression N (with N being a number, as in 0.24) is deprecated.
  847. We keep the --compression 0..9 for now to not break scripts, but it is
  848. deprecated and will be removed later, so better fix your scripts now:
  849. --compression 0 (as in 0.24) is the same as --compression zlib,0 (now).
  850. BUT: if you do not want compression, you rather want --compression none
  851. (which is the default).
  852. --compression 1 (in 0.24) is the same as --compression zlib,1 (now)
  853. --compression 9 (in 0.24) is the same as --compression zlib,9 (now)
  854. New features:
  855. - create --compression none (default, means: do not compress, just pass through
  856. data "as is". this is more efficient than zlib level 0 as used in borg 0.24)
  857. - create --compression lz4 (super-fast, but not very high compression)
  858. - create --compression zlib,N (slower, higher compression, default for N is 6)
  859. - create --compression lzma,N (slowest, highest compression, default N is 6)
  860. - honor the nodump flag (UF_NODUMP) and do not backup such items
  861. - list --short just outputs a simple list of the files/directories in an archive
  862. Bug fixes:
  863. - fixed --chunker-params parameter order confusion / malfunction, fixes #154
  864. - close fds of segments we delete (during compaction)
  865. - close files which fell out the lrucache
  866. - fadvise DONTNEED now is only called for the byte range actually read, not for
  867. the whole file, fixes #158.
  868. - fix issue with negative "all archives" size, fixes #165
  869. - restore_xattrs: ignore if setxattr fails with EACCES, fixes #162
  870. Other changes:
  871. - remove fakeroot requirement for tests, tests run faster without fakeroot
  872. (test setup does not fail any more without fakeroot, so you can run with or
  873. without fakeroot), fixes #151 and #91.
  874. - more tests for archiver
  875. - recover_segment(): don't assume we have an fd for segment
  876. - lrucache refactoring / cleanup, add dispose function, py.test tests
  877. - generalize hashindex code for any key length (less hardcoding)
  878. - lock roster: catch file not found in remove() method and ignore it
  879. - travis CI: use requirements file
  880. - improved docs:
  881. - replace hack for llfuse with proper solution (install libfuse-dev)
  882. - update docs about compression
  883. - update development docs about fakeroot
  884. - internals: add some words about lock files / locking system
  885. - support: mention BountySource and for what it can be used
  886. - theme: use a lighter green
  887. - add pypi, wheel, dist package based install docs
  888. - split install docs into system-specific preparations and generic instructions
  889. Version 0.24.0 (2015-08-09)
  890. ---------------------------
  891. Incompatible changes (compared to 0.23):
  892. - borg now always issues --umask NNN option when invoking another borg via ssh
  893. on the repository server. By that, it's making sure it uses the same umask
  894. for remote repos as for local ones. Because of this, you must upgrade both
  895. server and client(s) to 0.24.
  896. - the default umask is 077 now (if you do not specify via --umask) which might
  897. be a different one as you used previously. The default umask avoids that
  898. you accidentally give access permissions for group and/or others to files
  899. created by borg (e.g. the repository).
  900. Deprecations:
  901. - "--encryption passphrase" mode is deprecated, see #85 and #97.
  902. See the new "--encryption repokey" mode for a replacement.
  903. New features:
  904. - borg create --chunker-params ... to configure the chunker, fixes #16
  905. (attic #302, attic #300, and somehow also #41).
  906. This can be used to reduce memory usage caused by chunk management overhead,
  907. so borg does not create a huge chunks index/repo index and eats all your RAM
  908. if you back up lots of data in huge files (like VM disk images).
  909. See docs/misc/create_chunker-params.txt for more information.
  910. - borg info now reports chunk counts in the chunk index.
  911. - borg create --compression 0..9 to select zlib compression level, fixes #66
  912. (attic #295).
  913. - borg init --encryption repokey (to store the encryption key into the repo),
  914. fixes #85
  915. - improve at-end error logging, always log exceptions and set exit_code=1
  916. - LoggedIO: better error checks / exceptions / exception handling
  917. - implement --remote-path to allow non-default-path borg locations, #125
  918. - implement --umask M and use 077 as default umask for better security, #117
  919. - borg check: give a named single archive to it, fixes #139
  920. - cache sync: show progress indication
  921. - cache sync: reimplement the chunk index merging in C
  922. Bug fixes:
  923. - fix segfault that happened for unreadable files (chunker: n needs to be a
  924. signed size_t), #116
  925. - fix the repair mode, #144
  926. - repo delete: add destroy to allowed rpc methods, fixes issue #114
  927. - more compatible repository locking code (based on mkdir), maybe fixes #92
  928. (attic #317, attic #201).
  929. - better Exception msg if no Borg is installed on the remote repo server, #56
  930. - create a RepositoryCache implementation that can cope with >2GiB,
  931. fixes attic #326.
  932. - fix Traceback when running check --repair, attic #232
  933. - clarify help text, fixes #73.
  934. - add help string for --no-files-cache, fixes #140
  935. Other changes:
  936. - improved docs:
  937. - added docs/misc directory for misc. writeups that won't be included
  938. "as is" into the html docs.
  939. - document environment variables and return codes (attic #324, attic #52)
  940. - web site: add related projects, fix web site url, IRC #borgbackup
  941. - Fedora/Fedora-based install instructions added to docs
  942. - Cygwin-based install instructions added to docs
  943. - updated AUTHORS
  944. - add FAQ entries about redundancy / integrity
  945. - clarify that borg extract uses the cwd as extraction target
  946. - update internals doc about chunker params, memory usage and compression
  947. - added docs about development
  948. - add some words about resource usage in general
  949. - document how to backup a raw disk
  950. - add note about how to run borg from virtual env
  951. - add solutions for (ll)fuse installation problems
  952. - document what borg check does, fixes #138
  953. - reorganize borgbackup.github.io sidebar, prev/next at top
  954. - deduplicate and refactor the docs / README.rst
  955. - use borg-tmp as prefix for temporary files / directories
  956. - short prune options without "keep-" are deprecated, do not suggest them
  957. - improved tox configuration
  958. - remove usage of unittest.mock, always use mock from pypi
  959. - use entrypoints instead of scripts, for better use of the wheel format and
  960. modern installs
  961. - add requirements.d/development.txt and modify tox.ini
  962. - use travis-ci for testing based on Linux and (new) OS X
  963. - use coverage.py, pytest-cov and codecov.io for test coverage support
  964. I forgot to list some stuff already implemented in 0.23.0, here they are:
  965. New features:
  966. - efficient archive list from manifest, meaning a big speedup for slow
  967. repo connections and "list <repo>", "delete <repo>", "prune" (attic #242,
  968. attic #167)
  969. - big speedup for chunks cache sync (esp. for slow repo connections), fixes #18
  970. - hashindex: improve error messages
  971. Other changes:
  972. - explicitly specify binary mode to open binary files
  973. - some easy micro optimizations
  974. Version 0.23.0 (2015-06-11)
  975. ---------------------------
  976. Incompatible changes (compared to attic, fork related):
  977. - changed sw name and cli command to "borg", updated docs
  978. - package name (and name in urls) uses "borgbackup" to have less collisions
  979. - changed repo / cache internal magic strings from ATTIC* to BORG*,
  980. changed cache location to .cache/borg/ - this means that it currently won't
  981. accept attic repos (see issue #21 about improving that)
  982. Bug fixes:
  983. - avoid defect python-msgpack releases, fixes attic #171, fixes attic #185
  984. - fix traceback when trying to do unsupported passphrase change, fixes attic #189
  985. - datetime does not like the year 10.000, fixes attic #139
  986. - fix "info" all archives stats, fixes attic #183
  987. - fix parsing with missing microseconds, fixes attic #282
  988. - fix misleading hint the fuse ImportError handler gave, fixes attic #237
  989. - check unpacked data from RPC for tuple type and correct length, fixes attic #127
  990. - fix Repository._active_txn state when lock upgrade fails
  991. - give specific path to xattr.is_enabled(), disable symlink setattr call that
  992. always fails
  993. - fix test setup for 32bit platforms, partial fix for attic #196
  994. - upgraded versioneer, PEP440 compliance, fixes attic #257
  995. New features:
  996. - less memory usage: add global option --no-cache-files
  997. - check --last N (only check the last N archives)
  998. - check: sort archives in reverse time order
  999. - rename repo::oldname newname (rename repository)
  1000. - create -v output more informative
  1001. - create --progress (backup progress indicator)
  1002. - create --timestamp (utc string or reference file/dir)
  1003. - create: if "-" is given as path, read binary from stdin
  1004. - extract: if --stdout is given, write all extracted binary data to stdout
  1005. - extract --sparse (simple sparse file support)
  1006. - extra debug information for 'fread failed'
  1007. - delete <repo> (deletes whole repo + local cache)
  1008. - FUSE: reflect deduplication in allocated blocks
  1009. - only allow whitelisted RPC calls in server mode
  1010. - normalize source/exclude paths before matching
  1011. - use posix_fadvise to not spoil the OS cache, fixes attic #252
  1012. - toplevel error handler: show tracebacks for better error analysis
  1013. - sigusr1 / sigint handler to print current file infos - attic PR #286
  1014. - RPCError: include the exception args we get from remote
  1015. Other changes:
  1016. - source: misc. cleanups, pep8, style
  1017. - docs and faq improvements, fixes, updates
  1018. - cleanup crypto.pyx, make it easier to adapt to other AES modes
  1019. - do os.fsync like recommended in the python docs
  1020. - source: Let chunker optionally work with os-level file descriptor.
  1021. - source: Linux: remove duplicate os.fsencode calls
  1022. - source: refactor _open_rb code a bit, so it is more consistent / regular
  1023. - source: refactor indicator (status) and item processing
  1024. - source: use py.test for better testing, flake8 for code style checks
  1025. - source: fix tox >=2.0 compatibility (test runner)
  1026. - pypi package: add python version classifiers, add FreeBSD to platforms
  1027. Attic Changelog
  1028. ---------------
  1029. Here you can see the full list of changes between each Attic release until Borg
  1030. forked from Attic:
  1031. Version 0.17
  1032. ~~~~~~~~~~~~
  1033. (bugfix release, released on X)
  1034. - Fix hashindex ARM memory alignment issue (#309)
  1035. - Improve hashindex error messages (#298)
  1036. Version 0.16
  1037. ~~~~~~~~~~~~
  1038. (bugfix release, released on May 16, 2015)
  1039. - Fix typo preventing the security confirmation prompt from working (#303)
  1040. - Improve handling of systems with improperly configured file system encoding (#289)
  1041. - Fix "All archives" output for attic info. (#183)
  1042. - More user friendly error message when repository key file is not found (#236)
  1043. - Fix parsing of iso 8601 timestamps with zero microseconds (#282)
  1044. Version 0.15
  1045. ~~~~~~~~~~~~
  1046. (bugfix release, released on Apr 15, 2015)
  1047. - xattr: Be less strict about unknown/unsupported platforms (#239)
  1048. - Reduce repository listing memory usage (#163).
  1049. - Fix BrokenPipeError for remote repositories (#233)
  1050. - Fix incorrect behavior with two character directory names (#265, #268)
  1051. - Require approval before accessing relocated/moved repository (#271)
  1052. - Require approval before accessing previously unknown unencrypted repositories (#271)
  1053. - Fix issue with hash index files larger than 2GB.
  1054. - Fix Python 3.2 compatibility issue with noatime open() (#164)
  1055. - Include missing pyx files in dist files (#168)
  1056. Version 0.14
  1057. ~~~~~~~~~~~~
  1058. (feature release, released on Dec 17, 2014)
  1059. - Added support for stripping leading path segments (#95)
  1060. "attic extract --strip-segments X"
  1061. - Add workaround for old Linux systems without acl_extended_file_no_follow (#96)
  1062. - Add MacPorts' path to the default openssl search path (#101)
  1063. - HashIndex improvements, eliminates unnecessary IO on low memory systems.
  1064. - Fix "Number of files" output for attic info. (#124)
  1065. - limit create file permissions so files aren't read while restoring
  1066. - Fix issue with empty xattr values (#106)
  1067. Version 0.13
  1068. ~~~~~~~~~~~~
  1069. (feature release, released on Jun 29, 2014)
  1070. - Fix sporadic "Resource temporarily unavailable" when using remote repositories
  1071. - Reduce file cache memory usage (#90)
  1072. - Faster AES encryption (utilizing AES-NI when available)
  1073. - Experimental Linux, OS X and FreeBSD ACL support (#66)
  1074. - Added support for backup and restore of BSDFlags (OSX, FreeBSD) (#56)
  1075. - Fix bug where xattrs on symlinks were not correctly restored
  1076. - Added cachedir support. CACHEDIR.TAG compatible cache directories
  1077. can now be excluded using ``--exclude-caches`` (#74)
  1078. - Fix crash on extreme mtime timestamps (year 2400+) (#81)
  1079. - Fix Python 3.2 specific lockf issue (EDEADLK)
  1080. Version 0.12
  1081. ~~~~~~~~~~~~
  1082. (feature release, released on April 7, 2014)
  1083. - Python 3.4 support (#62)
  1084. - Various documentation improvements a new style
  1085. - ``attic mount`` now supports mounting an entire repository not only
  1086. individual archives (#59)
  1087. - Added option to restrict remote repository access to specific path(s):
  1088. ``attic serve --restrict-to-path X`` (#51)
  1089. - Include "all archives" size information in "--stats" output. (#54)
  1090. - Added ``--stats`` option to ``attic delete`` and ``attic prune``
  1091. - Fixed bug where ``attic prune`` used UTC instead of the local time zone
  1092. when determining which archives to keep.
  1093. - Switch to SI units (Power of 1000 instead 1024) when printing file sizes
  1094. Version 0.11
  1095. ~~~~~~~~~~~~
  1096. (feature release, released on March 7, 2014)
  1097. - New "check" command for repository consistency checking (#24)
  1098. - Documentation improvements
  1099. - Fix exception during "attic create" with repeated files (#39)
  1100. - New "--exclude-from" option for attic create/extract/verify.
  1101. - Improved archive metadata deduplication.
  1102. - "attic verify" has been deprecated. Use "attic extract --dry-run" instead.
  1103. - "attic prune --hourly|daily|..." has been deprecated.
  1104. Use "attic prune --keep-hourly|daily|..." instead.
  1105. - Ignore xattr errors during "extract" if not supported by the filesystem. (#46)
  1106. Version 0.10
  1107. ~~~~~~~~~~~~
  1108. (bugfix release, released on Jan 30, 2014)
  1109. - Fix deadlock when extracting 0 sized files from remote repositories
  1110. - "--exclude" wildcard patterns are now properly applied to the full path
  1111. not just the file name part (#5).
  1112. - Make source code endianness agnostic (#1)
  1113. Version 0.9
  1114. ~~~~~~~~~~~
  1115. (feature release, released on Jan 23, 2014)
  1116. - Remote repository speed and reliability improvements.
  1117. - Fix sorting of segment names to ignore NFS left over files. (#17)
  1118. - Fix incorrect display of time (#13)
  1119. - Improved error handling / reporting. (#12)
  1120. - Use fcntl() instead of flock() when locking repository/cache. (#15)
  1121. - Let ssh figure out port/user if not specified so we don't override .ssh/config (#9)
  1122. - Improved libcrypto path detection (#23).
  1123. Version 0.8.1
  1124. ~~~~~~~~~~~~~
  1125. (bugfix release, released on Oct 4, 2013)
  1126. - Fix segmentation fault issue.
  1127. Version 0.8
  1128. ~~~~~~~~~~~
  1129. (feature release, released on Oct 3, 2013)
  1130. - Fix xattr issue when backing up sshfs filesystems (#4)
  1131. - Fix issue with excessive index file size (#6)
  1132. - Support access of read only repositories.
  1133. - New syntax to enable repository encryption:
  1134. attic init --encryption="none|passphrase|keyfile".
  1135. - Detect and abort if repository is older than the cache.
  1136. Version 0.7
  1137. ~~~~~~~~~~~
  1138. (feature release, released on Aug 5, 2013)
  1139. - Ported to FreeBSD
  1140. - Improved documentation
  1141. - Experimental: Archives mountable as fuse filesystems.
  1142. - The "user." prefix is no longer stripped from xattrs on Linux
  1143. Version 0.6.1
  1144. ~~~~~~~~~~~~~
  1145. (bugfix release, released on July 19, 2013)
  1146. - Fixed an issue where mtime was not always correctly restored.
  1147. Version 0.6
  1148. ~~~~~~~~~~~
  1149. First public release on July 9, 2013