usage.rst 26 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571572573574575576577578579580581582583584585586587588589590591592593594595596597598599600601602603604605606607608609610611612613614615616617618619620621622623624625626627628629630631632633634635636637638639640641642643644645646647648649650651652653654655656657658659660661662663664665666667668669670671672673674675676677678679680681682683684685686687688689690691692693694695696697698699700701702703704705706707708709710711712713714715716717718719720721722723724725726727728729730
  1. .. include:: global.rst.inc
  2. .. highlight:: none
  3. .. _detailed_usage:
  4. Usage
  5. =====
  6. |project_name| consists of a number of commands. Each command accepts
  7. a number of arguments and options. The following sections will describe each
  8. command in detail.
  9. General
  10. -------
  11. .. include:: usage_general.rst.inc
  12. In case you are interested in more details (like formulas), please see
  13. :ref:`internals`.
  14. Common options
  15. ++++++++++++++
  16. All |project_name| commands share these options:
  17. .. include:: usage/common-options.rst.inc
  18. .. include:: usage/init.rst.inc
  19. Examples
  20. ~~~~~~~~
  21. ::
  22. # Local repository, repokey encryption, BLAKE2b (often faster, since Borg 1.1)
  23. $ borg init --encryption=repokey-blake2 /path/to/repo
  24. # Local repository (no encryption)
  25. $ borg init --encryption=none /path/to/repo
  26. # Remote repository (accesses a remote borg via ssh)
  27. $ borg init --encryption=repokey-blake2 user@hostname:backup
  28. # Remote repository (store the key your home dir)
  29. $ borg init --encryption=keyfile user@hostname:backup
  30. .. include:: usage/create.rst.inc
  31. Examples
  32. ~~~~~~~~
  33. ::
  34. # Backup ~/Documents into an archive named "my-documents"
  35. $ borg create /path/to/repo::my-documents ~/Documents
  36. # same, but list all files as we process them
  37. $ borg create --list /path/to/repo::my-documents ~/Documents
  38. # Backup ~/Documents and ~/src but exclude pyc files
  39. $ borg create /path/to/repo::my-files \
  40. ~/Documents \
  41. ~/src \
  42. --exclude '*.pyc'
  43. # Backup home directories excluding image thumbnails (i.e. only
  44. # /home/*/.thumbnails is excluded, not /home/*/*/.thumbnails)
  45. $ borg create /path/to/repo::my-files /home \
  46. --exclude 're:^/home/[^/]+/\.thumbnails/'
  47. # Do the same using a shell-style pattern
  48. $ borg create /path/to/repo::my-files /home \
  49. --exclude 'sh:/home/*/.thumbnails'
  50. # Backup the root filesystem into an archive named "root-YYYY-MM-DD"
  51. # use zlib compression (good, but slow) - default is lz4 (fast, low compression ratio)
  52. $ borg create -C zlib,6 /path/to/repo::root-{now:%Y-%m-%d} / --one-file-system
  53. # Backup a remote host locally ("pull" style) using sshfs
  54. $ mkdir sshfs-mount
  55. $ sshfs root@example.com:/ sshfs-mount
  56. $ cd sshfs-mount
  57. $ borg create /path/to/repo::example.com-root-{now:%Y-%m-%d} .
  58. $ cd ..
  59. $ fusermount -u sshfs-mount
  60. # Make a big effort in fine granular deduplication (big chunk management
  61. # overhead, needs a lot of RAM and disk space, see formula in internals
  62. # docs - same parameters as borg < 1.0 or attic):
  63. $ borg create --chunker-params 10,23,16,4095 /path/to/repo::small /smallstuff
  64. # Backup a raw device (must not be active/in use/mounted at that time)
  65. $ dd if=/dev/sdx bs=10M | borg create /path/to/repo::my-sdx -
  66. # No compression (default)
  67. $ borg create /path/to/repo::arch ~
  68. # Super fast, low compression
  69. $ borg create --compression lz4 /path/to/repo::arch ~
  70. # Less fast, higher compression (N = 0..9)
  71. $ borg create --compression zlib,N /path/to/repo::arch ~
  72. # Even slower, even higher compression (N = 0..9)
  73. $ borg create --compression lzma,N /path/to/repo::arch ~
  74. # Use short hostname, user name and current time in archive name
  75. $ borg create /path/to/repo::{hostname}-{user}-{now} ~
  76. # Similar, use the same datetime format as borg 1.1 will have as default
  77. $ borg create /path/to/repo::{hostname}-{user}-{now:%Y-%m-%dT%H:%M:%S} ~
  78. # As above, but add nanoseconds
  79. $ borg create /path/to/repo::{hostname}-{user}-{now:%Y-%m-%dT%H:%M:%S.%f} ~
  80. # Backing up relative paths by moving into the correct directory first
  81. $ cd /home/user/Documents
  82. # The root directory of the archive will be "projectA"
  83. $ borg create /path/to/repo::daily-projectA-{now:%Y-%m-%d} projectA
  84. .. include:: usage/extract.rst.inc
  85. Examples
  86. ~~~~~~~~
  87. ::
  88. # Extract entire archive
  89. $ borg extract /path/to/repo::my-files
  90. # Extract entire archive and list files while processing
  91. $ borg extract --list /path/to/repo::my-files
  92. # Verify whether an archive could be successfully extracted, but do not write files to disk
  93. $ borg extract --dry-run /path/to/repo::my-files
  94. # Extract the "src" directory
  95. $ borg extract /path/to/repo::my-files home/USERNAME/src
  96. # Extract the "src" directory but exclude object files
  97. $ borg extract /path/to/repo::my-files home/USERNAME/src --exclude '*.o'
  98. # Restore a raw device (must not be active/in use/mounted at that time)
  99. $ borg extract --stdout /path/to/repo::my-sdx | dd of=/dev/sdx bs=10M
  100. .. Note::
  101. Currently, extract always writes into the current working directory ("."),
  102. so make sure you ``cd`` to the right place before calling ``borg extract``.
  103. .. include:: usage/check.rst.inc
  104. .. include:: usage/rename.rst.inc
  105. Examples
  106. ~~~~~~~~
  107. ::
  108. $ borg create /path/to/repo::archivename ~
  109. $ borg list /path/to/repo
  110. archivename Mon, 2016-02-15 19:50:19
  111. $ borg rename /path/to/repo::archivename newname
  112. $ borg list /path/to/repo
  113. newname Mon, 2016-02-15 19:50:19
  114. .. include:: usage/list.rst.inc
  115. Examples
  116. ~~~~~~~~
  117. ::
  118. $ borg list /path/to/repo
  119. Monday Mon, 2016-02-15 19:15:11
  120. repo Mon, 2016-02-15 19:26:54
  121. root-2016-02-15 Mon, 2016-02-15 19:36:29
  122. newname Mon, 2016-02-15 19:50:19
  123. ...
  124. $ borg list /path/to/repo::root-2016-02-15
  125. drwxr-xr-x root root 0 Mon, 2016-02-15 17:44:27 .
  126. drwxrwxr-x root root 0 Mon, 2016-02-15 19:04:49 bin
  127. -rwxr-xr-x root root 1029624 Thu, 2014-11-13 00:08:51 bin/bash
  128. lrwxrwxrwx root root 0 Fri, 2015-03-27 20:24:26 bin/bzcmp -> bzdiff
  129. -rwxr-xr-x root root 2140 Fri, 2015-03-27 20:24:22 bin/bzdiff
  130. ...
  131. $ borg list /path/to/repo::archiveA --list-format="{mode} {user:6} {group:6} {size:8d} {isomtime} {path}{extra}{NEWLINE}"
  132. drwxrwxr-x user user 0 Sun, 2015-02-01 11:00:00 .
  133. drwxrwxr-x user user 0 Sun, 2015-02-01 11:00:00 code
  134. drwxrwxr-x user user 0 Sun, 2015-02-01 11:00:00 code/myproject
  135. -rw-rw-r-- user user 1416192 Sun, 2015-02-01 11:00:00 code/myproject/file.ext
  136. ...
  137. .. include:: usage/diff.rst.inc
  138. Examples
  139. ~~~~~~~~
  140. ::
  141. $ borg init -e=none testrepo
  142. $ mkdir testdir
  143. $ cd testdir
  144. $ echo asdf > file1
  145. $ dd if=/dev/urandom bs=1M count=4 > file2
  146. $ touch file3
  147. $ borg create ../testrepo::archive1 .
  148. $ chmod a+x file1
  149. $ echo "something" >> file2
  150. $ borg create ../testrepo::archive2 .
  151. $ rm file3
  152. $ touch file4
  153. $ borg create ../testrepo::archive3 .
  154. $ cd ..
  155. $ borg diff testrepo::archive1 archive2
  156. [-rw-r--r-- -> -rwxr-xr-x] file1
  157. +135 B -252 B file2
  158. $ borg diff testrepo::archive2 archive3
  159. added 0 B file4
  160. removed 0 B file3
  161. $ borg diff testrepo::archive1 archive3
  162. [-rw-r--r-- -> -rwxr-xr-x] file1
  163. +135 B -252 B file2
  164. added 0 B file4
  165. removed 0 B file3
  166. .. include:: usage/delete.rst.inc
  167. Examples
  168. ~~~~~~~~
  169. ::
  170. # delete a single backup archive:
  171. $ borg delete /path/to/repo::Monday
  172. # delete the whole repository and the related local cache:
  173. $ borg delete /path/to/repo
  174. You requested to completely DELETE the repository *including* all archives it contains:
  175. repo Mon, 2016-02-15 19:26:54
  176. root-2016-02-15 Mon, 2016-02-15 19:36:29
  177. newname Mon, 2016-02-15 19:50:19
  178. Type 'YES' if you understand this and want to continue: YES
  179. .. include:: usage/prune.rst.inc
  180. Examples
  181. ~~~~~~~~
  182. Be careful, prune is a potentially dangerous command, it will remove backup
  183. archives.
  184. The default of prune is to apply to **all archives in the repository** unless
  185. you restrict its operation to a subset of the archives using ``--prefix``.
  186. When using ``--prefix``, be careful to choose a good prefix - e.g. do not use a
  187. prefix "foo" if you do not also want to match "foobar".
  188. It is strongly recommended to always run ``prune -v --list --dry-run ...``
  189. first so you will see what it would do without it actually doing anything.
  190. There is also a visualized prune example in ``docs/misc/prune-example.txt``.
  191. ::
  192. # Keep 7 end of day and 4 additional end of week archives.
  193. # Do a dry-run without actually deleting anything.
  194. $ borg prune -v --list --dry-run --keep-daily=7 --keep-weekly=4 /path/to/repo
  195. # Same as above but only apply to archive names starting with the hostname
  196. # of the machine followed by a "-" character:
  197. $ borg prune -v --list --keep-daily=7 --keep-weekly=4 --prefix='{hostname}-' /path/to/repo
  198. # Keep 7 end of day, 4 additional end of week archives,
  199. # and an end of month archive for every month:
  200. $ borg prune -v --list --keep-daily=7 --keep-weekly=4 --keep-monthly=-1 /path/to/repo
  201. # Keep all backups in the last 10 days, 4 additional end of week archives,
  202. # and an end of month archive for every month:
  203. $ borg prune -v --list --keep-within=10d --keep-weekly=4 --keep-monthly=-1 /path/to/repo
  204. .. include:: usage/info.rst.inc
  205. Examples
  206. ~~~~~~~~
  207. ::
  208. $ borg info /path/to/repo::root-2016-02-15
  209. Name: root-2016-02-15
  210. Fingerprint: 57c827621f21b000a8d363c1e163cc55983822b3afff3a96df595077a660be50
  211. Hostname: myhostname
  212. Username: root
  213. Time (start): Mon, 2016-02-15 19:36:29
  214. Time (end): Mon, 2016-02-15 19:39:26
  215. Command line: /usr/local/bin/borg create --list -C zlib,6 /path/to/repo::root-2016-02-15 / --one-file-system
  216. Number of files: 38100
  217. Original size Compressed size Deduplicated size
  218. This archive: 1.33 GB 613.25 MB 571.64 MB
  219. All archives: 1.63 GB 853.66 MB 584.12 MB
  220. Unique chunks Total chunks
  221. Chunk index: 36858 48844
  222. .. include:: usage/mount.rst.inc
  223. .. include:: usage/umount.rst.inc
  224. Examples
  225. ~~~~~~~~
  226. borg mount
  227. ++++++++++
  228. ::
  229. $ borg mount /path/to/repo::root-2016-02-15 /tmp/mymountpoint
  230. $ ls /tmp/mymountpoint
  231. bin boot etc home lib lib64 lost+found media mnt opt root sbin srv tmp usr var
  232. $ borg umount /tmp/mymountpoint
  233. ::
  234. $ borg mount -o versions /path/to/repo /tmp/mymountpoint
  235. $ ls -l /tmp/mymountpoint/home/user/doc.txt/
  236. total 24
  237. -rw-rw-r-- 1 user group 12357 Aug 26 21:19 doc.txt.cda00bc9
  238. -rw-rw-r-- 1 user group 12204 Aug 26 21:04 doc.txt.fa760f28
  239. $ fusermount -u /tmp/mymountpoint
  240. borgfs
  241. ++++++
  242. ::
  243. $ echo '/mnt/backup /tmp/myrepo fuse.borgfs defaults,noauto 0 0' >> /etc/fstab
  244. $ echo '/mnt/backup::root-2016-02-15 /tmp/myarchive fuse.borgfs defaults,noauto 0 0' >> /etc/fstab
  245. $ mount /tmp/myrepo
  246. $ mount /tmp/myarchive
  247. $ ls /tmp/myrepo
  248. root-2016-02-01 root-2016-02-2015
  249. $ ls /tmp/myarchive
  250. bin boot etc home lib lib64 lost+found media mnt opt root sbin srv tmp usr var
  251. .. Note::
  252. ``borgfs`` will be automatically provided if you used a distribution
  253. package, ``pip`` or ``setup.py`` to install |project_name|. Users of the
  254. standalone binary will have to manually create a symlink (see
  255. :ref:`pyinstaller-binary`).
  256. .. include:: usage/key_export.rst.inc
  257. .. include:: usage/key_import.rst.inc
  258. .. _borg-change-passphrase:
  259. .. include:: usage/key_change-passphrase.rst.inc
  260. Examples
  261. ~~~~~~~~
  262. ::
  263. # Create a key file protected repository
  264. $ borg init --encryption=keyfile -v /path/to/repo
  265. Initializing repository at "/path/to/repo"
  266. Enter new passphrase:
  267. Enter same passphrase again:
  268. Remember your passphrase. Your data will be inaccessible without it.
  269. Key in "/root/.config/borg/keys/mnt_backup" created.
  270. Keep this key safe. Your data will be inaccessible without it.
  271. Synchronizing chunks cache...
  272. Archives: 0, w/ cached Idx: 0, w/ outdated Idx: 0, w/o cached Idx: 0.
  273. Done.
  274. # Change key file passphrase
  275. $ borg key change-passphrase -v /path/to/repo
  276. Enter passphrase for key /root/.config/borg/keys/mnt_backup:
  277. Enter new passphrase:
  278. Enter same passphrase again:
  279. Remember your passphrase. Your data will be inaccessible without it.
  280. Key updated
  281. Fully automated using environment variables:
  282. ::
  283. $ BORG_NEW_PASSPHRASE=old borg init -e=repokey repo
  284. # now "old" is the current passphrase.
  285. $ BORG_PASSPHRASE=old BORG_NEW_PASSPHRASE=new borg key change-passphrase repo
  286. # now "new" is the current passphrase.
  287. .. include:: usage/serve.rst.inc
  288. Examples
  289. ~~~~~~~~
  290. borg serve has special support for ssh forced commands (see ``authorized_keys``
  291. example below): it will detect that you use such a forced command and extract
  292. the value of the ``--restrict-to-path`` option(s).
  293. It will then parse the original command that came from the client, makes sure
  294. that it is also ``borg serve`` and enforce path restriction(s) as given by the
  295. forced command. That way, other options given by the client (like ``--info`` or
  296. ``--umask``) are preserved (and are not fixed by the forced command).
  297. ::
  298. # Allow an SSH keypair to only run borg, and only have access to /path/to/repo.
  299. # Use key options to disable unneeded and potentially dangerous SSH functionality.
  300. # This will help to secure an automated remote backup system.
  301. $ cat ~/.ssh/authorized_keys
  302. command="borg serve --restrict-to-path /path/to/repo",no-pty,no-agent-forwarding,no-port-forwarding,no-X11-forwarding,no-user-rc ssh-rsa AAAAB3[...]
  303. .. include:: usage/upgrade.rst.inc
  304. Examples
  305. ~~~~~~~~
  306. ::
  307. # Upgrade the borg repository to the most recent version.
  308. $ borg upgrade -v /path/to/repo
  309. making a hardlink copy in /path/to/repo.upgrade-2016-02-15-20:51:55
  310. opening attic repository with borg and converting
  311. no key file found for repository
  312. converting repo index /path/to/repo/index.0
  313. converting 1 segments...
  314. converting borg 0.xx to borg current
  315. no key file found for repository
  316. .. _borg_key_migrate-to-repokey:
  317. Upgrading a passphrase encrypted attic repo
  318. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  319. attic offered a "passphrase" encryption mode, but this was removed in borg 1.0
  320. and replaced by the "repokey" mode (which stores the passphrase-protected
  321. encryption key into the repository config).
  322. Thus, to upgrade a "passphrase" attic repo to a "repokey" borg repo, 2 steps
  323. are needed, in this order:
  324. - borg upgrade repo
  325. - borg key migrate-to-repokey repo
  326. .. include:: usage/recreate.rst.inc
  327. Examples
  328. ~~~~~~~~
  329. ::
  330. # Make old (Attic / Borg 0.xx) archives deduplicate with Borg 1.x archives
  331. # Archives created with Borg 1.1+ and the default chunker params are skipped (archive ID stays the same)
  332. $ borg recreate /mnt/backup --chunker-params default --progress
  333. # Create a backup with little but fast compression
  334. $ borg create /mnt/backup::archive /some/files --compression lz4
  335. # Then compress it - this might take longer, but the backup has already completed, so no inconsistencies
  336. # from a long-running backup job.
  337. $ borg recreate /mnt/backup::archive --compression zlib,9
  338. # Remove unwanted files from all archives in a repository
  339. $ borg recreate /mnt/backup -e /home/icke/Pictures/drunk_photos
  340. # Change archive comment
  341. $ borg create --comment "This is a comment" /mnt/backup::archivename ~
  342. $ borg info /mnt/backup::archivename
  343. Name: archivename
  344. Fingerprint: ...
  345. Comment: This is a comment
  346. ...
  347. $ borg recreate --comment "This is a better comment" /mnt/backup::archivename
  348. $ borg info /mnt/backup::archivename
  349. Name: archivename
  350. Fingerprint: ...
  351. Comment: This is a better comment
  352. ...
  353. .. include:: usage/with-lock.rst.inc
  354. .. include:: usage/break-lock.rst.inc
  355. Miscellaneous Help
  356. ------------------
  357. .. include:: usage/help.rst.inc
  358. Debug Commands
  359. --------------
  360. There is a ``borg debug`` command that has some subcommands which are all
  361. **not intended for normal use** and **potentially very dangerous** if used incorrectly.
  362. For example, ``borg debug put-obj`` and ``borg debug delete-obj`` will only do
  363. what their name suggests: put objects into repo / delete objects from repo.
  364. Please note:
  365. - they will not update the chunks cache (chunks index) about the object
  366. - they will not update the manifest (so no automatic chunks index resync is triggered)
  367. - they will not check whether the object is in use (e.g. before delete-obj)
  368. - they will not update any metadata which may point to the object
  369. They exist to improve debugging capabilities without direct system access, e.g.
  370. in case you ever run into some severe malfunction. Use them only if you know
  371. what you are doing or if a trusted |project_name| developer tells you what to do.
  372. Additional Notes
  373. ----------------
  374. Here are misc. notes about topics that are maybe not covered in enough detail in the usage section.
  375. --chunker-params
  376. ~~~~~~~~~~~~~~~~
  377. The chunker params influence how input files are cut into pieces (chunks)
  378. which are then considered for deduplication. They also have a big impact on
  379. resource usage (RAM and disk space) as the amount of resources needed is
  380. (also) determined by the total amount of chunks in the repository (see
  381. `Indexes / Caches memory usage` for details).
  382. ``--chunker-params=10,23,16,4095`` results in a fine-grained deduplication
  383. and creates a big amount of chunks and thus uses a lot of resources to manage
  384. them. This is good for relatively small data volumes and if the machine has a
  385. good amount of free RAM and disk space.
  386. ``--chunker-params=19,23,21,4095`` (default) results in a coarse-grained
  387. deduplication and creates a much smaller amount of chunks and thus uses less
  388. resources. This is good for relatively big data volumes and if the machine has
  389. a relatively low amount of free RAM and disk space.
  390. If you already have made some archives in a repository and you then change
  391. chunker params, this of course impacts deduplication as the chunks will be
  392. cut differently.
  393. In the worst case (all files are big and were touched in between backups), this
  394. will store all content into the repository again.
  395. Usually, it is not that bad though:
  396. - usually most files are not touched, so it will just re-use the old chunks
  397. it already has in the repo
  398. - files smaller than the (both old and new) minimum chunksize result in only
  399. one chunk anyway, so the resulting chunks are same and deduplication will apply
  400. If you switch chunker params to save resources for an existing repo that
  401. already has some backup archives, you will see an increasing effect over time,
  402. when more and more files have been touched and stored again using the bigger
  403. chunksize **and** all references to the smaller older chunks have been removed
  404. (by deleting / pruning archives).
  405. If you want to see an immediate big effect on resource usage, you better start
  406. a new repository when changing chunker params.
  407. For more details, see :ref:`chunker_details`.
  408. --umask
  409. ~~~~~~~
  410. If you use ``--umask``, make sure that all repository-modifying borg commands
  411. (create, delete, prune) that access the repository in question use the same
  412. ``--umask`` value.
  413. If multiple machines access the same repository, this should hold true for all
  414. of them.
  415. --read-special
  416. ~~~~~~~~~~~~~~
  417. The --read-special option is special - you do not want to use it for normal
  418. full-filesystem backups, but rather after carefully picking some targets for it.
  419. The option ``--read-special`` triggers special treatment for block and char
  420. device files as well as FIFOs. Instead of storing them as such a device (or
  421. FIFO), they will get opened, their content will be read and in the backup
  422. archive they will show up like a regular file.
  423. Symlinks will also get special treatment if (and only if) they point to such
  424. a special file: instead of storing them as a symlink, the target special file
  425. will get processed as described above.
  426. One intended use case of this is backing up the contents of one or multiple
  427. block devices, like e.g. LVM snapshots or inactive LVs or disk partitions.
  428. You need to be careful about what you include when using ``--read-special``,
  429. e.g. if you include ``/dev/zero``, your backup will never terminate.
  430. Restoring such files' content is currently only supported one at a time via
  431. ``--stdout`` option (and you have to redirect stdout to where ever it shall go,
  432. maybe directly into an existing device file of your choice or indirectly via
  433. ``dd``).
  434. To some extent, mounting a backup archive with the backups of special files
  435. via ``borg mount`` and then loop-mounting the image files from inside the mount
  436. point will work. If you plan to access a lot of data in there, it likely will
  437. scale and perform better if you do not work via the FUSE mount.
  438. Example
  439. +++++++
  440. Imagine you have made some snapshots of logical volumes (LVs) you want to backup.
  441. .. note::
  442. For some scenarios, this is a good method to get "crash-like" consistency
  443. (I call it crash-like because it is the same as you would get if you just
  444. hit the reset button or your machine would abrubtly and completely crash).
  445. This is better than no consistency at all and a good method for some use
  446. cases, but likely not good enough if you have databases running.
  447. Then you create a backup archive of all these snapshots. The backup process will
  448. see a "frozen" state of the logical volumes, while the processes working in the
  449. original volumes continue changing the data stored there.
  450. You also add the output of ``lvdisplay`` to your backup, so you can see the LV
  451. sizes in case you ever need to recreate and restore them.
  452. After the backup has completed, you remove the snapshots again. ::
  453. $ # create snapshots here
  454. $ lvdisplay > lvdisplay.txt
  455. $ borg create --read-special /path/to/repo::arch lvdisplay.txt /dev/vg0/*-snapshot
  456. $ # remove snapshots here
  457. Now, let's see how to restore some LVs from such a backup. ::
  458. $ borg extract /path/to/repo::arch lvdisplay.txt
  459. $ # create empty LVs with correct sizes here (look into lvdisplay.txt).
  460. $ # we assume that you created an empty root and home LV and overwrite it now:
  461. $ borg extract --stdout /path/to/repo::arch dev/vg0/root-snapshot > /dev/vg0/root
  462. $ borg extract --stdout /path/to/repo::arch dev/vg0/home-snapshot > /dev/vg0/home
  463. .. _append_only_mode:
  464. Append-only mode
  465. ~~~~~~~~~~~~~~~~
  466. A repository can be made "append-only", which means that Borg will never overwrite or
  467. delete committed data (append-only refers to the segment files, but borg will also
  468. reject to delete the repository completely). This is useful for scenarios where a
  469. backup client machine backups remotely to a backup server using ``borg serve``, since
  470. a hacked client machine cannot delete backups on the server permanently.
  471. To activate append-only mode, edit the repository ``config`` file and add a line
  472. ``append_only=1`` to the ``[repository]`` section (or edit the line if it exists).
  473. In append-only mode Borg will create a transaction log in the ``transactions`` file,
  474. where each line is a transaction and a UTC timestamp.
  475. In addition, ``borg serve`` can act as if a repository is in append-only mode with
  476. its option ``--append-only``. This can be very useful for fine-tuning access control
  477. in ``.ssh/authorized_keys`` ::
  478. command="borg serve --append-only ..." ssh-rsa <key used for not-always-trustable backup clients>
  479. command="borg serve ..." ssh-rsa <key used for backup management>
  480. Example
  481. +++++++
  482. Suppose an attacker remotely deleted all backups, but your repository was in append-only
  483. mode. A transaction log in this situation might look like this: ::
  484. transaction 1, UTC time 2016-03-31T15:53:27.383532
  485. transaction 5, UTC time 2016-03-31T15:53:52.588922
  486. transaction 11, UTC time 2016-03-31T15:54:23.887256
  487. transaction 12, UTC time 2016-03-31T15:55:54.022540
  488. transaction 13, UTC time 2016-03-31T15:55:55.472564
  489. From your security logs you conclude the attacker gained access at 15:54:00 and all
  490. the backups where deleted or replaced by compromised backups. From the log you know
  491. that transactions 11 and later are compromised. Note that the transaction ID is the
  492. name of the *last* file in the transaction. For example, transaction 11 spans files 6
  493. to 11.
  494. In a real attack you'll likely want to keep the compromised repository
  495. intact to analyze what the attacker tried to achieve. It's also a good idea to make this
  496. copy just in case something goes wrong during the recovery. Since recovery is done by
  497. deleting some files, a hard link copy (``cp -al``) is sufficient.
  498. The first step to reset the repository to transaction 5, the last uncompromised transaction,
  499. is to remove the ``hints.N`` and ``index.N`` files in the repository (these two files are
  500. always expendable). In this example N is 13.
  501. Then remove or move all segment files from the segment directories in ``data/`` starting
  502. with file 6::
  503. rm data/**/{6..13}
  504. That's all to it.
  505. Drawbacks
  506. +++++++++
  507. As data is only appended, and nothing removed, commands like ``prune`` or ``delete``
  508. won't free disk space, they merely tag data as deleted in a new transaction.
  509. Be aware that as soon as you write to the repo in non-append-only mode (e.g. prune,
  510. delete or create archives from an admin machine), it will remove the deleted objects
  511. permanently (including the ones that were already marked as deleted, but not removed,
  512. in append-only mode).
  513. Note that you can go back-and-forth between normal and append-only operation by editing
  514. the configuration file, it's not a "one way trip".
  515. Further considerations
  516. ++++++++++++++++++++++
  517. Append-only mode is not respected by tools other than Borg. ``rm`` still works on the
  518. repository. Make sure that backup client machines only get to access the repository via
  519. ``borg serve``.
  520. Ensure that no remote access is possible if the repository is temporarily set to normal mode
  521. for e.g. regular pruning.
  522. Further protections can be implemented, but are outside of Borg's scope. For example,
  523. file system snapshots or wrapping ``borg serve`` to set special permissions or ACLs on
  524. new data files.