usage.rst 27 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571572573574575576577578579580581582583584585586587588589590591592593594595596597598599600601602603604605606607608609610611612613614615616617618619620621622623624625626627628629630631632633634635636637638639640641642643644645646647648649650651652653654655656657658659660661662663664665666667668669670671672673674675676677678679680681682683684685686687688689690691692693694695696697698699700701702703704705706707708709710711712713714715716717
  1. .. include:: global.rst.inc
  2. .. _detailed_usage:
  3. Usage
  4. =====
  5. |project_name| consists of a number of commands. Each command accepts
  6. a number of arguments and options. The following sections will describe each
  7. command in detail.
  8. General
  9. -------
  10. Type of log output
  11. ~~~~~~~~~~~~~~~~~~
  12. The log level of the builtin logging configuration defaults to WARNING.
  13. This is because we want |project_name| to be mostly silent and only output
  14. warnings (plus errors and critical messages).
  15. Use ``--verbose`` or ``--info`` to set INFO (you will get informative output then
  16. additionally to warnings, errors, critical messages).
  17. Use ``--debug`` to set DEBUG to get output made for debugging.
  18. All log messages created with at least the set level will be output.
  19. Log levels: DEBUG < INFO < WARNING < ERROR < CRITICAL
  20. While you can set misc. log levels, do not expect that every command will
  21. give different output on different log levels - it's just a possibility.
  22. .. warning:: While some options (like ``--stats`` or ``--list``) will emit more
  23. informational messages, you have to use INFO (or lower) log level to make
  24. them show up in log output. Use ``-v`` or a logging configuration.
  25. Return codes
  26. ~~~~~~~~~~~~
  27. |project_name| can exit with the following return codes (rc):
  28. ::
  29. 0 = success (logged as INFO)
  30. 1 = warning (operation reached its normal end, but there were warnings -
  31. you should check the log, logged as WARNING)
  32. 2 = error (like a fatal error, a local or remote exception, the operation
  33. did not reach its normal end, logged as ERROR)
  34. 128+N = killed by signal N (e.g. 137 == kill -9)
  35. If you use ``--show-rc``, the return code is also logged at the indicated
  36. level as the last log entry.
  37. Environment Variables
  38. ~~~~~~~~~~~~~~~~~~~~~
  39. |project_name| uses some environment variables for automation:
  40. General:
  41. BORG_REPO
  42. When set, use the value to give the default repository location. If a command needs an archive
  43. parameter, you can abbreviate as `::archive`. If a command needs a repository parameter, you
  44. can either leave it away or abbreviate as `::`, if a positional parameter is required.
  45. BORG_PASSPHRASE
  46. When set, use the value to answer the passphrase question for encrypted repositories.
  47. BORG_DISPLAY_PASSPHRASE
  48. When set, use the value to answer the "display the passphrase for verification" question when defining a new passphrase for encrypted repositories.
  49. BORG_LOGGING_CONF
  50. When set, use the given filename as INI_-style logging configuration.
  51. BORG_RSH
  52. When set, use this command instead of ``ssh``. This can be used to specify ssh options, such as
  53. a custom identity file ``ssh -i /path/to/private/key``. See ``man ssh`` for other options.
  54. TMPDIR
  55. where temporary files are stored (might need a lot of temporary space for some operations)
  56. Some automatic "answerers" (if set, they automatically answer confirmation questions):
  57. BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK=no (or =yes)
  58. For "Warning: Attempting to access a previously unknown unencrypted repository"
  59. BORG_RELOCATED_REPO_ACCESS_IS_OK=no (or =yes)
  60. For "Warning: The repository at location ... was previously located at ..."
  61. BORG_CHECK_I_KNOW_WHAT_I_AM_DOING=NO (or =YES)
  62. For "Warning: 'check --repair' is an experimental feature that might result in data loss."
  63. BORG_DELETE_I_KNOW_WHAT_I_AM_DOING=NO (or =YES)
  64. For "You requested to completely DELETE the repository *including* all archives it contains:"
  65. Note: answers are case sensitive. setting an invalid answer value might either give the default
  66. answer or ask you interactively, depending on whether retries are allowed (they by default are
  67. allowed). So please test your scripts interactively before making them a non-interactive script.
  68. Directories:
  69. BORG_KEYS_DIR
  70. Default to '~/.config/borg/keys'. This directory contains keys for encrypted repositories.
  71. BORG_CACHE_DIR
  72. Default to '~/.cache/borg'. This directory contains the local cache and might need a lot
  73. of space for dealing with big repositories).
  74. Building:
  75. BORG_OPENSSL_PREFIX
  76. Adds given OpenSSL header file directory to the default locations (setup.py).
  77. BORG_LZ4_PREFIX
  78. Adds given LZ4 header file directory to the default locations (setup.py).
  79. Please note:
  80. - be very careful when using the "yes" sayers, the warnings with prompt exist for your / your data's security/safety
  81. - also be very careful when putting your passphrase into a script, make sure it has appropriate file permissions
  82. (e.g. mode 600, root:root).
  83. .. _INI: https://docs.python.org/3.4/library/logging.config.html#configuration-file-format
  84. Resource Usage
  85. ~~~~~~~~~~~~~~
  86. |project_name| might use a lot of resources depending on the size of the data set it is dealing with.
  87. CPU:
  88. It won't go beyond 100% of 1 core as the code is currently single-threaded.
  89. Especially higher zlib and lzma compression levels use significant amounts
  90. of CPU cycles.
  91. Memory (RAM):
  92. The chunks index and the files index are read into memory for performance
  93. reasons.
  94. Compression, esp. lzma compression with high levels might need substantial
  95. amounts of memory.
  96. Temporary files:
  97. Reading data and metadata from a FUSE mounted repository will consume about
  98. the same space as the deduplicated chunks used to represent them in the
  99. repository.
  100. Cache files:
  101. Contains the chunks index and files index (plus a compressed collection of
  102. single-archive chunk indexes).
  103. Chunks index:
  104. Proportional to the amount of data chunks in your repo. Lots of chunks
  105. in your repo imply a big chunks index.
  106. It is possible to tweak the chunker params (see create options).
  107. Files index:
  108. Proportional to the amount of files in your last backup. Can be switched
  109. off (see create options), but next backup will be much slower if you do.
  110. Network:
  111. If your repository is remote, all deduplicated (and optionally compressed/
  112. encrypted) data of course has to go over the connection (ssh: repo url).
  113. If you use a locally mounted network filesystem, additionally some copy
  114. operations used for transaction support also go over the connection. If
  115. you backup multiple sources to one target repository, additional traffic
  116. happens for cache resynchronization.
  117. In case you are interested in more details, please read the internals documentation.
  118. Units
  119. ~~~~~
  120. To display quantities, |project_name| takes care of respecting the
  121. usual conventions of scale. Disk sizes are displayed in `decimal
  122. <https://en.wikipedia.org/wiki/Decimal>`_, using powers of ten (so
  123. ``kB`` means 1000 bytes). For memory usage, `binary prefixes
  124. <https://en.wikipedia.org/wiki/Binary_prefix>`_ are used, and are
  125. indicated using the `IEC binary prefixes
  126. <https://en.wikipedia.org/wiki/IEC_80000-13#Prefixes_for_binary_multiples>`_,
  127. using powers of two (so ``KiB`` means 1024 bytes).
  128. Date and Time
  129. ~~~~~~~~~~~~~
  130. We format date and time conforming to ISO-8601, that is: YYYY-MM-DD and
  131. HH:MM:SS (24h clock).
  132. For more information about that, see: https://xkcd.com/1179/
  133. Unless otherwise noted, we display local date and time.
  134. Internally, we store and process date and time as UTC.
  135. .. include:: usage/init.rst.inc
  136. Examples
  137. ~~~~~~~~
  138. ::
  139. # Local repository (default is to use encryption in repokey mode)
  140. $ borg init /mnt/backup
  141. # Local repository (no encryption)
  142. $ borg init --encryption=none /mnt/backup
  143. # Remote repository (accesses a remote borg via ssh)
  144. $ borg init user@hostname:backup
  145. # Remote repository (store the key your home dir)
  146. $ borg init --encryption=keyfile user@hostname:backup
  147. Important notes about encryption:
  148. It is not recommended to disable encryption. Repository encryption protects you
  149. e.g. against the case that an attacker has access to your backup repository.
  150. But be careful with the key / the passphrase:
  151. If you want "passphrase-only" security, use the ``repokey`` mode. The key will
  152. be stored inside the repository (in its "config" file). In above mentioned
  153. attack scenario, the attacker will have the key (but not the passphrase).
  154. If you want "passphrase and having-the-key" security, use the ``keyfile`` mode.
  155. The key will be stored in your home directory (in ``.config/borg/keys``). In
  156. the attack scenario, the attacker who has just access to your repo won't have
  157. the key (and also not the passphrase).
  158. Make a backup copy of the key file (``keyfile`` mode) or repo config file
  159. (``repokey`` mode) and keep it at a safe place, so you still have the key in
  160. case it gets corrupted or lost. Also keep the passphrase at a safe place.
  161. The backup that is encrypted with that key won't help you with that, of course.
  162. Make sure you use a good passphrase. Not too short, not too simple. The real
  163. encryption / decryption key is encrypted with / locked by your passphrase.
  164. If an attacker gets your key, he can't unlock and use it without knowing the
  165. passphrase.
  166. Be careful with special or non-ascii characters in your passphrase:
  167. - |project_name| processes the passphrase as unicode (and encodes it as utf-8),
  168. so it does not have problems dealing with even the strangest characters.
  169. - BUT: that does not necessarily apply to your OS / VM / keyboard configuration.
  170. So better use a long passphrase made from simple ascii chars than one that
  171. includes non-ascii stuff or characters that are hard/impossible to enter on
  172. a different keyboard layout.
  173. You can change your passphrase for existing repos at any time, it won't affect
  174. the encryption/decryption key or other secrets.
  175. .. include:: usage/create.rst.inc
  176. Examples
  177. ~~~~~~~~
  178. ::
  179. # Backup ~/Documents into an archive named "my-documents"
  180. $ borg create /mnt/backup::my-documents ~/Documents
  181. # same, but verbosely list all files as we process them
  182. $ borg create -v --list /mnt/backup::my-documents ~/Documents
  183. # Backup ~/Documents and ~/src but exclude pyc files
  184. $ borg create /mnt/backup::my-files \
  185. ~/Documents \
  186. ~/src \
  187. --exclude '*.pyc'
  188. # Backup home directories excluding image thumbnails (i.e. only
  189. # /home/*/.thumbnails is excluded, not /home/*/*/.thumbnails)
  190. $ borg create /mnt/backup::my-files /home \
  191. --exclude 're:^/home/[^/]+/\.thumbnails/'
  192. # Do the same using a shell-style pattern
  193. $ borg create /mnt/backup::my-files /home \
  194. --exclude 'sh:/home/*/.thumbnails'
  195. # Backup the root filesystem into an archive named "root-YYYY-MM-DD"
  196. # use zlib compression (good, but slow) - default is no compression
  197. $ borg create -C zlib,6 /mnt/backup::root-{now:%Y-%m-%d} / --one-file-system
  198. # Make a big effort in fine granular deduplication (big chunk management
  199. # overhead, needs a lot of RAM and disk space, see formula in internals
  200. # docs - same parameters as borg < 1.0 or attic):
  201. $ borg create --chunker-params 10,23,16,4095 /mnt/backup::small /smallstuff
  202. # Backup a raw device (must not be active/in use/mounted at that time)
  203. $ dd if=/dev/sdx bs=10M | borg create /mnt/backup::my-sdx -
  204. # No compression (default)
  205. $ borg create /mnt/backup::repo ~
  206. # Super fast, low compression
  207. $ borg create --compression lz4 /mnt/backup::repo ~
  208. # Less fast, higher compression (N = 0..9)
  209. $ borg create --compression zlib,N /mnt/backup::repo ~
  210. # Even slower, even higher compression (N = 0..9)
  211. $ borg create --compression lzma,N /mnt/backup::repo ~
  212. # Format tags available for archive name:
  213. # {now}, {utcnow}, {fqdn}, {hostname}, {user}, {pid}
  214. # add short hostname, backup username and current unixtime (seconds from epoch)
  215. $ borg create /mnt/backup::{hostname}-{user}-{now:%s} ~
  216. .. include:: usage/extract.rst.inc
  217. Examples
  218. ~~~~~~~~
  219. ::
  220. # Extract entire archive
  221. $ borg extract /mnt/backup::my-files
  222. # Extract entire archive and list files while processing
  223. $ borg extract -v --list /mnt/backup::my-files
  224. # Extract the "src" directory
  225. $ borg extract /mnt/backup::my-files home/USERNAME/src
  226. # Extract the "src" directory but exclude object files
  227. $ borg extract /mnt/backup::my-files home/USERNAME/src --exclude '*.o'
  228. # Restore a raw device (must not be active/in use/mounted at that time)
  229. $ borg extract --stdout /mnt/backup::my-sdx | dd of=/dev/sdx bs=10M
  230. Note: currently, extract always writes into the current working directory ("."),
  231. so make sure you ``cd`` to the right place before calling ``borg extract``.
  232. .. include:: usage/check.rst.inc
  233. .. include:: usage/rename.rst.inc
  234. Examples
  235. ~~~~~~~~
  236. ::
  237. $ borg create /mnt/backup::archivename ~
  238. $ borg list /mnt/backup
  239. archivename Mon, 2016-02-15 19:50:19
  240. $ borg rename /mnt/backup::archivename newname
  241. $ borg list /mnt/backup
  242. newname Mon, 2016-02-15 19:50:19
  243. .. include:: usage/list.rst.inc
  244. Examples
  245. ~~~~~~~~
  246. ::
  247. $ borg list /mnt/backup
  248. Monday Mon, 2016-02-15 19:15:11
  249. repo Mon, 2016-02-15 19:26:54
  250. root-2016-02-15 Mon, 2016-02-15 19:36:29
  251. newname Mon, 2016-02-15 19:50:19
  252. ...
  253. $ borg list /mnt/backup::root-2016-02-15
  254. drwxr-xr-x root root 0 Mon, 2016-02-15 17:44:27 .
  255. drwxrwxr-x root root 0 Mon, 2016-02-15 19:04:49 bin
  256. -rwxr-xr-x root root 1029624 Thu, 2014-11-13 00:08:51 bin/bash
  257. lrwxrwxrwx root root 0 Fri, 2015-03-27 20:24:26 bin/bzcmp -> bzdiff
  258. -rwxr-xr-x root root 2140 Fri, 2015-03-27 20:24:22 bin/bzdiff
  259. ...
  260. $ borg list /mnt/backup::archiveA --list-format="{mode} {user:6} {group:6} {size:8d} {isomtime} {path}{extra}{NEWLINE}"
  261. drwxrwxr-x user user 0 Sun, 2015-02-01 11:00:00 .
  262. drwxrwxr-x user user 0 Sun, 2015-02-01 11:00:00 code
  263. drwxrwxr-x user user 0 Sun, 2015-02-01 11:00:00 code/myproject
  264. -rw-rw-r-- user user 1416192 Sun, 2015-02-01 11:00:00 code/myproject/file.ext
  265. ...
  266. # see what is changed between archives, based on file modification time, size and file path
  267. $ borg list /mnt/backup::archiveA --list-format="{mtime:%s}{TAB}{size}{TAB}{path}{LF}" |sort -n > /tmp/list.archiveA
  268. $ borg list /mnt/backup::archiveB --list-format="{mtime:%s}{TAB}{size}{TAB}{path}{LF}" |sort -n > /tmp/list.archiveB
  269. $ diff -y /tmp/list.archiveA /tmp/list.archiveB
  270. 1422781200 0 . 1422781200 0 .
  271. 1422781200 0 code 1422781200 0 code
  272. 1422781200 0 code/myproject 1422781200 0 code/myproject
  273. 1422781200 1416192 code/myproject/file.ext | 1454664653 1416192 code/myproject/file.ext
  274. ...
  275. .. include:: usage/delete.rst.inc
  276. Examples
  277. ~~~~~~~~
  278. ::
  279. # delete a single backup archive:
  280. $ borg delete /mnt/backup::Monday
  281. # delete the whole repository and the related local cache:
  282. $ borg delete /mnt/backup
  283. You requested to completely DELETE the repository *including* all archives it contains:
  284. repo Mon, 2016-02-15 19:26:54
  285. root-2016-02-15 Mon, 2016-02-15 19:36:29
  286. newname Mon, 2016-02-15 19:50:19
  287. Type 'YES' if you understand this and want to continue: YES
  288. .. include:: usage/prune.rst.inc
  289. Examples
  290. ~~~~~~~~
  291. Be careful, prune is a potentially dangerous command, it will remove backup
  292. archives.
  293. The default of prune is to apply to **all archives in the repository** unless
  294. you restrict its operation to a subset of the archives using ``--prefix``.
  295. When using ``--prefix``, be careful to choose a good prefix - e.g. do not use a
  296. prefix "foo" if you do not also want to match "foobar".
  297. It is strongly recommended to always run ``prune --dry-run ...`` first so you
  298. will see what it would do without it actually doing anything.
  299. ::
  300. # Keep 7 end of day and 4 additional end of week archives.
  301. # Do a dry-run without actually deleting anything.
  302. $ borg prune --dry-run --keep-daily=7 --keep-weekly=4 /mnt/backup
  303. # Same as above but only apply to archive names starting with "foo":
  304. $ borg prune --keep-daily=7 --keep-weekly=4 --prefix=foo /mnt/backup
  305. # Keep 7 end of day, 4 additional end of week archives,
  306. # and an end of month archive for every month:
  307. $ borg prune --keep-daily=7 --keep-weekly=4 --keep-monthly=-1 /mnt/backup
  308. # Keep all backups in the last 10 days, 4 additional end of week archives,
  309. # and an end of month archive for every month:
  310. $ borg prune --keep-within=10d --keep-weekly=4 --keep-monthly=-1 /mnt/backup
  311. .. include:: usage/info.rst.inc
  312. Examples
  313. ~~~~~~~~
  314. ::
  315. $ borg info /mnt/backup::root-2016-02-15
  316. Name: root-2016-02-15
  317. Fingerprint: 57c827621f21b000a8d363c1e163cc55983822b3afff3a96df595077a660be50
  318. Hostname: myhostname
  319. Username: root
  320. Time (start): Mon, 2016-02-15 19:36:29
  321. Time (end): Mon, 2016-02-15 19:39:26
  322. Command line: /usr/local/bin/borg create -v --list -C zlib,6 /mnt/backup::root-2016-02-15 / --one-file-system
  323. Number of files: 38100
  324. Original size Compressed size Deduplicated size
  325. This archive: 1.33 GB 613.25 MB 571.64 MB
  326. All archives: 1.63 GB 853.66 MB 584.12 MB
  327. Unique chunks Total chunks
  328. Chunk index: 36858 48844
  329. .. include:: usage/mount.rst.inc
  330. Examples
  331. ~~~~~~~~
  332. borg mount/borgfs
  333. +++++++++++++++++
  334. ::
  335. $ borg mount /mnt/backup::root-2016-02-15 /tmp/mymountpoint
  336. $ ls /tmp/mymountpoint
  337. bin boot etc home lib lib64 lost+found media mnt opt root sbin srv tmp usr var
  338. $ fusermount -u /tmp/mymountpoint
  339. borgfs
  340. ++++++
  341. ::
  342. $ echo '/mnt/backup /tmp/myrepo fuse.borgfs defaults,noauto 0 0' >> /etc/fstab
  343. $ echo '/mnt/backup::root-2016-02-15 /tmp/myarchive fuse.borgfs defaults,noauto 0 0' >> /etc/fstab
  344. $ mount /tmp/myrepo
  345. $ mount /tmp/myarchive
  346. $ ls /tmp/myrepo
  347. root-2016-02-01 root-2016-02-2015
  348. $ ls /tmp/myarchive
  349. bin boot etc home lib lib64 lost+found media mnt opt root sbin srv tmp usr var
  350. .. Note::
  351. ``borgfs`` will be automatically provided if you used a distribution
  352. package, ``pip`` or ``setup.py`` to install |project_name|. Users of the
  353. standalone binary will have to manually create a symlink (see
  354. :ref:`pyinstaller-binary`).
  355. .. include:: usage/change-passphrase.rst.inc
  356. Examples
  357. ~~~~~~~~
  358. ::
  359. # Create a key file protected repository
  360. $ borg init --encryption=keyfile -v /mnt/backup
  361. Initializing repository at "/mnt/backup"
  362. Enter new passphrase:
  363. Enter same passphrase again:
  364. Remember your passphrase. Your data will be inaccessible without it.
  365. Key in "/root/.config/borg/keys/mnt_backup" created.
  366. Keep this key safe. Your data will be inaccessible without it.
  367. Synchronizing chunks cache...
  368. Archives: 0, w/ cached Idx: 0, w/ outdated Idx: 0, w/o cached Idx: 0.
  369. Done.
  370. # Change key file passphrase
  371. $ borg change-passphrase -v /mnt/backup
  372. Enter passphrase for key /root/.config/borg/keys/mnt_backup:
  373. Enter new passphrase:
  374. Enter same passphrase again:
  375. Remember your passphrase. Your data will be inaccessible without it.
  376. Key updated
  377. .. include:: usage/serve.rst.inc
  378. Examples
  379. ~~~~~~~~
  380. borg serve has special support for ssh forced commands (see ``authorized_keys``
  381. example below): it will detect that you use such a forced command and extract
  382. the value of the ``--restrict-to-path`` option(s).
  383. It will then parse the original command that came from the client, makes sure
  384. that it is also ``borg serve`` and enforce path restriction(s) as given by the
  385. forced command. That way, other options given by the client (like ``--info`` or
  386. ``--umask``) are preserved (and are not fixed by the forced command).
  387. ::
  388. # Allow an SSH keypair to only run borg, and only have access to /mnt/backup.
  389. # Use key options to disable unneeded and potentially dangerous SSH functionality.
  390. # This will help to secure an automated remote backup system.
  391. $ cat ~/.ssh/authorized_keys
  392. command="borg serve --restrict-to-path /mnt/backup",no-pty,no-agent-forwarding,no-port-forwarding,no-X11-forwarding,no-user-rc ssh-rsa AAAAB3[...]
  393. .. include:: usage/upgrade.rst.inc
  394. Examples
  395. ~~~~~~~~
  396. ::
  397. # Upgrade the borg repository to the most recent version.
  398. $ borg upgrade -v /mnt/backup
  399. making a hardlink copy in /mnt/backup.upgrade-2016-02-15-20:51:55
  400. opening attic repository with borg and converting
  401. no key file found for repository
  402. converting repo index /mnt/backup/index.0
  403. converting 1 segments...
  404. converting borg 0.xx to borg current
  405. no key file found for repository
  406. Miscellaneous Help
  407. ------------------
  408. .. include:: usage/help.rst.inc
  409. Debug Commands
  410. --------------
  411. There are some more commands (all starting with "debug-") which are all
  412. **not intended for normal use** and **potentially very dangerous** if used incorrectly.
  413. They exist to improve debugging capabilities without direct system access, e.g.
  414. in case you ever run into some severe malfunction. Use them only if you know
  415. what you are doing or if a trusted |project_name| developer tells you what to do.
  416. Additional Notes
  417. ----------------
  418. Here are misc. notes about topics that are maybe not covered in enough detail in the usage section.
  419. Item flags
  420. ~~~~~~~~~~
  421. ``borg create -v --list`` outputs a verbose list of all files, directories and other
  422. file system items it considered (no matter whether they had content changes
  423. or not). For each item, it prefixes a single-letter flag that indicates type
  424. and/or status of the item.
  425. If you are interested only in a subset of that output, you can give e.g.
  426. ``--filter=AME`` and it will only show regular files with A, M or E status (see
  427. below).
  428. A uppercase character represents the status of a regular file relative to the
  429. "files" cache (not relative to the repo -- this is an issue if the files cache
  430. is not used). Metadata is stored in any case and for 'A' and 'M' also new data
  431. chunks are stored. For 'U' all data chunks refer to already existing chunks.
  432. - 'A' = regular file, added (see also :ref:`a_status_oddity` in the FAQ)
  433. - 'M' = regular file, modified
  434. - 'U' = regular file, unchanged
  435. - 'E' = regular file, an error happened while accessing/reading *this* file
  436. A lowercase character means a file type other than a regular file,
  437. borg usually just stores their metadata:
  438. - 'd' = directory
  439. - 'b' = block device
  440. - 'c' = char device
  441. - 'h' = regular file, hardlink (to already seen inodes)
  442. - 's' = symlink
  443. - 'f' = fifo
  444. Other flags used include:
  445. - 'i' = backup data was read from standard input (stdin)
  446. - '-' = dry run, item was *not* backed up
  447. - '?' = missing status code (if you see this, please file a bug report!)
  448. --chunker-params
  449. ~~~~~~~~~~~~~~~~
  450. The chunker params influence how input files are cut into pieces (chunks)
  451. which are then considered for deduplication. They also have a big impact on
  452. resource usage (RAM and disk space) as the amount of resources needed is
  453. (also) determined by the total amount of chunks in the repository (see
  454. `Indexes / Caches memory usage` for details).
  455. ``--chunker-params=10,23,16,4095`` results in a fine-grained deduplication
  456. and creates a big amount of chunks and thus uses a lot of resources to manage
  457. them. This is good for relatively small data volumes and if the machine has a
  458. good amount of free RAM and disk space.
  459. ``--chunker-params=19,23,21,4095`` (default) results in a coarse-grained
  460. deduplication and creates a much smaller amount of chunks and thus uses less
  461. resources. This is good for relatively big data volumes and if the machine has
  462. a relatively low amount of free RAM and disk space.
  463. If you already have made some archives in a repository and you then change
  464. chunker params, this of course impacts deduplication as the chunks will be
  465. cut differently.
  466. In the worst case (all files are big and were touched in between backups), this
  467. will store all content into the repository again.
  468. Usually, it is not that bad though:
  469. - usually most files are not touched, so it will just re-use the old chunks
  470. it already has in the repo
  471. - files smaller than the (both old and new) minimum chunksize result in only
  472. one chunk anyway, so the resulting chunks are same and deduplication will apply
  473. If you switch chunker params to save resources for an existing repo that
  474. already has some backup archives, you will see an increasing effect over time,
  475. when more and more files have been touched and stored again using the bigger
  476. chunksize **and** all references to the smaller older chunks have been removed
  477. (by deleting / pruning archives).
  478. If you want to see an immediate big effect on resource usage, you better start
  479. a new repository when changing chunker params.
  480. For more details, see :ref:`chunker_details`.
  481. --read-special
  482. ~~~~~~~~~~~~~~
  483. The option ``--read-special`` is not intended for normal, filesystem-level (full or
  484. partly-recursive) backups. You only give this option if you want to do something
  485. rather ... special -- and if you have hand-picked some files that you want to treat
  486. that way.
  487. ``borg create --read-special`` will open all files without doing any special
  488. treatment according to the file type (the only exception here are directories:
  489. they will be recursed into). Just imagine what happens if you do ``cat
  490. filename`` --- the content you will see there is what borg will backup for that
  491. filename.
  492. So, for example, symlinks will be followed, block device content will be read,
  493. named pipes / UNIX domain sockets will be read.
  494. You need to be careful with what you give as filename when using ``--read-special``,
  495. e.g. if you give ``/dev/zero``, your backup will never terminate.
  496. The given files' metadata is saved as it would be saved without
  497. ``--read-special`` (e.g. its name, its size [might be 0], its mode, etc.) -- but
  498. additionally, also the content read from it will be saved for it.
  499. Restoring such files' content is currently only supported one at a time via
  500. ``--stdout`` option (and you have to redirect stdout to where ever it shall go,
  501. maybe directly into an existing device file of your choice or indirectly via
  502. ``dd``).
  503. Example
  504. +++++++
  505. Imagine you have made some snapshots of logical volumes (LVs) you want to backup.
  506. .. note::
  507. For some scenarios, this is a good method to get "crash-like" consistency
  508. (I call it crash-like because it is the same as you would get if you just
  509. hit the reset button or your machine would abrubtly and completely crash).
  510. This is better than no consistency at all and a good method for some use
  511. cases, but likely not good enough if you have databases running.
  512. Then you create a backup archive of all these snapshots. The backup process will
  513. see a "frozen" state of the logical volumes, while the processes working in the
  514. original volumes continue changing the data stored there.
  515. You also add the output of ``lvdisplay`` to your backup, so you can see the LV
  516. sizes in case you ever need to recreate and restore them.
  517. After the backup has completed, you remove the snapshots again. ::
  518. $ # create snapshots here
  519. $ lvdisplay > lvdisplay.txt
  520. $ borg create --read-special /mnt/backup::repo lvdisplay.txt /dev/vg0/*-snapshot
  521. $ # remove snapshots here
  522. Now, let's see how to restore some LVs from such a backup. ::
  523. $ borg extract /mnt/backup::repo lvdisplay.txt
  524. $ # create empty LVs with correct sizes here (look into lvdisplay.txt).
  525. $ # we assume that you created an empty root and home LV and overwrite it now:
  526. $ borg extract --stdout /mnt/backup::repo dev/vg0/root-snapshot > /dev/vg0/root
  527. $ borg extract --stdout /mnt/backup::repo dev/vg0/home-snapshot > /dev/vg0/home