Merge pull request #3698 from ThomasWaldmann/backports-18

Backports 18

docs/conf.py

@@ -42,7 +42,7 @@ master_doc = 'index'
 
 # General information about the project.
 project = 'Borg - Deduplicating Archiver'
-copyright = '2010-2014 Jonas Borgström, 2015-2018 The Borg Collective (see AUTHORS file)'
+copyright = u'2010-2014 Jonas Borgström, 2015-2018 The Borg Collective (see AUTHORS file)'
 
 # The version info for the project you're documenting, acts as replacement for
 # |version| and |release|, also used in various other places throughout the

docs/quickstart.rst

@@ -55,6 +55,33 @@ Also helpful:
     - Other tasks fill the disk simultaneously
     - Hard quotas (which may not be reflected in statvfs(2))
 
+Important note about permissions
+--------------------------------
+
+Using root likely will be required if you want to backup files of other users
+or the operating system. If you only back up your own files, you neither need
+nor want to use root.
+
+Avoid to create a mixup of users and permissions in your repository (or cache).
+
+This can easily happen if you run borg using different user accounts (e.g. your
+non-privileged user and root) while accessing the same repo.
+
+Of course, a non-root user will have no permission to work with the files
+created by root (or another user) and borg operations will just fail with
+`Permission denied`.
+
+The easy way to avoid this is to always access the repo as the same user:
+
+For a local repository just always invoke borg as same user.
+
+For a remote repository: always use e.g. borg@remote_host. You can use this
+from different local users, the remote user accessing the repo will always be
+borg.
+
+If you need to access a local repository from different users, you can use the
+same method by using ssh to borg@localhost.
+
 Automating backups
 ------------------
 

docs/usage_general.rst.inc

@@ -86,8 +86,22 @@ If you have set BORG_REPO (see above) and an archive location is needed, use
 ``::archive_name`` - the repo URL part is then read from BORG_REPO.
 
 
-Type of log output
-~~~~~~~~~~~~~~~~~~
+Logging
+~~~~~~~
+
+Borg writes all log output to stderr by default. But please note that something
+showing up on stderr does *not* indicate an error condition just because it is
+on stderr. Please check the log levels of the messages and the return code of
+borg for determining error, warning or success conditions.
+
+If you want to capture the log output to a file, just redirect it:
+
+::
+
+    borg create repo::archive myfiles 2>> logfile
+
+
+Custom logging configurations can be implemented via BORG_LOGGING_CONF.
 
 The log level of the builtin logging configuration defaults to WARNING.
 This is because we want Borg to be mostly silent and only output

src/borg/archiver.py

@@ -1942,6 +1942,15 @@ class Archiver:
 
     helptext = collections.OrderedDict()
     helptext['patterns'] = textwrap.dedent('''
+        The path/filenames used as input for the pattern matching start from the
+        currently active recursion root. You usually give the recursion root(s)
+        when invoking borg and these can be either relative or absolute paths.
+
+        So, when you give `relative/` as root, the paths going into the matcher
+        will look like `relative/.../file.ext`. When you give `/absolute/` as root,
+        they will look like `/absolute/.../file.ext`. This is meant when we talk
+        about "full path" below.
+
         File patterns support these styles: fnmatch, shell, regular expressions,
         path prefixes and path full-matches. By default, fnmatch is used for
         ``--exclude`` patterns and shell-style is used for the experimental ``--pattern``
@@ -1961,8 +1970,8 @@ class Archiver:
             the path separator ('\\' for Windows and '/' on other systems) is not
             treated specially. Wrap meta-characters in brackets for a literal
             match (i.e. `[?]` to match the literal character `?`). For a path
-            to match a pattern, it must completely match from start to end, or
-            must match from the start to just before a path separator. Except
+            to match a pattern, the full path must match, or it must match
+            from the start of the full path to just before a path separator. Except
             for the root path, paths will never end in the path separator when
             matching is attempted.  Thus, if a given pattern ends in a path
             separator, a '\*' is appended before matching is attempted.
@@ -1976,7 +1985,7 @@ class Archiver:
 
         Regular expressions, selector `re:`
             Regular expressions similar to those found in Perl are supported. Unlike
-            shell patterns regular expressions are not required to match the complete
+            shell patterns regular expressions are not required to match the full
             path and any substring match is sufficient. It is strongly recommended to
             anchor patterns to the start ('^'), to the end ('$') or both. Path
             separators ('\\' for Windows and '/' on other systems) in paths are
@@ -1986,13 +1995,13 @@ class Archiver:
 
         Path prefix, selector `pp:`
             This pattern style is useful to match whole sub-directories. The pattern
-            `pp:/data/bar` matches `/data/bar` and everything therein.
+            `pp:root/somedir` matches `root/somedir` and everything therein.
 
         Path full-match, selector `pf:`
-            This pattern style is useful to match whole paths.
+            This pattern style is (only) useful to match full paths.
             This is kind of a pseudo pattern as it can not have any variable or
-            unspecified parts - the full, precise path must be given.
-            `pf:/data/foo.txt` matches `/data/foo.txt` only.
+            unspecified parts - the full path must be given.
+            `pf:root/file.ext` matches `root/file.txt` only.
 
             Implementation note: this is implemented via very time-efficient O(1)
             hashtable lookups (this means you can have huge amounts of such patterns
@@ -2075,7 +2084,16 @@ class Archiver:
             considered first (in the order of appearance). Then patterns from ``--patterns-from``
             are added. Exclusion patterns from ``--exclude-from`` files are appended last.
 
-            An example ``--patterns-from`` file could look like that::
+            Examples::
+
+                # backup pics, but not the ones from 2018, except the good ones:
+                # note: using = is essential to avoid cmdline argument parsing issues.
+                borg create --pattern=+pics/2018/good --pattern=-pics/2018 repo::arch pics
+
+                # use a file with patterns:
+                borg create --patterns-from patterns.lst repo::arch
+
+            The patterns.lst file could look like that::
 
                 # "sh:" pattern style is the default, so the following line is not needed:
                 P sh