Browse Source

normalize authenticated key modes

rename authenticated to authenticated-blake2, consistent with the other
blake2 key modes

add authenticated mode that fills the blank and is consistent with the
other "unqualified" key modes
Marian Beermann 8 years ago
parent
commit
221dc1c4c7
5 changed files with 165 additions and 55 deletions
  1. 65 15
      docs/man/borg-init.1
  2. 32 14
      docs/usage/init.rst.inc
  3. 34 16
      src/borg/archiver.py
  4. 15 6
      src/borg/crypto/key.py
  5. 19 4
      src/borg/testsuite/key.py

+ 65 - 15
docs/man/borg-init.1

@@ -1,6 +1,6 @@
 .\" Man page generated from reStructuredText.
 .\" Man page generated from reStructuredText.
 .
 .
-.TH BORG-INIT 1 "2017-05-17" "" "borg backup tool"
+.TH BORG-INIT 1 "2017-06-11" "" "borg backup tool"
 .SH NAME
 .SH NAME
 borg-init \- Initialize an empty repository
 borg-init \- Initialize an empty repository
 .
 .
@@ -81,6 +81,55 @@ a different keyboard layout.
 You can change your passphrase for existing repos at any time, it won\(aqt affect
 You can change your passphrase for existing repos at any time, it won\(aqt affect
 the encryption/decryption key or other secrets.
 the encryption/decryption key or other secrets.
 .SS Encryption modes
 .SS Encryption modes
+.TS
+center;
+|l|l|l|l|.
+_
+T{
+Hash/MAC
+T}	T{
+Not encrypted
+no auth
+T}	T{
+Not encrypted,
+but authenticated
+T}	T{
+Encrypted (AEAD w/ AES)
+and authenticated
+T}
+_
+T{
+SHA\-256
+T}	T{
+none
+T}	T{
+authenticated
+T}	T{
+repokey, keyfile
+T}
+_
+T{
+BLAKE2b
+T}	T{
+n/a
+T}	T{
+authenticated\-blake2
+T}	T{
+repokey\-blake2,
+keyfile\-blake2
+T}
+_
+.TE
+.sp
+On modern Intel/AMD CPUs (except very cheap ones), AES is usually
+hardware\-accelerated.
+BLAKE2b is faster than SHA256 on Intel/AMD 64\-bit CPUs,
+which makes \fIauthenticated\-blake2\fP faster than \fInone\fP and \fIauthenticated\fP\&.
+.sp
+On modern ARM CPUs, NEON provides hardware acceleration for SHA256 making it faster
+than BLAKE2b\-256 there. NEON accelerates AES as well.
+.sp
+Hardware acceleration is always used automatically when available.
 .sp
 .sp
 \fIrepokey\fP and \fIkeyfile\fP use AES\-CTR\-256 for encryption and HMAC\-SHA256 for
 \fIrepokey\fP and \fIkeyfile\fP use AES\-CTR\-256 for encryption and HMAC\-SHA256 for
 authentication in an encrypt\-then\-MAC (EtM) construction. The chunk ID hash
 authentication in an encrypt\-then\-MAC (EtM) construction. The chunk ID hash
@@ -90,26 +139,24 @@ These modes are compatible with borg 1.0.x.
 \fIrepokey\-blake2\fP and \fIkeyfile\-blake2\fP are also authenticated encryption modes,
 \fIrepokey\-blake2\fP and \fIkeyfile\-blake2\fP are also authenticated encryption modes,
 but use BLAKE2b\-256 instead of HMAC\-SHA256 for authentication. The chunk ID
 but use BLAKE2b\-256 instead of HMAC\-SHA256 for authentication. The chunk ID
 hash is a keyed BLAKE2b\-256 hash.
 hash is a keyed BLAKE2b\-256 hash.
-These modes are new and \fInot\fP compatible with borg 1.0.x.
+These modes are new and \fInot\fP compatible with Borg 1.0.x.
 .sp
 .sp
 \fIauthenticated\fP mode uses no encryption, but authenticates repository contents
 \fIauthenticated\fP mode uses no encryption, but authenticates repository contents
-through the same keyed BLAKE2b\-256 hash as the other blake2 modes (it uses it
-as the chunk ID hash). The key is stored like repokey.
+through the same HMAC\-SHA256 hash as the \fIrepokey\fP and \fIkeyfile\fP modes (it uses it
+as the chunk ID hash). The key is stored like \fIrepokey\fP\&.
 This mode is new and \fInot\fP compatible with borg 1.0.x.
 This mode is new and \fInot\fP compatible with borg 1.0.x.
 .sp
 .sp
-\fInone\fP mode uses no encryption and no authentication. It uses sha256 as chunk
+\fIauthenticated\-blake2\fP is like \fIauthenticated\fP, but uses the keyed BLAKE2b\-256 hash
+from the other blake2 modes.
+This mode is new and \fInot\fP compatible with Borg 1.0.x.
+.sp
+\fInone\fP mode uses no encryption and no authentication. It uses SHA256 as chunk
 ID hash. Not recommended, rather consider using an authenticated or
 ID hash. Not recommended, rather consider using an authenticated or
 authenticated/encrypted mode.
 authenticated/encrypted mode.
-This mode is compatible with borg 1.0.x.
-.sp
-Hardware acceleration will be used automatically.
-.sp
-On modern Intel/AMD CPUs (except very cheap ones), AES is usually
-hardware\-accelerated. BLAKE2b is faster than SHA256 on Intel/AMD 64bit CPUs,
-which makes \fIauthenticated\fP faster than \fInone\fP\&.
-.sp
-On modern ARM CPUs, NEON provides hardware acceleration for SHA256 making it faster
-than BLAKE2b\-256 there.
+Use it only for new repositories where no encryption is wanted \fBand\fP when compatibility
+with 1.0.x is important. If compatibility with 1.0.x is not important, use
+\fIauthenticated\-blake2\fP or \fIauthenticated\fP instead.
+This mode is compatible with Borg 1.0.x.
 .SH OPTIONS
 .SH OPTIONS
 .sp
 .sp
 See \fIborg\-common(1)\fP for common options of Borg commands.
 See \fIborg\-common(1)\fP for common options of Borg commands.
@@ -127,6 +174,9 @@ select encryption key mode \fB(required)\fP
 .TP
 .TP
 .B \-a\fP,\fB  \-\-append\-only
 .B \-a\fP,\fB  \-\-append\-only
 create an append\-only mode repository
 create an append\-only mode repository
+.TP
+.B \-\-storage\-quota
+Set storage quota of the new repository (e.g. 5G, 1.5T). Default: no quota.
 .UNINDENT
 .UNINDENT
 .SH EXAMPLES
 .SH EXAMPLES
 .INDENT 0.0
 .INDENT 0.0

+ 32 - 14
docs/usage/init.rst.inc

@@ -72,6 +72,26 @@ the encryption/decryption key or other secrets.
 Encryption modes
 Encryption modes
 ++++++++++++++++
 ++++++++++++++++
 
 
++----------+---------------+------------------------+--------------------------+
+| Hash/MAC | Not encrypted | Not encrypted,         | Encrypted (AEAD w/ AES)  |
+|          | no auth       | but authenticated      | and authenticated        |
++----------+---------------+------------------------+--------------------------+
+| SHA-256  | none          | authenticated          | repokey, keyfile         |
++----------+---------------+------------------------+--------------------------+
+| BLAKE2b  | n/a           | authenticated-blake2   | repokey-blake2,          |
+|          |               |                        | keyfile-blake2           |
++----------+---------------+------------------------+--------------------------+
+
+On modern Intel/AMD CPUs (except very cheap ones), AES is usually
+hardware-accelerated.
+BLAKE2b is faster than SHA256 on Intel/AMD 64-bit CPUs,
+which makes `authenticated-blake2` faster than `none` and `authenticated`.
+
+On modern ARM CPUs, NEON provides hardware acceleration for SHA256 making it faster
+than BLAKE2b-256 there. NEON accelerates AES as well.
+
+Hardware acceleration is always used automatically when available.
+
 `repokey` and `keyfile` use AES-CTR-256 for encryption and HMAC-SHA256 for
 `repokey` and `keyfile` use AES-CTR-256 for encryption and HMAC-SHA256 for
 authentication in an encrypt-then-MAC (EtM) construction. The chunk ID hash
 authentication in an encrypt-then-MAC (EtM) construction. The chunk ID hash
 is HMAC-SHA256 as well (with a separate key).
 is HMAC-SHA256 as well (with a separate key).
@@ -80,23 +100,21 @@ These modes are compatible with borg 1.0.x.
 `repokey-blake2` and `keyfile-blake2` are also authenticated encryption modes,
 `repokey-blake2` and `keyfile-blake2` are also authenticated encryption modes,
 but use BLAKE2b-256 instead of HMAC-SHA256 for authentication. The chunk ID
 but use BLAKE2b-256 instead of HMAC-SHA256 for authentication. The chunk ID
 hash is a keyed BLAKE2b-256 hash.
 hash is a keyed BLAKE2b-256 hash.
-These modes are new and *not* compatible with borg 1.0.x.
+These modes are new and *not* compatible with Borg 1.0.x.
 
 
 `authenticated` mode uses no encryption, but authenticates repository contents
 `authenticated` mode uses no encryption, but authenticates repository contents
-through the same keyed BLAKE2b-256 hash as the other blake2 modes (it uses it
-as the chunk ID hash). The key is stored like repokey.
+through the same HMAC-SHA256 hash as the `repokey` and `keyfile` modes (it uses it
+as the chunk ID hash). The key is stored like `repokey`.
 This mode is new and *not* compatible with borg 1.0.x.
 This mode is new and *not* compatible with borg 1.0.x.
 
 
-`none` mode uses no encryption and no authentication. It uses sha256 as chunk
+`authenticated-blake2` is like `authenticated`, but uses the keyed BLAKE2b-256 hash
+from the other blake2 modes.
+This mode is new and *not* compatible with Borg 1.0.x.
+
+`none` mode uses no encryption and no authentication. It uses SHA256 as chunk
 ID hash. Not recommended, rather consider using an authenticated or
 ID hash. Not recommended, rather consider using an authenticated or
 authenticated/encrypted mode.
 authenticated/encrypted mode.
-This mode is compatible with borg 1.0.x.
-
-Hardware acceleration will be used automatically.
-
-On modern Intel/AMD CPUs (except very cheap ones), AES is usually
-hardware-accelerated. BLAKE2b is faster than SHA256 on Intel/AMD 64bit CPUs,
-which makes `authenticated` faster than `none`.
-
-On modern ARM CPUs, NEON provides hardware acceleration for SHA256 making it faster
-than BLAKE2b-256 there.
+Use it only for new repositories where no encryption is wanted **and** when compatibility
+with 1.0.x is important. If compatibility with 1.0.x is not important, use
+`authenticated-blake2` or `authenticated` instead.
+This mode is compatible with Borg 1.0.x.

+ 34 - 16
src/borg/archiver.py

@@ -2434,34 +2434,52 @@ class Archiver:
         Encryption modes
         Encryption modes
         ++++++++++++++++
         ++++++++++++++++
 
 
+        +----------+---------------+------------------------+--------------------------+
+        | Hash/MAC | Not encrypted | Not encrypted,         | Encrypted (AEAD w/ AES)  |
+        |          | no auth       | but authenticated      | and authenticated        |
+        +----------+---------------+------------------------+--------------------------+
+        | SHA-256  | none          | authenticated          | repokey, keyfile         |
+        +----------+---------------+------------------------+--------------------------+
+        | BLAKE2b  | n/a           | authenticated-blake2   | repokey-blake2,          |
+        |          |               |                        | keyfile-blake2           |
+        +----------+---------------+------------------------+--------------------------+
+
+        On modern Intel/AMD CPUs (except very cheap ones), AES is usually
+        hardware-accelerated.
+        BLAKE2b is faster than SHA256 on Intel/AMD 64-bit CPUs,
+        which makes `authenticated-blake2` faster than `none` and `authenticated`.
+
+        On modern ARM CPUs, NEON provides hardware acceleration for SHA256 making it faster
+        than BLAKE2b-256 there. NEON accelerates AES as well.
+
+        Hardware acceleration is always used automatically when available.
+
         `repokey` and `keyfile` use AES-CTR-256 for encryption and HMAC-SHA256 for
         `repokey` and `keyfile` use AES-CTR-256 for encryption and HMAC-SHA256 for
         authentication in an encrypt-then-MAC (EtM) construction. The chunk ID hash
         authentication in an encrypt-then-MAC (EtM) construction. The chunk ID hash
         is HMAC-SHA256 as well (with a separate key).
         is HMAC-SHA256 as well (with a separate key).
-        These modes are compatible with borg 1.0.x.
+        These modes are compatible with Borg 1.0.x.
 
 
         `repokey-blake2` and `keyfile-blake2` are also authenticated encryption modes,
         `repokey-blake2` and `keyfile-blake2` are also authenticated encryption modes,
         but use BLAKE2b-256 instead of HMAC-SHA256 for authentication. The chunk ID
         but use BLAKE2b-256 instead of HMAC-SHA256 for authentication. The chunk ID
         hash is a keyed BLAKE2b-256 hash.
         hash is a keyed BLAKE2b-256 hash.
-        These modes are new and *not* compatible with borg 1.0.x.
+        These modes are new and *not* compatible with Borg 1.0.x.
 
 
         `authenticated` mode uses no encryption, but authenticates repository contents
         `authenticated` mode uses no encryption, but authenticates repository contents
-        through the same keyed BLAKE2b-256 hash as the other blake2 modes (it uses it
-        as the chunk ID hash). The key is stored like repokey.
-        This mode is new and *not* compatible with borg 1.0.x.
+        through the same HMAC-SHA256 hash as the `repokey` and `keyfile` modes (it uses it
+        as the chunk ID hash). The key is stored like `repokey`.
+        This mode is new and *not* compatible with Borg 1.0.x.
 
 
-        `none` mode uses no encryption and no authentication. It uses sha256 as chunk
+        `authenticated-blake2` is like `authenticated`, but uses the keyed BLAKE2b-256 hash
+        from the other blake2 modes.
+        This mode is new and *not* compatible with Borg 1.0.x.
+
+        `none` mode uses no encryption and no authentication. It uses SHA256 as chunk
         ID hash. Not recommended, rather consider using an authenticated or
         ID hash. Not recommended, rather consider using an authenticated or
         authenticated/encrypted mode.
         authenticated/encrypted mode.
-        This mode is compatible with borg 1.0.x.
-
-        Hardware acceleration will be used automatically.
-
-        On modern Intel/AMD CPUs (except very cheap ones), AES is usually
-        hardware-accelerated. BLAKE2b is faster than SHA256 on Intel/AMD 64bit CPUs,
-        which makes `authenticated` faster than `none`.
-
-        On modern ARM CPUs, NEON provides hardware acceleration for SHA256 making it faster
-        than BLAKE2b-256 there.
+        Use it only for new repositories where no encryption is wanted **and** when compatibility
+        with 1.0.x is important. If compatibility with 1.0.x is not important, use
+        `authenticated-blake2` or `authenticated` instead.
+        This mode is compatible with Borg 1.0.x.
         """)
         """)
         subparser = subparsers.add_parser('init', parents=[common_parser], add_help=False,
         subparser = subparsers.add_parser('init', parents=[common_parser], add_help=False,
                                           description=self.do_init.__doc__, epilog=init_epilog,
                                           description=self.do_init.__doc__, epilog=init_epilog,

+ 15 - 6
src/borg/crypto/key.py

@@ -781,10 +781,7 @@ class Blake2RepoKey(ID_BLAKE2b_256, RepoKey):
     MAC = blake2b_256
     MAC = blake2b_256
 
 
 
 
-class AuthenticatedKey(ID_BLAKE2b_256, RepoKey):
-    TYPE = 0x06
-    NAME = 'authenticated BLAKE2b'
-    ARG_NAME = 'authenticated'
+class AuthenticatedKeyBase(RepoKey):
     STORAGE = KeyBlobStorage.REPO
     STORAGE = KeyBlobStorage.REPO
 
 
     # It's only authenticated, not encrypted.
     # It's only authenticated, not encrypted.
@@ -825,9 +822,21 @@ class AuthenticatedKey(ID_BLAKE2b_256, RepoKey):
         return data
         return data
 
 
 
 
+class AuthenticatedKey(AuthenticatedKeyBase):
+    TYPE = 0x07
+    NAME = 'authenticated'
+    ARG_NAME = 'authenticated'
+
+
+class Blake2AuthenticatedKey(ID_BLAKE2b_256, AuthenticatedKeyBase):
+    TYPE = 0x06
+    NAME = 'authenticated BLAKE2b'
+    ARG_NAME = 'authenticated-blake2'
+
+
 AVAILABLE_KEY_TYPES = (
 AVAILABLE_KEY_TYPES = (
     PlaintextKey,
     PlaintextKey,
     PassphraseKey,
     PassphraseKey,
-    KeyfileKey, RepoKey,
-    Blake2KeyfileKey, Blake2RepoKey, AuthenticatedKey,
+    KeyfileKey, RepoKey, AuthenticatedKey,
+    Blake2KeyfileKey, Blake2RepoKey, Blake2AuthenticatedKey,
 )
 )

+ 19 - 4
src/borg/testsuite/key.py

@@ -8,8 +8,9 @@ import msgpack
 import pytest
 import pytest
 
 
 from ..crypto.key import Passphrase, PasswordRetriesExceeded, bin_to_hex
 from ..crypto.key import Passphrase, PasswordRetriesExceeded, bin_to_hex
-from ..crypto.key import PlaintextKey, PassphraseKey, KeyfileKey, RepoKey, Blake2KeyfileKey, Blake2RepoKey, \
-    AuthenticatedKey
+from ..crypto.key import PlaintextKey, PassphraseKey, AuthenticatedKey, RepoKey, KeyfileKey, \
+    Blake2KeyfileKey, Blake2RepoKey, Blake2AuthenticatedKey
+from ..crypto.key import ID_HMAC_SHA_256, ID_BLAKE2b_256
 from ..crypto.key import TAMRequiredError, TAMInvalid, TAMUnsupportedSuiteError, UnsupportedManifestError
 from ..crypto.key import TAMRequiredError, TAMInvalid, TAMUnsupportedSuiteError, UnsupportedManifestError
 from ..crypto.key import identify_key
 from ..crypto.key import identify_key
 from ..crypto.low_level import bytes_to_long, num_aes_blocks
 from ..crypto.low_level import bytes_to_long, num_aes_blocks
@@ -70,12 +71,13 @@ class TestKey:
         return tmpdir
         return tmpdir
 
 
     @pytest.fixture(params=(
     @pytest.fixture(params=(
-        KeyfileKey,
         PlaintextKey,
         PlaintextKey,
+        AuthenticatedKey,
+        KeyfileKey,
         RepoKey,
         RepoKey,
         Blake2KeyfileKey,
         Blake2KeyfileKey,
         Blake2RepoKey,
         Blake2RepoKey,
-        AuthenticatedKey,
+        Blake2AuthenticatedKey,
     ))
     ))
     def key(self, request, monkeypatch):
     def key(self, request, monkeypatch):
         monkeypatch.setenv('BORG_PASSPHRASE', 'test')
         monkeypatch.setenv('BORG_PASSPHRASE', 'test')
@@ -256,6 +258,19 @@ class TestKey:
     def test_authenticated_encrypt(self, monkeypatch):
     def test_authenticated_encrypt(self, monkeypatch):
         monkeypatch.setenv('BORG_PASSPHRASE', 'test')
         monkeypatch.setenv('BORG_PASSPHRASE', 'test')
         key = AuthenticatedKey.create(self.MockRepository(), self.MockArgs())
         key = AuthenticatedKey.create(self.MockRepository(), self.MockArgs())
+        assert AuthenticatedKey.id_hash is ID_HMAC_SHA_256.id_hash
+        assert len(key.id_key) == 32
+        plaintext = b'123456789'
+        authenticated = key.encrypt(plaintext)
+        # 0x07 is the key TYPE, 0x0100 identifies LZ4 compression, 0x90 is part of LZ4 and means that an uncompressed
+        # block of length nine follows (the plaintext).
+        assert authenticated == b'\x07\x01\x00\x90' + plaintext
+
+    def test_blake2_authenticated_encrypt(self, monkeypatch):
+        monkeypatch.setenv('BORG_PASSPHRASE', 'test')
+        key = Blake2AuthenticatedKey.create(self.MockRepository(), self.MockArgs())
+        assert Blake2AuthenticatedKey.id_hash is ID_BLAKE2b_256.id_hash
+        assert len(key.id_key) == 128
         plaintext = b'123456789'
         plaintext = b'123456789'
         authenticated = key.encrypt(plaintext)
         authenticated = key.encrypt(plaintext)
         # 0x06 is the key TYPE, 0x0100 identifies LZ4 compression, 0x90 is part of LZ4 and means that an uncompressed
         # 0x06 is the key TYPE, 0x0100 identifies LZ4 compression, 0x90 is part of LZ4 and means that an uncompressed