Merge pull request #7805 from ThomasWaldmann/fwd-port-tam-cve-fixes-master

some fwd ports of the archive-TAM/CVE related code/docs

docs/changes_1.x.rst

@@ -29,27 +29,29 @@ places. Borg now considers archives without TAM as garbage or an attack.
 
 We are not aware of others having discovered, disclosed or exploited this vulnerability.
 
-Below, if we speak of borg 1.2.5, we mean a borg version >= 1.2.5 **or** a
+Below, if we speak of borg 1.2.6, we mean a borg version >= 1.2.6 **or** a
 borg version that has the relevant security patches for this vulnerability applied
 (could be also an older version in that case).
 
 Steps you must take to upgrade a repository:
 
-1. Upgrade all clients using this repository to borg 1.2.5.
+1. Upgrade all clients using this repository to borg 1.2.6.
    Note: it is not required to upgrade a server, except if the server-side borg
    is also used as a client (and not just for "borg serve").
 
-   Do **not** run ``borg check`` with borg 1.2.5 before completing the upgrade steps.
+   Do **not** run ``borg check`` with borg > 1.2.4 before completing the upgrade steps.
 
-2. Run ``borg info --debug <repository> 2>&1 | grep TAM | grep -i manifest``.
+2. Run ``BORG_WORKAROUNDS=ignore_invalid_archive_tam borg info --debug <repo> 2>&1 | grep TAM | grep -i manifest``.
 
    a) If you get "TAM-verified manifest", continue with 3.
    b) If you get "Manifest TAM not found and not required", run
       ``borg upgrade --tam --force <repository>`` *on every client*.
 
-3. Run ``borg list --format='{name} {time} tam:{tam}{NL}' <repository>``.
+3. Run ``BORG_WORKAROUNDS=ignore_invalid_archive_tam borg list --format='{name} {time} tam:{tam}{NL}' <repo>``.
    "tam:verified" means that the archive has a valid TAM authentication.
    "tam:none" is expected as output for archives created by borg <1.0.9.
+   "tam:none" is also expected for archives resulting from a borg rename
+   or borg recreate operation (see #7791).
    "tam:none" could also come from archives created by an attacker.
    You should verify that "tam:none" archives are authentic and not malicious
    (== have good content, have correct timestamp, can be extracted successfully).
@@ -57,13 +59,15 @@ Steps you must take to upgrade a repository:
    In low-risk, trusted environments, you may decide on your own risk to skip step 3
    and just trust in everything being OK.
 
-4. If there are no tam:non archives left at this point, you can skip this step.
-   Run ``borg upgrade --archives-tam <repository>``.
-   This will make sure all archives are TAM authenticated (an archive TAM will be added
-   for all archives still missing one).
-   ``borg check`` would consider TAM-less archives as garbage or a potential attack.
-   Optionally run the same command as in step 3 to see that all archives now are "tam:verified".
+4. If there are no tam:none archives left at this point, you can skip this step.
+   Run ``BORG_WORKAROUNDS=ignore_invalid_archive_tam borg upgrade --archives-tam <repo>``.
+   This will unconditionally add a correct archive TAM to all archives not having one.
+   ``borg check`` would consider TAM-less or invalid-TAM archives as garbage or a potential attack.
+   To see that all archives now are "tam:verified" run: ``borg list --format='{name} {time} tam:{tam}{NL}' <repo>``
 
+5. Please note that you should never use BORG_WORKAROUNDS=ignore_invalid_archive_tam
+   for normal production operations - it is only needed once to get the archives in a
+   repository into a good state. All archives have a valid TAM now.
 
 Vulnerability time line:
 
@@ -71,7 +75,8 @@ Vulnerability time line:
 * 2023-06-13...: Work on fixing the issue, upgrade procedure, docs.
 * 2023-06-30: CVE was assigned via Github CNA
 * 2023-06-30 .. 2023-08-29: Fixed issue, code review, docs, testing.
-* 2023-08-30: Released fixed version 1.2.5
+* 2023-08-30: Released fixed version 1.2.5 (broken upgrade procedure for some repos)
+* 2023-08-31: Released fixed version 1.2.6 (fixes upgrade procedure)
 
 .. _hashindex_set_bug:
 

src/borg/archive.py

@@ -1999,13 +1999,17 @@ class ArchiveChecker:
                 # are likely looking at an archive item here, also check the TAM authentication:
                 try:
                     archive, verified, _ = self.key.unpack_and_verify_archive(data, force_tam_not_required=False)
-                except IntegrityError:
+                except IntegrityError as integrity_error:
                     # TAM issues - do not accept this archive!
                     # either somebody is trying to attack us with a fake archive data or
                     # we have an ancient archive made before TAM was a thing (borg < 1.0.9) **and** this repo
                     # was not correctly upgraded to borg 1.2.5 (see advisory at top of the changelog).
                     # borg can't tell the difference, so it has to assume this archive might be an attack
                     # and drops this archive.
+                    name = archive.get(b"name", b"<unknown>").decode("ascii", "replace")
+                    logger.error("Archive TAM authentication issue for archive %s: %s", name, integrity_error)
+                    logger.error("This archive will *not* be added to the rebuilt manifest! It will be deleted.")
+                    self.error_found = True
                     continue
                 # note: if we get here and verified is False, a TAM is not required.
                 archive = ArchiveItem(internal_dict=archive)
@@ -2271,6 +2275,7 @@ class ArchiveChecker:
                     # when upgrading to borg 1.2.5, users are expected to TAM-authenticate all archives they
                     # trust, so there shouldn't be any without TAM.
                     logger.error("Archive TAM authentication issue for archive %s: %s", info.name, integrity_error)
+                    logger.error("This archive will be *removed* from the manifest! It will be deleted.")
                     self.error_found = True
                     del self.manifest.archives[info.name]
                     continue