浏览代码

Buffer: fix wrong thread-local storage use (#2951)

Buffer: delete support for multiple threads

Thread-local objects are a fairly complex footgun, so avoid them in
the first place. When a Compressor uses a Buffer (for example),
just create multiple Compressor instances for each thread, owned by
each thread. Minimize multiple ownership of objects across
threads if at all possible!
enkore 7 年之前
父节点
当前提交
f7ed5d7220
共有 1 个文件被更改,包括 6 次插入8 次删除
  1. 6 8
      src/borg/helpers/datastruct.py

+ 6 - 8
src/borg/helpers/datastruct.py

@@ -1,6 +1,3 @@
-import threading
-from collections import namedtuple
-
 from .errors import Error
 
 
@@ -11,7 +8,9 @@ class StableDict(dict):
 
 
 class Buffer:
-    """provide a thread-local buffer"""
+    """
+    Provides a managed, resizable buffer.
+    """
 
     class MemoryLimitExceeded(Error, OSError):
         """Requested buffer size {} is above the limit of {}."""
@@ -23,13 +22,12 @@ class Buffer:
         """
         assert callable(allocator), 'must give alloc(size) function as first param'
         assert limit is None or size <= limit, 'initial size must be <= limit'
-        self._thread_local = threading.local()
         self.allocator = allocator
         self.limit = limit
         self.resize(size, init=True)
 
     def __len__(self):
-        return len(self._thread_local.buffer)
+        return len(self.buffer)
 
     def resize(self, size, init=False):
         """
@@ -41,7 +39,7 @@ class Buffer:
         if self.limit is not None and size > self.limit:
             raise Buffer.MemoryLimitExceeded(size, self.limit)
         if init or len(self) < size:
-            self._thread_local.buffer = self.allocator(size)
+            self.buffer = self.allocator(size)
 
     def get(self, size=None, init=False):
         """
@@ -50,4 +48,4 @@ class Buffer:
         """
         if size is not None:
             self.resize(size, init)
-        return self._thread_local.buffer
+        return self.buffer