Do not allow duplicate destinations in TarFileArchive.add_file()

The logic to ensure we have a proper tar record for each directory
in the path of files collected with addCopySpec*() ends up
generating lots of duplicate calls to add_file(). Filter them out
by checking for the existence of the new destination in
tarfile.getnames().

This reduces peak resident memory consumption by 50-75% and cuts
run times by 50-66%.

Related to Issue #81

1 files changed, 2 insertions, 0 deletions

diff --git a/sos/utilities.py b/sos/utilities.py
index 32e08012..a3859f1c 100644
--- a/sos/utilities.py
+++ b/sos/utilities.py
@@ -252,6 +252,8 @@ class TarFileArchive(Archive):
         else:
             dest = self.prepend(src)
 
+        if dest in self.tarfile.getnames():
+            return
         if src != '/':
             self.add_parent(src)