[cleaner] Replace encoding errors when opening files

A scenario was found that if a file had encoding issues when `sos clean`
went to open the file for obfuscation, we would stop processing the file
but leave it in the archive, which had the potential to leave
unobfuscated information in that file in the archive.

Fix this, by using the `errors='replace'` parameter when opening archive
files. This allows us to continue parsing the file normally, while
replacing the problematic characters with `?`s.

Closes: #3015

Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>

1 files changed, 20 insertions, 0 deletions

diff --git a/tests/test_data/fake_plugins/unicode_test.py b/tests/test_data/fake_plugins/unicode_test.py
new file mode 100644
index 00000000..541dfb5c
--- /dev/null
+++ b/tests/test_data/fake_plugins/unicode_test.py
@@ -0,0 +1,20 @@
+# This file is part of the sos project: https://github.com/sosreport/sos
+#
+# This copyrighted material is made available to anyone wishing to use,
+# modify, copy, or redistribute it subject to the terms and conditions of
+# version 2 of the GNU General Public License.
+#
+# See the LICENSE file in the source distribution for further information.
+
+from sos.report.plugins import Plugin, IndependentPlugin
+
+
+class UnicodeTest(Plugin, IndependentPlugin):
+    """Fake plugin to test the handling of a file with embedded unicode
+    """
+
+    plugin_name = 'unicode_test'
+    short_desc = 'Fake plugin to test unicode file handling'
+
+    def setup(self):
+        self.add_copy_spec('/tmp/sos-test-unicode.txt')