aboutsummaryrefslogtreecommitdiffstats
path: root/tests
Commit message (Collapse)AuthorAgeFilesLines
* [Policy] Add policy-controlled forbidden pathsJake Hunsaker2021-01-131-0/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This adds policy-controlled forbidden path checking, which should be the final part of implementing "global" forbidden paths. With this commit, policies may now add paths and glob matches for paths which should never be collected in any plugin. Combined with plugin-defined paths and user-defined paths already available, plugins should now be able to be properly restricted from sensitive collections. Note that the way this is implemented is that policies that define the `set_forbidden_paths()` classmethod *extend* this forbidden list as it is built from the subclass(es) that also define one. This way, "top-level" policies do not need to maintain independent copies of entire trees of paths just to add a few specific additional ones that are not forbidden within other policies. This initial commit adds paths that are either very well-known to be ones we should avoid, or are paths that have previously been part of reported issues where these paths/files should not be collected. Closes: #316 Closes: #796 Closes: #919 Closes: #1316 Resolves: #2360 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [PackageManager] Separate PackageManager from policiesJake Hunsaker2021-01-041-1/+2
| | | | | | | | | | | | Moves `PackageManager` out from `sos/policies/__init__.py` into a new `sos/policies/package_managers` subdir. Future commits will aim to canonicalize package manager subclasses for policies to use, and ease the creation of new reusable package managers. Related: #2349 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [policies] Move linux policies into new subdirJake Hunsaker2021-01-042-2/+4
| | | | | | | | | | | Moves the actual policy files for supported distributions out of `sos/policies` and into a new subdir `sos/policies/distros`. Note that `Policy()` still lives in the former, while `LinuxPolicy()` has been moved into `sos/policies/distros/__init__.py` Related: #2349 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [tests] Add test for newly loaded kernel modulesJake Hunsaker2020-12-111-2/+10
| | | | | | | | | | | | | | | Adds a test to `simple.sh` to check if we loaded any new kernel modules during a normal exeuction. This test will be somewhat limited in the fact that this currently only tests the plugins that are loaded by default on each tested distribution. Also included is a minor typo fix for reporting where IP addresses were not obfuscated, if any. This typo did not affect the actual test however. Resolves: #2326 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [tests] Update simple.sh test suiteJake Hunsaker2020-11-191-30/+171
| | | | | | | | | | | | | This patch overhauls tests/simple.sh to provide more granular details on why a test run may have failed, beyond checking for a non-zero exit code or if output was written to stderr. This should also serve as another step towards more easily extendible tests for our automated processes. Resolves: #2305 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [report] Allow users to specify commands and files to skipJake Hunsaker2020-11-032-0/+4
| | | | | | | | | | | | | | | Adds two new options, `--skip-commands` and `--skip-files`, that allow users to selectively skip specific command or file collection instead of having to disable whole plugins to skip those collections. These options are also exposed via `sos collect`, being gated by a version of 4.1 since that is the next scheduled release where we can guarantee this functionality will be present. Closes: #2203 Resolves: #2271 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [general] flake8 first passBryan Quigley2020-08-135-11/+5
| | | | | Signed-off-by: Bryan Quigley <code@bryanquigley.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [travis,style] Upgrade to 20.04 and cover more in testsBryan Quigley2020-08-137-51/+80
| | | | | Signed-off-by: Bryan Quigley <code@bryanquigley.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [tests] Update sos.conf locationJake Hunsaker2020-07-211-4/+5
| | | | | | Updates the check for sos.conf with the new location Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [cleaner] Add user-provided keyword obfuscationJake Hunsaker2020-06-171-0/+20
| | | | | | | | | Adds a new parser and map to allow user defined keyword obfuscation. Users may now use the `--keywords` option to have `SoSCleaner` scan lines for matching keywords, and replace them in place like we do for other parsers. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [cleaner] Add hostname parserJake Hunsaker2020-06-171-0/+40
| | | | | | | | | | | | Adds a hostname parser to `sos clean` that will attempt to obfuscate FQDNs matching the hostname of the system that generated the sosreport, as found in sos_commands/host/hostname. Additionally, any domains added via the `--domains` option will also be obfuscated, including any subdomains of the domain(s) specified by the option. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [tests] Add tests for SoSCleanerJake Hunsaker2020-06-172-0/+98
| | | | | | Adds tests for the mappings and parsers used by SoSCleaner Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [archive] Drop bzip2Bryan Quigley2020-05-261-1/+1
| | | | | | | | | | | | | | | | | | This just leaves xz with gzip as the automatic backup. Bzip2 took the most time by far and only does marginally better than gzip. time 27 Size 10.72MiB --batch -z xz --log-size 1 time 27 Size 16.32MiB --batch -z gzip --log-size 1 time 40 Size 14.17MiB --batch -z bzip2 --log-size 1 time 43 Size 105.21MiB --batch -z xz --all-logs time 64 Size 187.38MiB --batch -z gzip --all-logs time 135 Size 182.86MiB --batch -z bzip2 --all-logs Resolves: #2053 Signed-off-by: Bryan Quigley <bryan.quigley@canonical.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [tests] Add Timing, summary, and reduce IO for small filesBryan Quigley2020-05-041-9/+20
| | | | | | | | | | | | Timing - see how long each test takes Summary - output the key results at the end Use /dev/shm to store temporary files Make since command actually meaningful Resolves: #2039 Signed-off-by: Bryan Quigley <bryan.quigley@canonical.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [plugins] Convert docstrings to class attrsJake Hunsaker2020-04-301-1/+1
| | | | | | | | | | Converts all current plugin docstrings into a `short_desc` attribute, that is now referenced by `Plugin.get_description()`. Closes: #1960 Resolves: #2036 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [tests] Fix unittestsJake Hunsaker2020-04-212-3/+3
| | | | | | | | | | Fixes a unittest failure due to an attempt to pass print() a `file` keyword for Red Hat, which is no longer accepted in python3 Additionally fixes 2 string related errors in the test suite since changing from python2 to python3. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [global] Drop use of python six and other 2.X bitsBryan Quigley2020-04-093-15/+6
| | | | | | Resolves: #2006 Signed-off-by: Bryan Quigley <bryan.quigley@canonical.com>
* [travis] Update test script for new binaryJake Hunsaker2020-04-081-9/+9
| | | | | | | Updates the test script `simple.sh` used by Travis to use the new `sos` binary with the `report` subcommand. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [reporting] Move reporting under sos/reportJake Hunsaker2020-04-081-1/+1
| | | | | | Moves the HTML/JSON reporting bits under sos/report. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [global] Fix report import pathsJake Hunsaker2020-04-073-3/+3
| | | | | | | | | | | Updates the uses of 'from sos.plugins' to 'from sos.report.plugins' in order to fix imports across the project with the new tree organization. Additionally, the legacy `sosreport` binary now injects the user's current working directory into the path the python interpreter uses in order to allow local execution from a git checkout. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [Plugin] Add new method for iterating over storage devicesJake Hunsaker2020-04-072-20/+40
| | | | | | | | | | | | | | | | | | | | | | | Adds a new mechanism for iterating over a list of devices with a given command or list of commands. This is to allow us to stop doing ad-hoc device enumeration in multiple plugins and instead provide a consistent behavior for iterating over devices such as disks. Before setting up plugins, sos will now enumerate a list of block devices and fibre devices that is made available to the plugins. From there the new 'add_blockdev_cmd()' method may be passed a command or list of commands that will be iterated over for every device. Commands passed need to include '%(dev)s' in the appropriate place for device name substitution. By default this will iterate over all discovered block devices. The 'devices' parameter may be used to change this, either by passing 'fibre' to instead iterate over fibre devices, or by passing a list of devices to use directly. To filter out or restrict commands to running on specific types of devices, the blacklist and whitelist parameters can be used respectively and can accept either a single regex string or a list of regexes. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [Policy] Add Container Runtime abstraction classJake Hunsaker2020-04-071-32/+70
| | | | | | | | | | | | | | | | | Adds a `ContainerRuntime()` class to allow policies to specify a container runtime to allow plugins to utilize. The `ContainerRuntime` is intended to allow for the discovery of containers, including specific ones by name, and for the execution of commands inside those containers. This is meant to remove the overhead of manually defining ways to determine an active runtime and if a component is containerized within plugins. Related: #1866 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [tests] Make simple.sh consistentBryan Quigley2020-03-181-9/+1
| | | | | | | | | | | We no longer have to workaround an odd LXD/Travis interaction so make simple.sh not treat Travis special. Closes: #1921 Resolves: #1983 Signed-off-by: Bryan Quigley <bryan.quigley@canonical.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [tests] add license header to source filesBryn M. Reeves2020-02-269-0/+63
| | | | | | | Resolves: #1958 Signed-off-by: Bryn M. Reeves <bmr@redhat.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [tests] delete random shebangsBryn M. Reeves2020-02-264-8/+0
| | | | | | | | | | | | | Various unit test source files in tests/ have unnecessary shell "shebang" notation specifying a particular Python interpereter. Since these files are executed under the test suite runner there is no need for them to include these lines. Related: #1949 Signed-off-by: Bryn M. Reeves <bmr@redhat.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [tests] remove obsolete test_exe.py scriptBryn M. Reeves2020-02-261-4/+0
| | | | | | | Related: #1949 Signed-off-by: Bryn M. Reeves <bmr@redhat.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [tests] use echo instead of py script in test_{output,shell_out}()Bryn M. Reeves2020-02-261-4/+2
| | | | | | | Related: #1949 Signed-off-by: Bryn M. Reeves <bmr@redhat.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [tests] compare correct string types in test_nonexe_file()Bryn M. Reeves2020-02-261-1/+1
| | | | | | | | | Use a byte string instead of a regular string. Related: #1949 Signed-off-by: Bryn M. Reeves <bmr@redhat.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [tests] look for 'true' in test_exe_file() instead of py scriptBryn M. Reeves2020-02-261-2/+1
| | | | | | | | | | Use the executable 'true' when testing is_executable() on a non-absolute path. Related: #1949 Signed-off-by: Bryn M. Reeves <bmr@redhat.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [policies] fix LinuxPolicy performance regression (test suite)Bryn M. Reeves2020-02-142-21/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Commit 285873a introduces a regression in the performance of LinuxProfile class initialisation: commit 285873a4f753822a88d475a1b030ab622bf4c72e Author: Bryan Quigley <bryan.quigley@canonical.com> Date: Tue Feb 11 15:03:16 2020 -0800 [policies] Detect systemd use instead of hardcoding it All the patch does is to switch from testing self.init to determine whether to use the SystemdInit() class or the basic InitSystem() one, to looking for '/run/systemd/system' in the file system. This has more broad side effects than it might seem since the test suite uses LinuxPolicy() as a mock policy object for a large number of tests. Since SystemdInit() calls out to systemctl to obtain the state of the init system this both increases the run time for the tests and causes high resource use in systemd itself: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 1 root 20 0 177196 13504 4512 R 92.7 0.1 5:24.04 systemd This causes the run time for the Python2 and 3 test suite (plus pep8/pycodestyle) to grow from ~5s to over 1m20s on my test system: 285873a~1 real 0m5.683s 285873a real 1m20.353s Allow direct users of LinuxPolicy to override the init system detection by specifying an init= kwarg initialised to an InitSystem-like object directly, and use this in the test suite to avoid the cost of initialising the full SystemdInit() class. Resolves: #1953 Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [travis] Move to single test script and use travis job matrixBryan Quigley2020-02-041-0/+111
| | | | | | | | | | | | | | | | | | | | | | Move all the sosreport runs to a simple bash script: tests/simple.sh so it can be called multiple times. This also makes it possible to run locally. The travis job matrix let's us have 2 Ubuntu native sos runs against installed python, while letting us to also continue to do the old style python runs. Ideally, we can utilize this to do Fedora/CentOS/RHEL jobs in the future. Python 3.4 testing was also dropped - as it seems to not be as well used (many test failures due to not being available). Python 3.8 testing was added. pycodestyle will only run once with the bionic version. Resolves: #1896 Signed-off-by: Bryan Quigley <bryan.quigley@canonical.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [Plugin] Allow selectively disabling postprocessingJake Hunsaker2019-11-191-0/+22
| | | | | | | | | | | | | | | | | | Adds two mechanisms by which users can choose to disable postprocessing of collected information. First, is a global method exposed via the `--no-postproc` option. Using this option will skip postprocessing for all plugins. Second, is a per-plugin option exposed via a new 'postproc' plugin option. This is set to _True_ by default (meaning yes, perform postprocessing), which users can set to False or off to disable postprocessing for that plugin only; e.g. `-k podman.postproc=off` Closes: #286 Resolves: #1862 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [tests] fix test_copy_dir_forbidden_pathBryn M. Reeves2019-11-011-1/+2
| | | | | | | | | | Rather than call just Plugin.setup() and Plugin._do_copy_path(), add an add_copy_spec() call to the mock plugin setup() method, and invoke copying by calling the Plugin.collect() method. Related: #1845 Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [sosreport,reporting] replace HTML reports by Report subclassPavel Moravec2019-08-251-21/+52
| | | | | | | | | | | | | | | Current HTML report generation is slow and its implementation is full of scattered html code. We shall utilize Report class instead. Additionally, add JSON report format for easy automated parsing. Further, fixed decoding of unicode content of a report. Fixed report_tests suite including some pycodestyle errors. Resolves: #1713 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [sosreport] grab logs since datePavel Moravec2019-08-251-0/+1
| | | | | | | | | | | | Adds a --since switch that takes a date as an argument. This switch will skip the archive files with a mtime older than the date. Also, --since affects journalctl execution for --all. Resolves: #1678 Signed-off-by: David Vallee Delisle <dvd@redhat.com> Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [SoSPredicate] override kernel module tests if --allow-system-changesBryn M. Reeves2019-08-162-0/+2
| | | | | | | | | | If the --allow-system-changes option is in effect, bypass checking of kernel modules and allow commands to attempt to auto-load any missing modules when run. Related: #1678 Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [archive] Handle checking container sysroot in _make_leading_pathsPavel Moravec2019-07-091-1/+2
| | | | | | | | | | | | | | | Previously, in _make_leading_paths(), checking host file paths did not account for non / sysroots, for situations where sos is run in a container and the host's / is actually mounted under /host for example. This would lead to copy errors when trying to copy symlinks. This method now will use sysroot if one is set, thus avoiding copy errors. Resolves: #1705 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com> Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [sos] Implement sosreport --dry-runBryn M. Reeves2019-03-222-22/+40
| | | | Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [Plugin] clean up Plugin.get_option()Bryn M. Reeves2018-12-121-12/+4
| | | | | | | | | | | | | There's a lot of ancient junk in this method (and associated code strewn around sos.sosreport and tests). Remove the ability to pass a list of options to the method since nothing uses this, and also delete the incomplete implementation of global plugin options via the commons dictionary (this work was already completed some time ago by mapping these options directly to the command line args). Resolves: #1498 Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [sosreport] Add mechanism to encrypt final archiveJake Hunsaker2018-07-121-1/+2
| | | | | | | | | | | | | | | | | | | | | Adds an option to encrypt the resulting archive that sos generates. There are two methods for doing so: --encrypt-key Uses a key-pair for asymmetric encryption --encrypt-pass Uses a password for symmetric encryption For key-pair encryption, the key-to-be-used must be imported into the root user's keyring, as gpg does not allow for the use of keyfiles. If the encryption process fails, sos will not abort as the unencrypted archive will have already been created. The assumption being that the archive is still of use and/or the user has another means of encrypting it. Resolves: #1320 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com> Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [archive] fix add_string()/do_*_sub() regressionBryn M. Reeves2018-07-021-10/+2
| | | | | | | | | | | | | A change in the handling of add_string() operations in the archive class causes the Plugin string substitution methods to fail (since the archive was enforcing a check that the path did not already exist - for substitutions this is always the case). Maintain the check for content that is being copied into the archive anew, but make the add_string() method override this and disable the existence checks. Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [Plugin] set size limits automaticallyBryn M. Reeves2018-06-201-0/+1
| | | | | | | | | | | If the caller does not specific an explicit size limit (or 0 to disable limits) automatically set the sizelimit argument for the add_copy_spec(), add_cmd_output(), and add_journal() methods to the value of the "log_size" option. Resolves: #1325 Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [archive] use threads for xz compressionPavel Moravec2018-06-181-1/+2
| | | | | | | | | | Moving also building of the command from Archive to Plugin class. Closes: #1196 Resolves: #1338 Signed-off-by: Pavel Moravec <pmoravec@redhat.com> Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [archive] make FileCacheArchive methods re-entrantBryn M. Reeves2018-06-181-2/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Since plugins are now run concurrently the Archive class must be re-entrant: different plugins may attempt to create the same paths in the archive, and since they are executing in parallel and with no locking, there exists a TOCTOU race between the use of os.path.{exists,isdir, etc.}() and the subsequent creation of a new path. Address this by implementing a simple monitor-style locking scheme for the FileCacheArchive class. A `_path_lock` member is added to the class and this must be held across any operation that modifies the namespace of the archive (including file, directory, symlink, and special node creation). Additionally, the `_check_path()` Archive method (which checks for the existence of path components up to the basename of the file, creating directories as required) is extended to also carry out a test for the existence of the path, and if the path does exist, that the object is of the expected type. A mismatch in the object type generates a ValueError with an appropriate string description of the problem. Benchmarking shows a consistent small improvement with the patch: this is a result of reduced redundant copy operations that the previous archive structure allowed (since duplicate operations are now aborted as soon as the _path_lock is acquired). Resolves: #1340 Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [Plugin] Make Plugin.policy an object instead of a functionJake Hunsaker2018-04-172-14/+28
| | | | | | | | | | Moves Plugin.policy to be an object for plugins instead of a function, making it easier to leverage the active policy within sos plugins. Resolves: #1179 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com> Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [plugins] Dont apply sizelimit if --all-logs option usedPavel Moravec2017-11-011-1/+1
| | | | | | When --all-logs is used, ignore any sizelimits in add_copy_spec. Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [tests] update test_exe.py for python3Sandro Bonazzola2017-03-311-1/+1
| | | | | | | test_exe.py fails with nosetests-3. Updated to python 3 compatible syntax. Signed-off-by: Sandro Bonazzola <sbonazzo@redhat.com>
* [plugins] move the glob tests togetherMichael Adam2017-03-281-4/+4
| | | | Signed-off-by: Michael Adam <obnox@samba.org>
* [plugins] remove duplicate plugin testsMichael Adam2017-03-281-11/+0
| | | | Signed-off-by: Michael Adam <obnox@samba.org>
* [plugins] let all copy_spec tests use add_copy_spec()Michael Adam2017-03-281-9/+9
| | | | | | instead of add_copy_spec_limit() Signed-off-by: Michael Adam <obnox@samba.org>
generated by cgit (git 2.34.1) at 2024-11-16 10:44:38 +0000