aboutsummaryrefslogtreecommitdiffstats
path: root/tests/report_tests
Commit message (Collapse)AuthorAgeFilesLines
* [tests] Add test for using `--encrypt-pass`Jake Hunsaker2021-06-031-0/+51
| | | | | | | | | | Adds two test cases for using `--encrypt-pass`; one with just the bare option and another in conjunction with `--clean` to ensure both can be used in the same execution. Resolves: #2566 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [tests] Improve failed command reporting in test outputJake Hunsaker2021-05-262-2/+2
| | | | | | | | | | | | | | | | | | | | Improves error reporting for failed sos commands by logging stderr (or stdout if stderr is not populated) to the console, which was previously being truncated by the builtin error handling of avocado. Printed output is limited to the last 8k to avoid dumping several MBs at a time for scenarios such as timeouts where command failure may generate significant logging prior to failing. Included with this are 2 minor changes to existing tests. First, remove verbose output from the expected plugins test to reduce otherwise irrelevant output for command failures. Second limit the number of plugins run for the LogLevelTest, both to reduce overall run time for a test where we aren't testing specific plugins and to improve readability of failures for such a test. Resolves: #2556 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [tests] Add test for priority parameter in add_cmd_output()Jake Hunsaker2021-05-241-0/+54
| | | | | | | | | Adds a test that checks that the new `priority` parameter is working as expected for well known and common collections. Resolves: #2553 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [report] Do not fail on skipping non-existing pluginsJake Hunsaker2021-05-181-1/+12
| | | | | | | | | | | | | | | | | | | Specifying a non-existing plugin to skip is no longer considered a fatal error. This was previously in place, in part, for third party utilities calling sos to ensure adherence to known plugins. However, since `collect` has been integrated into sos natively, this is far less of a concern. Specfying non-existing plugins will now only generate a warning message for plugins wanting to be skipped (`-n`). If non-existing plugins are specified for enablement via `-e` or `-o`, that will still generate a fatal error. Closes: #537 Closes: #1723 Resolves: #2517 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [archive] Don't shell out for compressing the archiveJake Hunsaker2021-05-121-0/+35
| | | | | | | | | | As sos is now python3-only, we can avoid shelling-out to compression utilities like `xz` or `gzip`, and instead use the method provided by the built-in `tarfile` module. Resolves: #2523 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [tests|logs] Stream test data to journal, change inspected journalJake Hunsaker2021-05-051-9/+13
| | | | | | | | | | | | Changes the logs pre_sos_setup() to write to the journal using a stream rather than writing several MB directly all at once. This fixes an issue where using the `journal.send()` method from the systemd module would result in no data being written to the journal during the test setup. Second, switch over to inspecting the full journal rather than the most recent boot journal. Related: #2499
* [tests] Correct test package installationJake Hunsaker2021-05-051-7/+6
| | | | | | | | | | | | | | | | First, correct the logs plugin stagetwo test to specify the needed package in a list, and second move the import to within the test class that needs it so that we don't fail on the stageone tests if the package is not present. Second, correct the journal injections that this plugin test does to use `pre_sos_setup()` instead of overriding `setup_mocking()`, and call the former after the mocking has been setup. Finally, add the resulting size of the collected journal if we fail the size requirements of the test. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [tests] Add test class for when exceptions are expected in an executionJake Hunsaker2021-05-031-0/+43
| | | | | | | | | | | | | | Adds a new `StageOneReportExceptionTest` class that is to be used for tests that should ensure an exception IS hit, for example when testing plugin enablement or invalid options. Included with this commit is a small set of plugin enablement tests. Future tests for component and plugin specific exception handling should be built using this class and design approach. Resolves: #2518 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [Plugin] Make command output size limiting obviousJake Hunsaker2021-04-301-0/+4
| | | | | | | | | | | | | | | | | | | | | When command output is size limited it should be made obvious that it is not the whole output of the command, beyond the collected output (most likely) starting in the middle of a string. To do this, we add a 'truncated' key to the results dict returned by `sos_get_command_output()`, which is set based on if the `AsyncReader` was filled during command execution. From that, first log if a command was truncated. Second, instead of dropping the collected output in the normal plugin directory, instead drop it into `sos_strings` with a `.tailed` extention to mimic how we handled tailed file collections. Finally, add a symlink from the plugin dir with the original filename that points to the `sos_strings` path. Closes: #1332 Resolves: #2506 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [logging] Fix archive debug logging, adjust verbosity levelsJake Hunsaker2021-04-281-1/+20
| | | | | | | | | | | | | | | | | Fixes an issue where archive debug logging was controlled by the use of `--debug` rather than `--verbose`. Removes a superfluous log in `FileCacheArchive.add_link()`. Also, adjusts the different verbosity levels to be more meaninful. Now, the use of `-v` will enable debug logging but will not print those messages to console, `-vv` will print debug logging to console, while `-vvv` will enable archive debug logging which is expected to be significant due to most file operations being logging at some point with archive debug logging. Resolves: #2507 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [process] Collect per-process files from /procJake Hunsaker2021-04-161-0/+29
| | | | | | | | | | | | | | | | | | | | Adds collection of per-process files from /proc/$pid, limited to a number of pids controlled by the new `process.numprocs` option. This option will default to the first 2048 processes. Using a value of `0` for this option will result in all pids being collected. In testing, the initial limit of 2048 process did not significantly raise collection time or archive size. Note however, that collection of 20k or more processes does show a significant increase in plugin execution time and archive size. This is however tempered, by the whole-plugin timeout that sos imposes. Related: #436 Closes: #542 Resolves: #1783 Resolves: #2498 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [tests] Add a test for default expected plugin enablementJake Hunsaker2021-04-151-1/+56
| | | | | | | | | | | | | | | | Adds a test case to ensure that the plugins we can expect to always be enabled when a "normal" `sos report` command is run are in fact enabled. First, test the distro-independent plugins that do not have specific enablement triggers (and thus should always be enabled). Second, have a distro-specific test for distro-specific plugins that are also expected to always run on those systems. Closes: #2365 Resolves: #2431 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [tests] Add distro specific helper decoratorsJake Hunsaker2021-04-151-2/+2
| | | | | | | | | | | | | | | | | | | Adds helper decorators to define specific test methods for use on specific distributions only. Currently two decorators are available: @redhat_only Only run on fedora, centos, or rhel @ubuntu_only Only run on ubuntu or debian Note that these decorators are only intended for individual `test_*` methods, and will not function to define distro-specific test classes. These should make it easier to write plugin test cases where packaging differences between distributions otherwise makes plugin tests either impossible needlessly complex to write generically. Related: #2431 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [tests] Set local distro in StageOne to allow distro-specific testsJake Hunsaker2021-04-151-1/+2
| | | | | | | | | | | | | | Moves the detection of the local system's distribution to StageOne, and defines two constants to reference RH and Ubuntu distribution lists to allow for distro-specific tests. Further, make the expected_warnings_displayed test skipped if the local test system is not a RH-family distro, as those warnings are not expected on non RH-family systems. Related: #2431 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [tests] Add a pre-setup setup method and smoke testsJake Hunsaker2021-04-151-0/+56
| | | | | | | | | | | | | | | | First, add a "pre-setup setup" method, in the form of `pre_sos_setup()`, that can be used in the way the traditional `setUp()` method would be used (but can't because that's our entry point for executing our sos runs). This method will be executed _prior_ to any mocking. Second, add a smoke test that enables all plugins that exist in the local branch being tested. This will test that doing so does not generate any exceptions and that some expected warnings from select plugins are displayed. Related: #2431 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [tests] Add plugin tests for basic functionalityJake Hunsaker2021-04-153-0/+137
| | | | | | | | | | | | Adds several plugin tests for plugins that can be expected to run on all distributions, in particular these tests are including mostly Stage Two tests. As such, these tests are only recommended to be run on test machines. Related: #2431 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [tests] Add a StageTwo timeout testJake Hunsaker2021-04-151-0/+44
| | | | | | Adds a test to ensure our timeout control is working properly. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [tests] Start using avocado for test suiteJake Hunsaker2021-04-155-0/+181
This commit represents the start of an overhaul of the test suite used by sos. Note that several more commits to follow will be required in order for the test suite to be considered stable. The new test suite will use the avocado-framework to build out new tests. This first part adopts a new 'stageX' naming scheme for our tests as follows: stage0 -> Unittests stage1 -> Basic function tests, no mocking allowed stage2 -> Mocked tests for specific scenarios/regressions stage3 -> Complex setups for layered products/environments At the moment, these unittests are not updated for avocado, though most should still work with `nosetest` directly. A new set of base classes is defined in tests/sos_tests.py which provide the foundation for actual tests cases. This approach entails new test cases subclassing a base class, such as the new `StageOneReportTest`, and setting the `sos_cmd` class attr to the _options_ for an sos report run. By default `sos report --batch` will be run, and targeted to the test job's directory as a tmpdir. Each sos command will be executed once, and all test_* methods within a test case that subclasses `StageOneReportTest` will be checked against the output of that execution. Note that this diverges from avocado's typical approach where each test_* method is performed against a brand new instance of the class (thus meaning any setup including our sos report run would normally be run fresh). However, after speaking with the avocado devel team, this is still seen as a valid pattern for the framework. The current organizational approach is to separate the tests by component rather than stage. For example. `tests/report_tests/` should hold any report-centric tests, and the `plugin_tests` directory therein should be used for plugin-specific tests. As of this commit, there are basic functionality tests under `tests/report_tests/` and a single plugin test under `tests/report_tests/plugin_tests/` to act as a POC. Further, there is a `tests/vendor_tests/` directory for organizing vendor-specific bug/feature tests that are not covered by the generic project-wide tests. A POC test from RHBZ1928628 is available with this commit. Note that in order for these tests to be run properly _without_ installing the current branch to the local system, you will need to run the following command: `PYTHONPATH=tests/ avocado run -t stageone tests/` Related: #2431 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>