| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Consolidate testing, so that things can be run from CLI, CI and
various other means in a consistent way by using tox.
* Updated all the flake8 tests in the tests folder.
* Added pylint as a neutral test so that we can work on this in stages
and have some collaboration on what we test and don't
* The tox tests for unit, stageone, stagetwo testing makes is easier
for users to know how to run tests, and not have to do things
manually
* Using tox for CI doesn't make sense, as that will create virtual
envs and will disregard system/snap based python modules so may not
work
Signed-off-by: Arif Ali <arif.ali@canonical.com>
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Fix the issues reported by the tests on the new version of ubuntu
Resolves: #3587
Closes: #3588
Related: SET-594,SET-595
Co-authored-by: David Negreira <david.negreira@canonical.com>
Signed-off-by: Arif Ali <arif.ali@canonical.com>
Signed-off-by: David Negreira <david.negreira@canonical.com>
|
| |
|
|
|
|
|
|
|
|
| |
Commits a downstream test from RHEL to ensure that a given release has a
consistent version string in the UI report, manifest, and most
importantly for the package nvr for an sos package built from the
current branch. Note that this test is only implemented for RPM
installations at this time.
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
After all plugins have run, sos will now generate a "tag summary" and
add it to the report section of the manifest. This summary will be a
json-formatted entry that details all collections within the report that
have any tag associated with them at all. This should allow for easier
parsing of the manifest by analyzers such as Red Hat Insights.
As part of this change, commands will no longer be automatically tagged
with the name of the binary used in the command collection.
Additionally, manual collections performed by a plugin's `collect()`
method will now be recorded in the manifest in the same manner as file
and command output collections.
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
| |
|
|
|
|
|
|
|
|
| |
The debug level messages gated by `-v` are very helpful for diagnosing
test failures, but currently not all tests specify the use of verbosity.
Make use of verobsity a default parameter for all test runs to address
this.
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Improves error reporting for failed sos commands by logging stderr (or
stdout if stderr is not populated) to the console, which was previously
being truncated by the builtin error handling of avocado. Printed output
is limited to the last 8k to avoid dumping several MBs at a time for
scenarios such as timeouts where command failure may generate
significant logging prior to failing.
Included with this are 2 minor changes to existing tests. First, remove
verbose output from the expected plugins test to reduce otherwise
irrelevant output for command failures. Second limit the number of
plugins run for the LogLevelTest, both to reduce overall run time for a
test where we aren't testing specific plugins and to improve readability
of failures for such a test.
Resolves: #2556
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fixes an issue where archive debug logging was controlled by the use of
`--debug` rather than `--verbose`. Removes a superfluous log in
`FileCacheArchive.add_link()`.
Also, adjusts the different verbosity levels to be more meaninful. Now,
the use of `-v` will enable debug logging but will not print those
messages to console, `-vv` will print debug logging to console, while
`-vvv` will enable archive debug logging which is expected to be
significant due to most file operations being logging at some point
with archive debug logging.
Resolves: #2507
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
This commit represents the start of an overhaul of the test suite used
by sos. Note that several more commits to follow will be required in
order for the test suite to be considered stable.
The new test suite will use the avocado-framework to build out new
tests.
This first part adopts a new 'stageX' naming scheme for our tests as
follows:
stage0 -> Unittests
stage1 -> Basic function tests, no mocking allowed
stage2 -> Mocked tests for specific scenarios/regressions
stage3 -> Complex setups for layered products/environments
At the moment, these unittests are not updated for avocado, though most
should still work with `nosetest` directly.
A new set of base classes is defined in tests/sos_tests.py which provide
the foundation for actual tests cases. This approach entails new test
cases subclassing a base class, such as the new `StageOneReportTest`,
and setting the `sos_cmd` class attr to the _options_ for an sos report
run. By default `sos report --batch` will be run, and targeted to the
test job's directory as a tmpdir.
Each sos command will be executed once, and all test_* methods within a
test case that subclasses `StageOneReportTest` will be checked against
the output of that execution. Note that this diverges from avocado's
typical approach where each test_* method is performed against a brand
new instance of the class (thus meaning any setup including our sos
report run would normally be run fresh). However, after speaking with
the avocado devel team, this is still seen as a valid pattern for the
framework.
The current organizational approach is to separate the tests by
component rather than stage. For example. `tests/report_tests/` should
hold any report-centric tests, and the `plugin_tests` directory therein
should be used for plugin-specific tests. As of this commit, there are
basic functionality tests under `tests/report_tests/` and a single
plugin test under `tests/report_tests/plugin_tests/` to act as a POC.
Further, there is a `tests/vendor_tests/` directory for organizing
vendor-specific bug/feature tests that are not covered by the generic
project-wide tests. A POC test from RHBZ1928628 is available with this
commit.
Note that in order for these tests to be run properly _without_
installing the current branch to the local system, you will need to run
the following command:
`PYTHONPATH=tests/ avocado run -t stageone tests/`
Related: #2431
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
