| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
As CentOS Stream now runs ahead of RHEL, instead of behind, using it as
a testing distribution on GCE allows us to have more confidence on those
distributions than testing on Fedora alone.
As CentOS Stream does not have version locks, there is only one version
to test on, rather than two as is the case with Fedora and Ubuntu. The
image we build from will need to be updated more regularly however as
new versions are released following RHEL minor version releases.
Resolves: #2499
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Adds a new plugin for openstack deployments that collects information
about/from the underlying database for the deployment. This plugin
crosses multiple openstack components (for which there is typically one
component for each), since that is preferable to adding a host of
database-related options to each individual plugin.
By default, this plugin will only collect the output of `clustercheck`.
When using either the `dump` or `dumpall` plugin options, this plugin
will also collect a mysql dump of the database; either for specific
tables or all tables present respectively.
Closes: #1256
Closes: #1984
Resolves: #2519
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
Having just one whitespace before "Found key:" failed to match
the first group.
Resolves: #2521
Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Adds a new `StageOneReportExceptionTest` class that is to be used for
tests that should ensure an exception IS hit, for example when testing
plugin enablement or invalid options.
Included with this commit is a small set of plugin enablement tests.
Future tests for component and plugin specific exception handling should
be built using this class and design approach.
Resolves: #2518
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
Pulp-3 / pulpcore as a revolution from pulp-2 needs a separate
plugin, since both plugins have nothing in common and there might
be deployments where is active both pulp-2 and pulp-3.
Resolves: #2278
Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
Like we do for foreman and pulp, avoid collecting logs from the various
openstack components that write to /var/log/httpd/*, instead allowing
those plugins to set the proper collections.
Closes: #1950
Resolves: #2516
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
When using `reporting.py`, e.g. for `sos.txt`, sort the content on a
per-section basis so that comparison between multiple sos reports is
easier.
Closes: #1998
Resolves: #2515
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
| |
The `collect` component supports `--log-size` as a passthru option for
report. Add it to the `collect` manpage for completeness.
Related: #2510
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
Updates the description of `--log-size` in the manpage for `report` so
that it accurately reflects how size limitations are determined and
imposed during collections.
Closes: #1900
Resolves: #2513
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When command output is size limited it should be made obvious that it is
not the whole output of the command, beyond the collected output (most
likely) starting in the middle of a string.
To do this, we add a 'truncated' key to the results dict returned by
`sos_get_command_output()`, which is set based on if the `AsyncReader`
was filled during command execution.
From that, first log if a command was truncated. Second, instead of
dropping the collected output in the normal plugin directory, instead
drop it into `sos_strings` with a `.tailed` extention to mimic how we
handled tailed file collections. Finally, add a symlink from the plugin
dir with the original filename that points to the `sos_strings` path.
Closes: #1332
Resolves: #2506
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
Standardizes the `assertOutputContains` and `assertOutputNotContains`
methods to use `re.search()`, instead of separate parsing methods.
`search()` replaces `match()` for efficiency purposes against the
collected output.
Related: #2507
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fixes an issue where archive debug logging was controlled by the use of
`--debug` rather than `--verbose`. Removes a superfluous log in
`FileCacheArchive.add_link()`.
Also, adjusts the different verbosity levels to be more meaninful. Now,
the use of `-v` will enable debug logging but will not print those
messages to console, `-vv` will print debug logging to console, while
`-vvv` will enable archive debug logging which is expected to be
significant due to most file operations being logging at some point
with archive debug logging.
Resolves: #2507
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
The insights-client command should not be run within sosreport,
as it collects data sosreport already collects and is not cautious
about how it does so. Running it is therefore superfluous.
Closes: #2508
Resolves: #2509
Signed-off-by: Paul Wayper <paulway@redhat.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Refactor the collections for the different httpd versions collected by
the RedHatApache instance of this plugin, to remove the repetition of
similar collection points.
Moving forward, collection of newer httpd versions will only need to
specify the version number, rather than explicitly repeat the same file
collection paths.
Closes: #704
Resolves: #2505
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
Collect a verbose curl command output, run against RHSM server like
subscription-manager does. This helps troubleshooting with RHSM proxy
issues.
Resolves: #2503
Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Chrony 4.0 added better support for fragmented
configuration.
Chrony's configuration and NTP sources can now
be under:
NTP sources:
/etc/chrony/sources.d/
Chrony's configuration:
/etc/chrony/conf.d/
Resolves: #2501
Signed-off-by: Eric Desrochers <eric.desrochers@canonical.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Adds collection of per-process files from /proc/$pid, limited to a
number of pids controlled by the new `process.numprocs` option. This
option will default to the first 2048 processes. Using a value of `0`
for this option will result in all pids being collected.
In testing, the initial limit of 2048 process did not significantly
raise collection time or archive size. Note however, that collection of
20k or more processes does show a significant increase in plugin
execution time and archive size. This is however tempered, by the
whole-plugin timeout that sos imposes.
Related: #436
Closes: #542
Resolves: #1783
Resolves: #2498
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
Fixes an exception propagation from `cleanup()` where an attempt to look
for and remove a checksum file was made when an archive was not
generated (and thus no checksum would be present either).
Related: #2488
Resolves: #2494
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A redundant file skipping check was being executed within
`obfuscate_line()` that would cause subsequent archives being obfuscated
to skip line obfuscation within a file, despite iterating through the
entire file.
Remove this redundant check, thus allowing proper obfuscation.
Closes: #2490
Related: RHBZ#1930181
Resolves: #2492
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Previously, maps were being prepped via archives after extraction. This
reduced the amount of file IO being done, but made it so that necessary
obfuscations from later archives in a series would not be obfuscated in
the archives obfuscated before those later archives were extracted.
Fix this by extracting the map prep files into memory for each archive
to prep the maps before we enter the obfuscation loop entirely.
Closes: #2490
Related: RHBZ#1930181
Resolves: #2492
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Plugin version never really took off. A very few plugins
had it configured (e.g. jars, pacemaker)
It also generated unhelpful noise in version.txt.
Closes: #2495
Resolves: #2496
Signed-off-by: Eric Desrochers <eric.desrochers@canonical.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
On certain kernel configuration, devlink cmds may trigger
the module to load automatically.
Closes: #2468
Resolves: #2487
Signed-off-by: Eric Desrochers <eric.desrochers@canonical.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This commit adds collection of `ipmitool isel elist` and adds verbosity to
the regular `ipmitool sel list` collection.
Additionally, simplify the base command construction logic, and pull in
`ipmitool lan print` from #998. Note that network obfuscation is
relegated to the use of `sos clean` here, rather than doing plugin-based
scrubbing.
Closes: #998
Closes: #2015
Resolves: #2486
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Adds collection of several `cloud-init` command outputs, and
additionally updates the file collections performed based on the
behavior of `cloud-init collect-logs`.
Most notably, removes the collection of `/var/lib/cloud-init` to avoid
possible user sensitive data being collected.
Closes: #2080
Resolves: #2485
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
These policies haven't done anything since
IndependentPlugin was made a more common default.
Closes: #2470
Resolves: #2474
Signed-off-by: Bryan Quigley <code@bryanquigley.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Adds a test case to ensure that the plugins we can expect to always be
enabled when a "normal" `sos report` command is run are in fact enabled.
First, test the distro-independent plugins that do not have specific
enablement triggers (and thus should always be enabled).
Second, have a distro-specific test for distro-specific plugins that are
also expected to always run on those systems.
Closes: #2365
Resolves: #2431
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
| |
Adds `python3` as a package name for plugin enablement of the `python`
plugin for both RH and Ubuntu distributions.
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
| |
Updates `assertPlugin(Not)Included` to allow for lists as well as single
strings.
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Adds helper decorators to define specific test methods for use on
specific distributions only. Currently two decorators are available:
@redhat_only Only run on fedora, centos, or rhel
@ubuntu_only Only run on ubuntu or debian
Note that these decorators are only intended for individual `test_*`
methods, and will not function to define distro-specific test classes.
These should make it easier to write plugin test cases where packaging
differences between distributions otherwise makes plugin tests either
impossible needlessly complex to write generically.
Related: #2431
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
| |
Adds a predicate to gate collection of `ip xfrm` commands, dependent on
if the `xfrm_algo` and `xfrm_user` kmods are already loaded.
Discovered during testing for #2431
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
| |
Removes a `plugin_name` override in the `DebianPsacct` plugin that was
causing issues with plugin enablement/references by the overridden name.
Related: #2431
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Moves the detection of the local system's distribution to StageOne, and
defines two constants to reference RH and Ubuntu distribution lists to
allow for distro-specific tests.
Further, make the expected_warnings_displayed test skipped if the local
test system is not a RH-family distro, as those warnings are not
expected on non RH-family systems.
Related: #2431
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
First, add a "pre-setup setup" method, in the form of `pre_sos_setup()`,
that can be used in the way the traditional `setUp()` method would be
used (but can't because that's our entry point for executing our sos
runs). This method will be executed _prior_ to any mocking.
Second, add a smoke test that enables all plugins that exist in the
local branch being tested. This will test that doing so does not
generate any exceptions and that some expected warnings from select
plugins are displayed.
Related: #2431
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Adds several plugin tests for plugins that can be expected to run on all
distributions, in particular these tests are including mostly Stage Two
tests.
As such, these tests are only recommended to be run on test machines.
Related: #2431
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
Adds a new CirrusCI task to run our StageTwo tests, iff the stageone
task completed successfully. Note that for this task we will only
execute against the latest supported version of each distro we test.
Related: #2431
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
| |
Adds a test to ensure our timeout control is working properly.
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
In events where a user-provided keyword matches a filename before, or
instead of, any file _content_, that filename would go unobfuscated due
to how we match filenames to obfuscated values.
To resolve this, pre-generate an obfuscation map during parser
initialization for each keyword provided.
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This commits adds a `StageTwoReportTest` from which tests that need to
do "light" mocking should be defined. As the name implies, this is the
base class for our Stage 2 tests.
"Light" mocking for our purposes is currently defined as dropping files
in locations on the test system's filesystem, which may include
replacing binaries, and installing packages for the durtation of the
test case. At the end of the test, packages are uninstalled and mocked
files are removed. In the event that a file already exists at a given
location on the test system, that file is temporarily renamed with a
'.sostesting' extension. On test clean up, once mocked files are
removed, any files renamed previously are moved back to their original
location/name.
Mocking is controlled automatically during test setup, and is based upon
test class attributes, similar to how plugin enablement is dependent
upon plugin class attributes. For Stage 2 tests cases, this includes:
`files` - a list of file names. These names should represent the
"real" location on the filesystem where the mocked
files will be placed. The mocked content should be
placed under a similar file path under tests/test_data
E.G. using `files = ['/etc/foo/bar']` will place a
copy of `tests/test_data/etc/foo/bar` under
`/etc/foo/bar` on the test system.
`packages` - a dict of packages to install. The dict keys map to
the distribution the package names are used for,
e.g. `rhel` or `ubuntu`. The values are lists of
package names, optionally with version included.
This commit includes just the framework for these tests, and no actual
Stage 2 tests are included as of yet.
Related: #2431
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
| |
Updates the filepaths in our unittests for the temp data files to
reflect their new location in the updated directory layout.
Related: #2431
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
| |
Use avocado to run our test suite instead of the removed simple.sh.
Related: #2365
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This commit represents the start of an overhaul of the test suite used
by sos. Note that several more commits to follow will be required in
order for the test suite to be considered stable.
The new test suite will use the avocado-framework to build out new
tests.
This first part adopts a new 'stageX' naming scheme for our tests as
follows:
stage0 -> Unittests
stage1 -> Basic function tests, no mocking allowed
stage2 -> Mocked tests for specific scenarios/regressions
stage3 -> Complex setups for layered products/environments
At the moment, these unittests are not updated for avocado, though most
should still work with `nosetest` directly.
A new set of base classes is defined in tests/sos_tests.py which provide
the foundation for actual tests cases. This approach entails new test
cases subclassing a base class, such as the new `StageOneReportTest`,
and setting the `sos_cmd` class attr to the _options_ for an sos report
run. By default `sos report --batch` will be run, and targeted to the
test job's directory as a tmpdir.
Each sos command will be executed once, and all test_* methods within a
test case that subclasses `StageOneReportTest` will be checked against
the output of that execution. Note that this diverges from avocado's
typical approach where each test_* method is performed against a brand
new instance of the class (thus meaning any setup including our sos
report run would normally be run fresh). However, after speaking with
the avocado devel team, this is still seen as a valid pattern for the
framework.
The current organizational approach is to separate the tests by
component rather than stage. For example. `tests/report_tests/` should
hold any report-centric tests, and the `plugin_tests` directory therein
should be used for plugin-specific tests. As of this commit, there are
basic functionality tests under `tests/report_tests/` and a single
plugin test under `tests/report_tests/plugin_tests/` to act as a POC.
Further, there is a `tests/vendor_tests/` directory for organizing
vendor-specific bug/feature tests that are not covered by the generic
project-wide tests. A POC test from RHBZ1928628 is available with this
commit.
Note that in order for these tests to be run properly _without_
installing the current branch to the local system, you will need to run
the following command:
`PYTHONPATH=tests/ avocado run -t stageone tests/`
Related: #2431
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
Converts the `ata` plugin to using the `add_block_dev()` method instead
of generating it's own list of device paths.
Closes: #2430
Resolves: #2481
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Previously, `parse_string_for_keys()`, called by `obfuscate_string()`
for non-regex based obfuscations, would return on the first match in the
string found for each parser.
Instead, continue iterating over all items in each parser's dataset
before returning the (now fully) obfuscated string.
Resolves: #2480
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fixes an issue where an upload attempt on Ubuntu systems would fail with
a traceback due to the upload archive's name not being available when we
first check to make sure we have an upload location, since the Ubuntu
default location requires an archive/file name.
Related to this, stop clobbering `upload_archive` and assign the archive
name to an `upload_archive_name` variable instead.
Closes: #2472
Resolves: #2479
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In order to determine the available plugins on each node, `collect`
parses the `--list` output. This relies on several multiline regexes
that are built from the headers from each section printed in that
output.
Since there are now options available to every plugin, the order of
output sections has changed, and `collect` needs to break parsing
earlier to avoid scraping non-plugin-name strings into the available
plugin list for each node.
Resolves: #2477
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
Only extended life OSes are using upstart still.
http://upstart.ubuntu.com/ for more details (although it's a mostly
unmaintained site at this point).
Resolves: #2473
Signed-off-by: Bryan Quigley <code@bryanquigley.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
networkctl may be used to introspect the state
of the network links as seen by systemd-networkd.
Resolves: #2471
Signed-off-by: Eric Desrochers <eric.desrochers@canonical.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add --cmd-timeout option to configure command timeout. Plugin-specific
option of the same name (i.e. -k logs.cmd-timeout=60) can control the
timeout per plugin.
Option defaults and global/plugin-specific option preference follows the
--plugin-timeout rules.
Resolves: #2466
Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Files primarily containing package information, e.g. `installed-rpms` or
`installed-debs`, were previously being skipped by all parsers. In
reality, we only need to skip these for the IP parser due to the fact
that version numbers often generate a match for IP address regexes.
This will also fix a problem where if a system was the build host for
certain packages, the hostname would remain in these files as the
hostname parser was previously not checking these files.
Closes: #2400
Resolves: #2464
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
By default SSSD collects all memory cache files:
* /var/lib/sss/mc/passwd
* /var/lib/sss/mc/group
* /var/lib/sss/mc/initgroups
They all are included in 25MB size limit for sosreport.
This commits add memory cache files one - by - one,
this way 25MB size limit will be aplied per file.
Resolves: #2462
Signed-off-by: Paweł Poławski <ppolawsk@redhat.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|