aboutsummaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* [tests] Run unit tests under avocado instead of noseJake Hunsaker2021-10-114-48/+6
| | | | | | | | | | | | | | | | | | | | | | | `nose` is no longer maintained, and as of python-3.10 is functionally broken. As such, instead transition to running those tests via avocado, like we do with our integration test suite. The tests themselves do not need much modification, however due to the isolation provided for executing the tests we do need to explicitly set a new PYTHONPATH env var for those executions. This means we still need to run the unit tests as a separate step from the stageone tests. The changes needed are mostly around file paths relative to the pwd where the tests are executed from originally. Additionally, remove the sosreport_pexpect unit test as it is no longer useful in its own right, would need more significant changes to run properly with avocado, and the integration test suite provides better coverage for what it was testing. Closes: #2716 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [networking] retrieve devlink port attributesAntoine Tenart2021-10-111-0/+1
| | | | | | | | | | | | | | | | | | | | | | When using sub-functions[1] gathering the devlink port attributes does provide value when debugging. If there is no devlink port, the output is empty. Example: pci/0000:04:00.0/65535: type eth netdev enp4s0f0 flavour physical port 0 splittable false pci/0000:04:00.0/32768: type eth netdev en4f0pf0sf42 flavour pcisf controller 0 pfnum 0 sfnum 42 splittable false function: hw_addr 00:00:00:00:88:88 state active opstate attached pci/0000:04:00.0/32769: type eth netdev en4f0pf0sf1 flavour pcisf controller 0 pfnum 0 sfnum 1 splittable false function: hw_addr 00:00:00:00:00:00 state active opstate attached auxiliary/mlx5_core.sf.4/131072: type eth netdev enp4s0f0s42 flavour virtual port 0 splittable false auxiliary/mlx5_core.sf.5/196608: type eth netdev enp4s0f0s1 flavour virtual port 0 splittable false [1] https://www.kernel.org/doc/html/latest/networking/devlink/devlink-port.html#subfunction Signed-off-by: Antoine Tenart <atenart@kernel.org>
* [report] Overwrite pred=None before refering predicate attributesPavel Moravec2021-10-061-0/+2
| | | | | | | | | | During a dry run, add_journal method sets pred=None whilst log_skipped_cmd refers to predicate attributes. In that case, replace None predicate by a default / empty predicate. Resolves: #2711 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [gcp] Adding Google Cloud pluginMaciej Strzelczyk2021-10-061-0/+145
| | | | | | | | | | | | | | | | | Adding a plugin for Google Cloud Compute Engine VMs. The plugin will collect data about Google services running on the system (journalctl -u google*), data from the Metadata Server that's available for every instance and output of `gcloud auth list` - if available. Available option: keep-pii - if set, the plugin won't remove the project name and project number from the metadata.json file. Closes #2699 Signed-off-by: Maciej Strzelczyk <strzelczyk@google.com>
* [insights] collect connection check command outputPavel Moravec2021-10-051-0/+5
| | | | | | | | | | Collect 'insights-client --test-connection --net-debug' cmdout with a limited timeout to prevent plugin stuck for too long in case of a networking/proxy issue. Resolves: #2704 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [foreman] Collect puma status and statsPavel Moravec2021-10-051-1/+20
| | | | | | | | | Collect foreman-puma-status and 'pumactl [gc-|]stats', optionally using SCL (if detected). Resolves: #2712 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [Policy] Fix assignment of case id promptJake Hunsaker2021-09-301-4/+5
| | | | | | | | | | | | | | | | When prompting for a case id, `Policy` was not properly updating the option value, only assigning the value to `Policy` which meant that aspects outside of `Policy` could not always properly reference the (updated) case id. Fix this by assigning the case id prompt response back to the case_id option value. `Policy` still retains a local reference to case_id as existing logic was setting that based on the (assumed-to-be-updated) option value, which until this commit would have been superfluous. Closes: #2707 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [networking] prevent iptables-save commands to load nf_tables kmodPavel Moravec2021-09-301-5/+24
| | | | | | | | | | | | If iptables has built-in nf_tables kmod, then 'ip netns <foo> iptables-save' command requires the kmod which must be guarded by predicate. Analogously for ip6tables. Resolves: #2703 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [Systemd, Policy] Correct InitSystem chrooting when chroot is neededJake Hunsaker2021-09-304-32/+43
| | | | | | | | | | | | | | | | | | | | | | | | | | | This commit resolves a situation in which `sos` is being run in a container but the `SystemdInit` InitSystem would not properly load information from the host, thus causing the `Plugin.is_service*()` methods to erroneously fail or return `False`. Fix this scenario by pulling the `_container_init()` and related logic to check for a containerized host sysroot out of the Red Hat specific policy and into the base `LinuxPolicy` class so that the init system can be initialized with the correct sysroot, which is now used to chroot the calls to the relevant `systemctl` commands. For now, this does impose the use of looking for the `container` env var (automatically set by docker, podman, and crio regardless of distribution) and the use of the `HOST` env var to read where the host's `/` filesystem is mounted within the container. If desired in the future, this can be changed to allow policy-specific overrides. For now however, this extends host collection via an sos container for all distributions currently shipping sos. Note that this issue only affected the `InitSystem` abstraction for loading information about local services, and did not affect init system related commands called by plugins as part of those collections. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [collect] Abstract transport protocol from SoSNodeJake Hunsaker2021-09-2713-344/+705
| | | | | | | | | | | | | | | | | | Since its addition to sos, collect has assumed the use of a system installation of SSH in order to connect to the nodes identified for collection. However, there may be use cases and desires to use other transport protocols. As such, provide an abstraction for these protocols in the form of the new `RemoteTransport` class that `SoSNode` will now leverage. So far an abstraction for the currently used SSH ControlPersist function is provided, along with a psuedo abstraction for local execution so that SoSNode does not directly need to make more "if local then foo" checks than are absolutely necessary. Related: #2668 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [origin] Removing unused configurationVladislav Walek2021-09-241-34/+19
| | | | | | | | | | | | | | | Removed the diagnostics part as it is no longer maintained and doesn't work on Openshift. Adding additional projects to collect. Removed getting all namespaces as it is not needed for troubleshooting and project names are sensitive for some customers. Adding condition to collect the logs from systemd openshift services if not running as static pods. Signed-off-by: Vladislav Walek <22072258+vwalek@users.noreply.github.com>
* [ceph] split the ceph pluginNikhil Kshirsagar2021-09-246-94/+347
| | | | | | | | | | | | | This work distributes the ceph plugin into plugins for individual ceph components (mon,osd,mds,rgw,mgr), so that additional data collection can then be added in each of the components. Work for additional data collection in each component plugin will follow in a later commit. Closes: #1945 Signed-off-by: Nikhil Kshirsagar <nkshirsagar@gmail.com>
* [build] Add archives to setup.py packagesJake Hunsaker2021-09-211-1/+1
| | | | | | | Adds the newly abstracted `sos.cleaner.archives` package to `setup.py` so that manual builds will properly include it. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [runc] remove redundant pluginPavel Moravec2021-09-211-37/+0
| | | | | | | | | See discussion in #2689 Resolves: #2689 Closes: #2692 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [Plugin] Improve add_string_as_file collection and manifest recordingJake Hunsaker2021-09-179-20/+79
| | | | | | | | | | | | | | | | | | | This commit allows plugins that call `add_string_as_file` to specify if the string should be written to `sos_strings/$name/` (current behavior) or if it should be written to `sos_commands/$name/` which may be desireable for organizational purposes for plugin collections. `add_string_as_file()` has also been updated to write to a plugin's manifest section for any files written this way. Accordingly, the method will now accept a `tags` parameter to add specified tags to the manifest entry. Certain plugins directly calling this method have been updated, but the existing logic to write truncated data to `sos_strings/` remains untouched, and will not generate manifest entries (as those should already be handled by the method that trigged the truncated collection). Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [docs] Update README content and formattingJake Hunsaker2021-09-151-10/+33
| | | | | | | Fixes a few formatting issues with the README, as well as expanding and clarifying a few sections based on recent changes. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* Add snap configuration fileArif Ali2021-09-141-0/+37
| | | | | | | | | | | | | After discussion with the snapcraft team, this fits the bill for a classic snap Closes: #1992 Reference: https://forum.snapcraft.io/t/25673 Reference: https://forum.snapcraft.io/t/26447 Signed-off-by: Nicolas Bock nicolas.bock@canoical.com Signed-off-by: Arif Ali <arif.ali@canonical.com>
* [kernel] Capture Pressure Stall InformationJose Castillo2021-09-141-1/+2
| | | | | | | | | | Kernel 4.20 includes PSI metrics for CPU, memeory and IO. The feature is enabled after adding "psi=1" as kernel boot parameter. The information is captured in files in the directory /proc/pressure. Signed-off-by: Jose Castillo <jcastillo@redhat.com>
* Add `editorconfig` for basic editor settingsNicolas Bock2021-09-141-0/+9
| | | | | | | | | EditorConfig is fairly editor agnostic and is understood by a wide range of editors/IDEs [1]. Signed-off-by: Nicolas Bock <nicolas.bock@canonical.com> [1] https://editorconfig.org/
* [openstack_octavia] Stop collecting Loadbalancer detailsPurandhar Sairam Mannidi2021-09-131-27/+42
| | | | | | | | | | | | | | Currently, we're collecting loadbalancer details from nodes by running specific commands irrespective of whether octavia client package is installed or OS_* env variables are set required for collecting LoadBalancer or its associated subcommands information. This patch adds a check for both environment variables and whether octavia package is installed. Resolves: #2670 Signed-off-by: Purandhar Sairam Mannidi <pmannidi@redhat.com>
* [cleaner] Add support for Insights client archivesJake Hunsaker2021-09-133-1/+46
| | | | | | | Adds a new type of `SoSObfuscationArchive` to add support for obfuscating archives generated by the Insights project. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [hostname_map] Add a catch for single-character hostnamesJake Hunsaker2021-09-131-1/+8
| | | | | | | | | | | | If a log file was truncated at a specific boundary in a string of the FQDN of the host such that we only get a couple characters before the rest of the domain, we would previously bodly replace all instances of that character with the obfuscated short name; not very helpful. Instead, don't sanitize the short name if this happens and instead obfuscate the whole FQDN as 'unknown.example.com'. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [clean] Provide archive abstractions to obfuscate more than sos archivesJake Hunsaker2021-09-1314-179/+423
| | | | | | | | | | | | | | | | | | | | | This commit removes the restriction imposed on `sos clean` since its introduction in sos-4.0 to only work against known sos report archives or build directories. This is because there has been interest in using the obfuscation bits of sos in other data-collector projects. The `SoSObfuscationArchive()` class has been revamped to now be an abstraction for different types of archives, and the cleaner logic has been updated to leverage this new abstraction rather than assuming we're working on an sos archive. Abstractions are added for our own native use cases - that being `sos report` and `sos collect` for at-runtime obfuscation, as well as standalone archives previously generated. Further generic abstractions are available for plain directories and tarballs however these will not provide the same level of coverage as fully supported archive types, as is noted in the manpage for sos-clean. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [subscription_manager] collect syspurpose data via sub-manPavel Moravec2021-09-091-1/+2
| | | | | | | | | As syspurpose utility is being integrated into subscription-manager, we need to call both legacy and sub-man version of the command. Resolves: #2681 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [report] Implement --estimate-onlyPavel Moravec2021-09-092-3/+84
| | | | | | | | | Add report option --estimate-only to estimate disk space requirements when running a sos report. Resolves: #2673 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [firewalld] collect nft rules in firewall_tables onlyPavel Moravec2021-09-092-11/+6
| | | | | | | | | | | | We collect 'nft list ruleset' in both plugins, while: - nft is not shipped by firewalld package, so we should not collect it in firewalld plugin - running the command requires both nf_tables and nfnetlink kmods, so we should use both kmods in the predicate Resolves: #2679 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [foreman] Correct Apache package on DebianEwoud Kohl van Wijngaarden2021-09-092-2/+2
| | | | | | | On Debian the Apache package is named apache2 and this is also used in directory paths such as /etc/apache2 and /var/log/apache2. Signed-off-by: Ewoud Kohl van Wijngaarden <ewoud@kohlvanwijngaarden.nl>
* [cirrus] Upload logs from failed tests to GCE storage bucketJake Hunsaker2021-09-092-4/+37
| | | | | | | | | | | Adds failure handling to cirrus tasks that actually run sos so that if an error is encountered in the test suite, the logs from the tests are uploaded to the GCE cloud storage bucket associated with the GCE sos project. This will also make those logs available from the cirrus task page. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [tests] Add tests for proper handling and logging of conf file optionsJake Hunsaker2021-09-082-0/+63
| | | | | | | Adds a new report test that ensures that options set in `sos.conf` are picked up, handled properly, and logged as we expect them to be. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [tests] Allow StageTwo to place files with different namesJake Hunsaker2021-09-081-13/+29
| | | | | | | | | | | | | | | Up until now the `files` attr for Stage Two tests was simply a list that dropped files of the same name from our `tests/test_data/` directory into the relevant paths on the test system. This however limited us to single copies of files within this directory. Instead, also allow `files` to take a list of tuples in the form of `(source_path, dest_path)` so that we may have multiple copies of say `sos.conf` within `tests/test_data` for different test cases while still being able to temporarily override the test system's actual `/etc/sos/sos.conf`. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [options] Fix logging on plugopts in effective sos commandJake Hunsaker2021-09-082-14/+18
| | | | | | | | | | | | | | | | | | First, provide a special-case handling for plugin options specified in sos.conf in `SoSOptions.to_args().has_value()` that allows for plugin options to be included in the "effective options now" log message. Second, move the logging of said message (and thus the merging of preset options, if used), to being _prior_ to the loading of plugin options. Combined, plugin options specified in sos.conf will now be logged properly and this logging will occur before we set (and log the setting of) those options. Resolves: #2663 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [tests] Pin avocado test runner to known working versionCleber Rosa2021-08-311-3/+3
| | | | | | | | | | | | | | | | Avocado has been developing a new test runner (dubbed nrunner) and the upcoming release 91.0 will activate the new runner as the default one. While most of the tests and jobs should work under the new runner, there are a few caveats, and possibly unknown issues. To avoid regressions in the test execution and CI status, let's pin the test runner to the known working version. A switch to use the new runner may be done as a separate future change after compatibility is fully investigated and validated. Reference: https://avocado-framework.readthedocs.io/en/90.0/releases/90_0.html#important-announcement Signed-off-by: Cleber Rosa <crosa@redhat.com>
* [unpackaged] deal with recursive loop of symlinks properlyPavel Moravec2021-08-311-2/+3
| | | | | | | | | | When the plugin processes a recursive loop of symlinks, it currently hangs in an infinite loop trying to follow the symlinks. Use pathlib.Path.resolve() method to return the target directly. Resolves: #2664 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [processor] Apply sizelimit to /sys/devices/system/cpu/cpuXPavel Moravec2021-08-301-1/+8
| | | | | | | | | | | | Copy /sys/devices/system/cpu/cpuX with separately applied sizelimit. This is required for systems with tens/hundreds of CPUs where the cumulative directory size exceeds 25MB or even 100MB. Resolves: #2639 Closes: #2665 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [omnipath_client] Ensure opacapture runs only with allow-system-changesJose Castillo2021-08-301-2/+7
| | | | | | | | | | | | | | | | | | | | While omnipath_client plugin is collecting "opacapture", `depmod -a` command is executed to regenerates some files under /usr/lib/modules/$kernel. modules.dep modules.dep.bin modules.devname modules.softdep modules.symbols modules.symbols.bin This patch ensures that the command is only run when the option --allow-system-changes is used. Fixes: RHBZ#1998433 Signed-off-by: Jose Castillo <jcastillo@redhat.com>
* [openvswitch] Get userspace datapath implementationsKevin Traynor2021-08-301-1/+7
| | | | | | | | | OVS has options for the userspace datapath so certain functionality can use implementations based on SIMD instructions if available. Add some commands which show the implementations available and used. Signed-off-by: Kevin Traynor <ktraynor@redhat.com>
* [ssh] Include info about ~/.ssh folders of users.Maciej Strzelczyk2021-08-301-0/+33
| | | | | | | | | | | The SSH demon will not accept connection for users whose `.ssh` forlders don't have proper permissions set. This change makes the ssh plugin to list (ls -laZ) the `~/.ssh` folders of all users found in the system. Closes #2658 Signed-off-by: Maciej Strzelczyk <strzelczyk@google.com>
* [udisks] New udisks pluginJose Castillo2021-08-301-0/+30
| | | | | | | | | | | This new plugin captures the udisks config in: /etc/udisks/ And status and object dump via udisksctl command. Signed-off-by: Jose Castillo <jcastillo@redhat.com>
* [plugins] Update existing plugin options to new classJake Hunsaker2021-08-3067-247/+313
| | | | | | | Updates all existing plugin options to use the new `PluginOpt` class introduced in the previous commit. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [Plugin] Make plugin options their own classJake Hunsaker2021-08-305-117/+163
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Up until now plugin options were defined via tuples, with positional significance of the tuple elements. This made plugin option creation fairly easy, but option handling could easily become confusing. Instead, create a new `PluginOpt` class that plugin options from here on out will need to build from in order to function. These will still be applied to plugins by inserting them into the `option_list` class attr in order to retain an easy way to expand plugin options for authors. Internally, options are now assigned to a dict which is then directly accessed for plugin option manipulations. PluginOpt default values are retained separate from their current value, and elements are assigned directly to meaningful identifiers within the class. This should alleviate some of the overhead when handling plugin options within sos. Not all current tuple elements have been carried over into the new `PluginOpt` class - for example, the 'speed' attribute has been dropped as it does not have a current function. In the planned `sos info` component, the time effects of a plugin option should be documented in the `long_desc` attribute instead. Additionally, the `Plugin.get_option_as_list()` method has been removed as it was not being used anywhere. Note that this particular commit only introduces the new class, and the loading options used by `SoSReport()`. As such, plugins using options currently will report errors during test runs. A commit following this one will shuffle existing plugin options into the new class structure and allow the plugins to execute normally. Resolves: #274 Resolves: #452 Resolves: #1597 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [rabbitmq] Capture Erlang crash dump from RabbitMQTakashi Kajinami2021-08-261-0/+7
| | | | | | | | | | When RabbitMQ process crashes, it sometimes generates erl_crash.dump in its log directory. This file is is useful to understand the cause. This change ensures that file is captured in sosreport even without --all-logs. Signed-off-by: Takashi Kajinami <tkajinam@redhat.com>
* [unpackaged] Limit package_manager file list to relevant pathsJake Hunsaker2021-08-251-5/+8
| | | | | | | | | | | | | | | | Since the `unpackaged` plugin looks for all files within $PATH, we should in turn limit the list of files returned by the package manager that we're comparing against to the same paths. Otherwise, we are iterating over a needlessly large list of packaged files outside of the filesystem paths that we actually care about. On systems with a large number of installed packages, this can result in a packaged file list that is smaller by tens or even hundreds of thousands of entries and in turn results in a much faster `setup()` time for the plugin. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [collect] Remove double prompt for case IDJake Hunsaker2021-08-231-5/+0
| | | | | | | | Remove the case ID prompt within `collect`, since the one from `Policy` will always apply. This avoids a double prompt for a case ID if the user does not originally provide one. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [Ceph] Add support for containerized Ceph setupTeoman ONAY2021-08-231-4/+37
| | | | | | | RH Ceph 4 can either be installed as RPM or as containers. The changes permits to collect the ceph config on both kind of setup. Signed-off-by: Teoman ONAY <tonay@redhat.com>
* [collect,docs] Drop `--master` and update internal referencesJake Hunsaker2021-08-2315-122/+102
| | | | | | | | | | | | | | | | This commit follows up on the previous changes from #2555 by fully dropping the `--master` option in favor of `--primary`, `--manager`, and `--controller`. Internal references have been updated as well, using the term `primary` across all instances. Note that within OCP cluster profile, 'primary' labeling and option setting still relies on the 'master' label/role reported by the environment, as that remains the term used to identify such nodes. Resolves: #2329 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [tests] Update Cirrus testing for Foreman 2.5Jake Hunsaker2021-08-231-7/+5
| | | | | | | | | | | | | Updates the Foreman tests to use new images created for Foreman 2.5. Additionally, adds an option to the prep script to prevent new Debian release suite changes from causing the prep scripts to abort the tests. For out purposes, suite changes are not important during test runs as these images are built on known versions. Resolves: #2656 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [policies] add openeuler policy supportlijingwei2021-08-201-0/+40
| | | | Signed-off-by: lijingwei <lijingwei@uniontech.com>
* [pylint] Drop pylintrcJake Hunsaker2021-08-191-354/+0
| | | | | | Drops the pylintrc file from the repo, as it has been unused for years. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [sapnw] Add condition for length of fields varJan Jansky2021-08-161-0/+2
| | | | | | | | | | For better complexity of code i am proposing check for len(fields) >= 7 as we later in code getting fields[7] values. Resolves: #2647 Signed-off-by: Jan Jansky <jjansky@redhat.com>
* [build] Bump version to 4.24.2Jake Hunsaker2021-08-163-4/+7
| | | | | | This commit marks the relase of sos-4.2 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
generated by cgit (git 2.34.1) at 2024-11-10 23:39:52 +0000