aboutsummaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* [origin] Removing unused configurationVladislav Walek2021-09-241-34/+19
| | | | | | | | | | | | | | | Removed the diagnostics part as it is no longer maintained and doesn't work on Openshift. Adding additional projects to collect. Removed getting all namespaces as it is not needed for troubleshooting and project names are sensitive for some customers. Adding condition to collect the logs from systemd openshift services if not running as static pods. Signed-off-by: Vladislav Walek <22072258+vwalek@users.noreply.github.com>
* [ceph] split the ceph pluginNikhil Kshirsagar2021-09-246-94/+347
| | | | | | | | | | | | | This work distributes the ceph plugin into plugins for individual ceph components (mon,osd,mds,rgw,mgr), so that additional data collection can then be added in each of the components. Work for additional data collection in each component plugin will follow in a later commit. Closes: #1945 Signed-off-by: Nikhil Kshirsagar <nkshirsagar@gmail.com>
* [build] Add archives to setup.py packagesJake Hunsaker2021-09-211-1/+1
| | | | | | | Adds the newly abstracted `sos.cleaner.archives` package to `setup.py` so that manual builds will properly include it. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [runc] remove redundant pluginPavel Moravec2021-09-211-37/+0
| | | | | | | | | See discussion in #2689 Resolves: #2689 Closes: #2692 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [Plugin] Improve add_string_as_file collection and manifest recordingJake Hunsaker2021-09-179-20/+79
| | | | | | | | | | | | | | | | | | | This commit allows plugins that call `add_string_as_file` to specify if the string should be written to `sos_strings/$name/` (current behavior) or if it should be written to `sos_commands/$name/` which may be desireable for organizational purposes for plugin collections. `add_string_as_file()` has also been updated to write to a plugin's manifest section for any files written this way. Accordingly, the method will now accept a `tags` parameter to add specified tags to the manifest entry. Certain plugins directly calling this method have been updated, but the existing logic to write truncated data to `sos_strings/` remains untouched, and will not generate manifest entries (as those should already be handled by the method that trigged the truncated collection). Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [docs] Update README content and formattingJake Hunsaker2021-09-151-10/+33
| | | | | | | Fixes a few formatting issues with the README, as well as expanding and clarifying a few sections based on recent changes. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* Add snap configuration fileArif Ali2021-09-141-0/+37
| | | | | | | | | | | | | After discussion with the snapcraft team, this fits the bill for a classic snap Closes: #1992 Reference: https://forum.snapcraft.io/t/25673 Reference: https://forum.snapcraft.io/t/26447 Signed-off-by: Nicolas Bock nicolas.bock@canoical.com Signed-off-by: Arif Ali <arif.ali@canonical.com>
* [kernel] Capture Pressure Stall InformationJose Castillo2021-09-141-1/+2
| | | | | | | | | | Kernel 4.20 includes PSI metrics for CPU, memeory and IO. The feature is enabled after adding "psi=1" as kernel boot parameter. The information is captured in files in the directory /proc/pressure. Signed-off-by: Jose Castillo <jcastillo@redhat.com>
* Add `editorconfig` for basic editor settingsNicolas Bock2021-09-141-0/+9
| | | | | | | | | EditorConfig is fairly editor agnostic and is understood by a wide range of editors/IDEs [1]. Signed-off-by: Nicolas Bock <nicolas.bock@canonical.com> [1] https://editorconfig.org/
* [openstack_octavia] Stop collecting Loadbalancer detailsPurandhar Sairam Mannidi2021-09-131-27/+42
| | | | | | | | | | | | | | Currently, we're collecting loadbalancer details from nodes by running specific commands irrespective of whether octavia client package is installed or OS_* env variables are set required for collecting LoadBalancer or its associated subcommands information. This patch adds a check for both environment variables and whether octavia package is installed. Resolves: #2670 Signed-off-by: Purandhar Sairam Mannidi <pmannidi@redhat.com>
* [cleaner] Add support for Insights client archivesJake Hunsaker2021-09-133-1/+46
| | | | | | | Adds a new type of `SoSObfuscationArchive` to add support for obfuscating archives generated by the Insights project. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [hostname_map] Add a catch for single-character hostnamesJake Hunsaker2021-09-131-1/+8
| | | | | | | | | | | | If a log file was truncated at a specific boundary in a string of the FQDN of the host such that we only get a couple characters before the rest of the domain, we would previously bodly replace all instances of that character with the obfuscated short name; not very helpful. Instead, don't sanitize the short name if this happens and instead obfuscate the whole FQDN as 'unknown.example.com'. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [clean] Provide archive abstractions to obfuscate more than sos archivesJake Hunsaker2021-09-1314-179/+423
| | | | | | | | | | | | | | | | | | | | | This commit removes the restriction imposed on `sos clean` since its introduction in sos-4.0 to only work against known sos report archives or build directories. This is because there has been interest in using the obfuscation bits of sos in other data-collector projects. The `SoSObfuscationArchive()` class has been revamped to now be an abstraction for different types of archives, and the cleaner logic has been updated to leverage this new abstraction rather than assuming we're working on an sos archive. Abstractions are added for our own native use cases - that being `sos report` and `sos collect` for at-runtime obfuscation, as well as standalone archives previously generated. Further generic abstractions are available for plain directories and tarballs however these will not provide the same level of coverage as fully supported archive types, as is noted in the manpage for sos-clean. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [subscription_manager] collect syspurpose data via sub-manPavel Moravec2021-09-091-1/+2
| | | | | | | | | As syspurpose utility is being integrated into subscription-manager, we need to call both legacy and sub-man version of the command. Resolves: #2681 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [report] Implement --estimate-onlyPavel Moravec2021-09-092-3/+84
| | | | | | | | | Add report option --estimate-only to estimate disk space requirements when running a sos report. Resolves: #2673 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [firewalld] collect nft rules in firewall_tables onlyPavel Moravec2021-09-092-11/+6
| | | | | | | | | | | | We collect 'nft list ruleset' in both plugins, while: - nft is not shipped by firewalld package, so we should not collect it in firewalld plugin - running the command requires both nf_tables and nfnetlink kmods, so we should use both kmods in the predicate Resolves: #2679 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [foreman] Correct Apache package on DebianEwoud Kohl van Wijngaarden2021-09-092-2/+2
| | | | | | | On Debian the Apache package is named apache2 and this is also used in directory paths such as /etc/apache2 and /var/log/apache2. Signed-off-by: Ewoud Kohl van Wijngaarden <ewoud@kohlvanwijngaarden.nl>
* [cirrus] Upload logs from failed tests to GCE storage bucketJake Hunsaker2021-09-092-4/+37
| | | | | | | | | | | Adds failure handling to cirrus tasks that actually run sos so that if an error is encountered in the test suite, the logs from the tests are uploaded to the GCE cloud storage bucket associated with the GCE sos project. This will also make those logs available from the cirrus task page. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [tests] Add tests for proper handling and logging of conf file optionsJake Hunsaker2021-09-082-0/+63
| | | | | | | Adds a new report test that ensures that options set in `sos.conf` are picked up, handled properly, and logged as we expect them to be. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [tests] Allow StageTwo to place files with different namesJake Hunsaker2021-09-081-13/+29
| | | | | | | | | | | | | | | Up until now the `files` attr for Stage Two tests was simply a list that dropped files of the same name from our `tests/test_data/` directory into the relevant paths on the test system. This however limited us to single copies of files within this directory. Instead, also allow `files` to take a list of tuples in the form of `(source_path, dest_path)` so that we may have multiple copies of say `sos.conf` within `tests/test_data` for different test cases while still being able to temporarily override the test system's actual `/etc/sos/sos.conf`. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [options] Fix logging on plugopts in effective sos commandJake Hunsaker2021-09-082-14/+18
| | | | | | | | | | | | | | | | | | First, provide a special-case handling for plugin options specified in sos.conf in `SoSOptions.to_args().has_value()` that allows for plugin options to be included in the "effective options now" log message. Second, move the logging of said message (and thus the merging of preset options, if used), to being _prior_ to the loading of plugin options. Combined, plugin options specified in sos.conf will now be logged properly and this logging will occur before we set (and log the setting of) those options. Resolves: #2663 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [tests] Pin avocado test runner to known working versionCleber Rosa2021-08-311-3/+3
| | | | | | | | | | | | | | | | Avocado has been developing a new test runner (dubbed nrunner) and the upcoming release 91.0 will activate the new runner as the default one. While most of the tests and jobs should work under the new runner, there are a few caveats, and possibly unknown issues. To avoid regressions in the test execution and CI status, let's pin the test runner to the known working version. A switch to use the new runner may be done as a separate future change after compatibility is fully investigated and validated. Reference: https://avocado-framework.readthedocs.io/en/90.0/releases/90_0.html#important-announcement Signed-off-by: Cleber Rosa <crosa@redhat.com>
* [unpackaged] deal with recursive loop of symlinks properlyPavel Moravec2021-08-311-2/+3
| | | | | | | | | | When the plugin processes a recursive loop of symlinks, it currently hangs in an infinite loop trying to follow the symlinks. Use pathlib.Path.resolve() method to return the target directly. Resolves: #2664 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [processor] Apply sizelimit to /sys/devices/system/cpu/cpuXPavel Moravec2021-08-301-1/+8
| | | | | | | | | | | | Copy /sys/devices/system/cpu/cpuX with separately applied sizelimit. This is required for systems with tens/hundreds of CPUs where the cumulative directory size exceeds 25MB or even 100MB. Resolves: #2639 Closes: #2665 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [omnipath_client] Ensure opacapture runs only with allow-system-changesJose Castillo2021-08-301-2/+7
| | | | | | | | | | | | | | | | | | | | While omnipath_client plugin is collecting "opacapture", `depmod -a` command is executed to regenerates some files under /usr/lib/modules/$kernel. modules.dep modules.dep.bin modules.devname modules.softdep modules.symbols modules.symbols.bin This patch ensures that the command is only run when the option --allow-system-changes is used. Fixes: RHBZ#1998433 Signed-off-by: Jose Castillo <jcastillo@redhat.com>
* [openvswitch] Get userspace datapath implementationsKevin Traynor2021-08-301-1/+7
| | | | | | | | | OVS has options for the userspace datapath so certain functionality can use implementations based on SIMD instructions if available. Add some commands which show the implementations available and used. Signed-off-by: Kevin Traynor <ktraynor@redhat.com>
* [ssh] Include info about ~/.ssh folders of users.Maciej Strzelczyk2021-08-301-0/+33
| | | | | | | | | | | The SSH demon will not accept connection for users whose `.ssh` forlders don't have proper permissions set. This change makes the ssh plugin to list (ls -laZ) the `~/.ssh` folders of all users found in the system. Closes #2658 Signed-off-by: Maciej Strzelczyk <strzelczyk@google.com>
* [udisks] New udisks pluginJose Castillo2021-08-301-0/+30
| | | | | | | | | | | This new plugin captures the udisks config in: /etc/udisks/ And status and object dump via udisksctl command. Signed-off-by: Jose Castillo <jcastillo@redhat.com>
* [plugins] Update existing plugin options to new classJake Hunsaker2021-08-3067-247/+313
| | | | | | | Updates all existing plugin options to use the new `PluginOpt` class introduced in the previous commit. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [Plugin] Make plugin options their own classJake Hunsaker2021-08-305-117/+163
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Up until now plugin options were defined via tuples, with positional significance of the tuple elements. This made plugin option creation fairly easy, but option handling could easily become confusing. Instead, create a new `PluginOpt` class that plugin options from here on out will need to build from in order to function. These will still be applied to plugins by inserting them into the `option_list` class attr in order to retain an easy way to expand plugin options for authors. Internally, options are now assigned to a dict which is then directly accessed for plugin option manipulations. PluginOpt default values are retained separate from their current value, and elements are assigned directly to meaningful identifiers within the class. This should alleviate some of the overhead when handling plugin options within sos. Not all current tuple elements have been carried over into the new `PluginOpt` class - for example, the 'speed' attribute has been dropped as it does not have a current function. In the planned `sos info` component, the time effects of a plugin option should be documented in the `long_desc` attribute instead. Additionally, the `Plugin.get_option_as_list()` method has been removed as it was not being used anywhere. Note that this particular commit only introduces the new class, and the loading options used by `SoSReport()`. As such, plugins using options currently will report errors during test runs. A commit following this one will shuffle existing plugin options into the new class structure and allow the plugins to execute normally. Resolves: #274 Resolves: #452 Resolves: #1597 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [rabbitmq] Capture Erlang crash dump from RabbitMQTakashi Kajinami2021-08-261-0/+7
| | | | | | | | | | When RabbitMQ process crashes, it sometimes generates erl_crash.dump in its log directory. This file is is useful to understand the cause. This change ensures that file is captured in sosreport even without --all-logs. Signed-off-by: Takashi Kajinami <tkajinam@redhat.com>
* [unpackaged] Limit package_manager file list to relevant pathsJake Hunsaker2021-08-251-5/+8
| | | | | | | | | | | | | | | | Since the `unpackaged` plugin looks for all files within $PATH, we should in turn limit the list of files returned by the package manager that we're comparing against to the same paths. Otherwise, we are iterating over a needlessly large list of packaged files outside of the filesystem paths that we actually care about. On systems with a large number of installed packages, this can result in a packaged file list that is smaller by tens or even hundreds of thousands of entries and in turn results in a much faster `setup()` time for the plugin. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [collect] Remove double prompt for case IDJake Hunsaker2021-08-231-5/+0
| | | | | | | | Remove the case ID prompt within `collect`, since the one from `Policy` will always apply. This avoids a double prompt for a case ID if the user does not originally provide one. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [Ceph] Add support for containerized Ceph setupTeoman ONAY2021-08-231-4/+37
| | | | | | | RH Ceph 4 can either be installed as RPM or as containers. The changes permits to collect the ceph config on both kind of setup. Signed-off-by: Teoman ONAY <tonay@redhat.com>
* [collect,docs] Drop `--master` and update internal referencesJake Hunsaker2021-08-2315-122/+102
| | | | | | | | | | | | | | | | This commit follows up on the previous changes from #2555 by fully dropping the `--master` option in favor of `--primary`, `--manager`, and `--controller`. Internal references have been updated as well, using the term `primary` across all instances. Note that within OCP cluster profile, 'primary' labeling and option setting still relies on the 'master' label/role reported by the environment, as that remains the term used to identify such nodes. Resolves: #2329 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [tests] Update Cirrus testing for Foreman 2.5Jake Hunsaker2021-08-231-7/+5
| | | | | | | | | | | | | Updates the Foreman tests to use new images created for Foreman 2.5. Additionally, adds an option to the prep script to prevent new Debian release suite changes from causing the prep scripts to abort the tests. For out purposes, suite changes are not important during test runs as these images are built on known versions. Resolves: #2656 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [policies] add openeuler policy supportlijingwei2021-08-201-0/+40
| | | | Signed-off-by: lijingwei <lijingwei@uniontech.com>
* [pylint] Drop pylintrcJake Hunsaker2021-08-191-354/+0
| | | | | | Drops the pylintrc file from the repo, as it has been unused for years. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [sapnw] Add condition for length of fields varJan Jansky2021-08-161-0/+2
| | | | | | | | | | For better complexity of code i am proposing check for len(fields) >= 7 as we later in code getting fields[7] values. Resolves: #2647 Signed-off-by: Jan Jansky <jjansky@redhat.com>
* [build] Bump version to 4.24.2Jake Hunsaker2021-08-163-4/+7
| | | | | | This commit marks the relase of sos-4.2 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [sapnw] Fix IndexError exceptionJose Castillo2021-08-121-1/+2
| | | | | | | | | | | | | | | | This patch solves the situation where the command: /usr/sap/hostctrl/exe/saphostctrl -function ListInstances returns: No instances found And a non-zero return code. Signed-off-by: Jose Castillo <jcastillo@redhat.com>
* [sosnode] Fix passing of plugin options when using `--only-plugins`Jake Hunsaker2021-08-112-19/+20
| | | | | | | | | | | | | Fixes the handling of plugin options passed by `sos collect` to each node by first aligning the SoSOption name to those of `report` (`plugopts`), and second re-arranges the handling of plugin options and preset options passed by the user when also using `--only-plugins` so that the former are preserved and passed only with the `--only-plugins` option value. Resolves: #2641 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [sos] Read config section for clean when `--clean` is usedJake Hunsaker2021-08-115-0/+72
| | | | | | | | | | | | | | | | When a `report` or `collect` run would use `--clean` or `--mask` to do in-line obfuscation of collected reports, sos would not read the config section for clean - it would only be read if `sos clean` was called directly. As such, users would need to manually specify config file values for each run. Alleviate this gap by reading the config section for `clean` if either of the cleaner options are used. Do this before we apply cmdline options so that we maintain our order of precedence. Related: RHBZ#1950350 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [openstack_instack] Capture yaml files to define overridesTakashi Kajinami2021-08-101-1/+17
| | | | | | | | | | | | | Currently TripleO supports using additional yaml files to define resource overrides. - hieradata_override - net_config_override Because these files are useful when checking configurations in an undercloud node, this change makes sosreport capture these yaml files. The undercloud.conf file is parsed to look up these yaml files. Signed-off-by: Takashi Kajinami <tkajinam@redhat.com>
* [tests] Allow entire test classes to be distro specificJake Hunsaker2021-08-101-0/+21
| | | | | | | | | | | | | | Previously new test cases had to be written in such a way that they could be run on any supported distribution, even if their individual tests were all marked for specific distributions. Use of the distro-only decorators would fail when applied to test cases as a whole due to how test instantiation errors are handled. To address this, provide a distro check within `BaseSoSTest` so that tests can be made distro-specific and we can signal to avocado to skip all tests within that test case without failing the test suite entirely. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [opencontrail] adds a plugin for opencontrailmcriswell2021-08-041-0/+40
| | | | Signed-off-by: Matt Criswell <mcriswell@mirantis.com>
* [tests] Add test for cleaning and existing archiveJake Hunsaker2021-08-044-25/+110
| | | | | | | | | | | | | | | | Adds a test that ensures we are properly extracting and cleaning an already existing archive (one is being included under tests/test_data/ now), much the same as we test for a proper `sos report --clean` run. As part of this, some aspects of the `full_report_run.py` test are moved into the base test classes, and the new `existing_archive.py` test will explicitly run `sos clean` instead of `sos report`. The included archive for obfuscation testing is taken from a stock Fedora 34 VM with a known configuration which is reflected in the items being tested for. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [cleaner] Use a nested ProcessPoolExecutor for extractionJake Hunsaker2021-08-041-5/+19
| | | | | | | | | | | | | | | | | | | | This commit inserts a nested ProcessPoolExecutor into the extraction workflow for archives that are being obfuscated by `sos clean`. Previously, the extraction was handled inside the same thread as the rest of the obfuscation routines for each archive. However, it has been found that when very large archives are manipulated concurrently, performance can take a massive hit during the extraction process. This is due to GIL limitations. In this aspect 'very large archives' implies many tens of thousands of files - e.g. 50K+. Because TarFile uses a 10K internal buffer, we end up spinning a lot of time processing each file via the interpreter. By shunting each extraction off into a new process space, we can avoid the GIL issues altogether. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [rhui] Update the plugin on several placesPavel Moravec2021-08-041-8/+6
| | | | | | | | | | | - obfuscate "rhui_manager_password: xxx" in /root/.rhui/answers.yaml* - no need to collect or obfuscate anything from /etc/ansible/facts.d - newly detect the plugin via /etc/rhui/rhui-tools.conf file or rhui-manager command (only) Resolves: #2637 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [pulp] adding obfuscation of proxy_password inbvassova2021-08-031-2/+2
| | | | | | | | /etc/pulp/server/plugins.conf.d/ Resolves: #2564 Signed-off-by: Barbora Vassova bvassova@redhat.com
generated by cgit (git 2.34.1) at 2024-11-15 06:06:38 +0000