aboutsummaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* [report] Log a warning message when trying to encrypt with --buildJake Hunsaker2021-06-211-0/+3
| | | | | | | | | | | Since --build does not produce a tarball, we cannot encrypt any collections. However, the encryption options are set in the global option group so we cannot add a report-only option to that mutex group. Instead, print a warning informing the user of this limitation. Closes: #2568 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [cleaner] Fix checksum and archive pruning from archive listJake Hunsaker2021-06-212-2/+4
| | | | | | | | Fixes an issue where checksums may have gotten into the list of archives to be cleaned, which would cause further issues later. Additionally, prevents nested sosreports from top-level archives (such as from `collect`) from being removed for being a binary file when that top-level archive gets obfuscated.
* [cleaner] Don't obfuscate default 'core' userJake Hunsaker2021-06-211-0/+1
| | | | | | | | The 'core' user is a common default user on containerized hosts, and obfuscation of it is not advantageous, much like the default 'ubuntu' user for that distribution. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [ocp] Refine OCP node options in cluster profileJake Hunsaker2021-06-212-4/+65
| | | | | | | | Adds explicit setting of primary/node sos options for the `openshift` plugin within the cluster, rather than relying on default configurations and best practices to avoid duplicate collections. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [hostname parser,map] Attempt to detect strings with FQDN substringsJake Hunsaker2021-06-213-11/+81
| | | | | | | | | | | | | | This commit updates the hostname parser and associated map to be able to better detect and obfuscate FQDN substrings within file content and file names, particularly when the regex patterns failed to match a hostname that is formatted with '_' characters rather than '.' characters. The `get()` method has been updated to alow preserve characters and certain extensions that are not part of the FQDN, but are brought in by the regex pattern due to the fact that we need to use word boundary indicators within the pattern. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [cleaner] Explicitly obfuscate directory names within archivesJake Hunsaker2021-06-215-3/+54
| | | | | | | | | | | | | | | This commits adds a step to `obfuscate_report()` that explicitly walks through all directories in the archive, and obfuscates the directory names if necessary. Since this uses `obfuscate_string()` for the directory names, a `skip_keys` list has been added to maps to allow parsers/maps to specify matched keys (such as short names for the hostname parser) that should not be considered when obfuscating directory names (e.g. 'www'). Closes: #2465 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [cleaner] Iterate over matches with most precise match firstJake Hunsaker2021-06-213-5/+9
| | | | | | | | | | When matching strings in parsers to do obfuscation, we should be using the most precise matches found first, rather than matching in the order a match is hit. This ensures that we correctly obfuscate an entire string, rather than potentially only partial substring(s) that exist within the entire match. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [cleaner] Improve handling of symlink obfuscationJake Hunsaker2021-06-211-19/+46
| | | | | | | | | | Improves handling of symlink obfuscation by only performing the obfuscaiton on the ultimate target of any symlinks encountered. Now, when a symlink is encountered, clean will obfuscate the link name and re-write it in the archive, pointing to the (potentially obfuscated) target name. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [collect|sosnode] Avoiding clobbering sos options between nodesJake Hunsaker2021-06-213-38/+67
| | | | | | | | | | | | | | | | | | | | | This commit overhauls the function of `finalize_sos_cmd()` in several ways. First, assign the sos report plugin related options directly to private copies of those values for each node, so that the shared cluster profile does not clober options between nodes. Second, provide a default Lock mechanism for clusters that need to perform some node-comparison logic when assigning options based on node role. Finally, finalize the sos command for each node _prior_ to the call to `SoSNode.sosreport()` so that we can be sure that clusters are able to appropriately compare and assign sos options across nodes before some nodes have already started and/or finished their own sos report collections. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [sosnode] Support passing env vars to `run_command()`Jake Hunsaker2021-06-211-3/+24
| | | | | | | | | | | | | | Updates `run_command()` to support passing new environment variables to the command being run, for that command alone. This parameter takes a dict, and if set we will first copy the existing set of env vars on the node and then update that set of variables using the passed dict. Additionally, `execute_sos_command()` will now try to pass a new `sos_env_vars` dict (default empty) so that clusters may set environment variables specifically for the sos command being run, without having to modify the actual sos command being executed. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [SoSNode] Allow individually setting node optionsJake Hunsaker2021-06-212-2/+14
| | | | | | | | | Like we now do for primary nodes, add the ability to individually set node options via a new `set_node_options()` method for when blanket setting options across all nodes via the options class attrs is not sufficient. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [github] Merge Closes and Resolves linesBryan Quigley2021-06-161-2/+1
| | | | | | | | While using Github merging we don't need to track Resolves to close PRs anymore. If existing (and not the PR currently on) it still makes sense to use Resolves though. Signed-off-by: Bryan Quigley <code@bryanquigley.com>
* [plugins] Add containers enablement triggerJake Hunsaker2021-06-168-4/+8
| | | | | | | | | | | | | | | Adds a `containers` enablement trigger for plugins that currently check for specifically named containers as part of their `setup()` collections. This is primarily the openstack plugins checking for their respective component api's to be running in a container. In these cases the name pattern used in the `container_exists()` checks are used for the enablement trigger as well. Resolves: #2561 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [Plugin] Allow plugin activation by container presenceJake Hunsaker2021-06-162-7/+13
| | | | | | | | | | | Adds an enablement trigger tuple, `containers`, that allows for enabling a plugin based on the presence of a _running_ container matching a given name or regex. Note that this is only functional for the default runtime discovered by a policy during initialization. Resolves: #2561 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [report] Increase readability and reliability of top-level ThreadPoolJake Hunsaker2021-06-161-5/+4
| | | | | | | | | | | | | | | | | In the "top-level" `ThreadPoolExecutor` used for threading plugin execution, increase readability and reliability by wrapping it in a `with` context manager, so that the pool's cleanup is run in all situations explicitly. This is a continuation of #2294, bringing it up to date with the current state of the project and iterating over the plugin list as a discreet unit rather than individual plugins. Closes: #2294 Resolves: #2578 Co-authored-by: Erik Bernoth <ebernoth@redhat.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [vmware] Update command and log collectionsJake Hunsaker2021-06-161-4/+24
| | | | | | | | | | | | | | Updates the `vmware` plugin for more current collections based on the use of the `open-vm-tools` package which is the current recommendation from VMware to use instead of the legacy VMWare-Tools package. Command collections are expanded and updated to use `vmware-toolbox-cmd`. Closes: #2580 Resolves: #2582 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [cleaner] Use built-in TarFile compression functionalityJake Hunsaker2021-06-152-30/+28
| | | | | | | | | As we did with `report` in #2523, update `clean` to use the built-in compression capability of TarFile to re-compress unpacked archives. Resolves: #2576 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [pulpcore] obfuscate two passwords from 'dynaconf list'Pavel Moravec2021-06-151-0/+10
| | | | | | | | | The command also prints content of /etc/pulp/settings.py where we need to also obfuscate the SECRET_KEY and PASSWORD values. Resolves: #2583 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [cleaner] Remove binary files by defaultJake Hunsaker2021-06-158-7/+167
| | | | | | | | | | | | | | | | | | Binary files generally speaking cannot be obfuscated, and as such we should remove them from archives being obfuscated by default so that sensitive data is not mistakenly included in an obfuscated archive. This commits adds a new `--keep-binary-files` option that if used will keep any encountered binary files in the final archive. The default option of `false` will ensure that encountered binary files are removed. The number of removed binary files per archive is reported when obfuscation is completed for that archive. Closes: #2478 Resolves: #2524 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [man] unify some syntax in manpagesPavel Moravec2021-06-144-4/+9
| | | | | | | | | | Unify capitalisation of name and synopsis. Add references to sos.conf to SEE ALSO of all binaries. Resolves: #2581 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [Plugin] Add default journal collection for Plugin servicesJake Hunsaker2021-06-1417-29/+64
| | | | | | | | | | | | | | | | | Adds automatic collection of journals for any service defined in a plugin's `services` tuple, if that service exists on the system. Updates several plugins to define a `services` tuple in place of manually calling `add_journal()` (and as a bonus, removing redundant `add_service_status()` calls that are already performed based on the tuple). Additionally, add an appropriate tag to each `add_journal` call for easier collection lookup in the manifest. Resolves: #2579 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [usbguard] New pluginRenaud Métrich2021-06-101-0/+29
| | | | | | | | This captures the persistent and runtime configuration of usbguard. Resolves: #2577 Signed-off-by: Renaud Métrich <rmetrich@redhat.com>
* [cleaner] Don't strip empty lines from substituted filesJake Hunsaker2021-06-081-2/+4
| | | | | | | | | | | | | Fixes an issue where empty lines would be stripped from files that have other obfuscations in them. Those empty lines may be important for file structure and/or readability, so we should instead simply not pass empty lines to the parsers rather than skipping them wholesale in the flow of writing obfuscations to a temp file before replacing the source file with a potentially changed temp file. Resolves: #2562 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [collect] Deprecate 'master' in favor of 'primary'Jake Hunsaker2021-06-086-11/+36
| | | | | | | | | | | | | | | | | | | | | | | | | | This commit begins the deprecation of the use of `master` in favor of `primary`. For this initial step, `sos collect` will deprecate the `--master` option and print a notice whenever it is used. This option is being replaced by `--primary` with aliases `--manager` and `--controller`. This deprecation will last through 4.2, with `--master` being removed in 4.3. Additionally, the `exec_master_cmd()` method for cluster profiles has been similarly deprecated in favor of `exec_primary_cmd()`, with a deprecation note logged whenever a cluster profile uses the former method. The internal assignment of the `SoSNode` object as `master` remains for now, as this initial step is to deprecate the user-visible bits. By full removal of `--master` in 4.3, these internal assignments should all be changed to `primary` as well. Related: #2329 Resolves: #2555 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [cirrus] Update Fedora testing for F34Jake Hunsaker2021-06-071-2/+2
| | | | | | | | Updates the testing matrix for F34, dropping F32. Resolves: #2574 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [sssd] sssd plugin when sssd-commonJan Jansky2021-06-031-1/+1
| | | | | | | | | | | | | | We have reports that sssd logs are not collected, when we investigated we found associate wants to collect sssd related logs also when only sssd-common package is installed. We got this confirmed by sbr-idm. Resolves: #2571 Signed-off-by: Jan Jansky <jjansky@redhat.com>
* Addd information about loop devicesJose Castillo2021-06-031-0/+2
| | | | | | | | This patch captures information from loop devices via 'losetup -a' and the content of /sys/block/loopN/loop/ directory. Signed-off-by: Jose Castillo <jcastillo@redhat.com>
* [tests] Add test for using `--encrypt-pass`Jake Hunsaker2021-06-032-2/+85
| | | | | | | | | | Adds two test cases for using `--encrypt-pass`; one with just the bare option and another in conjunction with `--clean` to ensure both can be used in the same execution. Resolves: #2566 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [ocp] Add cluster profile for OCP4Jake Hunsaker2021-06-022-8/+109
| | | | | | | | | Removes the previous OCP cluster profile and replaces it with an updated one for OCP4 which is entirely separated from the kubernetes profile. Resolves: #2544 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [sosnode] Mark sos commands as explicitly needing root for containersJake Hunsaker2021-06-021-3/+3
| | | | | | | | | | | | Fixes an issue where the sos inspection commands were not properly marked as needing to be run as root (either directly or via sudo) for containerized hosts, which would lead to incorrect sos command formatting. Mark those commands, and the final container removal command, as explicitly needing root permissions. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [sosnode] Don't fail on sos-less bastion nodes used for node listsJake Hunsaker2021-06-021-5/+12
| | | | | | | | | | | If the master node is determined to not have sos installed, that is not necessarily a fatal error for scenarios where the 'master' node is only being used to enumerate node lists and is not actually part of the cluster. This can happen when a user is using a bastion node to enumerate and connect to the cluster environment, or if the local host is being used to enumerate nodes via cluster client tooling. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [sosnode] Allow clusters to set options only for master nodesJake Hunsaker2021-06-022-0/+27
| | | | | | | Adds a method the `Cluster` that allows a profile to set sos options specifically for master nodes. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [collect] Add options to provide registry auth for pulling imagesJake Hunsaker2021-06-026-13/+140
| | | | | | | | | | | | | | | | | | Adds options that allow a user to specify registry authentication, either via username/password or an authfile, to allow pulling an image that exists on a non-public registry. If a username/password is provided, that will be used. If not, we will attempt to use an authfile - either provided by the user or by a cluster profile. Also adds an option to forcibly pull a new(er) version of the specified image, to alleviate conditions where a too-old version of the image already exists on the host. Closes: #2534 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [plugins] Add insights tags for numerous pluginsJake Hunsaker2021-06-0234-42/+206
| | | | | | | | | | | This commit adds tags to numerous plugins for use in conjunction with Red Hat Insights. Tags are added as generically as possible, however in cases where that is not feasible or logical, tags are added with the 'insights_' prefix. Resolves: #2536 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [Plugin] Add `cmd_as_tag` option to `add_cmd_output`Jake Hunsaker2021-06-021-3/+15
| | | | | | | Adds a new `cmd_as_tag` parameter to `add_cmd_output()` that, if `True`, will convert the command string to a tag. Default is `False`. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [rpm] Use query tags and drop use of piping to awkJake Hunsaker2021-06-011-23/+14
| | | | | | | | | | | | Removes the use of awk in the `rpm` command that generates the `installed-rpms` output, instead relying on rpm's builtin query tags. Further removes the `add_rpm_cmd` method and directly calls `add_cmd_output` with the proper parameters now. Resolves: #2548 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [Red Hat] Update policy to use SFTP instead of legacy FTP dropboxJake Hunsaker2021-06-011-39/+76
| | | | | | | | | | | | | | | | | | | As the FTP dropbox for Red Hat is being decomissioned and replaced with an SFTP alternative, update the policy to fallback to the SFTP host and remove legacy FTP host references. The default behavior for --upload remains the same, just targeting a different location. If a username, password, and case number are given, the first attempt will be to upload to the Red Hat Customer Portal. If any are missing, or are invalid, then we will fallback to SFTP. During the fallback if a valid username and password are not provided, sos will attempt to obtain an anonymous token for the upload before failing out entirely. Closes: #2467 Resolves: #2552 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [Policy] Add SFTP upload supportJake Hunsaker2021-06-014-5/+125
| | | | | | | | | | Adds support for uploading via SFTP. This is done via pexpect calling the system's locally available SFTP binary. If either that binary or pexpect are unavailable on the local system, we will exit gracefully and report the issue. This allows sos to keep python3-pexpect as a recommends rather than a hard dependency. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [juju] Don't allow collection of kvmArif Ali2021-06-011-0/+1
| | | | | | | | | | | With --all-logs, it could potentially collect the kvms that are created by juju, and hence the sos report size could be significant. This change will ensure that the kvm images are not collected as part of sos report. Closes: #2563 Signed-off-by: Arif Ali <arif.ali@canonical.com>
* [archive] skip copying SELinux context for /proc and /sys everytimePavel Moravec2021-06-012-18/+56
| | | | | | | | | | | A supplement of #1399 fix, now also for adding strings or special device files. Also adding a (vendor) test case for it. Resolves: #2560 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [candlepin] collect /etc/candlepin/broker.xmlPavel Moravec2021-05-261-0/+5
| | | | | | | | | | | The file contains important config about internal candlepin broker routing. Also, apply scrubbing of two passwords on a single line / in one URI. Resolves: #2558 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [tests] Improve failed command reporting in test outputJake Hunsaker2021-05-263-3/+13
| | | | | | | | | | | | | | | | | | | | Improves error reporting for failed sos commands by logging stderr (or stdout if stderr is not populated) to the console, which was previously being truncated by the builtin error handling of avocado. Printed output is limited to the last 8k to avoid dumping several MBs at a time for scenarios such as timeouts where command failure may generate significant logging prior to failing. Included with this are 2 minor changes to existing tests. First, remove verbose output from the expected plugins test to reduce otherwise irrelevant output for command failures. Second limit the number of plugins run for the LogLevelTest, both to reduce overall run time for a test where we aren't testing specific plugins and to improve readability of failures for such a test. Resolves: #2556 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [tests] Add test for priority parameter in add_cmd_output()Jake Hunsaker2021-05-242-0/+68
| | | | | | | | | Adds a test that checks that the new `priority` parameter is working as expected for well known and common collections. Resolves: #2553 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [plugins] Assign priority values to known long running commandsJake Hunsaker2021-05-2410-19/+23
| | | | | | | Assigns new `priority` values to known long running commands, to ensure those commands run last in their respective plugins' execution. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [Plugin] Allow plugins to influence order of command executionJake Hunsaker2021-05-241-12/+28
| | | | | | | | | | | | | | | | Adds a `priority` parameter to `add_cmd_output()` and `add_journal()`, that can be used to influence the order in which commands are executed within a plugin. This allows for plugins to specify that long-running commands should be run last, regardless of where in the plugin those commands are defined. As part of accounting for this, minor fixups in the evaluation of run times for individual commands and how those are reflected in the report's manifest are included. Closes: #1585 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [foreman] Collect all apache foreman* logs in foreman pluginPavel Moravec2021-05-241-1/+1
| | | | | | | | | | Some important logs are written to foreman_[access|error].log in /var/log/httpd dir. foreman plugin should collect all the logs that apache plugin skips to prevent duplicit collection. Resolves: #2554 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [foreman] Follow sizelimit to foreman maintain and installer logsPavel Moravec2021-05-241-2/+2
| | | | | | | | | | | Specifying add_copy_spec to a directory means all files from the directory are collected. That is ridiculous for foremain-installer and namely foreman-maintain logs, where we should stick to the size limit by default. Hence collect files from the dir "specifically". Resolves: #2554 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [networking] collect also tc filter show ingressPavel Moravec2021-05-201-1/+2
| | | | | | | | | Both "tc -s filter show dev %eth [|ingress]" commands required as they provide different output. Resolves: #2550 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [tests] Add test for plugin design conformanceJake Hunsaker2021-05-191-0/+57
| | | | | | | | | | Adds a unittest that ensures conformance for plugin design, e.g. ensuring that enablement triggers are proper tuples and not inadvertent strings due to a missing trailing comma for single-tuples. Resolves: #2549 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [plugins] Add missing plugin namesJake Hunsaker2021-05-195-4/+5
| | | | | | | | Several plugins did not set a `plugin_name` attribute, instead relying on a fallback using the class name. Set the `plugin_name` appropriately in these plugins. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
generated by cgit (git 2.34.1) at 2024-11-17 15:54:38 +0000