aboutsummaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* [Policy] Add SFTP upload supportJake Hunsaker2021-06-014-5/+125
| | | | | | | | | | Adds support for uploading via SFTP. This is done via pexpect calling the system's locally available SFTP binary. If either that binary or pexpect are unavailable on the local system, we will exit gracefully and report the issue. This allows sos to keep python3-pexpect as a recommends rather than a hard dependency. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [juju] Don't allow collection of kvmArif Ali2021-06-011-0/+1
| | | | | | | | | | | With --all-logs, it could potentially collect the kvms that are created by juju, and hence the sos report size could be significant. This change will ensure that the kvm images are not collected as part of sos report. Closes: #2563 Signed-off-by: Arif Ali <arif.ali@canonical.com>
* [archive] skip copying SELinux context for /proc and /sys everytimePavel Moravec2021-06-012-18/+56
| | | | | | | | | | | A supplement of #1399 fix, now also for adding strings or special device files. Also adding a (vendor) test case for it. Resolves: #2560 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [candlepin] collect /etc/candlepin/broker.xmlPavel Moravec2021-05-261-0/+5
| | | | | | | | | | | The file contains important config about internal candlepin broker routing. Also, apply scrubbing of two passwords on a single line / in one URI. Resolves: #2558 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [tests] Improve failed command reporting in test outputJake Hunsaker2021-05-263-3/+13
| | | | | | | | | | | | | | | | | | | | Improves error reporting for failed sos commands by logging stderr (or stdout if stderr is not populated) to the console, which was previously being truncated by the builtin error handling of avocado. Printed output is limited to the last 8k to avoid dumping several MBs at a time for scenarios such as timeouts where command failure may generate significant logging prior to failing. Included with this are 2 minor changes to existing tests. First, remove verbose output from the expected plugins test to reduce otherwise irrelevant output for command failures. Second limit the number of plugins run for the LogLevelTest, both to reduce overall run time for a test where we aren't testing specific plugins and to improve readability of failures for such a test. Resolves: #2556 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [tests] Add test for priority parameter in add_cmd_output()Jake Hunsaker2021-05-242-0/+68
| | | | | | | | | Adds a test that checks that the new `priority` parameter is working as expected for well known and common collections. Resolves: #2553 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [plugins] Assign priority values to known long running commandsJake Hunsaker2021-05-2410-19/+23
| | | | | | | Assigns new `priority` values to known long running commands, to ensure those commands run last in their respective plugins' execution. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [Plugin] Allow plugins to influence order of command executionJake Hunsaker2021-05-241-12/+28
| | | | | | | | | | | | | | | | Adds a `priority` parameter to `add_cmd_output()` and `add_journal()`, that can be used to influence the order in which commands are executed within a plugin. This allows for plugins to specify that long-running commands should be run last, regardless of where in the plugin those commands are defined. As part of accounting for this, minor fixups in the evaluation of run times for individual commands and how those are reflected in the report's manifest are included. Closes: #1585 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [foreman] Collect all apache foreman* logs in foreman pluginPavel Moravec2021-05-241-1/+1
| | | | | | | | | | Some important logs are written to foreman_[access|error].log in /var/log/httpd dir. foreman plugin should collect all the logs that apache plugin skips to prevent duplicit collection. Resolves: #2554 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [foreman] Follow sizelimit to foreman maintain and installer logsPavel Moravec2021-05-241-2/+2
| | | | | | | | | | | Specifying add_copy_spec to a directory means all files from the directory are collected. That is ridiculous for foremain-installer and namely foreman-maintain logs, where we should stick to the size limit by default. Hence collect files from the dir "specifically". Resolves: #2554 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [networking] collect also tc filter show ingressPavel Moravec2021-05-201-1/+2
| | | | | | | | | Both "tc -s filter show dev %eth [|ingress]" commands required as they provide different output. Resolves: #2550 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [tests] Add test for plugin design conformanceJake Hunsaker2021-05-191-0/+57
| | | | | | | | | | Adds a unittest that ensures conformance for plugin design, e.g. ensuring that enablement triggers are proper tuples and not inadvertent strings due to a missing trailing comma for single-tuples. Resolves: #2549 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [plugins] Add missing plugin namesJake Hunsaker2021-05-195-4/+5
| | | | | | | | Several plugins did not set a `plugin_name` attribute, instead relying on a fallback using the class name. Set the `plugin_name` appropriately in these plugins. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [Plugin] Set option_list by default to empty listJake Hunsaker2021-05-191-2/+1
| | | | | | | | Rather than relying on a `getattr()` check returning False during a plugin's initialization, simply set `option_list` to default to an empty list. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [plugins] Add missing short_desc to pluginsJake Hunsaker2021-05-194-3/+5
| | | | | | Adds a `short_desc` attribute to several plugins missing it. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [plugins] Fix incorrect tuples in plugin attributesJake Hunsaker2021-05-1926-42/+32
| | | | | | | | | | Fixes a number of plugins whose enablement attributes (profiles, packages, commands, files) were incorrectly formatted and not actual tuples. Related: #2549 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [report] Do not fail on skipping non-existing pluginsJake Hunsaker2021-05-182-4/+19
| | | | | | | | | | | | | | | | | | | Specifying a non-existing plugin to skip is no longer considered a fatal error. This was previously in place, in part, for third party utilities calling sos to ensure adherence to known plugins. However, since `collect` has been integrated into sos natively, this is far less of a concern. Specfying non-existing plugins will now only generate a warning message for plugins wanting to be skipped (`-n`). If non-existing plugins are specified for enablement via `-e` or `-o`, that will still generate a fatal error. Closes: #537 Closes: #1723 Resolves: #2517 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [networking] Add nstat command supportSeiichi Ikarashi2021-05-141-0/+2
| | | | | | As netstat command is being deprecated, we need nstat as an alternative to "netstat -s". Signed-off-by: Seiichi Ikarashi <s.ikarashi@fujitsu.com>
* [docs] Clarify profile in manpage and how to enable further pluginsJake Hunsaker2021-05-141-6/+14
| | | | | | | | | | | | | | | Updates the language in the manpage to clarify that in order to enable additional plugins outside of a profile when a profile is specified, that users must use `-o` instead of `-e`. Remove the 'current profiles' list from the manpage as it is outdated and it is better to refer users to the list reported by the local installation. Closes: #504 Resolves: #2543 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [Policy] Allow users to toggle SSL verification for HTTPS uploadsJake Hunsaker2021-05-144-9/+28
| | | | | | | | | | | Adds a new `--upload-no-ssl-verify` option that controls if we verify the SSL certificate for an upload target. This will default to `False` to retain previous default behavior of performing SSL verification. Closes: #2497 Resolves: #2540 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [Policy] Correct and clarify https upload internalsJake Hunsaker2021-05-146-17/+29
| | | | | | | | | | | | | | | | | | | | | | | | The python requests module does not provide actual support for streaming, that is provided by requests-toolchain. The support in requests is for easy multipart uploads. The internal methods and variables for https uploads within `Policy` however revolve around streaming enablement, but don't actually influence the use of streaming or not. This was due to placeholders during development just being carried forward rather than corrected before merge. This was largely forgotten about, until recent reports of upload issues for user-provided endpoints. This commit serves to correct the language around the currently supported https upload functionality. That is, we either use a 'put' or 'post' based on the loaded policy defaults. Further, allow users to control this setting with a new `--upload-method` option. This will allow users to specify upload urls that may require the opposite HTTP method than what the policy defaults dictate. Related: #2497 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [cirrus] Make auto_cancellation global instead of environmentalJake Hunsaker2021-05-141-4/+4
| | | | | | | | Moves auto_cancellation to a global setting, instead of environmental. Related: #2541 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [policy,collector] State explicitly that caseid is optionalPavel Moravec2021-05-142-3/+4
| | | | | | | | | While providing case id is fully optional, the current phrasing might be insintepreted it is mandatory. Resolves: #2542 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* Capture missing files in /etc/auditd/Jose Castillo2021-05-141-0/+2
| | | | | | | This commit captures the file audit-stop.rules, and directory plugins.d. Signed-off-by: Jose Castillo <jcastillo@redhat.com>
* [cirrus] Enable auto-cancellation of jobs and specify type for GCEJake Hunsaker2021-05-141-3/+6
| | | | | | | | | | | | | | | | | | | | | | Modifies the cirrus configuration to enable auto-cancellation of jobs for the master branch (it is already enabled for PR branches by default). This will prevent batch merges from kicking off a test job for each merge. While it may arise that two or more independent commits that pass testing on their own branches combine to form an unexpected failure, this possibility seems remote for sos. This is a cost saving decision - currently for each commit to master we are spinning up a minimum of 13 VMs, which can very quickly snowball if we do multiple merges in a short amount of time (which is historically how the project does merges). Second, modify the `gce_instance` fields in the cirrus config to use pre-defined machine types, as these match our current "custom" specifications already, and are cheaper to run. Resolves: #2541 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [tests] Add integration testing for ForemanJake Hunsaker2021-05-133-30/+143
| | | | | | | | | | | | | | | Adds an integration test for the upstream Foreman project. After stageone tests have passed, and if the PR has changes relevant to the Foreman plugins (or the cirrus config), launch a pre-built GCE image that has a Foreman deployment on it. Per request of the Foreman devel team, these tests will be run against both a CentOS and Debian based Foreman image. Currently, testing is performed against Foreman 2.4. Resolves: #2531 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [convert2rhel] Fix packages and profiles attrs as tuplesJake Hunsaker2021-05-121-2/+2
| | | | | | | | | Fixes the definition of `packages` and `profiles` within the plugin to be tuples. Resolves: #2537 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [archive] Don't shell out for compressing the archiveJake Hunsaker2021-05-125-62/+70
| | | | | | | | | | As sos is now python3-only, we can avoid shelling-out to compression utilities like `xz` or `gzip`, and instead use the method provided by the built-in `tarfile` module. Resolves: #2523 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [kernel]:Patch to update kernel plugin to collect debug infoMamatha Inamdar2021-05-111-0/+2
| | | | | | | | | This patch is to update kernel plugin to collect page allocation failure info Resolves: #2504 Signed-off-by: Mamatha Inamdar <mamatha4@linux.vnet.ibm.com>
* [snapper]:Ptach to update snapper plugin to collect snapper infoMamatha Inamdar2021-05-111-0/+27
| | | | | | | | | | This patch is to Introduce snapper plugin to collect /usr/lib/snapper/ information to check executable permission for installation-helper command Resolves: #2504 Signed-off-by: Mamatha Inamdar <mamatha4@linux.vnet.ibm.com>
* [block]:Patch to update block pluging to collect disk infoMamatha Inamdar2021-05-111-0/+1
| | | | | | | | | This patch is to update block plugin to collect state of sda Resolves: #2504 Signed-off-by: Mamatha Inamdar <mamatha4@linux.vnet.ibm.com>
* [saphana] remove redundant unused argument of get_inst_infoPavel Moravec2021-05-111-1/+1
| | | | | | | | get_inst_info does not use and isnt called with 'prefix' argument Resolves: #2535 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [virtwho] improve regexp processing performance on huge linesPavel Moravec2021-05-101-1/+1
| | | | | | | | We can ignore .* at the start of a RE. Resolves: #2527 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [zvm] Add new pluginJake Hunsaker2021-05-101-0/+109
| | | | | | | | | Adds a new plugin for z/VM collection, based on collections performed by the existing s390 dbginfo.sh support script. Related: #2514 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [s390] Integrate missing commands from IBM s390x dbginfo.shJake Hunsaker2021-05-102-3/+12
| | | | | | | | | | Adds additional s390-relevant commands from the dbginfo.sh support script for s390/s390X systems. Closes: #2228 Resolves: #2514 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [ssh] SSH and SSHD configurations may be fragmentedKevin C Myers2021-05-101-2/+21
| | | | | | | | | | | | The SSH configurations may have an include directive. This adjustment to the plugin will copy in the main configs and then read those for any include keywords and attempt to copy those as well. Resolves: #2528 Signed-off-by: Kevin C Myers kemyers at redhat dot com Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [candlepin] move tomcat logs from foreman pluginPavel Moravec2021-05-102-8/+8
| | | | | | | | | | | | | | Candlepin as a tomcat application needs to collect the logs, not foreman. Ideally, the tomcat plugin should be responsible for colecting the logs, but doing so would pollute non-candlepin usage (cf. apache logs in foreman plugin as well). Resolves: #2530 Signed-off-by: Pavel Moravec <pmoravec@redhat.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [foreman] apply config scrubbing also to installer logsPavel Moravec2021-05-101-20/+17
| | | | | | | | | | | | | As logs contain diffs of configs, we must apply the same config scrubbing to the logs as well. Further, extend logs files to obfuscate to the whole installer dir to cover also upstream-only filenames. Resolves: #2532 Signed-off-by: Pavel Moravec <pmoravec@redhat.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [tests|logs] Stream test data to journal, change inspected journalJake Hunsaker2021-05-051-9/+13
| | | | | | | | | | | | Changes the logs pre_sos_setup() to write to the journal using a stream rather than writing several MB directly all at once. This fixes an issue where using the `journal.send()` method from the systemd module would result in no data being written to the journal during the test setup. Second, switch over to inspecting the full journal rather than the most recent boot journal. Related: #2499
* [grub2] Add predicate on dm_mod for grub2-mkconfigJake Hunsaker2021-05-051-1/+2
| | | | | | | | The `grub2-mkconfig` command will load the `dm_mod` module if it is not already loaded. As such, add a predicate to skip command execution if that module is not loaded already. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [tests] assertFileGlobInArchive should only fail if file is presentJake Hunsaker2021-05-051-4/+8
| | | | | | | | | | | Like `assertFileCollected`, `assertFileGlobInArchive` should only fail if the referenced collection is not in the archive *and* it is present on the test system. If it is not present on the test system, sos cannot collect it and it is thus not an error to be missing from the archive. Related: #2499 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [devicemapper] Add check for dm_modJake Hunsaker2021-05-051-2/+4
| | | | | | | | | | | | | | It was discovered during testing that some distributions will have `dm_mod` as an unloaded kmod rather than being a builtin. For the moment, we cannot rely on checking loaded kernel mods as a predicate for `dm_mod` as it does not get listed when it is a builtin. Instead, check for the presence of `/sys/module/dm_mod` as the marker for if `dm_mod` is either a builtin or a currently loaded module. Related: #2499 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [policies] Improve kernel module loading to include builtinsJake Hunsaker2021-05-051-1/+38
| | | | | | | | | | | | | | | Adds determination of builtin modules to extend the list of kernel modules that are loaded via our parsing of `lsmod`. This is first done by parsing `/usr/lib/modules/$(uname -r)/modules.builtin`, and then by further inspecting `/boot/config-$(uname -r)` for specific kconfig strings that are well-known to build kernel modules into the running kernel, but for those that do not appear in the modules.builtin file above. Closes: #2500 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [tests] Correct test package installationJake Hunsaker2021-05-052-10/+9
| | | | | | | | | | | | | | | | First, correct the logs plugin stagetwo test to specify the needed package in a list, and second move the import to within the test class that needs it so that we don't fail on the stageone tests if the package is not present. Second, correct the journal injections that this plugin test does to use `pre_sos_setup()` instead of overriding `setup_mocking()`, and call the former after the mocking has been setup. Finally, add the resulting size of the collected journal if we fail the size requirements of the test. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [cirrus] Add testing on CentOS Stream 8Jake Hunsaker2021-05-051-10/+17
| | | | | | | | | | | | | | | As CentOS Stream now runs ahead of RHEL, instead of behind, using it as a testing distribution on GCE allows us to have more confidence on those distributions than testing on Fedora alone. As CentOS Stream does not have version locks, there is only one version to test on, rather than two as is the case with Fedora and Ubuntu. The image we build from will need to be updated more regularly however as new versions are released following RHEL minor version releases. Resolves: #2499 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [openstack_database] Add new pluginJake Hunsaker2021-05-051-0/+91
| | | | | | | | | | | | | | | | | | | Adds a new plugin for openstack deployments that collects information about/from the underlying database for the deployment. This plugin crosses multiple openstack components (for which there is typically one component for each), since that is preferable to adding a host of database-related options to each individual plugin. By default, this plugin will only collect the output of `clustercheck`. When using either the `dump` or `dumpall` plugin options, this plugin will also collect a mysql dump of the database; either for specific tables or all tables present respectively. Closes: #1256 Closes: #1984 Resolves: #2519 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [foreman] fix unmatched group error in scrubbing installer logsPavel Moravec2021-05-041-1/+1
| | | | | | | | | | Having just one whitespace before "Found key:" failed to match the first group. Resolves: #2521 Signed-off-by: Pavel Moravec <pmoravec@redhat.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [tests] Add test class for when exceptions are expected in an executionJake Hunsaker2021-05-032-6/+89
| | | | | | | | | | | | | | Adds a new `StageOneReportExceptionTest` class that is to be used for tests that should ensure an exception IS hit, for example when testing plugin enablement or invalid options. Included with this commit is a small set of plugin enablement tests. Future tests for component and plugin specific exception handling should be built using this class and design approach. Resolves: #2518 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [pulpcore] add plugin for pulp-3Pavel Moravec2021-05-031-0/+120
| | | | | | | | | | | Pulp-3 / pulpcore as a revolution from pulp-2 needs a separate plugin, since both plugins have nothing in common and there might be deployments where is active both pulp-2 and pulp-3. Resolves: #2278 Signed-off-by: Pavel Moravec <pmoravec@redhat.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [apache] Forbid collecting openstack logsJake Hunsaker2021-04-301-3/+13
| | | | | | | | | | | Like we do for foreman and pulp, avoid collecting logs from the various openstack components that write to /var/log/httpd/*, instead allowing those plugins to set the proper collections. Closes: #1950 Resolves: #2516 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
generated by cgit (git 2.34.1) at 2024-11-17 21:32:12 +0000