aboutsummaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* [Policy] Allow users to toggle SSL verification for HTTPS uploadsJake Hunsaker2021-05-144-9/+28
| | | | | | | | | | | Adds a new `--upload-no-ssl-verify` option that controls if we verify the SSL certificate for an upload target. This will default to `False` to retain previous default behavior of performing SSL verification. Closes: #2497 Resolves: #2540 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [Policy] Correct and clarify https upload internalsJake Hunsaker2021-05-146-17/+29
| | | | | | | | | | | | | | | | | | | | | | | | The python requests module does not provide actual support for streaming, that is provided by requests-toolchain. The support in requests is for easy multipart uploads. The internal methods and variables for https uploads within `Policy` however revolve around streaming enablement, but don't actually influence the use of streaming or not. This was due to placeholders during development just being carried forward rather than corrected before merge. This was largely forgotten about, until recent reports of upload issues for user-provided endpoints. This commit serves to correct the language around the currently supported https upload functionality. That is, we either use a 'put' or 'post' based on the loaded policy defaults. Further, allow users to control this setting with a new `--upload-method` option. This will allow users to specify upload urls that may require the opposite HTTP method than what the policy defaults dictate. Related: #2497 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [cirrus] Make auto_cancellation global instead of environmentalJake Hunsaker2021-05-141-4/+4
| | | | | | | | Moves auto_cancellation to a global setting, instead of environmental. Related: #2541 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [policy,collector] State explicitly that caseid is optionalPavel Moravec2021-05-142-3/+4
| | | | | | | | | While providing case id is fully optional, the current phrasing might be insintepreted it is mandatory. Resolves: #2542 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* Capture missing files in /etc/auditd/Jose Castillo2021-05-141-0/+2
| | | | | | | This commit captures the file audit-stop.rules, and directory plugins.d. Signed-off-by: Jose Castillo <jcastillo@redhat.com>
* [cirrus] Enable auto-cancellation of jobs and specify type for GCEJake Hunsaker2021-05-141-3/+6
| | | | | | | | | | | | | | | | | | | | | | Modifies the cirrus configuration to enable auto-cancellation of jobs for the master branch (it is already enabled for PR branches by default). This will prevent batch merges from kicking off a test job for each merge. While it may arise that two or more independent commits that pass testing on their own branches combine to form an unexpected failure, this possibility seems remote for sos. This is a cost saving decision - currently for each commit to master we are spinning up a minimum of 13 VMs, which can very quickly snowball if we do multiple merges in a short amount of time (which is historically how the project does merges). Second, modify the `gce_instance` fields in the cirrus config to use pre-defined machine types, as these match our current "custom" specifications already, and are cheaper to run. Resolves: #2541 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [tests] Add integration testing for ForemanJake Hunsaker2021-05-133-30/+143
| | | | | | | | | | | | | | | Adds an integration test for the upstream Foreman project. After stageone tests have passed, and if the PR has changes relevant to the Foreman plugins (or the cirrus config), launch a pre-built GCE image that has a Foreman deployment on it. Per request of the Foreman devel team, these tests will be run against both a CentOS and Debian based Foreman image. Currently, testing is performed against Foreman 2.4. Resolves: #2531 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [convert2rhel] Fix packages and profiles attrs as tuplesJake Hunsaker2021-05-121-2/+2
| | | | | | | | | Fixes the definition of `packages` and `profiles` within the plugin to be tuples. Resolves: #2537 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [archive] Don't shell out for compressing the archiveJake Hunsaker2021-05-125-62/+70
| | | | | | | | | | As sos is now python3-only, we can avoid shelling-out to compression utilities like `xz` or `gzip`, and instead use the method provided by the built-in `tarfile` module. Resolves: #2523 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [kernel]:Patch to update kernel plugin to collect debug infoMamatha Inamdar2021-05-111-0/+2
| | | | | | | | | This patch is to update kernel plugin to collect page allocation failure info Resolves: #2504 Signed-off-by: Mamatha Inamdar <mamatha4@linux.vnet.ibm.com>
* [snapper]:Ptach to update snapper plugin to collect snapper infoMamatha Inamdar2021-05-111-0/+27
| | | | | | | | | | This patch is to Introduce snapper plugin to collect /usr/lib/snapper/ information to check executable permission for installation-helper command Resolves: #2504 Signed-off-by: Mamatha Inamdar <mamatha4@linux.vnet.ibm.com>
* [block]:Patch to update block pluging to collect disk infoMamatha Inamdar2021-05-111-0/+1
| | | | | | | | | This patch is to update block plugin to collect state of sda Resolves: #2504 Signed-off-by: Mamatha Inamdar <mamatha4@linux.vnet.ibm.com>
* [saphana] remove redundant unused argument of get_inst_infoPavel Moravec2021-05-111-1/+1
| | | | | | | | get_inst_info does not use and isnt called with 'prefix' argument Resolves: #2535 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [virtwho] improve regexp processing performance on huge linesPavel Moravec2021-05-101-1/+1
| | | | | | | | We can ignore .* at the start of a RE. Resolves: #2527 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [zvm] Add new pluginJake Hunsaker2021-05-101-0/+109
| | | | | | | | | Adds a new plugin for z/VM collection, based on collections performed by the existing s390 dbginfo.sh support script. Related: #2514 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [s390] Integrate missing commands from IBM s390x dbginfo.shJake Hunsaker2021-05-102-3/+12
| | | | | | | | | | Adds additional s390-relevant commands from the dbginfo.sh support script for s390/s390X systems. Closes: #2228 Resolves: #2514 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [ssh] SSH and SSHD configurations may be fragmentedKevin C Myers2021-05-101-2/+21
| | | | | | | | | | | | The SSH configurations may have an include directive. This adjustment to the plugin will copy in the main configs and then read those for any include keywords and attempt to copy those as well. Resolves: #2528 Signed-off-by: Kevin C Myers kemyers at redhat dot com Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [candlepin] move tomcat logs from foreman pluginPavel Moravec2021-05-102-8/+8
| | | | | | | | | | | | | | Candlepin as a tomcat application needs to collect the logs, not foreman. Ideally, the tomcat plugin should be responsible for colecting the logs, but doing so would pollute non-candlepin usage (cf. apache logs in foreman plugin as well). Resolves: #2530 Signed-off-by: Pavel Moravec <pmoravec@redhat.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [foreman] apply config scrubbing also to installer logsPavel Moravec2021-05-101-20/+17
| | | | | | | | | | | | | As logs contain diffs of configs, we must apply the same config scrubbing to the logs as well. Further, extend logs files to obfuscate to the whole installer dir to cover also upstream-only filenames. Resolves: #2532 Signed-off-by: Pavel Moravec <pmoravec@redhat.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [tests|logs] Stream test data to journal, change inspected journalJake Hunsaker2021-05-051-9/+13
| | | | | | | | | | | | Changes the logs pre_sos_setup() to write to the journal using a stream rather than writing several MB directly all at once. This fixes an issue where using the `journal.send()` method from the systemd module would result in no data being written to the journal during the test setup. Second, switch over to inspecting the full journal rather than the most recent boot journal. Related: #2499
* [grub2] Add predicate on dm_mod for grub2-mkconfigJake Hunsaker2021-05-051-1/+2
| | | | | | | | The `grub2-mkconfig` command will load the `dm_mod` module if it is not already loaded. As such, add a predicate to skip command execution if that module is not loaded already. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [tests] assertFileGlobInArchive should only fail if file is presentJake Hunsaker2021-05-051-4/+8
| | | | | | | | | | | Like `assertFileCollected`, `assertFileGlobInArchive` should only fail if the referenced collection is not in the archive *and* it is present on the test system. If it is not present on the test system, sos cannot collect it and it is thus not an error to be missing from the archive. Related: #2499 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [devicemapper] Add check for dm_modJake Hunsaker2021-05-051-2/+4
| | | | | | | | | | | | | | It was discovered during testing that some distributions will have `dm_mod` as an unloaded kmod rather than being a builtin. For the moment, we cannot rely on checking loaded kernel mods as a predicate for `dm_mod` as it does not get listed when it is a builtin. Instead, check for the presence of `/sys/module/dm_mod` as the marker for if `dm_mod` is either a builtin or a currently loaded module. Related: #2499 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [policies] Improve kernel module loading to include builtinsJake Hunsaker2021-05-051-1/+38
| | | | | | | | | | | | | | | Adds determination of builtin modules to extend the list of kernel modules that are loaded via our parsing of `lsmod`. This is first done by parsing `/usr/lib/modules/$(uname -r)/modules.builtin`, and then by further inspecting `/boot/config-$(uname -r)` for specific kconfig strings that are well-known to build kernel modules into the running kernel, but for those that do not appear in the modules.builtin file above. Closes: #2500 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [tests] Correct test package installationJake Hunsaker2021-05-052-10/+9
| | | | | | | | | | | | | | | | First, correct the logs plugin stagetwo test to specify the needed package in a list, and second move the import to within the test class that needs it so that we don't fail on the stageone tests if the package is not present. Second, correct the journal injections that this plugin test does to use `pre_sos_setup()` instead of overriding `setup_mocking()`, and call the former after the mocking has been setup. Finally, add the resulting size of the collected journal if we fail the size requirements of the test. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [cirrus] Add testing on CentOS Stream 8Jake Hunsaker2021-05-051-10/+17
| | | | | | | | | | | | | | | As CentOS Stream now runs ahead of RHEL, instead of behind, using it as a testing distribution on GCE allows us to have more confidence on those distributions than testing on Fedora alone. As CentOS Stream does not have version locks, there is only one version to test on, rather than two as is the case with Fedora and Ubuntu. The image we build from will need to be updated more regularly however as new versions are released following RHEL minor version releases. Resolves: #2499 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [openstack_database] Add new pluginJake Hunsaker2021-05-051-0/+91
| | | | | | | | | | | | | | | | | | | Adds a new plugin for openstack deployments that collects information about/from the underlying database for the deployment. This plugin crosses multiple openstack components (for which there is typically one component for each), since that is preferable to adding a host of database-related options to each individual plugin. By default, this plugin will only collect the output of `clustercheck`. When using either the `dump` or `dumpall` plugin options, this plugin will also collect a mysql dump of the database; either for specific tables or all tables present respectively. Closes: #1256 Closes: #1984 Resolves: #2519 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [foreman] fix unmatched group error in scrubbing installer logsPavel Moravec2021-05-041-1/+1
| | | | | | | | | | Having just one whitespace before "Found key:" failed to match the first group. Resolves: #2521 Signed-off-by: Pavel Moravec <pmoravec@redhat.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [tests] Add test class for when exceptions are expected in an executionJake Hunsaker2021-05-032-6/+89
| | | | | | | | | | | | | | Adds a new `StageOneReportExceptionTest` class that is to be used for tests that should ensure an exception IS hit, for example when testing plugin enablement or invalid options. Included with this commit is a small set of plugin enablement tests. Future tests for component and plugin specific exception handling should be built using this class and design approach. Resolves: #2518 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [pulpcore] add plugin for pulp-3Pavel Moravec2021-05-031-0/+120
| | | | | | | | | | | Pulp-3 / pulpcore as a revolution from pulp-2 needs a separate plugin, since both plugins have nothing in common and there might be deployments where is active both pulp-2 and pulp-3. Resolves: #2278 Signed-off-by: Pavel Moravec <pmoravec@redhat.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [apache] Forbid collecting openstack logsJake Hunsaker2021-04-301-3/+13
| | | | | | | | | | | Like we do for foreman and pulp, avoid collecting logs from the various openstack components that write to /var/log/httpd/*, instead allowing those plugins to set the proper collections. Closes: #1950 Resolves: #2516 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [reporting] Sort order of report contents by nameJake Hunsaker2021-04-301-1/+4
| | | | | | | | | | | When using `reporting.py`, e.g. for `sos.txt`, sort the content on a per-section basis so that comparison between multiple sos reports is easier. Closes: #1998 Resolves: #2515 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [man] Add missing log-size report option to collect manpageJake Hunsaker2021-04-301-0/+13
| | | | | | | | | The `collect` component supports `--log-size` as a passthru option for report. Add it to the `collect` manpage for completeness. Related: #2510 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [man] Properly document size limits in manpageJake Hunsaker2021-04-301-3/+11
| | | | | | | | | | | Updates the description of `--log-size` in the manpage for `report` so that it accurately reflects how size limitations are determined and imposed during collections. Closes: #1900 Resolves: #2513 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [Plugin] Make command output size limiting obviousJake Hunsaker2021-04-303-5/+35
| | | | | | | | | | | | | | | | | | | | | When command output is size limited it should be made obvious that it is not the whole output of the command, beyond the collected output (most likely) starting in the middle of a string. To do this, we add a 'truncated' key to the results dict returned by `sos_get_command_output()`, which is set based on if the `AsyncReader` was filled during command execution. From that, first log if a command was truncated. Second, instead of dropping the collected output in the normal plugin directory, instead drop it into `sos_strings` with a `.tailed` extention to mimic how we handled tailed file collections. Finally, add a symlink from the plugin dir with the original filename that points to the `sos_strings` path. Closes: #1332 Resolves: #2506 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [tests] Use re.search for assertOutput(Not)ContainsJake Hunsaker2021-04-281-2/+4
| | | | | | | | | | | Standardizes the `assertOutputContains` and `assertOutputNotContains` methods to use `re.search()`, instead of separate parsing methods. `search()` replaces `match()` for efficiency purposes against the collected output. Related: #2507 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [logging] Fix archive debug logging, adjust verbosity levelsJake Hunsaker2021-04-284-4/+30
| | | | | | | | | | | | | | | | | Fixes an issue where archive debug logging was controlled by the use of `--debug` rather than `--verbose`. Removes a superfluous log in `FileCacheArchive.add_link()`. Also, adjusts the different verbosity levels to be more meaninful. Now, the use of `-v` will enable debug logging but will not print those messages to console, `-vv` will print debug logging to console, while `-vvv` will enable archive debug logging which is expected to be significant due to most file operations being logging at some point with archive debug logging. Resolves: #2507 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [insights] Removeinsights-client command as it's superfluousPaul Wayper2021-04-261-5/+0
| | | | | | | | | | | | The insights-client command should not be run within sosreport, as it collects data sosreport already collects and is not cautious about how it does so. Running it is therefore superfluous. Closes: #2508 Resolves: #2509 Signed-off-by: Paul Wayper <paulway@redhat.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [apache] Refactor duplicate code for Jboss file collectionsJake Hunsaker2021-04-261-38/+34
| | | | | | | | | | | | | | | Refactor the collections for the different httpd versions collected by the RedHatApache instance of this plugin, to remove the repetition of similar collection points. Moving forward, collection of newer httpd versions will only need to specify the version number, rather than explicitly repeat the same file collection paths. Closes: #704 Resolves: #2505 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [subscription_manager] call curl to the RHSM serverPavel Moravec2021-04-261-0/+52
| | | | | | | | | | | Collect a verbose curl command output, run against RHSM server like subscription-manager does. This helps troubleshooting with RHSM proxy issues. Resolves: #2503 Signed-off-by: Pavel Moravec <pmoravec@redhat.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [chrony] configuration can now be fragmentedEric Desrochers2021-04-261-0/+2
| | | | | | | | | | | | | | | | | | | Chrony 4.0 added better support for fragmented configuration. Chrony's configuration and NTP sources can now be under: NTP sources: /etc/chrony/sources.d/ Chrony's configuration: /etc/chrony/conf.d/ Resolves: #2501 Signed-off-by: Eric Desrochers <eric.desrochers@canonical.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [process] Collect per-process files from /procJake Hunsaker2021-04-162-1/+47
| | | | | | | | | | | | | | | | | | | | Adds collection of per-process files from /proc/$pid, limited to a number of pids controlled by the new `process.numprocs` option. This option will default to the first 2048 processes. Using a value of `0` for this option will result in all pids being collected. In testing, the initial limit of 2048 process did not significantly raise collection time or archive size. Note however, that collection of 20k or more processes does show a significant increase in plugin execution time and archive size. This is however tempered, by the whole-plugin timeout that sos imposes. Related: #436 Closes: #542 Resolves: #1783 Resolves: #2498 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [sosnode] Avoid checksum cleanup if archive was not generatedJake Hunsaker2021-04-161-4/+4
| | | | | | | | | | | Fixes an exception propagation from `cleanup()` where an attempt to look for and remove a checksum file was made when an archive was not generated (and thus no checksum would be present either). Related: #2488 Resolves: #2494 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [clean] Remove duplicate file skipping within obfuscate_line()Jake Hunsaker2021-04-162-10/+3
| | | | | | | | | | | | | | | A redundant file skipping check was being executed within `obfuscate_line()` that would cause subsequent archives being obfuscated to skip line obfuscation within a file, despite iterating through the entire file. Remove this redundant check, thus allowing proper obfuscation. Closes: #2490 Related: RHBZ#1930181 Resolves: #2492 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [clean] Load maps from all archives before obfuscation loopJake Hunsaker2021-04-162-37/+45
| | | | | | | | | | | | | | | | Previously, maps were being prepped via archives after extraction. This reduced the amount of file IO being done, but made it so that necessary obfuscations from later archives in a series would not be obfuscated in the archives obfuscated before those later archives were extracted. Fix this by extracting the map prep files into memory for each archive to prep the maps before we enter the obfuscation loop entirely. Closes: #2490 Related: RHBZ#1930181 Resolves: #2492 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [global] Drop plugin versionEric Desrochers2021-04-154-9/+0
| | | | | | | | | | | | | Plugin version never really took off. A very few plugins had it configured (e.g. jars, pacemaker) It also generated unhelpful noise in version.txt. Closes: #2495 Resolves: #2496 Signed-off-by: Eric Desrochers <eric.desrochers@canonical.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [networking] check for the presence of devlinkEric Desrochers2021-04-151-7/+13
| | | | | | | | | | | On certain kernel configuration, devlink cmds may trigger the module to load automatically. Closes: #2468 Resolves: #2487 Signed-off-by: Eric Desrochers <eric.desrochers@canonical.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [ipmitool] Add elist and lan command outputJake Hunsaker2021-04-151-7/+6
| | | | | | | | | | | | | | | | This commit adds collection of `ipmitool isel elist` and adds verbosity to the regular `ipmitool sel list` collection. Additionally, simplify the base command construction logic, and pull in `ipmitool lan print` from #998. Note that network obfuscation is relegated to the use of `sos clean` here, rather than doing plugin-based scrubbing. Closes: #998 Closes: #2015 Resolves: #2486 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [cloud_init] Collect command output and update file collectionsJake Hunsaker2021-04-151-2/+8
| | | | | | | | | | | | | | Adds collection of several `cloud-init` command outputs, and additionally updates the file collections performed based on the behavior of `cloud-init collect-logs`. Most notably, removes the collection of `/var/lib/cloud-init` to avoid possible user sensitive data being collected. Closes: #2080 Resolves: #2485 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [IBM] Remove unneeded policiesBryan Quigley2021-04-152-94/+0
| | | | | | | | | | | These policies haven't done anything since IndependentPlugin was made a more common default. Closes: #2470 Resolves: #2474 Signed-off-by: Bryan Quigley <code@bryanquigley.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
generated by cgit (git 2.34.1) at 2024-11-11 12:12:31 +0000