aboutsummaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* [component] Use sysroot from Policy when opts doesn't specify itPavel Moravec2020-12-151-2/+2
| | | | | | | | | | Until --sysroot option is specified, Archive (sub)classes should be called with sysroot determined from Policy. Resolves: #2346 Signed-off-by: Pavel Moravec <pmoravec@redhat.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [Openvswitch] Add ovs-configuration journalJuan-Luis de Sousa-Valadas Castaño2020-12-151-0/+1
| | | | | | | | | | | | OpenShift 4.6 Introduces a new service called ovs-configuration, usually this information is also acquired in the must-gather but if precisely this service is at fault we may not be able to run must-gather. Resolves: #2345 Signed-off-by: Juan Luis de Sousa-Valadas <jdesousa@redhat.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [policy] Add /sbin to PATH for SuSE policy.Daria Bukharina2020-12-151-1/+1
| | | | | | | | | | | These changes fix iscsi plugin for SuSE distributions because `iscsiadm` util is placed in /sbin. Resolves: #2344 Closes: #2340 Signed-off-by: Daria Bukharina d.bukharina@yadro.com Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [subscription_manager] scrub passwords on one line onlyPavel Moravec2020-12-151-2/+2
| | | | | | | | | | | Fix regexp to scrub proxy_password on one line and don't obfuscate subsequent line when the password is empty. Closes: #2328 Resolves: #2343 Signed-off-by: Pavel Moravec <pmoravec@redhat.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [report] collect broken symlinksPavel Moravec2020-12-141-6/+10
| | | | | | | | | | | | Information about broken symlink destination is useful information that sos report should collect. Currently it stops doing so as stat-ing the symlink to determine filesize fails. Closes: #2333 Resolves: #2338 Signed-off-by: Pavel Moravec <pmoravec@redhat.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [collect] Add unavailable warning to `sos collect --help`Jake Hunsaker2020-12-131-0/+23
| | | | | | | | | | | | Adds a warning message to the end of `sos collect --help` output when the `collect` component is not available due to a missing dependency or missing sub package. Closes: #2336 Resolves: #2341 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com> Signed-off-by: Bryan Quigley <code@bryanquigley.com>
* [networking] Add kmod predicate to `ufw` commandsJake Hunsaker2020-12-131-1/+5
| | | | | | | | | | | The `ufw` commands will load the `bpfilter` and `iptable_filter` kernel modules if they are not already present. As such, add a `SoSPredicate` for those kernel modules to gate collection of `ufw` commands. Related: #2326 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com> Signed-off-by: Bryan Quigley <code@bryanquigley.com>
* [lxd] Add predicate to lxd/lxc commands for network kmodsJake Hunsaker2020-12-111-3/+24
| | | | | | | | | | | The `lxd` and `lxc` commands will load a handful of networking related kernel modules if those modules are not already loaded. As such, gate running those commands from the `lxd` plugin with a proper `SoSPredicate`. Related: #2326 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [wireless] Update enablement triggers and add predicate for iw commandsJake Hunsaker2020-12-111-13/+6
| | | | | | | | | | | | | | First, update the enablement trigger to use the `commands` tuple to use a policy's defined PATH to search for the `iw` command, rather than splitting out subclasses based on the `files` tuple. Second, add a predicate that requires the `cfg80211` kmod to be loaded before running any `iw` commands, as this command will load the module if it is not already loaded. Related: #2326 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [tests] Add test for newly loaded kernel modulesJake Hunsaker2020-12-111-2/+10
| | | | | | | | | | | | | | | Adds a test to `simple.sh` to check if we loaded any new kernel modules during a normal exeuction. This test will be somewhat limited in the fact that this currently only tests the plugins that are loaded by default on each tested distribution. Also included is a minor typo fix for reporting where IP addresses were not obfuscated, if any. This typo did not affect the actual test however. Resolves: #2326 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [pmem] Updated for ipmctl v2.xSteve Scargall2020-12-111-6/+11
| | | | | | | | | | ipmctl version 2.x introduces new commands and options. It also deprecates some older commands. Resolves: #2161 Signed-off-by: Steve Scargall <steve.scargall@intel.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [options] Fix --log-size=0 being ignored and unreported otherwiseJake Hunsaker2020-12-102-1/+4
| | | | | | | | | | | | | | | | | | The `--log-size` option was being silently ignored, due to a too-loose conditional in `Component.apply_options_from_cmdline()` which was inadvertently filtering out the option when a user set it to 0. Note that this did not affect `sos.conf` settings for this same value. Similarly, reporting the effective options after preset and cmdline merging was skipping log-size when it was set to 0, since we normally want to filter out null-value options (which imply they were not invoked). Adding an explicit check for `log-size` here is the easiest route forward to allow the reporting we expect. Closes: #2334 Resolves: #2335 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [crio] collect "crio config"Pavel Moravec2020-12-101-1/+4
| | | | | | | | | | The command dumps layered configuration currently used, which sometimes supersets config files. Resolves: #2337 Signed-off-by: Pavel Moravec <pmoravec@redhat.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [collector] allow overriding plain --cluster-typePavel Moravec2020-12-091-0/+1
| | | | | | | | | | | In few user scenarios, it is useful to force sos collect to override cluster type, but let it generate list of nodes by itself. For that, it is sufficient to set the self.cluster_type accordingly. Resolves: #2331 Signed-off-by: Pavel Moravec <pmoravec@redhat.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [plugins] Dont stop collecting by empty specfile when sizelimit=0Pavel Moravec2020-12-091-1/+1
| | | | | | | | | | When sizelimit=0, collecting an empty file would set limit_reached wrongly and stop collecting further files. Let fix this corner case. Resolves: #2330 Signed-off-by: Pavel Moravec <pmoravec@redhat.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [fibrechannel] Add optional argument for plugin to collect debug infoDaria Bukharina2020-12-021-0/+10
| | | | | | | | | | | Add a possibility to enable collecting debug logs from vendor specific paths. To enable this feature you need to pass optional `debug` argument. Resolves: #2324 Signed-off-by: Daria Bukharina d.bukharina@yadro.com Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [apt] Move unattended-upgrades log collectionEric Desrochers2020-11-302-2/+3
| | | | | | | | | | | | | | | | Having u-u logs collection inside the apt plugin make more sense as it is using apt functionnalities and python modules such as apt, apt_inst, apt_pkg. u-u is a mechanism to automatically install security upgrades on Debian/Ubuntu. It is enabled by default at installation. Resolves: #2323 Signed-off-by: Eric Desrochers <eric.desrochers@canonical.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [openstack_ceilometer] Add backend_url and transport_urlJose Castillo2020-11-301-3/+2
| | | | | | | | | | | | | | to connection keys This patch adds the password and url in backend_url and transport_url to the list of connection keys, so it gets properly masked. Related: RHBZ#1892692 Resolves: #2298 Signed-off-by: Jose Castillo <jose.mfcastillo@gmail.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [sos] Use parse_args() instead of parse_known_args()Jake Hunsaker2020-11-301-1/+1
| | | | | | | | | | | | | | | | | When sos-4.0 work first went in, the way we extracted cmdline args for the parser was to call `parse_known_args()` in an effort to avoid passing unknown args to subcmds that don't support those specific args. As the parsers were built out however, this became unnecessary. The default behavior of `parse_args()` is smart enough to filter out args that are not supported by the specific component being run. With this being the case, we no longer need to manually check for unknown args being passed, and with that clear a LGTM warning. Resolves: #2299 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [openstack_barbican] Add new pluginWilliam Konitzer2020-11-301-0/+54
| | | | | | | Resolves: #2315 Signed-off-by: William Konitzer <wkonitzer@mirantis.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [plugins] Align sosreport plugins with Mirantis Cloud PlatformWilliam Konitzer2020-11-3010-18/+80
| | | | | | | | | | | installation: - Update Salt/SaltMaster plugins; - Update OpenStack services plugins; - Update Keepalived as IndependentPlugin class Signed-off-by: William Konitzer <wkonitzer@mirantis.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [systemd] Add tmpfiles.d conf filesAkshay Gaikwad2020-11-301-1/+4
| | | | | | | | | systemd-tmpfiles uses these configuration files to create volatile files and directories. These configurations needs to gathered by sosreport for analyzing what tmpfiles rule are included. Signed-off-by: Akshay Gaikwad <akgaikwad001@gmail.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [readme] Update badgingJake Hunsaker2020-11-231-2/+2
| | | | | | | Update the build badge for the repo README to point to Cirrus-CI instead of Travis now that we've switched CI platforms. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [networking] Remove superfluous if/out_ns and re-indent per-ns codeMauricio Faria de Oliveira2020-11-231-46/+45
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Split this into its own commit as it's mostly indentation changes. From: ``` if self.g_n_n(): out_ns = self.g_n_n(<filters>) for namespace in out_ns: ... if self.get_option('ethtool_namespaces'): for namespace in out_ns: ... ``` To: ``` for namespace in self.g_n_n(<filters>): ... if self.get_option('ethtool_namespaces'): for namespace in self.g_n_n(<filters>): ... ``` Resolves: #2308 Signed-off-by: Mauricio Faria de Oliveira <mfo@canonical.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [Plugin,networking] Move namespace filtering to get_network_namespaces()Mauricio Faria de Oliveira2020-11-232-27/+37
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Copy the logic over from the networking plugin to `filter_namespaces()`, which is generic, taking any list of namespaces; and then wrap it in `get_network_namespaces()` for the particular network namespace type. Now other plugins may be able to filter namespaces as well, and not only network namespaces, but other types that may be added (which can get convenience wrappers `get_X_namespaces()` then.) In order to make the review simpler, the changes for the networking plugin are intentionally small (keep the optional `if self.g_n_n():` and only filter in `out_ns` which is used in `ethtool_namespaces`.) The next patch addresses that, with the larger indentation changes. Testing with the networking plugins' options to filter namespaces by a regex pattern, and limit by the number: ``` $ ip netns ns3 ns2 ns1 ``` Option `networking.namespace_pattern` (only namespaces with trailing 2): ``` $ sudo ./bin/sos report -o networking \ -k networking.namespace_pattern='.*2' --batch $ find sosreport-*/sos_commands/networking \ | cut -d/ -f2- | grep ip_netns_exec sos_commands/networking/ip_netns_exec_ns2_iptables-save sos_commands/networking/ip_netns_exec_ns2_ip_-s_-s_neigh_show sos_commands/networking/ip_netns_exec_ns2_ip_address_show sos_commands/networking/ip_netns_exec_ns2_ip_route_show_table_all sos_commands/networking/ip_netns_exec_ns2_ethtool_-i_lo sos_commands/networking/ip_netns_exec_ns2_ethtool_-k_lo sos_commands/networking/ip_netns_exec_ns2_ethtool_-S_lo sos_commands/networking/ip_netns_exec_ns2_ethtool_lo sos_commands/networking/ip_netns_exec_ns2_ip6tables-save sos_commands/networking/ip_netns_exec_ns2_netstat_-W_-agn sos_commands/networking/ip_netns_exec_ns2_netstat_-W_-neopa sos_commands/networking/ip_netns_exec_ns2_ip_rule_list sos_commands/networking/ip_netns_exec_ns2_netstat_-s ``` Option `networking.namespaces` (only one namespace): ``` $ sudo ./bin/sos report -o networking \ -k networking.namespaces=1 --batch $ grep 'Limiting namespace iteration' sosreport-*/sos_logs/sos.log 2020-11-18 22:48:00,263 WARNING: [plugin:networking] \ Limiting namespace iteration to first 1 namespaces found $ find sosreport-*/sos_commands/networking \ | cut -d/ -f2- | grep ip_netns_exec sos_commands/networking/ip_netns_exec_ns3_ethtool_-i_lo sos_commands/networking/ip_netns_exec_ns3_ip_rule_list sos_commands/networking/ip_netns_exec_ns3_ip_route_show_table_all sos_commands/networking/ip_netns_exec_ns3_netstat_-W_-neopa sos_commands/networking/ip_netns_exec_ns3_netstat_-s sos_commands/networking/ip_netns_exec_ns3_ethtool_-k_lo sos_commands/networking/ip_netns_exec_ns3_netstat_-W_-agn sos_commands/networking/ip_netns_exec_ns3_iptables-save sos_commands/networking/ip_netns_exec_ns3_ip_address_show sos_commands/networking/ip_netns_exec_ns3_ip_-s_-s_neigh_show sos_commands/networking/ip_netns_exec_ns3_ip6tables-save sos_commands/networking/ip_netns_exec_ns3_ethtool_-S_lo sos_commands/networking/ip_netns_exec_ns3_ethtool_lo ``` Signed-off-by: Mauricio Faria de Oliveira <mfo@canonical.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [report,conntrack,ebpf,networking] Introduce get_network_namespaces()Mauricio Faria de Oliveira2020-11-235-41/+42
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Put the list of network namespaces in `commons['namespaces']['network']` as with eg, existing block devices in `commons['devices']['block']`. Then provide the shortcut function `get_network_namespaces()` for such. And convert the conntrack/ebpf/networking plugins to use it, which deduplicates their code and simplifies the plugins per-ns handling. ``` $ ip netns ns3 ns2 ns1 $ sudo python3 >>> from sos import SoS >>> from sos.report.plugins import Plugin >>> >>> sos = SoS(['report']) >>> plugin = Plugin(sos._component.get_commons()) >>> plugin.commons['namespaces']['network'] ['ns3', 'ns2', 'ns1'] >>> plugin.commons['devices']['block'] ['loop1', 'vdb', 'vda', ...] ``` The contents of the `sos_commands/{contrack,ebpf,networking}` directories are identical before/after this patch (no changes). Before and After: ``` $ sudo ./bin/sos report -o conntrack,ebpf,networking --batch ``` Compare: ``` $ find sosreport-<OLD>/sos_commands/ | cut -d/ -f2- | sort \ > find-sosreport.old $ find sosreport-<NEW>/sos_commands/ | cut -d/ -f2- | sort \ > find-sosreport.new $ diff find-sosreport.{old,new} $ $ md5sum find-sosreport.{old,new} 6fb6d8db55b4bc7197de3c65358c3576 find-sosreport.old 6fb6d8db55b4bc7197de3c65358c3576 find-sosreport.new ``` For reference, listing the commands run for `ns1`: ``` $ find sosreport-sos-g-2020-11-18-ryrcztt/sos_commands/ \ | cut -d/ -f2- | grep ns1 sos_commands/networking/ip_netns_exec_ns1_ip_rule_list sos_commands/networking/ip_netns_exec_ns1_netstat_-W_-neopa sos_commands/networking/ip_netns_exec_ns1_netstat_-s sos_commands/networking/ip_netns_exec_ns1_iptables-save sos_commands/networking/ip_netns_exec_ns1_ip_address_show sos_commands/networking/ip_netns_exec_ns1_ethtool_lo sos_commands/networking/ip_netns_exec_ns1_ip_route_show_table_all sos_commands/networking/ip_netns_exec_ns1_netstat_-W_-agn sos_commands/networking/ip_netns_exec_ns1_ip6tables-save sos_commands/networking/ip_netns_exec_ns1_ip_-s_-s_neigh_show sos_commands/networking/ip_netns_exec_ns1_ethtool_-i_lo sos_commands/networking/ip_netns_exec_ns1_ethtool_-k_lo sos_commands/networking/ip_netns_exec_ns1_ethtool_-S_lo sos_commands/conntrack/ip_netns_exec_ns1_conntrack_-S sos_commands/conntrack/ip_netns_exec_ns1_conntrack_-L_-o_extended sos_commands/ebpf/ip_netns_exec_ns1_bpftool_net_list ``` Signed-off-by: Mauricio Faria de Oliveira <mfo@canonical.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [conntrack] gather per-namespace dataMauricio Faria de Oliveira2020-11-231-0/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Run conntrack commands per-namespace, as the ebpf/networking plugins: Test-case: ``` # ip netns ns3 ns2 ns1 ``` Before: ``` # ./bin/sos report -o conntrack,ebpf,networking --batch # tar tf /tmp/sosreport-*.tar.xz | grep ip_netns_exec .../sos_commands/ebpf/ip_netns_exec_ns1_bpftool_net_list .../sos_commands/ebpf/ip_netns_exec_ns2_bpftool_net_list .../sos_commands/ebpf/ip_netns_exec_ns3_bpftool_net_list ... .../sos_commands/networking/ip_netns_exec_ns1_ip_address_show ... .../sos_commands/networking/ip_netns_exec_ns2_ip_address_show ... .../sos_commands/networking/ip_netns_exec_ns3_ip_address_show ... ``` After: ``` # ./bin/sos report -o conntrack,ebpf,networking --batch # tar tf /tmp/sosreport-*.tar.xz | grep ip_netns_exec .../sos_commands/conntrack/ip_netns_exec_ns1_conntrack_-L_-o_extended .../sos_commands/conntrack/ip_netns_exec_ns1_conntrack_-S .../sos_commands/conntrack/ip_netns_exec_ns2_conntrack_-L_-o_extended .../sos_commands/conntrack/ip_netns_exec_ns2_conntrack_-S .../sos_commands/conntrack/ip_netns_exec_ns3_conntrack_-L_-o_extended .../sos_commands/conntrack/ip_netns_exec_ns3_conntrack_-S .../sos_commands/ebpf/ip_netns_exec_ns1_bpftool_net_list .../sos_commands/ebpf/ip_netns_exec_ns2_bpftool_net_list .../sos_commands/ebpf/ip_netns_exec_ns3_bpftool_net_list ... .../sos_commands/networking/ip_netns_exec_ns1_ip_address_show ... .../sos_commands/networking/ip_netns_exec_ns2_ip_address_show ... .../sos_commands/networking/ip_netns_exec_ns3_ip_address_show ... ``` Signed-off-by: Mauricio Faria de Oliveira <mfo@canonical.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [networking] Ignore stderr from ip netns for list of namespacesMauricio Faria de Oliveira2020-11-231-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The sosreport of a system with this issue: ``` # ip netns Error: Peer netns reference is invalid. Error: Peer netns reference is invalid. test-ns ``` Shows that the networking plugin runs commands for the `Error:` lines: (note the ebpf plugin does not due to differences addressed in PR #2306) ``` # ./bin/sos report -o ebpf,networking --batch # tar tf /tmp/sosreport-*.tar.xz | grep ip_netns_exec .../sos_commands/ebpf/ip_netns_exec_test-ns_bpftool_net_list .../sos_commands/networking/ip_netns_exec_Error_ip6tables-save .../sos_commands/networking/ip_netns_exec_Error_ip6tables-save.1 ... .../sos_commands/networking/ip_netns_exec_test-ns_ip6tables-save ... ``` This happens because the networking plugin calls `collect_cmd_output()` with default `stderr=True` and does not handle such line type. For the purposes of getting the list of network namespaces, it is OK to ignore `stderr` since it does not provide that information. However, we do want it in the archive, so it is fully documented for analysis/debug. So change the call from `collect_cmd_output()` to `add_cmd_output()` to include both `stdout` and `stderr` in the archive, and call `exec_cmd()` that ignores `stderr` to get the list of network namespaces. Note that the plugin _currently_ does not need `exec_cmd(stderr=False)` to ignore `stderr`, as described in PR#2306, but will once/if that is applied. However, with the next patch, `stderr=False` won't be needed. Before: ``` # tar tf /tmp/sosreport-*.tar.xz | grep ip_netns_exec .../sos_commands/ebpf/ip_netns_exec_test-ns_bpftool_net_list .../sos_commands/networking/ip_netns_exec_Error_ip6tables-save .../sos_commands/networking/ip_netns_exec_Error_ip6tables-save.1 ... .../sos_commands/networking/ip_netns_exec_test-ns_ip6tables-save ... ``` After: ``` # tar tf /tmp/sosreport-*.tar.xz | grep ip_netns_exec .../sos_commands/ebpf/ip_netns_exec_test-ns_bpftool_net_list .../sos_commands/networking/ip_netns_exec_test-ns_ip6tables-save .../sos_commands/networking/ip_netns_exec_test-ns_ip_-s_-s_neigh_show ... ``` And the `ip netns` contents with `stderr` still remain in the archive: ``` # tar xf /tmp/sosreport-*.tar.xz --to-stdout \ --wildcards '*/sos_commands/networking/ip_netns' Error: Peer netns reference is invalid. Error: Peer netns reference is invalid. test-ns ``` Test suite: ``` # ./tests/simple.sh ... Everything worked! ``` Signed-off-by: Mauricio Faria de Oliveira <mfo@canonical.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [grub2] Stylistic cleanup for grub2-mkconfig callsJake Hunsaker2020-11-231-4/+2
| | | | | | | | | | Small stylistic cleanups for the `grub2-mkconfig` environment variable settings and command formatting. Closes: #1070 Resolves: #2317 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [Plugin] Add method to get all containers matching a regexJake Hunsaker2020-11-231-0/+18
| | | | | | | | | | | | Adds a new method, `Plugin.get_all_containers_by_regex()`, that will return all known containers on the system with a name matching a provided regex string. Optionally, this method may also return terminated containers in addition to those that are active and running. Closes: #2176 Resolves: #2319 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [redhat] Add podman to valid container env varsJake Hunsaker2020-11-231-1/+1
| | | | | | | | | | | In newer versions of podman, the `container` env var set inside the container will be set to `podman` instead of `oci`. Add this value to the list of valid values to use to determine if sos is running inside a container or not, for when we want to capture from the host system. Resolves: #2320 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* add lang files to sos.specPavel Moravec2020-11-231-1/+1
| | | | | | | | | | Fix packit/copr build failures by adding lang files to sos.spec Closes: #2318 Resolves: #2321 Signed-off-by: Pavel Moravec <pmoravec@redhat.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [flake8] Fix minor indentation issueJake Hunsaker2020-11-191-1/+1
| | | | | | Resolve a minor indentation issue. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [cleaner] Simplify MAC addr parsing from #2311Jake Hunsaker2020-11-191-6/+13
| | | | | | | | | | | | | | | | | The previously merged #2311 was thought to be sufficient for remaining cases where MAC addrs were followed by punctuation characters, however it was found to be too restrictive in that it would leave quoted and other forms of MAC addrs unobfuscated when those addrs were immediately followed by other characters. Simply the regex match to catch all of these, and further update the `parse_lines()` override to properly trim down the match to just the address substring. Related: #2311 Related: #2305 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [travis] Drop testing via TravisJake Hunsaker2020-11-191-65/+0
| | | | | | | | | | | We are dropping the use of Travis for our CI testing, and moving to Cirrus-CI. Resolves: #2305 Closes: #2246 Closes: #2048 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [tests] Update simple.sh test suiteJake Hunsaker2020-11-191-30/+171
| | | | | | | | | | | | | This patch overhauls tests/simple.sh to provide more granular details on why a test run may have failed, beyond checking for a non-zero exit code or if output was written to stderr. This should also serve as another step towards more easily extendible tests for our automated processes. Resolves: #2305 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [Cirrus] Add testing via Cirrus CIJake Hunsaker2020-11-191-0/+101
| | | | | | | | | | | | | | | | | | | | | | | Adds functional CI testing via Cirrus in conjunction with GCE. Tests can now be run on Fedora as well as Ubuntu hosts, and minor tasks can be run in containers. This will allow us to easily expand our testing base across more distributions provided those distributions can be run on GCP instances. As new releases of supported distributions are made available, the maintainers will need to build and push updated Fedora (or more generally, RH-family) images to the GCP project. Ubuntu has cloud images on GCE already, so when new releases of Ubuntu are pushed we will simply need to update .cirrus.yml to point to the new public images. Note that at the moment testing on RHEL is not enabled, though it should follow the same framework as the Fedora tests and should hopefully be coming before too long. Closes: #1885 Resolves: #2305 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [component] Add log verbosity from presetsErik Bernoth2020-11-191-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | Closes: #2289 The main problem this tries to solve was that preset verbosity was ignored in logging. With a simple test this could be reproduced: sudo sh -c "source /path/to/repo/sosreport/venv/bin/activate; \ cd /tmp/foo; sos report --preset sostestpreset; cd -" The bug is that without a change of code there are no messages from the plugin `host` (no lines of output start wiht "[plugin:host]"). The problem is that the logging is set in the inherited __init__() method from Component, but the presets are only handled afterwards in the Report's __init__(). Since it is good to have logging configured from the beginning, the only option is to reconfigure it after the preset config is known. The simplest method is to reinitialize the logging, although maybe not the most efficient. Signed-off-by: Erik Bernoth <ebernoth@redhat.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [cleaner] Match MAC addrs in lines that end in punctuationJake Hunsaker2020-11-181-1/+22
| | | | | | | | | | | | | | | It was found that sometimes MAC addresses would not be obfuscated from certain files when the MAC address was followed by punctuation, such as periods. This was because the regex used to match IPv4 MAC addresses needs to avoid matching a substring within IPv6 mac addresses. Update the regex to compensate, and then override the `parse_line()` method to pull the matched punctuation out of the matched MAC substring so that obfuscation remains consistent. Resolves: #2311 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [auditd] Collect all fanotify message by defaultThorsten Scherf2020-11-181-1/+1
| | | | | | | | | The auditd plugin should collect all fanotify messages by default. Resolves: #2313 Signed-off-by: Thorsten Scherf <tscherf@redhat.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [postgresql] reorganize postgres from SCLPavel Moravec2020-11-171-30/+24
| | | | | | | | | | | - enable RedHatPostgreSQL also by rh-postgresql12 SCL - collect configs and logs regardless of running service - collect "du -sh" for SCL directories as well Resolves: #2309 Signed-off-by: Pavel Moravec <pmoravec@redhat.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [Plugin] exec_cmd(): pass stderr along to sos_get_command_output()Mauricio Faria de Oliveira2020-11-171-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The sosreport of a system with this issue: ``` # ip netns Error: Peer netns reference is invalid. Error: Peer netns reference is invalid. test-ns ``` Shows this difference between the plugins ebpf and networking (callers of `ip netns`): ``` # ./bin/sos report -o ebpf,networking --batch # tar tf /tmp/sosreport-*.tar.xz | grep ip_netns_exec .../sos_commands/ebpf/ip_netns_exec_test-ns_bpftool_net_list .../sos_commands/networking/ip_netns_exec_Error_ip6tables-save .../sos_commands/networking/ip_netns_exec_Error_ip6tables-save.1 ... .../sos_commands/networking/ip_netns_exec_test-ns_ip6tables-save ... ``` Only the networking plugin called `ip netns exec <ns> <cmd>` on `Error:` lines. Hmm. ... The networking plugin calls `collect_cmd_output()`, which has parameter stderr=True` as default: ``` def collect_cmd_output(self, cmd, suggest_filename=None, ... stderr=True, ... ``` The ebpf plugin calls `exec_cmd()`, BUT it has parameter `stderr=True` as default AS WELL: ``` def exec_cmd(self, cmd, timeout=cmd_timeout, stderr=True, ... ``` So, why the difference? ... Well, it turns out `exec_cmd()` does NOT pass the `stderr` parameter along to `sos_get_command_output()`: [but `collect_cmd_output()` does.] ``` return sos_get_command_output(cmd, timeout=timeout, chroot=root, chdir=runat, binary=binary, env=env, foreground=foreground) ``` And `sos_get_command_output()` has `stderr=False` as default: ``` def sos_get_command_output(command, timeout=300, stderr=False, ... ``` Thus, despite `exec_cmd()` has `stderr=True` as default it is _ignored_, as its callee `sos_get_command_output()` has `stderr=False` as default. This explains the output difference between the ebpf/networking plugins. ... Looking back for details, the origin of the change is in PR#1807 [1], with commit e8bb94c (refactor command functions/introduce `exec_cmd()`) and commit e51d3e6 (update caller plugins). The PR has no mentions/discussion around `stderr` in comments/review, so it's apparently an oversight during the refactor, I guess. Because, previously the refactored callers of `sos_get_command_output()` (`get_command_output()`, `call_ext_prog()`, `[_]get_cmd_output_now()`) all had `stderr=True` as parameter/default, passing it along to callee. (The only exception is in `_collect_cmd_output()` which does the first call in the chroot with `stderr`, and falls back to non-chroot without `stderr`, but it was already that way. Thus not changing this case.) ... So, it seems right and safe and opportune to fix this inconsistency between default `stderr` in `exec_cmd()` / `sos_get_command_output()` by just passing it along between them: Right: because it restores the previous behavior (before refactor) assumed/accepted by callers that did not specify `stderr=False`. Safe: because it restores the actual behavior to previous behavior, and if plugins handled `stderr` lines previously, they should also handle it now, for the most part (i.e., not any new `stderr` lines, but they are currently ignoring those, and should get them covered.) Opportune: because if any plugin breaks _now_ because of this, it is a new/post-refactor change that decided to use `exec_cmd()` with the default that is inconsistent with actual behavior, and it seems okay to take bugs for that, so to identify and fix such cases. Hopefully those points are reasonable with the project's philosophy. ... Test-case: Setup: ``` $ sudo python3 from sos import SoS from sos.report.plugins import Plugin sos = SoS(['report']) plugin = Plugin(sos._component.get_commons()) ``` Before: ``` >>> print(plugin.exec_cmd('ip netns')['output']) test-ns >>> print(plugin.exec_cmd('ip netns', stderr=True)['output']) test-ns >>> print(plugin.exec_cmd('ip netns', stderr=False)['output']) test-ns ``` After: ``` >>> print(plugin.exec_cmd('ip netns')['output']) Error: Peer netns reference is invalid. Error: Peer netns reference is invalid. test-ns >>> print(plugin.exec_cmd('ip netns', stderr=True)['output']) Error: Peer netns reference is invalid. Error: Peer netns reference is invalid. test-ns >>> print(plugin.exec_cmd('ip netns', stderr=False)['output']) test-ns ``` ... Test-suite: ``` # ./tests/simple.sh <...> Summary failures false time 2 -l failures false time 3 --list-presets failures false time 3 --list-profiles failures false time 17 --batch --build --no-env-vars failures false time 3 Size 14.23KiB --batch --no-report -o hardware failures false time 20 Size 2.99MiB --batch --label TEST -a -c never failures false time 20 Size 2.81MiB --batch --debug --log-size 0 -c always failures false time 20 Size 2.84MiB --batch -z xz --log-size 1 failures false time 19 Size 4.01MiB --batch -z gzip failures false time 32 Size 2.76MiB --batch -t 1 -n hardware failures false time 19 --batch --quiet -e opencl -k kernel.with-timer failures false time 22 Size 4.31MiB --batch --case-id 10101 --all-logs --since=20201110 failures false time 21 Size 3.06MiB --batch --verbose --no-postproc failures false time 39 Size 2.82MiB --batch --mask Everything worked! ``` [1] https://github.com/sosreport/sos/pull/1807 Resolves: #2306 Signed-off-by: Mauricio Faria de Oliveira <mfo@canonical.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [openstack_tripleo] Add new path for log collectionKevin Carter2020-11-171-7/+6
| | | | | | | | | | | | | This change essentially adds /var/lib/tripleo to the collection path. All of the collection targets have been moved into a class object which is easily shared across methods. This will alow us to easily collect more in the future and ensure all of our collected objects are sanitized. Resolves: #2300 Signed-off-by: Kevin Carter <kecarter@redhat.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [sos] Fix argparse error output formattingJake Hunsaker2020-11-101-1/+2
| | | | | | | | | | | | | | | | | | | Error message propagation from the subparsers was causing the error message to be mangled with the usage string from the "main" parser when an invalid argument value was passed. After discussion with python upstream, while this is an issue with the subparser formatting (as %(prog) is being set to the main parser's usage), there are simple paths around this behavior so a code change there will not be forthcoming. Instead, override the subparser's prog to be a single line string instead of the main parser's usage string. Closes: #2285 Resolves: #2301 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [ds] collect listing of DB filesPavel Moravec2020-11-101-0/+2
| | | | | | | | | The listing helps to troubleshoot possible DB corruption issues. Resolves: #2303 Signed-off-by: Pavel Moravec <pmoravec@redhat.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [ovirt] collect /etc/pki/ovirt-engine/.truststorePavel Moravec2020-11-101-2/+5
| | | | | | | | | | | .truststore contains useful public CAs but a_c_s skips collecting that hidden file. Closes: #2296 Resolves: #2297 Signed-off-by: Pavel Moravec <pmoravec@redhat.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [component] execute() as Abstract classErik Bernoth2020-11-101-0/+3
| | | | | | | | | | | | | | Resolves: #2295 `execute()` is a method that should be implemented by subclasses of Component. And the python documentation recommends to also add the method to the parent class and raise a specific error to indicate that the parent class should not implement it. See: https://docs.python.org/3/library/exceptions.html#NotImplementedError Signed-off-by: Erik Bernoth <ebernoth@redhat.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [report] remove redundant test of compression methodsPavel Moravec2020-11-041-9/+0
| | | | | | | | | | | Argparse processing of compression method already contains this inclusion test, let delete it from here. Closes: #2286 Resolves: #2293 Signed-off-by: Pavel Moravec <pmoravec@redhat.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [rhui] remove obsolete pluginPavel Moravec2020-11-041-41/+0
| | | | | | | | | | This is python2 only specific plugin, remove it for now. Closes: #2287 Resolves: #2292 Signed-off-by: Pavel Moravec <pmoravec@redhat.com> Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [fibrechannel] Add remote_ports to files tupleDaria Bukharina2020-11-041-1/+1
| | | | | | | | | | | Adds the `fc_remote_paths` path to the files tuple for plugin enablement. This also indirectly fixes a bug where the plugin would also be enabled due to a malformed 1-tuple. Resolves: #2284 Signed-off-by: Daria Bukharina d.bukharina@yadro.com Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
generated by cgit (git 2.34.1) at 2024-11-15 11:25:01 +0000