aboutsummaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* [sosnode] Use `parse_version()` to replace `LooseVersion()`Jake Hunsaker2023-01-162-4/+5
| | | | | | | | | | Switch to using `parse_version()` from setuptools instead of `LooseVersion()` from distutils, since distutils is being removed in python 3.12. Related: #3093 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [build] Convert to setuptoolsJake Hunsaker2023-01-164-90/+13
| | | | | | | | | | | | | | | | | | | | In python 3.12 distutils will be removed. As such, we need to update to the replacement `setuptools`. This commit makes the basic change over in `setup.py`, so that an `sdist` source tarball can be generated. Note that while this source tarball will still have the `.po` files in it any build tarball (`bdist`) produced via the new `setup.py` will *not* have `.mo` translation files compiled and included at this point. In reviewing this change, it was found that our internationalization is currently broken and very out of date. Future work will focus on fixing that situation, but for now the immediate packaging needs are being addressed. Resolves: #3093 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [utilities] Enhance timeout mechanismJunius Gao2023-01-162-1/+3
| | | | | | | Enhance on timeout mechanism. Force the parent process to exit when it has a deadlock child. Signed-off-by: Junius Gao <Junius.Gao@veritas.com>
* [testing] Change location of mocked files for testsJake Hunsaker2023-01-1229-22/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | Previously, mocked files were kept under the `tests/test_data/` directory and generally mimic'd the file location they would be temporarily copied to during the execution of their relevant tests. This has a few maintainability drawbacks, and the handling of the `files` attribute for test cases as either strings or tuples is at best confusing. Improve on this by instead making the `files` references relative to where the test case file is written. This enables easier maintenance by keeping all test requirements closer together, rather than spread across the repo. As such, the `files` attribute now requires a list of tuples, taking the form `(relative_src, absolute_dest)`. Additionally, fake plugins for tests that need them to artificially test a specific criteria should also be included in the test's subdir now. Along with this change, move several StageTwo tests to their own subdirs that now contain both the test cases and the needed files for mocking. This should be the new design pattern going forward - if a test needs to mock files of any kind, put it in a new subdirectory (and if it doesn't need to mock files, continue to keep it in the relevant directory within the test suite). Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [ceph_mon] collect mon status on octopusNikhil Kshirsagar2023-01-111-1/+1
| | | | Signed-off-by: Nikhil Kshirsagar <nikhil.kshirsagar@canonical.com>
* Replace deprecated Logger.warn by Logger.warningPavel Moravec2023-01-118-14/+15
| | | | | | Resolves: #3109 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [cleaner] Convert print()s to leveraging ui loggingJake Hunsaker2023-01-111-10/+14
| | | | | | | | | | | | It was recently found that some condition will cause Avocado to not capture trailing `print()` statements in our test suite, and it is reasonable to assume other automation may also have similar edge cases. Resolve this by switching potentially problematic `print()`s to use the ui logging stream, which will still print to console even after the file handler has been closed. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [policies] Add policy for OpenCloudOSzoedong2023-01-091-0/+42
| | | | Signed-off-by: zoedong <zoedong@tencent.com>
* [policies] Directly use a Transport for remote commandsJake Hunsaker2023-01-093-19/+55
| | | | | | | | | | | | | | | | | | | Previously, remote command executions handled by policies were done by moodifying the command string based on the `remote_exec` property of the given `SoSTransport` in use for the node that the policy was loaded for. While this worked well for SSH connections, newer transports may need to do some manipulation of returned data in order for the rest of `sos collect` to function as intended. As such, switch to directly using a transport's `run_command()` method, which will ideally handle any needed manipulations of either how the command is execute and/or how the returned data is presented to the calling component. Related: #3087 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [Plugin] Fix docstring in add_copy_specSamuel Walladge2023-01-091-2/+2
| | | | | | | The string or list strings in the param `copyspecs` are processed as globs, not regular expressions. Signed-off-by: Samuel Walladge <samuel.walladge@canonical.com>
* fix typolilinjie2023-01-091-1/+1
| | | | Signed-off-by: lilinjie <lilinjie@uniontech.com>
* [collector] Prevent appending local host in strict_node_list modePavel Moravec2023-01-051-6/+8
| | | | | | | | | | | | The changes in respecting strict_node_list are three-fold: 1) Don't add local hostname among "list of nodes to collect from:" 2) Skip explicit adding of the primary node to client_list 3) Apply strict_node_list to reduce_node_list (as it can purge away hostname or IP address of the local host, otherwise) Resolves: #3096 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [ceph_mgr] Update plugin for newer versions of CephJake Hunsaker2023-01-031-45/+73
| | | | | | | Updates the plugin to account for newer versions of Ceph, similar to the previous few commits focusing on the ceph plugins. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [ceph_osd] Update plugin for newer versions of cephJake Hunsaker2023-01-031-41/+70
| | | | | | | | | | | | Newer versions of Ceph, such as for RHCS 5, make some changes to the container names as well as how to leverage `ceph daemon` commands. Update the plugin to reflect this, and use the available admin sockets on the host to capture that output, rather than always running it inside the osd container. Further, account for the use of an fsid in directory paths for newer ceph. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [ceph_mon] Update plugin for newer ceph versionsJake Hunsaker2023-01-031-58/+141
| | | | | | | | | | | | | | | | Newer versions of ceph do not have running processes directly on the node. As such, the current method of determining the ID for the `ceph tell` commands is not functional. Fix this by using the output of `ceph status` on RHCS 5 nodes to get the node IDs. Further, while ceph containers can be used to enable this plugin, those containers by default cannot run various `ceph` commands collected by the plugin. However, those commands are functional directly on the host, so no longer attempt to execute within the containers. Finally, update `postproc()` for newer versions as well. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [ceph_*] Correct enablement trigger for ceph pluginsJake Hunsaker2023-01-036-27/+11
| | | | | | | | | | | | | | | | | The previous changes that broke up the ceph plugin into smaller component specific plugins inadvertently broke the plugin enablement by overriding the `check_enabled()` function to check for file presence under a directory. This in turn replaced the standard checks, such as the presence of certain containers. Fix this by removing the method override, and leveraging the `files` tuple against the component-specific directory into which the globs were trying to check. Further, update the container name regexes for enablement as they have changed slightly since the initial plugin creation. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [ceph] Add new collect clust profile for RHCS5Jake Hunsaker2023-01-021-0/+67
| | | | | | | | | | Adds a new cluster profile for RHCS 5 for `sos collect`. This profile depends upon the use of `cephadm` which is used to both deploy and manage the cluster. Users may optionally restrict the list of nodes to collect from by using the `-c ceph.labels` option to specify a set of label(s) to filter node results with. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [clusters] Allow cluster profiles to specify sos options directlyJake Hunsaker2023-01-023-14/+37
| | | | | | | | | | | | | | | Clusters can already pass plugin options to nodes, and can also set arbitrary options on individual nodes or primary nodes. So, rather than requiring a cluster profile specify sos options to both nodes and primaries, instead allow profiles to specify options via a simple `sos_options` dict that will get applied to every node automatically, as is the case with plugin options. Note that user values for these options will override cluster values. For example, if a cluster spceifies a `--log-size` value, and the user does on the command line, then the user's value will have precedence. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [policies] Add policy for Circle LinuxBella Zhang2023-01-021-0/+49
| | | | | | Circle Linux is an production-ready distribution from RHEL. Signed-off-by: Bella Zhang <bella@cclinux.org>
* [devel] Ignore entire .idea/ directory from pycharmJake Hunsaker2022-12-161-10/+1
| | | | | | | Ignore the entirety of the `.idea/` directory within the repo for anyone using PyCharm as an IDE. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [packit] Update packit configuration for Copr srpm buildsJake Hunsaker2022-12-151-12/+19
| | | | | | | | | | | Packit is moving srpm builds from sandbox to copr, which means we need to specify the build dependencies for packit to know how to create the build environment. As part of this change, re-organize the config file to be easier to read, and update some of the deprecated keys or nesting per docs. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [plugins] Count in estimate mode real disk usagePavel Moravec2022-12-121-6/+11
| | | | | | | | | | | | | | Currently, --estimate-only mode summarizes sizes of all files as returned by stat. That corresponds to "du --apparent-size" calculation. Meanwhile, the user is rather concerned about the real disk usage reported by "du" output. Let update the estimate mode from "du --apparent-size" style to "du". Resolves: #3084 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [openstack_mistral] Collect mistral content itselfCédric Jeanneret2022-12-072-2/+48
| | | | | | | | | | | | | | | | | | | | | | Logs as well as deploy information are located in /var/lib/mistral as well. Until now, this location was overlooked, and important information about the deploy state, configuration and potential errors were missing from the initial SOS-Report, leading to time loss for both Customer and Support. This patch intends to correct this situation, by ensuring the content is taken from the Undercloud. Notes: - this is especially important for OSP<17.0 - the location may content multiple subdirectories - the location will contain the history of the different actions done by the operator - mistral logs are moved from the openstack_instack plugin to the new openstack_mistral for the sake of consistency. The "instack" name is deprecated in OSP. Signed-off-by: Cédric Jeanneret <cjeanner@redhat.com>
* [nfs] add nfsd information collectionThiago Rafael Becker2022-12-051-0/+2
| | | | | | | | Collect nfsd information: - The contents of /proc/fs/nfsd - The output of nfsdclnts Signed-off-by: Thiago Rafael Becker <thiago.becker@gmail.com>
* [collect] Remove --sos-cmd optionJake Hunsaker2022-12-023-29/+0
| | | | | | | | | | | Removes the `--sos-cmd` option for `sos collect`. Allowing passthru options in this manner is inherently flawed, and any attempts at sanitizing potentially malicious/dangerous values will always be a losing battle. Instead, `sos collect` should leverage available `report` options that are vetted and handled via the existing per-node capabilities checks that is well-defined for explicit passthru options. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [cleaner,ipv6] Add support for IPv6 obfuscationJake Hunsaker2022-11-306-6/+426
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit adds a new parser and accompanying map for obfuscating IPv6 addresses. This new parser will attempt to capture valid IPv6 networks and addresses, and produce a mostly-randomized obfuscated pair. Due to the multiple formats an IPv6 address can take, some identifiers are necessary to preserve relevant information while still obfuscating actual addresses and networks. For example, global unicast addresses that have more than one defined hextet (greater than /16 prefix) will always generate an obfuscated address starting with `534f` (or 'so', continuing the style of our mac address handling that uses 'sos' as an identifier). Addresses with a /16 prefix or less, will start with simply '53'. Private addresses, which start with `fd` will generate an obfuscated address starting with `fd53`, so that the contextual understanding that it is a private network/address can remain. Link-local addresses which start with `fe80::` will remain that way, only having the device hextets obfuscated - again, keeping the contextual information that it is a link-local interface intact, as otherwise these obfuscations may confuse end users reviewing an sos report for problems. Note that the address `::1` and `::/0` are explicitly skipped and never obfuscated, for the same reasons given above. Additionally, this parser/map will write data to the default map (and any per-run private maps) differently than previous parsers. Rather than simply dumping the obfuscation pairs into the map, it is broken up via network, with hosts belonging to that network nested inside those network entries (still being json-formatted). Users will also note that the ipv6 entries in the map also have a `version` key, which is intended to be used for handling future updates to the parser/map when upgrading from an older sos version to a newer one. This may or may not be carried over to future updates to other parsers. Closes: #3008 Related: RHBZ#2134906 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [postfix] Exclude password filesIwao Miyake2022-11-301-0/+39
| | | | | | | | | | | | Changed to sosreport exclude files set by following options. - `lmtp_sasl_password_maps` - `smtp_sasl_password_maps` - `postscreen_dnsbl_reply_map` - `smtp_sasl_auth_cache_name` Resolvs : #3073 Signed-off-by: Iwao Miyake <miyake.iwao@fujitsu.com>
* [virsh] Collect the node device infoHan Han2022-11-291-0/+7
| | | | | | | | | Implement the virsh sub-command `nodedev-list --tree` and `nodedev-dumpxml` to virsh plugins. Resolves: #3079 Signed-off-by: Han Han <hhan@redhat.com>
* [autofs] Scrub autofs_ldap_auth secretsStepan Broz2022-11-281-0/+19
| | | | | | | | | | | | | | | | | | | | | | | The XML configuration file /etc/autofs_ldap_auth.conf may contain an authentication secret in the <autofs_ldap_sasl_conf/> tag. This patch makes sure the secret or encoded_secret gets scrubbed. Example of scrubbing of the secret: secret="abc" or encoded_secret = 'abc' to: secret="********" or encoded_secret = '********' Resolves: #3068 Signed-off-by: Stepan Broz <sbroz@redhat.com>
* [virsh]: Collect more info from hostHan Han2022-11-281-2/+6
| | | | | | | | | | Implement sub-command to collect the info from host, including the free memory of all numa nodes(freecell --all), the storage pool capabilities(pool-capabilities), the number of CPUs and the online CPUs(nodecpumap), the max number of vcpus supported by kvm(maxvcpus kvm), and the hypervisor sysinfo(sysinfo) Signed-off-by: Han Han <hhan@redhat.com>
* [dnf] Scrub passwords from dnf.confStepan Broz2022-11-211-0/+19
| | | | | | | | | | | | | | | | | | | Currently the dnf plugin scrubs passwords from the repository files and DNF variables, however "password" and "proxy_password" can be defined in "/etc/dnf/dnf.conf". This patch ensures that passwords are scrubbed from dnf.conf too. Example of scrubbing: Before: proxy_password = hackme After: proxy_password = ******** Resolves: #3072 Signed-off-by: Stepan Broz <sbroz@redhat.com>
* [networking] Nest namespace iteration inside namespace conditionalJake Hunsaker2022-11-211-43/+44
| | | | | | | | | | | | A CodeQL error alert was highlighted that the iptables predicates might be unset during execution of the plugin, which in turn could potentially cause the plugin to throw an exception. Fix this by nesting the interation of command collections over namespaces that might use this predicate in the same conditional that defines the predicate. Related: #3066 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [collector] Handle potential edge case exception in node list enumerationJake Hunsaker2022-11-211-3/+4
| | | | | | | | | Adds handling to catch the condition where we get something other than a string or list from `get_nodes()` to `format_node_list()`. Related: #3066 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [report] Merge redundant conditional in calling `display_results()`Jake Hunsaker2022-11-211-3/+2
| | | | | | | | | | | | | During our final processing to report the results of an archive to the user at the end of an execution, there was a redundant conditional checking if we were running with `--build` to decide how to call `display_results()` (with or without stat information). Merge this into the previous conditional that determines that stat information, which in turn resolves a CodeQL error alert. Related: #3066 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [openstack_instack] Refine regex matching for stack idsJake Hunsaker2022-11-211-5/+6
| | | | | | | | | | CodeQL highlighted on a potential issue on the regex used to extract stack IDs for collection iterations. Resolve this by refining the regex pattern to better match the stack IDs in the output. Related: #3066 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [collect] Restrict permissions on host group filesJake Hunsaker2022-11-211-1/+1
| | | | | | | | | Tighten the permissions on the host group file(s) written by `sos collect` to no longer be world readable. Related: #3066 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [utilities] If writing to disk, close file during exception handlingJake Hunsaker2022-11-211-0/+2
| | | | | | | | | | | | As identified by CodeQL, there was a case where a potential open file we are/were writing to as part of a command collection would not be closed if we hit an exception during that execution. Add explicit closure as part of the exception handling to resolve this. Related: #3066 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [testing] Switch to CodeQL from LGTMJake Hunsaker2022-11-213-1/+47
| | | | | | | | | | | LGTM is shutting down shortly, as it has been folded into GH CodeQL. As such, we need to switch over to using CodeQL to maintain automatic code analysis. Do this by using the provided CodeQL workflow configuration, and add a config file to limit the analysis to the appropriate `sos/` directory. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [report] Prevent obfuscating tmpDir path also in --build modePavel Moravec2022-11-211-2/+2
| | | | | | | | | | Likewise #3065, we should not obfuscate path to directory when --build option is used. Resolves: #3071 Relates: #3065 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [report] Prevent obfuscating tmpDir path before tarbal movePavel Moravec2022-11-141-4/+4
| | | | | | | | | When moving sos tarball from a private directory to /var/tmp, apply filename obfuscation just to the file and not the tmpDir path itself. Resolves: #3065 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [build] Require the sos aliasArif Ali2022-11-081-1/+2
| | | | | | | | When installing via --danegerous, the aliases are not done automatically, so create he sos alias, so that we can run via "sos report" Signed-off-by: Arif Ali <arif.ali@canonical.com>
* [build] use full path for snap checkArif Ali2022-11-071-1/+1
| | | | | | | | The environment for the build doesn't seem to be refreshing between runs, and hence sos from the snab binary location is not working. Having the full path ensures that this doesn't fail Signed-off-by: Arif Ali <arif.ali@canonical.com>
* [build] add concurrency to workflowArif Ali2022-11-071-1/+4
| | | | | | | | | | Add concutrrency to the job, so that if there is one already running, then that would be cancelled Remove the deb package, and that takes precedance with the path do the test doesn't quite work Signed-off-by: Arif Ali <arif.ali@canonical.com>
* [pulpcore] obfuscate PASSWORD in non-greedy mannerPavel Moravec2022-11-041-2/+4
| | | | | | | | | | | | Since PASSWORD can be in a one-liner list, we must mark the password value in a non-greedy manner until first ',' or '}' is found. This works well also for multi-line lists where any line terminates by a comma. Resolves: #3058 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [build] Update snap push workflowArif Ali2022-11-041-1/+3
| | | | | | | | * snapcraft-channel was the old method, updating to release * fetch all history, so that the version of the snap will be based on the latest tag plus the number of commits since Signed-off-by: Arif Ali <arif.ali@canonical.com>
* [build] Update snap install testJake Hunsaker2022-11-031-1/+1
| | | | | | | Adds `--classic` to the snap installation test as part of the new GH Action to push a new snap on `main` updates. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [build] Build snap and push to latest/edgeArif Ali2022-11-031-0/+26
| | | | | | | When a any MRs or commits are added to the main branch then automatically create a snap, and push to latest/edge channel Signed-off-by: Arif Ali <arif.ali@canonical.com>
* [testing] Revert testfarm integrationJake Hunsaker2022-11-024-27/+0
| | | | | | | | The RH team has decided to no longer pursue pushing the downstream beakerlib test suite into upstream to be run via testfarm. Instead, those tests will be ported to the avocado test suite over time. Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
* [ovn_central] Add commands to get OVN DBs statsFernando Royo2022-10-311-9/+14
| | | | | | | | | | | | | | | | In order to automate the parsing of the plugin output files, by systems could only read file contentit, seems useful to add an output cmd file where data, such as the size or permissions, of the OVN DBs is directly available. This patch modifies the current behaviour to copy the db files of each OVN DB to create a file containing the ls -lan output of the files. The list of possible locations of these *.db files is maintained for backwards compatibility but it has been necessary to iterate over them to only obtain the information file of those that really exist. Signed-off-by: Fernando Royo <froyo@redhat.com>
* [ovn_central] Query ovn_controller SBDB connection with ovn-appctlFernando Royo2022-10-311-1/+30
| | | | | | | | | | | | This additional command will tell us the ovn_controller connection status to OVN SBDB. This is done thanks to ovn-appctl command using -t option to specify the target over the ovn_controller socket, a function has been created in order to located the ovn_controller socket path that is name is different from each controller node. Signed-off-by: Fernando Royo <froyo@redhat.com>
generated by cgit (git 2.34.1) at 2024-09-20 02:29:28 +0000