| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
| |
Adds a hostname parser to `sos clean` that will attempt to obfuscate
FQDNs matching the hostname of the system that generated the sosreport,
as found in sos_commands/host/hostname.
Additionally, any domains added via the `--domains` option will also be
obfuscated, including any subdomains of the domain(s) specified by the
option.
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
Adds a manpage for `sos clean`, with a link for `sos mask`.
Updates `man sos` as well to include the options moved into the global
group as part of the SoSCleaner patchset.
Related: #1987
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
| |
Adds tests for the mappings and parsers used by SoSCleaner
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Adds functionality to SoSCollector runs to pass collected archives
through `--clean`, much the same as the functionality has been extended
to base `report` runs.
Note that running this way will, like report, only result in a single
obfuscated archive and the private mapping file. If users desire an
unobfuscated copy as well, then `sos collect` and `sos clean` should be
run separately.
Related: #1987
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Adds a new option, `--clean` or `--mask`, to hook a report into
`SoSCleaner`.
This is done after collection is complete, and before the archive is
built and compressed. Data is substituted in place for the report, which
means there will not be an obfuscated copy on disk. There will however
still be a mapping file produced and located in the same directory as
the final archive.
If it desired to have both an obfuscated and unobfuscated copy, it is
recommended to run `sos report` followed by a separate `sos clean`.
Related: #1987
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Adds a new component/subcommand `SoSCleaner`, accessible via `sos clean`
or `sos mask`.
This component is intended to bring similar functionality that is
available in the standalone `soscleaner` utility directly into the sos
project. It is designed to take either an untarr'ed sos directory, an sos
archive, or an archive of sos archives and obfuscate sensitive network
information (and optionally other data) from the report(s) that are
typically not able to be sanely scrubbed via the `postproc()` method of
plugins.
As of this first commit, users may execute `sos clean|mask $archive`
directly from the command line and expect to have an obfuscated archive
generated. Note that this obfuscated archive does NOT replace the
original archive on disk. Currently there is support for IPv4 IP
addresses, MAC addresses, and 64-bit IPv6 MAC addresses.
Future commits will aim to provide hooks for this functionality into
both `report` and `collect`. Additionally, more parsers will be added to
handle more types of data that needs consistent obfuscation.
Closes: #1987
Closes: #311
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
With the volume of information collected by the redesigned openshift
plugin, no longer restrict subdir creation with plugin directories to a
single level. Allow arbitrary depth to allow more efficient data
organization.
Resolves: #2052
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch updates the current `openshift` plugin from 2.x (now EOL) to
4.x (3.x is handled via the `origin` plugin). This has been done at the
request of the Red Hat OCP support team.
While OCP 4.x is still built on top of kubernetes, the collection
requirements differ far to much to be able to simply extend the
kubernetes plugin. Most notably, the OCP4 plugin is designed to collect
a significantly greater volume of data by default due to the complexity
of even a basic installation of OCP4 clusters. Additionally, there are
API login requirements to obtain `oc` command output even as root, which
is not the case with base kubernetes. In order to use the API, users
should supply a bearer token either via the `openshift.token` plugin
option, or by setting the `SOSOCPTOKEN` environment variable to avoid
the token being included in output collected in the archive.
By default, the plugin will capture cluster-level information under the
'cluster_resources' directory within the plugin directory, organized in
subdirs by resource names. Similarly, each default namespace (project)
that exists at deployment time will be collected from, with each
namespace having a distinct subdir under the 'namespaces' directory
within the plugin directory. This list of namespaces to collect from can
be added to or overriden with the `--add-namespaces` and
`--only-namespaces` options respectively.
During testing it was observed that over 50 of these default namespaces
exist out of the box, and as such both the setup() and collect() phases
can take several minutes each. Total runtime of just the openshift
plugin from start to finish averages around 10 minutes, and so if the
plugin collects from the API a notification is printed to indicate this
so that end users do not erroneously think sos has hung.
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The KVM plugin get triggered in a container
(e.g lxd) because of "/sys/module/kvm"
inheritance from the kernel host.
Not only it's a waste of sosreport time,
but running it inside a container may
unintentionnaly reveal details from its
host. Which is a undesired behaviour.
Switching to /dev/kvm, is more appropriate
and follow current standard as used by tool
such as cpu-checker (kvm-ok) for instance.
And taking benefit of this change to get rid
of the check_enabled() overwrite in favor of
using "files=" trigger.
Closes: #2062
Resolves: #2063
Signed-off-by: Eric Desrochers <eric.desrochers@canonical.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Updates our exit path when we encounter a fatal filesystem error
(ENOSPC, EROFS) to use `os._exit(1)` rather than our own `_exit()`
method so that we can properly exit from the ThreadPool.
This does leave the temporary directory in place on the filesystem, so
the exit message has been updated to highlight that fact to the end
user.
Closes: #2071
Resolves: #2108
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
| |
Resolves: #2110
Signed-off-by: Ganesh Payelkar <gpayelka@redhat.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This plugin is only executed if Kubernetes is deployed using
Canonical's Charmed Distribution of Kubernetes. The plugin
should also be run if other deployment methods are used.
Also adding service names if the Ubuntu Snaps are used for
the kubernetes control plane.
Closes: #2103
Signed-off-by: Nick Niehoff <nick.niehoff@canonical.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
Iterate via "gluster volue info" output split to lines,
and dont truncate the trailing character (a relict from past different
content parsing).
Resolves: #2106
Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This plugin is only executed if docker.io is installed but
if docker-ce or docker-ee from docker.com are installed this
plugin should also run.
Also collecting /etc/containerd to align with the collection
on RHEL.
Closes: #2100
Resolves: #2101
Signed-off-by: Nick Niehoff <nick.niehoff@canonical.com>
Signed-off-by: Bryan Quigley <bryan.quigley@canonical.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
With the removal of mistral from undercloud, we would be
using $HOME/config-download for the config downloaded playbooks.
Added it to the collected locations on undercloud.
Also collects ansible.log from the new location $HOME.
Resolves: #2031
Signed-off-by: Rabi Mishra <ramishra@redhat.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
This patch updates memory plugin to collect
hugepage memory information
Resolves: #2078
Signed-off-by: Mamatha Inamdar <mamatha4@linux.vnet.ibm.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch updates scsi plugin to collect
virtual I/O server information.
This patch reads lspath, lsmap and lsnports data
which provides information about virtual device
path, ports and mapping Information
Related: #2078
Signed-off-by: Mamatha Inamdar <mamatha4@linux.vnet.ibm.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
This patch updates Infiniband plugin to collect
InfiniBand switch node information.
Related: #2078
Signed-off-by: Mamatha Inamdar <mamatha4@linux.vnet.ibm.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch updates kdump plugin to collect kdump and
fadump information.
In this patch we are reading, kdump configuration,panic
and kexec_loaded files which are helpful to debug kdump
failure issues
This patch also reads sys/kernel/fadump_enabled
and /sys/kernel/fadump/enabled both files as there
is upstream work to deprecate these files
Related: #2078
Signed-off-by: Mamatha Inamdar <mamatha4@linux.vnet.ibm.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
As part of #2035, we recurse into directory copyspecs in order to
properly enumerate all files under that path for size limiting reasons.
However, it was found that copyspecs that are regexes that match only
directories get added effectively twice - once for the top-level
directory directly and then again for the files underneathe.
Fix this by removing the top-level path from the returned set of paths
to collect from any given copyspec, if that top-level path is a
non-empty directory.
Resolves: #2089
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
HW mode clock settings and clock drift factor, valuable for hwclock.
Closes: #2094
Resolves: #2095
Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
dnf could wait for some interactive input from a user.
Use --assumeno, to keep going without modifying dnf's state.
Resolves: #2091
Signed-off-by: Emmanuel Roullit <emmanuel.roullit@gmail.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
Since reorganization of sosreport directory tree for 4.0 preparation,
Makefie stopped working. Let apply the reorg to also here.
Resolves: #2096
Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Adds facilities to capture metadata information about a particular sos
run, and save it to any resulting archive as `manifest.json`. This
manifest will include component-specific information that should assist
in analysis of sos runs.
For `report`, this is an extension of existing reporting functionality,
that records information on a per-plugin basis regarding command output
and file collection. However, more information is recorded in this
collection than the JSON or HTML reporting structures that review a
finished archive.
Manifest data is added at the time of collection; for commands this includes
a breakout of command, parameters, full execution syntax, filepath relative
to archive root, and the actual return code of the command, even if non-zero.
For files, the copied files are grouped by the specification set in
`add_copy_spec()` calls, I.E. for any file pattern regex that collects
at least one file, all files collected by that regex are listed.
Additionally, this adds a `tags` parameter to `add_copy_spec()`,
`add_cmd_output()`, and `collect_cmd_output()` that if used will add
tagging information to the manifest entries for those collections. This
is intended to assist in future sos work (E.G. `sos examine`) and third
party utilities in identifying relevant/connected data that is present
in the archive. Note that all collections specified by a single call to
any of these collection methods will inherit the same tags.
For `collect` we now record the cluster type, node list, and a separate
entry for each node in the node list that includes the sos version, node
sos policy, and the archive within the collect tarball that was
collected for that host.
Closes: #1990
Resolves: #2050
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This just leaves xz with gzip as the automatic backup.
Bzip2 took the most time by far and only does marginally
better than gzip.
time 27 Size 10.72MiB --batch -z xz --log-size 1
time 27 Size 16.32MiB --batch -z gzip --log-size 1
time 40 Size 14.17MiB --batch -z bzip2 --log-size 1
time 43 Size 105.21MiB --batch -z xz --all-logs
time 64 Size 187.38MiB --batch -z gzip --all-logs
time 135 Size 182.86MiB --batch -z bzip2 --all-logs
Resolves: #2053
Signed-off-by: Bryan Quigley <bryan.quigley@canonical.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
It's possible that a deployment is using ovs-bonds rather than
dpdk bond ports. In that case, it's required to pull the iface
information as well.
Resolves: #2051
Signed-off-by: Aaron Conole <aconole@redhat.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
| |
The connectivity fault mgmt, qos and bond data can all be retrieved
on a per-port basis.
Signed-off-by: Aaron Conole <aconole@redhat.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
| |
Signed-off-by: Aaron Conole <aconole@redhat.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
| |
Sometimes the permissions or selinux attributes are modified in an
unexpected way.
Signed-off-by: Aaron Conole <aconole@redhat.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
| |
Most instances of ovs-vsctl were run without a previous check. If
the vswitchd is in a hung state, the command may never timeout. Ensure
that most places are using the timeout option.
Signed-off-by: Aaron Conole <aconole@redhat.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This includes flows, ct stats, frag stats, and conntrack information
for all of the datapath types that are active. Additionally, pull the
tunnel information for datapaths.
Stop using 'ovs-dpctl' in favor of 'ovs-appctl dpctl/*' - for multiple
reasons. The 'ovs-dpctl' command doesn't support the userspace
datapath. 'ovs-dpctl' cannot pull all attribute types (including
type=offloaded) in newer OVS versions. Also, the default attribute
type is 'all' so adding type=offloaded is redundant.
Signed-off-by: Aaron Conole <aconole@redhat.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
| |
Pulls additional flow stats, features, forwarding information and mac
layer information.
Signed-off-by: Aaron Conole <aconole@redhat.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
| |
The bridge protocol support has been extended for some time,
so ensure that we actually pull this protocol information
if it is configured that way.
Signed-off-by: Aaron Conole <aconole@redhat.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
| |
Ensures that the get-mempool-info command will be present.
Signed-off-by: Aaron Conole <aconole@redhat.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Collect the output of the 'iotop' command
by specifying the number of samples in
the command line with the option 'samples',
with 0.5 seconds delay between each one.
By default we capture 20 samples.
Resolves: #2024
Signed-off-by: Jose Castillo <jcastillo@redhat.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
| |
The string 'id' is a Python keyword: do not use it as a local
variable name.
Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
| |
The os-release file may contain blank and comment lines. Skip
them when parsing values.
Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
An exception in SoS._init_component() currently leads to a fairly
terse error even with --debug:
# sos report -vv --batch --debug
Could not initialize 'report': dictionary update sequence element #0 has length 1; 2 is required
By propagating the exception when --debug is given we get the full
backtrace on the terminal:
# sos report -vv --batch --debug
Could not initialize 'report': dictionary update sequence element #0 has length 1; 2 is required
Traceback (most recent call last):
File "/home/breeves/src/git/sos/bin/sos", line 21, in <module>
sos = SoS(sys.argv[1:])
File "/home/breeves/src/git/sos/sos/__init__.py", line 112, in __init__
self._init_component()
File "/home/breeves/src/git/sos/sos/__init__.py", line 153, in _init_component
raise err
File "/home/breeves/src/git/sos/sos/__init__.py", line 149, in _init_component
self.cmdline)
File "/home/breeves/src/git/sos/sos/report/__init__.py", line 118, in __init__
super(SoSReport, self).__init__(parser, args, cmdline)
File "/home/breeves/src/git/sos/sos/component.py", line 98, in __init__
self.policy = sos.policies.load(sysroot=self.opts.sysroot)
File "/home/breeves/src/git/sos/sos/policies/__init__.py", line 60, in load
if policy.check(remote=remote_check):
File "/home/breeves/src/git/sos/sos/policies/cos.py", line 30, in check
os_release = dict(line.strip().split('=') for line in fp
ValueError: dictionary update sequence element #0 has length 1; 2 is required
Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Commands like:
machinectl -q shell user1@ ..
hang if not called on foreground / with terminal.
Resolves: #2082
Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
| |
Apply os.path.expanduser on ~[user], not ~%s .
Relates to: #2082
Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This adds a plugin for gathering information about the loaded XDP programs
on a system, using the xdp-loader tool in the xdp-tools package. This is
complementary with the existing eBPF plugin, because this plugin supports
getting status for multiple XDP programs on each interface as supported by
libxdp.
For now, this just prints the output of 'xdp-loader status' which will list
any XDP programs attached. We don't currently support structured
output (such as JSON) from xdp-tools, but should we add that in the future
we can of course update this plugin.
Signed-off-by: Toke Høiland-Jørgensen <toke@redhat.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
Apparently I just broke travis entirely with trying allow-
downgrades.
Resolves: #2077
Signed-off-by: Bryan Quigley <bryan.quigley@canonical.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
Since kernel-4, iptables / ip6tables is newly provided by nf_tables
kernel module. Therefore, collecting ip[,6]tables commands should
be gated by presence of also this kernel module.
Resolves: #2054
Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
Signed-off-by: Bryan Quigley <bryan.quigley@canonical.com>
|
|
|
|
|
|
|
|
|
| |
Skip just collecting some SSL stuff (not required, potentially sensitive).
Resolves: #2075
Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
nfsserver plugin enabledness relies on legacy init scripts. A more
appropriate way to idenfity a NFS server is via nfs-utils package
(though it could be present also on a NFS client).
As that package enables nfs plugin, it is reasonable to merge the
plugins into one.
Closes: #2061
Resolves: #2073
Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
'general' plugin was replaced by 'host' plugin years ago.
Let update an example in man pages accordingly.
Resolves: #2072
Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
This allows inspection of the rule order, as well as insert/edit/delete of
the rules by that numbered id (ufw delete 23).
More details can be found here: https://help.ubuntu.com/community/UFW
Resolves: #2065
Signed-off-by: David A. Desrosiers <setuid@gmail.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
The documentation references ./sosreport , but
the command now lives inside ./bin, so this
commit updates such references.
Resolves: #2060
Signed-off-by: Jose Castillo <jcastillo@redhat.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add the ability to collect data/info about rootless podman/buildah
containers, in particular:
- containers_common plugopt 'rootlessusers' as a list of users to inspect
- for each user, collect:
- its containers config
- [podman|buildah] info and [UID|GID] map
- collect user-status and few user-related config files
Resolves: #2055
Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Moving tmpdir specifications out of policy gave us the ability to have
logging within `Policy`, however it was unnoticed before that this
stopped the redirection from container filesystem to host filesystem
when sos was running in a container (and we wanted to save the archive
to the host and not the container).
Fix this by standardizing a check for a `HOST` environment variable,
when a `container` env var is set as well. This has been used by the Red
Hat policy for some time to determine chroot locations for when we're
running in a container, and has been working well. The `container`
environment variable should be set at container setup by modern
container runtimes.
If either the `container` or `HOST` environment variable is not set, but
we are running in a container, then we will continue to write to the
container's filesystem (while policy will still determine any chroot
requirements separately).
Resolves: #2041
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
Signed-off-by: Bryan Quigley <bryan.quigley@canonical.com>
|