| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Dropping 'ubuntu-support-status' as
is it confusing and not very useful
in favor of 'ubuntu-security-status'
which, for now, is only available
starting with Focal/20.04LTS.
Ubuntu bug report:
https://launchpad.net/bugs/1873362
Closes: #2139
Resolves: #2140
Signed-off-by: Eric Desrochers <eric.desrochers@canonical.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Collect journalctl logs also when journal is configured to store logs
in memory only.
Further, apply --since option to journal catalog logs.
Further, when --all-logs is provided, collect the transient logs in
/var/log/journal dir as well.
Resolves: #2130
Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
On RHEL 8, the option `--config="global{locking_type=0}"` is deprecated.
This option is included in the lvm_opts variable and thus produces
deprecation warnings in the outputs of pvscan, vgscan, pvs, vgs, and
lvs:
WARNING: locking_type (0) is deprecated, using --nolocking.
This patch fixes the issue by removing "locking_type=0" from the config
string and appending "--nolocking" to lvm_opts on systems where
"--nolocking" is supported.
Related to RHBZ#1849248.
Resolves: #2127
Signed-off-by: Reid Wahl <nrwahl@protonmail.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
It was reported that certain arches may create subdir structures under
/proc/bus/pci differently than others - most notably that the first
device subdir could be '0000:00' instead of just '00'.
Rather than chase these different layouts, update the gating check for
running `lspci` commands to being that /proc/bus/pci exists and it has
more than just the `devices` file present, as this file may be present
but empty when nothing else exists under `/proc/bus/pci`.
Resolves: #2138
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
| |
Fix several typos within the manpage
Resolves: #2133
Signed-off-by: Ponnuvel Palaniyappan <ponnuvel.palaniyappan@canonical.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
| |
Fixes a typo in detecting LUKS partitions for cryptsetup output
collection.
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
Adds a check on a per-component basis for if that component requires
root permissions to run. By default, SoSComponent sets this value to
`False`, so set it explicitly to `True` for `SoSReport`.
Closes: #1989
Resolves: #2126
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
Include the truststore used by the oVirt engine
as it is supposed to contain CA certificates only
Closes: #2122
Resolves: #2124
Signed-off-by: Miguel Martín <mmartinv@redhat.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
Include all the certificates in '/etc/pki/vdsm' and
make sure no keys are collected.
Closes: #2121
Resolves: #2123
Signed-off-by: Miguel Martín <mmartinv@redhat.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
Too many logs, audits or fact_names suggest performance problems,
so it is worth collecting stats/counts of them.
Resolves: #2117
Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
This patch dumps all struct_ops currently existing in
the system via the commant 'bpftool struct_ops dump'.
Resolves: #2116
Signed-off-by: Jose Castillo <jcastillo@redhat.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Increase the default size limit for /var/lib/selinux, as the global
default of 25 MB is too small to collect both the 'active' and
'targeted' subdirs in many test scenarios. Previously this had been
copied in full due to the size limiting bug that #2035 resolved, so now
we need to properly scope the size limit for this collection.
Resolves: #2090
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
| |
Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
If the password contains one or more equal signs ('='), only the substring
after the final equal sign is scrubbed. The rest of the password appears in
plain text.
This patch modifies the scrub regex to scrub all characters after the first
equal sign.
Related to RHBZ#1845386.
Resolves: #2109
Signed-off-by: Reid Wahl <nrwahl@protonmail.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
Obfuscate passwords in logs like
..'cpdb --create .. --password='Qg6Fej9wekDGiWrMDR9j8WWgwcq4dyP3' ..
Resolves: #2115
Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
As Satellite deployments get more stable, it is less important to verify
packages installed.
Further, as Satellite produces much logs worth to collect, the preset should
increase log size to 100 MB.
Resolves: #2112
Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
| |
Adds a new parser and map to allow user defined keyword obfuscation.
Users may now use the `--keywords` option to have `SoSCleaner` scan
lines for matching keywords, and replace them in place like we do for
other parsers.
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
| |
Updates SoSReport and SoSCollector to better handle any exceptions
raised by SoSCleaner when `--clean` is used, so that we no longer try to
use `SoSCleaner` in subsequent renaming methods (when that object is
likely uninstantiated now due to the exception)
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Updates `SoSCleaner` to add basic stats from runs to the manifest used
by either `report` or `collect`.
When either `report` or `collect` write their manifest to the archive,
it will now include cleaner data and will be obfuscated appropriately
after being written, just before the archive is finalized. Both `report`
and `collect` have had their final archive operations updated to
facilitate this.
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
When run via `report` or `collect`, the log entries from `SoSCleaner`
will be included in the "normal" logging for those components. However,
when run directly from the CLI, `sos clean` does not add its own logging
to the archive(s) processed.
Instead, now write the log file to disk in the same directory the
processed archive gets written to, rather than losing the logging
entirely.
|
|
|
|
|
|
|
|
|
|
|
|
| |
For each file processed, `sos clean` will now check the filename and
attemept to obfuscate it of any _known_ items, for example host names
used as part of command collections.
This must be done against known items, rather than regex parsing for
potential new items, due to the nature of filenames having a high
propensity to trigger false positives in regex patterns.
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
First, stop filtering out comments and process them the same as any
other line.
Second, improve the logging made for obfuscating files by including the
archive name and the relative path, rather than the absolute path within
the tmpdir we use. Note that this also changed the format of files to
skip to the relative path within the archive.
|
|
|
|
|
|
|
| |
Adds the abiltiy for parsers to individually skip parsing lines if they
meet some regex set in `self.skip_line_patterns`.
The IP parser has been set to skip lines from DNF output.
|
|
|
|
|
|
|
|
|
|
| |
Extends the clean/mask options to the parsers built for `report` and
`collect` so that those components hooking into `SoSCleaner` still allow
full functionality to the user.
Related: #1987
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Adds a hostname parser to `sos clean` that will attempt to obfuscate
FQDNs matching the hostname of the system that generated the sosreport,
as found in sos_commands/host/hostname.
Additionally, any domains added via the `--domains` option will also be
obfuscated, including any subdomains of the domain(s) specified by the
option.
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
Adds a manpage for `sos clean`, with a link for `sos mask`.
Updates `man sos` as well to include the options moved into the global
group as part of the SoSCleaner patchset.
Related: #1987
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
| |
Adds tests for the mappings and parsers used by SoSCleaner
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Adds functionality to SoSCollector runs to pass collected archives
through `--clean`, much the same as the functionality has been extended
to base `report` runs.
Note that running this way will, like report, only result in a single
obfuscated archive and the private mapping file. If users desire an
unobfuscated copy as well, then `sos collect` and `sos clean` should be
run separately.
Related: #1987
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Adds a new option, `--clean` or `--mask`, to hook a report into
`SoSCleaner`.
This is done after collection is complete, and before the archive is
built and compressed. Data is substituted in place for the report, which
means there will not be an obfuscated copy on disk. There will however
still be a mapping file produced and located in the same directory as
the final archive.
If it desired to have both an obfuscated and unobfuscated copy, it is
recommended to run `sos report` followed by a separate `sos clean`.
Related: #1987
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Adds a new component/subcommand `SoSCleaner`, accessible via `sos clean`
or `sos mask`.
This component is intended to bring similar functionality that is
available in the standalone `soscleaner` utility directly into the sos
project. It is designed to take either an untarr'ed sos directory, an sos
archive, or an archive of sos archives and obfuscate sensitive network
information (and optionally other data) from the report(s) that are
typically not able to be sanely scrubbed via the `postproc()` method of
plugins.
As of this first commit, users may execute `sos clean|mask $archive`
directly from the command line and expect to have an obfuscated archive
generated. Note that this obfuscated archive does NOT replace the
original archive on disk. Currently there is support for IPv4 IP
addresses, MAC addresses, and 64-bit IPv6 MAC addresses.
Future commits will aim to provide hooks for this functionality into
both `report` and `collect`. Additionally, more parsers will be added to
handle more types of data that needs consistent obfuscation.
Closes: #1987
Closes: #311
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
With the volume of information collected by the redesigned openshift
plugin, no longer restrict subdir creation with plugin directories to a
single level. Allow arbitrary depth to allow more efficient data
organization.
Resolves: #2052
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch updates the current `openshift` plugin from 2.x (now EOL) to
4.x (3.x is handled via the `origin` plugin). This has been done at the
request of the Red Hat OCP support team.
While OCP 4.x is still built on top of kubernetes, the collection
requirements differ far to much to be able to simply extend the
kubernetes plugin. Most notably, the OCP4 plugin is designed to collect
a significantly greater volume of data by default due to the complexity
of even a basic installation of OCP4 clusters. Additionally, there are
API login requirements to obtain `oc` command output even as root, which
is not the case with base kubernetes. In order to use the API, users
should supply a bearer token either via the `openshift.token` plugin
option, or by setting the `SOSOCPTOKEN` environment variable to avoid
the token being included in output collected in the archive.
By default, the plugin will capture cluster-level information under the
'cluster_resources' directory within the plugin directory, organized in
subdirs by resource names. Similarly, each default namespace (project)
that exists at deployment time will be collected from, with each
namespace having a distinct subdir under the 'namespaces' directory
within the plugin directory. This list of namespaces to collect from can
be added to or overriden with the `--add-namespaces` and
`--only-namespaces` options respectively.
During testing it was observed that over 50 of these default namespaces
exist out of the box, and as such both the setup() and collect() phases
can take several minutes each. Total runtime of just the openshift
plugin from start to finish averages around 10 minutes, and so if the
plugin collects from the API a notification is printed to indicate this
so that end users do not erroneously think sos has hung.
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The KVM plugin get triggered in a container
(e.g lxd) because of "/sys/module/kvm"
inheritance from the kernel host.
Not only it's a waste of sosreport time,
but running it inside a container may
unintentionnaly reveal details from its
host. Which is a undesired behaviour.
Switching to /dev/kvm, is more appropriate
and follow current standard as used by tool
such as cpu-checker (kvm-ok) for instance.
And taking benefit of this change to get rid
of the check_enabled() overwrite in favor of
using "files=" trigger.
Closes: #2062
Resolves: #2063
Signed-off-by: Eric Desrochers <eric.desrochers@canonical.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Updates our exit path when we encounter a fatal filesystem error
(ENOSPC, EROFS) to use `os._exit(1)` rather than our own `_exit()`
method so that we can properly exit from the ThreadPool.
This does leave the temporary directory in place on the filesystem, so
the exit message has been updated to highlight that fact to the end
user.
Closes: #2071
Resolves: #2108
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
| |
Resolves: #2110
Signed-off-by: Ganesh Payelkar <gpayelka@redhat.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This plugin is only executed if Kubernetes is deployed using
Canonical's Charmed Distribution of Kubernetes. The plugin
should also be run if other deployment methods are used.
Also adding service names if the Ubuntu Snaps are used for
the kubernetes control plane.
Closes: #2103
Signed-off-by: Nick Niehoff <nick.niehoff@canonical.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
Iterate via "gluster volue info" output split to lines,
and dont truncate the trailing character (a relict from past different
content parsing).
Resolves: #2106
Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This plugin is only executed if docker.io is installed but
if docker-ce or docker-ee from docker.com are installed this
plugin should also run.
Also collecting /etc/containerd to align with the collection
on RHEL.
Closes: #2100
Resolves: #2101
Signed-off-by: Nick Niehoff <nick.niehoff@canonical.com>
Signed-off-by: Bryan Quigley <bryan.quigley@canonical.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
With the removal of mistral from undercloud, we would be
using $HOME/config-download for the config downloaded playbooks.
Added it to the collected locations on undercloud.
Also collects ansible.log from the new location $HOME.
Resolves: #2031
Signed-off-by: Rabi Mishra <ramishra@redhat.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
This patch updates memory plugin to collect
hugepage memory information
Resolves: #2078
Signed-off-by: Mamatha Inamdar <mamatha4@linux.vnet.ibm.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch updates scsi plugin to collect
virtual I/O server information.
This patch reads lspath, lsmap and lsnports data
which provides information about virtual device
path, ports and mapping Information
Related: #2078
Signed-off-by: Mamatha Inamdar <mamatha4@linux.vnet.ibm.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
This patch updates Infiniband plugin to collect
InfiniBand switch node information.
Related: #2078
Signed-off-by: Mamatha Inamdar <mamatha4@linux.vnet.ibm.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch updates kdump plugin to collect kdump and
fadump information.
In this patch we are reading, kdump configuration,panic
and kexec_loaded files which are helpful to debug kdump
failure issues
This patch also reads sys/kernel/fadump_enabled
and /sys/kernel/fadump/enabled both files as there
is upstream work to deprecate these files
Related: #2078
Signed-off-by: Mamatha Inamdar <mamatha4@linux.vnet.ibm.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
As part of #2035, we recurse into directory copyspecs in order to
properly enumerate all files under that path for size limiting reasons.
However, it was found that copyspecs that are regexes that match only
directories get added effectively twice - once for the top-level
directory directly and then again for the files underneathe.
Fix this by removing the top-level path from the returned set of paths
to collect from any given copyspec, if that top-level path is a
non-empty directory.
Resolves: #2089
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
HW mode clock settings and clock drift factor, valuable for hwclock.
Closes: #2094
Resolves: #2095
Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
dnf could wait for some interactive input from a user.
Use --assumeno, to keep going without modifying dnf's state.
Resolves: #2091
Signed-off-by: Emmanuel Roullit <emmanuel.roullit@gmail.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
Since reorganization of sosreport directory tree for 4.0 preparation,
Makefie stopped working. Let apply the reorg to also here.
Resolves: #2096
Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Adds facilities to capture metadata information about a particular sos
run, and save it to any resulting archive as `manifest.json`. This
manifest will include component-specific information that should assist
in analysis of sos runs.
For `report`, this is an extension of existing reporting functionality,
that records information on a per-plugin basis regarding command output
and file collection. However, more information is recorded in this
collection than the JSON or HTML reporting structures that review a
finished archive.
Manifest data is added at the time of collection; for commands this includes
a breakout of command, parameters, full execution syntax, filepath relative
to archive root, and the actual return code of the command, even if non-zero.
For files, the copied files are grouped by the specification set in
`add_copy_spec()` calls, I.E. for any file pattern regex that collects
at least one file, all files collected by that regex are listed.
Additionally, this adds a `tags` parameter to `add_copy_spec()`,
`add_cmd_output()`, and `collect_cmd_output()` that if used will add
tagging information to the manifest entries for those collections. This
is intended to assist in future sos work (E.G. `sos examine`) and third
party utilities in identifying relevant/connected data that is present
in the archive. Note that all collections specified by a single call to
any of these collection methods will inherit the same tags.
For `collect` we now record the cluster type, node list, and a separate
entry for each node in the node list that includes the sos version, node
sos policy, and the archive within the collect tarball that was
collected for that host.
Closes: #1990
Resolves: #2050
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This just leaves xz with gzip as the automatic backup.
Bzip2 took the most time by far and only does marginally
better than gzip.
time 27 Size 10.72MiB --batch -z xz --log-size 1
time 27 Size 16.32MiB --batch -z gzip --log-size 1
time 40 Size 14.17MiB --batch -z bzip2 --log-size 1
time 43 Size 105.21MiB --batch -z xz --all-logs
time 64 Size 187.38MiB --batch -z gzip --all-logs
time 135 Size 182.86MiB --batch -z bzip2 --all-logs
Resolves: #2053
Signed-off-by: Bryan Quigley <bryan.quigley@canonical.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
It's possible that a deployment is using ovs-bonds rather than
dpdk bond ports. In that case, it's required to pull the iface
information as well.
Resolves: #2051
Signed-off-by: Aaron Conole <aconole@redhat.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|