| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Adds a new `--low-priority` option to report, which will attempt to
constrain the process priority for the report generation. We do this by
attempting to set ourselves to an 'idle' IO class, as well as setting
our niceness to 19 to avoid contending for CPU time.
This is also exposed via `sos collect`, however users should note that
this will not be effective until the sos-4.5.1 release.
Closes: #3127
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Historically, journal sizes have been limited to the *higher* of 100MB
or `--log-size`. While this had the benefit of potentially controlling
both logs and journals with the same option, it was not immediately
intuitive to end users and downright prevented collecting less than
100MB of journals.
Address this by separating journal size limiting from `--log-size` by
adding a new `--journal-size` option (default 100). This will allow
users to individually control journal sizes without any "gotcha"
scenarios with relation to general log size limiting.
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
Removes the `--sos-cmd` option for `sos collect`. Allowing passthru
options in this manner is inherently flawed, and any attempts at
sanitizing potentially malicious/dangerous values will always be a
losing battle. Instead, `sos collect` should leverage available `report`
options that are vetted and handled via the existing per-node
capabilities checks that is well-defined for explicit passthru options.
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This commit adds a new parser and accompanying map for obfuscating IPv6
addresses.
This new parser will attempt to capture valid IPv6 networks and
addresses, and produce a mostly-randomized obfuscated pair. Due to the
multiple formats an IPv6 address can take, some identifiers are
necessary to preserve relevant information while still obfuscating
actual addresses and networks.
For example, global unicast addresses that have more than one defined
hextet (greater than /16 prefix) will always generate an obfuscated
address starting with `534f` (or 'so', continuing the style of our mac
address handling that uses 'sos' as an identifier). Addresses with a /16
prefix or less, will start with simply '53'. Private addresses, which
start with `fd` will generate an obfuscated address starting with
`fd53`, so that the contextual understanding that it is a private
network/address can remain. Link-local addresses which start with
`fe80::` will remain that way, only having the device hextets obfuscated
- again, keeping the contextual information that it is a link-local
interface intact, as otherwise these obfuscations may confuse end
users reviewing an sos report for problems.
Note that the address `::1` and `::/0` are explicitly skipped and never
obfuscated, for the same reasons given above.
Additionally, this parser/map will write data to the default map (and
any per-run private maps) differently than previous parsers. Rather than
simply dumping the obfuscation pairs into the map, it is broken up via
network, with hosts belonging to that network nested inside those
network entries (still being json-formatted). Users will also note that
the ipv6 entries in the map also have a `version` key, which is intended
to be used for handling future updates to the parser/map when upgrading
from an older sos version to a newer one. This may or may not be carried
over to future updates to other parsers.
Closes: #3008
Related: RHBZ#2134906
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
Adds a new `--disable-parsers` option that allows users to selectively
disable parsers for a given execution of `sos clean`.
This may be useful in specific scenarios where obfuscation is not
strictly needed for all the types of data we obfuscate, and where the
user trusts whomever may be receiving the archive for review.
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Adds a new `--encrypt` option that may be used in place of the existing
encrypt-related options. If used alongside `--batch`, this new option
will cause sos to set encryption options based on the `SOSENCRYPTKEY` or
`SOSENCRYPTPASS` environment variables. This allows users to leverage
this functionality without potentially leaking the values into `ps`-like
output inside the archive.
If `--batch` is not used, then this option prompts the user to select
the type of method to use and then provide the value in-line, or to use
the env var option.
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
| |
Set --force-pull-image=True by default, can be turned off with
--force-pull-image=False
Signed-off-by: Nadia Pinaeva <npinaeva@redhat.com>
|
|
|
|
|
|
| |
Related to #2860
Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This commit marks the beginning of the addition of a new `help`
component for sos, which will be used to display more in-depth help
information surrounding all the different components of sos.
The command is intended to be invoked with a specific help topic in
mind, that mirrors our project layout. E.G. to get help on the report
component, a user would use `sos help report`, whereas to get help on a
specific plugin a user would use `sos help report.plugins.$plugin`.
This first commit includes both the initial framework for the `help`
component, as well as updating `SoSReport` and `Plugin` to provide a
basic implementation for the new subcommand. Additionally, `Policy` is
given a basic framework for existing policies to report certain default
values as well as available presets. A stub section is provided for the
base `RedHatPolicy` to serve as a redirector for the actually used
policies that subclass it.
Closes: #2205
|
|
|
|
|
|
|
|
|
|
|
|
| |
As --estimate-only calculates disk usage based on `stat` data that
differs from outputs of other commands like `du`, enhance the warning
about reliability of the calculated estimation.
Also add a rule-of-thumb recommendation of real disk space requirements.
Resolves: #2815
Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Adds a new `--container-runtime` option that allows users to control
what default container runtime is used by plugins for container based
collections, effectively overriding policy defaults.
If no runtimes are active, this option is effectively ignored. If
however runtimes are active, but the requested one is not, raise an
exception to abort collection with an appropriate message to the user.
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This commit adds a new transport for `sos collect` by leveraging a
locally available `oc` binary that has been properly configured for
access to an OCP cluster.
This transport will allow users to use `sos collect` to collect reports
from an OCP cluster without directly connecting to any of the nodes
involved. We do this by using the `oc` binary to first launch a pod on
target node(s) and then exec our discovery commands and eventual `sos
report` command to that pod. This in turn is dependent on a function API
for the `oc` binary to communicate with. In the event that `oc` is not
__locally__ available or is not properly configured, we will fallback to
the current default of using SSH ControlPersist to directly connect to
the nodes. Otherwise, the OCP cluster will attempt to automatically use
this new transport.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Adds a new `--transport` option for users to be able to specify the type
of transport to use when connecting to nodes. The default value of
`auto` will defer to the cluster profile to set the transport type,
which will continue to default to use OpenSSH's ControlPersist feature.
Clusters may override the new `set_transport_type()` method to change
the default transport used.
If `--transport` is anything besides `auto`, then the cluster profile
will not be deferred to when choosing a transport for each remote node.
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
| |
Adds a new type of `SoSObfuscationArchive` to add support for
obfuscating archives generated by the Insights project.
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This commit removes the restriction imposed on `sos clean` since its
introduction in sos-4.0 to only work against known sos report archives
or build directories. This is because there has been interest in using
the obfuscation bits of sos in other data-collector projects.
The `SoSObfuscationArchive()` class has been revamped to now be an
abstraction for different types of archives, and the cleaner logic has
been updated to leverage this new abstraction rather than assuming we're
working on an sos archive.
Abstractions are added for our own native use cases - that being `sos
report` and `sos collect` for at-runtime obfuscation, as well as
standalone archives previously generated. Further generic abstractions
are available for plain directories and tarballs however these will not
provide the same level of coverage as fully supported archive types, as
is noted in the manpage for sos-clean.
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
| |
Add report option --estimate-only to estimate disk space requirements
when running a sos report.
Resolves: #2673
Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This commit follows up on the previous changes from #2555 by fully
dropping the `--master` option in favor of `--primary`, `--manager`, and
`--controller`.
Internal references have been updated as well, using the term `primary`
across all instances. Note that within OCP cluster profile, 'primary'
labeling and option setting still relies on the 'master' label/role
reported by the environment, as that remains the term used to identify
such nodes.
Resolves: #2329
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Adds a global `--namespaces` option that can be used to limit the
number of namespaces all plugins will iterate over. If a plugin provides
a specific plugin option, such as the `networking.namespaces` option,
then if set that plugin option will override the global option value.
The global option defaults to not limiting namespaces, matching current
behavior.
Closes: #2092
Resolves: #2547
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Binary files generally speaking cannot be obfuscated, and as such we
should remove them from archives being obfuscated by default so that
sensitive data is not mistakenly included in an obfuscated archive.
This commits adds a new `--keep-binary-files` option that if used will
keep any encountered binary files in the final archive. The default
option of `false` will ensure that encountered binary files are removed.
The number of removed binary files per archive is reported when
obfuscation is completed for that archive.
Closes: #2478
Resolves: #2524
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
Unify capitalisation of name and synopsis.
Add references to sos.conf to SEE ALSO of all binaries.
Resolves: #2581
Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Adds options that allow a user to specify registry authentication,
either via username/password or an authfile, to allow pulling an image
that exists on a non-public registry.
If a username/password is provided, that will be used. If not, we will
attempt to use an authfile - either provided by the user or by a cluster
profile.
Also adds an option to forcibly pull a new(er) version of the specified
image, to alleviate conditions where a too-old version of the image
already exists on the host.
Closes: #2534
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
Adds support for uploading via SFTP. This is done via pexpect calling
the system's locally available SFTP binary. If either that binary or
pexpect are unavailable on the local system, we will exit gracefully and
report the issue. This allows sos to keep python3-pexpect as a
recommends rather than a hard dependency.
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Updates the language in the manpage to clarify that in order to enable
additional plugins outside of a profile when a profile is specified,
that users must use `-o` instead of `-e`.
Remove the 'current profiles' list from the manpage as it is outdated
and it is better to refer users to the list reported by the local
installation.
Closes: #504
Resolves: #2543
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
Adds a new `--upload-no-ssl-verify` option that controls if we verify the
SSL certificate for an upload target. This will default to `False` to
retain previous default behavior of performing SSL verification.
Closes: #2497
Resolves: #2540
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The python requests module does not provide actual support for
streaming, that is provided by requests-toolchain. The support in
requests is for easy multipart uploads. The internal methods and
variables for https uploads within `Policy` however revolve around
streaming enablement, but don't actually influence the use of streaming
or not. This was due to placeholders during development just being
carried forward rather than corrected before merge.
This was largely forgotten about, until recent reports of upload issues
for user-provided endpoints.
This commit serves to correct the language around the currently
supported https upload functionality. That is, we either use a 'put' or
'post' based on the loaded policy defaults. Further, allow users to
control this setting with a new `--upload-method` option. This will
allow users to specify upload urls that may require the opposite HTTP
method than what the policy defaults dictate.
Related: #2497
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
| |
The `collect` component supports `--log-size` as a passthru option for
report. Add it to the `collect` manpage for completeness.
Related: #2510
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
Updates the description of `--log-size` in the manpage for `report` so
that it accurately reflects how size limitations are determined and
imposed during collections.
Closes: #1900
Resolves: #2513
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fixes an issue where archive debug logging was controlled by the use of
`--debug` rather than `--verbose`. Removes a superfluous log in
`FileCacheArchive.add_link()`.
Also, adjusts the different verbosity levels to be more meaninful. Now,
the use of `-v` will enable debug logging but will not print those
messages to console, `-vv` will print debug logging to console, while
`-vvv` will enable archive debug logging which is expected to be
significant due to most file operations being logging at some point
with archive debug logging.
Resolves: #2507
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add --cmd-timeout option to configure command timeout. Plugin-specific
option of the same name (i.e. -k logs.cmd-timeout=60) can control the
timeout per plugin.
Option defaults and global/plugin-specific option preference follows the
--plugin-timeout rules.
Resolves: #2466
Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
This patch fixes references to sosreport, to the
preferred 'sos report'. Also adds "SEE ALSO" consistently
for all man pages, and fixes a MAINTAINER line.
Resolves: #2432
Signed-off-by: Jose Castillo <jcastillo@redhat.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The default timeout for all plugins is 300, but
the global timeout is set as -1 in the option list. This is
because this timeout is unset by default. This patch
attempts to clarify the -1 exposed to the user when using
commands like 'sos report -l', via the option description
as well as a note clarifying it in the man page.
Closes: #2003
Resolves: #2415
Signed-off-by: Jose Castillo <jcastillo@redhat.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This commit adds the ability for users to provide a text file with a
list of newline-delimited keywords that should be obfuscated, rather
than requiring all keywords be specified either by the `--keywords`
option or configuration file settings.
Files may be provided via the new `--keyword-file` option which is
available to `clean`, `collect`, and `report`.
Closes: #2401
Resolves: #2408
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
The option 'cleaner' was not implemented
as an alias. This patch enables it for both
'sos report' and 'sos collect'.
Resolves: #2405
Signed-off-by: Jose Castillo <jcastillo@redhat.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This commit alters several option long-form names or destination names
to align those values in a sensible way. This serves to not only remove
some abiguity in option naming in code, but also to make it so that the
"effective options" line logged in every sos execution can be direction
copy-pasted as a working command.
Closes: #2288
Resolves: #2398
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Cleaning up references of --ticket-number, as it was fully replaced
by --case-id.
The credit goes to @mamatha4 .
Resolves: #2375
Relates to: #2374
Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Adds two new options, `--skip-commands` and `--skip-files`, that allow
users to selectively skip specific command or file collection instead of
having to disable whole plugins to skip those collections.
These options are also exposed via `sos collect`, being gated by a
version of 4.1 since that is the next scheduled release where we can
guarantee this functionality will be present.
Closes: #2203
Resolves: #2271
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The username and password used by `Policy` when `--upload` is used may
now be read via environment variables to avoid sensitive information
from being included in `ps` output if they were provided via
command-line options instead of the interactive prompts.
Upload username may now be provided via the SOSUPLOADUSER variable.
Upload password may now be provided via the SOSUPLOADPASSWORD variable.
Resolves: #2216
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Updates the manpage for further changes to the configuration design of
sos. `man sos.conf` now includes information on non-root user overrides,
as well as information regarding the subdirectories under `/etc/sos/` or
`$HOME/.config/sos/`.
The `sos_extras` plugin docstring has been copied into the man page as
well for better coverage of how users are expected to use that plugin.
The docstring remains in place in the actual plugin as well.
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Moves the default config file we look for to /etc/sos/sos.conf instead
of /etc/sos.conf. Extends the config file to look for a section matching
the name of the component being used. Renames the "general" section to
"global" and the "tunables" section to "plugin_options".
Updates the default sos.conf to this style and adds some comments to the
file. Update the man page for sos.conf.
Note that this commit does NOT update sos.spec to drop the default
configuration file in the new location, as that will be handled by a
later commit to update the specfile wholesale.
Closes: #2125
Resolves: #2136
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
| |
Fix several typos within the manpage
Resolves: #2133
Signed-off-by: Ponnuvel Palaniyappan <ponnuvel.palaniyappan@canonical.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
| |
Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
|
|
|
|
|
|
|
|
|
| |
Adds a new parser and map to allow user defined keyword obfuscation.
Users may now use the `--keywords` option to have `SoSCleaner` scan
lines for matching keywords, and replace them in place like we do for
other parsers.
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Adds a hostname parser to `sos clean` that will attempt to obfuscate
FQDNs matching the hostname of the system that generated the sosreport,
as found in sos_commands/host/hostname.
Additionally, any domains added via the `--domains` option will also be
obfuscated, including any subdomains of the domain(s) specified by the
option.
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
Adds a manpage for `sos clean`, with a link for `sos mask`.
Updates `man sos` as well to include the options moved into the global
group as part of the SoSCleaner patchset.
Related: #1987
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
'general' plugin was replaced by 'host' plugin years ago.
Let update an example in man pages accordingly.
Resolves: #2072
Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
Replaces the previous `--threads` option in collect with a `--jobs`
option to determine the number of concurrent collections to run.
Uses `--threads` as a passthru option now if the node supports it
(sos-3.6+), in an effort to unify option meanings between components.
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
| |
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Adds a general manpage for the sos command that then points to the
available components.
Renames sosreport.1 to sos-report.1 to match the component style, and
then symlinks sosreport.1 to it.
Adds a symlink from sos-collector.1 to sos-collect.1.
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
Adds the manpage from sos-collector to the tree as 'sos-collect', and
updates the instances of 'sos-collector' to 'sos collect' within the
manpage.
Additionally updates the man page to include newly available options.
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|
|
|
|
|
|
|
| |
Adds manage entries for the various --upload options added as part of
the previous patches adding upload functionality.
Signed-off-by: Jake Hunsaker <jhunsake@redhat.com>
|