aboutsummaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* [sos] add SoSOptions.dict()Bryn M. Reeves2018-06-201-1/+15
| | | | | | | | | | | Add a method to convert an SoSOptions() instance into a dict: the keys of the dictionary are the argument names and the values are the current option values. Any argument with a deque() value is converted to a plain list prior to processing. Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [sos] replace hard-coded format tuple with list comprehensionBryn M. Reeves2018-06-201-8/+1
| | | | | | | | | | Instead of hard-coding a tuple for every member of SoSOptions (% (self.foo, self.bar, ...)), use a list comprehension on the existing _arg_names list to obtain the attribute values via getattr(). This avoids the need to maintain the tuple manually as options are added or removed. Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [sosreport] add --list-presetsBryn M. Reeves2018-06-202-3/+30
| | | | | | Add a command line option to list out presets and their options: Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [policies] add on-disk preset supportBryn M. Reeves2018-06-202-4/+111
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add the ability for the Policy class to load presets in JSON format: a PresetDefaults object maps directly to the JSON file structure: the outer container is a dictionary whose keys are the names of the contained presets, and each preset may contain a "name", "desc", "note", and "opts" member. Methods are provided to load, write, delete, user presets, and to register policy-defined built-in presets: PresetDefaults.write() PresetDefaults.delete() Policy.register_presets() Policy.load_presets() policy.add_preset() policy.del_preset() The name is used to select a preset (in the same way as a policy defined preset), and the description and note give further information on the preset and its behaviour (for e.g. performance impact). The "opts" key maps to a further dictionary mapping sos argument names to their preset values. Loaded presets are added to the active policie's presets store and are available to the user via --preset. Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [sos] add automatic preset detection infrastructureBryn M. Reeves2018-06-203-3/+17
| | | | | | | | | | | | Add a new special value for the --preset argument, 'auto', which enables automatic detection of the preset identifier string by the policy class. A new method, probe_preset() is added which should be implemented by derived classes of Policy to return a ProductDefaults() object initialised appropriately for the running preset. Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [sosreport] apply preset defaults to command argumentsBryn M. Reeves2018-06-201-0/+15
| | | | Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [policies/redhat] add RHEL preset definitionsBryn M. Reeves2018-06-201-2/+57
| | | | | | | | Add initial RHEL product family presets to the RHELPolicy class and implement automatic preset detection for known Red Hat products (RHEL, Satellite, and Atomic). Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [policies] add PresetDefaults infrastructureBryn M. Reeves2018-06-201-1/+61
| | | | Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [sos,sosreport] add new SoSOptions.preset / --preset optionBryn M. Reeves2018-06-202-3/+6
| | | | Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [sos] add __str__ and __repr__ to SoSOptionsBryn M. Reeves2018-06-201-1/+31
| | | | Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [sos] make SoSOptions.profiles default an empty listBryn M. Reeves2018-06-201-1/+1
| | | | | | | | To be consistent with other deque()-typed arguments make the defined value of SoSOptions.profile be [] (this simplifies JSON encoding of the object). Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [sos] alphabetise SoSOptions attribute listBryn M. Reeves2018-06-201-17/+17
| | | | Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [sosreport] move SoSOptions from sos.sosreport to sosBryn M. Reeves2018-06-202-87/+118
| | | | | | | | | The options object now needs to be referenced in both the main sosreport script, and the policy classes: move its definition from the sos.sosreport module to sos to avoid recursive import errors between sosreport and policy. Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [sosreport] add SoSOptions.merge()Bryn M. Reeves2018-06-201-3/+30
| | | | | | | | | | | Add a method to merge a pair of SoSOptions, replacing unset or default values with those from the source options (and optionally overwriting already set values). This will be used to allow the policy framework to define default command line behaviour for specific products. Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [sosreport] add command line defaults dictionaryBryn M. Reeves2018-06-201-16/+22
| | | | | | | Add a dictionary mapping all argument names with a non-zero default value to the appropriate default. Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [sosreport] make --debug a booleanBryn M. Reeves2018-06-201-2/+1
| | | | | | Debugging is either on or off: make the argument match the option. Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [sosreport] alphabetise argument listBryn M. Reeves2018-06-201-54/+54
| | | | Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [sosreport] simplify SoSOptionsBryn M. Reeves2018-06-201-457/+157
| | | | | | | | | | | Simplify the SoSOptions class by removing the wrapping of the command line options and individual property methods. This removes some validation for API users implemented via the property methods: this can be re-added later if required as the interface is formalised. Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [Plugin] set size limits automaticallyBryn M. Reeves2018-06-202-0/+12
| | | | | | | | | | | If the caller does not specific an explicit size limit (or 0 to disable limits) automatically set the sizelimit argument for the add_copy_spec(), add_cmd_output(), and add_journal() methods to the value of the "log_size" option. Resolves: #1325 Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [Plugin] fix Plugin.add_journal() kwarg useBryn M. Reeves2018-06-201-3/+2
| | | | | | | The add_journal() method _must_ use explicit kwarg notation when calling the _add_cmd_output() helper. Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [Plugin] fix Plugin.add_cmd_output() kwarg useBryn M. Reeves2018-06-201-3/+4
| | | | | | | The add_cmd_output() method _must_ use explicit kwarg notation when calling the _add_cmd_output() helper. Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [sosreport] Correct verbosity level when -v is not specifiedJake Hunsaker2018-06-201-2/+2
| | | | | | | | | | | | | | | | | Fixes an issue where verbosity would be set to 'None' rather than 0 in the event that a sosreport was run with options, when those options did not include -v at all. This meant that if any options were specified, a newline would be printed for each started plugin when instead we should still have been refreshing a single line. Related to this, the previous newline behavior was beneficial when running with --batch, so that behavior is maintained if --batch is specified. Resolves: #1355 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com> Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [global] Transition to pycodestyleJake Hunsaker2018-06-2039-74/+76
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The 'pep8' tool has been renamed to 'pycodestyle' to avoid confusion with pep8 the document. pep8 (the tool) will no longer be updated and as such we should transition to the new pycodestyle. This commit fixes a number of PEP8 issues with the new pycodestyle linter that has tests for new additions to PEP8 that the pep8 tool did not include. Most of these are exception handling fixes like: sos/plugins/logs.py:96:13: E722 do not use bare 'except' Which are relatively straight forward to address, by instead catching just the applicable exceptions, e.g. IOError, ValueError, etc... In cases where there were many possible exceptions or where the "bare 'except'" was a final catch-all, we now use 'except Exception' to catch anything from the base exception class. Another frequent correction was for escape sequencing, such as: sos/plugins/logs.py:48:15: W605 invalid escape sequence '\S' The solution for this is to mark these regex strings as raw strings, which is preferred by `re` as future releases of that module may/will become more strict in its parsing of regex strings. Resolves: #1344 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com> Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [elastic] handle cases when config file isnt presentPavel Moravec2018-06-201-11/+14
| | | | | | | | | Catch and log any exception and set port as string before concatenating it. Resolves: #1352 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [fibrechannel] do not attempt to list non-existent directoriesBryn M. Reeves2018-06-201-6/+4
| | | | | | | | | | | Use a list comprehension to both generate the list of paths to collect, and to filter out any directories in the standard list of fibrechannel paths that do not exist on the running system. Resolves: #1353 Related: #1341 Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [sosreport] Fix finished running plugins reportJake Hunsaker2018-06-201-1/+1
| | | | | | | | | | | Adds the condition that no more plugins are supposed to run when there are no plugins running to report that sosreport has finished running plugins. Resolves: #1354 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com> Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [sosreport] use private plugin list for ThreadPoolExecutorBryn M. Reeves2018-06-191-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In some cases the concurrent plugins patch will produce output like the following (with a starting set of plugins of [activemq, apache, dbus, dhcp, manageiq, openshift, openstack_swift, puppet]: Starting 1/8 activemq [Running: activemq] Starting 2/8 apache [Running: apache] Starting 4/8 dhcp [Running: apache dhcp] Starting 5/8 manageiq [Running: apache dhcp manageiq] Starting 6/8 openshift [Running: apache openshift] Starting 7/8 openstack_swift [Running: apache openshift openstack_swift] Starting 8/8 puppet [Running: apache openshift puppet] Plugin #3 on the list (dbus) has vanished from the plugin run list. The ThreadPoolExecutor class uses itertools.izip() to walk the passed iterable of arguments. Since this references the same SoSReport.pluglist list that is being concurrently modified by the collect_plugin() methods it is possible for the pool iterator to "lose" an entry (another entry is removed by during the call to its collect_plugin() method, causing the list indices to shift, and making the index used in the next iteration of the pool's map() submission loop point one past the correct next entry. Avoid this problem by initialising a fresh copy of the pluglist to pass into the pool. Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [puppet] add package checksBryan Quigley2018-06-181-0/+2
| | | | | | | | | | | | | Previously this had no checks and would run for every sosreport, many of which don't have puppet installed. There have been several name changes so I included a lot of package names to try to cover all of the possible versions/distros. Resolves: #1345 Signed-off-by: Bryan Quigley <bryan.quigley@canonical.com> Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [skydive] add pluginMasco Kaliyamoorthy2018-06-181-0/+61
| | | | | | | | | | skydive is an open source real-time network topology and protocols analyzer for more info: http://skydive.network/ Resolves: #1351 Signed-off-by: Masco Kaliyamoorthy <mkaliyam@redhat.com> Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [sosreport] call traceback.print_exc without argumentsPavel Moravec2018-06-181-2/+1
| | | | | | | | | | to prevent uncaught exception on py3 also dont import already imported traceback class Resolves: #1349 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [plugins] Check read access before trying to list a dirBryan Quigley2018-06-181-1/+1
| | | | | | | | | | | I found a permission error when trying to copy /sys/fs/pstore when running on LXD containers. Adding a permission check first allows it to skip folders it can't list. Resolves: #1348 Signed-off-by: Bryan Quigley <bryan.quigley@canonical.com> Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [ipa] collect more logs and httpd alias configsPavel Moravec2018-06-181-1/+14
| | | | | | Resolves: #1346 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [ovirt_provider_ovn] fix typo in provider file/config variable namePavel Moravec2018-06-181-1/+1
| | | | | | Resolves: #1343 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [infiniband] test if dir exists before listing itPavel Moravec2018-06-181-1/+1
| | | | | | Resolves: #1342 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [fibrechannel] test if dir exists before listing itPavel Moravec2018-06-181-1/+2
| | | | | | Resolves: #1341 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [sosreport] --batch option does not display resultsPavel Moravec2018-06-181-2/+1
| | | | | | | | Fixing a regression caused by 8e69d36 . Resolves: #1337 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [archive] use threads for xz compressionPavel Moravec2018-06-184-9/+32
| | | | | | | | | | Moving also building of the command from Archive to Plugin class. Closes: #1196 Resolves: #1338 Signed-off-by: Pavel Moravec <pmoravec@redhat.com> Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [policies] add auxiliary function _get_pkg_name_for_binaryPavel Moravec2018-06-182-0/+12
| | | | | | | | | | In some cases, we need to determine what package provides given binary. This auxiliary function implements it - so far it works for and will be used by archive methods. Related to: #1196 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [sosreport] Fix threads execution with no options and output formattingJake Hunsaker2018-06-181-9/+13
| | | | | | | | | | | | | | | | | | | Fixes an issue where sosreport would not run due to an unset number of threads if 'sosreport' was run bare, with no options. Any option (threads or otherwise) would mask this problem due to how the default threads value was set by the parser. Now sosreport can be run without any options and run normally, with the concurrency provided by the threads. Additionally, changes the status line printed during plugin execution to be of a fixed length, so that output lines are not mangled when the status line is updated, rather than a new line being printed (such as what happens when using --batch). Resolves: #1336 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com> Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [elastic] Add Elasticsearch service pluginAmit Ghadge2018-06-181-0/+60
| | | | | | | | Closes: #1334 Resolves: #1335 Signed-off-by: Amit Ghadge <amitg.b14@gmail.com> Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [mssql] Add new pluginTakayoshi Tanaka2018-06-181-0/+92
| | | | | | | | | | | | | Adds a new plugin for Microsoft SQL Server on Linux. Collect log files. Collect journal logs. Collect keytab file info if AD auth is configured. Resolves: #1326 Signed-off-by: Takayoshi Tanaka <tatanaka@redhat.com> Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [origin] Collect {resolv,node-dnsmasq}.conf under /etc/origin/nodeKenjiro Nakayama2018-06-181-0/+2
| | | | | | | | | | | | | | | Since OCP 3.6+, there are some DNS config under /etc/origin/node/ such as `node-dnsmasq.conf` and `resolv.conf`. Although current sosreport does not gather them, these are important for troubleshooting of DNS issues. This patch updates origin.py to collect {resolv,node-dnsmasq}.conf under /etc/origin/node/. Resolves: #1311 Signed-off-by: Kenjiro Nakayama <nakayamakenjiro@gmail.com> Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [openstack_instack] add support for containerized undercloudEmilien Macchi2018-06-181-3/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 1) Do the copy in one call with variables, so it's more efficient. 2) Collect the config tarball It'll be very useful to collect the config tarball which containers TripleO Heat Templates used to deploy the undercloud and the Ansible playbooks generated by config-download as well. Both of them are archived and compressed in /home/stack/undercloud-install-*.tar.bzip2 files everytime install/upgrade commands are run. 3) Collect ephemeral Heat logs /var/log/heat-launcher is a new directory that contains the heat-installer sqlite, logs and configuration. The content is small because it's only used once and it's ephemeral (purged everytime an operator install or upgrade an undercloud). However, collecting these files will be useful to help customers. 4) Collect new logfile When containerizing the undercloud, we're moving install-undercloud.log into $HOME which makes it more accessible for the users, while we're removing instack(-undercloud) bits over time. This patch add $HOME/install-undercloud.log file and document why we're doing this. Note that we still collect old file for previous releases. Resolves: #1279 Signed-off-by: Emilien Macchi <emilien@redhat.com> Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [archive] make FileCacheArchive methods re-entrantBryn M. Reeves2018-06-182-74/+177
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Since plugins are now run concurrently the Archive class must be re-entrant: different plugins may attempt to create the same paths in the archive, and since they are executing in parallel and with no locking, there exists a TOCTOU race between the use of os.path.{exists,isdir, etc.}() and the subsequent creation of a new path. Address this by implementing a simple monitor-style locking scheme for the FileCacheArchive class. A `_path_lock` member is added to the class and this must be held across any operation that modifies the namespace of the archive (including file, directory, symlink, and special node creation). Additionally, the `_check_path()` Archive method (which checks for the existence of path components up to the basename of the file, creating directories as required) is extended to also carry out a test for the existence of the path, and if the path does exist, that the object is of the expected type. A mismatch in the object type generates a ValueError with an appropriate string description of the problem. Benchmarking shows a consistent small improvement with the patch: this is a result of reduced redundant copy operations that the previous archive structure allowed (since duplicate operations are now aborted as soon as the _path_lock is acquired). Resolves: #1340 Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* netplan conf files can exist in 3 locationsEric Desrochers2018-06-071-0/+2
| | | | | | | | | | | | | | | | | Configuration files can exist in three different locations with the precedence from most important to least as follows: /run/netplan/*.yaml /etc/netplan/*.yaml /lib/netplan/*.yaml Alphabetically later files, no matter what directory in, will amend keys if the key does not already exist and override previous keys if they do. Resolves: #1318 Signed-off-by: Eric Desrochers <eric.desrochers@canonical.com> Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [Ubuntu] Add ubuntu-advantage status, u-s-s to better default and hweBryan Quigley2018-06-071-12/+5
| | | | | | | | | | | | | | | | | Add ubuntu-advantage status which is installed by default on Ubuntu and shows when various UA items are enabled - esm, fips, and livepatch. Switch ubuntu-support-status to always --show-all. It takes exactly the same amount of time to run (in both cases it's relatively slow at 3 seconds), and only adds 30k uncompressed/12k compressed. Add hwe-support-status to give more details about Ubuntu Hardware enablement if present. Resolves: #1330 Signed-off-by: Bryan Quigley <bryan.quigley@canonical.com> Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [omnipath_*] Add two pluginsPavel Moravec2018-06-072-0/+109
| | | | | | | | | | | plugins added: - omnipath_manager for OPA fabric subnet manager - omnipath_client for management/head node of the cluster Resolves: #1291 Signed-off-by: Pavel Moravec <pmoravec@redhat.com> Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [archive] switch compression preset from -1 to -2Bryan Quigley2018-06-071-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | I've noted an uptick of users recompressing sosreports to get a better compression rate for uploading over slow links. My tests indicate that -2 is a better default and should remove any benefit of compressing again. In tested cases -2 does not cause any significant CPU increase and always has shown an improvment - sometimes substancial. The worst was 3seconds more for a 15MB savings. I would generally expect to see similar percentage savings as sosreports get bigger. -3 and higher were evaluated but had diminshing returns that didn't seem worth it. Tests results: NVMe drive xz -a --batch -1 5.9M 30.766s -2 5.7M 30.673s -3 5.6M 30.101s -4 5.5M 35.882s bz2 bz2 -a --batch -1 11M 36.675s -2 8.3M 38.792s xz -a --all-logs --batch -1 87M 1m8.994s -2 72M 1m11.926s -3 69M 1m18.210s -4 70M 2m33.930s Old hard drive -a --batch -1 5.3M 14.935s -2 5.2M 15.611s -a --batch --all-logs -1 35M 38.881s -2 26M 38.337s Resolves: #1307 Signed-off-by: Bryan Quigley <bryan.quigley@canonical.com> Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [filesys] Add permission and selinux check for /tmpJake Hunsaker2018-06-071-1/+8
| | | | | | | | | | | | | | | | Adds a check for the /tmp directory for permissions and selinux context. If either of these are incorrect, the system in question can exhibit strange behavior and otherwise be difficult for support teams to detect. On systems where SELinux is disabled, the permission check is still relevant and the command still produces useful output with the -Z option. Resolves: #1327 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com> Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [docker] Collect events and storage informationJake Hunsaker2018-06-071-1/+4
| | | | | | | | | | | Adds collection of the previous day's worth of events handled by the daemon, a summarized output of storage needs, and a listing of volumes known to the daemon. Resolves: #1328 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com> Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
generated by cgit (git 2.34.1) at 2024-09-29 17:25:48 +0000