aboutsummaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* [sosreport] improve --list-preset argument filteringBryn M. Reeves2018-06-201-3/+2
| | | | | | | Preset arguments should not be included in presets: simplify the filtering to use lists of filtered argument names. Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [sos] improve preset argument filteringBryn M. Reeves2018-06-201-2/+2
| | | | | | | Preset arguments should not be included in presets: simplify the filtering to use lists of filtered argument names. Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [policies] return None in find_preset() if preset not foundBryn M. Reeves2018-06-202-4/+3
| | | | Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [sosreport] do not display {add,del}_preset in --list-presetsBryn M. Reeves2018-06-201-0/+8
| | | | Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [sosreport] add --add-preset and --del-presetBryn M. Reeves2018-06-202-6/+71
| | | | | | | | | Allow user-defined named presets, stored by default in the /var/lib/sos/presets directory. A description and note for user presets may be optionally spefified using the --desc/--description and --note options. Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [sos] add add_preset and del_preset to SoSOptionsBryn M. Reeves2018-06-201-5/+8
| | | | Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [sos] add SoSOptions.dict()Bryn M. Reeves2018-06-201-1/+15
| | | | | | | | | | | Add a method to convert an SoSOptions() instance into a dict: the keys of the dictionary are the argument names and the values are the current option values. Any argument with a deque() value is converted to a plain list prior to processing. Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [sos] replace hard-coded format tuple with list comprehensionBryn M. Reeves2018-06-201-8/+1
| | | | | | | | | | Instead of hard-coding a tuple for every member of SoSOptions (% (self.foo, self.bar, ...)), use a list comprehension on the existing _arg_names list to obtain the attribute values via getattr(). This avoids the need to maintain the tuple manually as options are added or removed. Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [sosreport] add --list-presetsBryn M. Reeves2018-06-202-3/+30
| | | | | | Add a command line option to list out presets and their options: Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [policies] add on-disk preset supportBryn M. Reeves2018-06-202-4/+111
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add the ability for the Policy class to load presets in JSON format: a PresetDefaults object maps directly to the JSON file structure: the outer container is a dictionary whose keys are the names of the contained presets, and each preset may contain a "name", "desc", "note", and "opts" member. Methods are provided to load, write, delete, user presets, and to register policy-defined built-in presets: PresetDefaults.write() PresetDefaults.delete() Policy.register_presets() Policy.load_presets() policy.add_preset() policy.del_preset() The name is used to select a preset (in the same way as a policy defined preset), and the description and note give further information on the preset and its behaviour (for e.g. performance impact). The "opts" key maps to a further dictionary mapping sos argument names to their preset values. Loaded presets are added to the active policie's presets store and are available to the user via --preset. Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [sos] add automatic preset detection infrastructureBryn M. Reeves2018-06-203-3/+17
| | | | | | | | | | | | Add a new special value for the --preset argument, 'auto', which enables automatic detection of the preset identifier string by the policy class. A new method, probe_preset() is added which should be implemented by derived classes of Policy to return a ProductDefaults() object initialised appropriately for the running preset. Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [sosreport] apply preset defaults to command argumentsBryn M. Reeves2018-06-201-0/+15
| | | | Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [policies/redhat] add RHEL preset definitionsBryn M. Reeves2018-06-201-2/+57
| | | | | | | | Add initial RHEL product family presets to the RHELPolicy class and implement automatic preset detection for known Red Hat products (RHEL, Satellite, and Atomic). Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [policies] add PresetDefaults infrastructureBryn M. Reeves2018-06-201-1/+61
| | | | Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [sos,sosreport] add new SoSOptions.preset / --preset optionBryn M. Reeves2018-06-202-3/+6
| | | | Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [sos] add __str__ and __repr__ to SoSOptionsBryn M. Reeves2018-06-201-1/+31
| | | | Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [sos] make SoSOptions.profiles default an empty listBryn M. Reeves2018-06-201-1/+1
| | | | | | | | To be consistent with other deque()-typed arguments make the defined value of SoSOptions.profile be [] (this simplifies JSON encoding of the object). Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [sos] alphabetise SoSOptions attribute listBryn M. Reeves2018-06-201-17/+17
| | | | Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [sosreport] move SoSOptions from sos.sosreport to sosBryn M. Reeves2018-06-202-87/+118
| | | | | | | | | The options object now needs to be referenced in both the main sosreport script, and the policy classes: move its definition from the sos.sosreport module to sos to avoid recursive import errors between sosreport and policy. Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [sosreport] add SoSOptions.merge()Bryn M. Reeves2018-06-201-3/+30
| | | | | | | | | | | Add a method to merge a pair of SoSOptions, replacing unset or default values with those from the source options (and optionally overwriting already set values). This will be used to allow the policy framework to define default command line behaviour for specific products. Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [sosreport] add command line defaults dictionaryBryn M. Reeves2018-06-201-16/+22
| | | | | | | Add a dictionary mapping all argument names with a non-zero default value to the appropriate default. Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [sosreport] make --debug a booleanBryn M. Reeves2018-06-201-2/+1
| | | | | | Debugging is either on or off: make the argument match the option. Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [sosreport] alphabetise argument listBryn M. Reeves2018-06-201-54/+54
| | | | Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [sosreport] simplify SoSOptionsBryn M. Reeves2018-06-201-457/+157
| | | | | | | | | | | Simplify the SoSOptions class by removing the wrapping of the command line options and individual property methods. This removes some validation for API users implemented via the property methods: this can be re-added later if required as the interface is formalised. Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [Plugin] set size limits automaticallyBryn M. Reeves2018-06-202-0/+12
| | | | | | | | | | | If the caller does not specific an explicit size limit (or 0 to disable limits) automatically set the sizelimit argument for the add_copy_spec(), add_cmd_output(), and add_journal() methods to the value of the "log_size" option. Resolves: #1325 Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [Plugin] fix Plugin.add_journal() kwarg useBryn M. Reeves2018-06-201-3/+2
| | | | | | | The add_journal() method _must_ use explicit kwarg notation when calling the _add_cmd_output() helper. Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [Plugin] fix Plugin.add_cmd_output() kwarg useBryn M. Reeves2018-06-201-3/+4
| | | | | | | The add_cmd_output() method _must_ use explicit kwarg notation when calling the _add_cmd_output() helper. Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [sosreport] Correct verbosity level when -v is not specifiedJake Hunsaker2018-06-201-2/+2
| | | | | | | | | | | | | | | | | Fixes an issue where verbosity would be set to 'None' rather than 0 in the event that a sosreport was run with options, when those options did not include -v at all. This meant that if any options were specified, a newline would be printed for each started plugin when instead we should still have been refreshing a single line. Related to this, the previous newline behavior was beneficial when running with --batch, so that behavior is maintained if --batch is specified. Resolves: #1355 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com> Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [global] Transition to pycodestyleJake Hunsaker2018-06-2039-74/+76
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The 'pep8' tool has been renamed to 'pycodestyle' to avoid confusion with pep8 the document. pep8 (the tool) will no longer be updated and as such we should transition to the new pycodestyle. This commit fixes a number of PEP8 issues with the new pycodestyle linter that has tests for new additions to PEP8 that the pep8 tool did not include. Most of these are exception handling fixes like: sos/plugins/logs.py:96:13: E722 do not use bare 'except' Which are relatively straight forward to address, by instead catching just the applicable exceptions, e.g. IOError, ValueError, etc... In cases where there were many possible exceptions or where the "bare 'except'" was a final catch-all, we now use 'except Exception' to catch anything from the base exception class. Another frequent correction was for escape sequencing, such as: sos/plugins/logs.py:48:15: W605 invalid escape sequence '\S' The solution for this is to mark these regex strings as raw strings, which is preferred by `re` as future releases of that module may/will become more strict in its parsing of regex strings. Resolves: #1344 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com> Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [elastic] handle cases when config file isnt presentPavel Moravec2018-06-201-11/+14
| | | | | | | | | Catch and log any exception and set port as string before concatenating it. Resolves: #1352 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [fibrechannel] do not attempt to list non-existent directoriesBryn M. Reeves2018-06-201-6/+4
| | | | | | | | | | | Use a list comprehension to both generate the list of paths to collect, and to filter out any directories in the standard list of fibrechannel paths that do not exist on the running system. Resolves: #1353 Related: #1341 Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [sosreport] Fix finished running plugins reportJake Hunsaker2018-06-201-1/+1
| | | | | | | | | | | Adds the condition that no more plugins are supposed to run when there are no plugins running to report that sosreport has finished running plugins. Resolves: #1354 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com> Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [sosreport] use private plugin list for ThreadPoolExecutorBryn M. Reeves2018-06-191-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In some cases the concurrent plugins patch will produce output like the following (with a starting set of plugins of [activemq, apache, dbus, dhcp, manageiq, openshift, openstack_swift, puppet]: Starting 1/8 activemq [Running: activemq] Starting 2/8 apache [Running: apache] Starting 4/8 dhcp [Running: apache dhcp] Starting 5/8 manageiq [Running: apache dhcp manageiq] Starting 6/8 openshift [Running: apache openshift] Starting 7/8 openstack_swift [Running: apache openshift openstack_swift] Starting 8/8 puppet [Running: apache openshift puppet] Plugin #3 on the list (dbus) has vanished from the plugin run list. The ThreadPoolExecutor class uses itertools.izip() to walk the passed iterable of arguments. Since this references the same SoSReport.pluglist list that is being concurrently modified by the collect_plugin() methods it is possible for the pool iterator to "lose" an entry (another entry is removed by during the call to its collect_plugin() method, causing the list indices to shift, and making the index used in the next iteration of the pool's map() submission loop point one past the correct next entry. Avoid this problem by initialising a fresh copy of the pluglist to pass into the pool. Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [puppet] add package checksBryan Quigley2018-06-181-0/+2
| | | | | | | | | | | | | Previously this had no checks and would run for every sosreport, many of which don't have puppet installed. There have been several name changes so I included a lot of package names to try to cover all of the possible versions/distros. Resolves: #1345 Signed-off-by: Bryan Quigley <bryan.quigley@canonical.com> Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [skydive] add pluginMasco Kaliyamoorthy2018-06-181-0/+61
| | | | | | | | | | skydive is an open source real-time network topology and protocols analyzer for more info: http://skydive.network/ Resolves: #1351 Signed-off-by: Masco Kaliyamoorthy <mkaliyam@redhat.com> Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [sosreport] call traceback.print_exc without argumentsPavel Moravec2018-06-181-2/+1
| | | | | | | | | | to prevent uncaught exception on py3 also dont import already imported traceback class Resolves: #1349 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [plugins] Check read access before trying to list a dirBryan Quigley2018-06-181-1/+1
| | | | | | | | | | | I found a permission error when trying to copy /sys/fs/pstore when running on LXD containers. Adding a permission check first allows it to skip folders it can't list. Resolves: #1348 Signed-off-by: Bryan Quigley <bryan.quigley@canonical.com> Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [ipa] collect more logs and httpd alias configsPavel Moravec2018-06-181-1/+14
| | | | | | Resolves: #1346 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [ovirt_provider_ovn] fix typo in provider file/config variable namePavel Moravec2018-06-181-1/+1
| | | | | | Resolves: #1343 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [infiniband] test if dir exists before listing itPavel Moravec2018-06-181-1/+1
| | | | | | Resolves: #1342 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [fibrechannel] test if dir exists before listing itPavel Moravec2018-06-181-1/+2
| | | | | | Resolves: #1341 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [sosreport] --batch option does not display resultsPavel Moravec2018-06-181-2/+1
| | | | | | | | Fixing a regression caused by 8e69d36 . Resolves: #1337 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [archive] use threads for xz compressionPavel Moravec2018-06-184-9/+32
| | | | | | | | | | Moving also building of the command from Archive to Plugin class. Closes: #1196 Resolves: #1338 Signed-off-by: Pavel Moravec <pmoravec@redhat.com> Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [policies] add auxiliary function _get_pkg_name_for_binaryPavel Moravec2018-06-182-0/+12
| | | | | | | | | | In some cases, we need to determine what package provides given binary. This auxiliary function implements it - so far it works for and will be used by archive methods. Related to: #1196 Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
* [sosreport] Fix threads execution with no options and output formattingJake Hunsaker2018-06-181-9/+13
| | | | | | | | | | | | | | | | | | | Fixes an issue where sosreport would not run due to an unset number of threads if 'sosreport' was run bare, with no options. Any option (threads or otherwise) would mask this problem due to how the default threads value was set by the parser. Now sosreport can be run without any options and run normally, with the concurrency provided by the threads. Additionally, changes the status line printed during plugin execution to be of a fixed length, so that output lines are not mangled when the status line is updated, rather than a new line being printed (such as what happens when using --batch). Resolves: #1336 Signed-off-by: Jake Hunsaker <jhunsake@redhat.com> Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [elastic] Add Elasticsearch service pluginAmit Ghadge2018-06-181-0/+60
| | | | | | | | Closes: #1334 Resolves: #1335 Signed-off-by: Amit Ghadge <amitg.b14@gmail.com> Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [mssql] Add new pluginTakayoshi Tanaka2018-06-181-0/+92
| | | | | | | | | | | | | Adds a new plugin for Microsoft SQL Server on Linux. Collect log files. Collect journal logs. Collect keytab file info if AD auth is configured. Resolves: #1326 Signed-off-by: Takayoshi Tanaka <tatanaka@redhat.com> Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [origin] Collect {resolv,node-dnsmasq}.conf under /etc/origin/nodeKenjiro Nakayama2018-06-181-0/+2
| | | | | | | | | | | | | | | Since OCP 3.6+, there are some DNS config under /etc/origin/node/ such as `node-dnsmasq.conf` and `resolv.conf`. Although current sosreport does not gather them, these are important for troubleshooting of DNS issues. This patch updates origin.py to collect {resolv,node-dnsmasq}.conf under /etc/origin/node/. Resolves: #1311 Signed-off-by: Kenjiro Nakayama <nakayamakenjiro@gmail.com> Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [openstack_instack] add support for containerized undercloudEmilien Macchi2018-06-181-3/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 1) Do the copy in one call with variables, so it's more efficient. 2) Collect the config tarball It'll be very useful to collect the config tarball which containers TripleO Heat Templates used to deploy the undercloud and the Ansible playbooks generated by config-download as well. Both of them are archived and compressed in /home/stack/undercloud-install-*.tar.bzip2 files everytime install/upgrade commands are run. 3) Collect ephemeral Heat logs /var/log/heat-launcher is a new directory that contains the heat-installer sqlite, logs and configuration. The content is small because it's only used once and it's ephemeral (purged everytime an operator install or upgrade an undercloud). However, collecting these files will be useful to help customers. 4) Collect new logfile When containerizing the undercloud, we're moving install-undercloud.log into $HOME which makes it more accessible for the users, while we're removing instack(-undercloud) bits over time. This patch add $HOME/install-undercloud.log file and document why we're doing this. Note that we still collect old file for previous releases. Resolves: #1279 Signed-off-by: Emilien Macchi <emilien@redhat.com> Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
* [archive] make FileCacheArchive methods re-entrantBryn M. Reeves2018-06-182-74/+177
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Since plugins are now run concurrently the Archive class must be re-entrant: different plugins may attempt to create the same paths in the archive, and since they are executing in parallel and with no locking, there exists a TOCTOU race between the use of os.path.{exists,isdir, etc.}() and the subsequent creation of a new path. Address this by implementing a simple monitor-style locking scheme for the FileCacheArchive class. A `_path_lock` member is added to the class and this must be held across any operation that modifies the namespace of the archive (including file, directory, symlink, and special node creation). Additionally, the `_check_path()` Archive method (which checks for the existence of path components up to the basename of the file, creating directories as required) is extended to also carry out a test for the existence of the path, and if the path does exist, that the object is of the expected type. A mismatch in the object type generates a ValueError with an appropriate string description of the problem. Benchmarking shows a consistent small improvement with the patch: this is a result of reduced redundant copy operations that the previous archive structure allowed (since duplicate operations are now aborted as soon as the _path_lock is acquired). Resolves: #1340 Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
generated by cgit (git 2.34.1) at 2024-11-16 23:45:18 +0000