aboutsummaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* Disable archive recursion tests for TarFileArchiveBryn M. Reeves2012-12-181-10/+14
| | | | | | The TarFileArchive class currently doesn't allow recursive directory addition using the add_file() interface - disable those tests for now.
* Fix import statement in archives_test.pyBryn M. Reeves2012-12-181-1/+1
|
* Separate archive classes from utilities moduleBryn M. Reeves2012-12-185-244/+276
| | | | | | | | | | | | | | | This is the start of the archive reworking to address Issue #86. In this commit the Archive class hierarchy is separated from the generic code in the utilities module and moved into its own 'archive' module. This is taken from the changes committed to Jesse's directory_archive branch. The next step is to decide how we will handle the problems discovered with the in-line tarfile generation (and similar problems affecting Zip files) and whether to revert to building an in-filesystem tree (as Jesse's current branch does) or whether to fix the problems with in-lining post-processing hooks.
* Rename command output collection interfacesBryn M. Reeves2012-12-1764-360/+360
| | | | | | | | | | | The method names for collecting external output are ugly and awkward to type. Rename them to be more consistent with the file collection API and easier to remember and type. This change touches virtually every plugin class but is a very simple search/replace (plugin changes were done with sed). Fixes Issue #92.
* Merge pull request #94 from battlemidget/featureJesse Jaggars2012-12-141-1/+1
|\ | | | | Make Ubuntu/Debian depend on python-selinux (Closed GH:#93)
| * Make Ubuntu/Debian depend on python-selinux (Closed GH:#93)Adam Stokes2012-12-141-1/+1
| | | | | | | | Signed-off-by: Adam Stokes <hackr@cypherbook.com>
* | Add output of ntptime to ntp moduleBryn M. Reeves2012-12-131-1/+4
|/
* Delete vestigial coloured output codeBryn M. Reeves2012-12-132-11/+3
| | | | | | Colour output was removed from sos a long time ago. Kill off the remaining opts variable, command line option and documentation references for increased sanity.
* Remove sar file root symlinksBryn M. Reeves2012-12-131-2/+1
|
* Fix sar file collection and error reportingBryn M. Reeves2012-12-131-3/+5
|
* Fix typo in sar moduleBryn M. Reeves2012-12-131-1/+1
|
* Fix sar plugin error messageBryn M. Reeves2012-12-131-1/+1
|
* Fix exception when unreadable files are collectedBryn M. Reeves2012-12-131-3/+8
| | | | | | | | Files in /proc/, /sys/ and other kernel file systems may have read permissions but not implement a read() syscall. Catch the exception that occurs when accessing these files. Fixes Issue #87
* Fix relative paths in reportsBryn M. Reeves2012-12-132-5/+11
| | | | | | | Fix the relative paths used by the legacy HTML reporting methods and passed by sosreport to the new Report class. Fixes Issue #88
* Change collection of /lib/systemdBryn M. Reeves2012-12-131-1/+6
| | | | | | | Since /lib/systemd contains binaries don't collect the whole thing. Grab /lib/systemd/sysem and /lib/systemd/user as these contain unit files and symlinks but just take a directory listing for /lib/systemd itself and the other subdirectories.
* Teach pam module about multilibBryn M. Reeves2012-12-131-1/+1
| | | | | The pam module collects a list of files in /lib/modules which may be empty on 64-bit systems. Collect both possible paths.
* Do not allow duplicate destinations in TarFileArchive.add_file()Bryn M. Reeves2012-12-121-0/+2
| | | | | | | | | | | | | The logic to ensure we have a proper tar record for each directory in the path of files collected with addCopySpec*() ends up generating lots of duplicate calls to add_file(). Filter them out by checking for the existence of the new destination in tarfile.getnames(). This reduces peak resident memory consumption by 50-75% and cuts run times by 50-66%. Related to Issue #81
* Add profiling to command and file substitution methodsBryn M. Reeves2012-12-121-7/+21
| | | | | | | | | | | | | | | | | | | | | Add profiling calls around the regex substitution methods. Shows that substitution costs are considerable compared to copy: copied: /root/anaconda-ks.cfg time: 0.002293 copied: /var/log/anaconda/anaconda.log time: 0.002204 copied: /var/log/anaconda/syslog time: 0.002709 copied: /var/log/anaconda/anaconda.packaging.log time: 0.002658 copied: /var/log/anaconda/ks-script-YGuewK.log time: 0.002483 copied: /var/log/anaconda/anaconda.program.log time: 0.002416 copied: /var/log/anaconda/anaconda.storage.log time: 0.002263 copied: /var/log/anaconda/anaconda.xlog time: 0.002299 copied: /var/log/anaconda/anaconda.ifcfg.log time: 0.002461 copied: /var/log/anaconda/ks-script-kdAaa4.log time: 0.002558 subst : /root/anaconda-ks.cfg time: 0.021056 I.e. subsituting one file is on average eight times more costly than simple collection.
* Work around non-writable directories in host file systemsBryn M. Reeves2012-12-121-7/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Distributions that have moved to reduced capabilities and replaced suid and sgid binaries with fscaps have tightened the permissions on several standard system paths. For e.g.: dr-xr-xr-x. 18 root root 4096 Nov 23 19:04 / -rw-r--r--. 1 root root 1174 Dec 11 18:05 /etc/passwd ----------. 1 root root 742 Dec 11 18:05 /etc/shadow dr-xr-x---. 8 root root 4096 Dec 12 19:06 /root Processes that need to write to these paths on the host system must possess cap_dac_override in order to work but in the archive this creates problems when unpacking the archive without this capability. For files this is not a problem since the user only requires write permissions to the containing directory to remove the file. For directories it causes real problems for unprivileged users working with sosreport archives. This includes problems unpacking the archive (since directories are created without write permissions causing subsequent file creation beneath that path to fail) as well as problems cleaning up archives as a typical 'rm -rf' fails to remove these paths. These problems make it impossible to create archives that are both failthful to the host system and easy to work with for typical analysis users. Ultimately this may necessitate dropping permission preservation in the archive and instead storing these (and other information like ACLs and SELinux context) in files within the archive instead). Works around Issue #85
* Add logging to regex subsitution methodsBryn M. Reeves2012-12-121-0/+9
| | | | | | | | | | | | | | Currently we don't log anything when applying file or command output regex substitutions and any errors are silently discarded. This could cause a user to think that passwords etc. have been obscured when in fact they have not. Log our intent to substitute at debug level and log any exception at error level along with the path or glob, module and exception text. Fixes Issue #84
* Re-work sunrpc module to make porting easierBryn M. Reeves2012-12-121-4/+28
| | | | | | | | | | | | | | Re-organise the sunrpc module so that the service check is in the base (Plugin-derived) class but uses a service string defined only in the subclasses. This allows the same checkenabled() to be used for any policy that implements the needed runlevelByService() API. This can serve as a prototype for a generic enabled-by-service facility (mimicking existing files and packages checks) although it currently needs considerable work to review and revise the service and runlevel interfaces in the policy modules. Related to Issue #77
* Use paths list in anaconda pluginBryn M. Reeves2012-12-111-3/+1
| | | | | The previous commit introduced a paths list for the module to collect but did not use it: fix the addCopySpecs call.
* Update anaconda supportBryn M. Reeves2012-12-111-7/+18
| | | | | Update the anaconda module to support new log locations in recent versions.
* Fix regression TarFileArchive regressionBryn M. Reeves2012-12-111-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Commit 179d9bb introduced a regression in the TarFileArchive class: (Pdb) bt /usr/sbin/sosreport(23)<module>() -> main(sys.argv[1:]) /usr/lib/python2.7/site-packages/sos/sosreport.py(908)main() -> sos.execute() /usr/lib/python2.7/site-packages/sos/sosreport.py(901)execute() -> return self.final_work() /usr/lib/python2.7/site-packages/sos/sosreport.py(773)final_work() -> self._finish_logging() /usr/lib/python2.7/site-packages/sos/sosreport.py(376)_finish_logging() -> self.archive.add_file(self.sos_log_file.name, dest=os.path.join('sos_logs', 'sos.log')) /usr/lib/python2.7/site-packages/sos/utilities.py(280)add_file() -> self.tarfile.addfile(tar_info, fileobj) /usr/lib64/python2.7/tarfile.py(2015)addfile() -> buf = tarinfo.tobuf(self.format, self.encoding, self.errors) /usr/lib64/python2.7/tarfile.py(996)tobuf() -> return self.create_gnu_header(info) /usr/lib64/python2.7/tarfile.py(1027)create_gnu_header() -> return buf + self._create_header(info, GNU_FORMAT) /usr/lib64/python2.7/tarfile.py(1112)_create_header() -> itn(info.get("mtime", 0), 12, format), > /usr/lib64/python2.7/tarfile.py(212)itn() -> raise ValueError("overflow in number field") The tarinfo mtime field is a float but the pax headers take a string-encoded value. The tarinfo field was inadvertently converted to a formatted string. The problem is hard to track down because it's not always triggered; the bug depends on the value of the string-encoded mtime date.
* Collect file lists for /var/libBryn M. Reeves2012-12-111-0/+2
| | | | | Collect a list of files and directories under /var/lib using ls -lR and tree (if available).
* Collect additional files and command output for infinibandBryn M. Reeves2012-12-111-2/+9
| | | | | Collect output of ibstat, ibstatus, ibhosts and the opensm and openib configuration files.
* Add mkqdisk -L output to cluster moduleBryn M. Reeves2012-12-111-0/+1
|
* Add /sys/firmware/acpi/tables to hardware moduleBryn M. Reeves2012-12-111-0/+1
| | | | Collect the ACPI tables present on the host system
* Add SysV IPC moduleBryn M. Reeves2012-12-111-0/+29
| | | | | Add a module to collect SysV IPC related information. Collect the files in /proc/sysvipc/* as well as the traditional ipcs -a output.
* Add more commands to iscsi initiator moduleBryn M. Reeves2012-12-111-1/+7
| | | | | Add additional iscsiadm command collection to the iscsi module and check for the iscsi-initiator-utils package on Red Hat systems.
* Remove analyze() wiringBryn M. Reeves2012-12-114-16/+3
| | | | | | | The analyze() methods haven't been used since commit 6ea48cb and their functonality has now been moved or removed in all modules. Delete the last bits of wiring and docs references.
* Remove diagnose() wiringBryn M. Reeves2012-12-113-68/+1
| | | | | | Complete the removal of the diagnose functionality by removing the methods in the Plugin class and sosreport module, command line options and documentation.
* Remove analyze() method from yum moduleBryn M. Reeves2012-12-111-9/+0
| | | | | The yum module's analyze method is pretty useless. It just checks for extra /etc/yum.repos.d files on RHEL systems. Delete it.
* Delete kernel analyze() method and add more sysfs module filesBryn M. Reeves2012-12-111-41/+3
| | | | | | | | | The analyze methods are no longer used and the kernel module's regex-table based taint checking is kinda klunky and a maintenance problem. Delete this code but collect the initstate, refcnt and taint files from /sys/modules/*.
* Delete selinux module's analyze() methodBryn M. Reeves2012-12-111-8/+9
| | | | | | | | | | | | The analyze machinery hasn't been hooked up since commit 6ea48cb. The use of these methods has also not been especially clear: some modules trigger additional file/command collection here while others only add alerts (the original purpose of analyze()). Other module call the addAlert() interface from their setup() method. Move the SELinux module's analyze functionality into the main setup() method but make it conditional on setroubleshoot being installed as it's necessary for the current implementation.
* Remove diagnose() methods from pluginsBryn M. Reeves2012-12-114-57/+0
| | | | | | Delete the diagnose() methods from all modules still using it. The diagnostics feature never worked well and is better implemented to run after-the-fact on the data collected in the tarball.
* Do not attempt to store SELinux context for /proc and /sysBryn M. Reeves2012-12-101-3/+7
| | | | | | | | | | | | | | | | | | | | | | | | | Storing a (correct) SELinux context for files in procfs and sysfs leads to tonnes of ugly spew when the tarball is unpacked as these contexts are not permitted on "normal" file systems: tar: sosreport-rhel7-vm1-20121210145629/proc/sys/vm: Cannot setfilecon: Permission denied tar: sosreport-rhel7-vm1-20121210145629/proc/sys/vm/percpu_pagelist_fraction: Cannot setfilecon: Permission denied tar: sosreport-rhel7-vm1-20121210145629/proc/sys/vm: Cannot setfilecon: Permission denied tar: sosreport-rhel7-vm1-20121210145629/proc/sys/vm/scan_unevictable_pages: Cannot setfilecon: Permission denied Etc. Check for these path prefixes in TarFileArchive.add_file() and skip generating a pax header for them. This isn't a perfect fix; it may be better to move this up to a higer layer (i.e. within the Plugin class's file handling routines) and also to filter by context rather than path (since SELinux contexts are a property of an inode not a path and a user could mount these file systems at other locations). This partially fixes Issue #79 but I'll keep it open for the time being to track these improvements.
* Fix typo in systemd moduleBryn M. Reeves2012-12-101-4/+2
| | | | | Forgot to refresh the copy of the module in HEAD; pickup two additional fixes added in testing.
* Add systemd moduleBryn M. Reeves2012-12-101-0/+46
| | | | | | | | | Add a new module to collect systemd information. This is the start of a general re-organisation of the init related information captured by sos. Initially collect systemctl and journalctl command output as well as (some) configuration files owned by the package.
* Move sysctl collection to kernel module and add new locationsBryn M. Reeves2012-12-102-1/+3
| | | | | | | The sysctl configuration and state belongs to the kernel. Reflect this by moving the collection of sysctl.conf to the kernel module (where sysctl -a is already captured) and add collection of the /etc/sysctl.d and /lib/sysctl.d directories.
* Remove obsolete sysreport references from sos.specBryn M. Reeves2012-12-071-2/+0
|
* Obscure password in /etc/keystone/keystone.confBryn M. Reeves2012-12-071-0/+5
|
* Make sar module more robustBryn M. Reeves2012-12-071-7/+27
| | | | | Make the sar module more tolerant of broken environments and users force-enabling the module when its own enabled check fails.
* Remove stray debug code in copy_permissions_filter()Bryn M. Reeves2012-12-071-1/+0
|
* Make TarArchive preserve permissions and SELinux contextBryn M. Reeves2012-12-071-18/+53
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Make the TarArchive class preserve permissions and SELinux context on all files and directories it collects. This requires us to recursively add the parent directories of any files copied in by full path (or the directory would only be implicitly created and so receive default ownership and permissions and no SELinux context). To make this work, don't call tarfile.add() to recursively add directories - the doCopyFileOrDir machinery in the Plugin class already handles directory recursion properly. Instead when a directory is passed to add_file(), just create a TarInfo record of type tarfile.DIRTYPE and propagate the permissions from the host file system. The SELinux contexts have the unfortunate side-effect of spewing errors if unpacked without --no-selinux as some contexts cannot be placed on some file systems. It may be more useful to eventually store the contexts in the in a file rather than directly in the tarball. Before: drwxr-xr-x. 9 root root 640 Dec 7 00:57 proc lrwxrwxrwx. 1 root root 30 Dec 7 00:57 ps -> sos_commands/process/ps_auxwww drwxr-xr-x. 2 root root 60 Dec 7 00:57 root drwxr-xr-x. 2 root root 80 Dec 7 00:57 sbin After: dr-xr-xr-x. 9 root root 640 Dec 5 19:07 proc lrwxrwxrwx. 1 root root 30 Dec 7 00:58 ps -> sos_commands/process/ps_auxwww dr-xr-x---. 2 root root 60 Dec 7 00:48 root dr-xr-xr-x. 2 root root 80 Dec 7 00:57 sbin More fixes issue #76
* Fix typo in i18n moduleBryn M. Reeves2012-12-061-1/+1
|
* Add /etc/localtime to general moduleBryn M. Reeves2012-12-061-0/+1
|
* Add /etc/locale.conf to i18n moduleBryn M. Reeves2012-12-061-1/+1
|
* Fix exception when sar module enabled on system without sysstatBryn M. Reeves2012-12-061-3/+5
| | | | | The sar module assumes that /var/log/sa exists when it is enabled. Check for its presence and exit from setup early.
* Preserve permissions and ownership for files stored as stringsBryn M. Reeves2012-12-061-1/+11
| | | | | | | | | | Files that are stored as strings bypass the permissions code added in commit 74c1476 since they are added to the archive via add_string(). Add a check to see if a destination exists in the host file system. If it does propagate the permissions to the stored string's tar record.
generated by cgit (git 2.34.1) at 2024-11-11 23:15:00 +0000