diff options
| author | Bryn M. Reeves <bmr@redhat.com> | 2018-06-19 12:45:50 +0100 |
|---|---|---|
| committer | Bryn M. Reeves <bmr@redhat.com> | 2018-06-19 12:45:50 +0100 |
| commit | ab15a0f10c7bfa98ebf0bf619bccb60a31b4545a (patch) | |
| tree | 08c203907a897fa2f33692a5da5d0c65666890a0 | |
| parent | e51ea1d847406f3efa16ba8e78bbf02924e7b0b1 (diff) | |
| download | sos-ab15a0f10c7bfa98ebf0bf619bccb60a31b4545a.tar.gz | |
[sosreport] use private plugin list for ThreadPoolExecutor
In some cases the concurrent plugins patch will produce output
like the following (with a starting set of plugins of [activemq,
apache, dbus, dhcp, manageiq, openshift, openstack_swift,
puppet]:
Starting 1/8 activemq [Running: activemq]
Starting 2/8 apache [Running: apache]
Starting 4/8 dhcp [Running: apache dhcp]
Starting 5/8 manageiq [Running: apache dhcp manageiq]
Starting 6/8 openshift [Running: apache openshift]
Starting 7/8 openstack_swift [Running: apache openshift openstack_swift]
Starting 8/8 puppet [Running: apache openshift puppet]
Plugin #3 on the list (dbus) has vanished from the plugin run
list.
The ThreadPoolExecutor class uses itertools.izip() to walk the
passed iterable of arguments. Since this references the same
SoSReport.pluglist list that is being concurrently modified by
the collect_plugin() methods it is possible for the pool iterator
to "lose" an entry (another entry is removed by during the call to
its collect_plugin() method, causing the list indices to shift,
and making the index used in the next iteration of the pool's
map() submission loop point one past the correct next entry.
Avoid this problem by initialising a fresh copy of the pluglist
to pass into the pool.
Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
| -rw-r--r-- | sos/sosreport.py | 4 |
1 files changed, 3 insertions, 1 deletions
diff --git a/sos/sosreport.py b/sos/sosreport.py index 42ff8938..4ef9abcb 100644 --- a/sos/sosreport.py +++ b/sos/sosreport.py @@ -1287,7 +1287,9 @@ class SoSReport(object): self.pluglist.append((plugruncount, i[0])) try: self.plugpool = ThreadPoolExecutor(self.opts.threads) - self.plugpool.map(self._collect_plugin, self.pluglist, chunksize=1) + # Pass the plugpool its own private copy of self.pluglist + self.plugpool.map(self._collect_plugin, list(self.pluglist), + chunksize=1) self.plugpool.shutdown(wait=True) self.ui_log.info("") except KeyboardInterrupt: |