Issue 3272: Multiprocessing hangs when multiprocessing.Pool methods are called (original) (raw)

multiprocessing hangs, when multiprocessing.Pool methods map, imap, imap_unordered, apply, apply_async, map_async are called. Here is an example:

Python 3.0b1+ (py3k:64686M, Jul 3 2008, 13:06:13) [GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2 Type "help", "copyright", "credits" or "license" for more information.

from multiprocessing import Pool Pool(processes=4).imap(lambda x: x, range(10)) <multiprocessing.pool.IMapIterator object at 0x83bcc8c> Pool(processes=4).map(lambda x: x, range(10)) Exception in thread Thread-13: Traceback (most recent call last): File "/usr/local/lib/python3.0/threading.py", line 492, in _bootstrap_inner self.run() File "/usr/local/lib/python3.0/threading.py", line 447, in run self._target(*self._args, **self._kwargs) File "/usr/local/lib/python3.0/multiprocessing/pool.py", line 225, in _handle_tasks put(task) File "/usr/local/lib/python3.0/pickle.py", line 1319, in dumps Pickler(f, protocol).dump(obj) File "/usr/local/lib/python3.0/multiprocessing/util.py", line 311, in _reduce_method if m.self is None: AttributeError: 'function' object has no attribute 'self'

Process PoolWorker-28: Traceback (most recent call last): File "/usr/local/lib/python3.0/multiprocessing/process.py", line 232, in _bootstrap self.run() File "/usr/local/lib/python3.0/multiprocessing/process.py", line 88, in run self._target(*self._args, **self._kwargs) File "/usr/local/lib/python3.0/multiprocessing/pool.py", line 57, in worker task = get() File "/usr/local/lib/python3.0/multiprocessing/queues.py", line 337, in get Process PoolWorker-27: Traceback (most recent call last): File "/usr/local/lib/python3.0/multiprocessing/process.py", line 232, in _bootstrap self.run() File "/usr/local/lib/python3.0/multiprocessing/process.py", line 88, in run self._target(*self._args, **self._kwargs) File "/usr/local/lib/python3.0/multiprocessing/pool.py", line 57, in worker task = get() File "/usr/local/lib/python3.0/multiprocessing/queues.py", line 337, in get Process PoolWorker-26: Traceback (most recent call last): File "/usr/local/lib/python3.0/multiprocessing/process.py", line 232, in _bootstrap Process PoolWorker-25: Traceback (most recent call last): File "/usr/local/lib/python3.0/multiprocessing/process.py", line 232, in _bootstrap self.run() File "/usr/local/lib/python3.0/multiprocessing/process.py", line 88, in run self._target(*self._args, **self._kwargs) File "/usr/local/lib/python3.0/multiprocessing/pool.py", line 57, in worker task = get() File "/usr/local/lib/python3.0/multiprocessing/queues.py", line 337, in get self.run() File "/usr/local/lib/python3.0/multiprocessing/process.py", line 88, in run self._target(*self._args, **self._kwargs) File "/usr/local/lib/python3.0/multiprocessing/pool.py", line 57, in worker task = get() File "/usr/local/lib/python3.0/multiprocessing/queues.py", line 339, in get return recv() KeyboardInterrupt racquire() KeyboardInterrupt racquire() KeyboardInterrupt racquire() KeyboardInterrupt Terminated

The same happens on latest trunk of Python 2.6. Note, map and apply methods hang Python interpreter completely, so I have to use kill utility to exit Python.