Issue 5155: Multiprocessing.Queue created by sub-process fails when used in sub-sub-process ("bad file descriptor" in q.get()) (original) (raw)

(Note: This issue only happens on linux - on Windows things work fine.)

Please see the attached script.

The script creates a process (TestProcess()). That process runs and creates a sub-process (TestSubProcess()). When TestSubProcess tries to access a Queue created by TestProcess, the .get() call fails with the following stack trace:

Process Process-1:1: Traceback (most recent call last): File "/opt/python2.6/lib/python2.6/multiprocessing/process.py", line 231, in _bootstrap self.run() File "/opt/python2.6/lib/python2.6/multiprocessing/process.py", line 88, in run self._target(*self._args, **self._kwargs) File "queue_test.py", line 7, in TestSubProcess item = q.get() # Fails with IOError if queue created in TestProcess File "/opt/python2.6/lib/python2.6/multiprocessing/queues.py", line 91, in get res = self._recv() IOError: [Errno 9] Bad file descriptor

If the same queue is created by main (top level), TestSubProcess accesses it with no errors. This behavior can be stimulated by uncommenting the q = multiprocessing.Queue() line in TestProcess.

Tested using: Python 2.6.1 (r261:67517, Dec 4 2008, 16:51:00) [MSC v.1500 32 bit (Intel)] on win32 (Installed using binary installer, windows XPSP2)

No problems

Python 2.6.1 (r261:67515, Jan 6 2009, 15:23:19) [GCC 4.2.4 (Ubuntu 4.2.4-1ubuntu3)] on linux2 (Compiled from source due to ubuntu 8.04 (Hardy) not having python2.6 yet)

Fails on q.get() in TestSubProcess

Quick note: Have tracked this down to what appears to be buggy behaviour on the part of os.pipe() when called from within a Process.

First invocation of os.pipe() in a Process returns (0,x) - stdin(?) resulting in the 'bad file descriptor' error.

Interactive prompt test follows:

import multiprocessing as MP, os def r(): print os.pipe(), os.pipe(), os.pipe() ... r() (9, 10) (11, 12) (13, 14) MP.Process(target=r).start() (0, 15) (16, 17) (18, 19) MP.Process(target=r).start() (0, 15) (16, 17) (18, 19)