Multiprocessing pool.py gets SimpleQueue objects as inqueue and outqueue. when it terminates, it doesn't call the close() method of the queues' readers and writers. As a results, 4 file pipes leak in one pool termination. Expected: The pool closes reader and writer pipes of the inqueue and outqueue when it terminates. What did happen: the pool doesn't close the pipes. 4 pipes leak.
I did some investigating using a test script and Python 3.7.0a0 from multiprocessing import Pool import os import time def f(x): time.sleep(30) return x*x if __name__=='__main__': print('Main pid {0}'.format(os.getpid())) p = Pool(5) p.map(f, [1,2,3]) print('Returned') time.sleep(30) and grepping for pipe and the parentpid in the output from lsof ( lsof | grep python.*.*pipe ). The pipes opened at the start of the script are still open even after the line print('Returned') is executed. I suppose this is expected because I did not call *p.close()*. All pipes are cleaned up after the parent process finishes. When I repeat the experiment calling p.close() after p.map returns, all that is left is the 9 pipes opened by the parent. All pipes are cleaned up after parent script exits. @shani - could you please clarify how you were able to detect the leaking pipes?
I think this issue is mistaken. The reader and writer objects are closed automatically when they are destroyed (see Connection.__del__). The only thing that may lack is a way to close them more eagerly. In any case, I'm closing as a duplicate of issue 30966.
