Issue 1236: subprocess is not thread-safe (original) (raw)

The following test program crashes:

======================================== import threading, sys, subprocess

subprocess._cleanup = lambda: None

def doit(): for i in xrange(0, 1000): p = subprocess.Popen( "true" ) p.wait()

t = threading.Thread( target=doit ) t.start() doit()

==============================

It crashes because when one thread calls subprocess.Popen(), subprocess calls this _cleanup() function, which might reap the subprocess started in another thread ! The other thread might be inside subprocess.Popen.wait(), just about to call waitpid(), and kill itself.

If you uncomment the commented line, then the program runs with no problems.

I imagine the purpose of _cleanup is to protect users from themselves, i.e., protect a user who calls subprocess.Popen() a lot without ever calling wait(). I suggest either:

(1) eliminating this _cleanup() mechanism completely; people who do not wait() deserve the zombies they get; (2) synchronizing _cleanup() with wait() through a lock; or, (3) having wait() simply retry if it gets ECHILD. On the retry, it will discover that returncode is set, and return normally.

-Ken

This is a duplicate of bug# 1731717.

I asked Donovan Baarda, who told me:

""" Last time I looked this had been fixed, admittedly in a bit of an ugly way, on the svn head. I offered to do a patch to make it a bit cleaner, but as it isn't really broken anymore it was a bit of a low priority and I haven't done it.

This bug seems to have a good repeatable test-case that we can probably use in the unittests to show that it's now fixed... """