[Python-Dev] PEP 324: popen5 - New POSIX process module (original) (raw)

Guido van Rossum guido at python.org
Mon Jan 5 10:12:37 EST 2004

Peter,

After seeing the discussion that your PEP has sparked (without reading all of it) I am torn. I agree that Python's subprocess management facilities can use a lot of work. I agree that your code is by and large an improvement over popen2 and friends. But I'm not sure that it is enough of an improvement to be accepted into the standard library.

What I'm looking for in standard library additions is "category-killers". IMO good recent examples are optparse and the logging module. Both were in active use and had independent distributions with happy users before they were accepted into the standard library.

Note that in my definition, a category-killer doesn't have to have all conceivable features -- it has to strike the right balance between feature completeness and a set of properties that are usually referred to by words like elegance, simplicity, ease-of-use, ease-of-learning. For example, optparse specifically does not support certain styles of option parsing -- it strives to encourage uniformity in option syntax, and that rules out certain features.

(I'm not saying that your popen5 has too many features -- I'm just warning that I'm not asking you to make it a category-killer by adding tons more features. I don't want more features, I want the satisfying feeling that this is the "best" solution, for my very personal definition of "best".)

I note that your source code at

http://cvs.lysator.liu.se/viewcvs/viewcvs.cgi/popen5/?cvsroot=python-popen5

has some useful comments that were missing from the PEP, e.g. how to replace various older APIs with calls to popen5. It also has some motivation for the API choices you made that are missing from the PEP (e.g. why there are options for setting cwd).

So what's missing? For the PEP, I would say that a lot of explanatory and motivational text from the source code comments and doc strings should be added. (PEPs don't have to be terse formal documents that describe the feature in as few words as possible; that can be one section of the PEP, but in general a PEP needs to provide motivation and insight as well as specification.)

For the code, I think that a Windows version is essential. It would be okay if a small amount of code (e.g. making some basic Windows APIs available, like CreateProcess) had to be coded in C, as long as the bulk could still be in Python. I really like having this kind of code in Python -- it helps understanding the fine details, and it allows subclassing more easily.

I also wonder if more support for managing a whole flock of subprocesses might not be a useful addition; this is a common need in some applications. And with this, I'd like to see explicit support (somehow -- maybe through a separate class) for managing "daemon" processes. This certainly is a common need!

I would like this to replace all other "high-level" subprocess management facilities, in particular os.system() and os.popen() and the popen2 module, on all platforms. Oh, and also os.spawn*(). But not os.fork() and os.exec*(), since those are the building blocks (on Unix, anyway).

--Guido van Rossum (home page: http://www.python.org/~guido/)

More information about the Python-Dev mailing list