[Python-Dev] PEP 324: popen5 - New POSIX process module (original) (raw)

Martin v. Loewis martin at v.loewis.de
Sat Jan 3 19:14:52 EST 2004

Peter Astrand wrote:

I don't agree. I have used all of the existing mechanism in lots of apps, and it's just a pain. There are lots of functions to choose between, but none does what you really want.

So enhance them, instead of replacing them.

- Cross-process exceptions: Exceptions happening in the child before the new process has started to execute are re-raised in the parent.

This is a bug in popen2, IMO. Fixing it is a good thing, but does not require a new module. "Fixing popen2" would mean a break old applications; exceptions will happen, which apps are not prepared of.

I find that an acceptable incompatibility, and it will likely break no existing application. Applications usually expect that the program they start actually exists; it is a good thing that they now can detect the error that the missing/non-executable application.

Errors should never pass silently.

- A hook for executing custom code between fork and exec. This can be used for, for example, changing uid.

Such a hook could be merged as a keyword argument into the existing API. Into which module/method/function?

For example, popen2.popen2, as argument preexec_fn.

There is no one flexible enough. The case for redirecting only stderr is just one example; this is simple not possible with the current API.

Can you elaborate? What is the specific problem, how does your preexec function look like, and how is it used with popen5. I can then show you how it could be used with popen2, if that was enhanced appropriately.

- All combinations of file descriptor redirection is possible. For example, the "python-dialog" [2] needs to spawn a process and redirect stderr, but not stdout. This is not possible with current functions, without using temporary files.

Sounds like a new function on the popen2 module. To support all combinations, 12 different functions are necessary. Who will remember what popen2.popen11() means?

Why is that? Just add a single function, with arguments stdin/stdout/stderr. No need for 12 functions. Then explain the existing functions in terms of your new function (if possible).

- Support for connecting several subprocesses (shell "pipe").

Isn't this available already, as the shell supports pipe creation, anyway? With popen5, you can do it without using the shell.

Why is that a good thing?

There's already a bug about this; bug 788035. This is what one of the comment says:

"But this whole popen{,2,3,4} section of posixmodule.c is so fiendishly complicated with all the platform special cases that I'm loath to touch it..." I haven't checked if this is really true, though.

You really should work with the existing code base. Ignoring it is a guarantee that your PEP will be rejected. (Studying it, and then providing educated comments about it, might get you through)

I think this is the core problem of your approach: You throw away all past history, and imply that you can do better than all prior contributors could. Honestly, this is doubtful. The current code is so complicated because implementing pipes is complicated.

Well, I don't see how this could be done easily: The current API is not flexible enough, and some things (like cross-process exceptions) breaks compatibility.

I never said it would be easy. However, introducing a new popen module is a major change, and there must be strong indications that the current API cannot be enhanced before throwing it away.

There should be one-- and preferably only one --obvious way to do it.

As for breaking compatibility: This is what the PEP should study in detail. It is sometimes acceptable to break compatibility, if applications are likely to be improved by the change. Any change can, in principle, break compatibility. Suppose I had an application that did

from popen5 import open

This application might break if your proposed change is implemented, as a new module is added. So you can't claim "I will break no programs".

Writing a good popen module is hard. Providing cross-platform support (for Windows, for example) is even harder. Trying to retrofit a good popen implementation into an old API without breaking compatibility seems impossible to me. I'm not prepared to try.

So I continue to be -1 with your PEP.

Regards, Martin

More information about the Python-Dev mailing list