[Python-ideas] Tulip / PEP 3156 (original) (raw)

[Python-ideas] Tulip / PEP 3156 - subprocess events

Paul Moore p.f.moore at gmail.com
Mon Jan 21 00:25:12 CET 2013

On 20 January 2013 22:53, Andrew Bennetts <andrew at bemusement.org> wrote:

Guido van Rossum wrote: […]

I have a more-or-less working but probably incomplete version checked into the tulip repo: http://code.google.com/p/tulip/source/browse/tulip/subprocesstransport.py

Note that this completely ignores stderr -- this makes the code simpler while still useful (there's plenty of useful stuff you can do without reading stderr), and avoids the questions Greg Ewing brought up about needing two transports (one for stdout, another for stderr). Although 3 pipes to a subprocess (stdin, stdout, stderr) is the usual convention, it's not the only possibility, so that configuration shouldn't be hard-coded. On POSIX some programs can and do make use of the ability to have more pipes to a subprocess; e.g. the various *fd options of gnupg (--status-fd, --logger-fd, --command-fd, and so on). And some programs give the child process file descriptors that aren't pipes, like sockets (e.g. an inetd-like server that accepts a socket then spawns a subprocess to serve it). So I hope tulip will support these possibilities (although obviously the stdin/out/err style should be the convenient default). You will be unsurprised to hear that Twisted does :)

My plan is to modify Guido's current code to take a subprocess.Popen object when creating a connection to a subprocess. So you'd use the existing API to start the process, and then tulip to interact with it. Having said that, I have no idea if or how subprocess.Popen would support the extra fds you are talking about. If you can show me some sample code, I can see what would be needed to handle it. But as far as I know, subprocess.Popen objects only have the 3 standard handles exposed as attributes - stdin, stdout and stderr.

If you have to create your own pipes and manage them yourself in "normal" code, then I would expect that you'd have to do the same with tulip. That may indicate a need for (yet another) event loop API to create a pipe which can then be used with subprocess. Or you could use the add_reader/add_writer interfaces, at the expense of portability.

Paul

PS The above is still my plan. But at the moment, every PC in my house seems to have decided to stop working, so I'm rebuilding PCs rather than doing anything useful :-( Normal service will be resumed in due course...

More information about the Python-ideas mailing list