(original) (raw)

On 16 January 2013 17:52, Guido van Rossum <guido@python.org> wrote:
\> On Wed, Jan 16, 2013 at 7:12 AM, Paul Moore <p.f.moore@gmail.com> wrote:
\>> I've so far been lurking on the tulip/async discussions, as although
\>> I'm interested, I have no specific need for writing high-performance
\>> network code.
\>>
\>> However, I hit a use case today which seems to me to be ideal for an
\>> async-style approach, and yet I don't think it's covered by the
\>> current PEP. Specifically, I am looking at monitoring a
\>> subprocess.Popen object. This is basically an IO loop, but monitoring
\>> the 3 pipes to the subprocess (well, only stdout and stderr in my
\>> case...). Something like add\_reader/add\_writer would be fine, except
\>> for the fact that (a) they are documented as low-level not for the
\>> user, and (b) they don't work in all cases (e.g. in a select-based
\>> loop on Windows).
\>>
\>> I'd like PEP 3156 to include some support for waiting on IO from (one
\>> or more) subprocesses like this in a cross-platform way. If there's
\>> something in there to do this at the moment, that's great, but it
\>> wasn't obvious to me when I looked...
\>
\> This is a great use case. The right approach would probably be to
\> define a new Transport (and an event loop method to create one) that
\> wraps pipes going into and out of a subprocess. The new method would
\> have a standard API (probably similar to that of subprocess), whereas
\> there would be different implementations of the Transport based on
\> platform and event loop implementation (similar to the way the
\> subprocess module has quite different implementations).
\>
\> Can you check out the Tulip source code (code.google.com/p/tulip) and
\> come up with a patch to do this? I'll gladly review it. It's fine to
\> only cover the UNIX case for now.

I'll have a look. There \*is\* one problem, though - I imagine it will
be relatively easy to put something together that works on Unix, as
waiting on pipes is covered by the existing select/poll mechanisms.
But I'm on Windows, so I won't be able to test it. And on Windows,
there's no mechanism in place to wait on arbitrary filehandles, so the
process wait mechanism is a much harder nut to crack. Chicken and egg
problem...

Maybe I'll start by looking at waiting on arbitrary filehandles, and

use that to build the process approach. Unfortunately, I don't think

IOCP is any more able to wait on arbitrary files than select - see my

followup to an older thread on Richard's work there. Or maybe I'll set

up a hacking environment in a Linux VM or something. That'd be a fun

experience in any case.

I'll have to get my brain round the existing spec as well. I'm finding

it hard to understand why there are so many methods on the event loop

that are specific to particular use cases (for this example, your

suggested method to create the new type of Transport).

My instinct

says that this should *also* be a good test case for a user coming up

with a new type of "event source" and wanting to plug it into the

event loop. Having to add a new method to the event loop seems to

imply this isn't possible.

OK, off to do a lot of spec reading and then some coding. With luck,
you'll be patient with dumb questions from me on the way :-)