[Python-ideas] Tulip / PEP 3156 (original) (raw)

[Python-ideas] Tulip / PEP 3156 - subprocess events

Paul Moore p.f.moore at gmail.com
Fri Jan 18 10:33:09 CET 2013

On 18 January 2013 08:38, Nick Coghlan <ncoghlan at gmail.com> wrote:

Transports are about the communications channel - network sockets - OS pipes - shared memory - CANbus - protocol tunneling

Transports should only be platform specific at the base layer where they actually need to interact with the OS through the event loop. Higher level transports should be connected to lower level protocols based on APIs provided by those transports and protocols themselves. The whole point of the protocol vs transport model is to allow you to write adaptive stacks.

Interesting. On that basis, the whole subprocess interaction scenario is not a low level transport at all (contrary to what I understood from Guido's suggestion of an event loop method) and so should be built in user code (OK, probably as a standard library helper, but definitely not as specialist methods on the event loop) layered on the low-level pipe transport.

That was my original instinct, but it fell afoul of

  1. The Windows implementation of a low level pipe transport doesn't exist (yet) and I don't know enough about IOCP to write it [1].
  2. I don't understand the programming model well enough to understand how to write a transport/protocol layer (coroutine head explosion issue).

I have now (finally!) got Guido's point that implementing a process protocol will give me a good insight into how this stuff is meant to work. I'm still struggling to understand why he thinks it needs a dedicated method on the event loop, rather than being a higher-level layer like you're suggesting, but I'm at least starting to understand what questions to ask.

Paul

[1] There is some stuff in the IOCP documentation about handles having to be opened in OVERLAPPED mode, which worries me here as it may imply that arbitrary pipes (such as the ones subprocess.Popen uses) can't be plugged in. It's a bit like setting a filehandle to nonblocking in Unix, but it has to be done at open time, IIUC. I think I saw an email about this that I need to hunt out.

More information about the Python-ideas mailing list