[Python-Dev] Status of PEP 3145 (original) (raw)

[Python-Dev] Status of PEP 3145 - Asynchronous I/O for subprocess.popen

Josiah Carlson josiah.carlson at gmail.com
Fri Mar 28 21:26:48 CET 2014

On Fri, Mar 28, 2014 at 12:42 PM, Terry Reedy <tjreedy at udel.edu> wrote:

On 3/28/2014 12:45 PM, Josiah Carlson wrote:

If it makes you feel any better, I spent an hour this morning building a 2-function API for Linux and Windows, both tested, not using ctypes, and not even using any part of asyncio (the Windows bits are in msvcrt and winapi). It works in Python 3.3+. You can see it here: http://pastebin.com/0LpyQtU5

Thank you. The docs gave me the impression that I could simply write proc.stdin and read proc.stdout. I failed with even a simple echo server (on Windows) and your code suggests why. So it does not get lost, I attached your code to http://bugs.python.org/issue18823 My interest is with Idle. It originally ran user code in the same process as the Shell and Editor code. Then Guido added an option to os.spawn a separate process and communicate through a socket connection and the option became the default with same process (requested by -N on the command line) as a backup option. 3.2 switched to using subprocess, but still with a socket. The problem is that the socket connection intermittently fails. Firewalls are, or at least used to be one possible cause, but there are others -- unknown. (While it works, the suggestion to restart with -N is a mystery to people who have never seen a command line.) This is one of the biggest sources of complaints about Idle. A pipe connection method that always worked on Windows, *x, and Mac would be great in itself and would also allow code simplification by removing the -n option. (Roger Serwy has suggested the latter as having two modes makes patching trickier.) The current socket connection must be non-blocking. Even though the exec loop part of the Shell window waits for a response after sending a user statement, everything else is responsive. One can select text in the window, use the menus, or switch to another window. So Idle definitely needs non-blocking write and read. In my ignorance, I have no idea whether the approach in your code or that in Viktor's code is better. Either way, I will appreciate any help you give, whether by writing, reviewing, or testing, to make communication with subprocesses easier and more dependable.

One of my other use-cases for this was using this in my editor (PyPE), which I wrote (in 2003) because I lost work in Idle. This lost work was due to the same-process interpreter crashing during an interactive session. IIRC, this is partly what pushed Guido to have Idle use os.spawn() + socket. I ended up using wxPython's built-in external process support at the time, but that's obviously not useful in core Python with Idle :P

This is all coming back full circle. :)

More information about the Python-Dev mailing list