Issue 28287: Refactor subprocess.Popen to let a subclass handle IO asynchronously (original) (raw)

Created on 2016-09-27 14:26 by martius, last changed 2022-04-11 14:58 by admin.

Files
File name Uploaded Description Edit
popen_execute_child_refactoring.patch martius,2016-09-27 14:26 popen_execut_child_refactoring.patch, should work with python 3.4+ review
Pull Requests
URL Status Linked Edit
PR 6878 closed python-dev,2018-05-15 21:06
Messages (4)
msg277517 - (view) Author: Martin Richard (martius) * Date: 2016-09-27 14:26
Hi, Currently, subprocess.Popen performs blocking IO in its constructor (at least on Unix): it reads on a pipe in order to detect outcome of the pre-exec and exec phase in the new child. There is no way yet to modify this behavior as this blocking call is part of a long Popen._execute_child() method. This is a problem in asyncio (asyncio.subprocess_exec and asyncio.subprocess_shell). I would like to submit a patch which breaks Popen.__init__() and Popen._execute_child() in several methods so it becomes possible to avoid blocking calls (read on pipe and waitpid) by overriding a few private methods without duplicating too much code. The goal is to use it in asyncio, as described in this pull request (which currently monkey-patches Popen): https://github.com/python/asyncio/pull/428 This patch only targets the unix implementation. Thanks for your feedback.
msg298358 - (view) Author: Giampaolo Rodola' (giampaolo.rodola) * (Python committer) Date: 2017-07-14 16:09
Is the main goal to just make Popen.__init__ non-blocking? If not, #1191964 aims at providing non-blocking reads/writes (including on Windows).
msg300839 - (view) Author: Martin Richard (martius) * Date: 2017-08-25 14:01
Yes, the goal is to isolate the blocking IO in __init__ into other methods so Popen can be subclassed in asyncio. The end goal is to ensure that when asyncio calls Popen(), it doesn't block the process. In the context of asyncio, there's no need to make Popen() IOs non-blocking as they will be performed with the asyncio API (rather than the IO methods provided by the Popen object).
msg301024 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2017-08-30 12:35
By the way, we're using GitHub PRs now for development, though you can still submit patch files if you prefer that.
History
Date User Action Args
2022-04-11 14:58:37 admin set github: 72474
2018-05-15 21:06:57 python-dev set pull_requests: + <pull%5Frequest6550>
2018-05-14 19:06:36 gregory.p.smith set versions: + Python 3.8, - Python 3.7
2017-08-30 21:09:21 gregory.p.smith set assignee: gregory.p.smith
2017-08-30 12:35:13 pitrou set nosy: + pitroumessages: +
2017-08-30 12:34:23 pitrou set nosy: + gregory.p.smith
2017-08-30 11:48:06 pitrou set stage: patch review
2017-08-25 14:01:26 martius set messages: +
2017-07-14 16:09:07 giampaolo.rodola set nosy: + giampaolo.rodolamessages: +
2016-09-30 18:44:38 terry.reedy set versions: + Python 3.7, - Python 3.6
2016-09-30 18:43:59 terry.reedy set nosy: + yselivanov
2016-09-27 14:26:43 martius create