bpo-35621: Support running subprocesses in asyncio when loop is execu… · python/cpython@0d671c0 (original) (raw)
`@@ -117,6 +117,7 @@ asyncio ships with the following built-in policies:
`
117
117
``
118
118
` .. availability:: Windows.
`
119
119
``
``
120
`+
.. _asyncio-watchers:
`
120
121
``
121
122
`Process Watchers
`
122
123
`================
`
`@@ -129,10 +130,11 @@ In asyncio, child processes are created with
`
129
130
`` :func:create_subprocess_exec and :meth:loop.subprocess_exec
``
130
131
`functions.
`
131
132
``
132
``
`` -
asyncio defines the :class:AbstractChildWatcher abstract base class,
``
133
``
`-
which child watchers should implement, and has two different
`
134
``
`` -
implementations: :class:SafeChildWatcher (configured to be used
``
135
``
`` -
by default) and :class:FastChildWatcher.
``
``
133
`` +
asyncio defines the :class:AbstractChildWatcher abstract base class, which child
``
``
134
`+
watchers should implement, and has four different implementations:
`
``
135
`` +
:class:ThreadedChildWatcher (configured to be used by default),
``
``
136
`` +
:class:MultiLoopChildWatcher, :class:SafeChildWatcher, and
``
``
137
`` +
:class:FastChildWatcher.
``
136
138
``
137
139
`` See also the :ref:Subprocess and Threads <asyncio-subprocess-threads>
``
138
140
`section.
`
`@@ -184,23 +186,64 @@ implementation used by the asyncio event loop:
`
184
186
``
185
187
``` Note: loop may be None.
`186`
`188`
``
``
`189`
`+
.. method:: is_active()
`
``
`190`
`+`
``
`191`
``` +
Return ``True`` if the watcher is ready to use.
``
192
+
``
193
`+
Spawning a subprocess with inactive current child watcher raises
`
``
194
`` +
:exc:RuntimeError.
``
``
195
+
``
196
`+
.. versionadded:: 3.8
`
``
197
+
187
198
` .. method:: close()
`
188
199
``
189
200
` Close the watcher.
`
190
201
``
191
202
` This method has to be called to ensure that underlying
`
192
203
` resources are cleaned-up.
`
193
204
``
194
``
`-
.. class:: SafeChildWatcher
`
``
205
`+
.. class:: ThreadedChildWatcher
`
``
206
+
``
207
`+
This implementation starts a new waiting thread for every subprocess spawn.
`
``
208
+
``
209
`+
It works reliably even when the asyncio event loop is run in a non-main OS thread.
`
``
210
+
``
211
`+
There is no noticeable overhead when handling a big number of children (O(1) each
`
``
212
`+
time a child terminates), but stating a thread per process requires extra memory.
`
``
213
+
``
214
`+
This watcher is used by default.
`
``
215
+
``
216
`+
.. versionadded:: 3.8
`
195
217
``
196
``
`-
This implementation avoids disrupting other code spawning processes
`
``
218
`+
.. class:: MultiLoopChildWatcher
`
``
219
+
``
220
`` +
This implementation registers a :py:data:SIGCHLD signal handler on
``
``
221
`+
instantiation. That can break third-party code that installs a custom handler for
`
``
222
`` +
SIGCHLD. signal).
``
``
223
+
``
224
`+
The watcher avoids disrupting other code spawning processes
`
197
225
`` by polling every process explicitly on a :py:data:SIGCHLD signal.
``
198
226
``
199
``
`-
This is a safe solution but it has a significant overhead when
`
``
227
`+
There is no limitation for running subprocesses from different threads once the
`
``
228
`+
watcher is installed.
`
``
229
+
``
230
`+
The solution is safe but it has a significant overhead when
`
200
231
` handling a big number of processes (O(n) each time a
`
201
232
`` :py:data:SIGCHLD is received).
``
202
233
``
203
``
`-
asyncio uses this safe implementation by default.
`
``
234
`+
.. versionadded:: 3.8
`
``
235
+
``
236
`+
.. class:: SafeChildWatcher
`
``
237
+
``
238
`+
This implementation uses active event loop from the main thread to handle
`
``
239
`` +
:py:data:SIGCHLD signal. If the main thread has no running event loop another
``
``
240
`` +
thread cannot spawn a subprocess (:exc:RuntimeError is raised).
``
``
241
+
``
242
`+
The watcher avoids disrupting other code spawning processes
`
``
243
`` +
by polling every process explicitly on a :py:data:SIGCHLD signal.
``
``
244
+
``
245
`` +
This solution is as safe as :class:MultiLoopChildWatcher and has the same O(N)
``
``
246
`+
complexity but requires a running event loop in the main thread to work.
`
204
247
``
205
248
`.. class:: FastChildWatcher
`
206
249
``
`@@ -211,6 +254,9 @@ implementation used by the asyncio event loop:
`
211
254
` There is no noticeable overhead when handling a big number of
`
212
255
` children (O(1) each time a child terminates).
`
213
256
``
``
257
`+
This solution requires a running event loop in the main thread to work, as
`
``
258
`` +
:class:SafeChildWatcher.
``
``
259
+
214
260
``
215
261
`Custom Policies
`
216
262
`===============
`