bpo-33966, multiprocessing: Pool wait for worker start by vstinner · Pull Request #7966 · python/cpython (original) (raw)

The problem is this will make the Pool constructor much slower (especially with the "spawn" method).

I ran a benchmark on the master branch on Linux (Python compiled in release mode):

$ ./python -m perf timeit -s 'import multiprocessing; ctx = multiprocessing.get_context("spawn")' 'ctx.Pool(1)'

Result:

Mean +- std dev: [ref] 9.02 ms +- 0.50 ms -> [event] 43.0 ms +- 5.0 ms: 4.76x slower (+376%)

So yes, the creation of the Pool object becomes much slower. But if you use the Pool, the overall performance shouldn't change. It's just that currently, the worker startup time it delayed to the first usage of the pool.

Do you think that the slowdown is an issue?

Would you prefer to wait until the worker started before terminating it, to avoid the race condition?

Note: If you missed it, I tried a different approach but it changed the lifetime of handles in the parent process and so I abandonned it.
#7966 (comment)