gh-100228: Warn from os.fork() if other threads exist. by gpshead · Pull Request #100229 · python/cpython (original) (raw)

Where can I get your input on an issue I'm seeing where I dont actually see why this warning is being emitted? This has to do with the use of the pytest-xdist plugin which uses a tool called execnet. When using this plugin, if our own test uses fork, we get the warning. however, I've created a suite that runs through threading.enumerate() in the parent process, and in the child process - there is no additional thread spawned in either, there is only a single thread, the "main" thread.

output can be seen as follows - as you can see I'm not able to find any additional threads that are known to the Python interpreter. So....execnet is spawning threads outside of the interpreter? is that what has to be happening? is that a bug for them?

initialized: 1/1 workerINFO:main:THREAD FROM MAIN PROCESS 14974: <_MainThread(MainThread, started 140612028988288)>

1 worker [1 item]      
INFO:main:THREAD FROM CHILD PROCESS 14975 (parent: 14974): <_MainThread(MainThread, started 139972663720832)>

.
=========================================================================================== warnings summary ============================================================================================
test.py::TestWhatever::test_thing
  /usr/lib64/python3.12/multiprocessing/popen_fork.py:66: DeprecationWarning: This process (pid=14975) is multi-threaded, use of fork() may lead to deadlocks in the child.
    self.pid = os.fork()

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
===================================================================================== 1 passed, 1 warning in 10.17s =====================================================================================

I've looked at execnet a few times and so far it's opaque to me what it's doing or how pytest-xdist uses it.