bpo-31783: Fix a race condition creating workers during shutdown by brianquinlan · Pull Request #13171 · python/cpython (original) (raw)

The race condition occurs with the following set of context switches:

Main Thread                           Thread  #1
========= .                         ========
1.                                   ThreadPoolExecutor.submit()
                                        ...
                                        if _shutdown:

                    <=====

2. thread._python_exit
      _shutdown = true
      items = list(_threads_queues.items())

                    =====>

3.                                      ...
                                         _threads_queues[t] = self._work_queue

                    <=====

4. 
    for t, q in items:
        q.put(None)
    for t, q in items:
        t.join()

At this point there is a new thread added to _threads_queues but thread._python_exit won't join it because it was added to _threads_queues after a copy was made (i.e.
items = list(_threads_queues.items()))

The bug has a patch with time.sleep() to force the pathological sequencing and a test script to demonstrate the problem.