bpo-36780: Add wait_at_exit to ThreadPoolExecutor.shutdown. by hniksic · Pull Request #13250 · python/cpython (original) (raw)

@brianquinlan This was intentional - I tested shutdown(wait=False) with ProcessPoolExecutor, and found that it raised exceptions and hanged the process at exit. (Not just hanged in the sense of waiting for the pending futures, but completely hanged, even when the futures exited.) So the new functionality is only available in and documented for ThreadPoolExecutor.

The expected behavior is for the program to print 1 and 2 and then to wait for 5 seconds before exiting. Instead, it prints 1 and 2, but hangs at exit with the following output:

$ python3.7 ~/Desktop/x
1
2
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/usr/lib/python3.7/concurrent/futures/process.py", line 101, in _python_exit
    thread_wakeup.wakeup()
  File "/usr/lib/python3.7/concurrent/futures/process.py", line 89, in wakeup
    self._writer.send_bytes(b"")
  File "/usr/lib/python3.7/multiprocessing/connection.py", line 183, in send_bytes
    self._check_closed()
  File "/usr/lib/python3.7/multiprocessing/connection.py", line 136, in _check_closed
    raise OSError("handle is closed")
Exception in thread QueueManagerThread:
Traceback (most recent call last):
  File "/usr/lib/python3.7/threading.py", line 917, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.7/threading.py", line 865, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python3.7/concurrent/futures/process.py", line 368, in _queue_management_worker
    thread_wakeup.clear()
  File "/usr/lib/python3.7/concurrent/futures/process.py", line 92, in clear
    while self._reader.poll():
  File "/usr/lib/python3.7/multiprocessing/connection.py", line 255, in poll
    self._check_closed()
  File "/usr/lib/python3.7/multiprocessing/connection.py", line 136, in _check_closed
    raise OSError("handle is closed")
OSError: handle is closed
OSError: handle is closed