[Python-Dev] PEP 3148 ready for pronouncement (original) (raw)

Brian Quinlan brian at sweetapp.com
Fri May 28 08:52:39 CEST 2010

On May 28, 2010, at 1:39 PM, Scott Dial wrote:

On 5/27/2010 4:13 AM, Brian Quinlan wrote:

On 27 May 2010, at 17:53, Floris Bruynooghe wrote:

On Thu, May 27, 2010 at 01:46:07PM +1200, Greg Ewing wrote:

On 27/05/10 00:31, Brian Quinlan wrote:

You have two semantic choices here: 1. let the interpreter exit with the future still running 2. wait until the future finishes and then exit

I'd go for (1). I don't think it's unreasonable to expect a program that wants all its tasks to finish to explicitly wait for that to happen. I'd got for (1) as well, it's no more then reasonable that if you want a result you wait for it. And I dislike libraries doing magic you can't see, I'd prefer if I explicitly had to shut a pool down. I'm glad I'm not alone in preferring (1) tough. Keep in mind that this library magic is consistent with the library magic that the threading module does - unless the user sets Thread.daemon to True, the interpreter does not exit until the thread does. Given your rationale, I don't understand from the PEP: shutdown(wait=True) Signal the executor that it should free any resources that it is using when the currently pending futures are done executing. Calls to Executor.submit and Executor.map and made after shutdown will raise RuntimeError. If wait is True then the executor will not return until all the pending futures are done executing and the resources associated with the executor have been freed. Can you tell me what is the expected execution time of the following: executor = ThreadPoolExecutor(maxworkers=1) executor.submit(lambda: time.sleep(1000)) executor.shutdown(wait=False) sys.exit(0) I believe it's 1000 seconds, which seems to defy my request of shutdown(wait=False) because "secretly" the Python exit is going to wait anyways.

It would take 1000 seconds. "...then the executor will not return..."
should read "...then the method will not return...".

ISTM, it is much easier to get behavior #2 if you have behavior #1, and it would also seem rather trivial to make ThreadPoolExecutor take an optional argument specifying which behavior you want.

Adding a daemon option would be reasonable. If you don't shutdown your
executors you are pretty much guaranteed to get random traceback
output on exit through.

Your reference implementation does not actually implement the specification given in the PEP, so it's quite impossible to check this myself. There is no wait=True option for shutdown() in the reference implementation, so I can only guess what that implementation might look like.

Look at around line 129 in: http://code.google.com/p/pythonfutures/source/browse/branches/feedback/python3/futures/thread.py

Cheers, Brian

More information about the Python-Dev mailing list