[Python-Dev] Another threading idea (original) (raw)

Guido van Rossum guido at python.org
Tue Mar 14 23:08:09 CET 2006

Isn't this a job for threading.BoundedSpemaphore()?

On 3/14/06, Raymond Hettinger <raymond.hettinger at verizon.net> wrote:

FWIW, I've been working on a way to simplify the use of queues with daemon consumer threads

Sometimes, I launch one or more consumer threads that wait for a task to enter a queue and then work on the task. A recurring problem is that I sometimes need to know if all of the tasks have been completed so I can exit or do something with the result. If each thread only does a single task, I can use t.join() to wait until the task is done. However, if the thread stays alive and waits for more Queue entries, then there doesn't seem to be a good way to tell when all the processing is done. So, the idea is to create a subclass of Queue that increments a counter when objects are enqueued, that provides a method for worker threads to decrement the counter when the work is done, and offers a blocking join() method that waits until the counter is zero # main thread q = TaskQueue() for t in workerthreads(): t.start() for task in tasklist: q.put(task) # increments the counter and enqueues a task q.join() # all of the tasks are done (counter is zero) doworkonresults()

# worker thread while 1: task = q.get() # task is popped but the counter is unchanged dowork(task) q.decrement() # now the counter gets reduced The idea is still in its infancy (no implementation and it hasn't been tried in real-world code) but I would like to get some feedback. If it works out, I'll post a recipe to ASPN and see how it goes. Raymond

Python-Dev mailing list Python-Dev at python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org

-- --Guido van Rossum (home page: http://www.python.org/~guido/)

More information about the Python-Dev mailing list