[Python-Dev] Another threading idea (original) (raw)

Paul Moore p.f.moore at gmail.com
Tue Mar 14 23:10:47 CET 2006

On 3/14/06, Raymond Hettinger <raymond.hettinger at verizon.net> wrote:

FWIW, I've been working on a way to simplify the use of queues with daemon consumer threads

Sometimes, I launch one or more consumer threads that wait for a task to enter a queue and then work on the task. A recurring problem is that I sometimes need to know if all of the tasks have been completed so I can exit or do something with the result. [...] So, the idea is to create a subclass of Queue that increments a counter when objects are enqueued, that provides a method for worker threads to decrement the counter when the work is done, and offers a blocking join() method that waits until the counter is zero

I've also hit this problem, and would find this pattern useful.

FWIW, in my code, I bypassed the problem by spawning one worker per task, and waiting on them all. This works, but is sub-optimal (there's no need for 100+ subthreads, when 20 or so generic workers would have done...)

Paul.

More information about the Python-Dev mailing list