Issue 12155: queue example doesn't stop worker threads (original) (raw)

Created on 2011-05-23 10:22 by vstinner, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Messages (5)
msg136601 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2011-05-23 10:22
The queue doc contains the following example: ------------------ def worker(): while True: item = q.get() do_work(item) q.task_done() q = Queue() for i in range(num_worker_threads): t = Thread(target=worker) t.daemon = True t.start() for item in source(): q.put(item) q.join() # block until all tasks are done ------------------ http://docs.python.org/library/queue.html It doesn't define do_work(), num_worker_threads or do_work(), but my concern is that it doesn't stop worker threads. I consider "t.daemon = True" as an hack to not care about stopping threads. The example should pass a special value to each worker to stop it. For example: while True: job = queue.get() if job is None: break audio.play(*job) queue.task_done() Main thread: ... threads = [] for i in range(num_worker_threads): t = Thread(target=worker) threads.append(t) t.start() ... for i in range(num_worker_threads): queue.put(None) queue.join() for thread in threads: thread.join()
msg136710 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2011-05-24 01:17
> It doesn't define do_work(), num_worker_threads or do_work() Is it unclear to you what those mean? They are placeholders for the user's actual task at hand. > my concern is that it doesn't stop worker threads. Stopping the threads wasn't the intent of the example. > I consider "t.daemon = True" as an hack to not care > about stopping threads. If you post a high-quality self-contained example somewhere on the net, I would be happy to link to it.
msg136724 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2011-05-24 07:32
> Is it unclear to you what those mean? Well, it's clear, but I like when I can simply copy/paste the example and it does just work: > If you post a high-quality self-contained example somewhere > on the net, I would be happy to link to it. I just propose to change the example to stop the threads: ------------------ def worker(): while True: item = q.get() if item is None: break do_work(item) q.task_done() q = Queue() threads = [] for i in range(num_worker_threads): t = Thread(target=worker) threads.append(t) t.start() for item in source(): q.put(item) q.join() # block until all tasks are done for i in range(num_worker_threads): q.put(None) for t in threads: t.join() ------------------
msg137153 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2011-05-28 20:55
The proposed change adds about 7 lines to show the 'trick' of putting num-worker Nones on the queue. I think that is worth it.
msg238426 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-03-18 13:06
New changeset b44ec269abda by Victor Stinner in branch 'default': Issue #12155: Fix queue doc example to join threads https://hg.python.org/cpython/rev/b44ec269abda
History
Date User Action Args
2022-04-11 14:57:17 admin set github: 56364
2015-03-18 13:06:34 vstinner set status: open -> closedresolution: fixedversions: + Python 3.5, - Python 3.1, Python 2.7, Python 3.2, Python 3.3
2015-03-18 13:06:20 python-dev set nosy: + python-devmessages: +
2011-05-28 20:55:18 terry.reedy set nosy: + terry.reedymessages: +
2011-05-24 07:32:36 vstinner set messages: +
2011-05-24 01:17:49 rhettinger set priority: normal -> lowmessages: +
2011-05-24 01:08:33 rhettinger set assignee: docs@python -> rhettinger
2011-05-23 10:49:17 pitrou set nosy: + rhettinger
2011-05-23 10:22:30 vstinner create