Issue 4593: Documentation for multiprocessing - Pool.apply() (original) (raw)

Issue4593

Created on 2008-12-08 15:45 by beazley, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Messages (4)
msg77312 - (view)	Author: David M. Beazley (beazley)	Date: 2008-12-08 15:45
The documentation for the apply() and apply_async() methods of a Pool object might emphasize that these operations execute func(*args,**kwargs) in only one of the pool workers and that func() is not being executed in parallel on all workers. On first reading, I actually thought it might be the latter (and had to do some experimentation to figure out what was actually happening).
msg77316 - (view)	Author: Jesse Noller (jnoller) *	Date: 2008-12-08 16:11
Ah, I see what you're talking about David. Good catch. It wouldn't make sense to run the apply() in all of the workers, as apply() is a single function(...) call. map() however is done in parallel amongst all workers.
msg77336 - (view)	Author: David M. Beazley (beazley)	Date: 2008-12-08 19:11
Actually, you shouldn't discount the potential usefulness of running apply() in all of the worker nodes. A lot of people coming from parallel programming know about things like global broadcasts, reductions, and so forth. For example, if I wanted to perform a global operation (maybe some kind of configuration) on all workers, I could see doing some kind of global apply() operation to do it. That said, I'm not actually asking for any new functionality. I'd just make it more clear that apply() is not performing a function call on all pool workers. Also, given that apply() blocks, I'm not exactly sure how useful it is in the context of actually performing work in parallel. You might want to emphasize that apply_async() is better suited for that (the only other way I could think of to take advantage of apply() in parallel would be to call it from separate threads in the process that created the pool).
msg80385 - (view)	Author: Jesse Noller (jnoller) *	Date: 2009-01-22 22:06
fixed in trunk, r68862, merged to 3k in r68863