[Python-Dev] Issues with multiprocessing (original) (raw)

Jesse Noller jnoller at gmail.com
Wed Dec 9 13:52:46 CET 2009

On Wed, Dec 9, 2009 at 6:22 AM, Michael Foord <fuzzyman at voidspace.org.uk> wrote:

On 09/12/2009 08:51, Joshi, Yateen wrote:

Hi,

I have an application that uses multiprocessing pools (multiprocessing.Pool(processes=…..)). There are multiple such pools and each pool has configurable number of processes. Once the process is spawned, it keeps on executing and does the needed processing. If there is nothing to process (like ftp’ing files from some source, if the files are not there, the process would sleep for some time, and then again check for files, that way, it is a infinite loop with some sleep), the process ‘sleeps’ for some time and continues. I am using a T5220, Solaris box with Solaris 10. Problem –there are multiple pools and multiple processes, i am seeing that not all the processes get spawned. They get spawned when the sleep time is increased (say from 0.1 sec to 1 sec). If I further increase the number of processes, again some process do not get spawned. For that, I further need to increase the sleep time (say to 2 sec), then the processes get spawned. Typically, in a multiprocessing I would expect that if a process sleeps for even a small time, other processes should get their chance to execute, but this does not seem to be happening here. Can you please throw some light on it? Hello Joshi, This email list is for the development of Python, not for developing with Python. A better place to try would be the python-list (comp.lang.python - which has both google groups and gmane gateways). All the best, Michael Foord

Additionally, if you suspect there is a reproducible box (preferably not just on solaris, which I do not have access to) please file it in the tracker.

More information about the Python-Dev mailing list