(original) (raw)

> > Pablo's issue35378 evolved to add a weak reference in iterators to try
> > to detect when the Pool is destroyed: raise an exception from the
> > iterator, if possible.

> That's an ok fix for me.

I am playing with weakreferences inside the iterator and result objects,
but this may not be enough/a complete solution. For example, take the
code of ApplyResult.get:

def get(self, timeout=None):
if self.\_pool() is None: # self.\_pool is a weakref
raise RuntimeError("The pool is dead. Aborting") <--- new code
self.wait(timeout)

It can be that the pool is alive when we check for it (self.\_pool() is None) but while
the code is waiting with no timeout (timeout=None) the pool dies, effectively leaving the
program deadlocked with no error.

--

I agree that misusage of the pool should not be encouraged but in this situation the fact that
this code hangs:

import multiprocessing  

for x in multiprocessing.Pool().imap(int, \["4", "3"\]):  
 print(x)

is a bit worriying because although is incorrect and an abuse of the API, users can do this easily with
no error message other than a misterious hang. I have found this on several places and people were
very confused because usually the interpreter throws some kind of error indication. In my humble opinion,
we should try to avoid hanging as a consequence of the misusage, whatever we do.

Pablo