bpo-35424: emit ResourceWarning at multiprocessing.Pool destruction by vstinner · Pull Request #10974 · python/cpython (original) (raw)

I reverted https://bugs.python.org/issue34172 change because it introduced a backward incompatible change in stable branches. I also reverted the change in the master branch to open a more general discussion: https://mail.python.org/pipermail/python-dev/2018-December/155946.html

The reverted change isn't magic: it doesn't fix all issues. Even with this fix, the destructor still doesn't join many threads and so regrtest will still complain. The .join() method has to be called explicitly to really make sure that a test doesn't leak resources.

The reverted change had a test on "del pool" and this test caused issues on buildbots because it leaked resources (processes, threads). IMHO testing that the destructor calls terminate() deserves its own separated test.

In this PR, the test is restricted to check that the destructor emits a ResourceWarning.

Moreover, to come back to https://bugs.python.org/issue34172 maybe you haven't noticed, by this PR should also help https://bugs.python.org/issue34172 because the problematic example ("for x in multiprocessing.Pool().imap(int, ["4", "3"]): print(x)") should now emit a ResourceWarning (once the reference cycle will be broken again). It's better than nothing to warn developers that something is wrong :-) See also https://bugs.python.org/issue35378

... Sorry for the disgression, but it seems like you asked for it :-)