[Python-Dev] bpo-34837: Multiprocessing.Pool API Extension (original) (raw)

[Python-Dev] bpo-34837: Multiprocessing.Pool API Extension - Pass Data to Workers w/o Globals

Michael Selik mike at selik.org
Sat Sep 29 15:00:09 EDT 2018

On Sat, Sep 29, 2018 at 5:24 AM Sean Harrington <seanharr11 at gmail.com> wrote:

On Fri, Sep 28, 2018 at 4:39 PM Sean Harrington <seanharr11 at gmail.com> wrote: > My simple argument is that the developer should not be constrained to make the objects passed globally available in the process, as this MAY break encapsulation for large projects.

I could imagine someone switching from Pool to ThreadPool and getting into trouble, but in my mind using threads is caveat emptor. Are you worried about breaking encapsulation in a different scenario? >> Without a specific example on-hand, you could imagine a tree of function calls that occur in the worker process (even newly created objects), that should not necessarily have access to objects passed from parent -> worker. In every case given the current implementation, they will.

Echoing Antoine: If you want some functions to not have access to a module's globals, you can put those functions in a different module. Note that multiprocessing already encapsulates each subprocesses' globals in essentially a separate namespace.

Without a specific example, this discussion is going to go around in circles. You have a clear aversion to globals. Antoine and I do not. No one else seems to have found this conversation interesting enough to participate, yet.

More information about the Python-Dev mailing list