Issue 8289: multiprocessing.Process.init pickles all arguments (original) (raw)

Issue8289

Created on 2010-04-02 15:06 by cool-RR, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Messages (6)
msg102172 - (view)	Author: Ram Rachum (cool-RR) *	Date: 2010-04-02 15:06
Currently, when you create a Process, all arguments you pass to its __init__ get pickled. I understood this is done because arguments to __init__ almost always become attributes to the Process. Like this: def MyProcess(multiprocessing.Process): def __init__(self, whatever): self.whatever = whatever Of course, attributes must be pickled so they can be accessed from the separate process (on Windows). And indeed in most cases all arguments to __init__ become attributes, so this makes sense. But, in some cases you pass in arguments to __init__ that do not become attributes. In my case, __init__ takes an object, and takes some attributes of this object as attributes to itself. The object is unpicklable, but the attributes are. So I had to make some ugly workaround to make the program run. So I think it would be better if Process would be smart enough to pickle only the arguments that get set as attributes.
msg102173 - (view)	Author: Ram Rachum (cool-RR) *	Date: 2010-04-02 15:07
Sorry, I accidentally typed "def" instead of "class" in my code sample.
msg162525 - (view)	Author: Richard Oudkerk (sbt) *	Date: 2012-06-08 12:46
As long as you don't pass the arguments on to Process.__init__() when you call it there should be no problem. The following program works, but will fail with RuntimeError if you uncomment the comment line: from multiprocessing import Process class Unpicklable(object): def __reduce__(self): raise RuntimeError class MyProcess(Process): def __init__(self, foo, unpicklable_bar): Process.__init__(self, #args=(foo, unpicklable_bar) ) self.foo = foo self.baz = str(unpicklable_bar) def run(self): print(self.foo) print(self.baz) if __name__ == '__main__': p = MyProcess([1,2,3], Unpicklable()) p.start() p.join()
msg162624 - (view)	Author: Richard Oudkerk (sbt) *	Date: 2012-06-11 15:46
I don't think there is any problem here since you have control over which arguments you pass to __init__. Without a reason why that is not a solution I will eventually close the issue as rejected.
msg162627 - (view)	Author: Ram Rachum (cool-RR) *	Date: 2012-06-11 16:21
I opened this issue 2 years ago, and I don't remember it being easily solvable back then. But I've long forgotten what the problems were, and I've lost personal interest in it, so I guess we'll just let it go.
msg162630 - (view)	Author: Richard Oudkerk (sbt) *	Date: 2012-06-11 17:04
OK, I'll close.