Issue 1398: Can't pickle partial functions (original) (raw)

Created on 2007-11-07 15:27 by danhs, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Messages (6)
msg57200 - (view)	Author: Daniel (danhs)	Date: 2007-11-07 15:27
Creating a function using functools.partial results in a function which cannot be pickled. Attached is a small testcase.
msg57260 - (view)	Author: Stefan Sonnenberg-Carstens (pythonmeister)	Date: 2007-11-08 16:49
You are using an old protocol version pickle.dumps(partial_f,2) does the trick: >>> pickle.dumps(partial_f,2) '\x80\x02cfunctools\npartial\nq\x00)\x81q\x01}q\x02b.'
msg71171 - (view)	Author: Nicolas Grilly (ngrilly)	Date: 2008-08-15 15:16
It seems using protocol version 2 is not enough: >>> s = pickle.dumps(partial_f, 2) >>> f = pickle.loads(s) Traceback (most recent call last): ... TypeError: type 'partial' takes at least one argument Am I missing something?
msg71177 - (view)	Author: Alexandre Vassalotti (alexandre.vassalotti) *	Date: 2008-08-15 18:33
I agree that this a bug. However, the liberal functools.partial constructor makes it hard to pickle partial instances correctly. Ideally, we would add the __getnewargs__ special method and be done with it. But, this won't work in this case due to the *args and **kwargs arguments of partial. Since pickle supports neither, we would need to use apply(), which going to be removed in Python 3.0, as follow: >>> import pickletools >>> pickletools.dis("c__builtin__\napply\n(cfunctools\npartial\n(c__main__\nf\nt(S'b'\nK\x01dtR.") 0: c GLOBAL '__builtin__ apply' 19: ( MARK 20: c GLOBAL 'functools partial' 39: ( MARK 40: c GLOBAL '__main__ f' 52: t TUPLE (MARK at 39) 53: ( MARK 54: S STRING 'b' 59: K BININT1 1 61: d DICT (MARK at 53) 62: t TUPLE (MARK at 19) 63: R REDUCE 64: . STOP Unfortunately, pickle.Pickler cannot generate a such pickle stream. So this bug is symptom of the bigger issue that classes with *args and/or **kwargs argument cannot be made picklable.
msg95940 - (view)	Author: Florent Xicluna (flox) *	Date: 2009-12-03 19:23
It seems fixed on trunk. It is fixed on Python 3.1 and 3.2, too. With Python 2.6 I see the error: >>> ./python partial_bug.py TypeError: can't pickle partial objects Documentation for "trunk" and "py3k" is OK?
msg96254 - (view)	Author: Alexandre Vassalotti (alexandre.vassalotti) *	Date: 2009-12-11 13:34
Indeed. Jack Diederich added support for pickling partial functions in r70931.