Issue 1692335: Fix exception pickling: Move initial args assignment to BaseException.new (original) (raw)
Created on 2007-04-01 13:46 by zseil, last changed 2022-04-11 14:56 by admin. This issue is now closed.
Messages (64)
Author: Ziga Seilnacht (zseil) * 
Date: 2007-04-01 13:46
Pickling exceptions fails when an Exception class requires an argument in the constructor, but doesn't call its base class' constructor. See this mail for details:
http://mail.python.org/pipermail/python-dev/2007-April/072416.html
This patch simply moves initial args assignment to BaseException.new. This should fix most of the problems, because it is very unlikely that an exception overwrites the new method; exceptions used to be old style classes, which don't support the new special method.
The args attribute is still overwritten in all the init methods, so there shouldn't be any backward compatibility problems.
Author: Ziga Seilnacht (zseil) *
Date: 2007-04-01 13:47
File Added: exc_args_25.diff
Author: Ziga Seilnacht (zseil) *
Date: 2007-04-01 21:50
I'm attaching a test that Eric Huss sent in private mail.
The test fails, because old exception pickles don't have args for the reconstructor, but their init() gets called anyway because they are new style classes now.
The problem is in cPickle.InstanceNew() and pickle.Unpickler._instantiate(). Those methods behave differently depending on the type of the object instantiated; they avoid the init() call when type is an old style class.
There is nothing that can be done in the exception classes to fix this issue; the fix would need to be in the pickle and cPickle module. File Added: test_exception_pickle.py
Author: Eric Huss (ehuss)
Date: 2007-06-15 00:34
I have stumbled across another scenario where unpickling fails. If your exception takes arguments, but you call Exception.init with a different number of arguments, it will fail. As in:
class D(Exception): def init(self, foo): self.foo = foo Exception.init(self)
Author: Eric Huss (ehuss)
Date: 2007-06-27 18:45
Added patch #1744398 as an alternate solution.
Author: Georg Brandl (georg.brandl) *
Date: 2007-08-11 09:06
Attaching a new patch that fixes the
class D(Exception): def init(self, foo): self.foo = foo Exception.init(self)
scenario by keeping the original args to new as an exception attribute. File Added: exception-pickling.diff
Author: Neal Norwitz (nnorwitz) *
Date: 2007-08-11 18:27
self->args needs to be initialized to NULL because if the alloc of args failed, self->args would be uninitialized and deallocated. (I realize the alloc of an empty tuple will realistically never fail currently.)
I'm not sure if this could cause any new problems because of the behavior change, but the code itself looked fine to me. Hopefully someone with more knowledge of exceptions could take a look at the idea.
Author: Eric Huss (ehuss)
Date: 2007-08-12 01:33
Georg's new patch looks good to me, it seems to pass all the scenarios that I know of. You can close out my patch if you check this in. Thanks for looking into this!
Author: Georg Brandl (georg.brandl) *
Date: 2007-08-12 08:00
The only question left is what to do if something reassigns exceptioninstance.args.
Author: Ziga Seilnacht (zseil) *
Date: 2007-08-12 11:23
There is also the problem that Jim Fulton mentioned in bug #1742889; if the user modifies the 'message' attribute of an exception (or any other attribute that is not stored in the dict), it won't get serialized on pickling.
Attaching a new patch that fixes this by using the getstate() pickling hook. It uses tp_members and tp_getset members of the type object to find all the attributes that need to be serialized.
The 2.5 patch also contains a fix for migrating old exception pickles to Python 2.5. For that I had to change cPickle and pickle. It would be nice if Jim could comment if this change is needed, since ZODB is probably the biggest user of these modules.
The downside is that this patch is fairly big.
File Added: exception_pickling_25.diff
Author: Ziga Seilnacht (zseil) *
Date: 2007-08-12 11:24
File Added: exception_pickling_26.diff
Author: Georg Brandl (georg.brandl) *
Date: 2007-08-12 12:13
I wouldn't care too much about .message; it was new in 2.5 and is already deprecated in 2.6.
Author: Ziga Seilnacht (zseil) *
Date: 2007-08-12 12:34
It's not just the message attribute; the problem is that the current code expects that the exception won't be modified between creation and unpickling.
For example:
import pickle e = EnvironmentError() e.filename = "x" new = pickle.dumps(pickle.loads(e)) print new.filename None e = SyntaxError() e.lineno = 10 new = pickle.loads(pickle.dumps(e)) print new.lineno None
Author: Georg Brandl (georg.brandl) *
Date: 2007-08-12 13:24
Yes, this is what I meant with my last comment.
Author: Georg Brandl (georg.brandl) *
Date: 2007-08-23 17:29
Raising priority.
Author: Sean Reifschneider (jafo) *
Date: 2007-09-17 10:01
It's not clear to me what the next step is here, does one or more of the other folks need to provide some input? Georg: If you are looking for something from someone, can you assign the ticket to them?
Author: Georg Brandl (georg.brandl) *
Date: 2007-09-20 08:47
IMO zseil's latest patch is the best one, but I need any other developer to review and approve it.
Author: Jaroslav Pachola (jarpa)
Date: 2008-01-17 22:13
While zseil's patch for Python 2.5 works for me (on the current 2.5.1 download), svn version of Python 2.6 rejects the 2.6 patch. Attaching fixed 2.6 patch (2 rejects, 1 fuzz fixed, patch works without complains for me). I would be very glad if someone could review the patches and maybe commit them.
Author: Facundo Batista (facundobatista) *
Date: 2008-01-19 14:03
The last patch (exception_pickling_26.diff) applies cleanly, and all the tests run ok.
System: Python 2.6a0 (trunk:60073M, Jan 19 2008, 11:41:33) [GCC 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)] on linux2
Author: Guido van Rossum (gvanrossum) *
Date: 2008-02-01 17:38
I'm against including this in 2.5.2. It reeks of a new feature, and the code looks like it would risk breaking other apps.
(I'm fine with adding this to 2.6 of course.)
Author: Jaroslav Pachola (jarpa)
Date: 2008-02-02 08:07
To me it seems more like a bug fix - in Python 2.4 and older the pickling works well and the current 2.5 behavior really breaks existing code. That's why I plead for inclusion in 2.5.2; because this issue prevents our company to move to 2.5.
Author: Guido van Rossum (gvanrossum) *
Date: 2008-02-02 11:36
Understood. Can you get other developers on python-dev to weigh in? Maybe I am over-estimating the danger.
Author: Martin v. Löwis (loewis) *
Date: 2008-02-03 19:04
I tested exception_pickling_25.diff, and it may break existing code. In 2.5.1, Exception("Hello,4).reduce() gives (<type 'exceptions.Exception'>, ('Hello', 4))
With the patch, it gives
TypeError: can't pickle Exception objects
IMO, that is an unacceptable change for a bugfix release.
Aside: please give unique file names to the patches, or remove patches if you want to replace a previous patch.
Author: Benjamin Peterson (benjamin.peterson) *
Date: 2008-07-31 01:51
How is this coming? Can we apply this to 2.6?
Author: Ziga Seilnacht (zseil) *
Date: 2008-11-17 11:37
Sorry for the long silence. I think that this patch is not relevant anymore. The code that uses exception pickling already had to be adapted to changes in Python 2.5, so there is no need to change the pickling again and risk breaking user code. I'll close it in a few days if nobody objects.
Author: Eric Huss (ehuss)
Date: 2008-11-18 22:06
I'm disappointed to see this closed. Exception pickling is still broken in some cases in 2.6.
Author: Guido van Rossum (gvanrossum) *
Date: 2008-11-18 22:52
OK, reopening. Can you post an example that fails today?
Author: Eric Huss (ehuss)
Date: 2008-11-18 23:07
In the attached test_exception_pickle.py file, class C and D cannot be unpickled (raises TypeError).
class C(Exception): """Extension with values, args not set.""" def init(self, foo): self.foo = foo
class D(Exception): """Extension with values, init called with no args.""" def init(self, foo): self.foo = foo Exception.init(self)
There are also some other exceptions that fail to be handled properly. See the exception_pickling_26.diff for some unittests. In particular, I see SlotedNaiveException failing.
Author: Kyle VanderBeek (kylev)
Date: 2010-04-13 00:22
Ping? Is anyone working on this? Have Eric's concerns been addressed and can we pickle/unpickle all exceptions yet?
Author: Tres Seaver (tseaver) *
Date: 2010-05-31 21:13
The attached patch adds Mark's examples to test_pickle as a failing test.
Author: Alexander Belopolsky (belopolsky) *
Date: 2010-06-29 22:18
The case in are probably not pointing to a bug. If subclass' init passes its args to the base class init (as it probably should), pickling works:
class E(Exception): """Extension with values, init called with args.""" def init(self, foo): self.foo = foo Exception.init(self, foo)
from pickle import * loads(dumps(E(1))) E(1,) _.foo 1
What would be a use case for not setting args? Even if there was a valid use case for it, it would be easy to simply provide custom getinitargs or reduce to support it.
Note that with definitions, eval(repr(C('foo'))) also raises a TypeError, so properly supporting args hiding in exception subclasses would probably involve overriding repr as well.
Author: Eric Huss (ehuss)
Date: 2010-06-30 03:04
Alexander, the use case I was involved with was an RPC system which allowed exceptions to propagate over the connection. In this case, you do not have absolute control over which exceptions may be raised, or who wrote the code that is raising the exception. There are many cases, even in the standard library, where people do not pass all arguments to the base exception class. Some of these are probably mistakes, but in general pickle shouldn't fail on otherwise legitimate objects.
There are other cases where passing all arguments up is not wanted, or at least cumbersome. If you have multiple levels of inheritance, and subclasses add additional arguments to the init, they wouldn't have a way to include those arguments to the base class unless all classes were written with *args in the init function, which many people would not know or remember to do.
Author: Jason R. Coombs (jaraco) *
Date: 2010-09-22 13:39
In , I believe belopolsky is mistaken in stating that "it would be easy to simply provide custom getinitargs or reduce to support it". It appears getinitargs does not work on Python 2.5 or Python 2.7. Exceptions of the following class still raise a TypeError on unpickling:
class D(Exception): """Extension with values, init called with no args.""" def init(self, foo): self.foo = foo Exception.init(self)
def __getinitargs__(self):
return self.foo,Using reduce does seem to work. I suspect this is because Exceptions are extension types.
I think the fundamental problem is that pickling exceptions does not follow the principle of least surprise. In particular:
- Other built-in objects (dicts, lists, etc) don't require special handling (replace Exception with dict in the above example and it works).
- It's not obvious how to write an Exception subclass that takes additional arguments and make it pickleable.
- Neither the pickle documentation nor the Exception documentation describe how pickling is implemented in Exceptions.
Eric has provided some good use cases. Furthermore, it seems counter-intuitive to me to pass a subclass' custom arguments to the parent class. Indeed, even the official tutorial defines exception classes that are unpickleable (http://docs.python.org/tutorial/errors.html#tut-userexceptions).
If the use case is obvious enough that it shows up in the hello world tutorial, I don't think there should be any argument that it's not a common use case.
At the very least, there should be a section in the pickle documentation or Exception documentation describing how one should make a pickleable subclass. What would be better is for Exceptions to behave like most other classes when being pickled.
Author: Jason R. Coombs (jaraco) *
Date: 2010-09-22 14:58
After some further reading, I found that PEP-352 (http://www.python.org/dev/peps/pep-0352/) explicitly states that "Including programmatic information (e.g., an error code number) should be stored as a separate attribute in a subclass [and not in the args attribute]." The parameter to Exception.init should always be a single, human-readable string. The default pickler doesn't handle this officially-prescribed use-case without overriding reduce.
class F(Exception): def init(self, message, code): Exception.init(self, message) self.code = code
Of course, as belopolsky observed, repr must also be overridden.
Author: Alexander Belopolsky (belopolsky) *
Date: 2010-09-22 18:39
On Wed, Sep 22, 2010 at 9:39 AM, Jason R. Coombs <report@bugs.python.org> wrote:
.. It appears getinitargs does not work on Python 2.5 or Python 2.7.
Yes, getinitargs is only used for old style classes. I was wrong on that point.
Exceptions of the following class still raise a TypeError on unpickling:
class D(Exception): """Extension with values, init called with no args.""" def init(self, foo): self.foo = foo Exception.init(self)
def getinitargs(self): return self.foo,
The problem with your D class is that it does not provide correct args attribute which is expected from an exception class:
()
Using reduce does seem to work. I suspect this is because Exceptions are extension types.
There are two ways to fix this class. Both fix the args issue as well:
Pass foo to Exception.init like this: Exception.init(self, foo) in D.init.
Explicitly initialize self.args: self.args = foo,
I think the fundamental problem is that pickling exceptions does not follow the principle of least surprise. In particular:
- Other built-in objects (dicts, lists, etc) don't require special handling (replace Exception with dict in the above example and it works).
Other built-in objects don't provide an API for retrieving their init arguments.
- It's not obvious how to write an Exception subclass that takes additional arguments and make it pickleable.
AFAICT, this is python subclassing 101: if base class init uses arguments, they should be passed to it by subclass' init.
- Neither the pickle documentation nor the Exception documentation describe how pickling is implemented in Exceptions.
Exception documentation may be improved by adding a section on subclassing. Note that the args argument was not even mentioned in 2.7 documentation, so some discussion of its role may be helpful somewhere.
Eric has provided some good use cases. Furthermore, it seems counter-intuitive to me to pass a subclass' custom arguments to the parent class. Indeed, even the official tutorial defines exception classes that are unpickleable (http://docs.python.org/tutorial/errors.html#tut-userexceptions).
Well, the tutorial examples should probably be changed. In these examples base class init is not called at all which is probably not a good style.
If the use case is obvious enough that it shows up in the hello world tutorial, I don't think there should be any argument that it's not a common use case.
I am not arguing against simplifying Exception subclassing. I just don't see an easy solution that would not promote subclasses with unusable args attribute. I also disagree with this issue classified as a bug. It may be a valid feature request, but not a bug.
In any case, no proponent of this feature has come up with a patch for 3.2 so far and in my view, this would be a prerequisite for moving this forward.
At the very least, there should be a section in the pickle documentation or Exception documentation describing how one should make a pickleable subclass. ..
I agree, but again someone has to step in to write such section. Improving documentation may also be the only solution for the 2.x series.
Author: Ben Bass (bpb)
Date: 2011-04-12 13:52
Perhaps this should be addressed separately, but subprocess.CalledProcessError is subject to this problem (can't be unpickled) (it has separate returncode and cmd attributes, but no args).
It's straightforward to conform user-defined Exceptions to including .args and having reasonable init functions, but not possible in the case of stdlib exceptions.
import subprocess, pickle try: ... subprocess.check_call('/bin/false') ... except Exception as e: ... pickle.loads(pickle.dumps(e)) ... Traceback (most recent call last): File "", line 2, in File "/usr/lib/python3.1/subprocess.py", line 435, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '/bin/false' returned non-zero exit status 1
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "", line 4, in File "/usr/lib/python3.1/pickle.py", line 1363, in loads encoding=encoding, errors=errors).load() TypeError: init() takes at least 3 positional arguments (1 given)
Author: Faheem Mitha (fmitha)
Date: 2012-01-11 08:01
What is the status on this? It contains to be an issue. See http://bugs.python.org/issue13751 and http://bugs.python.org/issue13760
Author: Tal Einat (taleinat) *
Date: 2012-01-11 16:57
I'd just like to weigh in and say that this is a major issue for me at the moment. Not being able to indiscriminately pickle/unpickle exceptions is making my parallel-processing work very painful, because of problematic stdlib exceptions.
I'm surprised this hasn't been fixed way back in 2.x. FWIW, for my project having this fixed in 3.x could be a significant incentive to finally ditch 2.x.
Author: Richard Oudkerk (sbt) *
Date: 2012-07-24 21:43
ISTM the simplest approach would be to just set self->args in BaseException.new() (like in Georg's patch) but to ignore the possibility that the user might later set self.args to something stupid "wrong":
diff -r 51ac5f06dd04 Objects/exceptions.c --- a/Objects/exceptions.c Tue Jul 24 03:45:39 2012 -0700 +++ b/Objects/exceptions.c Tue Jul 24 22:12:49 2012 +0100 @@ -44,12 +44,17 @@ self->traceback = self->cause = self->context = NULL; self->suppress_context = 0;
- self->args = PyTuple_New(0);
- if (!self->args) {
Py_DECREF(self);return NULL;
if (!args) {
args = PyTuple_New(0);if (!args) {Py_DECREF(self);return NULL;}} else {
Py_INCREF(args);}
self->args = args; return (PyObject *)self; }
Certainly it will not work for all cases (like calling a base classes' init with different arguments), but it does cover the very common case where init() is defined but does not call the base classes' init().
Such a patch is minimally invasive and, as far as I can see, would not break currently working code.
Would this be acceptable for a bugfix release?
Author: Richard Oudkerk (sbt) *
Date: 2012-07-26 12:52
I see that the originally proposed patch is more or less what I suggested above. Since this has been a critical issue for 5 years, I think such a minimal patch should be committed even though it is not a complete solution.
It seems to me that the more complicated patch exception_pickling_26.diff has a significant overlap with _common_reduce() in typeobject.c.
The patch actually attempts to be more general than _common_reduce() since it collects data from tp_getset and tp_members, whereas _common_reduce() only collects data from slots. The patch does not seem to take account of read-only members/getsetters, which I think would cause exceptions when unpickling.
An alternative implementation piggy-backing on _common_reduce() (and using reduce rather than getstate) would be
def reduce(self): slots = None func, args, state = object.reduce(self, 2)[:3] if type(state) is tuple: state, slots = state newstate = {'args': self.args} if state: newstate.update(state) if slots: newstate.update(slots) return func, args, newstate
This deals correctly with slots and works with all protocols. However, the pickles produced can only be unpickled on versions where exceptions are new-style classes, ie Python 2.5 and later.
This would have the side effect that init() would no longer be called while unpickling.
Author: Brett Cannon (brett.cannon) *
Date: 2012-07-26 21:34
While I would be happy to see this issue finally closed, but it's up to Georg if this goes into 3.3 or not as one could argue it's a feature or a bugfix/oversight.
As for the actual fix, classic classes shouldn't hold back a good solution since that is a 2.x thing that only affects Python 2.4 which is too old to worry about; I think it's an acceptable limitation.
Author: Richard Oudkerk (sbt) *
Date: 2012-07-26 23:16
I realize now that the idea of using object.reduce(..., 2) would not really work since many exception classes use non-slot descriptors (unless 'slots' attributes were also added as hints of what to serialize).
I think there are two options simple enough to sneak in to 3.3:
(1) The trivial patch of initially setting self->args in new().
(2) Georg's idea of additionally setting a newargs attribute in new() and using it in reduce(). However, I would store newargs directly in the struct to avoid always triggering creation of a dict for the instance.
Author: Antoine Pitrou (pitrou) *
Date: 2012-07-26 23:23
I think there are two options simple enough to sneak in to 3.3:
(1) The trivial patch of initially setting self->args in new().
(2) Georg's idea of additionally setting a newargs attribute in new() and using it in reduce(). However, I would store newargs directly in the struct to avoid always triggering creation of a dict for the instance.
At this point of the release process, the trivial approach sounds safer to me (but is it?).
Author: Richard Oudkerk (sbt) *
Date: 2012-07-27 11:00
Here is a minimal patch against default.
It is a clear improvement on the current situation, even though it still cannot handle the case
class Error(Exception): def init(self, x): Exception.init(self) self.x = x
Author: Antoine Pitrou (pitrou) *
Date: 2012-07-27 11:40
There should be an actual pickling / unpickling test, so that this doesn't regress by mistake.
Le vendredi 27 juillet 2012 à 11:01 +0000, Richard Oudkerk a écrit :
Changes by Richard Oudkerk <shibturn@gmail.com>:
Added file: http://bugs.python.org/file26537/init_args.patch
Author: Richard Oudkerk (sbt) *
Date: 2012-07-27 12:09
ExceptionTests.testAttributes already checks the attributes of unpickled copies (using all protocols).
Author: Antoine Pitrou (pitrou) *
Date: 2012-07-27 12:11
ExceptionTests.testAttributes already checks the attributes of unpickled copies (using all protocols).
Ok, sorry for the noise then.
Author: Brett Cannon (brett.cannon) *
Date: 2012-07-27 14:58
Temporarily making this a blocker to see if Georg considers this a bugfix or feature.
Author: Richard Oudkerk (sbt) *
Date: 2012-07-27 16:26
BTW, BaseException_init() has the code
Py_XDECREF(self->args);
self->args = args;
Py_INCREF(self->args);Presumably the Py_XDECREF(self->args) would be better replaced by Py_CLEAR(self->args)?
Author: Antoine Pitrou (pitrou) *
Date: 2012-07-27 17:42
Presumably the Py_XDECREF(self->args) would be better replaced by Py_CLEAR(self->args)?
Ah, yes, definitely.
Author: Antoine Pitrou (pitrou) *
Date: 2012-07-27 17:47
Or you could simply Py_INCREF(args) before the Py_XDECREF...
Author: Richard Oudkerk (sbt) *
Date: 2012-07-27 19:00
Or you could simply Py_INCREF(args) before the Py_XDECREF...
But won't self->args point to a broken object while any callbacks triggered by Py_XDECREF() are run?
An alternative would be
tmp = self->args;
self->args = args;
Py_INCREF(self->args);
Py_XDECREF(tmp);As far as I can see the idiom Py_?DECREF(self->...) is rarely safe outside of a deallocator unless you are sure the pointed to object has a "safe" type (or you are sure the refcount cannot fall to zero).
Author: Antoine Pitrou (pitrou) *
Date: 2012-07-27 19:12
But won't self->args point to a broken object while any callbacks triggered by Py_XDECREF() are run?
An alternative would be
tmp = self->args; self->args = args; Py_INCREF(self->args); Py_XDECREF(tmp);
You are right, that's better.
Author: Richard Oudkerk (sbt) *
Date: 2012-07-27 20:16
Patch which adds fix for BaseException_init().
Actually this class of problem seem to be quite common.
BaseException_set_tb() appears to be affected too, as is the code in the tutorial which introduces setters.
Author: Georg Brandl (georg.brandl) *
Date: 2012-07-28 09:19
OK, finally I think this can go in.
Author: Roundup Robot (python-dev) 
Date: 2012-07-28 17:11
New changeset 68e2690a471d by Richard Oudkerk in branch 'default': Issue #1692335: Move initial args assignment to BaseException.new http://hg.python.org/cpython/rev/68e2690a471d
Author: Georg Brandl (georg.brandl) *
Date: 2012-07-29 06:01
Richard, can the issue be closed?
Author: Richard Oudkerk (sbt) *
Date: 2012-07-29 12:24
Richard, can the issue be closed?
I guess so (although the change could arguably be back ported).
Pickling of Exception classes is still somewhat dodgy because an example like
class Error(Exception):
def __init__(self, x):
Exception.__init__(self)
self.x = xis still not picklable.
Any further improvements can be discussed in a separate issue, so I will close.
Author: Georg Brandl (georg.brandl) *
Date: 2012-07-29 17:07
Please reopen if you think it should be backported.
Author: Roundup Robot (python-dev)
Date: 2013-02-27 14:04
New changeset 2c9f7ed28384 by R David Murray in branch '3.2': #17296: backport fix for issue 1692335, naive exception pickling. http://hg.python.org/cpython/rev/2c9f7ed28384
New changeset 67c27421b00b by R David Murray in branch '3.3': Null merge for issue 1692335 backport. http://hg.python.org/cpython/rev/67c27421b00b
New changeset 94f107752e83 by R David Murray in branch 'default': Null merge for issue 1692335 backport. http://hg.python.org/cpython/rev/94f107752e83
Author: Ofer (slallum)
Date: 2018-01-28 11:43
Perhaps this is a problem for a different issue, but pickling custom exceptions fails when the exception gets more than one argument:
import pickle class MultipleArgumentsError(Exception): def init(self, a, b): self.a = a self.b = b Exception.init(self, a)
pickle.loads(pickle.dumps(MultipleArgumentsError('a', 'b')))
this code produces the following error:
Traceback (most recent call last): File "/tmp/multiple_arguments_exception.py", line 8, in pickle.loads(pickle.dumps(MultipleArgumentsError('a', 'b'))) File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 1388, in loads return Unpickler(file).load() File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 864, in load dispatchkey File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 1139, in load_reduce value = func(*args) TypeError: init() takes exactly 3 arguments (2 given)
Author: Jason R. Coombs (jaraco) *
Date: 2018-01-28 15:04
@slallum, That does seem to be a problem, though I do observe that the issue reported by bdb with CalledProcessError is no longer an issue:
import subprocess, pickle try: ... subprocess.check_call(['python', '-c', 'raise SystemExit(1)']) ... except Exception as e: ... pickle.loads(pickle.dumps(e)) ... CalledProcessError(1, ['python', '-c', 'raise SystemExit(1)'])
Looking into how CalledProcessError is defined (https://github.com/python/cpython/blob/79db11ce99332d62917be9d03b31494b1ff2f96a/Lib/subprocess.py#L60) may shed some light on the recommended way to make a pickleable Exception class that takes more than one argument.
Hmm. It seems it does it by not calling the superclass init. Indeed, following that model it seems to work:
import pickle class MultipleArgumentsError(Exception): def init(self, a, b): self.a = a self.b = b
err = pickle.loads(pickle.dumps(MultipleArgumentsError('a', 'b'))) assert err.a == 'a' assert err.b == 'b'
Author: Ofer (slallum)
Date: 2018-01-28 15:37
@jason.coombs as far as I can tell, this only works in python3, but not in python2, where it still produces the same error.
Author: Guido van Rossum (gvanrossum) *
Date: 2018-01-28 15:51
Please don't discuss on closed issues. Open a new issue.
History
Date
User
Action
Args
2022-04-11 14:56:23
admin
set
github: 44791
2018-10-11 14:43:02
orivej
set
nosy: + orivej
2018-01-28 15:51:04
gvanrossum
set
messages: +
2018-01-28 15:37:56
slallum
set
messages: +
versions: + Python 2.7, - Python 3.3
2018-01-28 15:04:51
jaraco
set
messages: +
2018-01-28 11:43:18
slallum
set
nosy: + slallum
messages: +
2013-02-27 14:04:40
python-dev
set
messages: +
2012-07-29 17:07:07
georg.brandl
set
messages: +
2012-07-29 12:24:55
sbt
set
status: open -> closed
resolution: fixed
messages: +
2012-07-29 06:01:43
georg.brandl
set
priority: release blocker -> critical
messages: +
2012-07-28 20:01:01
lukasz.langa
set
assignee: lukasz.langa -> sbt
stage: patch review -> commit review
versions: - Python 2.7, Python 3.2
2012-07-28 17:11:07
python-dev
set
nosy: + python-dev
messages: +
2012-07-28 09:19:06
georg.brandl
set
messages: +
2012-07-27 20:16:23
sbt
set
files: + init_args.patch
messages: +
2012-07-27 19:12:28
pitrou
set
messages: +
2012-07-27 19:00:40
sbt
set
messages: +
2012-07-27 17:47:45
pitrou
set
messages: +
2012-07-27 17:42:59
pitrou
set
messages: +
2012-07-27 16:26:24
sbt
set
messages: +
2012-07-27 14:58:52
brett.cannon
set
priority: critical -> release blocker
nosy: + benjamin.peterson
messages: +
2012-07-27 12:11:12
pitrou
set
messages: +
2012-07-27 12:09:00
sbt
set
messages: +
2012-07-27 11:40:43
pitrou
set
messages: +
2012-07-27 11:01:26
sbt
set
files: + init_args.patch
2012-07-27 11:00:58
sbt
set
messages: +
2012-07-26 23:23:16
pitrou
set
messages: +
2012-07-26 23:16:38
sbt
set
messages: +
2012-07-26 21:34:22
brett.cannon
set
messages: +
2012-07-26 12:52:54
sbt
set
messages: +
2012-07-24 21:43:37
sbt
set
nosy: + sbt
messages: +
2012-03-11 10:17:28
eric.araujo
set
nosy: + pitrou
2012-03-08 22:01:28
lukasz.langa
set
assignee: lukasz.langa
nosy: + lukasz.langa
2012-01-16 15:17:58
zbysz
set
nosy: + zbysz
2012-01-13 20:53:28
georg.brandl
set
assignee: georg.brandl -> (no value)
2012-01-11 16:57:09
taleinat
set
nosy: + taleinat
messages: +
2012-01-11 08:01:58
fmitha
set
nosy: + fmitha
messages: +
2011-06-12 18:35:56
terry.reedy
set
versions: + Python 3.3, - Python 3.1
2011-04-12 13:52:51
bpb
set
nosy: + bpb
messages: +
2011-04-05 11:34:02
vstinner
set
nosy: + vstinner
2010-09-22 18:39:40
belopolsky
set
messages: +
2010-09-22 14:58:05
jaraco
set
messages: +
2010-09-22 13:39:06
jaraco
set
nosy: + jaraco
messages: +
2010-09-17 12:58:19
BreamoreBoy
set
versions: + Python 2.7, Python 3.2, - Python 2.6
2010-06-30 03:04:12
ehuss
set
messages: +
2010-06-29 22🔞58
belopolsky
set
nosy: + belopolsky
messages: +
2010-05-31 21:13:58
tseaver
set
files: + issue1692335-tests.patch
nosy: + tseaver
messages: +
2010-04-13 00:22:15
kylev
set
nosy: + kylev
messages: +
2009-05-15 02:22:51
ajaksu2
set
stage: patch review
type: behavior
versions: + Python 3.1, - Python 2.5
2008-11-18 23:07:21
ehuss
set
messages: +
2008-11-18 22:54:39
benjamin.peterson
set
nosy: - benjamin.peterson
2008-11-18 22:52:19
gvanrossum
set
status: closed -> open
resolution: out of date -> (no value)
messages: +
2008-11-18 22:06:07
ehuss
set
messages: +
2008-11-17 17:46:34
gvanrossum
set
status: open -> closed
resolution: out of date
2008-11-17 11:37:01
zseil
set
messages: +
2008-07-31 01:51:23
benjamin.peterson
set
nosy: + benjamin.peterson
messages: +
2008-02-04 19:44:49
alexandre.vassalotti
set
nosy: + alexandre.vassalotti
2008-02-03 19:04:04
loewis
set
nosy: + loewis
messages: +
2008-02-02 11:36:55
gvanrossum
set
messages: +
2008-02-02 08:07:22
jarpa
set
messages: +
2008-02-01 17:38:02
gvanrossum
set
nosy: + gvanrossum
messages: +
2008-01-19 14:03:52
facundobatista
set
nosy: + facundobatista
messages: +
2008-01-17 22:13:52
jarpa
set
files: + exception_pickling_26.diff
nosy: + jarpa
messages: +
2007-11-20 02:00:03
brett.cannon
set
nosy: + brett.cannon
2007-09-20 08:47:33
georg.brandl
set
messages: +
2007-09-17 10:01:29
jafo
set
assignee: georg.brandl
messages: +
nosy: + jafo
2007-08-28 08:40:38
jafo
set
messages: -
2007-08-28 08:40:18
jafo
set
messages: -
2007-08-23 17:29:54
georg.brandl
set
priority: normal -> critical
versions: + Python 2.6
messages: +
severity: normal -> major
title: Move initial args assignment to BaseException.__new__ -> Fix exception pickling: Move initial args assignment to BaseException.__new__
2007-08-23 17:28:00
georg.brandl
link
2007-08-23 17:24:47
georg.brandl
link
2007-04-01 13:46:15
zseil
create