Issue 13479: pickle too picky on re-defined classes (original) (raw)

Created on 2011-11-25 09:02 by kxroberto, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Messages (4)
msg148311 - (view) Author: kxroberto (kxroberto) Date: 2011-11-25 09:02
When a class definition was re-executed (reload, exec ..) , pickling of existing instances fails for to picky reason (class object id mismatch). Solved by the one liner patch below. Rational: Python is dynamic. Like with any normal attribute lookup: When its the same module/..name this class is really meant - no matter about its id. (During unpickling its another id anyway, the code may have evolved years ...) diff -ur --strip _orig/pickle.py ./pickle.py --- _orig/pickle.py 2008-09-08 10:58:32 +0000 +++ ./pickle.py 2011-11-24 15:47:11 +0000 @@ -747,7 +747,7 @@ "Can't pickle %r: it's not found as %s.%s" % (obj, module, name)) else: - if klass is not obj: + if klass.__name__ != obj.__name__: raise PicklingError( "Can't pickle %r: it's not the same object as %s.%s" % (obj, module, name))
msg148337 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2011-11-25 16:58
Hm, this change allows many other *undesirable* objects pass the test as well. I'd prefer to stick to the rule, "when in doubt, raise an error". Maybe using == instead of 'is' as the test would be acceptable?
msg149223 - (view) Author: kxroberto (kxroberto) Date: 2011-12-11 11:03
Well, "==" whould allow the wanted feature by exception through meta classes for concerned classes: >>> class X: ... a=1 ... >>> Y=X >>> class X: ... a=1 ... >>> Y==X False >>> class XCompare(type): ... def __eq__(self, other): ... print "tolerant class __eq__" ... return self.__name__ == other.__name__ ... >>> class X: ... __metaclass__ = XCompare ... a=1 ... >>> Y=X >>> class X: ... a=1 ... >>> Y==X tolerant class __eq__ True >>> Better than nothing. Its a improvement generally, independently. But thinking about my acutal use cases and all: It still doesn't satisfy. I don't want to introduce this extra magic on all those classes just for that feature - because when needed, the majority of classes are concerned (see below). One can have only one meta class ... its too tricky and off-road to guess for most programmers ... "when in doubt, raise an error": That is IMHO too rigid here, and generally when a feature is then hindered too much. Aren't warnings the right tool for such case? If really rarely there is problem, should it surface easily already during dev & test time? Compared to the everday life danger of Pythons dynamic attribute access, version incompatibilities, etc. its about a rather harmless issue here. Now I'd vote for a warnings.warn upon "==" (or old "is") failing , and then an error only when the .__name__ is not matching too. A warning at dev & test time should be enough, when just "==" (or "is") fails. I mainly like the tolerance during development: e.g. fast reload style edit-run cycles (reload sometimes improved with special reload fix code), because I noticed that 95% of code changes/bug fixes do not require a full expensive app-restart. This pays off particularly with bigger GUI app development/fixing and similar, where lot of status is accumulated expensively during run time. But I wished that feature already for a deployed app too.
msg149250 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2011-12-11 20:18
What you're seeing here is just one of may things that go subtly wrong when you reload a class. I don't think we should fix this one aspect while leaving so many other bugs due to the same root cause. It would be better to focus your energy on a way to improve reloading, e.g. make it so that the identity of global functions and classes doesn't change when their module is reloaded. (You'll find it a tough problem, but note that it's been solved for at least one specific instance: modules *do* retain their identity, so maybe you can use that as a model.)
History
Date User Action Args
2022-04-11 14:57:24 admin set github: 57688
2011-12-11 20🔞30 gvanrossum set status: open -> closedresolution: wont fixmessages: +
2011-12-11 11:03:04 kxroberto set messages: +
2011-11-25 16:58:26 gvanrossum set nosy: + gvanrossummessages: +
2011-11-25 09:03:34 kxroberto set title: pickle to picky on re-defined classes -> pickle too picky on re-defined classes
2011-11-25 09:02:42 kxroberto create