[Python-Dev] Pickling instances of nested classes (original) (raw)

Walter Dörwald walter at livinglogic.de
Fri Apr 1 13:17:24 CEST 2005

Samuele Pedroni wrote:

[...] And having the full name of the class available would certainly help in debugging. that's probably the only plus point but the names would be confusing wrt modules vs. classes.

You'd propably need a different separator in repr. XIST does this:

from ll.xist.ns import html html.a.Attrs.href <attribute class ll.xist.ns.html:a.Attrs.href at 0x8319284>

My point was that enabling reduce hooks at the metaclass level has propably other interesting applications, is far less complicated than your proposal to implement, it does not further complicate the notion of what happens at class creation time, and indeed avoids the implementation costs (for all python impls) of your proposal and still allows fairly generic solutions to the problem at hand because the solution can be formulated at the metaclass level.

Pickling classes like objects (i.e. by using the pickling methods in their (meta-)classes) solves only the second part of the problem: Finding the nested classes in the module on unpickling. The other problem is to add additional info to the inner class, which gets pickled and makes it findable on unpickling.

If pickle.py is patched along these lines [*] (strawman impl, not much tested but testpickle.py still passes, needs further work to support reduceex and cPickle would need similar changes) then this example works:

class HierarchMeta(type): """metaclass such that inner classes know their outer class, with pickling support""" def new(cls, name, bases, dic): sub = [x for x in dic.values() if isinstance(x,HierarchMeta)]

I did something similar to this in XIST, but the problem with this approach is that in:

class Foo(Elm): pass

class Bar(Elm): Baz = Foo

the class Foo will get its outer set to Bar although it shouldn't.

[...] def reduce(cls): if hasattr(cls, 'outer'): return getattr, (cls.outer, cls.name) else: return cls.name

I like this approach: Instead of hardcoding how references to classes are pickled (pickle the name), deligate it to the metaclass.

BTW, if classes and functions are pickable, why aren't modules:

import urllib, cPickle cPickle.dumps(urllib.URLopener) 'curllib\nURLopener\np1\n.' cPickle.dumps(urllib.splitport) 'curllib\nsplitport\np1\n.' cPickle.dumps(urllib) Traceback (most recent call last): File "", line 1, in ? File "/usr/local/lib/python2.4/copy_reg.py", line 69, in _reduce_ex raise TypeError, "can't pickle %s objects" % base.name TypeError: can't pickle module objects

We'd just have to pickle the module name.

Bye, Walter Dörwald

More information about the Python-Dev mailing list