[Python-Dev] instancemethod_getattro seems to be partially wrong (original) (raw)

Guido van Rossum guido at python.org
Sun Nov 23 20:26:09 EST 2003

Arghh! This is in fact harder than I was aware of. You have a setter, for its existance, although it won't set, for the readonly flag. Without criticism, you are for sure not finally happy with the solution, which sounds more like a working proof of concept than a real concept which you are happy to spread on the world. I'm better off to keep my hands off and not touch it now.

Actually, I like it fine. There really are four categories:

  1. not a descriptor
  2. overridable descriptor (used for methods) 2a) read-only non-overridable descriptor (used for read-only data) 2b) writable non-overridable descriptor (used for writable data)

Case (0) is recognized by not having get at all. Case (1) has get but not set. Cases (2a) and (2b) have get and set; case (2a) has a set that raises an exception. There are other (older) examples of setattr implementations that always raise an exception.

I patches pickle.py and cPickle.c to do essentially what Armin said: """ So I'm just saying that pickle.py in wrong in just one place:

reduce = getattr(obj, "reduce", None) if reduce: rv = reduce() should be: reduce = getattr(type(obj), "reduce", None) if reduce: rv = reduce(obj) """

Right. (That's what I was trying to say, too. :-)

An almost trivial change, although I also had to change copy.py, and overall I was unhappy since this extends my patch set to more than replacing python2x.dll, but I hope this will become an official patch and back-patch.

Give it to me baby. (On SF. :-)

> What's the reason for wanting to make cPickle non-recursive?

Several reasons. For one, the same reason why I started arguing about deeply recursive destruction code, and implemented the initial elevator destructor, you remember. (trashcan)

Yeah. Maybe I should get out of the serious language implementation business, because I still liked it better before. It may work, but it is incredibly ugly, and also had bugs for the longest time (and those bugs were a lot harder to track down than the bug it was trying to fix). With Tim's version I can live with it -- but I just don't like this kind of complexification of the implementation, even if it works better.

Same reason. When del crashes, cPickle will crash as well.

Please don't call it del. del is a user-level finalization callback with very specific properties and problems. You were referring to tp_dealloc, which has different issues.

Now that I can pickle tracebacks and very deep recursions, I don't want them to crash.

Several people asked on the main list, how to pickle deeply nested structures without crashing pickle. Well, my general answer was to rewrite pickle in a non-recursive manner.

I guess it's my anti-Scheme attitude. I just think the problem is in the deeply nested structures. There usually is a less nested data structure that doesn't have the problem. But I'll shut up, because this rant is not productive. :-(

On the other hand, my implementation for tracebacks and tasklets (with large chains of frames attached) was different: In order to avoid cPickle's shortcomings of recursion, I made the tasklets produce a list of all related frames, instead of having them refer to each other via fback. I did the same for tracebacks, by making the leading traceback object special, to produce a list of all other traceback objects in the chain.

Hey, just what I said. :-)

Armin once said, "rewrite the pickle code", which I'd happily do, but I do think, the above layout changes are not that bad, anyway. WHile frame chains and traceback chains are looking somewhat recursive, they aren't really. I think, they are lists/tuples by nature, and pickling them as that not only makes the result of reduce more readable and usable, but the pickle is also a bit shorter than that of a deeply nested structure.

Well, unclear. Frame chains make sense as chains because they are reference-counted individually.

Well, after Armin's input, I dropped my special case, and instead I will submit a patch for 2.2 and 2.3, which uses your proposed way to use reduce from pickle and copy. This is completely compatible and does what we want!

Wonderful! Please send me the SF issue, I don't subscribe to SF any more. (I've done my checkin in case you wondered.)

--Guido van Rossum (home page: http://www.python.org/~guido/)

More information about the Python-Dev mailing list