[Python-Dev] Re: [Python-checkins]python/dist/src/Modules

gcmodule.c,2.33.6.5,2.33.6.6 (original) (raw)

Jeremy Hylton jeremy@alum.mit.edu
05 Apr 2003 21:02:04 -0500

On Sat, 2003-04-05 at 14:34, Tim Peters wrote:

While a getattr side effect may resurrect an object in gc's unreachable list, gc has no way to know that an object has been resurrected short of starting over again. In the absence of that, the object remains in gc's unreachable list, and its tpclear slot eventually gets called. The internal C stuff remains self-consistent, so this won't cause a segfault (etc), but it may (as above) be surprising. I don't see a sane way to fix this so long as asking whether del exists can execute arbitrary mounds of Python code.

I think I'll second the thought that there are no satisfactory answers here. We've made a big step forward by fixing the core dumps.

If we want to document the current behavior, we would say that garbage collection may leave reachable objects in an "invalid state" in the presence of "problematic objects." A "problematic object" is an instance of a classic class that defines a getattr hook (getattr) but not a finalizer (del). An object an in "invalid state" has had its tp_clear slot executed; in the case of instances, this means the dict will be empty. Specifically, if a problematic object is part of unreachable cycle, the garbage collector will execute the code in its getattr hook; if executing that code makes any object in the cycle reachable again, it will be left in an invalid state.

If we document this for 2.2, it's more complicated because instances of new-style classes are also affected. What's worse, a new-style class with a getattribute hook is affected regardless of whether it has a finalizer.

Here are a couple of thoughts about how to avoid leaving objects in an invalid state. It's pretty unlikely for it to happen, but speaking from experience it's baffling when it does.

#1. (I think this was Fred's suggestion on Friday.) Don't do a hasattr() check on the object, do it on the class. This is what happens with new-style classes in Python 2.3: If a new-style class doesn't define an del method, then its instances don't have finalizer. It doesn't matter whether the specific instance has an del attribute.

Limitations: This is a change in semantics, although it only covers a nearly insane corner case. The other limitation is that things could still go wrong, although only in the presence of a classic metaclass!

#2. If an object has a getattr hook and it's involved in a cycle, just put it in gc.garbage. Forget about checking for a finalizer. That seems fine for 2.3, since we're only talking about classic classes with getattr hooks. But it doesn't sound very pleasant for 2.2, since it covers an class instance with a getattr hook.

I think #1 is pretty reasonable. I'd like to see something fixed for 2.2.3, but I worry that the semantic change may be unacceptable for a bug fix release. (But maybe not, the semantics are pretty insane right now :-).

Jeremy