[Python-Dev] Boom (original) (raw)

Tim Peters tim.one@comcast.net
Thu, 03 Apr 2003 23:08:54 -0500

While enduring dental implant surgery earlier today, I thought to myself "oops -- I bet this program will crash Python". Turns out it does, in current CVS, and almost certainly in every version of Python since cyclic gc was added:

""" import gc

class C: def getattr(self, attr): del self.attr raise AttributeError

a = C() b = C() a.attr = b b.attr = a

del a, b gc.collect() """

Short course: a and b are in a trash cycle. gcmodule's move_finalizers() finds one of them and calls has_finalizer() to see whether it's collectible. Say it's b. has_finalizer() calls (in effect) hasattr(b, "del"), and b.getattr() deletes b.attr as a side effect before saying b.del doesn't exist. That drops the refcount on a to 0, which in turn drops the refcount on a.dict to 0. Those two are the killers: a and a.dict become untracked (by gc) as part of cleaning them up, but the move_finalizers() "next" local still points to one of them (to the dict, in the run I happened to step thru). As a result, the next trip around the move_finalizer() loop calls has_finalizer() on memory that's already been free()ed. Hilarity ensues.

The anesthesia is wearing off and I won't speculate about solutions now. I suspect it's easy, or close to intractable. PLabs folks, I'm unsure whether this relates to the ZODB test failure we've been bashing away at. All, ZODB is a persistent database, and at one point in this test gc determines that "a ghost" is unreachable. When gc's has_finalizer() asks whether the ghost has a del method, the persistence machinery kicks in, sucking the ghost's state off of disk, and executing a lot of Python code as a result. Part of the Python code executed does appear (if hazy memory serves) to delete some previously unreachable objects that were also in (or hanging off of) the ghost's cycle, and so in the unreachable list gc's move_finalizers() is crawling over.

The kind of blowup above could be one bad effect, and Jeremy was seeing blowups with move_finalizers() in the traceback. Unfortunately, the test doesn't blow up under CVS Python, and 2.2.2 doesn't have the telltale 0xdbdbdbdb filler 2.3's debug PyMalloc sprays into free()ed memory.