[Python-Dev] safety of Py_CLEAR and self (original) (raw)

Tim Peters tim.peters at gmail.com
Mon Feb 12 20:23:11 CET 2007

[Jeremy Hylton]

I was wondering today how I could convince myself that a sequence of PyCLEAR() calls in a tpclear function was safe. Take for example a really trivial sequence of code on frameclear():

PyCLEAR(f->fexctype); PyCLEAR(f->fexcvalue); PyCLEAR(f->fexctraceback); PyCLEAR(f->ftrace); We use PyCLEAR() so that multiple calls to frameclear() are safe. The first time this runs it sets exctype to NULL before calling DECREF. This guarantees that a subsequent frameclear() or framedealloc() won't attempt to DECREF it a second time. I think I understand why that's desireable and why it works. The primary risk is that via DECREF we may execute an arbitrary amount of Python code via weakref callbacks, finalizers, and code in other threads that gets resumed while the callbacks and finalizers are running. My question, specifically, then: Why it is safe to assume that f doesn't point to trash after a particular call to PyCLEAR()? Any particular call to PyCLEAR() could break the cycle that the object is involved in an lead to a call to framedealloc(). The frame could get returned to an obmalloc pool, returned to the OS, or just re-used by another object before we get back to PyCLEAR(). It seems like such behavior would be exceedingly unlikely, but I can't convince myself that it is impossible. Which is it: improbable or impossible? If it is only improbable, should we figure out how to write code that is safe against such an improbable event?

As Guido pointed out, tp_clear is called by gc from only one place, which sticks an incref/decref pair around the call so that the refcount on f can't fall to 0 while frame_clear() is active.

That doesn't mean frame_clear is always safe to call, it only means that gc's use of the tp_clear slot is safe. Nobody else "should be" calling frame_clear (and it so happens nothing else in the core does), but it's something to be dimly aware of. For example, IIRC, some of ZODB's C code internally invokes its XXX_clear() functions directly, as part of removing persistent object state (unrelated to object deallocation). Determining whether those kinds of uses are safe requires case-by-case analysis.

More information about the Python-Dev mailing list