[Python-Dev] Debugging opportunity :-) (original) (raw)

Phillip J. Eby pje at telecommunity.com
Sat Apr 15 01:30:01 CEST 2006

At 01:11 AM 4/15/2006 +0200, Thomas Wouters wrote:

On 4/15/06, Thomas Wouters <<mailto:thomas at python.org>thomas at python.org> wrote:

(I first thought the problem was caused by gendealloc doing 'PyDECREF(gen->genframe)' instead of 'frame = gen->genframe; gen->genframe = NULL; PyDECREF(frame)', but that isn't the case. It should do it that way, I believe, but it's not the cause of this crash.) fixes the crash.Ah, found the problem. After I hit 'send', I realized I hadn't checked frameobject's tpclear, and sure enough, it calls PyXDECREF in-place. That explains why the first generator object gets dealloced twice, although it doesn't explain why it doesn't blow up when it reaches a negative refcount.

You're missing another piece of the puzzle: the problem is that since generators don't have a tp_clear, they don't know they're pointing to an invalid frame; it appears to still be running. So when the frame object releases items off the stack, it releases its reference to the generator, which is released normally, dropping out the local variable reference back to the generator whose frame is being cleared. That generator doesn't know it's holding a garbage frame, and thus proceeds to finalize it by resuming it... which then pops the blockstack, and f_stacktop is still valid, so popping the blockstack decrefs the frame a second time.

Fixing frameobject and genobject to both use PyCLEAR() makes both the 'minimal' testcase and testgenerators work.

It seems to me that frame_clear() should also set f_stacktop to NULL before doing any clearing; otherwise it's possible for a generator to think that the frame is still executable, and the double-decref could thus be replaced by a null pointer dereference.

More information about the Python-Dev mailing list