[Python-Dev] [Python-checkins] r43358 - python/trunk/Modules/itertoolsmodule.c (original) (raw)

Thomas Wouters thomas at python.org
Tue Mar 28 11:02:31 CEST 2006

On 3/27/06, Neal Norwitz <nnorwitz at gmail.com> wrote:

On 3/27/06, Thomas Wouters <thomas at python.org> wrote: > > The teeobject has GC (hence the word 'and' in 'itertools.tee and its > internal teedataobject' ;-) The problem with testgenerators is that this > also leaks: > > def leak(): > def gen(): > while True: > yield g > g = gen() > > leak() Please add a test for this to the leakers and remove the tee test in leakers if that no longer leaks.

I added this test, but I didn't remove test_tee, because it still leaks. I also checked in a test of code that used to leak, but doesn't anymore. The cycle this nested generator creates, which is also involved in the test_tee leak, is not cleanable by the cycle-gc, and it looks like it hasn't been since the yield-expr/coroutine patch was included in the trunk. I believe the culprit to be this part of that patch:

Index: Modules/gcmodule.c

--- Modules/gcmodule.c (revision 39238) +++ Modules/gcmodule.c (revision 39239) @@ -413,10 +413,8 @@ assert(delstr != NULL); return _PyInstance_Lookup(op, delstr) != NULL; }

}

/* Move the objects in unreachable with del methods into finalizers.

Now, reverting that part of the patch in revision 39239 triggers an assertion failure, but removing the assertion makes it work right; the above nested-generator cycle gets cleaned up again. Later in the trunk, the assertion was changed anyway, and it no longer fails if I just revert the gcmodule change. Of course, the reason the cycle is uncleanable is because generators grew a tp_del method, and that throws cycle-gc in a hissy fit; reverting the above patch just makes cycle-gc ignore the tp_del of heaptypes. I don't know enough about the cycle-gc to say whether that's good or bad, but not having generators be cleanable is not very good. For what it's worth, reverting the gcmodule patch doesn't seem to break any tests.

However, fixing both those things (itertools.tee lack of GC, and GC's inability to clean generators) still does not fix the 254 refleaks in test_generators. When I backport the itertools.tee-GC changes to r39238 (one checkin before coroutines), test_generators doesn't leak, neither the r39238 version of test_generators, nor the trunk version. One revision later, r39239, either test_generators leaks 15 references. 'Fixing' gcmodule, which fixes the nested-generator leak, does not change the number of leaks. And, as you all may be aware of, in the trunk, test_generators leaks 254 references; this is also the case with the gcmodule fix. So there's more huntin' afoot.

In the mean time, if people knowledgeable about the cycle-GC care to comment about the gcmodule change wrt. heaptypes, please do.

-- Thomas Wouters <thomas at python.org>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060328/c31c1407/attachment.htm

More information about the Python-Dev mailing list