[Python-Dev] recursive closures - reference cycle (original) (raw)

"Martin v. Löwis" martin at v.loewis.de
Wed Dec 9 23:37:11 CET 2009

Yes, and a number of different workarounds. That's not really the issue. The issue is that what looks like a perfectly safe idiom (calling a function recursively) happens to create a reference cycle if that function is a closure. This is a non-obvious "gotcha" that I must now educate my team about.

I'm not sure why you call that non-obvious. As Antoine(?) pointed out, any recursive function calls a reference cycle, and that is not surprising, because the function refers to itself, so the function definition is obviously cyclic (i.e. from within the function body, you can get back to the function object).

I know you dismissed the global function example as non-relevant, because the cycle is through the globals dictionary (i.e. any global function is in a cycle, whether it is recursive or not), and because it is possible to break the cycle (by deleting the function from the globals). However, that's beside the point, because a) it is an optimization that closures are only cyclic when they need to be (i.e. a nested function only keeps references to those name bindings that are mentioned in its body, rather than referencing the entire closure), and b) it is also possible to break the cycle in the closure case, by setting the cell target to None (say), and c) the actual problem that you are having is not that the nested functions create cycles, but that you get a fresh one every time you call the outer function (so that you get the impression of a memory leak).

Note that at least parts of the library strive to avoid reference cycles even though "gc" exists. An example being xlm.minidom.Document.unlink()

That doesn't avoid reference cycles. Instead, it helps breaking them (as a minidom tree is cyclic). minidom has this explicit API for breaking cycles because it predated the introduction of cyclic GC in Python.

Regards, Martin

More information about the Python-Dev mailing list