[Python-Dev] reference leaks, del, and annotations (original) (raw)
Thomas Wouters thomas at python.org
Fri Mar 31 15:40:47 CEST 2006
- Previous message: [Python-Dev] reference leaks, __del__, and annotations
- Next message: [Python-Dev] Class decorators
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
On 3/31/06, Jim Jewett <jimjjewett at gmail.com> wrote:
The checkins list has been struggling with generator reference leaks; the latest conclusion was that some are unavoidable because of del cycles.
That was Tim's conclusion, but I wasn't quite done thinking about it ;)
That sort of defeats the purpose of resource managers. (Yes, it can be worked around on a case-by-case basis.)
As I see it, part of the problem is that (1) When there is a cycle, python refuses to guess. (2) There is no way for a del method to hint at ordering constraints. (3) There is no lightweight version of del to say "I don't care about ordering constraints."
An additional (and much more complicating) problem is that del can (and is allowed to) revive 'self'. That means that halfway through your cleanup, however you decided to do it, you can suddenly find out that you shouldn't be cleaning up at all. And of course any given del can rely on all of the parts of the cycle, even if one of those parts claims it can safely break the cycle.
I think there are three different scenarios involving cycles and/or del methods:
An object may conciously create a cycle, and know how to resolve it. A 'breakcycle' method or such may be the right way to handle those cases. It would have to be pretty sure that no one outside itself can (or should) rely on the attribute or closure it breaks the cycle with, though. (Sensible exceptions ought to be fine, IMHO.)
An object may not care about cycles, but want to do some cleanup when it is deleted. The C types have 'tp_dealloc' for this, and that's what PyFile uses to close files. If you want to emulate this behaviour in Python, you are forced to use del, creating the unreclaimable cycle problem. Perhaps a dealloc class/staticmethod makes sense; it would be passed a dictionary of instancedata, but not 'self', so it can never revive 'self' and can be sanely used in cycles -- that is, some of its instancedata may still suddenly be None, but that's a problem with del methods and global variables, too.
An object may care about cycles, actually need a del method that can revive the object, but not have a sane way to say 'break the cycle at this point'. Generators may be an example of that kind of object, although I'm not sure: they could throw() an exception to end the cycle. I don't think we can reclaim cycles where none of the objects can fairly break the cycle.
-- Thomas Wouters <thomas at python.org>
Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20060331/624508fa/attachment.html
- Previous message: [Python-Dev] reference leaks, __del__, and annotations
- Next message: [Python-Dev] Class decorators
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]