[Python-Dev] update on memory leaks in 2.2 (original) (raw)

Guido van Rossum guido@python.org
Fri, 07 Dec 2001 13:02:30 -0500

Guido van Rossum wrote:

> What's a potential leak? Could it be that these are in cycles? The > reported blocks are all weak reference objects; most of then are > allocated by addsubclass(), which uses weak refs for the list of > subclasses of a given (new-style) class. I suppose most weak refs are > in cycles (else there would be no need for weak refs).

Neal replied (in private email):

From page 157 of this doc:

http://www.rational.com/support/documentation/manuals/unixsuites/pdf/purify/purify.pdf A PLK message describes heap memory that you might have leaked. You have pointers only to the middle of the region. PLK is a warning message. Memory in use can sometimes appear as a PLK if the pointer returned by malloc is offset. A common cause is referencing a substring within a large string. Another example is when a pointer to a C++ object is cast to the second or later base class of a multiply-inherited object. It is offset past the other base class objects. Truly leaked memory can sometimes appear as a PLK, if some non-pointer integer within the program space, when interpreted as a pointer, points within an otherwise leaked block of memory. This is rather rare. Inspect the code to differentiate between these causes of PLK reports.

Thanks, that's very helpful! Here's how I now understand what's happened. Weak references are garbage collected, which means that the malloc pointer points 12 bytes before the start of the object as the rest of Python knows about it. Objects in the free list are linked together via their object pointer, so they are only known via a pointer into their "middle". We don't see this warning for other GC'ed objects, because they are in one of the collector's linked lists, which point to the start of the GC header which is the malloc'ed pointer. But freed weak references are not known to the collector!

So I'm confident that these are not leaks.

--Guido van Rossum (home page: http://www.python.org/~guido/)