[Python-Dev] C API for gc.enable() and gc.disable() (original) (raw)

Tim Peters tim.peters at gmail.com
Sun Jun 22 23:10:52 CEST 2008

[Antoine Pitrou]

Would it be helpful if the GC was informed of memory growth by the Python memory allocator (that is, each time it either asks or gives back a block of memory to the system allocator) ?

[Martin v. Löwis]

I don't see how. The garbage collector is already informed about memory growth; it learns exactly when a container object is allocated or deallocated. That the allocator then requests memory from the system only confirms what the garbage collector already knew: that there are lots of allocated objects. From that, one could infer that it might be time to perform garbage collection - or one could infer that all the objects are really useful, and no garbage can be collected.

Really the same conundrum we currently face: cyclic gc is currently triggered by reaching a certain /excess/ of allocations over deallocations. From that we /do/ infer it's time to perform garbage collection -- but, as some examples here showed, it's sometimes really the case that the true meaning of the excess is that "all the objects are really useful, and no garbage can be collected -- and I'm creating a lot of them".

pymalloc needing to allocate a new arena would be a different way to track an excess of allocations over deallocations, and in some ways more sensible (since it would reflect an excess of /bytes/ allocated over bytes freed, rather than an excess in the counts of objects allocated-over-freed regardless of their sizes -- an implication is, e.g., that cyclic gc would be triggered much less frequently by mass creation of small tuples than of small dicts, since a small tuple consumes much less memory than a small dict).

Etc. ;-)

More information about the Python-Dev mailing list