[Python-Dev] extremely slow exit for program having huge (45G) dict (python 2.5.2) (original) (raw)

Antoine Pitrou solipsis at pitrou.net
Sat Dec 20 22:13:11 CET 2008

Leif Walsh <leif.walsh gmail.com> writes:

It might be a semantic change that I'm looking for here, but it seems to me that if you turn off the garbage collector, you should be able to expect that either it also won't run on exit, or it should have a way of letting you tell it not to run on exit. [...]

I'm skeptical that it's a garbage collector problem. The script creates one dict containing lots of strings and ints. The thing is, strings and ints aren't tracked by the GC as they are simple atomic objects. Therefore, the /only/ object created by the script which is tracked by the GC is the dict. Moreover, since there is no cycle created, the dict should be directly destroyed when its last reference dies (the "del" statement), not go through the garbage collection process.

Given that the problem is reproduced on certain systems and not others, it can be related to an interaction between allocation patterns of the dict implementation, the Python memory allocator, and the implementation of the C malloc() / free() functions. I'm no expert enough to find out more on the subject.

More information about the Python-Dev mailing list