[Python-Dev] extremely slow exit for program having huge (45G) dict (python 2.5.2) (original) (raw)

Alexandre Vassalotti alexandre at peadrop.com
Sun Dec 21 00:40:25 CET 2008

[Sorry, for the previous garbage post.]

On Fri, Dec 19, 2008 at 6:29 PM, Mike Coleman <tutufan at gmail.com> wrote: I have a program that creates a huge (45GB) defaultdict. (The keys are short strings, the values are short lists of pairs (string, int).) Nothing but possibly the strings and ints is shared.

Could you give us more information about the dictionary. For example, how many objects does it contain? Is 45GB the actual size of the dictionary or of the Python process?

That is, after executing the final statement (a print), it is apparently spending a huge amount of time cleaning up before exiting.

Most of this time is probably spent on DECREF'ing objects in the dictionary. As other mentioned, it would useful to have self-contained example to examine the behavior more closely.

I have done 'gc.disable()' for performance (which is hideous without it)--I have no reason to think there are any loops.

Have you seen any significant difference in the exit time when the cyclic GC is disabled or enabled?

-- Alexandre

More information about the Python-Dev mailing list