[Python-Dev] Dictionary sparseness (original) (raw)

Guido van Rossum guido@python.org
Mon, 05 May 2003 07:47:39 -0400

> Hm. Maybe so, but it is a feature that there are no user controls > over dictionary behavior, based on the observation that for every user > who knows enough about the dict implementation to know how to tweak > it, there are at least 1000 who don't, and the latter, in their > ill-advised quest for more speed, will use the tweakage API to their > detriment.

Perhaps there should be safety-belts and kindergarten controls: d.pack(fat=False) --> None. Reclaims deleted entries. If optional fat argument is true, the internal size is doubled resulting in potentially faster lookups at the expense of slower iteration and more memory. This ought to be both safe and simple.

And a waste of time except in the most rare circumstances.

Raymond Hettinger

P.S. Also, I think it worthwhile to at least transform dictresize() into PyDictResize() so that C extensions will have some control. This would make it possible for us to add a single line making the builtin dictionary more sparse and providing a 75% first probe hit rate.

And that would give how much of a performance improvement of typical applications?

Sorry, I really think that you're complexificating APIs here without sufficient gain. I really value the work you've done on figuring out how to improve dicts, but I think you've come to know the code too well to see the other side of the coin.

--Guido van Rossum (home page: http://www.python.org/~guido/)