[Python-Dev] Tunable parameters in dictobject.c (was dictnotes.txt out of date?) (original) (raw)

Terry Reedy tjreedy at udel.edu
Thu Jun 14 04:22:00 CEST 2012

On 6/13/2012 9:15 PM, Raymond Hettinger wrote:

On Jun 13, 2012, at 2:37 PM, Mark Shannon wrote:

I think that for combined tables a growth factor of x2 is best, but I don't have any hard evidence to back that up. I believe that change should be reverted. You've undone work that was based on extensive testing and timings of many python apps. In particular, it harms the speed of building-up all large dictionaries, and it greatly harms apps with steady-size dictionaries with changing keys. The previously existing parameter were well studied and have been well-reviewed by the likes of Tim Peters. They shouldn't be changed without deep thought and study. Certainly, "I think a growth factor of x2 is best" is insufficient.

The use cases 'directly used dict that is possibly the primary data structure and possibly gigabytes in size' is sufficiently different from 'hidden instance dicts that are typically a small, constant size (for a given class) but possibly repeated a million times' are sufficiently different that different tuning parameters might be appropriate. So I agree with Raymond think you should leave the parameters for the standard dicts alone until there is good evidence for a change. (Good == apparent to more than just one person ;-).

If you improved the resize speed of standard dicts, great. However, that does not necessarily mean that the resize factor should change.

[clipped by Raymond]

I had gained the impression from comments on this list, comments on the tracker and the web in general that the memory consumption of CPython was more of an issue than its performance.

In 16 years, I seemed to have missed that. I assure you there are many who feel the opposite.

-- Terry Jan Reedy

More information about the Python-Dev mailing list