[Python-Dev] PyBench DictCreation (was Re: Performance compares) (original) (raw)

Tim Peters tim.one@home.com
Fri, 18 May 2001 15:26:02 -0400

[MAL]

It [pybench] doesn't claim "typical use". pybench is aimed at finding out performance issues about hot-spots -- there's no such thing as a "typical program", so pybench gives you low level performance compares for very specific tasks, e.g. dictionary creation or for-loop performance.

I have found it to be rather successful at that. At least gives some good hints at where to look...

There must be a misunderstanding here. I understand and appreciate all that!

From the instant you created it, PyBench became the best performance canary we have ("canary" in the sense of bringing a bird into the coal mine with you, because when a potentially fatal buildup of gasses occurs, the canary will pass out before you even notice).

My point was that making a decision based solely on that PyBench happens to create millions of dicts of exactly size 3, and relatively few of any other size, would be crazy -- which I'm sure you understand and appreciate too.

... I found that instance dictionaries are usual within the 8 slot range. You normally have a few heavy wheight instances and many light wheight ones which only have two or three attributes in their instance dict.

Matches my observations too.

[on dict resize parameters]

Why not make those parameters macros at the top of dictobject.c which can then be tuned to whatever the programmer needs/wants ?!

Bad idea, IMO. If someone understands the dict implementation well enough to be competent to change these without, e.g., opening a door to infinite loops, then they already know where these parameters appear, and can change the hardcoded #s themselves. Thr max load factor simply wasn't intended to be adjustable; and if it were, it would be a per-dict decision.