[Python-Dev] collections module (original) (raw)

Kurt B. Kaiser kbk at shore.net
Wed Jan 14 15:04:15 EST 2004

Guido van Rossum <guido at python.org> writes:

before: 15773 pystones, 29.9 mPB after: 12690 pystones, 31.4 mPB What's the mPB number? If its the time to complete parrotbench, this is actually a slowdown, just as for pystone.

Switch back to CVS:

19531, 19379, 19531, 19379, 19305, 19084 pystones, 31.4 mPB Switch back to Tim's patch: 15015, 15480, 14588 pystones should not need, but: make clean && make (just make on previous, only listobject.c being recompiled) Still patched: 13158, 14006, 13193; restart python; 15923, 16722, 16023 restart python; 12953, 13698, 12953, 13513 Well, pystones are all over the map on this box. But the improvement is there for punchit and the parrotbench benchmark is solid across the change. Unclear. Please clarify mPB.

milliParrotBenches/sec. 1000/(time to complete Parrotbench [sec])

In a few years we'll be spec'ing kPB :-)

So the data so far is 29.9 mPB unpatched 31.4 mPB patched 31.4 mPB unpatched This morning: 30.7, 30.4, 30.1, 32.1, 32.2, 31.9 mPB patched (using make time)

rm Objects/listobject.c && cvs up Objects/listobject.c make clean && make

30.1, 30.2, 30.3, 32.1, 29.6, 29.9, 28.2 mPB unpatched

Average unpatched: 30.2 mPB Average patched: 31.3 mPB

So a 3 - 4% improvement, maybe. Needs more data.

I reported the equivalent of 31.1 mPB on 2Jan04, which was the best of several trials. The 'best of 3' pystones at that time was 17182.

Pystone

My impression is that Pystone isn't a good benchmark for this box unless run many times over a long period with python restarts, perhaps a cron at night. It may be the 128kB cache, but the results are very variable.

This morning, with patched listobject.c, it's 15015, 15060, 14881; restart python; 16891, 17182, 17361 restart python; 14925, 14970, 14925, 14492, 14662, 14836, 15060, 15106, 15060 append sys.path and import a module: 13889, 13927, 14084 do some stuff, then: 14662, 14368, 14577 restart python; 14836, 14881, 14836 restart python; 15337, 15337, 15337, 15773, 15873, 15674, 15479, 15432, 14792

Following the switch back to unpatched mentioned above: 18182, 18050, 18248, 18450, 18248, 18181 restart python; 14881, 14970, 14705, 14836, 14749, 14970 restart python; 16835, 16835, 16778, 17064, 17182, 16892 restart python; append sys.path, import module; 20000, 20408, 20080, 20808, 19763, 20000 do some stuff; 19305, 19762, 19607, 19084, 19455, 19455 restart python; 17605, 17731, 17482, 17544, 17544, 17483

On these two sets:

Average unpatched: 17865 pystones Average patched: 15136 pystones

A 15% reduction with the patch.

This may be significant, but requires a lot more data here and investigation by others. On this box the pystone rating jumps between python restarts.

I haven't calculated any more advanced statistics; IMHO with small data sets, especially jumpy, ragged ones like these, you can often get a better feel by just eyeballing it. The downside being, you tend to see what you want to see....

KBK

More information about the Python-Dev mailing list