[Python-Dev] Python Benchmarks (original) (raw)

M.-A. Lemburg mal at egenix.com
Fri Jun 2 23:20:06 CEST 2006

M.-A. Lemburg wrote:

That's why the timers being used by pybench will become a parameter that you can then select to adapt pybench it to the OS your running pybench on. Wasn't that decision a consequence of the problems found during the sprint? It's a consequence of a discussion I had with Steve Holden and Tim Peters: I believe that using wall-clock timers for benchmarking is not a good approach due to the high noise level. Process time timers typically have a lower resolution, but give a better picture of the actual run-time of your code and also don't exhibit as much noise as the wall-clock timer approach. Of course, you have to run the tests somewhat longer to get reasonable accuracy of the timings. Tim thinks that it's better to use short running tests and an accurate timer, accepting the added noise and counting on the user making sure that the noise level is at a minimum. I just had an idea: if we could get each test to run inside a single time slice assigned by the OS scheduler, then we could benefit from the better resolution of the hardware timers while still keeping the noise to a minimum. I suppose this could be achieved by: * making sure that each tests needs less than 10ms to run * calling time.sleep(0) after each test run Here's some documentation on the Linux scheduler: http://www.samspublishing.com/articles/article.asp?p=101760&seqNum=2&rl=1 Table 3.1 has the minimum time slice: 10ms. What do you think ? Would this work ?

I ran some tests related to this and it appears that provide the test itself uses less than 1ms, chances are high that you don't get any forced context switches in your way while running the test.

It also appears that you have to use time.sleep(10e6) to get the desired behavior. time.sleep(0) seems to receive some extra care, so doesn't have the intended effect - at least not on Linux.

I've checked this on AMD64 and Intel Pentium M. The script is attached - it will run until you get more than 10 forced context switches in 100 runs of the test, incrementing the runtime of the test in each round.

It's also interesting that the difference between max and min run-time of the tests can be as low as 0.2% on the Pentium, whereas the AMD64 always stays around 4-5%. On an old AMD Athlon, the difference rare goes below 50% - this might also have to do with the kernel version running on that machine which is 2.4 whereas the AMD64 and Pentium M are running 2.6.

Note that is needs to the resource module, so it won't work on Windows.

It's interesting that even pressing a key on your keyboard will cause forced context switches.

-- Marc-Andre Lemburg eGenix.com

Professional Python Services directly from the Source (#1, Jun 02 2006)

Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: microbench.py Url: http://mail.python.org/pipermail/python-dev/attachments/20060602/f8671cce/attachment.asc

More information about the Python-Dev mailing list