[Python-Dev] CPython optimization: storing reference counters outside of objects (original) (raw)

Artur Siekielski artur.siekielski at gmail.com
Tue May 24 17:39:06 CEST 2011

2011/5/24 Sturla Molden <sturla at molden.no>:

Den 24.05.2011 11:55, skrev Artur Siekielski:

PYRO/multiprocessing proxies isn't a comparable solution because of ORDERS OF MAGNITUDE worser performance. You compare here direct memory access vs serialization/message passing through sockets/pipes. The bottleneck is likely the serialization, but only if you serialize large objects. IPC is always very fast, at least on localhost .

It cannot be "fast" compared to direct memory access. Here is a benchmark: summing numbers in a small list in a child process using multiprocessing "manager": http://dpaste.org/QzKr/ , and using implicit copy of the structure after fork(): http://dpaste.org/q3eh/. The first is 200 TIMES SLOWER. It means if the work finishes in 20 seconds using fork(), the same work will require more than one hour using multiprocessing manager.

If a database is too slow, I am rather sure you need something else than Python as well.

Disk access is about 1000x slower than memory access in C, and Python in a worst case is 50x slower than C, so there is still a huge win (not to mention that in a common case Python is only a few times slower).

Artur

More information about the Python-Dev mailing list