[Python-Dev] Thoughts fresh after EuroPython (original) (raw)

Guido van Rossum guido at python.org
Mon Jul 26 16:36:13 CEST 2010

On Mon, Jul 26, 2010 at 3:00 AM, Michael Foord <fuzzyman at voidspace.org.uk> wrote:

At Resolver Systems we created a "calculation system" that does large calculations on background threads using IronPython. Doing them on a background thread allows the ui to remain responsive. Several calculations could run simultaneously using multiple cores.

As the calculation operates on a large object graph (which the ui then needs access to in order to display it) using multiprocessing would have imposed a very big overhead due to serialization / deserialization (the program runs on windows). Using CPython would have made the program a lot slower due to the GIL.

Sure. Note that using threads with the GIL, it is not a problem to keep the UI responsive even if background calculations are going on (at worst it requires some tweaking of sys.setcheckinterval() or its new-GIL equivalent). However with the GIL multiple calculations would be limited to a single core.

According to CSP advicates, this approach will break down when you need more than 8-16 cores since cache coherence breaks down at 16 cores. Then you would have to figure out a message-passing approach (but the messages would have to be very fast).

-- --Guido van Rossum (python.org/~guido)

More information about the Python-Dev mailing list