[Python-Dev] Thoughts fresh after EuroPython (original) (raw)

Ronald Oussoren ronaldoussoren at mac.com
Mon Jul 26 16:33:48 CEST 2010

On 26 Jul, 2010,at 12:00 PM, Michael Foord <fuzzyman at voidspace.org.uk> wrote:

On 26/07/2010 04:42, Guido van Rossum wrote:

On Sun, Jul 25, 2010 at 8:31 PM, Peter Portante <peter.a.portante at gmail.com> wrote:

FWIW: We use Python at Tabblo, straddled across Python 2.5.4 and 2.6.5. They work. And they work well. But we make light use of threads (mostly background I/O handling), and heavy use of multiple processes because we can't take advantage of our multi-core systems otherwise. Isn't this an indication that the GIL is, in fact, not (much of) a problem? I wish those trying to get rid of the GIL well But it may not be the panacea some folks are hoping for. Multi-threaded programming remains hard (and removing the GIL might actually make it harder). Jython and IronPython don't have a GIL, and I think PyPy may not either. Does anyone have experience with GIL-free programming in one of those? At Resolver Systems we created a "calculation system" that does large calculations on background threads using IronPython. Doing them on a background thread allows the ui to remain responsive. Several calculations could run simultaneously using multiple cores.

As the calculation operates on a large object graph (which the ui then needs access to in order to display it) using multiprocessing would have imposed a very big overhead due to serialization / deserialization (the program runs on windows).

Using CPython would have made the program a lot slower due to the GIL.  

I have a simular usecase, although in my case it's more using large blobs instead of complicated datastructures. I'm not hurt by the GIL because most threads run C code most of the time, which enables us to use multiple CPU cores without getting hurt by the GIL.

In my opinion the GIL is a weak point of CPython and it would be nice if it could be fixed.  That is however easier said than done, a number of people have tried in the past and ran into implementation limitations like our refcounting garbage collector that make hard to remove the GIL without either rewriting lots of code, or running into a brick wall performance-wise.

The HotPy presentation at EuroPython shows that it is possible to remove the GIL, although at the cost of replacing the garbage collector and most likely breaking existing C extensions (although the HotPy author seemed to have a possible workaround for that)

Ronald -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20100726/52cf2592/attachment.html>

More information about the Python-Dev mailing list