[Python-ideas] Add faster locks to the threading module? (original) (raw)

Sturla Molden sturla at molden.no
Wed Jul 21 02:51:53 CEST 2010

Thread synchronization with threading.Lock can be expensive. But consider this: Why should the active thread need to aquire a mutex, when it already holds one? That would be the GIL.

Instead of acqiring a lock (and possibly inducing thread swiching etc.), it could just deny the other threads access to the GIL for a while. The cost of that synchronization method would be completely amortized, as check intervals happen anyway.

Here is what a very naïve implementation would look like in ctypes (real code could use C instead, or perhaps should not attempt this at all...):

from contextlib import contextmanager import ctypes _Py_Ticker = ctypes.c_int.in_dll(ctypes.pythonapi,"_Py_Ticker")

@contextmanager def threadsafe(): tmp = _Py_Ticker.value _Py_Ticker.value = 0x7fffffff yield _Py_Ticker.value = tmp

Now we can do this:

with threadsafe(): # the GIL is mine, # for as long as I want pass

The usecase for this "gillock" is about the same as for a spinlock C. We want synchronization for a breif period of time, but don't want the overhead of aquiring a mutex.

In Python this gillock has one big advantage over a spinlock: We don't have to wait, so we don't risk a tread switch on enter/aquire. But there can be only one instance of this lock, as there is only one GIL. That is the drawback compared to a spinlock.

Therefore I think both a spinlock and a gillock should be added to the threading module. These are synchronization methods that should be available.

P.S. A gillock working like the ctypes code above is of course very dangerous. If a C extension releases the GIL while _Py_Ticker is astronomic, we have a very bad situation... But real code could try to safeguard against this. E.g. Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS could be defined to do nothing if _Py_Ticker is above some ridiculous threshold.

P.P.S. Yes I know about the newgil. But I have not thought about how to achieve similar effect with that.

Sturla

-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-ideas/attachments/20100721/6125bd53/attachment.html>

More information about the Python-ideas mailing list