[Python-Dev] Software Transactional Memory for Python (original) (raw)

Guido van Rossum guido at python.org
Sun Aug 28 18:43:33 CEST 2011

On Sat, Aug 27, 2011 at 6:08 AM, Armin Rigo <arigo at tunes.org> wrote:

Hi Nick,

On Sat, Aug 27, 2011 at 2:40 PM, Nick Coghlan <ncoghlan at gmail.com> wrote: 1. How does the patch interact with C code that explicitly releases the GIL? (e.g. IO commands inside a "with atomic:" block) As implemented, any code in a "with atomic" is prevented from explicitly releasing and reacquiring the GIL: the GIL remain acquired until the end of the "with" block.  In other words PyBEGINALLOWTHREADS has no effect in a "with" block.  This gives semantics that, in a full multi-core STM world, would be implementable by saying that if, in the middle of a transaction, you need to do I/O, then from this point onwards the transaction is not allowed to abort any more.  Such "inevitable" transactions are already supported e.g. by RSTM, the C++ framework I used to prototype a C version (https://bitbucket.org/arigo/arigo/raw/default/hack/stm/c ).

2. Whether or not Jython and IronPython could implement something like that, since they're free threaded with fine-grained locks. If they can't then I don't see how we could justify making it part of the standard library. Yes, I can imagine some solutions.  I am no Jython or IronPython expert, but let us assume that they have a way to check synchronously for external events from time to time (i.e. if there is some equivalent to sys.setcheckinterval()).  If they do, then all you need is the right synchronization: the thread that wants to start a "with atomic" has to wait until all other threads are paused in the external check code.  (Again, like CPython's, this not a properly multi-core STM-ish solution, but it would give the right semantics.  (And if it turns out that STM is successful in the future, Java will grow more direct support for it )) A bientôt, Armin.

This sounds like a very interesting idea to pursue, even if it's late, and even if it's experimental, and even if it's possible to cause deadlocks (no news there). I propose that we offer a C API in Python 3.3 as well as an extension module that offers the proposed decorator. The C API could then be used to implement alternative APIs purely as extension modules (e.g. would a deadlock-detecting API be possible?).

I don't think this needs a PEP, it's not a very pervasive change. We can even document the API as experimental. But (if I may trust Armin's reasoning) it's important to add support directly to CPython, as currently it cannot be done as a pure extension module.

-- --Guido van Rossum (python.org/~guido)

More information about the Python-Dev mailing list