[Python-Dev] PyFAQ: thread-safe interpreter operations (original) (raw)

Armin Rigo arigo at tunes.org
Tue Nov 21 13:08:37 CET 2006

Hi Martin,

On Mon, Nov 20, 2006 at 11:55:42PM +0100, "Martin v. L?wis" wrote:

In general, += isn't atomic: it may invoke add or iadd on the left-hand side, or radd on the right-hand side.

If you only look at the actual operation, the these aren't atomic:

x.field = y # may invoke setattr, may also be a property D[x] = y # may invoke x.hash, and x.eq

I think this list of examples isn't meant to be read that way. Half of them can invoke custom methods, not just the two you mention here. I think the idea is that provided only "built-in enough" objects are involved, the core operation described by each line works atomically, in the sense e.g. that if two threads do 'L.append(x)' you really add two items to the list (only the order is unspecified), and if two threads perform x.field = y roughly at the same time, and the type of x doesn't override the default setattr logic, then you know that the object x will end up with a 'field' that is present and has exactly one of the two values that the threads tried to put in.

Python programs rely on these kind of properties, and they are probably a good thing - at least, much better IMHO than having to put locks everywhere. I would even say that the distinction between "preventing the interpreter from crashing" and "getting sane results" is not really relevant. If your program doesn't crash the interpreter, but loose some append()s or produce similar nonsense if you forget a lock, then we get the drawbacks of the GIL without its benefits...

In practice, the list of operations that is atomic should (ideally) be documented more precisely -- one way to do that is to specify it at the level of built-in methods instead of syntax, e.g. saying that the method list.append() works atomically, and so does dict.setdefault() as long as all keys are "built-in enough" objects.

A bientot,

Armin

More information about the Python-Dev mailing list