[Python-Dev] Pythonic concurrency (original) (raw)

Jim Fulton jim at zope.com
Fri Oct 7 23:07:58 CEST 2005

Shane Hathaway wrote:

Antoine Pitrou wrote:

A relational database has a very strict and regular data model. Also, it has transactions. This makes it easy to precisely define concurrency at the engine level.

To apply the same thing to Python you would at least need : 1. a way to define a subset of the current bag of reachable objects which has to stay consistent w.r.t. transactions that are applied to it (of course, you would have several such subsets in any non-trivial application) 2. a way to start and end a transaction on a bag of objects (begin / commit / rollback) 3. a precise definition of the semantics of "consistency" here : for example, only one thread could modify a bag of objects at any given time, and other threads would continue to see the frozen, stable version of that bag until the next version is committed by the writing thread For 1), a helpful paradigm would be to define an object as being the "root" of a bag, and all its properties would automatically and recursively (or not ?) belong to this bag. One has to be careful that no property "leaks" and makes the bag become the set of all reachable Python objects (one could provide a means to say that a specific property must not be transitively put in the bag). Then, use myobject.begintransaction() and myobject.committransaction(). The implementation of 3) does not look very obvious ;-S Well, I think you just described ZODB. ;-) I'd be happy to explain how ZODB solves those problems, if you're interested. However, ZODB doesn't provide locking, and that bothers me somewhat. If two threads try to modify an object at the same time, one of the threads will be forced to abort, unless a method has been defined for resolving the conflict. If there are too many writers, ZODB crawls. ZODB's strategy works fine when there aren't many conflicting, concurrent changes, but the complex locking done by relational databases seems to be required for handling a lot of concurrent writers.

I don't think it would be all that hard to use a locking (rather than a time-stamp) strategy for ZODB, although ZEO would make this extra challenging.

In any case, the important thing to agree on here is that transactions provide a useful approach to concurrency control in the case where

Someone else pointed out essentially the same thing at the beginning of this thread.

Jim

-- Jim Fulton mailto:jim at zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org

More information about the Python-Dev mailing list