[Python-Dev] Removing the GIL (Me, not you!) (original) (raw)

Adam Olsen rhamph at gmail.com
Thu Sep 13 19:08:40 CEST 2007

On 9/13/07, Hrvoje Nikšić <hrvoje.niksic at avl.com> wrote:

On Thu, 2007-09-13 at 13:15 +0200, "Martin v. Löwis" wrote: > > To put it another way, would it actually matter if the reference > > counts for such objects became hopelessly wrong due to non-atomic > > adjustments? > > If they drop to zero (which may happen due to non-atomic adjustments), > Python will try to release the static memory, which will crash the > malloc implementation.

More precisely, Python will call the deallocator appropriate for the object type. If that deallocator does nothing, the object continues to live. Such objects could also start out with a refcount of sys.maxint or so to ensure that calls to the no-op deallocator are unlikely. The part I don't understand is how Python would know which objects are global/static. Testing for such a thing sounds like something that would be slower than atomic incref/decref.

I've explained my experiments here: http://www.artima.com/forums/flat.jsp?forum=106&thread=214235&start=30&msRange=15#279978

Basically though, atomic incref/decref won't work. Once you've got two threads modifying the same location the costs skyrocket. Even without being properly atomic you'll get the same slowdown on x86 (who's cache coherency is fairly strict.)

The only two options are: A) Don't modify an object on every incref/decref. Deletion must be delayed. This lets you share (thread-safe) objects. B) Don't share any objects. This is a process model (even if they're lightweight like erlang). For the near future, it's much easier to do this using real processes though.

Threading is much more powerful, but it remains to be proven that it can be done efficiently.

-- Adam Olsen, aka Rhamphoryncus

More information about the Python-Dev mailing list