[Python-Dev] Removing the GIL (Me, not you!) (original) (raw)

Tony Nelson tonynelson at georgeanelson.com
Fri Sep 14 18:44:25 CEST 2007

At 1:51 AM -0500 9/14/07, Justin Tulloss wrote:

On 9/14/07, Adam Olsen <<mailto:rhamph at gmail.com>rhamph at gmail.com> wrote:

Could be worth a try. A first step might be to just implement the atomic refcounting, and run that single-threaded to see if it has terribly bad effects on performance. I've done this experiment. It was about 12% on my box. Later, once I had everything else setup so I could run two threads simultaneously, I found much worse costs. All those literals become shared objects that create contention. It's hard to argue with cold hard facts when all we have is raw speculation. What do you think of a model where there is a global "thread count" that keeps track of how many threads reference an object? Then there are thread-specific reference counters for each object. When a thread's refcount goes to 0, it decrefs the object's thread count. If you did this right, hopefully there would only be cache updates when you update the thread count, which will only be when a thread first references an object and when it last references an object.

It's likely that cache line contention is the issue, so don't glom all the different threads' refcount for an object into one vector. Keep each thread's refcounts in a per-thread vector of objects, so only that thread will cache that vector, or make refcounts so large that each will be in its own cache line (usu. 64 bytes, not too horrible for testing purposes). I don't know all what would be required for separate vectors of refcounts, but each object could contain its index into the vectors, which would all be the same size (Go Virtual Memory!).

I mentioned this idea earlier and it's growing on me. Since you've actually messed around with the code, do you think this would alleviate some of the contention issues?

Justin

Your idea can be combined with the maxint/2 initial refcount for non-disposable objects, which should about eliminate thread-count updates for them.

TonyN.:' <mailto:tonynelson at georgeanelson.com> ' <http://www.georgeanelson.com/> -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20070914/70897382/attachment.htm

More information about the Python-Dev mailing list