[Python-3000] the future of the GIL (original) (raw)

Guido van Rossum guido at python.org
Tue May 8 17:37:44 CEST 2007

On 5/8/07, Luis P Caamano <lcaamano at gmail.com> wrote:

On 5/7/07, "Guido van Rossum" wrote: > Around '99 Greg Stein and Mark Hammond tried to get rid of the GIL. > They removed most of the global mutable data structures, added > explicit locks to the remaining ones and to individual mutable > objects, and actually got the whole thing working. Unfortunately even > on the system with the fastest locking primitives (Windows at the > time) they measured a 2x slow-down on a single CPU due to all the > extra locking operations going on.

That just breaks my heart. You gotta finish that sentence, it was a slow down on single CPU with a speed increase with two or more CPUs, leveling out at 4 CPUs or so. This was the same situation on every major OS kernel, including AIX, HPUX, Linux, Tru64, etc., when they started supporting SMP machines, which is why all of them at some time sported two kernels, one for SMP machines with the spinlock code and one for single processor machines with the spinlock code #ifdef'ed out. For some, like IBM/AIX and HPUX, eventually and as expected, all their servers became MPs and then they stopped delivering the SP kernel. The same would've been true for the python interpreter, one for MP and one for SP, and eventually, even in the PC world, everything would be MP and the SP interpreter would disappear.

The difference is, for an OS kernel, there really isn't any other way to benefit from multiple CPUs. But for Python, there is -- run multiple processes instead of threads!

People need to understand though that the GIL is not as bad as one would initially think as most C extensions release the GIL and run concurrently on multiple CPUs. It takes a bit of researching through old emails in the python list and a bit of time to really understand that. Nevertheless, when the itch is bad enough, it'll get scratched.

I think you're overestimating the sophistication of the average extension developer, and the hardware to which they have access.

Nevertheless, you're right the GIL is not as bad as you would initially think: you just have to undo the brainwashing you got from Windows and Java proponents who seem to consider threads as the only way to approach concurrent activities.

Just because Java was once aimed at a set-top box OS that didn't support multiple address spaces, and just because process creation in Windows used to be slow as a dog, doesn't mean that multiple processes (with judicious use of IPC) aren't a much better approach to writing apps for multi-CPU boxes than threads.

Just Say No to the combined evils of locking, deadlocks, lock granularity, livelocks, nondeterminism and race conditions.

-- --Guido van Rossum (home page: http://www.python.org/~guido/)

More information about the Python-3000 mailing list