[Python-Dev] GIL, Python 3, and MP vs. UP (was Re: Variant of removing GIL.) (original) (raw)

Guido van Rossum guido at python.org
Mon Sep 19 02:50:13 CEST 2005

On 9/17/05, John J Lee <jjl at pobox.com> wrote:

Given the points you make, and the facts that both Python 3 and real problems with continuing to scale down semiconductor chip feature sizes are on the horizon, it seems that now would be an excellent time to start work on this, with the goal of introducing it at the same time as Python 3.

a. Python 3.0 will break lots of code anyway, so the extension module issue becomes far less significant. b. In x years time (x < 10?) it seems likely multiprocessor (MP) users will be in the majority. (As a result, the uniprocessor (UP) slowdown becomes less important in practice, and also Python has the opportunity of avoiding the risk of being sidelined by a real or perceived lack of MP performance.)

That assumes a very specific model for how all that MP power is going to be used. I personally don't think the threaded programming model as found in Java works all that well; without locks you end up with concurrent modification errors, with locks you get deadlocks and livelocks. A typical programmer has a hard enough time keeping track of a bunch of variables being modified by a single thread; add multiple threads acting simultaneously on the same variables to the mix, and it's a nightmare.

If my hunch is right, I expect that instead of writing massively parallel applications, we will continue to write single-threaded applications that are tied together at the process level rather than at the thread level.

c. Since time is needed to iron out bugs (and perhaps also to reimplememt some pieces of code "from scratch"), very early in the life of Python 3 seems like the least-worst time to begin work on such a change.

I realize that not all algorithms (nor all computational problems) scale well to MP hardware. Is it feasible to usefully compile both MP and a UP binaries from one Python source code base?

That's an understatement. I expect that most problems (even most problems that we will be programming 10-20 years from now) get little benefit out of MP.

(I'm also quite aware that the GIL does not prevent all means of achieving efficient use of multiprocessors. I'm just concious that different parellisation problems are presumably best expressed using different tools, and that Python 3 and increased prevalance of MP systems might tip the balance in favour of removing the GIL.)

Of course, it still takes a (anti-)hero to step forward and do the work...

Be my guest. Prove me wrong. Talk is cheap; instead of arguing my points (all of which can be argued ad infinitum), come back when you've got a working GIL-free Python. Doesn't have to be CPython-based -- C# would be fine too.

-- --Guido van Rossum (home page: http://www.python.org/~guido/)

More information about the Python-Dev mailing list