[Python-Dev] Concurrency and Python (original) (raw)

James Y Knight foom at fuhm.net
Fri Jan 7 21:56:55 CET 2005

On Jan 7, 2005, at 2:44 PM, Edward C. Jones wrote:

Is Python a suitable language for concurrent programming?

Depends on what you mean. Python is not very good for shared-memory
concurrent programming. (That is, threads). The library doesn't have
enough good abstractions for locks/synchronization/etc, and, of course
the big issue of CPython only allowing one thread to execute bytecode
at a time.

At the moment, threads are the fad, but I don't believe that will be
scaling very well. As you scale up the number of CPUs, the amount of
time wasted on memory synchronization similarly goes up, until you're
wasting more time on memory consistency than doing actual work.

Thus, I expect the trend to be more towards async message passing
architectures (that is, multiple processes each with their own memory),
instead, and I think Python is about as good for that as any existing
language. Which is to say: reasonable, but not insanely great.

What is the state of programming theory for concurrency?

For an example of the kind of new language being developed around a
asynchronous message passing model, see IBM's poorly-named "X10"
language. I saw a talk on it and thought it sounded very promising.
What it adds over the usual message passing system is an easier way to
name and access remote data and to spawn parallel activities that
operates on that data. The part about arrays of data spread out over a
number of different "places" (roughly, a CPU and its own memory) and
how to operate on them I found especially interesting.

I tried to find their project website, but since their name conflicts
with the home automation system, it's hard to google for. Or perhaps
they don't have a website.

Short summary information: http://www.csail.mit.edu/events/eventcalendar/calendar.php? show=event&id=131

Talk slides: http://www.cs.ualberta.ca/~amaral/cascon/CDP04/slides/sarkar.pdf

More talk slides, and a video: http://www.research.ibm.com/vee04/video.html#sarkar "Vivek Sarkar, Language and Virtual Machine Challenges for Large-Scale
Parallel Systems"

James

More information about the Python-Dev mailing list