[Python-Dev] Exploration PEP : Concurrency for moderately massive (4 to 32 cores) multi-core architectures (original) (raw)

Krishna Sankar ksankar at doubleclix.net
Wed Sep 19 23:00:48 CEST 2007

Steve,

Thanks.
a)   Yep, SMP for now. Agreed on the need for asymmetric

architectures like cell-processor. We need to start somewhere and then can extend to more exotic realms. b) Yep, need to scale to arbitrary number of cores. But as a start, I wanted to differentiate from massive parallelism. c) Yep, we can have message passing semantics at the interface level and then underneath share the memory (even optimize with the copy on write patter). I was thinking that we would need to cross process space; for example federate 8 separate py processes (in an 8 core machine) and have a shared data path between them, based on shared memory allocated at configuration time.

Cheers Steven H. Rogers wrote:

Krishna Sankar wrote:

Folks, As a follow-up to the py3k discussions started by Bruce and Guido, I pinged Brett and he suggested I submit an exploratory proposal. Would appreciate insights, wisdom, the good, the bad and the ugly.

A) Does it make sense ? B) Which application sets should we consider in designing the interfaces and implementations C) In this proposal, parallelism and concurrency are used in an interchangeable fashion. Thoughts ? D) Please suggest pertinent links, discussions and insights. E) I have kept the proposal to a minimum to start the discussions and to explore if this is the right thing to do. Collaboratively, as we zero-in on one or two approaches, the idea is to expand it to a crisp and clear PEP. Need to do some more formatting as well. Cheers P.S : I had sent this to python-ideas couple of days ago and received two comments (Thanks Leonardo, Thanks Adam) I haven't incorporated their comments yet. Folks who are on both lists, pardon me for the spam. # Proto-PEP elided. Other than number of cores, you don't mention hardware architecture. I presume that you're thinking of symmetric multiprocessor architectures. If so, this should be explicit. You should also consider that SMP may not be the predominant multi-core architecture in the future, the Cell processor has one general purpose processor and eight more specialized processors. You might not want to limit the PEP to 32 cores, I know of startups working on 40 and 64 core chips. Shared memory may be necessary for good performance, but it doesn't have to be exposed at the language level. While Erlang has strictly message passing semantics, I believe that it uses shared memory in the low level implementation. # Steve

Python-Dev mailing list Python-Dev at python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ksankar%40doubleclix.net

More information about the Python-Dev mailing list