[Python-Dev] Pythonic concurrency - cooperative MT (original) (raw)

Guido van Rossum guido at python.org
Fri Sep 30 16:38:27 CEST 2005

On 9/30/05, Antoine Pitrou <solipsis at pitrou.net> wrote:

Le vendredi 30 septembre 2005 à 08:32 -0500, skip at pobox.com a écrit : > >> (C) That scheduler is non-preemptive. A single greedy generator can > >> starve all the others. > > Antoine> Instead of looking at this as a problem, you could look at it > Antoine> as a feature. > > Apple looked at it as a feature for years. Not anymore.

You are missing the context: - at the OS level, it is not good to have cooperative MT because any badly-written app (there are lots of ;-)) can cause lack of responsiveness for the whole machine - at the application level, cooperative MT can be good to enforce robust semantics without disrupting other apps The latter is what we are discussing AFAIK: a different concurrency scheme inside a single application - not accross the whole OS or desktop.

I like this analysis.

Actually, I think different concurrency schemes must be chosen and mixed depending on the semantics. For example, when you have a networked GUI app, it can be good to have the network in one preemptive thread (using e.g. Twisted) and the GUI in another preemptive thread (using GTK, wx...). This is because GUI and network have different latency characteristics and requirements. (of course, you can replace "preemptive thread" with "process" in the above description)

I'm skeptical of this. I think a traditional GUI+network app can combine the network and UI tasks in one process -- usually they interact somewhat and that's where the bugs show up. Also note that this is not an application area where MP is very interesting -- most of the time you're blocked waiting for the user or for a socket (or likely both :) so a single CPU is all you need. I've never heard someone complain that the GIL is in the way for these types of apps.

So whatever innovatice concurrency scheme Python may come out, it should still be mixable with more traditional concurrency schemes, because required properties vary wildly even inside a single app.

I don't think you've proved that yet.

-- --Guido van Rossum (home page: http://www.python.org/~guido/)

More information about the Python-Dev mailing list