[Python-Dev] Pythonic concurrency (original) (raw)

Guido van Rossum guido at python.org
Tue Oct 11 02🔞15 CEST 2005

On 10/10/05, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:

I'm wondering whether Python threads should be non-preemptive by default. Preemptive threading is massive overkill for many applications. You don't need it, for example, if you just want to use threads to structure your program, overlap processing with I/O, etc.

I recall using a non-preemptive system in the past; in Amoeba, to be precise.

Initially it worked great.

But as we added more powerful APIs to the library, we started to run into bugs that were just as if you had preemptive scheduling: it wouldn't always be predictable whether a call into the library would need to do I/O or not (it might use some sort of cache) so it would sometimes allow other threads to run and sometimes not. Or a change to the library would change this behavior (making a call that didn't use to block into sometimes-blocking).

Given the tendency of Python developers to build layers of abstractions I don't think it will help much.

Preemptive threading would still be there as an option to turn on when you really need it.

Or perhaps there could be a priority system, with a thread only able to be preempted by a thread of higher priority. If you ignore priorities, all your threads default to the same priority, so there's no preemption. If you want a thread that can preempt others, you give it a higher priority.

If you ask me, priorities are worse than the problem they are trying to solve.

-- --Guido van Rossum (home page: http://www.python.org/~guido/)

More information about the Python-Dev mailing list