[Python-Dev] Extension modules, Threading, and the GIL (original) (raw)

Mark Hammond markh@skippinet.com.au
Thu, 9 Jan 2003 13:42:43 +1100

[Martin]

Mark Hammond wrote: > While I concede that this is likely to work in the general > case, I am not sure it is "correct". If no threading semantics > will be broken by having one thread use multiple thread-states, > then I must ask what purpose thread-states (as opposed to the GIL) have.

That is easy to answer (even though it is out of my area of expertise): it carries the Python stack, in particular for exceptions.

It also carries the profiler and debugger hooks, a general purpose "thread state" dictionary and other misc. details such as the tick count, recursion depth protection etc.

Now, if you have multiple thread states in a single thread, the question is how a Python exception should propagate through the C stack.

Actually, I think the question is still "why would a single thread have multiple thread-states?". (Or maybe "should a thread-state be renamed to, say, an "invocation frame"?)

With multiple thread states, the exception "drops off" in the callback, which usually

"usually" is the key word here. Python isn't designed only to handle what programs "usually" do. A strategy I have seen recently here, which is to argue that any other requirements are self-evidently broken, is not helpful.

We could possibly argue that exceptions are OK to handle this way. Similar amounts of text could also possibly convince that the profiler, debugger and thread-switch items also will not be too badly broken by having multiple thread states per thread, or that such breakage is "desirable" (ie, can be justified). You will have more trouble convincing me that future items stored in a Python thread state will not be broken, but I am past arguing about it.

Please correct me if I am wrong, but it seems your solution to this is:

If so, I am afraid I was hoping for just a little more .

Mark.