[Python-Dev] A proposed solution for Issue 502236: Asyncrhonousexceptions between threads (original) (raw)

Robert Brewer fumanchu at aminus.org
Sat Jul 12 19:08:31 CEST 2008

Josiah Carlson wrote:

This doesn't need to be an interpreter thing; it's easy to implement by the user (I've done it about a dozen times using a single global flag). If you want it to be automatic, it's even possible to make it happen automatically using sys.settrace() and friends (you can even make it reasonably fast if you use a C callback).

Agreed. If someone wants a small library to help do this, especially in web servers, the latest version of Cherrpy includes a 'process' subpackage under a generous license. It does all the things Andy describes via a Bus object:

Andy Scott wrote: > 1. Put in place a new function call sys.exitapplication, what this > would do is: > a. Mark a flag in t0's data structure saying a request to > shutdown has been made

This is bus.exit(), which publishes a 'stop' message to all subscribed 'stop' listeners, and then an 'exit' message to any 'exit' listeners.

> b. Raise a new exception, SystemShuttingDown, in t1.

That's up to the listener.

> 2. As the main interpreter executes it checks the "shutting down > flag" in the per thread data and follows one of two paths: > If it is t0: > a. Stops execution of the current code sequence > b. Iterates over all extant threads ... > c. Enters a timed wait loop where it will allow the other > threads time to see the signal. It will iterate this loop > a set number of times to avoid being blocked on any given > thread.

This is implemented as [t.join() for t in threading.enumerate()] in the main thread.

> d. When all threads have exited, or been forcefully closed, > raise the SystemShuttingDown exception

The bus just lets the main thread exit at this point.

> P1. If the thread is in a tight loop will it see the exception? Or > more generally: when should the exception be raised?

That's dependent enough on what work the thread is doing that a completely generic approach is generally not sufficient. Therefore, the process.bus sends a 'stop' message, and leaves the implementation of the receiver up to the author of that thread's logic. Presumably, one wouldn't register a listener for the 'stop' message unless one knew how to actually stop.

> P2. When should the interpreter check this flag? > > I think the answer to both of these problems is to check the flag, > and hence raise the exception, in the following circumstances: > - When the interpreter executes a back loop. So this should catch > the jump back to the top of a "while True:" loop > - Just before the interpreter makes a call to a hooked in non- > Python system function, e.g. file I/O, networking &c.

This is indeed how most well-written apps do it already.

> Checking at these points should be the minimal required, I think, to > ensure that a given thread can not ignore the exception. It may be > possible, or even required, to perform the check every time a Python > function call is made.

PLEASE don't make Python function calls slower.

> 1. The Python interpreter has per thread information. > 2. The Python interpreter can tell if the system, t0, thread is > running. > 3. The Python engine has (or can easily obtain) a list of all > threads it created. > 4. It is possible to raise exceptions as the byte code is executing.

Replace 'Python interpreter' with 'your application' and those become relatively simple architectural issues: maintain a list of threads, have them expose an interface to determine if they're running, and make them monitor a flag to know when another thread is asking them to stop.

Robert Brewer fumanchu at aminus.org

More information about the Python-Dev mailing list