[Python-Dev] Twisted Isn't Specific (was Re: Trial balloon: microthreads library in stdlib) (original) (raw)

"Martin v. Löwis" martin at v.loewis.de
Thu Feb 15 17:35:37 CET 2007

Larry Hastings schrieb:

Bob Ippolito wrote:

There is no single PerfectReactor. There are several use cases where you need to wait on >1 different event systems, which guarantees at least two OS threads (and two event loops). In general it's nice to have a single Python event loop ("the reactor") to act on said threads (e.g. something just sitting on a mutex waiting for messages) but waiting for IO to occur should probably happen on one or more ancillary threads -- one per event system (e.g. select, GTK, WaitForMultipleEvents, etc.) Why couldn't PerfectReactor be a reactor for other reactors? A sort of concentrator for these multiple event systems and multiple threads.

Because you can't write a single function that blocks (idle) until an event in one of the systems occurs. None of the system is truly asynchronous. All APIs, eventually, rely on synchronous, blocking, operating system calls. In Unix, the call to make is typically select or poll, and you know it is safe to make because you have all even sources as file descriptors and you know nothing else needs to be done before one of the events occurs.

Now, for these generalized event loops, it may not be possible anymore to combine all event sources into a single blocking call. If you have timeouts, you have to find out what the minimum timeout is (to make it the timeout for the blocking call, assuming that supports a timeout). If you have idle tasks, you have to invoke all idle tasks of all reactors (which you may not be able to get because there is no API to fetch them).

Even more difficult: if these use different OS event mechanisms (on systems that have multiple such systems). E.g. try combinging a Win32 GetMessage/PostMessage loop with a WaitForMultipleObjects loop with a WinSock select call; then add the Win32 RPC libraries to that (i.e. try to determine the named pipe handles you have to pass to WaitForMultipleObjects).

You ask to listen to sockets, so it instantiates a singleton PerfectReactor which instantiates a select() reactor and listens to it directly in a single-threaded manner. If you then ask to listen to Win32 messages, the PerfectReactor instantiates a GetMessage() reactor. Then, realizing it has two "event systems", it spawns a thread for each child reactor with a listener that serializes the incoming events into the PerfectReactor's queue.

Ah, threads :-( It turns out that you need to invoke GetMessage in the context of the thread in which the window was created. In a different thread, you won't get any messages. (This is indeed one of the biggest limitations of USER32, which was only overcome in Vista when the "native" GUI programming model is based on DirectX - the problem is then of course solved only for applications that use the DirectX UI API).

Bingo, your application doesn't need to be written thread-safe, PerfectReactor is platform-agnostic, and you don't have to know in advance all the event types you might ever listen to.

Sorry if this is a dumb question (or if I'm mangling the terminology),

Integrating with threads might be a solution in some cases, and a problem in others. You can't assume it is a universal solution.

Regards, Martin

More information about the Python-Dev mailing list