[Python-Dev] threadsafe patch for asynchat (original) (raw)

Andrew Bennetts andrew-pythondev at puzzling.org
Wed Feb 8 14:57:04 CET 2006

Donovan Baarda wrote:

On Wed, 2006-02-08 at 02:33 -0500, Steve Holden wrote: > Martin v. Löwis wrote: > > Tim Peters wrote: [...] > > What is the reason that people want to use threads when they can have > > poll/select-style message processing? Why does Zope require threads? > > IOW, why would anybody want a "threadsafe patch for asynchat"? > > > In case the processing of events needed to block? If I'm processing web > requests in an async* dispatch loop and a request needs the results of a > (probably lengthy) database query in order to generate its output, how > do I give the dispatcher control again to process the next asynchronous > network event? > > The usual answer is "process the request in a thread". That way the > dispatcher can spring to life for each event as quickly as needed.

I believe that Twisted does pretty much this with it's "deferred" stuff. It shoves slow stuff off for processing in a separate thread that re-syncs with the event loop when it's finished.

Argh! No. Threading is completely orthogonal to Deferreds.

Deferreds are just an abstraction for managing callbacks for an asychronous operation. They don't magically invoke threads, or otherwise turn synchronous code into asynchronous code for you.

This seems to be a depressingly common misconception. I wish I knew how to stop it.

They're much simpler than people seem to think. They're an object a function returns to say "I don't have a result for you yet, but if you attach callbacks to this I'll run those when I do." We've do this because it's much nicer than having to pass callbacks into functions, particularly when you want to deal with chains of callbacks and error handling.

There is a single utility function in Twisted called "deferToThread" that will run a function in a threadpool, and arrange for a Deferred to be fired with the result (in the event loop thread, of course). This is just one of many possible uses for Deferreds, and not an especially common one.

I'm happy to provide pointers to several Twisted docs if anyone is at all unclear on this.

While they are very useful, I don't think they're an essential part of a minimal Twisted replacement for asyncore/asynchat -- in fact, they'd work just fine with asyncore/asynchat, because they do so little.

-Andrew.

More information about the Python-Dev mailing list