[Python-Dev] [PEP 3148] futures - execute computations asynchronously (original) (raw)

Phillip J. Eby pje at telecommunity.com
Sat Mar 6 07:11:14 CET 2010

At 01:03 AM 3/5/2010, Brian Quinlan wrote:

Hi all,

I recently submitted a daft PEP for a package designed to make it easier to execute Python functions asynchronously using threads and processes. It lets the user focus on their computational problem without having to build explicit thread/process pools and work queues. The package has been discussed on stdlib-sig but now I'd like this group's feedback.

My immediate reaction is that this would be a lot more useful if it built on an API for coroutine yielding/interaction, similar to what's in say, Eventlet. That would seem to make it easier to write synchronous-looking code that operates on futures, and allow futures to be composed more cleanly.

ISTM that if futures were standardized before a coroutine API, it would lead to effective orphaning ala what happened with asyncore, especially since the new library is, well, new.

I'm somewhat concerned that, as described, the proposed API adds little over what's relatively easy to do with a mature coroutine framework like Eventlet, while at the same time creating yet another alternative (and mutually incompatible) event loop system in the stdlib, beyond the ones that are already in asyncore, tkinter, and the various SocketServer subclasses.

As far as naming goes, Twisted uses the term "Deferred" for this concept (and also has a very mature API for handling them).

And speaking of Twisted, it seems to me that the PEP would be much improved in general by learning from some of the lessons of other systems. I don't think that Java's example is really the best one to follow in this instance, compared to the many existing async frameworks that have Python-specific experience and APIs to learn from.

Or, to put it another way, something that worries me about this PEP is that nearly all of its Python-related citations are for discussions of futures, with the only previous Python implementation cited being a crude sketch of a cookbook recipe. The PEP also doesn't address questions of interoperability with existing solutions, compare features with them, or even so much as say, "There are other production implementations of this concept in Python, but we are going to pretend they don't exist." ;-)

More information about the Python-Dev mailing list