[Python-Dev] [PEP 3148] futures - execute computations asynchronously (original) (raw)

Jeffrey Yasskin jyasskin at gmail.com
Sun Mar 7 19:59:50 CET 2010

On Sun, Mar 7, 2010 at 9:57 AM, P.J. Eby <pje at telecommunity.com> wrote:

At 08:39 AM 3/7/2010 -0800, Jeffrey Yasskin wrote:

Do you have an example of a language or library that uses the term "future" to refer to what you're talking about? I'm curious to see what it looks like. The wikipedia page menetioned earlier in the discussion has several examples.  Twisted's Deferreds are probably the best example of such a system in Python, though, as far as I know.  The features proposed in the PEP are basically akin to Twisted's deferToThread(), i.e. "return a future for running this code in some other thread".

Nearly all of the wikipedia page's examples refer to exactly this kind of system, but with syntax to support it instead of putting it in a library. The distinction between futures and deferreds is that futures are designed to block, while deferreds are designed to continue through callbacks. We could say that futures are to deferreds as threading.Lock is to mutex.mutex.

 That way you can write synchronous-looking code that nonetheless executes asynchronously.

"Asynchronous" in the PEP refers to how the task behind the future runs, not how users wait for the future. Is there a better way to say that?

Twisted uses Deferreds extensively, both for I/O and IPC, and of course for database access.  However, the 'defer' module itself is conceptually distinct from both the I/O system, event loop, and threads: it's a largely-generic framework for managing asynchronous computation.  Parallel task queueing is simply one possible application of Deferreds; their actual purpose is to provide a common API for working with values that aren't yet known.

That is to say, futures. In contrast, the PEP manages only a very limited sort of "future" - the completion of a single task, which must be synchronously waited for.  If one "future" in the PEP needs to wait for another, you tie up that thread or process in a busy loop...  which rather limits how much parallelism you can have.

So is it that you just don't like the idea of blocking, and want to stop anything that relies on it from getting into the standard library?

Given the set_result and set_exception methods, it's pretty straightforward to fill in the value of a future from something that isn't purely computational. Given a way to register "on-done" callbacks with the future, it would be straightforward to wait for a future without blocking, too.

Jeffrey

More information about the Python-Dev mailing list