[Python-Dev] futures API (original) (raw)

Raymond Hettinger raymond.hettinger at gmail.com
Thu Dec 9 23:39:38 CET 2010

On Dec 9, 2010, at 9:02 AM, Brian Quinlan wrote:

On Dec 9, 2010, at 4:26 AM, Thomas Nagy wrote:

Hello,

I am looking forward to replacing a piece of code (http://code.google.com/p/waf/source/browse/trunk/waflib/Runner.py#86) by the futures module which was announced in python 3.2 beta. I am a bit stuck with it, so I have a few questions about the futures: 1. Is the futures API frozen? Yes.

Yes, unless the current API is defective in some way. A beta1 release is a chance for everyone to exercise the new API and discover whether it is problematic in any real world applications.

2. How hard would it be to return the tasks processed in an output queue to process/consume the results while they are returned? The code does not seem to be very open for monkey patching. You can associate a callback with a submitted future. That callback could add the future to your queue.

The would be a good example for the docs.

3. How hard would it be to add new tasks dynamically (after a task is executed) and have the futures object never complete? I'm not sure that I understand your question. You can submit new work to an Executor at until time until it is shutdown and a work item can take as long to complete as you want. If you are contemplating tasks that don't complete then maybe you could be better just scheduling a thread. 4. Is there a performance evaluation of the futures code (execution overhead) ? No. Scott Dial did make some performance improvements so he might have a handle on its overhead.

FWIW, the source code is short and readable. From my quick read, it looks to be a relatively thin wrapper/adapter around existing tools. Most of the work still gets done by the threads or processes themselves. Think of this as a cleaner, more centralized API around the current toolset -- there is no deep new technology under the hood.

Raymond

More information about the Python-Dev mailing list