[Python-Dev] [PEP 3148] futures - execute computations asynchronously (original) (raw)

Dj Gilcrease digitalxero at gmail.com
Mon Mar 8 21:56:24 CET 2010

On Mon, Mar 8, 2010 at 12:04 PM, Dj Gilcrease <digitalxero at gmail.com> wrote:

A style I have used in my own code in the past is a Singleton class with register and create methods, where the register takes a name(string) and the class and the create method takes the name and *args, **kwargs and acts as a factory.

So I decided to play with this design a little and since I made it a singleton I decided to place all the thread/process tracking and exit handle code in it instead of having the odd semi-global scoped _shutdown, _thread_references, _remove_dead_thread_references and _python_exit objects floating around in each executor file, seems to work well. The API would be

from concurrent.futures import executors

executor = executors.create(NAME, *args, **kwargs) # NAME is 'process' or 'thread' by default

To create your own executor you create your executor class and add the following at the end

from concurrent.futures import executors, ExecutorBase class MyExecutor(ExecutorBase): ... executors.register(NAME, MyExecutor)

It checks to see if your executor is a subclass of ExecutorBase, but only does a UserWarning if it is not since you should know what methods are required to be an executor like object, so if you are not subclassing ExecutorBase you should suppress the UserWarning before you register you class and un-suppress it after

Some Helper Methods/Properties on the executors Singleton add_joinable_ref - This replaces the _thread_references.add system of tracking threads, and it allows for adding processes as well hence the joinable_ref name. It does check to make sure the ref being passes has a join method then creates a weakref to it and adds it to a set. For every thread or process your executor creates you should call this with so it will be tracked properly

cleanup_joinable_refs - This replaces the _remove_dead_thread_references that had to be written for each executor individually. This should be called periodically, currently it is only called when you create a new executor since it is a blocking method (it uses a thread lock to make sure the set of references does not change while it is discarding old ones)

shutdown - is a readonly property and replaces the _shutdown global var that had to be created for each executor individually, it is set in the executors destructor

del - replaces the _python_exit method that had to be written for each executor individually

If this API change isnt accepted its no big deal, since I am only changing it due to personal pet peeves and the only real issue I had with the package was scoping which has already been addressed by adding it to a concurrent package

More information about the Python-Dev mailing list