[Python-Dev] Active Objects in Python (original) (raw)

Phillip J. Eby pje at telecommunity.com
Tue Sep 27 19:40:22 CEST 2005

At 11:18 AM 9/27/2005 -0600, Bruce Eckel wrote:

Yes, defining an class as "active" would: 1) Install a worker thread and concurrent queue in each object of that class. 2) Automatically turn method calls into tasks and enqueue them 3) Prevent any other interaction other than enqueued messages

#3 is the sticky bit. Enforcing such restrictions in Python is hard. Each active object would have to have its own sys.modules, for example, because modules and classes in Python are mutable and use dictionaries, and modifying them from two thread simultaneously would be fatal.

For built-in types it's not so bad, so they could be shared... except for the fact that they have reference counts, which need lock protection as well to avoid fouling up GC. So really, this idea doesn't really help with GIL removal at all.

What it does help with, is effective use of multiprocessor machines on platforms where fork() is available, if the API works across processes as well as threads.

So yes, the best way for this to work might be some kind of enforced implementation -- but forcing you to do something is not particularly Pythonic,

But keeping the interpreter from dumping core due to program bugs is Pythonic, which is why the GIL would need to stay. :)

so I would think that it would be possible to build an active object framework along with some checking tools to warn you if you are breaking the rules.

Well, you could pickle and unpickle the objects you send from one function to another, and for cross-process communication, you'll need to do something like that anyway, or else use that shared-memory objects thing. PySHM? I don't remember its name, but it's an extension that lets you store Python objects in shared memory and use them from multiple processes, modulo certain strict limitations.

But what's not clear is whether this would require knowledge of the innards of the Python interpreter (which I don't have) or if it could be built using libraries.

If you're not trying to get rid of the GIL or truly "enforce" things, you can do everything you need for CSP in plain old Python. peak.events has a generator-based microthread facility that's CSP-ish, for example, although it doesn't have decorators for async methods. They're not hard to write, though; Chandler has some for methods that get called across thread barriers now. That is, methods that get called from Chandler's UI thread but are run in the Twisted reactor thread. I've occasionally also thought about implementing async C#'s "chord" features in Python for peak.events, but I haven't actually had a use case that needed that much generality yet.

More information about the Python-Dev mailing list