[Python-Dev] Pythonic concurrency - cooperative MT (original) (raw)
Christopher Armstrong radeex at gmail.com
Sun Oct 2 04:00:03 CEST 2005
- Previous message: [Python-Dev] Pythonic concurrency - cooperative MT
- Next message: [Python-Dev] Why does __getitem__ slot of builtin call sequence methods first?
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
On 10/2/05, Martin Blais <blais at furius.ca> wrote:
One of the problems that you have with using generators like this, is that automatic "yield" on resource access does not occur automatically, like it does in threading. With threads, the kernel is invoked when access to a low-level resource is requested, and may decide to put your process in the wait queue when it judges necessary. I don't know how you would do that with generators. To implement that explicitly, you would need an asynchronous version of all the functions that may block on resources (e.g. file open, socket write, etc.), in order to be able to insert a yield statement at that point, after the async call, and there should be a way for the scheduler to check if the resource is "ready" to be able to put your generator back in the runnable queue.
(A question comes to mind here: Twisted must be doing something like this with their "deferred objects", no? I figure they would need to do something like this too. I will have to check.)
As I mentioned in the predecessor of this thread (I think), I've written a thing called "Defgen" or "Deferred Generators" which allows you to write a generator to yield control when waiting for a Deferred to fire. So this is basically "yield or resource access". In the Twisted universe, every asynchronous resource-retrieval is done by returning a Deferred and later firing that Deferred. Generally, you add callbacks to get the value, but if you use defgen you can say stuff like (in Python 2.5 syntax) try: x = yield getPage('http://python.org/') except PageNotFound: print "Where did Python go!" else: assert "object-oriented" in x
Many in the Twisted community get itchy about over-use of defgen, since it makes it easier to assume too much consistency in state, but it's still light-years beyond pre-emptive shared-memory threading when it comes to that.
-- Twisted | Christopher Armstrong: International Man of Twistery Radix | -- http://radix.twistedmatrix.com | Release Manager, Twisted Project \\V/// | -- http://twistedmatrix.com |o O| | w----v----w-+
- Previous message: [Python-Dev] Pythonic concurrency - cooperative MT
- Next message: [Python-Dev] Why does __getitem__ slot of builtin call sequence methods first?
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]