[Python-Dev] PEP 492: async/await in Python; version 4 (original) (raw)

Wolfgang Langner tds333+pydev at gmail.com
Wed May 6 10:38:52 CEST 2015

Hi Yury,

On 05.05.2015 20:25, Yury Selivanov wrote:

We forget to address the major problems here. How can someone in a "sync" script use this async stuff easy. How can async and sync stuff cooperate and we don't need to rewrite the world for async stuff. How can a normal user access the power of async stuff without rewriting all his code. So he can use a simple asyc request library in his code. How can a normal user learn and use all this in an easy way. asyncio and twisted answered these questions ;) The answer is that you have to write async implementations. gevent has a different answer, but greenlents/stackless is something that will never be merged in CPython and other implementations.

I think monkeypatching and the gevent way is wrong. And explicit is better than implicit.

I have to clear this. I meant how can we make this async stuff more accessible to the average sync user. Sometime even without writing or knowing how to write coroutines or other async stuff.

Let me explain it with a deeper example (Some of it is related to Python 2 and twisted):

I had the same problem for a server application using twisted. Provide easy interfaces to my users most not aware of async stuff.

My solution was to write my own decorator similar to twisted's @inlineCallbacks. On top of it I added one more level to the decorator and distinguish if it was called from the main thread (also running the mainloop) and other threads. This object usable also as decorator is called "task" and has some utility methods. And returns a "deferred". (in asyncio this would be a Future)

Resulting in code like:

@task def myAsyncFunction(webaddress): result = yield fetch(webaddress)

only to show sleep example

yield task.sleep(0.1) task.Return(result)

Usable in a sync context (extra script thread):

def do(): content = myAsyncFunction("http://xyz")

or async context:

@task def ado(): content = yield myAsyncFunction("http://xyz")

The task decorator has functionality to check if something is called from the main thread (-> also a task and async) or it is called from another thread (-> sync or script).

So this async interface is usable from both worlds. If someone operates async he/she must only know the task decorator and when to yield. If he/she uses it in sync mode nothing special has to be done.

To allow all this the server starts the async main loop in the main thread and executes the script in an extra script thread. The user has every time his own thread, also for rpc stuff. The only way to switch into the main loop is to decorate a function as @task, every task is a coroutine and executed in the main thread (also thread of main loop).

Benefit of all this:

This is what I mean if I say we must address the bridging problem between the worlds. It think it is the wrong way to divide it in async and sync stuff and duplicate all networking libraries in the sync and async ones.

For me the answer is to write one async netowrk library and use it in both, a sync script and in an async main loop. With an easy interface and not forcing the user to know this is an async library I have to do something special.

And in future go one step further and use all this combined with PyParallel to solve the multiprocessing problem in Python. (Script in extra thread, mainloop in main thread, executed and managed via PyParallel avoiding the gil) But this is only a vision/dream of mine.

And for all this we still can't tell them "oh the async stuff solves the multiprocessing problem of Python learn it and switch to version 3.5". It does not and it is only most useful for networking stuff nothing more. "networking stuff", and in particular, web, is a huge part of current Python usage. Please don't underestimate that.

I do not. But for most they want only use it as a client and the main concern for most is "I want to get this web page" and not "I will implement a web client have to do this async and get it".

Regards,

Wolfgang

More information about the Python-Dev mailing list