[Python-Dev] PEP 492: async/await in Python; version 4 (original) (raw)

Paul Moore p.f.moore at gmail.com
Tue May 5 22:39:12 CEST 2015

(Yury gave similar responses, so (a) I'll just respond here, and (b) it's encouraging that you both responded so quickly with the same message)

On 5 May 2015 at 20:44, Brett Cannon <brett at python.org> wrote:

That's not to say that everything needs to be beginner-friendly, but it does mean that it's hard for the wider Python community to meaningfully comment, or evaluate or sanity-check the design. We're left with a sense of "trust us, it makes sense if you need it, everyone else can ignore it". Watch David Beazley's talk from PyCon this year and you can watch him basically re-implement asyncio on stage in under 45 minutes. It's not as complicated as it seems when you realize there is an event loop driving everything (which people have been leaving out of the conversation since it doesn't tie into the syntax directly).

I'll watch that - it should be fun. But I have seen things like that before, and I've got an idea how to write an event loop. You're right that it's easy to lose track of the fundamentally simple idea in all the complex discussions. To me that feels like a peculiar failure of the abstraction, in that in some circumstances it makes things feel more complex than they are :-)

Personally, I feel as if PEP 492 is looking a little premature - maybe the focus should be on making asyncio more accessible first, and then adding syntax. I think this ties the concept of adding syntax to Python to make coroutine-based programming easier too much to asyncio; the latter is just an implementation of the former. This PEP doesn't require asyncio beyond the fact that will be what provides the initial event loop in the stdlib.

It's very hard to separate coroutines from asyncio, because there's no other example (not even a toy one) to reason about.

It would probably be helpful to have a concrete example of a basic event loop that did nothing but schedule tasks. No IO waiting or similar, just scheduling. I have a gut feeling that event loops are more than just asyncio, but without examples to point to it's hard to keep a focus on that fact. And even harder to isolate "what is an event loop mechanism" from "what is asyncio specific". For example, asyncio.BaseEventLoop has a create_connection method. That's obviously not a fundamental aspect of a generic event loop, But call_soon (presumably) is. Having a documented "basic event loop" interface would probably help emphasise the idea than event loops don't have to be asyncio. (Actually, what is the minimal event loop interface that is needed for the various task/future mechanisms to work, independently of asyncio? And what features of an event loop etc are needed for the PEP, if it's being used outside of asyncio?)

I guess the other canonical event loop use case is GUI system message dispatchers.

You can argue that the syntax is needed to help make async more accessible - but if that's the case then the terminology debates and confusion are clear evidence that it's not succeeding in that goal. Perhaps, but arguing about the nitty-gritty details of something doesn't automatically lead to a clearer understanding of the higher level concept. Discussing how turning a steering wheel in a car might help you grasp how cars turn, but it isn't a requirement to get "turn the wheel left to make the car go left".

Fair point. If only I could avoid driving into walls :-)

Of course, that's based on my perception of one of the goals of the PEP as being "make coroutines and asyncio more accessible", If the actual goals are different, my conclusion is invalid. I think the goal is "make coroutines easier to use" and does not directly relate to asyncio.

OK. But in that case, some examples using a non-asyncio toy "just schedule tasks" event loop might help.

Well, twisted always had defertothread. Asyncio has runinexecutor, but that seems to be callback-based rather than coroutine-based? Yep.

... and so you can't use it with async/await?

Many people use requests for their web access. There are good reasons for this. Are you saying that until someone steps up and writes an async implementation of requests, I have to make a choice - requests or asyncio? I believe so; you need something to implement await. This is true in any language that implements co-routines.

Unfortunately, I can't see myself choosing asyncio in that situation. Which again means that asyncio becomes "something that the average user can't use". Which in turn further entrenches it as a specialist-only tool. You forgot to append "... yet" to that statement. Just because something isn't available out of the box without some effort to support doesn't mean it will never happen, else there would be absolutely no Python 3 users out there.

Fair point. Yuri mentioned aiohttp, as well. The one difference between this and Python 2/3, is that here you have to have two separate implementations. There's no equivalent of a "shared source" async and synchronous implementation of requests. So the typical "please support Python 3" issue that motivates projects to move forward doesn't exist in the same way. It's not to say that there won't be async versions of important libraries, it's just hard to see how the dynamics will work. I can't see myself raising an issue on cx_Oracle saying "please add asyncio support", and I don't know who else I would ask...

Co-routine-based asynchronous programming is new to Python, so as a community we don't have it as something everyone learns over time. If you don't come from an area that supports it then it will be foreign to you and not make sense without someone giving you a good tutorial on it. But considering C#, Dart, and Ecmascript 6 (will) have co-routine support -- and those are just the languages I can name off the top of my head -- using the exact same keywords suggests to me that it isn't that difficult of a topic to teach people. This is just one of those PEPs where you have to trust the people with experience in the area are making good design decisions for those of us who aren't in a position to contribute directly without more experience in the domain.

That's also a fair point, and it seems to me that there is reasonably general feeling that the experts can be trusted on the basic principles. There's also a huge amount of bikeshedding, but that's pretty much inevitable :-)

But I do think that unless someone does something to offer some non-asyncio examples of coroutine-based asynchronous programming in Python, the link in people's minds between async and asyncio will become more and more entrenched. While asyncio is the only real event loop implementation, saying "async can be used for things other than asyncio" is a rather theoretical point.

Is there anyone who feels they could write a stripped down but working example of a valid Python event loop without the asyncio aspects? Or is that what David Beazley's talk does? (I got the impression from what you said that he was aiming at async IO rather than just a non-IO event loop). Can asyncio.Future and asyncio.Task be reused with such an event loop, or would those need to be reimplemented as well? Writing your own event loop seems like a plausible exercise. Writing your own version of the whole task/future/coroutine/queue/synchronisation mechanisms seems like a lot to expect. And the event loop policy mechanism says that it works with loops that implement asyncio.BaseEventLoop (which as noted includes things like create_connection, etc).

Paul

More information about the Python-Dev mailing list