[Python-Dev] PEP 492 quibble and request (original) (raw)

Nick Coghlan ncoghlan at gmail.com
Thu Apr 30 03:52:11 CEST 2015

On 30 April 2015 at 10:33, Yury Selivanov <yselivanov.ml at gmail.com> wrote:

To really understand all implementation details of this line you need to read PEP 3156 and experiment with asyncio. There is no easier way, unfortunately. I can't add a super detailed explanation how event loops can be implemented in PEP 492, that's not in its scope.

This request isn't about understanding the implementation details, it's about understanding what Python users will gain from the PEP without needing to understand the implementation details.

For that, I'd like to see some not-completely-trivial example code in (or at least linked from) the PEP written using:

The intent of PEP 492, like PEP 380 and PEP 342 before it, is "make asynchronous programming in Python easier". I think it will actually succeed in that goal, but I don't think it currently does an especially job of explaining that to folks that aren't both already deeply invested in the explicitly asynchronous programming model and thoroughly aware of the fact that most of us need asynchronous programming to look as much like synchronous programming as possible in order for it to fit our brains. Some folks can fit ravioli code with callbacks going everywhere in their brains, but I can't, and it's my experience that most other folks can't either. This lack means the PEP that gets confused objections from folks that wish explicitly asynchronous programming models would just go away entirely (they won't), as well as from folks that already understand it and don't see why we can't continue treating it as a special case of other features that folks have to learn how to use from first principles, rather than saving them that up front learning cost by doing the pattern extraction to make event driven explicitly asynchronous programming its own first class concept with dedicated syntax (a hint on that front: with statements are just particular patterns for using try/except/finally, decorators are just particular patterns in using higher order functions, for statements are just particular patterns in using while statements and builtins, and even imports and classes just represent particular patterns in combining dictionaries, code execution and the metaclass machinery - the pattern extraction and dedicated syntax associated with all of them makes it possible to learn to use these concepts without first having to learn how to implement them)

From my own perspective, I've spent a reasonable amount of time attempting to explain to folks the "modal" nature of generators, in that you can use them both as pure iterable data sources and as coroutines (as per PEP 342).

The problem I've found is that our current approach ends up failing the "conceptually different things should also look superficially different" test: outside certain data pipeline processing problems, generators-as-iterators and generators-as-coroutines mostly end up being fundamentally different ways of approaching a programming problem, but the current shared syntax encourages users to attempt to place them in the same mental bucket. Those users will remain eternally confused until they learn to place them in two different buckets despite the shared syntax (the 5 or so different meanings of "static" in C++ come to mind at this point...).

With explicitly asynchronous development*, this problem of needing to learn to segment the world into two pieces is particularly important, and this wonderful rant on red & blue functions published a couple of months ago helps explain why: http://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/

The dedicated async/await syntax proposed in PEP 492 lets the end user mostly not care about all the messiness that needs to happen under the covers to make explicitly asynchronous programming as close to normal synchronous programming as possible. The fact that under the hood in CPython normal functions, coroutines, and generators are implemented largely as variant behaviours of the same type (producing respectively the requested result, an object expecting to be driven by an event loop to produce the requested result, and an object expecting to be driven be an iterative loop to produce a succession of requested values rather than a single result) can (and at least arguably should) be irrelevant to the mental model formed by future Python programmers (see http://uxoslo.com/2014/01/14/ux-hints-why-mental-models-matter/ for more on the difference between the representational models we present directly to end users and the underlying implementation models we use as programmers to actually make things work)

Regards, Nick.

P.S. *While it's not a substitute for explicitly asynchronous development, implicitly asynchronous code still has an important role to play as one of the 3 models (together with thread pools and blocking on the event loop while running a coroutine to completion) that lets synchronous code play nice with asynchronous code: http://python-notes.curiousefficiency.org/en/latest/pep_ideas/async_programming.html#gevent-and-pep-3156

-- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia

More information about the Python-Dev mailing list