[Python-Dev] PEP 492: What is the real goal? (original) (raw)

Yury Selivanov yselivanov.ml at gmail.com
Thu Apr 30 21:27:09 CEST 2015

Jim,

On 2015-04-30 2:41 PM, Jim J. Jewett wrote: [...]

Does it really permit making them [asynchronous calls], or does it just signal that you will be waiting for them to finish processing anyhow, and it doesn't need to be a busy-wait? I does. Bad phrasing on my part. Is there anything that prevents an asynchronous call (or waiting for one) without the "async with"? If so, I'm missing something important. Either way, I would prefer different wording in the PEP.

Yes, you can't use 'yield from' in exit/enter in current Python.

It uses the yield from implementation with an extra step of validating its argument. await only accepts an awaitable, which can be one of: What justifies this limitation? We want to avoid people passing regular generators and random objects to 'await', because it is a bug. Why? Is it a bug just because you defined it that way? Is it a bug because the "await" makes timing claims that an object not making such a promise probably won't meet? (In other words, a marker interface.) Is it likely to be a symptom of something that wasn't converted correctly, and there are likely to be other bugs caused by that same lack of conversion?

Same as 'yield from' is expecting an iterable, await is expecting an awaitable. That's the protocol.

You can't pass random objects to 'with' statements, 'yield from', 'for..in', etc.

If you write

def gen(): yield 1
await gen()

then it's a bug.

For coroutines in PEP 492: await_ = anext is the same as call = _next await_ = aiter is the same as call = _iter That tells me that it will be OK sometimes, but will usually be either a mistake or an API problem -- and it explains why. Please put those 3 lines in the PEP.

There is a line like that: https://www.python.org/dev/peps/pep-0492/#await-expression Look for "Also, please note..." line.

This is OK. The point is that you can use 'await log' in aenter. If you don't need awaits in aenter you can use them in aexit. If you don't need them there too, then just define a regular context manager. Is it an error to use "async with" on a regular context manager? If so, why? If it is just that doing so could be misleading, then what about "async with mgr1, mgr2, mgr3" -- is it enough that one of the three might suspend itself?

'with' requires an object with enter and exit

'async with' requires an object with aenter and aexit

You can have an object that implements both interfaces.

class AsyncContextManager: def aenter(self): log('entering context') aenter must return an awaitable Why? Is there a fundamental reason, or it is just to avoid the hassle of figuring out whether or not the returned object is a future that might still need awaiting?

The fundamental reason why 'async with' is proposed is because you can't suspend execution in enter and exit. If you need to suspend it there, use 'async with' and its a* methods, but they have to return awaitable (see https://www.python.org/dev/peps/pep-0492/#new-syntax and look what 'async with' is semantically equivalent to)

Is there an assumption that the scheduler will let the thing-being awaited run immediately, but look for other tasks when it returns, and a further assumption that something which finishes the whole task would be too slow to run right away?

It doesn't make any sense in using 'async with' outside of a coroutine. The interpeter won't know what to do with them: you need an event loop for that. So does the PEP also provide some way of ensuring that there is an event loop? Does it assume that self-suspending coroutines will only ever be called by an already-running event loop compatible with asyncio.geteventloop()? If so, please make these contextual assumptions explicit near the beginning of the PEP.

You need some kind of loop, but it doesn't have to the one from asyncio. There is at least one place in the PEP where it's mentioned that the PEP introduses a generic concept that can be used by asyncio and other frameworks.

It is a TypeError to pass a regular iterable without _aiter_ method to async for. It is a SyntaxError to use async for outside of a coroutine. The same questions about why -- what is the harm? I can imagine that as an implementation detail, the async for wouldn't be taken advtange of unless it was running under an event loop that knew to look for "aync for" as suspension points.

Event loop doesn't need to know anything about 'async with' and 'async for'. For loop it's always one thing -- something is awaiting somewhere for some result.

I'm not seeing what the actual harm is in either not happening to suspend (less efficient, but still correct), or in suspending between every step of a regular iterator (because, why not?)

For debugging this kind of mistakes there is a special debug mode in asyncio, in which @coroutine ... decorator makes the decision of whether to wrap or not to wrap based on an OS environment variable PYTHONASYNCIODEBUG. (1) How does this differ from the existing asynchio.coroutine? (2) Why does it need to have an environment variable? (Sadly, the answer may be "backwards compatibility", if you're really just specifying the existing asynchio interface better.) (3) Why does it need [set]getcoroutinewrapper, instead of just setting the asynchio.coroutines.coroutine attribute? (4) Why do the get/set need to be in sys?

That section describes some hassles we had in asyncio to enable better debugging.

(3) because it allows to enable debug selectively when we need it

(4) because it's where functions like 'set_trace' live. set_coroutine_wrapper() also requires some modifications in the eval loop, so sys looks like the right place.

Is the intent to do anything more than preface execution with: import asynchio.coroutines asynchio.coroutines.DEBUG = True

This won't work, unfortunately. You need to set the debug flag before you import asyncio package (otherwise we would have an unavoidable performance cost for debug features). If you enable it after you import asyncio, then asyncio itself won't be instrumented. Please see the implementation of asyncio.coroutine for details.

set_coroutine_wrapper solves these problems.

Yury

More information about the Python-Dev mailing list