Add asyncio.console module to progamatically access the asyncio REPL (original) (raw)

February 8, 2025, 12:03pm 1

What?

Asyncio has a repl (python -m asyncio). I would like to propose to provide functionality analogous to the stdlib’s code.interact() that allows to programatically invoke this repl.

The current limitations?

The asyncio REPL originally introduced in Implement asyncio repl · Issue #81209 · python/cpython · GitHub can not be invoked programatically due to the way it is implemented. Specifically:

  1. All the asyncio repl code is implemented in asyncio.__main__, providing no (proper) way to import it
  2. A significant chunk of it is written as top-level code directly under an if __name__ == "__main__" block
  3. The code relies on global variables, so even if one were to import things from asyncio.__main__, quite a few not so nice tricks must be employed to get it running outside a python -m asyncio call

Why?

The rationale is basically the same as for why code.interact exists. It’s convenient to build custom REPLs, which, while not a super common feature, can be quite useful to implement things like Django’s shell.

All the functionality already exists in the stdlib, so this would merely surface it.

The proposal

  1. Add a new asyncio.console module, implementing the asyncio REPL (basically just move all the code over from asyncio.__main__)
  2. Remove the current implementation’s reliance on global variable manipulation
  3. Expose that functionality via an interact() function, mimicking the interface of code.interact()
  4. Refactor asyncio.__main__ code to simply invoke asyncio.console.interact()

In fact, I have already done just that here (minus documentation and such): add asyncio.console module · provinzkraut/cpython@fc2a561 · GitHub :slight_smile:

Alternatives

Other repls

There are alternative Python repls, like IPython which could be used if such functionality is desired.

However, they are an external dependency, and since the stlib already implements all the functionality necessary for this, it seems logical to simply expose that.

Standalone implementation

Another way to achieve this would be a simple “backport” of the functionality provided in asyncio.__main__, but as it would basically be a copy-paste without adding any significant functionality, I think the same reasoning as for pre-existing alternative repls applies.

Potential downsides

If there’s interest in this, I’d of course be happy to propose a PEP and a PR with the implementation.

PS: Excuse the lack of links to the things I’m referencing, but it seems that as a new user I’m not allowed more than two of those in a post

asvetlov (Andrew Svetlov) February 9, 2025, 11:03am 2

Could you please describe what public API should be exported?
code module provides InteractiveInterpreter, InteractiveConsole, and interact.
Exporting only sync asyncio.console.interact() has very limited usage: it cannot be used from async code, it cannot use a custom loop, etc.
Exporting AsyncIOInteractiveConsole and REPLThread is not an option; these classes belongs to implementation details and could be widely changed between Python releases.

Another thing that bothers me is: asyncio console creates a daemon thread and never waits for the thread execution. It is ok if the lifetime of the function is equal to lifetime of the whole Python process, but spawning daemon threads by api call is dangerous idea I guess.

provinzkraut (Janek Nouvertné) February 9, 2025, 12:24pm 3

Thank you for the feedback @asvetlov :slight_smile:

I was thinking of, for now, simply exporting just interact. It has indeed a fairly limited number of use cases, but is that necessarily a bad thing?

Supplying a custom loop would definitely a good feature to add, as would re-using an already running loop.

Aside from that, did you have any concrete use case in mind that wouldn’t be covered by it?

In my mind, this was purely intended to broaden the utility of what’s already there, admittedly inspired by my own use case (adding a custom repl to a library with support for asyncio), so I might be missing some other obvious cases where the limited functionality of this might be unexpected or undesirable.

Since it just copied the existing code from asyncio.__main__, the interact implementation I’ve proposed calls sys.exit() at the end.

interact should also not be able to return - as far as I can tell - with the thread still running, since the thread internally calls run_multiline_interactive_console or InteractiveColoredConsole.interact, both of which block, and there’s no other busy-loop going on in the thread.

We could also join the thread there of course, but unless a user intentionally catches the SystemExit, that shouldn’t make a difference.

Would you consider that safe enough, as far as the daemon thread is concerned?

Second thoughts

Now that I think of it interact calling sys.exit does diverge from the behaviour of code.interact, which does not call sys.exit(). I guess it makes more sense to join the repl thread once we’ve left the repl and move the sys.exit call to asyncio.__main__.

I think that should address the concerns of the daemonic thread as well?

loic-simon (Loïc Simon) February 10, 2025, 2:13am 4

FWIW, I wrote a small package a few years ago precisely exposing anAsyncInteractiveInterpreter and an AsyncInteractiveConsole classes: https://pypi.org/project/asyncode

I used it to provide a REPL in a (private) Discord bot; worked well!

asvetlov (Andrew Svetlov) February 10, 2025, 9:18am 5

IIRC daemon thread mode was added to finish an interpreter when the thread is waiting for console input (readline call); maybe the new console has a way to interrupt the reader gracefully.

provinzkraut (Janek Nouvertné) February 10, 2025, 5:58pm 6

Hmm. Getting rid of the thread would arguably the cleanest option. I’ll see if I can come up with something :slight_smile:

provinzkraut (Janek Nouvertné) February 10, 2025, 6:00pm 7

@loic-simon thanks for sharing, I’ll check this one out!