Calling coroutines from sync code (original) (raw)

January 22, 2023, 12:03pm 1

Suppose I want to call async code from a regular function and wait for its return. This function could be part of a library or could be far down a stack that is being iteratively converted to asyncio, so the fact that it’s using asyncio internally should be transparent to any callers. Maybe it’s called from async contexts and there already is an event loop, maybe there isn’t. What’s currently the best way to call a coroutine in these contexts?

async def fetch_something(): ...

def sync_func():  # could be called from async or sync contexts
    ret = call_async(fetch_something())
    ...

Options I have considered for call_async:

asyncio.run() – This is wrong, according to the documentation: “This function cannot be called when another asyncio event loop is running in the same thread. […] It should be used as a main entry point for asyncio programs, and should ideally only be called once.”
asyncio.get_event_loop().run_until_complete() – This has started printing warnings when called from a synchronous context.
asyncio.new_event_loop().run_until_complete()?
Write a custom utility that calls get_running_loop() and new_event_loop() if that throws an exception?

Also, how can I handle the various housekeeping that asyncio.run() does?

srittau (Sebastian Rittau) January 24, 2023, 8:20pm 2

Another solution might be a custom function like this:

def call_async(coro):
    try:
        loop = asyncio.get_running_loop()
    except RuntimeError:
        return asyncio.run(coro)
    else:
        return loop.run_until_complete(coro)

This should take care of all necessary cleanup in case we’re not running in an event loop. Would that make sense?

csm10495 (Charles Machalow) January 28, 2023, 6:10am 3

I tend to do something similar to your call_async(). Though I’m also not sure if its the ‘right way’.

dougransom (Doug Ransom) August 5, 2023, 5:03pm 4

This works but why make everyone repeat this code?

why not have loop.run_until_complete create a loop if necessary?

elis.byberi (Elis Byberi) August 5, 2023, 9:42pm 5

You can make main() asynchronous, and then you can schedule a coroutine using asyncio.create_task(asynchronous()).
You can continue using the synchronous functions as before.

import asyncio
import time


async def asynchronous():
    print('asynchronous()')

    synchronous()

    await asyncio.sleep(0.0)


def synchronous():
    print('synchronous()')
    time.sleep(1.0)

    asyncio.create_task(asynchronous())


async def main():
    synchronous()

    await asyncio.sleep(0.0)


def run():
    loop = asyncio.new_event_loop()
    try:
        loop.run_until_complete(main())
        loop.run_forever()

    finally:
        loop.close()


run()

ofer-pd (Ofer Pd) October 1, 2024, 8:21pm 6

I have exactly the same question that Sebastian articulated perfectly - calling async code from sync code and waiting for the result.

I tried asyncio.get_running_loop().run_until_complete( coro ), but got this error:

RuntimeError: This event loop is already running

elis.byberi (Elis Byberi) October 1, 2024, 9:46pm 7

Use asyncio.create_task(asynchronous()) instead.

ofer-pd (Ofer Pd) October 7, 2024, 4:20pm 8

That does not answer the question. If you call create_task(), it will schedule that coro and return a task object, and then your code will continue.

The question was how to call async code and wait for the result from a function that is not async.

After all my research and experimentation, the conclusion I’ve come to is: “You can’t.”

If a function is async, then the entire stack of calls below it must be async all the way down to the one and only call to asyncio.run() in the app.

elis.byberi (Elis Byberi) October 7, 2024, 5:24pm 9

That object is awaitable, and you can wait on it from main() to get its result. I thought this was obvious.

It boils down to the simple fact that you use synchronous code to create an event loop and get results from asynchronous code. At first, I was sure this was just a request for a code sample. But here are some complete examples.

import asyncio
import time


async def asynchronous():
    print('asynchronous()')

    await asyncio.sleep(0.0)

    return 'Hello async World!'

def synchronous():
    print('synchronous()')
    time.sleep(1.0)

    return asyncio.create_task(asynchronous())

async def main():
    task = synchronous()
    result = await task
    print('result:', result)

def run():
    loop = asyncio.new_event_loop()
    try:
        loop.run_until_complete(main())
        # loop.run_forever()

    finally:
        loop.close()

run()

# synchronous()
# asynchronous()
# result: Hello async World!

def main():
    async def coro():
        await asyncio.sleep(0)
        return 'I\'m coro!'

    result = asyncio.run(coro())
    print('coro:', result)
    
main()

# coro: I'm coro!

That is a new constraint; you cannot wait on an awaitable from a synchronous function. However, you can call (schedule) a coroutine from a synchronous function.

elis.byberi (Elis Byberi) October 7, 2024, 5:50pm 10

Another complete example similar to asyncio.run(), but you can use it when an existing event loop is running:

import asyncio

def synchronous():
    async def coro():
        await asyncio.sleep(1)
        return 'I\'m coro!'

    try:
        loop = asyncio.get_running_loop()
    except RuntimeError:
        loop = asyncio.new_event_loop()
        asyncio.set_event_loop(loop)

    result = loop.run_until_complete(coro())
    print('coro:', result)

async def main():
    # Ensure synchronous() runs without issues in an async context
    await asyncio.to_thread(synchronous)
    print('running...')
    await asyncio.sleep(1)

asyncio.run(main())

# coro: I'm coro!
# running...