[FIXED] Alternative to asyncio.gather which I can keep adding coroutines to at runtime?

Issue

I need to be able to keep adding coroutines to the asyncio loop at runtime. I tried using create_task() thinking that this would do what I want, but it still needs to be awaited.

This is the code I had, not sure if there is a simple edit to make it work?

async def get_value_from_api():
    global ASYNC_CLIENT
    return ASYNC_CLIENT.get(api_address)


async def print_subs():
    count = await get_value_from_api()
    print(count)


async def save_subs_loop():
    while True:
        asyncio.create_task(print_subs())
        time.sleep(0.1)


async def start():
    global ASYNC_CLIENT
    async with httpx.AsyncClient() as ASYNC_CLIENT:
        await save_subs_loop()


asyncio.run(start())

Solution

asyncio.create_task() works as you describe it. The problem you are having here is that you create an infinite loop here:

async def save_subs_loop():
    while True:
        asyncio.create_task(print_subs())
        time.sleep(0.1) # do not use time.sleep() in async code EVER

save_subs_loop() keeps creating tasks but control is never yielded back to the event loop, because there is no await in there. Try

async def save_subs_loop():
    while True:
        asyncio.create_task(print_subs())
        await asyncio.sleep(0.1) # yield control back to loop to give tasks a chance to actually run

This problem is so common I'm thinking python should raise a RuntimeError if it detects time.sleep() within a coroutine :-)

Answered By - thisisalsomypassword