[FIXED] Why doesn't asyncio.TaskGroup wait for the queue to finish?

Issue

Im python asyncio newbie, In my code, total_slept_for value is 0. I want to calculate the total time it takes for all the values inside the queue to finish.

Any help or idea?? Thanks.

This is my code: python version : 3.11.4

import asyncio
import random
import time


async def worker(name, queue):
    while True:
        # queue.get()
        try:
            sleep_for = queue.get_nowait()
        except asyncio.QueueEmpty:
            break
        print(f"sleep_for : {sleep_for}")

        await asyncio.sleep(sleep_for)

        # Notify the queue work item has been processed
        queue.task_done()

        print(f"{name} has slept for {sleep_for:.2f} seconds")


async def main():
    # Create Quere
    q = asyncio.Queue()

    total_sleep_time = 0
    for _ in range(20):
        sleep_for = random.uniform(0.05, 1.0)
        total_sleep_time += sleep_for
        q.put_nowait(sleep_for)

    # Create 3 workers
    async with asyncio.TaskGroup() as tg:
        for idx in range(3):
            tg.create_task(worker(f"worker-{idx}", q))

    # Calc total running time
    started_at = time.monotonic()
    # Waiting finish Queue
    await q.join()
    print(time.monotonic())
    total_slept_for = time.monotonic() - started_at

    print("====")
    print(f"3 workers slept in parallel for {total_slept_for:.2f} seconds")
    print(f"total expected sleep time: {total_sleep_time:.2f} seconds")


asyncio.run(main())

Solution

The async with asyncio.TaskGroup() as tg: waits for the task group to finish. That means all the tasks are already done sleeping by the time you start timing.

You need to put the timing code and await q.join() inside the async with.

Answered By - user2357112