[FIXED] Async Generator Comprehension

Issue

The requirement is to concurrently perform a time consuming operation for a list of data.

My current implementation:

async def expensive_routine(service) -> Optional[Any]:
    await asyncio.sleep(5)
    if service % 2:
        return service
    return None

async def producer():
    # let's say
    services = range(10)
    #
    for future in asyncio.as_completed(
            [expensive_routine(service) for service in services]
        ):
            result = await future
            if result:
                yield result

This is then used by:

  async for x, y in producer():
        print(f"I have my {x} and {y}")

the function expensive_routine returns Optional[Any]. I want to yield only the not None results. Is there a way to perform this more efficiently or using a Comprehension ?

Solution

You mean if you can cram your nice little producer coroutine into a single-line generator expression abomination of unreadability? Why, yes!

import asyncio
import random


async def expensive_routine(service):
    await asyncio.sleep(random.randint(0, 5))
    if random.choice([0, 1]):
        return service
    return None


async def main():
    async for x in (
        res
        for coro in asyncio.as_completed(
            [expensive_routine(service) for service in range(10)]
        )
        if (res := (await coro)) # Python 3.8+
    ):
        print(x)


asyncio.run(main())

Jokes aside, I don't see anything wrong with your code and I'm not sure what you mean by more efficient, since your speed here is dominated by the slow expensive_routine.

I wrote this small example because I think this is what you meant with a comprehension, but I would prefer your much more readable producer.

Answered By - thisisalsomypassword