[FIXED] How to build an asynchronous generator?

Issue

When I run the code

async def main(*args, **kwargs):
    await sub_process1()

async def sub_process1():
    iter = await sub_process2()
    for i in iter:
        yield i

async def sub_process2():
    return [1, 2, 3]

I get

    async def main(*args, **kwargs):
>       await sub_process1()
E       TypeError: object async_generator can't be used in 'await' expression

The same result is obtained if I use

async def sub_process1():
    iter = await sub_process2()
    async for i in iter:
        yield i

instead.

How can I fix this?

Solution

a generator (async or not) doesn't do any work when it is created, it starts doing work when you iterate over it (for loop / async for loop) or equivalently when you call next or anext on it.

the difference between an async generator and a normal generator is that an async one can use await to suspend itself which in turn suspends whoever is iterating over it asynchronously, while not stalling the eventloop, the eventloop can still process other async tasks while waiting.

in order to iterate over an async generator you need to use an async for

the following example illustrates the difference

import asyncio
import time

async def main(*args, **kwargs):
    async for i in sub_process1(): # async generator async for
        print(i) 

async def sub_process1():
    async for i in sub_process2():  # async generator async for
        yield i
    
    # what happens under the hood
    generator = sub_process2() 
    try:
        while True:
           i = await anext(generator)
           yield i
    except StopAsyncIteration:
        pass
    
async def sub_process2(): # async generator
    for i in range(10):  # non async generator normal for
        yield i
        await asyncio.sleep(1) ## suspends sub_process1 and main but not the eventloop
        # this could be any awaitable IO, not a sleep
    for i in range(10):
        yield i
        time.sleep(1) ## suspends the eventloop
        # this could be any blocking IO, not a sleep

def sub_process3(): # non async generator
    for i in range(10):
        yield i
        # await asyncio.sleep(1) ## won't compile 

asyncio.run(main())

Answered By - Ahmed AEK