[FIXED] In Python asyncio, why does line-per-file processing blocks method?

Issue

I'm totally new to python's asyncio. I understand the idea, but even the most simple task won't work due to a lack of understanding on my side.

Here's my code which tries to read a file (and ultimately process each line of it) reguarily:

#!/usr/bin/env python3

import asyncio
import aiofiles

async def main():
    async def work():
        while True:
            async with aiofiles.open('../v2.rst', 'r') as f:
                async for line in f:
                    # real work will happen here
                    pass
            print('loop')
            await asyncio.sleep(2)

    tasks = asyncio.gather(
        work(),
    )

    await asyncio.sleep(10)

    # Cancel tasks
    tasks.add_done_callback(lambda r: r.exception())
    tasks.cancel()

if __name__ == '__main__':
    asyncio.run(main())

The work-function should read a file, do some line-per-line processing and then wait 2 seconds.

What happens is, that the function does "nothing". It blocks, I never see loop printed.

Where is my error in understanding asyncio?

Solution

The code hides the exception because the callback installed with add_done_callback retrieves the exception, only to immediately discard it. This prevents the (effectively unhandled) exception from getting logged by asyncio, which happens if you comment out the line with add_done_callback.

Also:

• the code calls gather without awaiting it, either immediately after the call or later.
• it unnecessarily invokes gather with a single coroutine. If the idea is to run the coroutine in the background, the idiomatic way to do so is with asyncio.create_task(work()).

Answered By - user4815162342