[FIXED] asyncio: Why is awaiting a cancelled Future not showing CancelledError?

Issue

Given the following program:

import asyncio

async def coro():
    future = asyncio.Future()
    future.cancel()
    print(future)  # <Future cancelled>
    await future  # no output


loop = asyncio.get_event_loop()
loop.create_task(coro())
loop.run_forever()

Why is the CancelledError thrown by await future not shown? Explicitly wrapping await future in try/except shows that it occurs. Other unhandled errors are shown with Task exception was never retrieved, like this:

async def coro2():
    raise Exception('foo')


loop.create_task(coro2())

Why is this not the case for awaiting cancelled Futures?

Additional question: What happens internally if a coroutine awaits a cancelled Future? Does it wait forever? Do I have to do any "cleanup" work?

Solution

Why is the CancelledError thrown by await future not shown?

The exception is not shown because you're never actually retrieving the result of coro. If you retrieved it in any way, e.g. by calling result() method on the task or just by awaiting it, you would get the expected error at the point where you retrieve it. The easiest way to observe the resulting traceback is by changing run_forever() to run_until_complete(coro()).

What happens internally if a coroutine awaits a cancelled Future? Does it wait forever? Do I have to do any "cleanup" work?

It doesn't wait forever, it receives the CancelledError at the point of await. You have discovered this already by adding a try/except around await future. The cleanup you need to do is the same like for any other exception - either nothing at all, or using with and finally to make sure that the resources you acquired get released in case of an exit.

Other unhandled errors are shown with Task exception was never retrieved [...] Why is this not the case for awaiting cancelled Futures?

Because Future.cancel intentionally disables the logging of traceback. This is to avoid spewing the output with "exception was never retrieved" whenever a future is canceled. Since CancelledError is normally injected from the outside and can happen at (almost) any moment, very little value is derived from retrieving it.

If it sounds strange to show the traceback in case of one exception but not in case of another, be aware that the tracebacks of failed tasks are displayed on a best-effort basis to begin with. Tasks created with create_task and not awaited effectively run "in the background", much like a thread that is not join()ed. But unlike threads, coroutines have the concept of a "result", either an object returned from the coroutine or an exception raised by it. The return value of the coroutine is provided by the result of its task. When the coroutine exits with an exception, the result encapsulates the exception, to be automatically raised when the result is retrieved. This is why Python cannot immediately print the traceback like it does when a thread terminates due to an unhandled exception - it has to wait for someone to actually retrieve the result. It is only when a Future whose result holds an exception is about to get garbage-collected that Python can tell that the result would never be retrieved. It then displays the warning and the traceback to avoid the exception passing silently.

Answered By - user4815162342