[FIXED] How to split a long coroutine without using await?

Issue

I have a coroutine that is getting too big and I'd like to split it for readability.

async def handle_message(self, message):
    message_type = message.get('type')

    if message_type == 'broadcast':
        ...
        for n in self._neighbors:
            await self.send_message(n, message)

    elif message_type == 'graph':
        ...

I'd like to extract the portion which handles broadcast messages into a private method like this:

async def handle_message(self, message):
    message_type = message.get('type')
    ...

    if message_type = 'broadcast':
        await self._handle_broadcast(message)
    elif message_type = 'graph':
        ...

The problem is that this changes the behavior of the code, since the _handle_broadcast part is a coroutine and its execution might be delayed since I call it with await.

What is the way to ensure that the coroutine runs immediately and isn't delayed?

Solution

In short: split the coroutine exactly like you started, by using await.

The problem is that this changes the behavior of the code, since the _handle_broadcast part is a coroutine and its execution might be delayed since I call it with await.

For better or worse, this premise is false. When given a coroutine, await immediately starts executing it without an intermediate delay. It is only if that coroutine calls something that causes it to suspend (such as asyncio.sleep or a network read that doesn't have data yet) that your coroutine gets suspended along with it - which is precisely what you would get had the code stayed inline.

In that sense await <some coroutine> works like the coroutine equivalent of a regular function call, allowing precisely the kind of non-semantics-changing refactoring that you need. This can be demonstrated with an example:

import asyncio

async def heartbeat():
    while True:
        print('tick')
        await asyncio.sleep(1)

async def noop():
    pass

async def coro():
    # despite "await", this blocks the event loop!
    while True:
        await noop()

loop = asyncio.get_event_loop()
loop.create_task(heartbeat())
loop.create_task(coro())
loop.run_forever()

The above code blocks the event loop - even though coro does nothing except await in a loop. So await is not a guarantee of yielding to the event loop, the coroutine has to do it with other means. (This behavior can also be a source of bugs.)

In the above case, one can get the event loop "un-stuck" by inserting an await asyncio.sleep(0). But that kind of thing should never be needed in production asyncio code, where the program should be structured so that each coroutine does comparatively little work, and then uses await to obtain more data.

Answered By - user4815162342