[FIXED] Start async task now, await later

Issue

C# programmer trying to learn some Python. I am trying to run a CPU intensive calc while letting an IO bound async method quietly chug away in the background. In C#, I would typically set the awaitable going, then kick off the CPU intensive code, then await the IO task, then combine results.

Here's how I'd do it in C#

static async Task DoStuff() {
    var ioBoundTask = DoIoBoundWorkAsync();
    int cpuBoundResult = DoCpuIntensizeCalc();
    int ioBoundResult = await ioBoundTask.ConfigureAwait(false);

    Console.WriteLine($"The result is {cpuBoundResult + ioBoundResult}");
}

static async Task<int> DoIoBoundWorkAsync() {
    Console.WriteLine("Make API call...");
    await Task.Delay(2500).ConfigureAwait(false); // non-blocking async call
    Console.WriteLine("Data back.");
    return 1;
}

static int DoCpuIntensizeCalc() {
    Console.WriteLine("Do smart calc...");
    Thread.Sleep(2000);  // blocking call. e.g. a spinning loop
    Console.WriteLine("Calc finished.");
    return 2;
}

And here's the equivalent code in python

import time
import asyncio

async def do_stuff():
    ioBoundTask = do_iobound_work_async()
    cpuBoundResult = do_cpu_intensive_calc()
    ioBoundResult = await ioBoundTask
    print(f"The result is {cpuBoundResult + ioBoundResult}")

async def do_iobound_work_async(): 
    print("Make API call...")
    await asyncio.sleep(2.5)  # non-blocking async call
    print("Data back.")
    return 1

def do_cpu_intensive_calc():
    print("Do smart calc...")
    time.sleep(2)  # blocking call. e.g. a spinning loop
    print("Calc finished.")
    return 2

await do_stuff()

Importantly, please note that the CPU intensive task is represented by a blocking sleep that cannot be awaited and the IO bound task is represented by a non-blocking sleep that is awaitable.

This takes 2.5 seconds to run in C# and 4.5 seconds in Python. The difference is that C# runs the asynchronous method straight away whereas python only starts the method when it hits the await. Output below confirms this. How can I achieve the desired result. Code that would work in Jupyter Notebook would be appreciated if at all possible.

--- C# ---
Make API call...
Do smart calc...
Calc finished.
Data back.
The result is 3

--- Python ---
Do smart calc...
Calc finished.
Make API call...
Data back.
The result is 3

---

Update 1

Inspired by knh190's answer, it seems that I can get most of the way there using asyncio.create_task(...). This achieves the desired result (2.5 secs): first, the asynchronous code is set running; next, the blocking CPU code is run synchronously; third the asynchronous code is awaited; finally the results are combined. To get the asynchronous call to actually start running, I had to put an await asyncio.sleep(0) in, which feels like a horrible hack. Can we set the task running without doing this? There must be a better way...

async def do_stuff():
    task = asyncio.create_task(do_iobound_work_async())
    await asyncio.sleep(0)  #   <~~~~~~~~~ This hacky line sets the task running

    cpuBoundResult = do_cpu_intensive_calc()
    ioBoundResult = await task

    print(f"The result is {cpuBoundResult + ioBoundResult}")

Solution

So with a bit more research it seems that this is possible but not quite as easy as in C#. The code for do_stuff() becomes:

async def do_stuff():
    task = asyncio.create_task(do_iobound_work_async())  # add task to event loop
    await asyncio.sleep(0)                               # return control to loop so task can start
    cpuBoundResult = do_cpu_intensive_calc()             # run blocking code synchronously
    ioBoundResult = await task                           # at last, we can await our async code

    print(f"The result is {cpuBoundResult + ioBoundResult}")

Versus C#, the two differences are:

1. asyncio.create_task(...) required to add the task to the running event loop
2. await asyncio.sleep(0) to temporarily return control back to the event loop so it can start the task.

The complete code sample is now:

import time
import asyncio

async def do_stuff():
    task = asyncio.create_task(do_iobound_work_async())  # add task to event loop
    await asyncio.sleep(0)                               # return control to loop so task can start
    cpuBoundResult = do_cpu_intensive_calc()             # run blocking code synchronously
    ioBoundResult = await task                           # at last, we can await our async code

    print(f"The result is {cpuBoundResult + ioBoundResult}")

async def do_iobound_work_async(): 
    print("Make API call...")
    await asyncio.sleep(2.5)  # non-blocking async call. Hence the use of asyncio
    print("Data back.")
    return 1

def do_cpu_intensive_calc():
    print("Do smart calc...")
    time.sleep(2)  # long blocking code that cannot be awaited. e.g. a spinning loop
    print("Calc finished.")
    return 2

await do_stuff()

---

I am not a big fan of having to remember to add that extra await asyncio.sleep(0) in order to start the task. It might be neater to have an awaitable function like begin_task(...) that starts the task running automatically such that it can be awaited at a later stage. Something like the below, for instance:

async def begin_task(coro):
    """Awaitable function that adds a coroutine to the event loop and sets it running."""
    task = asyncio.create_task(coro)
    await asyncio.sleep(0)
    return task

async def do_stuff():
    io_task = await begin_task(do_iobound_work_async())
    cpuBoundResult = do_cpu_intensive_calc()
    ioBoundResult = await io_task
    print(f"The result is {cpuBoundResult + ioBoundResult}")

Answered By - Big AL