[FIXED] How to schedule awaitables for sequential execution without awaiting, without prior knowing the number of awaitables?

Issue

What I would like to do is essentially described here.

However, I do not know how many awaitables I will execute.

If I were to use threading my code would be like this:

def foo():
   # some blocking call 

result_list = []
threads = []
for i in events:  # events is a list different every time
     newthread = threading.Thread(target=foo, args=())          
     threads.append(newthread)
     newthread.start()
print(result_list)

How can I turn this into async code?

I suppose foo() should look like this:

async def foo():
   global result_list
   result = await blocking_call()
   result_list.append(result)

I have tried to create tasks in another thread, but this approach doesn't seem to function properly.

EDIT

I would like to do something like this:

def foo():
   # some blocking call 

result_list = []
threads = []
for i in events:  # events is a list different every time
     time.sleep(i)
     newthread = threading.Thread(target=foo, args=())          
     threads.append(newthread)
     newthread.start()
print(result_list)

Solution

The other question you linked has the answer. The only thing you need to do differently, if you don't know the number of coroutines ahead of time, is put the calls and awaits of your coroutines in a loop.

But the helper task based approach is still the simplest solution in my opinion.

Here is a very simple demo:

from asyncio import create_task, run, sleep
from collections.abc import Iterable


async def foo(x: float) -> None:
    print(f"Executing foo({x=})")
    await sleep(x)
    print(f"Finished foo({x=})")


async def foo_sequentially(xs: Iterable[float]) -> None:
    for x in xs:
        await foo(x)


async def main() -> None:
    foo_inputs = [0.25, 0.5, 1., 1.5]
    foo_seq_task = create_task(foo_sequentially(foo_inputs))
    ...  # do other stuff, while `foo_seq_task` is scheduled/executes
    print("  Doing other stuff...")
    await sleep(1)
    print("  Doing more stuff...")
    await sleep(1)
    print("  Done with other stuff!")
    await foo_seq_task


if __name__ == "__main__":
    run(main())

Output:

  Doing other stuff...
Executing foo(x=0.25)
Finished foo(x=0.25)
Executing foo(x=0.5)
Finished foo(x=0.5)
Executing foo(x=1.0)
  Doing more stuff...
Finished foo(x=1.0)
Executing foo(x=1.5)
  Done with other stuff!
Finished foo(x=1.5)

As you can see the foo coroutines all executed sequentially with respect to one another, and viewed as a whole (wrapped in foo_sequentially) they executed concurrently to the rest of the things done by main.

That is because by definition an await statement will block the encompassing coroutine (in this case foo_sequentially), until the awaitable in that statement returns (in this case one of the foo coroutines); but the await statement will also yield control to the event loop allowing it to resume execution of some other coroutine (in this case main).

Answered By - Daniil Fajnberg