[FIXED] asyncio semaphore and wait task ordering patterns

Issue

Consider the following code for managing concurrency with identical async tasks

import asyncio

async def performTask(id):

  await asyncio.sleep(1)
  print(id)


async def runBatchItem(semaphore, task):
  await semaphore.acquire()
  await task
  semaphore.release()


async def main():

  # all tasks
  tasks = [performTask(i) for i in range(20)]

  # concurrency handler
  MAX_CONCURRENT = 3
  semaphore = asyncio.Semaphore(value=MAX_CONCURRENT)
  stasks = [runBatchItem(semaphore, task) for task in tasks]

  await asyncio.wait(stasks)


asyncio.run(main())

No matter how often I run it, I always end up with the following sequence of outputs

3 19 4 5 6 7 8 17 9 10 11 12 13 0 14 1 15 2 16 18

• Question 1. What is the logic to this ordering of my tasks?
• Question 2. What if I want the tasks to be processed in approximate insert order? I.e, like working through a queue with limited concurrency.

Thanks in advance!

Solution

As Andrew Svetlov (asyncio developer) answered here:

The order is undetermenistic by .wait() specification.

If you start your script on another machine you will get different result. If you want impose an order on task execution, you can just await for them in a loop or use asyncio synchronization primitive such as Event or Condition within coroutines.

Answered By - alex_noname