[FIXED] How to multiprocess async/await functions in python?

Issue

I need to run an event every 60 seconds to all my cars. My problem with the code below is the while loop doesn't end until the timeout (60) does and hence only the first car in cars is run.

class RunCars(BaseEvent):

    def __init__(self):
        interval_seconds = 60  # Set the interval for this event
        super().__init__(interval_seconds)

    # run() method will be called once every {interval_seconds} minutes
    async def run(self, client, cars):
        for car in cars:        
            channel = get_channel(client, "general")
            await client.send_message(channel, 'Running this '+str(car))
            await msg.add_reaction(str(get_emoji(':smiley:'))) 
            
            reaction = None
            while True:
                if str(reaction) == str(get_emoji(':smiley:'))
                    await client.send_message(channel, 'Finished with this '+str(car))
                try:
                    reaction, user = await client.wait_for('reaction_add', timeout=60, check=check)
                except:
                    break

I tried changing the code into a multithreaded process but had trouble with async/await inside the function and problems with pickling the function itself.

I'd appreciate any suggestions for how to go about this..

Solution

The asyncio module lets you execute multiple async method concurrently using the gather method. I think you can achieve the behavior you want by defining a method that runs a single car, and then replacing your for-loop with a call to gather, which will execute multiple run_one coroutines (methods) concurrently:

import asyncio

class RunCars(BaseEvent):

    def __init__(self):
        interval_seconds = 60  # Set the interval for this event
        super().__init__(interval_seconds)


    async def run(self, client, cars):
        coroutines = [self.run_one(client, car) for car in cars]
        asyncio.gather(*coroutines)

    async def run_one(self, client, car):
        channel = get_channel(client, "general")
        await client.send_message(channel, 'Running this '+str(car))
        await msg.add_reaction(str(get_emoji(':smiley:'))) 
          
        reaction = None
        while True:
            if str(reaction) == str(get_emoji(':smiley:'))
                await client.send_message(channel, 'Finished with this '+str(car))
            try:
                reaction, user = await client.wait_for('reaction_add', timeout=60, check=check)
            except:
                break

In general, when writing async code, you should try to replace sequential calls to async methods - basically for-loops that call async methods - with gather statements so they execute concurrently.

Answered By - mackorone