[FIXED] Best way to create a new thread from an async method in python

Issue

I'm currently writing a discord bot which needs to be able to run a task that could take anywhere from a few seconds to a minute while still being responsive to other commands. Forgive me if this is a fairly simple question, but I haven't been able to find a solution that worked yet.

Here is an abridged version of the code

class StableCog(commands.Cog, name='Stable Diffusion', description='Create images from natural language.'):
def __init__(self, bot):
    self.text2image_model = Text2Image()
    self.bot = bot

@commands.slash_command(description='Create an image.')
async def dream(self, -- a ton of arguments -- ):
    print(f'Request -- {ctx.author.name}#{ctx.author.discriminator} -- Prompt: {query}')
    asyncio.get_event_loop().create_task(src.bot.queue_system.dream_async( -- a ton of arguments -- ))

inside queue_system.py

async def dream_async(-- a ton of arguments --):
await ctx.interaction.response.send_message('Added to queue! You are # in queue')
embed = discord.Embed()
try:
    #lots of code, I've removed it since it doesn't have anything to do with the async
    await ctx.channel.send(embed=embed, file=discord.File(fp=buffer, filename=f'{seed}.png'))

except Exception as e:
    embed = discord.Embed(title='txt2img failed', description=f'{e}\n{traceback.print_exc()}', color=embed_color)
    await ctx.channel.send(embed=embed)

However, the discord bot becomes unresponsive until the code in queue_system.py finishes running. Every solution I've tried so far hasn't worked correctly since I'm trying to create a thread to run an asynchronous method. What would be the best way to do so? Ignore the name queue_system.py, it isn't quite a queue system yet, I'm just working out how to run the dream method asynchronously before I work that out.

Solution

What is blocking the event loop in the dream_async coroutine ? In that coroutine if you're calling some (non async) functions that have the potential to block the loop, that's an issue, but the real culprit must be in the "lots of code" part :)

A good option would be to use run_in_executor() to run the non async code in a threadpool, and therefore prevent that code from blocking dream_async.

def blocking_stuff(arg):
    # this will run in a thread
    ...
    
    return 'something'


async def dream_async(-- a ton of arguments --):
    loop = asyncio.get_event_loop()

    await ctx.interaction.response.send_message('Added to queue! You are # in queue')
    embed = discord.Embed()

    try:
        # Run the blocking part in a threadpool
        result = await loop.run_in_executor(None, blocking_stuff, 'test') 

        await ctx.channel.send(embed=embed, file=discord.File(fp=buffer, filename=f'{seed}.png'))

    except Exception as e:
        embed = discord.Embed(title='txt2img failed', description=f'{e}\n{traceback.print_exc()}', color=embed_color)
        await ctx.channel.send(embed=embed)

Hope i didn't misunderstand you.

Answered By - cipres