[FIXED] Execute a coroutine after asyncio server is started

Issue

I am working on a controller application that monitors and controls subprocesses which are independent python executeables. Basically what I want is that in controller.py running an asyncio.star_server. After the server is up and running the controller.py should execute other python files as clients which will connect to it. The controller server runs forever and create new client instances and also send shutdown message to them if necessary.

Unfortunately this does not work. No error received, it just hangs.

controller.py:

async def handleClient(reader, writer):
    #handling a connection
    addr = writer.get_extra_info("peername")
    print(f"connection from {addr}")

    data_ = await reader.readline()
    ...

async def startClient(client_py_file, host, port):
    # this executes another py file that will connect to this server
    await asyncio.sleep(0.1)
    subprocess.run(["python.exe", client_py_file, host, port])

async def main():
    server = await asyncio.start_server(handleClient, "127.0.0.1", 4000)
    await asyncio.ensure_future(startClient("client.py", "127.0.0.1", 4000)
    await server.wait_closed()

asyncio.run(main())

It seems it executes the client.py that starts, that connects to the server without any error.

client.py:

async def async_client(loop):
    reader, writer = await asyncio.open_connection(host, port, loop = loop)
    writer.writelines([json.dumps("key" : idstr, "msg" : "this is my message"}, b"\n"])
    await writer.drain()
    while True:
        data = await reader.readline()
        ....

now the client hangs on and waits for response from the server. But on the server the handleClient handler is not triggered. Have no idea what goes wrong. Could you please help me? Thank you in advance!

Solution

The problem is that subprocess.run is a blocking function, which waits for the client to finish. During this wait the event loop is blocked and unable to service the incoming connections.

The simplest fix is to replace subprocess.run(...) with subprocess.Popen(...) which does the same thing, but returning a handle to the subprocess without waiting for it to finish. If you need to communicate with the subprocess, you can also use asyncio.create_subprocess_exec(...) which also returns a handle, but one whose methods like wait() are coroutines.

Answered By - user4815162342