[FIXED] Handling a lot of concurrent connections in Python 3 asyncio

Issue

Iam trying to improve the performance of my application. It is a Python3.6 asyncio.Protocol based TCP server (SSL wrapped) handling a lot of requests.

It works fine and the performance is acceptable when only one connection is active, but as soon as another connection is opened, the client part of the application slows down. This is really noticeable once there are 10-15 client connection.

Is there a way to properly handle requests in parallel or should I resort to running multiple server instances?

/edit Added code

main.py

if __name__ == '__main__':
    import package.server
    server = package.server.TCPServer()
    server.join()

package.server

import multiprocessing, asyncio, uvloop
asyncio.set_event_loop_policy(uvloop.EventLoopPolicy())

from package.connection import Connection

class TCPServer(multiprocessing.Process):
    name = 'tcpserver'
    def __init__(self, discord_queue=None):
        multiprocessing.Process.__init__(self)
        self.daemon = True

        # some setup in here

        self.start()

    def run(self):
        loop = uvloop.new_event_loop()
        self.loop = loop

        # db setup, etc

        server = loop.create_server(Connection, HOST, PORT, ssl=SSL_CONTEXT)
        loop.run_until_complete(server)
        loop.run_forever()

package.connection

import asyncio, hashlib, os

from time import sleep, time as timestamp

class Connection(asyncio.Protocol):
    connections = {}

    def setup(self, peer):
        self.peer = peer

        self.ip, self.port = self.peer[0], self.peer[1]
        self.buffer = []

    @property
    def connection_id(self):
        if not hasattr(self, '_connection_id'):
            self._connection_id = hashlib.md5('{}{}{}'.format(self.ip, self.port, timestamp()).encode('utf-8')).hexdigest()
        return self._connection_id

    def connection_lost(self, exception):
        del Connection.connections[self.connection_id]

    def connection_made(self, transport):
        self.transport = transport
        self.setup(transport.get_extra_info('peername'))
        Connection.connections[self.connection_id] = self

    def data_received(self, data):
        # processing, average server side execution time is around 30ms
        sleep(0.030)
        self.transport.write(os.urandom(64))

The application runs on Debian 9.9 and is started via systemd

To "benchmark" I use this script:

import os, socket

from multiprocessing import Pool
from time import time as timestamp

def foobar(i):
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

    s.connect(('127.0.0.1', 60000))

    while True:
        ms = timestamp()*1000
        s.send(os.urandom(128))
        s.recv(1024*2)
        print(i, timestamp()*1000-ms)

if __name__ == '__main__':
    instances = 4
    with Pool(instances) as p:
        print(p.map(foobar, range(0, instances)))

Solution

To answer my own question here. I went with a solution that spawned multiple instances which were listening on base_port + x and I put a nginx TCP loadbalancer in front of it.

The individual TCPServer instances are still spawned as own process and communicate among themselves via a separate UDP connection and with the main process via multiprocessing.Queue.

While this does not "fix" the problem, it provides a somewhat scalable solution for my very specific problem.

Answered By - pistol-whip