[FIXED] Python Asynchronous FQDN Resolution

Issue

I have a list of 10k ips and I need to get their FQDN. Doing this synchronously takes ages, so I tried coding it asynchronously, but I don't see any difference in execution times.

Synchronous method:

def test_synch():
    start_time = time.time()
    for ip in ip_list:
        fqdn = socket.getfqdn(ip)
        print(fqdn)
    print("Time for synchronous requests: ", time.time()-start_time)

Execution time: 284 seconds for 100 ip addresses

Asynchronous method:

async def get_fqdn_async(ip):
    return socket.getfqdn(ip)


async def get_fqdn(ip):
    print("executed task for ip", ip)
    fqdn = await get_fqdn_async(ip)
    print("got fqdn ", fqdn, " for ip ", ip)
    return fqdn


async def main():
    tasks = []
    for ip in ip_list:
        task = asyncio.create_task(
            get_fqdn(ip))
        tasks.append(task)

    fqdns = await asyncio.gather(*tasks)
    print(fqdns)

def test_asynch():
    start_time = time.time()
    asyncio.run(main())
    print("Time for asynchornous requests: ", time.time()-start_time)

Execution time: 283 seconds for 100 ips

Obviously I am doing something wrong, but I can't figure out what.

Solution

, Seems to me that multithreading would be ideal here. Consider this:

from concurrent.futures import ThreadPoolExecutor
import socket
import json

list_of_ips = ['www.google.com', 'www.bbc.co.uk', 'www.tripdavisor.com', 'www.stackoverflow.com', 'www.facebook.com']

def getfqdn(ip):
    return ip, socket.getfqdn(ip)

results = dict()
with ThreadPoolExecutor() as executor:
    for future in [executor.submit(getfqdn, ip) for ip in set(list_of_ips)]:
        ip, fqdn = future.result()
        results[ip] = fqdn

with open('report.json', 'w') as j:
    json.dump(results, j, indent=4)

Answered By - JCaesar