[FIXED] "RuntimeError: <_overlapped.Overlapped object> still has pending operation at deallocation" while using aiohttp

Issue

I wrote the code where I am trying to parse data using aiohttp, bs4, and asyncio, but I get the following error. What's wrong?

This is my code:

import aiohttp
import asyncio
from bs4 import BeautifulSoup
import pandas as pd

urls = ['https://www.markt.de/frontenhausen/alte-pcs/hp+athlon+64+x2+4200/a/5a4d0c2a/',
        'https://www.markt.de/frontenhausen/desktop-pc/pc+amd+ryzen+9+3950x+32+gb/a/4d6f7c34/',
        'https://www.markt.de/schlieben/desktop-pc/fujitsu+esprimo+p410+85+core+i5+8gb+256gb+ssd+office+pc+win+10+p/a/5a2fcca5/']

async def get_page(session, url):
    async with session.get(url) as r:
        return await r.text()

async def get_all(session,urls):
    tasks = []
    for url in urls:
        task = asyncio.create_task(get_page(session, url))
        tasks.append(task)
    results = await asyncio.gather(*tasks)
    return results

async def main(urls):
    async with aiohttp.ClientSession() as session:
        data = await get_all(session, urls)
        return data

def parse(result):
    data = []
    for html in results:
        soup = BeautifulSoup(html, 'html.parser')
        items = soup.find_all('div', id='clsy-c-expose-wrapper')
        for item in items:
            data.append({'name': item.find('div', class_='clsy-contentsection--hor-padding').find_next('h1', class_='clsy-c-expose__subject').get_text()})
    print(data)
    return data

results = asyncio.run(main(urls))
parse(results)

I get this error:

Traceback (most recent call last):
  File "C:\Users\AppData\Local\Programs\Python\Python39\lib\asyncio\windows_events.py", line 434, in select self._poll(timeout)

RuntimeError: <_overlapped.Overlapped object at 0x000001A1B018F270> still has pending operation at deallocation, the process may crash

Traceback (most recent call last):
  File "C:\Users\AppData\Local\Programs\Python\Python39\lib\asyncio\windows_events.py", line 434, in select self._poll(timeout)

RuntimeError: <_overlapped.Overlapped object at 0x000001A1B018F270> still has pending operation at deallocation, the process may crash

Solution

Try this:

import platform
if platform.system() == 'Windows':
   asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())

Answered By - Igor Burdenyuk