[FIXED] How to iterate in each element?

Issue

import requests
from bs4 import BeautifulSoup

url = "https://bigpara.hurriyet.com.tr/borsa/hisse-fiyatlari/"

r = requests.get(url)
soup = BeautifulSoup(r.content,"lxml")
details = soup.find_all("div",attrs={"class":"tBody"})
#print(type(details))

for detail in details:
    
    print("{} {}".format(
        detail.a.string,
        detail.find("li",attrs={"class":"cell004"}).text.strip()
        ))

We need to take the stocks with BeautifulSoup. Code only iterates the first element. How to iterate each one?

Solution

First find all ul elements and iterate over it to find title and values according to class wise and append items to dict1 with key title

dict1={}
for ul in details.find_all("ul"):
    title=ul.find("li",class_="cell003 tal arrow").get_text()
    values=[i.get_text(strip=True) for i in ul.find_all("li",class_=["cell004","cell003"])][1:]
    dict1[title]=values

Now you can use pandas module to transform data to DataFrame using pd.DataFrame method and get data according to required

import pandas as pd

df=pd.DataFrame(dict1)
df=df.transpose()

columns=[i.get_text(strip=True) for i in soup.find("div",class_="tHead").find_all("li")][1:]
df.columns=columns

Output:

           Son    Dün   ( % )   Yüksek  Düşük   Ağ. Ort  Hacim(LOT) Hacim(TL)
ACSEL   51,00   50,55   0,89    51,45   50,25   50,84   183.227 9.315.538
ADEL    22,98   22,98   0,00    23,20   22,72   22,93   67.600  1.550.269
.....

Answered By - Bhavya Parikh