[FIXED] how can I approach scraping a real estate website with BeautifulSoup4

Issue

(new to this, so apologies if the question is stupid, I don't need a solution but just a push in the right direction)

I am trying to scrape a real estate website using bs4, problem is I can't find data I am looking for. when I parse the html I can't see the information I need from the website. Is it even possible to scrape this website?

This is what I did to see if I can find the right tag to extract housing listings from the website.

url = urllib.request.urlopen('https://www.sreality.cz/hledani/prodej/domy/rodinne-domy/praha,stredocesky-kraj?velikost=3-pokoje,4-pokoje,5-a-vice').read()
soup = bs(url,'lxml')
print(soup)

Solution

@baduker already has stated about it's dynamic nature and the way of the possible solution. The website is very lazy load and dynamic. So I use selenium with bs4.

Working code as an examole:

from selenium import webdriver
import time
from bs4 import BeautifulSoup
from selenium.webdriver.chrome.service import Service
import pandas as pd
webdriver_service = Service("./chromedriver") #Your chromedriver path
driver = webdriver.Chrome(service=webdriver_service)


driver.get('https://www.sreality.cz/hledani/prodej/domy/rodinne-domy/praha,stredocesky-kraj?velikost=3-pokoje,4-pokoje,5-a-vice')
driver.maximize_window()
time.sleep(10)


soup = BeautifulSoup(driver.page_source,"html.parser")
for card in soup.select('[class="text-wrap"]'): 
    title = card.h2.get_text(strip=True) 
    print(title)

Output:

Prodej  rodinného domu 641 m², pozemek 1 539 m²
Prodej  rodinného domu 250 m², pozemek 709 m²
Prodej  rodinného domu 533 m², pozemek 3 795 m²
Prodej  rodinného domu 68 m², pozemek 221 m²
Prodej  rodinného domu 191 m², pozemek 821 m²
Prodej  rodinného domu 126 m², pozemek 401 m²
Prodej  rodinného domu 137 m², pozemek 411 m²
Prodej  rodinného domu 320 m², pozemek 350 m²
Prodej  rodinného domu 335 m², pozemek 2 315 m²
Prodej  rodinného domu 122 m², pozemek 340 m²
Prodej  rodinného domu 125 m², pozemek 220 m²
Prodej  rodinného domu 111 m², pozemek 227 m²
Prodej  rodinného domu 111 m², pozemek 204 m²
Prodej  rodinného domu 161 m², pozemek 157 m²
Prodej  rodinného domu 172 m², pozemek 834 m²
Prodej  rodinného domu 96 m², pozemek 294 m²
Prodej  rodinného domu 137 m², pozemek 380 m²
Prodej  rodinného domu 191 m², pozemek 502 m²
Prodej  rodinného domu 641 m², pozemek 1 539 m²
Prodej  rodinného domu 449 m², pozemek 784 m²
Prodej  rodinného domu 221 m², pozemek 578 m²

Answered By - Fazlul