Issue
I'm trying to parse html
from this ecommerce website with selenium
, beautifulsoup
in python
to get following text
from product titles:
"Electric outlet 220V 32A IP44" and "34.60"
Problem:
I get empty values for my variables productCardTitle
and productPriceValue
.
This is my html:
<ul class="products-cards products-cards--view-grid grid">
<li data-uid="210a10d0-4976-4c94-9ab4-0e7d38459c02" class="products-cards__item product-card">
<div class="product-card__top">...</div>
<div class="product-card__content">...</div>
<div class="product-card__title-wrapper">
<h3 class="product-card__title">
<a href="/catalog/Electric-outlet-220V-32A-IP44">
Electric outlet 220V 32A IP44
</a>
</h3>
</div>
...
<div class="product-card__bottom">
<p class="product-card__price product-price product-price--small">...</p>
<p class="product-card__price product-price">
<span class="visually-hidden">Price:</span>
<span class="product-price__value">34.60</span>
<span class="product-price__currency">...</span>
...
</li>
</ul>
This is my code in python:
productCards = soup.find_all('li', class_="products-cards__item product-card")
for productCard in productCards:
productCardTitle = productCard.find_all('h3', class_="product-card__title")
for product in productCardTitle:
title = product.findChildren('a')[0].string.strip()
print(title)
productPriceValue = productCard.find_all('span', class_="product-price__value")
for product in productPriceValue:
price = product.string.strip()
print(price)
I would appreciate if someone could give me some help on how to solve this problem.
Solution
Not sure what is going wrong, that your results are empty. Tryed it with selenium
as well as with requests
and both provide valide information.
Anyway - It do not need that numbers of loops to grab the information, lets take a look how to make it more lean.
Select all cards like this:
soup.select('li.product-card')
Iterate once over the result set and create a list
of dicts
:
[
{'title': card.h3.get_text(strip=True),
'price': card.select_one('span.product-price__value').get_text()
}
for card in soup.select('li.product-card')
]
Example (selenium)
from bs4 import BeautifulSoup
from selenium import webdriver
driver = webdriver.Chrome('YOUR PATH TO CHROME')
driver.get('https://shop-aventa.ru/search?q=+%D0%A0%D0%B0%D0%B7%D1%8A%D0%B5%D0%BC+220+')
soup=BeautifulSoup(driver.page_source, 'html.parser')
data = [
{'title': card.h3.get_text(strip=True),
'price': card.select_one('span.product-price__value').get_text()
}
for card in soup.select('li.product-card')
]
data
Example (requests)
import requests
from bs4 import BeautifulSoup
headers ={
'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36'
}
r =requests.get('https://shop-aventa.ru/search?q=+%D0%A0%D0%B0%D0%B7%D1%8A%D0%B5%D0%BC+220+')
soup=BeautifulSoup(r.content, 'html.parser')
data = [
{'title': card.h3.get_text(strip=True),
'price': card.select_one('span.product-price__value').get_text()
}
for card in soup.select('li.product-card')
]
data
Output
[{'title': 'Разъем 220 ЭКФ (Розетка 233), 63А, IP67', 'price': '934.60'}, {'title': 'Разъем 220 ЭКФ (Розетка 223), 32А, IP44', 'price': '286.00'}, {'title': 'Разъем 220 ЭКФ (Вилка 023), 32А, IP44', 'price': '239.40'}, {'title': 'Разъем 220 ЭКФ (Розетка 123), 32А, IP44', 'price': '342.80'}, {'title': 'Разъем 220 ЭКФ (Розетка 213), 16А, IP44', 'price': '200.70'}, {'title': 'Разъем 220 ЭКФ (Розетка 113), 16А, IP44', 'price': '269.70'}, {'title': 'Разъем 220 ЭКФ (Вилка 013), 16А, IP44', 'price': '161.50'}, {'title': 'Разъем 220 ЭКФ (Розетка 413), 16А, IP44', 'price': '269.20'}, {'title': 'Разъем 220В DKC 32А 220В, 2P+E, наст. IP44', 'price': '595.50'}, {'title': 'Разъем 220 ИЭК (Вилка 513), 16А, IP44, MAGNUM', 'price': '533.58'}, {'title': 'Разъем 220 ИЭК (Вилка 033), 63А, IP67, MAGNUM', 'price': '1 727.41'}, {'title': 'Разъем 220 ИЭК (Розетка 223), 32А, IP 44', 'price': '348.21'}, {'title': 'Разъем 220 ИЭК (Розетка 113), 16А, IP44, MAGNUM', 'price': '427.15'}, {'title': 'Разъем 220 ИЭК (Розетка 133), 63А, IP67, MAGNUM', 'price': '2 607.28'}, {'title': 'Разъем 220 ИЭК (Розетка 233), 63А, IP44', 'price': '2 051.63'}, {'title': 'Разъем 220 ИЭК (Вилка 023), 32А, IP44, MAGNUM', 'price': '362.05'}, {'title': 'Разъем 220 ИЭК (Розетка 113), 16А, IP 44', 'price': '330.47'}, {'title': 'Разъем 220 ИЭК (Вилка 023), 32А, IP 44', 'price': '291.02'}, {'title': 'Разъем 220 ИЭК (Розетка 213), 16А, IP44, MAGNUM', 'price': '341.92'}, {'title': 'Разъем 220 ИЭК (Вилка 013), 16А, IP 44', 'price': '200.54'}, {'title': 'Разъем 220 ИЭК (Вилка 033), 63А, IP54', 'price': '1 460.27'}, {'title': 'Разъем 220 ИЭК (Розетка скрытая 413), 16А, IP44', 'price': '350.79'}, {'title': 'Разъем 220 ИЭК (Розетка 133), 63А, IP54', 'price': '2 069.92'}, {'title': 'Разъем 220 ИЭК (Розетка 423), 32А, IP44, MAGNUM', 'price': '567.41'}]
Answered By - HedgeHog
0 comments:
Post a Comment
Note: Only a member of this blog may post a comment.