Issue
url=http://www.mercadopublico.cl/Procurement/Modules/RFB/DetailsAcquisition.aspx?idlicitacion=3951-24-L122
print(i)
soup = BeautifulSoup(requests.get(i).content, "html.parser")
soup.select("#box_a2 .box_a_text_00")
licitation_number = soup.select_one("#lblNumLicitacion").text
responsable = soup.select_one("#lblResponsable").text
ficha = soup.select_one("#lblFicha2Reclamo").text
nombre_licitacion=soup.select_one("#lblNombreLicitacion").text
try:
estado_licitacion = soup.select_one("#lblFicha1Estado").text
except ValueError:
i have severals links where I make my loop
sometimas this part
estado_licitacion = soup.select_one("#lblFicha1Estado").text
get erros because does not exist, tryed to fixed with try and except but keep getting this error
AttributeError: 'NoneType' object has no attribute 'text'
when doesnt exist, the example is done for error, the loop doesnt fail all the time
Solution
You should check if your element is available in your soup
:
estado_licitacion = e.text if (e:=soup.select_one("#lblFicha1Estado")) else 'no estato licitation available'
Note walrus operator
requires Python 3.8
or later to work.*
Alternative for < Python 3.8
:
estado_licitacion = soup.select_one("#lblFicha1Estado").text if soup.select_one("#lblFicha1Estado") else 'no estato licitation available'
Example
import requests
from bs4 import BeautifulSoup
idList = ['3951-24-L122']
data = []
for idlicitacion in idList:
url=f'http://www.mercadopublico.cl/Procurement/Modules/RFB/DetailsAcquisition.aspx?idlicitacion={idlicitacion}'
soup = BeautifulSoup(requests.get(url).content)
data.append({
'licitation_number': e.text if (e:=soup.select_one("#lblNumLicitacion")) else 'no licitation number available',
'responsable': e.text if (e:=soup.select_one("#lblResponsable")) else 'no responsable available',
'ficha': e.text if (e:=soup.select_one("#lblFicha2Reclamo")) else 'no ficha available',
'estado_licitacion': e.text if (e:=soup.select_one("#lblFicha1Estado")) else 'no estato licitation available'
})
data
Output
[{'licitation_number': '3951-24-L122', 'responsable': 'CONSULTORIO GENERAL URBANO VICTOR DOMINGO SILVA, Consultorio Victor Domingo Silva', 'ficha': '0', 'estado_licitacion': 'no estato licitation available'}]
Answered By - HedgeHog
0 comments:
Post a Comment
Note: Only a member of this blog may post a comment.