[FIXED] Python Web scraping | How to handle missing elements using try and except so that it prints as Not available if element not is found?

Issue

from bs4 import BeautifulSoup as soup

from urllib.request import urlopen as uReq

import bs4

headers = {'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Brave Chrome/83.0.4103.116 Safari/537.36'}

my_url = 'https://www.jiomart.com/c/groceries/dairy-bakery/dairy/62'



uclient = uReq(my_url)

page_html = uclient.read()

uclient.close()


bs41 = soup(page_html, 'html.parser')


containers = bs41.find_all('div', {'col-md-3 p-0'})
#print(len(containers))


#print(soup.prettify(containers[0]))



for container in containers:
    p_name = container.find_all('span', {'class' : 'clsgetname'})
    productname = p_name[0].text

    o_p = container.find_all('span' , id = 'final_price' )
    offer_price = o_p[0].text


    try:
        ap = container.find_all('strike', id = 'price')
        actual_price = ap[0].text

    except:
        print('not available')

    

    print('Product name is', productname)
    print('Product Mrp is', offer_price)
    print('Product actual price', actual_price)
    
    
    print()


   

While performing the above code, There is a product which doesn't have a actual price and has offer price only. But other products are having both the values. When I'm trying to handle the exception via try and except by printing 'Not Available' it's not Working.

Rather It's printing it on the first-line as Not Available and it's also showing a actual price of rs 35 whereas actual price is null.

How should i deal with these things, so it may help me.

Solution

The issue is that even if it does not find the element, it still prints actual_price which is probably in an outer scope.

You have 2 ways to approach this.

• The 1st is to only print if the element was found, for which you can do:

    try:
        ap = container.find_all('strike', id = 'price')
        actual_price = ap[0].text
        print('Product name is', productname)
        print('Product Mrp is', offer_price)
        print('Product actual price', actual_price)

    except:
        print('not available')

• The 2nd is to set actual_price to "not available", so it prints not available next to 'Product actual price'. To make this work you just need to add actual_price = 'not found' in your except block, so your code would become:

    try:
        ap = container.find_all('strike', id = 'price')
        actual_price = ap[0].text

    except:
        print('not available')
        actual_price = 'not found'

Answered By - Berlm