[FIXED] web scraping help needed

Issue

I was wondering if someone could help me put together some code for https://finance.yahoo.com/quote/TSCO.l?p=TSCO.L

I currently use this code to scrape the current price

currentPriceData = soup.find_all('div', {'class':'My(6px) Pos(r) smartphone_Mt(6px)'})[0].find('span').text

This works fine but I occasionally get an error not really sure why as the links are all correct. but I would like to try to get the price again

so something like

try: 
    currentPriceData = soup.find_all('div', {'class':'My(6px) Pos(r) smartphone_Mt(6px)'})[0].find('span').text
except Exception:
    currentPriceData = soup.find('span', {'class':'Trsdu(0.3s) Fw(b) Fz(36px) Mb(-4px) D(ib)'})[0].text

The problem is that I can't get it to scrape the number using this method any help would be greatly appreciated.

Solution

EDIT (16.03.2023): Updated to working version

To get current price you can use next example (don't forget to set User-Agent HTTP header):

import requests
from bs4 import BeautifulSoup

ticker = 'TSCO.L'
url = f'https://finance.yahoo.com/quote/{ticker}?p={ticker}'
headers = {'User-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/114.0'}

soup = BeautifulSoup(requests.get(url, headers=headers).text, 'html.parser')

current = soup.select_one(f'fin-streamer[data-field="regularMarketPrice"][data-symbol="{ticker}"]')
print(current['value'])

Prints:

263.2

---

Previous version of this answer was parsing for JSON data embedded in the site's HTML which is no longer valid.

Answered By - Andrej Kesely