[FIXED] How to get the current stock price using fidelity's screener?

Issue

I am trying to get the current stock price using fidelity's screener. For example, the current price of AAPL is $165.02 on https://digital.fidelity.com/prgw/digital/research/quote/dashboard/summary?symbol=AAPL.

When I inspect the webpace, the price is shown here: <div _ngcontent-cxa-c16="" class="nre-quick-quote-price">$165.02</div>.

I used this code:

import requests
from bs4 import BeautifulSoup

def stock_price(symbol: str = "AAPL") -> str:

    url = f"https://digital.fidelity.com/prgw/digital/research/quote/dashboard/summary?symbol={symbol}"
    response = requests.get(url)
    soup = BeautifulSoup(response.text, "html.parser")
    price_tag = soup.find('div', class_='nre-quick-quote-price')
    current_price = price_tag['value']

    return current_price

But got this error:

Traceback (most recent call last):
  File "get_price.py", line 160, in <module>
    print(f"Current {symbol:<4} stock price is {stock_price(symbol):>8}")
  File "get_price.py", line 63, in stock_price
    current_price = price_tag['value']
TypeError: 'NoneType' object is not subscriptable

I also used this code:

from selenium import webdriver

def stock_price(symbol: str = "AAPL") -> str:

    driver = webdriver.Chrome()
    url = "https://digital.fidelity.com/prgw/digital/research/quote/dashboard/summary?symbol=" + symbol
    driver.get(url)

    current_price = driver.find_element('div.nre-quick-quote-price').text

    return current_price

But got this error:

Traceback (most recent call last):
  File "get_price.py", line 103, in <module>
    price_tag = driver.find_element('div.nre-quick-quote-price')
  File "C:\Users\X\miniconda3\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 831, in find_element
    return self.execute(Command.FIND_ELEMENT, {"using": by, "value": value})["value"]
  File "C:\Users\X\miniconda3\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 440, in execute
    self.error_handler.check_response(response)
  File "C:\Users\X\miniconda3\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 245, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.InvalidArgumentException: Message: invalid argument: invalid locator

Please help!

Solution

Root cause of your issue: Syntax is not correct. See below.

In your second code, change the below line from:

current_price = driver.find_element('div.nre-quick-quote-price').text

To:

current_price = driver.find_element(By.CSS_SELECTOR, 'div.nre-quick-quote-price').text

Full working code: Note: I have added explicitwaits in your code to ensure code is more consistent, this way even if the website is little slow in responding/loading your code will handle it.

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.wait import WebDriverWait


def stock_price(symbol: str = "AAPL") -> str:
    driver = webdriver.Chrome()
    url = "https://digital.fidelity.com/prgw/digital/research/quote/dashboard/summary?symbol=" + symbol
    driver.get(url)

    wait = WebDriverWait(driver, 10)
    current_price = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, 'div.nre-quick-quote-price'))).text
    return current_price

print(stock_price())

Result:

$165.02

Process finished with exit code 0

UPDATED code to run in headless mode:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.wait import WebDriverWait

def stock_price(symbol: str = "AAPL") -> str:
    options = webdriver.ChromeOptions()
    options.add_argument('--headless')
    options.add_argument("--user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.99 Safari/537.36")
    driver = webdriver.Chrome(options=options)
    url = "https://digital.fidelity.com/prgw/digital/research/quote/dashboard/summary?symbol=" + symbol
    driver.get(url)

    wait = WebDriverWait(driver, 10)
    current_price = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, 'div.nre-quick-quote-price'))).text
    return current_price

print(stock_price())

Answered By - Shawn