[FIXED] Copied xpath directly from chrome, and not working

Issue

Trying to scrape odds from fanduel, goal is to get the player's name. In this case Jayson Tatum.

https://sportsbook.fanduel.com/basketball/nba/philadelphia-76ers-@-boston-celtics-31137202?tab=player-points

Even when I copy the xpath directly from chrome it doesnt seem to work. Though it works when I hardcode and look for an element through xpath containing the text Jayson Tatum. This is my code

name = WebDriverWait(driver, 20).until(
                    EC.presence_of_element_located((By.XPATH,'//*[@id="root"]/div/div[2]/div[1]/div/div[2]/div[3]/div/div[2]/div/div[3]/div[1]/div/div/div[1]/span')))

Also tried this

name = driver.find_element(By.XPATH, '//*[@id="root"]/div/div[2]/div[1]/div/div[2]/div[3]/div/div[2]/div/div[3]/div[1]/div/div/div[1]/span')

Still get a NoSuchElement trying both ways.

Solution

To print the text Jayson Tatum you can use the following Locator Strategy:

• Using xpath and text attribute:

  print(driver.find_element(By.XPATH, "//span[text()='UNDER']//following::div[1]//span").text)
  

---

Ideally you need to induce WebDriverWait for the visibility_of_element_located() and you can use either of the following Locator Strategy:

• Using XPATH and get_attribute("innerHTML"):

  print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//span[text()='UNDER']//following::div[1]//span"))).get_attribute("innerHTML"))
  

• Note : You have to add the following imports :

  from selenium.webdriver.support.ui import WebDriverWait
  from selenium.webdriver.common.by import By
  from selenium.webdriver.support import expected_conditions as EC
  

You can find a relevant discussion in How to retrieve the text of a WebElement using Selenium - Python

---

References

Link to useful documentation:

• get_attribute() method Gets the given attribute or property of the element.
• text attribute returns The text of the element.
• Difference between text and innerHTML using Selenium

Answered By - DebanjanB