[FIXED] Selenium Python not scraping from this website

Issue

I was trying to interact with this website using Selenium in python. I wrote this code to select the radio button using XPATH. But some weird error is showing in my terminal. Can anyone please solve this problem? I tried but can't figure out the problem.

My code.

from select import select
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import csv
import time
PATH = "C:\Program Files (x86)\chromedriver.exe"

driver = webdriver.Chrome(PATH)

driver.get('https://www2.illinois.gov/idoc/Offender/Pages/InmateSearch.aspx')

button = driver.find_element_by_xpath('/html/body/table/tbody/tr/td/table[2]/tbody/tr/td[1]/table/tbody/tr/td/form/table/tbody/tr/td/input[2]')
button.click()

driver.implicitly_wait(10)

driver.quit()

Error :

DevTools listening on ws://127.0.0.1:62348/devtools/browser/ce37da62-856d-4159-ad45-9eca8e63115a
E:\Fiverr job\Orders\1\test.py:18: DeprecationWarning: find_element_by_xpath is deprecated. Please use find_element(by=By.XPATH, value=xpath) instead
  button = driver.find_element_by_xpath('/html/body/table/tbody/tr/td/table[2]/tbody/tr/td[1]/table/tbody/tr/td/form/table/tbody/tr/td/input[2]')
Traceback (most recent call last):
  File "E:\Fiverr job\Orders\1\test.py", line 18, in <module>
    button = driver.find_element_by_xpath('/html/body/table/tbody/tr/td/table[2]/tbody/tr/td[1]/table/tbody/tr/td/form/table/tbody/tr/td/input[2]')
  File "E:\Fiverr job\Orders\1\env\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 526, in find_element_by_xpath
    return self.find_element(by=By.XPATH, value=xpath)
  File "E:\Fiverr job\Orders\1\env\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 1251, in find_element
    return self.execute(Command.FIND_ELEMENT, {
  File "E:\Fiverr job\Orders\1\env\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 430, in execute
    self.error_handler.check_response(response)
  File "E:\Fiverr job\Orders\1\env\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 247, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/table/tbody/tr/td/table[2]/tbody/tr/td[1]/table/tbody/tr/td/form/table/tbody/tr/td/input[2]"}
  (Session info: chrome=102.0.5005.63)
Stacktrace:
Backtrace:
        Ordinal0 [0x0054D953+2414931]
        Ordinal0 [0x004DF5E1+1963489]
        Ordinal0 [0x003CC6B8+837304]
        Ordinal0 [0x003F9500+1021184]
        Ordinal0 [0x003F979B+1021851]
        Ordinal0 [0x00426502+1205506]
        Ordinal0 [0x004144E4+1131748]
        Ordinal0 [0x00424812+1198098]
        Ordinal0 [0x004142B6+1131190]
        Ordinal0 [0x003EE860+976992]
        Ordinal0 [0x003EF756+980822]
        GetHandleVerifier [0x007BCC62+2510274]
        GetHandleVerifier [0x007AF760+2455744]
        GetHandleVerifier [0x005DEABA+551962]
        GetHandleVerifier [0x005DD916+547446]
        Ordinal0 [0x004E5F3B+1990459]
        Ordinal0 [0x004EA898+2009240]
        Ordinal0 [0x004EA985+2009477]
        Ordinal0 [0x004F3AD1+2046673]
        BaseThreadInitThunk [0x7648FA29+25]
        RtlGetAppContainerNamedObjectPath [0x77BE7A7E+286]
        RtlGetAppContainerNamedObjectPath [0x77BE7A4E+238]

Solution

This is because the element you try to click is located into an iframe:

So you must first switch to it before finding and clicking the desired button:

import time

driver = webdriver.Chrome(options=options, desired_capabilities=capabilities)

driver.get('https://www2.illinois.gov/idoc/Offender/Pages/InmateSearch.aspx')

#wait a little to be sure the iframe is loaded
time.sleep(2)

#find and switch to the iframe
iframe = driver.find_element(By.XPATH, '//*[@id="soi-iframe"]')
driver.switch_to.frame(iframe)

button = driver.find_element(By.XPATH, '/html/body/table/tbody/tr/td/table[2]/tbody/tr/td[1]/table/tbody/tr/td/form/table/tbody/tr/td/input[2]')
button.click()

driver.quit()

Proof of work: (click on pic to zoom in)

Answered By - Drakax