[FIXED] Click a link based on value in another column of a table - webscrapping python

Issue

I am trying to scrape a website for a test case and the link. When I enter a certain value in the search box it return multiple values in the table and I would want to click on the link in the column ('License Number') when the value in the column 'License Name' matches with the string I am looking for.

For example I would want to click on the License Number 0747600 when the value in the column License Name == 'AON Consulting, Inc.'

So far I have gotten to the point below

driver = webdriver.Chrome('YOUR_PATH_TO_chromedriver.exe_FILE')
form_url = "https://cdicloud.insurance.ca.gov/cal/LicenseNumberSearch?handler=Search"
driver.get(form_url)

search_box = driver.find_element('id','SearchLicenseNumber').send_keys("0747600")

WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.ID, "btnSearch"))).click()

click_search=driver.find_element('id','btnSearch')
click_search.send_keys(Keys.ENTER)

page_source = driver.page_source

soup = BeautifulSoup(page_source, "html.parser")

tables = soup.find('table')

#  Looking for the table with the classes 'wikitable' and 'sortable'
table = soup.find('table', id='searchResult')

df = pd.DataFrame(columns=['license_name', 'license_number'])
license_name = []
license_number =[]
# Collecting Ddata
for row in table.tbody.find_all('tr'):    
    # Find all data for each column
    columns = row.find_all('td')

    if(columns != []):
        #license_name = columns[0].text.strip()
        license_name.append(columns[0].text.strip())
        license_number.append(columns[1].text.strip())

select_finder = "//td[contains(text(), 'AON Consulting, Inc.')]/td[1]/a"
driver.find_element("xpath",select_finder).click()

The last 2 lines of the code is throwing me the error

Message: no such element: Unable to locate element:

Solution

You can do this with just Selenium

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait

driver = webdriver.Chrome()
driver.maximize_window()

url = "https://cdicloud.insurance.ca.gov/cal/LicenseNumberSearch?handler=Search"
driver.get(url)

license_number = "0747600"
license_name = "AON CONSULTING, INC."
wait = WebDriverWait(driver, 10)
driver.find_element(By.ID, "SearchLicenseNumber").send_keys(license_number)
driver.find_element(By.ID, "btnSearch").click()
wait.until(EC.element_to_be_clickable((By.XPATH, f"//table[@id='searchResult']//tr[./td[text()='{license_name}']]/td[2]/a"))).click()
print(license_name)

The final XPath is the most complicated bit...

//table[@id='searchResult']//tr[./td[text()='{license_name}']]/td[2]/a
  ^ find a TABLE tag with the ID 'searchResult'
                             ^ that has a descendant TR
                               ^ that TR has a child TD that contains the text in the variable 'license_name'
                                                               ^ from that TR, get the second TD
                                                                     ^ and finally the child A

Answered By - JeffC