[FIXED] Choose a drop down option using Selenium Python Webscraping

Issue

I am trying to use selenium and python webscraping to choose a drop down option of routes based on my local public transit schedule. Here is the link to the site: https://tmweb.pacebus.com/TMWebWatch/LiveDepartureTimes

The issue I'm running into is that the line:

find_route_takin = driver.find_element(By.CLASS_NAME,"active-result") 

produces an error saying:

Message: no such element: Unable to locate element: {"method":"css selector","selector":".active-result"}

Here is the code for how I am trying to complete the task:

import time

from bs4 import BeautifulSoup
from datetime import datetime
from selenium import webdriver
from selenium.webdriver.common.by import By
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.support.ui import Select

driver = webdriver.Chrome(ChromeDriverManager().install())
driver.maximize_window()
driver.get("https://tmweb.pacebus.com/TMWebWatch/LiveDepartureTimes")

driver.find_element(By.ID, "MainContent_routeList_chosen")

find_route_takin = driver.find_element(By.CLASS_NAME,"active-result")
select_route = Select(find_route_takin)
select_route.select_by_visible_text("565 - Grand Avenue")
time.sleep(20)

Within the HTML of the site for the first drop-down for the routes, there is a ul object that holds many li objects that represent each individual route. I have tried looking through various YouTube videos, but I can't find a solution to my problem. I've read that perhaps I'll have to use xpath instead of CLASS_NAME, but when I tried copying the xpath by right-clicking and choosing copy xpath, I'm still prompted with the same error message. What I want is when I run the program, a chrome window is brought up and the 565 - Grand Avenue route is selected. The error I'm running into is

Message: no such element: Unable to locate element: {"method":"css selector","selector":".active-result"}

Solution

You may try this way:

import time
from selenium.webdriver import Chrome, ChromeOptions
from selenium.webdriver.common.by import By
from selenium.webdriver.support.wait import WebDriverWait
import selenium.webdriver.support.expected_conditions as EC

options = ChromeOptions()
options.add_argument("--start-maximized")

driver = Chrome(options=options)

def choose_route(opt):

    driver.get("https://tmweb.pacebus.com/TMWebWatch/LiveDepartureTimes")
    WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.ID, 'MainContent_routeList_chosen'))).click()

    routes = driver.find_element(By.CSS_SELECTOR, 'ul.chosen-results')
    route_options = routes.find_elements(By.TAG_NAME, 'li')

    for option in route_options:
        if option.text == opt:
            option.click()

    time.sleep(10)

choose_route("565 - Grand Avenue")
# choose_route("213 - Green Bay Road")

Things to note:

1. first we wait for the Route drop-down to get visibly located/loaded on the page in order to click it.
2. As we click, the drop-down gets open, and all the options get loaded on the page. So we find all the route options which are all the li tags under the ul tag with class name chosen-results
3. And finally choose the option based on the text and click.

That's all, and notice that we can not use Select because the drop-down options are not under a select tag.

Answered By - Ajeet Verma