[FIXED] How to handle dropdown without select in selenium python

Issue

Hello I would like to be able to change the value of "50 Profiles / Page" to "500 Profiles / Page", but the problem is that in the HTML there is no "Select" tag.

I tried doing this but it didn't work

from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

url = 'https://www.personality-database.com/profile?pid=1&sort=hot'

driver = webdriver.Chrome(ChromeDriverManager().install())
driver.maximize_window()
driver.implicitly_wait(30)
driver.get(url)

WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, '//*[@id="root"]/div/section/main/div[1]/div[2]/div/div[5]/ul/li[10]/div/div[1]/span[2][text()="500 Profiles / Page"]'))).click()

Here is the code The HTML code

<li class="rc-pagination-options">
    <div class="rc-select rc-pagination-options-size-changer rc-select-single rc-select-show-arrow">
        <span class="rc-select-arrow" unselectable="on" aria-hidden="true">
            <span class="rc-select-arrow-icon"></span></span>
                <div class="rc-select-dropdown rc-select-dropdown-placement-topLeft  rc-select-dropdown-hidden">
                        <div role="listbox" id="rc_select_0_list">
                            <div aria-label="20 Profiles / Page" role="option" id="rc_select_0_list_0"
                                aria-selected="false">20</div>
                        </div>
                        <div class="rc-virtual-list" style="position: relative;">
                            <div class="rc-virtual-list-holder">
                                    <div class="rc-virtual-list-holder-inner"
                                        style="display: flex; flex-direction: column;">
                                        <div aria-selected="false" class="rc-select-item rc-select-item-option"
                                            title="20 Profiles / Page">
                                            <div class="rc-select-item-option-content">20 Profiles / Page</div><span
                                                class="rc-select-item-option-state" unselectable="on" aria-hidden="true"
                                                style="user-select: none;"><span
                                                    class="rc-select-item-option-state-icon"></span></span>
                                        </div>
                                        <div aria-selected="false" class="rc-select-item rc-select-item-option"
                                            title="500 Profiles / Page">
                                            <div class="rc-select-item-option-content">500 Profiles / Page</div><span
                                                class="rc-select-item-option-state" unselectable="on" aria-hidden="true"
                                                style="user-select: none;"><span
                                                    class="rc-select-item-option-state-icon"></span></span>
                                        </div>
                ...
</li>

Solution

First we need to close the pop-ups and then try to click on pagination options. And using both Implicit wait and Explicit wait is not Recommended.

Try the following solution:

driver.get("https://www.personality-database.com/profile?pid=1&sort=hot")
wait = WebDriverWait(driver,30)
try:
    # Close the footer add
    wait.until(EC.element_to_be_clickable((By.XPATH,"//span[@id='ezmob-wrapper']/div/center/span/div/div/span"))).click()
    # Scroll a distance so that the Cookie pop up appears and Close it
    driver.execute_script("window.scrollBy(0,50);")
    wait.until(EC.element_to_be_clickable((By.XPATH,"//button[@id='rcc-confirm-button']"))).click()
except:
    print("no adds")
 
# click on the drop down option   
pagination = wait.until(EC.element_to_be_clickable((By.XPATH,"//li[@class='rc-pagination-options']")))
pagination.click()

# Click on the 500 profiles
option = wait.until(EC.element_to_be_clickable((By.XPATH,"//div[@class='rc-virtual-list-holder-inner']//div[text()='500 Profiles / Page']")))
option.click()

Answered By - pmadhu