[FIXED] Selenium: "ElementClickInterceptedException", Element is not clickable because another element obscures it

Issue

I am trying to click on the "CSV" button at this website https://covid19-vaccine-report.ecdc.europa.eu/#6_Reported_data using selenium.

However, with the below code I am receiving the error selenium.common.exceptions.ElementClickInterceptedException: Message: Element <button class="dt-button buttons-csv buttons-html5" type="button"> is not clickable at point (464,600) because another element <div class="cckBannerInner"> obscures it.

The code:

import os
import glob
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium import webdriver
from webdriver_manager.firefox import GeckoDriverManager
from selenium.webdriver.firefox.options import Options

# get directory of the current script
CURR_DIR = os.path.dirname(os.path.abspath(__file__))
URL = 'https://covid19-vaccine-report.ecdc.europa.eu/#6_Reported_data'
ACCEPT_COOKIES_ID = 'cckAcceptCookies'
VACCINES_XPATH = '//*[@id="DataTables_Table_0_wrapper"]/div[5]/button[2]'

# before downloading new files, remove old ones
for f in glob.glob(os.path.join(CURR_DIR, "dataset_2021-W*")):
    os.remove(f)

# this will prevent opening browser window
# (https://stackoverflow.com/a/60627463/1979665)
options = Options()
#options.add_argument("--headless")

# (https://stackoverflow.com/a/18440478/1979665)
# To prevent download dialog
profile = webdriver.FirefoxProfile()
profile.set_preference('browser.download.folderList', 2)  # custom location
profile.set_preference('browser.download.manager.showWhenStarting', False)
profile.set_preference('browser.download.dir', CURR_DIR)
profile.set_preference('browser.helperApps.neverAsk.saveToDisk', 'text/csv')
# (https://stackoverflow.com/a/32416545/1979665)
profile.set_preference("network.cookie.cookieBehavior", 2)

# (https://stackoverflow.com/a/58727916/1979665)
browser = webdriver.Firefox(profile,
                            executable_path=GeckoDriverManager().install(),
                            options=options)
browser.get(URL)
wait = WebDriverWait(browser, 2000000)

# accept cookies
accept_cookies = browser.find_element_by_id(ACCEPT_COOKIES_ID)
browser.execute_script("arguments[0].click();", accept_cookies)

# download vaccines as CSV
wait.until(EC.element_to_be_clickable((By.XPATH, VACCINES_XPATH))).click()
browser.quit()

The XPATH of the button was taken with the copy XPath feature of the Firefox web developer console.

I have read many other suggested answers and tried their solutions but none worked.

I use JavaScript to accept the cookies (browser.execute_script("arguments[0].click();", accept_cookies)), in order for the banner to go away, because when I tried with other methods I received the error could not be scrolled into view.

Then I am using WebDriverWait to wait for the desired button to be clickable (with element_to_be_clickable).

When the code runs, I see what's happening in the browser window, and I do see that the cookie are accepted and the banner goes away. Then the page is scrolled down to the CSV button, but I receive the aforementioned error, like if the button is still hidden by the cookie banner.

The exact error (I repeat it down here), indeed refers to the cookie banner this way:

selenium.common.exceptions.ElementClickInterceptedException: Message: Element <button class="dt-button buttons-csv buttons-html5" type="button"> is not clickable at point (464,600) because another element <p class="cckInform"> obscures it

I also tried increasing the wait time up to 5000000 but it did not make any difference.

WORKING CODE

Thanks to @Prophet I was able to make it work. The trick was to scroll down the page until the button becomes visible and only then click on it.

I also removed the "accept coockie" part this way, which is great given that now I have to deal with only one element this way.

The updated working code:

import os
import glob
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium import webdriver
from webdriver_manager.firefox import GeckoDriverManager
from selenium.webdriver.firefox.options import Options

# get directory of the current script
CURR_DIR = os.path.dirname(os.path.abspath(__file__))
URL = 'https://covid19-vaccine-report.ecdc.europa.eu/#6_Reported_data'
VACCINES_XPATH = '//*[@id="DataTables_Table_0_wrapper"]/div[5]/button[2]'

# before downloading new files, remove old ones
for f in glob.glob(os.path.join(CURR_DIR, "dataset_2021-W*")):
    os.remove(f)

# this will prevent opening browser window
# (https://stackoverflow.com/a/60627463/1979665)
options = Options()
#options.add_argument("--headless")

# (https://stackoverflow.com/a/18440478/1979665)
# To prevent download dialog
profile = webdriver.FirefoxProfile()
profile.set_preference('browser.download.folderList', 2)  # custom location
profile.set_preference('browser.download.manager.showWhenStarting', False)
profile.set_preference('browser.download.dir', CURR_DIR)
profile.set_preference('browser.helperApps.neverAsk.saveToDisk', 'text/csv')
# (https://stackoverflow.com/a/32416545/1979665)
profile.set_preference("network.cookie.cookieBehavior", 2)

# (https://stackoverflow.com/a/58727916/1979665)
driver = webdriver.Firefox(profile,
                            executable_path=GeckoDriverManager().install(),
                            options=options)
driver.get(URL)

wait = WebDriverWait(driver, 2000000)
vaccines = wait.until(EC.element_to_be_clickable((By.XPATH, VACCINES_XPATH)))
driver.execute_script("arguments[0].scrollIntoView();", vaccines)
vaccines.click()
driver.quit()

Solution

Try the following: scroll into view some other element that is below the desired button so that the desired button will not be covered by the cookies banner and then try clicking the button.

national_references = WebDriverWait(browser, 20).until(EC.presence_of_element_located((By.CSS_SELECTOR, "#national-references")))
actions = ActionChains(driver)
actions.move_to_element(national_references).perform()
wait.until(EC.element_to_be_clickable((By.XPATH, VACCINES_XPATH))).click()

UPD: Since scrolling to the national_references was not possible since it is out of bounds of viewport width (1366) and height (630) according to the exception thrown when trying so scroll it into the view with actions.move_to_element(national_references).perform() we can scroll page down until national_references becomes visible.
Now we can successfully click the vaccines CSV button.

Answered By - Prophet