[FIXED] Closing an ad button in a website using selenium

Issue

I was working on a web scraping project with Selenium and was trying to scrape news from the site https://www.businesstimes.com.sg/government-economy. But whenever I open the site with the selenium automated chrome window, one ad comes up in a popup which I want to close.

import selenium
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as LM
from selenium.common.exceptions import TimeoutException
from selenium.webdriver.common.by import By
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
from selenium.webdriver.support.ui import Select
import pandas as pd
import time

options = webdriver.ChromeOptions()

"""options.add_argument("enable-automation")
options.add_argument("--headless")"""
lists = ['disable-popup-blocking']

caps = DesiredCapabilities().CHROME
caps["pageLoadStrategy"] = "normal"

options.add_argument("--window-size=1920,1080")
options.add_argument("--disable-extensions")
options.add_argument("--disable-notifications")
options.add_argument("--disable-Advertisement")
options.add_argument("--disable-popup-blocking")


driver = webdriver.Chrome(executable_path= r"E:\chromedriver\chromedriver.exe", options=options, desired_capabilities=caps) #paste your own choromedriver path
driver.get('https://www.businesstimes.com.sg/government-economy')

I tried two methods, one was by the xpath method and one by the CSS selector method, but both failed.

#1st method
driver.find_element(By.CSS_SELECTOR, 'button[data-id="pclose-btn"]').click()

#2nd method
driver.find_element_by_xpath("//div[@class='bz-el bz-pclose-btn knd-BUTTON']").click()

Please help me with this. Thank you!

Solution

Element you trying to access is inside nested iframe.
This should work:

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
options = Options()
options.add_argument("start-maximized")

webdriver_service = Service('C:\webdrivers\chromedriver.exe')
driver = webdriver.Chrome(service=webdriver_service, options=options)
url = 'https://www.businesstimes.com.sg/government-economy'
driver.get(url)
wait = WebDriverWait(driver, 20)

wait.until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR, "iframe[id*='prestitial']")))
wait.until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR, "iframe")))

wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button[data-id='pclose-btn']"))).click()

Answered By - Prophet