[FIXED] Selenium only working in non-headless mode?

Issue

i want to use Selenium on this page: https://www.avis.com/en/home

Without the headless-mode everything works fine with that code:

import requests
from bs4 import BeautifulSoup
import os, sys, time
import xlwings as xw
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.action_chains import ActionChains
from random import choice
from selenium import webdriver     
from sys import platform 

WAIT = 1
link = f"https://www.avis.com/en/home"
cd = '/chromedriver.exe'
options = Options()
# options.add_argument('--headless')
options.add_experimental_option ('excludeSwitches', ['enable-logging'])            
options.add_argument("user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.106 Safari/537.36")    
driver = webdriver.Chrome (path + cd, options=options)
driver.get (link)
print(f"Read location VIE...")
driver.find_element_by_id("PicLoc_value").send_keys("VIE") 
driver.find_element_by_id("PicLoc_value").click()
time.sleep(WAIT)
print(f"Read from date 2021-07-06...")
driver.find_element_by_id("from").clear()
driver.find_element_by_id("from").click()
driver.find_element_by_id("from").send_keys("07/06/2021") 
time.sleep(WAIT)
print(f"Read to date 2021-07-21...")
driver.find_element_by_id("to").clear()
driver.find_element_by_id("to").click()
driver.find_element_by_id("to").send_keys("07/21/2021") 
time.sleep(WAIT)
driver.find_element_by_id("res-home-select-car").click()

But when i uncomment the line options.add_argument('--headless') it is not working anymore

I allways get this error message:

Read location VIE...
Read from date 2021-07-06...
Read to date 2021-07-21...
Traceback (most recent call last):
  File "C:\Users\Polzi\Documents\DEV\Upwork\JeffG\scrapeAvis.py", line 59, in <module>
    driver.find_element_by_id("res-home-select-car").click()
  File "C:\Users\Polzi\AppData\Local\Programs\Python\Python39\lib\site-packages\selenium\webdriver\remote\webelement.py", line 80, in click
    self._execute(Command.CLICK_ELEMENT)
  File "C:\Users\Polzi\AppData\Local\Programs\Python\Python39\lib\site-packages\selenium\webdriver\remote\webelement.py", line 633, in _execute
    return self._parent.execute(command, params)
  File "C:\Users\Polzi\AppData\Local\Programs\Python\Python39\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "C:\Users\Polzi\AppData\Local\Programs\Python\Python39\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.ElementClickInterceptedException: Message: element click intercepted: Element <button name="button" id="res-home-select-car" ng-class="{'margin-top-90':false &amp;&amp; vm.expandState &amp;&amp; isAuthenticated}" class="btn btn-red selectMyCar hideMeFix btn-primary-avis" ng-if="!vm.isOneClick" ng-mousedown="vm.selectCarClicked = true" ng-click="vm.getVehicles.submit(resForm)">...</button> is not clickable at point (391, 417). Other element would receive the click: <div class="flyoutWrapper background_img suppress-lazyloading" imgsrc="/content/dam/avis/na/us/common/offers/avis-redbackground-600x250.jpg/jcr:content/renditions/cq5dam.web.768.504.webp" request="background" style="background-image: url(&quot;/content/dam/avis/na/us/common/offers/avis-redbackground-600x250.jpg/jcr:content/renditions/cq5dam.web.768.504.webp&quot;); opacity: 1;">...</div>
  (Session info: headless chrome=91.0.4472.124)

Why is here selenium working only wihtout the headless mode?

Solution

When using headless mode you should set the screen size since default screen size in headless mode is 800x600.
That's why in headless mode you have the ElementClickInterceptedException error for the code that is running fine in the regular mode.
So, try adding

options.add_argument("--window-size=1920x1080")

Also, we usually add this:

options.add_argument('--no-sandbox')
options.add_argument('--disable-gpu')

Answered By - Prophet