[FIXED] Selenium can't open a second page

Issue

I am using Selenium to open different pages of a site. Have tried multiple times but the browser does not open a second webpage after the initial GET call. Have tried on both Chrome and Safari. Here is my code:

driver = webdriver.Chrome()
driver.get("https://www.justdial.com/Chennai/Hr-Consultancy-Services/nct-10258625/page-2")
driver.set_page_load_timeout(30)
driver.get("https://www.justdial.com/Chennai/Hr-Consultancy-Services/nct-10258625/page-3")

Here is the error I get for the second call:

The info from Network logs is Error 504, but I have verified that it works perfectly when done on another window of the browser, without automation

Solution

A bit of more information about your usecase would have helped to construct a more canonical answer. However I was able to access the Page 2 of justdial.com/Chennai/Hr-Consultancy-Services with a minimized code block as follows:

• Code Block:

  from selenium import webdriver
  
  options = webdriver.ChromeOptions() 
  options.add_argument("start-maximized")
  driver = webdriver.Chrome(options=options, executable_path=r'C:\WebDrivers\chromedriver.exe')
  driver.get("https://www.justdial.com/Chennai/Hr-Consultancy-Services/nct-10258625/page-2")
  

• Browser Snapshot:

But while sending multiple get() one after another:

driver.get("https://www.justdial.com/Chennai/Hr-Consultancy-Services/nct-10258625/page-2")
driver.get("https://www.justdial.com/Chennai/Hr-Consultancy-Services/nct-10258625/page-3")

It seems ChromeDriver initiated Chrome Browser gets detected and the following error is shown:

An error occurred while processing your request.
Reference #97.e5732c31.1612205693.6fd2708

---

Solution

To avoid the detection you can add the following option:

--disable-blink-features=AutomationControlled

---

Example

from selenium import webdriver

options = webdriver.ChromeOptions() 
options.add_argument("start-maximized")
options.add_argument('--disable-blink-features=AutomationControlled')
driver = webdriver.Chrome(options=options, executable_path=r'C:\WebDrivers\chromedriver.exe')
driver.get("https://www.justdial.com/Chennai/Hr-Consultancy-Services/nct-10258625/page-2")
driver.get("https://www.justdial.com/Chennai/Hr-Consultancy-Services/nct-10258625/page-3")

Answered By - undetected Selenium