[FIXED] Selenium leaves behind running processes?

Issue

When my selenium program crashes due to some error, it seems to leave behind running processes.

For example, here is my process list:

carol    30186  0.0  0.0 103576  7196 pts/11   Sl   00:45   0:00 /home/carol/test/chromedriver --port=51789
carol    30322  0.0  0.0 102552  7160 pts/11   Sl   00:45   0:00 /home/carol/test/chromedriver --port=33409
carol    30543  0.0  0.0 102552  7104 pts/11   Sl   00:48   0:00 /home/carol/test/chromedriver --port=42567
carol    30698  0.0  0.0 102552  7236 pts/11   Sl   00:50   0:00 /home/carol/test/chromedriver --port=46590
carol    30938  0.0  0.0 102552  7496 pts/11   Sl   00:55   0:00 /home/carol/test/chromedriver --port=51930
carol    31546  0.0  0.0 102552  7376 pts/11   Sl   01:16   0:00 /home/carol/test/chromedriver --port=53077
carol    31549  0.5  0.0      0     0 pts/11   Z    01:16   0:03 [chrome] <defunct>
carol    31738  0.0  0.0 102552  7388 pts/11   Sl   01:17   0:00 /home/carol/test/chromedriver --port=55414
carol    31741  0.3  0.0      0     0 pts/11   Z    01:17   0:02 [chrome] <defunct>
carol    31903  0.0  0.0 102552  7368 pts/11   Sl   01:19   0:00 /home/carol/test/chromedriver --port=54205
carol    31906  0.6  0.0      0     0 pts/11   Z    01:19   0:03 [chrome] <defunct>
carol    32083  0.0  0.0 102552  7292 pts/11   Sl   01:20   0:00 /home/carol/test/chromedriver --port=39083
carol    32440  0.0  0.0 102552  7412 pts/11   Sl   01:24   0:00 /home/carol/test/chromedriver --port=34326
carol    32443  1.7  0.0      0     0 pts/11   Z    01:24   0:03 [chrome] <defunct>
carol    32691  0.1  0.0 102552  7360 pts/11   Sl   01:26   0:00 /home/carol/test/chromedriver --port=36369
carol    32695  2.8  0.0      0     0 pts/11   Z    01:26   0:02 [chrome] <defunct>

Here is my code:

from selenium import webdriver

browser = webdriver.Chrome("path/to/chromedriver")
browser.get("http://stackoverflow.com")
browser.find_element_by_id('...').click()

browser.close()

Sometimes, the browser doesn't load the webpage elements quickly enough so Selenium crashes when it tries to click on something it didn't find. Other times it works fine.

This is a simple example for simplicity sake, but with a more complex selenium program, what is a guaranteed clean way of exiting and not leave behind running processes? It should cleanly exit on an unexpected crash and on a successful run.

Solution

Whats happening is that your code is throwing an exception, halting the python process from continuing on. As such, the close/quit methods never get called on the browser object, so the chromedrivers just hang out indefinitely.

You need to use a try/except block to ensure the close method is called every time, even when an exception is thrown. A very simplistic example is:

from selenium import webdriver

browser = webdriver.Chrome("path/to/chromedriver")
try:
    browser.get("http://stackoverflow.com")
    browser.find_element_by_id('...').click()

except:
    browser.close()
    browser.quit()  # I exclusively use quit

There are a number of much more sophisticated approaches you can take here, such as creating a context manager to use with the with statement, but its difficult to recommend one without having a better understanding of your codebase.

Answered By - Levi Noecker