[FIXED] Heroku chromedriver and google-chrome build packs are crashing upon use

Issue

Using scrapy and heroku for some spiders I have setup. It's worked fine for months, didn't change anything that I know would have any affect, and yet when I try to run a spider, I get this error.

2021-03-03 04:13:33 [selenium.webdriver.remote.remote_connection] DEBUG: Finished Request
2021-03-03 04:13:33 [twisted] CRITICAL: Unhandled error in Deferred:
2021-03-03 04:13:33 [twisted] CRITICAL: 
Traceback (most recent call last):
  File "/app/.heroku/python/lib/python3.7/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
    result = g.send(result)
  File "/app/.heroku/python/lib/python3.7/site-packages/scrapy/crawler.py", line 87, in crawl
    self.engine = self._create_engine()
  File "/app/.heroku/python/lib/python3.7/site-packages/scrapy/crawler.py", line 101, in _create_engine
    return ExecutionEngine(self, lambda _: self.stop())
  File "/app/.heroku/python/lib/python3.7/site-packages/scrapy/core/engine.py", line 69, in __init__
    self.downloader = downloader_cls(crawler)
  File "/app/.heroku/python/lib/python3.7/site-packages/scrapy/core/downloader/__init__.py", line 83, in __init__
    self.middleware = DownloaderMiddlewareManager.from_crawler(crawler)
  File "/app/.heroku/python/lib/python3.7/site-packages/scrapy/middleware.py", line 53, in from_crawler
    return cls.from_settings(crawler.settings, crawler)
  File "/app/.heroku/python/lib/python3.7/site-packages/scrapy/middleware.py", line 35, in from_settings
    mw = create_instance(mwcls, settings, crawler)
  File "/app/.heroku/python/lib/python3.7/site-packages/scrapy/utils/misc.py", line 150, in create_instance
    instance = objcls.from_crawler(crawler, *args, **kwargs)
  File "/app/.heroku/python/lib/python3.7/site-packages/scrapy_selenium/middlewares.py", line 71, in from_crawler
    browser_executable_path=browser_executable_path
  File "/app/.heroku/python/lib/python3.7/site-packages/scrapy_selenium/middlewares.py", line 51, in __init__
    self.driver = driver_klass(**driver_kwargs)
  File "/app/.heroku/python/lib/python3.7/site-packages/selenium/webdriver/chrome/webdriver.py", line 81, in __init__
    desired_capabilities=desired_capabilities)
  File "/app/.heroku/python/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 157, in __init__
    self.start_session(capabilities, browser_profile)
  File "/app/.heroku/python/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/app/.heroku/python/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/app/.heroku/python/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: exited abnormally.
  (unknown error: DevToolsActivePort file doesn't exist)
  (The process started from chrome location /app/.apt/opt/google/chrome/chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)

Solution

It's happening the same to me. I looked the git repository and found an issue and a pull request for it, so I think we just gonna have to wait for it be accepted.

Edit: There is a fix. I tested and it worked!

1. Add buildpack: https://github.com/heroku/heroku-buildpack-apt
2. Create a file name Aptfile and paste libxshmfence-dev in the file.

Answered By - Ana Paula Lima