[FIXED] Django app not connecting to PostgreSQL in setup created via docker-compose

Issue

I am trying to run a Django app with a PostgreSQL database via docker compose. Here is my docker-compose.yml:

version: '3.7'

services:

  web:
    build: .

    command: sh -c "python manage.py runserver"

    volumes: 
      - .:/immoscreen

    env_file:
      - .env
    
    environment: 
      - APP_DB_ENGINE=${APP_DB_ENGINE}
      - APP_DB_HOST=${APP_DB_HOST}
      - APP_DB_PORT=${APP_DB_PORT}
      - APP_DB_NAME=${APP_DB_NAME}
      - APP_DB_USER=${APP_DB_USER}
      - APP_DB_PASSWORD=${APP_DB_PASSWORD}

    image: immoscreen_app

    ports:
      - ${DJANGO_APP_PORT}:${DJANGO_APP_PORT}

    expose: 
      - ${DJANGO_APP_PORT}

    restart: "on-failure"

    depends_on:
      db:
        condition: service_healthy
    

  db:
    image: postgres

    volumes:
      - ./db/data/:/var/lib/postgresql/data/
      - ./db/init.sql/:/docker-entrypoint-initdb.d/

    env_file:
      - .env

    environment:
      - POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
      - APP_DB_NAME=${APP_DB_NAME}
      - APP_DB_USER=${APP_DB_USER}
      - APP_DB_PASSWORD=${APP_DB_PASSWORD}

    ports:
      - 5432:5432
    
    healthcheck:
      test: ["CMD-SHELL", "pg_isready", "-d", "${APP_DB_NAME}", "-U", "${APP_DB_USER}"]
      interval: 10s
      timeout: 10s
      retries: 5
      start_period: 5s

Here are the relevant parts of the .env file referenced in docker-compose.yml:

# PORT to run Django app on
DJANGO_APP_PORT=8000

# POSTGRES DB configuration
APP_DB_ENGINE=django.db.backends.postgresql
APP_DB_HOST=db
APP_DB_PORT=5432
APP_DB_NAME=immohub
APP_DB_USER=immohub
APP_DB_PASSWORD=thepassword

# POSTGRES DB configuration
POSTGRES_PASSWORD=postgres

This configuration uses a health check on the db service and a service_readycondition for the web service to not make the Django app start before the DB is ready.

The database is being initialized as expected with the initialization script stored in ./db/init.sql/.

I can connect to the database with a pgAdmin client on the machine I am running this on with both of the following settings:

• postgres:postgres@localhost:5432/postgres
• immohub:thepassword@localhost:5432/immohub

However, the Django app in the web container fails to connect to the database, even though the db container reports the database as rady to accept connections just before. Here's what the log shows:

immohub-db-1   | 
immohub-db-1   | PostgreSQL Database directory appears to contain a database; Skipping initialization
immohub-db-1   | 
immohub-db-1   | 
immohub-db-1   | 2023-12-19 11:34:20.723 UTC [1] LOG:  starting PostgreSQL 16.1 (Debian 16.1-1.pgdg120+1) on aarch64-unknown-linux-gnu, compiled by gcc (Debian 12.2.0-14) 12.2.0, 64-bit
immohub-db-1   | 2023-12-19 11:34:20.723 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
immohub-db-1   | 2023-12-19 11:34:20.723 UTC [1] LOG:  listening on IPv6 address "::", port 5432
immohub-db-1   | 2023-12-19 11:34:20.724 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
immohub-db-1   | 2023-12-19 11:34:20.730 UTC [30] LOG:  database system was shut down at 2023-12-19 11:32:35 UTC
immohub-db-1   | 2023-12-19 11:34:20.736 UTC [1] LOG:  database system is ready to accept connections
immohub-db-1   | 2023-12-19 11:34:30.348 UTC [41] FATAL:  role "root" does not exist
immohub-web-1  | Watching for file changes with StatReloader
immohub-web-1  | Performing system checks...
immohub-web-1  | 
immohub-web-1  | 
immohub-web-1  | System check identified no issues (0 silenced).
immohub-web-1  | Exception in thread django-main-thread:
immohub-web-1  | Traceback (most recent call last):
immohub-web-1  |   File "/usr/local/lib/python3.10/site-packages/django/db/backends/base/base.py", line 289, in ensure_connection
immohub-web-1  |     self.connect()
immohub-web-1  |   File "/usr/local/lib/python3.10/site-packages/django/utils/asyncio.py", line 26, in inner
immohub-web-1  |     return func(*args, **kwargs)
immohub-web-1  |   File "/usr/local/lib/python3.10/site-packages/django/db/backends/base/base.py", line 270, in connect
immohub-web-1  |     self.connection = self.get_new_connection(conn_params)
immohub-web-1  |   File "/usr/local/lib/python3.10/site-packages/django/utils/asyncio.py", line 26, in inner
immohub-web-1  |     return func(*args, **kwargs)
immohub-web-1  |   File "/usr/local/lib/python3.10/site-packages/django/db/backends/postgresql/base.py", line 275, in get_new_connection
immohub-web-1  |     
immohub-web-1  | connection = self.Database.connect(**conn_params)
immohub-web-1  |   File "/usr/local/lib/python3.10/site-packages/psycopg/connection.py", line 738, in connect
immohub-web-1  |     
immohub-web-1  | raise ex.with_traceback(None)
immohub-web-1  | psycopg.
immohub-web-1  | OperationalError: connection failed: Connection refused
immohub-web-1  |        Is the server running on that host and accepting TCP/IP connections?
immohub-web-1  | 
immohub-web-1  | The above exception was the direct cause of the following exception:
immohub-web-1  | 
immohub-web-1  | Traceback (most recent call last):
immohub-web-1  |   File "/usr/local/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
immohub-web-1  | 
immohub-web-1  |     self.run()
immohub-web-1  | 
immohub-web-1  | 
immohub-web-1  |   File "/usr/local/lib/python3.10/threading.py", line 953, in run
immohub-web-1  |     self._target(*self._args, **self._kwargs)
immohub-web-1  | 
immohub-web-1  |   File "/usr/local/lib/python3.10/site-packages/django/utils/autoreload.py", line 64, in wrapper
immohub-web-1  | 
immohub-web-1  |     
immohub-web-1  | fn(*args, **kwargs)
immohub-web-1  |   File "/usr/local/lib/python3.10/site-packages/django/core/management/commands/runserver.py", line 136, in inner_run
immohub-web-1  |     self.check_migrations()
immohub-web-1  |   File "/usr/local/lib/python3.10/site-packages/django/core/management/base.py", line 574, in check_migrations
immohub-web-1  |     executor = MigrationExecutor(connections[DEFAULT_DB_ALIAS])
immohub-web-1  |   File "/usr/local/lib/python3.10/site-packages/django/db/migrations/executor.py", line 18, in __init__
immohub-web-1  |     self.loader = MigrationLoader(self.connection)
immohub-web-1  |   File "/usr/local/lib/python3.10/site-packages/django/db/migrations/loader.py", line 58, in __init__
immohub-web-1  |     self.build_graph()
immohub-web-1  |   File "/usr/local/lib/python3.10/site-packages/django/db/migrations/loader.py", line 235, in build_graph
immohub-web-1  |     
immohub-web-1  | self.applied_migrations = recorder.applied_migrations()
immohub-web-1  |   File "/usr/local/lib/python3.10/site-packages/django/db/migrations/recorder.py", line 81, in applied_migrations
immohub-web-1  |     if self.has_table():
immohub-web-1  |   File "/usr/local/lib/python3.10/site-packages/django/db/migrations/recorder.py", line 57, in has_table
immohub-web-1  |     with self.connection.cursor() as cursor:
immohub-web-1  |   File "/usr/local/lib/python3.10/site-packages/django/utils/asyncio.py", line 26, in inner
immohub-web-1  |     return func(*args, **kwargs)
immohub-web-1  |   File "/usr/local/lib/python3.10/site-packages/django/db/backends/base/base.py", line 330, in cursor
immohub-web-1  |     return self._cursor()
immohub-web-1  |   File "/usr/local/lib/python3.10/site-packages/django/db/backends/base/base.py", line 306, in _cursor
immohub-web-1  |     
immohub-web-1  | self.ensure_connection()
immohub-web-1  |   File "/usr/local/lib/python3.10/site-packages/django/utils/asyncio.py", line 26, in inner
immohub-web-1  |     
immohub-web-1  | return func(*args, **kwargs)
immohub-web-1  |   File "/usr/local/lib/python3.10/site-packages/django/db/backends/base/base.py", line 288, in ensure_connection
immohub-web-1  |     
immohub-web-1  | with self.wrap_database_errors:
immohub-web-1  |   File "/usr/local/lib/python3.10/site-packages/django/db/utils.py", line 91, in __exit__
immohub-web-1  |     raise dj_exc_value.with_traceback(traceback) from exc_value
immohub-web-1  |   File "/usr/local/lib/python3.10/site-packages/django/db/backends/base/base.py", line 289, in ensure_connection
immohub-web-1  |     self.connect()
immohub-web-1  |   File "/usr/local/lib/python3.10/site-packages/django/utils/asyncio.py", line 26, in inner
immohub-web-1  |     
immohub-web-1  | return func(*args, **kwargs)
immohub-web-1  |   File "/usr/local/lib/python3.10/site-packages/django/db/backends/base/base.py", line 270, in connect
immohub-web-1  |     self.connection = self.get_new_connection(conn_params)
immohub-web-1  |   File "/usr/local/lib/python3.10/site-packages/django/utils/asyncio.py", line 26, in inner
immohub-web-1  |     return func(*args, **kwargs)
immohub-web-1  |   File "/usr/local/lib/python3.10/site-packages/django/db/backends/postgresql/base.py", line 275, in get_new_connection
immohub-web-1  |     connection = self.Database.connect(**conn_params)
immohub-web-1  |   File "/usr/local/lib/python3.10/site-packages/psycopg/connection.py", line 738, in connect
immohub-web-1  |     raise ex.with_traceback(None)
immohub-web-1  | django.db.utils.OperationalError: connection failed: Connection refused
immohub-web-1  |        Is the server running on that host and accepting TCP/IP connections?

I also tried with localhost as the DB host for the Django app, which fails the same way.

When I run ping db in a terminal of the web container, I get:

# ping db
PING db (172.22.0.2) 56(84) bytes of data.
64 bytes from immohub-db-1.immohub_default (172.22.0.2): icmp_seq=1 ttl=64 time=0.263 ms

Can anybody see/explain what am I missing here?

Solution

After some troubleshooting, this appears to be related to how docker-compose builds the network configuration. There is no entry for the dbservice in the web service's hosts file. After adding one manually, the web service can ping the db host. I have created a new post on the issue tracker for docker-compose on Github: https://github.com/docker/compose/issues/11293

Answered By - matt_jay