White Papper: Supercharging Django's Development Workflow with Docker Integration

django-white-papper.md

Supercharging Django's Development Workflow with Docker Integration

A White Paper by Laurent Declercq, CTO @ Agon Partners Innovation

---

Overview

Modern Django applications often depend on external services like PostgreSQL, Redis, and Maildev. While Django's runserver is excellent for rapid development, it lacks support for starting and managing these services.

This white paper introduces a pragmatic extension to runserver that integrates Docker service orchestration directly into the Django development cycle.

---

Problem Statement

Developers frequently encounter friction when working in containerized environments:

• Docker services must be started manually before running Django.
• Forgetting to start services leads to confusing runtime errors.
• Database initialization and migration are manual steps.
• When the dev server exits, dependent containers are often left running.

---

Solution

This approach redefines the developer workflow with a single command:

python manage.py runserver

This enhanced command:

• Automatically starts Docker services (PostgreSQL, Redis, Maildev).
• Waits for containers to become healthy before launching Django.
• Optionally restores the database from the latest dump.
• Detects and applies pending Django migrations.
• Gracefully shuts down services when development ends.

---

Architecture & Implementation

The solution extends Django’s runserver management command. Key features include:

• --no-pull to avoid pulling images during development.
• --no-pg-restore and --no-migrate for faster iteration.
• --only-database to start only the PostgreSQL service.
• Integration with Docker Compose using subprocess.run(...).
• Health check loop with timeout logic.
• Safe shutdown using Python’s atexit.

CLI Interface

python manage.py runserver [addr:port] \
    --no-pull \
    --no-pg-restore \
    --no-migrate \
    --only-database

---

Developer Experience

This system eliminates repetitive tasks. Developers no longer need to:

• Remember the dependent services to start.
• Manually check container health.
• Rerun migrations.
• Clean up orphaned containers.

Example Output

PUBLICA NEGOTIA DEVELOPMENT SERVICES

 🚀 Launching the dependent services ...

 ⏳ Waiting for the dependent services to be healthy ...
 🎉 All dependent services are healthy!

 📥 Restoring the PostgreSQL database from the latest dump ...
 📦 PostgreSQL database is already initialized. Skipping ...

 🛠️ Applying database migrations ...
 📦 All migrations are already applied. Skipping ...

 🚀 Launching built-in Django development server ...

Full Code Example

# Description: Start the Django development server with Docker dependent services.
#
# This command overrides the Django runserver command to pre-start the Docker dependent services (PostgreSQL, Redis,
# Maildev) using Docker Compose.
#
# By default, the command also initializes the database by restoring the latest dump and running the migrations if
# needed. The database initialization can be skipped using the --no-pg-restore and --no-migrate options.
#
# The command is designed to be used in conjunction with the built-in Django development server so, the dependencies are
# started before the server is started, that is, when the 'RUN_MAIN' environment variable is not set yet:
#
#      ┌───────────────────────────────────────┐
#      │ Parent proc ('RUN_MAIN' not set)      │
#      │  > Starts Docker dependent services   │
#      │  > Initialize Database if needed      │
#      │  > Run Database migration if needed   │
#      │  > Spawns child with RUN_MAIN=true    │
#      └──────────────────┬────────────────────┘
#                         │
#                         ▼
#      ┌───────────────────────────────────────┐
#      │ Django Dev server (RUN_MAIN=true)     │
#      │  > This restarts on every file change │
#      └───────────────────────────────────────┘
#
# When the Django development server is stopped, the dependent services are stopped as well.
#
# Author: Laurent Declercq <[email protected]>
# Version: 20250528

"""Start the Django development server with Docker dependent services."""

import atexit
import shutil
import os
import subprocess
import sys
import time
from pathlib import Path
from contextlib import chdir
from typing import Any

from django.core.management import CommandError, CommandParser, call_command
from django.core.management.commands.runserver import Command as RunserverCommand
from django.db import ProgrammingError, connections
from django.db.migrations.executor import MigrationExecutor


# List of services to start (dependent services).
SERVICES = [
    "publica-negotia-cache",
    "publica-negotia-database",
    "publica-negotia-smtp",
]

def check_for_requirements(binaries: list[str | dict[str, str]]) -> None:
    """
    Check for requirements availability in the PATH.

    :param binaries: List of required binaries to check.
    :raise: CommandError if a required binary is not found in the PATH.
    """
    for item in binaries:
        binary, icon = next(iter(item.items())) if isinstance(item, dict) else (item, "❌")

        if shutil.which(binary) is None:
            raise CommandError(f" {icon} Required binary {binary!r} not found in PATH.")

def determine_project_compose_file_path() -> Path:
    """
    Helper to determine the path to the project's docker-compose file.

    :return: Path to the docker-compose file.
    :raise: FileNotFoundError if the file is not found.
    :raise: PermissionError if the file is not readable.
    """
    path = determine_project_root_dir().joinpath("docker-compose.yml")

    if not path.exists():
        raise FileNotFoundError(f"docker-compose file not found at: {path}")

    if not path.is_file() or not os.access(path, os.R_OK):
        raise PermissionError(f"docker-compose file is not readable at: {path}")

    return path


def determine_project_root_dir() -> Path:
    """
    Helper to determine the project root directory.

    :return: Project root directory.
    """
    return Path(__file__).resolve().parents[4]


class Command(RunserverCommand):
    """Start the Django development server with Docker dependent services."""

    help = "🧩 Manage the Publica Negotia developments services"

    def add_arguments(self, parser: CommandParser) -> None:
        """
        Add command arguments.

        :param parser: Command parser.
        :return: None.
        """
        super().add_arguments(parser)

        parser.add_argument("--no-pull", action="store_true", help="Don't pull Docker images")

        parser.add_argument(
            "--no-pg-restore",
            action="store_true",
            help="Do not restore the PostgreSQL database, even if schema is empty.",
        )

        parser.add_argument(
            "--no-migrate",
            action="store_true",
            help="Do not run migrations, even if there are pending ones.",
        )

        # Add an option allowing to start only the database service.
        parser.add_argument(
            "--only-database",
            action="store_true",
            help="Start only the PostgreSQL database service without the other dependent services.",
        )

    def handle(self, *args: Any, **options: Any) -> str | None:
        """
        Handle the command.

        :param args: Command arguments.
        :param options: Command options.
        :return: None
        :raise: CommandError if an error occurred.
        """
        # When the --only-database option is set, we only start the database service.
        service = SERVICES[1] if options.get("only_database", False) else None

        # This block runs **once** before Django starts autoreload subprocesses.
        if os.environ.get("RUN_MAIN") != "true":
            pg_restore = not options.get("no_pg_restore", False)
            migrate = not options.get("no_migrate", False)
            no_pull = options.get("no_pull", False)

            check_for_requirements([{"docker": "🐳"}])

            self.stdout.write("PUBLICA NEGOTIA DEVELOPMENT SERVICES", self.style.SUCCESS, "\n\n")

            with chdir(determine_project_root_dir()):
                if ret := self._start_services(no_pull=no_pull, service=service):
                    raise CommandError(ret)

                self._initialize_database(pg_restore, migrate)

                self.stdout.write("\n 🚀 Launching built-in Django development server ...", self.style.NOTICE, "\n\n")

        if service:
            # If only the database service is requested, we don't need to start the server.
            self.stdout.write(
                " 🛑 The Django development server is not started because only the database service was requested.",
                self.style.WARNING
            )
            return None

        # Call the parent class handle method to start the development server.
        return super().handle(*args, **options)

    def _start_services(self, no_pull: bool = False, service: str = None) -> str | None:
        """
        Start the dependent (PostgreSQL, Redis, Maildev).

        :param no_pull: Do not pull Docker images.
        :param service: Optional specific service to start (if provided, only this service will be started).
        :return: A string if an error occurred, None otherwise.
        """
        services = SERVICES if service is None else [service]

        if service and service not in SERVICES:
            return self.style.ERROR(f" ❌ Invalid service: {service!r}. Valid services are: {', '.join(SERVICES)}")

        self.stdout.write(" 🚀 Launching the dependent services ...", self.style.NOTICE, "\n\n")

        try:
            # Start the development services using Docker Compose.
            subprocess.run(
                [
                    "docker",
                    "compose",
                    "--file",
                    str(determine_project_compose_file_path()),
                    "up",
                    "--pull",
                    ("never" if no_pull else "missing"),
                    "--detach",
                    # "--force-recreate",
                    "--remove-orphans",
                    *([service] if service else [])  # If a specific service is provided, only start that service.
                ],
                check=True,
            )

            self.stdout.write("\n ⏳  Waiting for the dependent services to be healthy ...", self.style.NOTICE, "\n")

            timeout = 60  # Timeout in seconds to wait for the services to become healthy (60 seconds).
            start_time = time.time()  # Record the start time to calculate the elapsed time.

            # Wait until all dependent services are healthy, checking every 3 seconds, and aborting if the timeout is
            # reached.
            while True:
                if all(self.__is_service_healthy(service) for service in services):
                    self.stdout.write(" 🎉 All dependent services are healthy!", self.style.SUCCESS, "\n\n")
                    break

                if time.time() - start_time > timeout:
                    return self.style.ERROR(" ⏰  Timeout waiting for dependent services to become healthy!")

                # Wait for 3 seconds before checking again health status.
                time.sleep(3)

            # Register graceful cleanup hook for stopping the services when the server is stopped, unless only the
            # database service is requested.
            if not service:
                atexit.register(self.__stop_services)
        except subprocess.CalledProcessError as e:
            return self.style.ERROR(f" 💥 Failed to start the dependent services: {e}")
        except KeyboardInterrupt:
            sys.stdout.write("\r")
            return self.style.ERROR("\n 💀 Operation cancelled. Bye!")

        return None

    def __stop_services(self) -> None:
        """
        Stop the dependent services (PostgreSQL, Redis, Maildev) when the server is stopped.

        :return: None
        """
        sys.stdout.write("\r")
        self.stdout.write(" 🛑 The Django development server has been stopped!", self.style.SUCCESS, "\n\n")
        self.stdout.write(" 🧹 Stopping the dependent services ...", self.style.NOTICE, "\n\n")

        try:
            subprocess.run(
                ["docker", "compose", "--file", str(determine_project_compose_file_path()), "stop"],
                check=True,
            )

            self.stdout.write("\n 🛑 The dependent services were stopped!", self.style.SUCCESS, "\n")
        except subprocess.CalledProcessError as e:
            self.stdout.write(f" 💥 Failed to stop the dependent services: {e}", self.style.ERROR, "\n")

    def _initialize_database(self, pg_restore: bool, migrate: bool) -> None:
        """
        Initialize the database.

        Initialize the database by restoring the latest dump and running the migrations. If the database is already
        initialized, just run the migrations if there are any pending.

        :param pg_restore: Restore the database dump.
        :param migrate: Run the migrations.
        """
        if pg_restore:
            self.stdout.write(" 📥 Restoring the PostgreSQL database from the latest dump ...", self.style.NOTICE)

            if self.__is_db_schema_empty():
                call_command("pg_restore")
            else:
                self.stdout.write(" 📦 PostgreSQL database is already initialized. Skipping ...", self.style.WARNING)

        if migrate:
            self.stdout.write("\n 🛠️ Applying database migrations ...", self.style.NOTICE)

        if self.__needs_migrations():
            call_command("migrate")
        else:
            self.stdout.write(" 📦 All migrations are already applied. Skipping ...", self.style.WARNING)

    @staticmethod
    def __is_service_healthy(service: str) -> bool:
        """
        Helper method to check if a Docker service is healthy.

        :param service: Service name.
        :return: True if the service is healthy, False otherwise.
        """
        result = subprocess.run(
            ["docker", "inspect", "--format", "{{.State.Health.Status}}", service],
            capture_output=True,
            text=True,
            check=True,
        )

        return result.stdout.strip() == "healthy"

    @staticmethod
    def __is_db_schema_empty() -> bool:
        """
        Helper method to check if the database schema is empty.

        :return: True if the database is initialized, False otherwise.
        """
        with connections["default"].cursor() as cursor:
            cursor.execute("SELECT COUNT(*) FROM pg_catalog.pg_tables WHERE schemaname = 'public';")
            count = cursor.fetchone()[0]
            connections["default"].close()

            return count == 0

    @staticmethod
    def __needs_migrations() -> bool:
        """
        Helper method to check if there are pending migrations.

        :return: True if there are pending migrations, False otherwise.
        """
        try:
            executor = MigrationExecutor(connections["default"])
            targets = executor.loader.graph.leaf_nodes()
            plan = executor.migration_plan(targets)
            return bool(plan)
        except ProgrammingError:
            # Likely that the 'django_migrations' table doesn't exist yet.
            return True

---

Future Directions

• Add --down for service-only shutdown.
• Integrate Compose logs directly in the command output.
• Extend support for frontend services (React, Vite).
• .env integration for service-specific environments.

---

About the Author

Laurent Declercq is a seasoned developer and systems architect with over 20 years of experience. As CTO of Agon Partners Innovation, he specializes in full-stack development, infrastructure automation, and developer experience design.

Known for his precision, architectural thinking, and relentless pursuit of maintainability, Laurent has created tools and frameworks used across public affairs platforms, secure infrastructure deployments, and AI-driven legislative analysis systems.

He is also the original author of i-MSCP, a world-renowned open-source web hosting control panel, and has contributed to numerous other open-source projects. His work has been recognized for its clarity, efficiency, and user-centric design.

This white paper reflects his ongoing commitment to crafting pragmatic and elegant solutions for real-world development workflows.

---

💬 For questions or feedback: [email protected]