Dockerfile + docker-compose.yml for development

README.md

my_awesome_app

NDA...

Installation

Local

1. Create env file:

cp envs/.env.example envs/.env

2. If you use pyenv, create and activate environment. You can read this article to get familiar with pyenv. Or you can just omit this step, and poetry will install venv for you.

pyenv install 3.11
pyenv virtualenv 3.11 my_awesome_app
pyenv local my_awesome_app

3. If you don't have Poetry installed run:

make poetry-download

4. Initialize poetry and install pre-commit hooks:

make install
make pre-commit-install

5. Run formatters, and linters. Make sure there is no errors.

make format lint

6. Run supporting services

docker compose --profile infra up -d

7. Run the API

make api

8. Run the task worker

make worker

You can open:

• API Documentation at http://localhost:8000/docs
• Minio at http://localhost:9001 (user, password)
• RabbitMQ at http://localhost:15672/ (guest, guest)

To stop all services run:

docker compose --profile dev --profile test down

Docker

1. Create env file:

cp envs/.env.docker.example envs/.env.docker

2. Build an image:

docker compose --profile dev build

3. Run the app:

docker compose --profile dev up -d

4. Run formatters and linters. Make sure there is no errors.

docker compose exec web make format lint

You can open:

• API Documentation at http://localhost:8000/docs
• Minio at http://localhost:9001 (user, password)
• RabbitMQ at http://localhost:15672/ (guest, guest)

To stop all services run:

docker compose --profile dev --profile test down

Tests

Local

Run supporting services for testing:

docker compose --profile test-infra up -d

Run tests:

make test

Stop all services:

docker compose --profile test-infra down

Docker

docker compose --profile test build

docker compose run test-runner

As simple as that

Migrations

Create migration:

alembic revision --autogenerate -m "Message"

Apply migrations:

alembic upgrade head

Revert last migration:

alembic downgrade -1

Makefile usage

Makefile contains a lot of functions for faster development.

1. Download and remove Poetry

To download and install Poetry run:

make poetry-download

To uninstall

make poetry-remove

2. Install all dependencies and pre-commit hooks

Install requirements:

make install

Pre-commit hooks could be installed after git init via

make pre-commit-install

3. Codestyle

Automatic formatting uses ruff.

make codestyle

# or use synonym
make format

Codestyle checks only, without rewriting files:

make check-codestyle

Update all dev libraries to the latest version using one comand

make update-dev-deps

4. Code security

make check-security

This command identifies security issues with Safety and Bandit.

make check-security

To validate pyproject.toml use

make check-poetry

5. Linting and type checks

Run static linting with pylint and mypy:

make static-lint

6. Tests with coverage

Run pytest

make test

7. All linters

Of course there is a command to rule run all linters in one:

make lint

the same as:

make test && make check-codestyle && make static-lint && make check-safety

8. Docker

make docker-build

which is equivalent to:

make docker-build VERSION=latest

Remove docker image with

make docker-remove

More information about docker.

9. Cleanup

Delete pycache files

make pycache-remove

Remove package build

make build-remove

Delete .DS_STORE files

make dsstore-remove

Remove .mypycache

make mypycache-remove

Or to remove all above run:

make cleanup

Credits

This project was generated with python-package-template

docker-compose.yml

version: "3.9"

services:

  web:
    build:
      context: .
      dockerfile: Dockerfile
      target: development
    command: bash -c "alembic upgrade head && uvicorn my_awesome_app.app:app --host 0.0.0.0 --port 8000 --reload"
    env_file:
      - ./envs/.env.docker
    ports:
      - "8000:8000"
    volumes:
      - .:/app
    depends_on:
      postgres:
        condition: service_healthy
    profiles:
      - dev

  worker:
    build:
      context: .
      dockerfile: Dockerfile
      target: development
    command: taskiq worker my_awesome_app.broker:broker my_awesome_app.tasks --reload
    env_file:
      - ./envs/.env.docker
    volumes:
      - .:/app
    depends_on:
      postgres:
        condition: service_healthy
    profiles:
      - dev

  postgres:
    image: postgres:16-alpine
    environment:
      POSTGRES_USER: user
      POSTGRES_PASSWORD: password
      POSTGRES_DB: my_awesome_app
      PGDATA: /data/postgres
    ports:
      - "5432:5432"
    volumes:
      - postgres-data:/data/postgres
    restart: unless-stopped
    healthcheck:
      test: [ "CMD-SHELL", "pg_isready -U user -d my_awesome_app" ]
      interval: 5s
      timeout: 5s
      retries: 5
    profiles:
      - dev
      - infra

  minio:
    image: minio/minio
    ports:
      - "9000:9000"
      - "9001:9001"
    volumes:
      - minio-data:/data
    environment:
      MINIO_ROOT_USER: user
      MINIO_ROOT_PASSWORD: password
    command: server --console-address ":9001" /data
    profiles:
      - dev
      - infra

  redis:
    image: redis:7-alpine
    ports:
      - "6379:6379"
    command: redis-server --save 60 1 --loglevel warning
    volumes:
      - redis-data:/data
    restart: unless-stopped
    profiles:
      - dev
      - infra

  rabbitmq:
    image: rabbitmq:3-management-alpine
    ports:
      - "5672:5672"
      - "15672:15672"
    restart: unless-stopped
    profiles:
      - dev
      - infra

  createbuckets:
    image: minio/mc
    depends_on:
      - minio
    entrypoint: >
      /bin/sh -c "
      /usr/bin/mc alias set myminio http://minio:9000 user password;
      /usr/bin/mc mb myminio/my_awesome_app;
      /usr/bin/mc anonymous set public myminio/my_awesome_app;
      exit 0;
      "
    profiles:
      - dev
      - infra

  test-postgres:
    image: postgres:16-alpine
    environment:
      POSTGRES_USER: user
      POSTGRES_PASSWORD: password
      POSTGRES_DB: postgres
    ports:
      - "5433:5432"
    restart: unless-stopped
    healthcheck:
      test: [ "CMD-SHELL", "pg_isready -U user -d test" ]
      interval: 5s
      timeout: 5s
      retries: 5
    profiles:
      - test
      - test-infra

  test-minio:
    image: minio/minio
    ports:
      - "9002:9000"
      - "9003:9003"
    environment:
      MINIO_ROOT_USER: user
      MINIO_ROOT_PASSWORD: password
    command: server --console-address ":9003" /data
    profiles:
      - test
      - test-infra

  test-runner:
    build:
      context: .
      dockerfile: Dockerfile
      target: development
    command: bash -c "alembic upgrade head && make test"
    env_file:
      - ./envs/.env.docker.test
    volumes:
      - .:/app
    depends_on:
      test-postgres:
        condition: service_healthy
      test-minio:
        condition: service_started
    profiles:
      - test


volumes:
  postgres-data:
  minio-data:
  redis-data:

Dockerfile

# Source: https://gist.github.com/usr-ein/c42d98abca3cb4632ab0c2c6aff8c88a

################################
# PYTHON-BASE
# Sets up all our shared environment variables
################################
FROM python:3.11-slim as python-base

    # python
ENV PYTHONUNBUFFERED=1 \
    # prevents python creating .pyc files
    PYTHONDONTWRITEBYTECODE=1 \
    \
    # pip
    PIP_DISABLE_PIP_VERSION_CHECK=on \
    PIP_DEFAULT_TIMEOUT=100 \
    \
    # poetry
    # https://python-poetry.org/docs/configuration/#using-environment-variables
    POETRY_VERSION=1.6.1 \
    # make poetry install to this location
    POETRY_HOME="/opt/poetry" \
    # make poetry create the virtual environment in the project's root
    # it gets named `.venv`
    POETRY_VIRTUALENVS_IN_PROJECT=true \
    # do not ask any interactive question
    POETRY_NO_INTERACTION=1 \
    \
    # paths
    # this is where our requirements + virtual environment will live
    PYSETUP_PATH="/opt/pysetup" \
    VENV_PATH="/opt/pysetup/.venv"

RUN apt-get update \
    && apt-get install --no-install-recommends -y \
        # deps for installing poetry
        curl \
        # deps for building python deps
        build-essential

# prepend poetry and venv to path
ENV PATH="$POETRY_HOME/bin:$VENV_PATH/bin:$PATH"


################################
# BUILDER-BASE
# Used to build deps + create our virtual environment
################################
FROM python-base as builder-base

# install poetry - respects $POETRY_VERSION & $POETRY_HOME
# The --mount will mount the buildx cache directory to where
# Poetry and Pip store their cache so that they can re-use it
RUN --mount=type=cache,target=/root/.cache \
    curl -sSL https://install.python-poetry.org | python3 -

# copy project requirement files here to ensure they will be cached.
WORKDIR $PYSETUP_PATH
COPY poetry.lock pyproject.toml ./

# install runtime deps - uses $POETRY_VIRTUALENVS_IN_PROJECT internally
RUN --mount=type=cache,target=/root/.cache \
    poetry install --without=dev


################################
# DEVELOPMENT
# Image used during development / testing
################################
FROM python-base as development
WORKDIR $PYSETUP_PATH

# copy in our built poetry + venv
COPY --from=builder-base $POETRY_HOME $POETRY_HOME
COPY --from=builder-base $PYSETUP_PATH $PYSETUP_PATH

# quicker install as runtime deps are already installed
RUN --mount=type=cache,target=/root/.cache \
    poetry install --with=dev

# will become mountpoint of our code
WORKDIR /app

CMD ["uvicorn", "my_awesome_app.app:app", "--host", "0.0.0.0", "--port", "8000", "--reload"]


################################
# PRODUCTION
# Final image used for runtime
################################
FROM python-base as production
COPY --from=builder-base $PYSETUP_PATH $PYSETUP_PATH

COPY ./my_awesome_app /app/my_awesome_app
COPY ./migrations /app/migrations
COPY ./alembic.ini /app/alembic.ini

WORKDIR /app

CMD ["uvicorn", "my_awesome_app.app:app", "--host", "0.0.0.0", "--port", "8000"]

Makefile

#* Variables
SHELL := /usr/bin/env bash
PYTHON := python3

#* Docker variables
IMAGE := my_awesome_app
VERSION := latest

#* Directories with source code
CODE = my_awesome_app tests
CODE_FORMAT = my_awesome_app tests migrations
TESTS = tests

#* Application
api:
	uvicorn my_awesome_app.app:app --reload

worker:
	taskiq worker my_awesome_app.broker:broker my_awesome_app.tasks --reload

#* Poetry
.PHONY: poetry-download
poetry-download:
	curl -sSL https://install.python-poetry.org | $(PYTHON) -

.PHONY: poetry-remove
poetry-remove:
	curl -sSL https://install.python-poetry.org | $(PYTHON) - --uninstall

#* Installation
.PHONY: install
install:
	poetry lock --no-interaction --no-update
	poetry install --no-interaction

.PHONY: pre-commit-install
pre-commit-install:
	poetry run pre-commit install

#* Formatters
.PHONY: codestyle
codestyle:
	poetry run ruff format $(CODE_FORMAT)
	poetry run ruff check $(CODE) --fix-only

.PHONY: format
format: codestyle

#* Test
.PHONY: test
test:
	poetry run pytest -n auto --cov $(TEST_FILTER)
	poetry run coverage xml

# Validate pyproject.toml
.PHONY: check-poetry
check-poetry:
	poetry check

.PHONY: check-codestyle
check-codestyle:
	poetry run ruff format $(CODE_FORMAT) --check

#* Static linters

.PHONY: check-ruff
check-ruff:
	poetry run ruff check $(CODE) --no-fix

.PHONY: check-mypy
check-mypy:
	poetry run mypy --install-types --non-interactive --config-file pyproject.toml $(CODE)

.PHONY: static-lint
static-lint: check-ruff check-mypy

#* Check safety

.PHONY: check-safety
check-safety:
	poetry run safety check --full-report

.PHONY: lint
lint: check-poetry check-codestyle static-lint

.PHONY: update-dev-deps
update-dev-deps:
	poetry add -G dev mypy@latest pre-commit@latest pytest@latest \
										coverage@latest safety@latest typeguard@latest ruff@latest

#* Docker
# Example: make docker-build VERSION=latest
# Example: make docker-build IMAGE=some_name VERSION=0.1.0
.PHONY: docker-build
docker-build:
	@echo Building docker $(IMAGE):$(VERSION) ...
	docker build -t $(IMAGE):$(VERSION) .

# Example: make docker-remove VERSION=latest
# Example: make docker-remove IMAGE=some_name VERSION=0.1.0
.PHONY: docker-remove
docker-remove:
	@echo Removing docker $(IMAGE):$(VERSION) ...
	docker rmi -f $(IMAGE):$(VERSION)

.PHONY: docker-up
docker-up:
	docker compose up

#* Cleaning
.PHONY: pycache-remove
pycache-remove:
	find . | grep -E "(__pycache__|\.pyc|\.pyo$$)" | xargs rm -rf

.PHONY: dsstore-remove
dsstore-remove:
	find . | grep -E ".DS_Store" | xargs rm -rf

.PHONY: mypycache-remove
mypycache-remove:
	find . | grep -E ".mypy_cache" | xargs rm -rf

.PHONY: ipynbcheckpoints-remove
ipynbcheckpoints-remove:
	find . | grep -E ".ipynb_checkpoints" | xargs rm -rf

.PHONY: pytestcache-remove
pytestcache-remove:
	find . | grep -E ".pytest_cache" | xargs rm -rf

.PHONY: ruffcache-remove
ruffcache-remove:
	find . | grep -E ".ruff_cache" | xargs rm -rf

.PHONY: build-remove
build-remove:
	rm -rf build/

.PHONY: reports-remove
reports-remove:
	rm -rf reports/

.PHONY: cleanup
cleanup: pycache-remove dsstore-remove mypycache-remove ruffcache-remove \
ipynbcheckpoints-remove pytestcache-remove reports-remove