Setup a self-hosted Graylog server (e.g. for Django applications)

So other machines can send their logs to the central server for inspection/debugging.

For ease of setup we are going to use the official docker-compose setup here. The example below will be using Debian 11.

Get a VPS with at least 4GB of RAM (less may crash as opensearch demands some resources).
ssh to that VPS.
Install Docker:

curl -fsSL https://get.docker.com -o get-docker.sh
sh ./get-docker.sh

nano ~/.bashrc
alias docker-compose="docker compose"
. ~/.bashrc

Install graylog using docker-compose:

apt-get install git -y
git clone https://github.com/Graylog2/docker-compose.git

cd open-core/
cp .env.example .env

# For GRAYLOG_PASSWORD_SECRET:
pwgen -N 1 -s 96

# For GRAYLOG_ROOT_PASSWORD_SHA2 (the login password's hash, user will be named 'admin'):
echo -n <login-pw> | shasum -a 256

nano .env  # Fill with password + hash from above.

Edit config as needed:

graylog:
  environment:
    GRAYLOG_ALLOW_HIGHLIGHTING: "true"
  ports:
  - "514:5140/tcp"   # Default syslog port is 514.

Start the graylog services:

docker-compose up -d

The graylog UI login will be reachable under localhost:9000 on the VPS, using the default 'admin' username and the password you added to the .env file.

In Graylog, add a Syslog TCP input under System > Inputs. Bind address should be 0.0.0.0 and the port 5140 to match the docker-compose config. Mark it as Global so restarting the containers won't interrupt the input.
On all client machines: Redirect syslog to the graylog server public IP (see ifconfig on the VPS), e.g. via /etc/rsyslog.d/graylog.conf:

*.* @@<public-vps-ip>:514;RSYSLOG_SyslogProtocol23Format

Tell your Django app (settings.py) to write its logs to syslog, so they also end up in graylog. This also uses django-guid for a correlation-ID prefix for each incoming request:

LOGGING = {
    "filters": {
        "correlation_id": {
            "()": "django_guid.log_filters.CorrelationId",
        }
    },
    "formatters": {
        "verbose": {
            "format": "%(levelname)s %(asctime)s [%(correlation_id)s] %(name)s %(message)s"
        },
    },
    "handlers": {
        "graylog": {
            "class": "logging.handlers.SysLogHandler",
            "address": "/dev/log",
            "level": "INFO",
            "formatter": "verbose",
            "filters": ["correlation_id"],
        },
    },
    "loggers": {
        "myproject.myapp": {
            "handlers": ["graylog"],
            "level": "INFO",
            "propagate": True,
        },
        # Also send all unhandled exceptions to graylog.
        "": {
            "handlers": ["graylog"],
            "level": "ERROR",
        },
    },
}

cb109/setup_graylog_server.md

Setup a self-hosted Graylog server (e.g. for Django applications)