Deployment

A single VPS running Docker Compose handles more traffic than most applications will ever see. A compiled Rust binary serving HTML fragments through Axum is fast enough that a 4 GB server comfortably sustains thousands of concurrent users. Start here. Add servers when monitoring shows you need them, not before.

This section covers the full deployment path: cross-compiling the application, packaging it as a Docker image, provisioning infrastructure with Terraform, orchestrating services with Docker Compose, and handling database backups. It starts with a single-server setup and describes how to grow into a multi-server architecture when the time comes.

The architecture progression

Three stages, from simplest to most complex:

Stage 1: Single VPS. One server runs everything: your application, PostgreSQL, Redis, Caddy, and the observability stack. This is the right starting point for most projects. It deploys in minutes and has no distributed-systems complexity.

Stage 2: Separate services. Split into two or three servers: one for your application, one for shared services (PostgreSQL, Redis, Grafana stack), and optionally one for your software forge (Forgejo). Block storage volumes hold persistent data. Tailscale connects the servers privately. This stage suits production workloads that need independent scaling or isolation between the database and the application.

Stage 3: Kubernetes. When Docker Compose on a small number of servers is genuinely constraining you, not before. The scaling strategy section covers the signals.

The Terraform configurations in this section provision stage 2 (separate stacks for volumes, services, and application). Collapsing them onto a single server for stage 1 is straightforward: put everything in one Compose file and skip the networking.

Hetzner Cloud

Hetzner Cloud offers VPS instances at a fraction of the price of AWS or DigitalOcean for equivalent specs. Check Hetzner’s pricing page for current rates.

Server lines

Line	CPU	Use case
CX	Shared Intel/AMD	General workloads
CAX	Shared ARM (Ampere Altra)	Best price-performance ratio
CPX	Dedicated AMD EPYC	CPU-intensive workloads
CCX	Dedicated high-memory	Databases, caching

For a Rust web application, a CX23 or CX33 (4 vCPU / 8 GB) is a strong starting point. The CAX (ARM) line offers better price-performance, but requires ARM64 Docker images, which adds cross-compilation complexity. Stick with x86 (CX line) unless you have a reason to use ARM.

Regions

Hetzner operates data centres in Nuremberg (NBG1), Falkenstein (FSN1), Helsinki (HEL1), Ashburn (ASH), Hillsboro (HIL), and Singapore (SIN). EU regions include 20-60 TB of traffic. US and Singapore regions cost more and include less traffic. Choose the region closest to your users.

Block storage

Hetzner Volumes provide block storage that attaches to a server. Volumes persist independently of the server lifecycle, which is the entire point: you can destroy and recreate a server without losing data.

• 10 GB minimum, 10 TB maximum
• A volume can only attach to one server at a time
• Volume and server must be in the same location
• Data is stored with triple replication

Use volumes for PostgreSQL data directories, Redis persistence, and any other state that must survive a server rebuild.

Building the application

Cross-compile on your development machine (or in CI), then package the binary into a minimal Docker image. This avoids slow in-Docker Rust compilation entirely.

Cross-compilation with cargo-zigbuild

cargo-zigbuild replaces the system linker with Zig’s cross-compilation toolchain. It produces Linux binaries from macOS (or any host) without Docker, a Linux VM, or a separate cross-compilation toolchain.

Install cargo-zigbuild and Zig:

cargo install --locked cargo-zigbuild
brew install zig    # or: pip3 install ziglang

Add the target and build:

rustup target add x86_64-unknown-linux-gnu
cargo zigbuild --release --target x86_64-unknown-linux-gnu

The binary lands in target/x86_64-unknown-linux-gnu/release/. It is dynamically linked against glibc, which is fine because the runtime Docker image includes glibc.

To pin a specific minimum glibc version (Debian 12 ships glibc 2.36):

cargo zigbuild --release --target x86_64-unknown-linux-gnu.2.36

The glibc version suffix ensures the binary runs on any system with that glibc version or newer. This prevents surprises where a binary compiled against a newer glibc fails on an older host.

The Dockerfile

The Docker image does not compile anything. It copies the pre-built binary into a minimal base image.

FROM gcr.io/distroless/cc-debian12:nonroot
COPY target/x86_64-unknown-linux-gnu/release/myapp /usr/local/bin/myapp
ENTRYPOINT ["/usr/local/bin/myapp"]

distroless/cc-debian12 includes glibc, libgcc, CA certificates, and timezone data. Nothing else: no shell, no package manager, no utilities. The :nonroot tag runs as UID 65534 instead of root.

The resulting image is roughly 30 MB. Builds take seconds because there is no compilation step.

Build and tag the image:

cargo zigbuild --release --target x86_64-unknown-linux-gnu
docker build -t git.example.com/myorg/myapp:v1.0.0 .

Pushing to the registry

This section assumes a running Forgejo instance with its built-in container registry. The registry is enabled by default.

Log in and push:

docker login git.example.com
docker push git.example.com/myorg/myapp:v1.0.0

Authenticate with a Forgejo personal access token that has package:read and package:write scopes. Create one under Settings > Applications in Forgejo.

Infrastructure with Terraform

Terraform provisions the servers, volumes, firewall rules, and networking. The hcloud provider (v1.60+) is officially maintained by Hetzner and covers the full API.

Stack separation

Split infrastructure into three Terraform stacks (separate state files):

1. Volumes stack: block storage for persistent data. Deploy first, destroy last (or never).
2. Services stack: VPS for PostgreSQL, Redis, observability. References volumes by data source.
3. Application stack: VPS for Caddy and the application. Can be destroyed and recreated without affecting data.

This separation protects persistent data. Rebuilding the application server does not touch the database volume. Rebuilding the services server does not touch the volumes themselves.

Shared variables

Each stack needs access to the Hetzner API token and common settings. Create a terraform.tfvars file (git-ignored) in each stack directory:

hcloud_token = "your-hetzner-api-token"
location     = "fsn1"
ssh_key_name = "deploy-key"

Stack 1: Volumes

# stacks/volumes/main.tf
terraform {
  required_providers {
    hcloud = {
      source  = "hetznercloud/hcloud"
      version = "~> 1.60"
    }
  }
}

variable "hcloud_token" { sensitive = true }
variable "location" { default = "fsn1" }

provider "hcloud" {
  token = var.hcloud_token
}

resource "hcloud_volume" "postgres" {
  name              = "postgres-data"
  size              = 50
  location          = var.location
  format            = "ext4"
  delete_protection = true

  labels = {
    service = "postgres"
  }
}

resource "hcloud_volume" "redis" {
  name              = "redis-data"
  size              = 10
  location          = var.location
  format            = "ext4"
  delete_protection = true

  labels = {
    service = "redis"
  }
}

resource "hcloud_volume" "grafana" {
  name              = "grafana-data"
  size              = 20
  location          = var.location
  format            = "ext4"
  delete_protection = true

  labels = {
    service = "grafana"
  }
}

output "postgres_volume_id" { value = hcloud_volume.postgres.id }
output "redis_volume_id" { value = hcloud_volume.redis.id }
output "grafana_volume_id" { value = hcloud_volume.grafana.id }

delete_protection = true prevents accidental terraform destroy from deleting the volumes. To remove a protected volume, you must first set delete_protection = false and apply, then destroy. This is intentional friction.

Stack 2: Services VPS

# stacks/services/main.tf
terraform {
  required_providers {
    hcloud = {
      source  = "hetznercloud/hcloud"
      version = "~> 1.60"
    }
  }
}

variable "hcloud_token" { sensitive = true }
variable "location" { default = "fsn1" }
variable "ssh_key_name" {}
variable "postgres_volume_id" {}
variable "redis_volume_id" {}
variable "grafana_volume_id" {}

provider "hcloud" {
  token = var.hcloud_token
}

data "hcloud_ssh_key" "deploy" {
  name = var.ssh_key_name
}

resource "hcloud_firewall" "services" {
  name = "services"

  # SSH (lock down to Tailscale once configured)
  rule {
    direction  = "in"
    protocol   = "tcp"
    port       = "22"
    source_ips = ["0.0.0.0/0", "::/0"]
  }

  # Tailscale UDP
  rule {
    direction  = "in"
    protocol   = "udp"
    port       = "41641"
    source_ips = ["0.0.0.0/0", "::/0"]
  }
}

resource "hcloud_server" "services" {
  name        = "services-1"
  server_type = "cx33"
  image       = "ubuntu-24.04"
  location    = var.location
  ssh_keys    = [data.hcloud_ssh_key.deploy.id]
  firewall_ids = [hcloud_firewall.services.id]

  user_data = file("${path.module}/cloud-init.yaml")
}

resource "hcloud_volume_attachment" "postgres" {
  volume_id = var.postgres_volume_id
  server_id = hcloud_server.services.id
  automount = true
}

resource "hcloud_volume_attachment" "redis" {
  volume_id = var.redis_volume_id
  server_id = hcloud_server.services.id
  automount = true
}

resource "hcloud_volume_attachment" "grafana" {
  volume_id = var.grafana_volume_id
  server_id = hcloud_server.services.id
  automount = true
}

output "services_ip" { value = hcloud_server.services.ipv4_address }

Pass the volume IDs from stack 1 via terraform.tfvars:

# stacks/services/terraform.tfvars
hcloud_token       = "your-token"
location           = "fsn1"
ssh_key_name       = "deploy-key"
postgres_volume_id = 12345678
redis_volume_id    = 12345679
grafana_volume_id  = 12345680

Stack 3: Application VPS

# stacks/app/main.tf
terraform {
  required_providers {
    hcloud = {
      source  = "hetznercloud/hcloud"
      version = "~> 1.60"
    }
  }
}

variable "hcloud_token" { sensitive = true }
variable "location" { default = "fsn1" }
variable "ssh_key_name" {}
variable "domain" {}

provider "hcloud" {
  token = var.hcloud_token
}

data "hcloud_ssh_key" "deploy" {
  name = var.ssh_key_name
}

resource "hcloud_firewall" "app" {
  name = "app"

  # HTTP
  rule {
    direction  = "in"
    protocol   = "tcp"
    port       = "80"
    source_ips = ["0.0.0.0/0", "::/0"]
  }

  # HTTPS
  rule {
    direction  = "in"
    protocol   = "tcp"
    port       = "443"
    source_ips = ["0.0.0.0/0", "::/0"]
  }

  # SSH (lock down to Tailscale once configured)
  rule {
    direction  = "in"
    protocol   = "tcp"
    port       = "22"
    source_ips = ["0.0.0.0/0", "::/0"]
  }

  # Tailscale UDP
  rule {
    direction  = "in"
    protocol   = "udp"
    port       = "41641"
    source_ips = ["0.0.0.0/0", "::/0"]
  }
}

resource "hcloud_server" "app" {
  name        = "app-1"
  server_type = "cx23"
  image       = "ubuntu-24.04"
  location    = var.location
  ssh_keys    = [data.hcloud_ssh_key.deploy.id]
  firewall_ids = [hcloud_firewall.app.id]

  user_data = file("${path.module}/cloud-init.yaml")
}

output "app_ip" { value = hcloud_server.app.ipv4_address }

Cloud-init for server setup

Both VPS stacks use a cloud-init.yaml that installs Docker and Tailscale on first boot:

#cloud-config
package_update: true
package_upgrade: true

packages:
  - curl
  - ca-certificates

runcmd:
  # Install Docker
  - curl -fsSL https://get.docker.com | sh
  - systemctl enable docker
  - systemctl start docker

  # Install Tailscale
  - curl -fsSL https://tailscale.com/install.sh | sh

  # Create application directory
  - mkdir -p /opt/app

After Terraform provisions the server, SSH in and run tailscale up with an auth key to join the tailnet. Subsequent deployments happen over Tailscale.

Tailscale for secure networking

Tailscale builds a WireGuard-based mesh VPN between your servers. Each server gets a stable IP on the 100.x.y.z range. All traffic between servers is encrypted end-to-end. The free tier supports up to 3 users and 100 devices.

Why Tailscale

Without Tailscale, the services VPS must expose PostgreSQL (port 5432), Redis (port 6379), and the Grafana stack to the public internet, even if firewalled to specific IPs. With Tailscale, these services bind only to the Tailscale interface. They are invisible to the public internet entirely.

Tailscale also simplifies SSH access. Once your servers are on the tailnet, you can close port 22 in the Hetzner firewall and SSH over the Tailscale IP instead.

Setting up servers

On each server, install Tailscale (already done via cloud-init) and authenticate:

# Generate an auth key in the Tailscale admin console
# Use a reusable, tagged key: --advertise-tags=tag:server
tailscale up --authkey tskey-auth-xxxxx --advertise-tags=tag:server

Tagged auth keys disable key expiry, so the server stays connected indefinitely. Generate auth keys in the Tailscale admin console.

Verify connectivity:

# From the app server, ping the services server by Tailscale hostname
ping services-1

Tailscale’s MagicDNS assigns each machine a hostname on your tailnet. Use these hostnames in your application’s DATABASE_URL and Redis connection strings instead of public IPs.

Docker sidecar pattern

For services running inside Docker containers, use a Tailscale sidecar container. Other containers share its network stack via network_mode: service::

services:
  tailscale:
    image: tailscale/tailscale:latest
    hostname: app-1
    environment:
      TS_AUTHKEY: ${TS_AUTHKEY}
      TS_EXTRA_ARGS: --advertise-tags=tag:server
      TS_STATE_DIR: /var/lib/tailscale
      TS_USERSPACE: "false"
    volumes:
      - ts-state:/var/lib/tailscale
    devices:
      - /dev/net/tun:/dev/net/tun
    cap_add:
      - net_admin
    restart: unless-stopped

  app:
    image: git.example.com/myorg/myapp:latest
    network_mode: service:tailscale
    depends_on:
      - tailscale

network_mode: service:tailscale makes the app container reachable at the Tailscale IP. The Tailscale state volume (ts-state) preserves the node identity across container restarts.

The sidecar approach is most useful when the application itself needs to be reachable over Tailscale. For the simpler case where the application only connects to services over Tailscale (and the host already has Tailscale installed), the host-level Tailscale installation is sufficient and the sidecar is not needed.

Production Docker Compose

Services VPS

The services VPS runs PostgreSQL, Redis, and the observability stack. Each service stores data on a Hetzner Volume mounted to the host.

# /opt/app/compose.yaml (services VPS)
services:
  postgres:
    image: postgres:17-alpine
    restart: unless-stopped
    ports:
      - "100.x.y.z:5432:5432"
    volumes:
      - /mnt/postgres-data/pgdata:/var/lib/postgresql/data
    environment:
      POSTGRES_USER: myapp
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_DB: myapp
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U myapp"]
      interval: 10s
      timeout: 5s
      retries: 5
    logging:
      driver: "json-file"
      options:
        max-size: "10m"
        max-file: "3"

  redis:
    image: redis:7-alpine
    restart: unless-stopped
    ports:
      - "100.x.y.z:6379:6379"
    volumes:
      - /mnt/redis-data:/data
    command: redis-server --appendonly yes --requirepass ${REDIS_PASSWORD}
    healthcheck:
      test: ["CMD", "redis-cli", "-a", "${REDIS_PASSWORD}", "ping"]
      interval: 10s
      timeout: 5s
      retries: 5
    logging:
      driver: "json-file"
      options:
        max-size: "10m"
        max-file: "3"

Replace 100.x.y.z with the services server’s Tailscale IP. Binding to the Tailscale IP means PostgreSQL and Redis accept connections only from the tailnet, not from the public internet.

Volume mount paths (/mnt/postgres-data/, /mnt/redis-data/) correspond to where Hetzner automounts the attached volumes. Check the mount points with lsblk or df -h after Terraform provisions the server.

The observability stack (Grafana, Loki, Tempo, Prometheus, OTel Collector) runs on the same VPS. See the observability section for Docker Compose configuration of those services.

Application VPS

The application VPS runs Caddy and the application.

# /opt/app/compose.yaml (app VPS)
services:
  app:
    image: git.example.com/myorg/myapp:${TAG:-latest}
    restart: unless-stopped
    expose:
      - "3000"
    env_file:
      - .env.production
    healthcheck:
      test: ["CMD", "/usr/local/bin/myapp", "healthcheck"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 15s
    logging:
      driver: "json-file"
      options:
        max-size: "10m"
        max-file: "3"

  caddy:
    image: caddy:2-alpine
    restart: unless-stopped
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./Caddyfile:/etc/caddy/Caddyfile:ro
      - caddy-data:/data
      - caddy-config:/config
    depends_on:
      app:
        condition: service_healthy

volumes:
  caddy-data:
  caddy-config:

The application reads DATABASE_URL, REDIS_URL, and other configuration from .env.production. These connection strings use Tailscale hostnames:

DATABASE_URL=postgres://myapp:password@services-1:5432/myapp
REDIS_URL=redis://:password@services-1:6379
OTEL_EXPORTER_OTLP_ENDPOINT=http://services-1:4317

See the configuration section for the full .env.production setup and how Terraform provisions it.

Health check implementation

The distroless runtime image has no shell, so curl and wget are not available. Implement a health check subcommand in the application binary:

fn main() {
    dotenvy::dotenv().ok();

    if std::env::args().nth(1).as_deref() == Some("healthcheck") {
        match healthcheck() {
            Ok(()) => std::process::exit(0),
            Err(e) => {
                eprintln!("healthcheck failed: {e}");
                std::process::exit(1);
            }
        }
    }

    run();
}

fn healthcheck() -> Result<(), Box<dyn std::error::Error>> {
    let port = std::env::var("PORT").unwrap_or_else(|_| "3000".to_string());
    let url = format!("http://127.0.0.1:{port}/health");
    let resp = ureq::get(&url).call()?;
    if resp.status() == 200 { Ok(()) } else { Err("non-200 status".into()) }
}

Add ureq (a minimal synchronous HTTP client) to the server crate’s dependencies. The health check subcommand calls the application’s /health endpoint and exits with code 0 or 1. Docker uses the exit code to determine container health.

Caddy as reverse proxy

Caddy handles TLS termination and reverse proxying. Its defining feature is automatic HTTPS: point a domain at your server, and Caddy obtains and renews a Let’s Encrypt certificate with zero configuration.

Caddyfile

app.example.com {
    reverse_proxy app:3000
}

That is the entire configuration. Caddy obtains a TLS certificate for app.example.com, terminates HTTPS, and proxies requests to the application container on port 3000.

For multiple services behind different subdomains:

app.example.com {
    reverse_proxy app:3000
}

grafana.example.com {
    reverse_proxy services-1:3000
}

Caddy resolves services-1 via Tailscale’s MagicDNS when the Caddy container shares the host’s Tailscale network (or uses the Tailscale sidecar pattern).

Requirements

Caddy’s automatic HTTPS needs two things:

1. A DNS A record pointing your domain to the server’s public IP.
2. Ports 80 and 443 open and routed to Caddy (for the ACME challenge and HTTPS traffic).

Both are handled by the Terraform firewall configuration and your DNS provider. If you manage DNS through Hetzner, the hcloud provider supports DNS zone and record resources (hcloud_zone, hcloud_zone_record).

Why Caddy over Nginx

Nginx requires certbot, cron jobs, and manual renewal configuration for Let’s Encrypt certificates. Caddy does this automatically. For a reverse proxy in front of a Rust application, Caddy’s simpler configuration and automatic certificate management eliminate an entire category of operational work. Nginx’s performance advantage is irrelevant at the traffic levels where you are running one or two VPS instances.

Database deployment and backups

PostgreSQL runs in a Docker container on the services VPS, with its data directory mounted on a Hetzner Volume. The volume provides persistence and triple replication at the storage layer. Backups provide recovery from logical errors (accidental deletes, bad migrations) that replication cannot protect against.

pg_dump with cron

A daily pg_dump via cron is the simplest backup strategy and covers the majority of use cases.

Create the backup script on the services VPS:

#!/usr/bin/env bash
# /opt/app/scripts/backup-db.sh
set -euo pipefail

BACKUP_DIR="/opt/app/backups"
RETENTION_DAYS=14
TIMESTAMP=$(date +%Y%m%d-%H%M%S)
DUMP_FILE="${BACKUP_DIR}/myapp-${TIMESTAMP}.dump"

mkdir -p "${BACKUP_DIR}"

# Dump the database (custom format, compressed)
docker exec postgres pg_dump \
  -U myapp \
  -Fc \
  myapp > "${DUMP_FILE}"

# Remove backups older than retention period
find "${BACKUP_DIR}" -name "myapp-*.dump" -mtime +${RETENTION_DAYS} -delete

echo "Backup complete: ${DUMP_FILE} ($(du -h "${DUMP_FILE}" | cut -f1))"

-Fc produces a custom-format dump: compressed, supports selective restore of individual tables, and works with pg_restore. It is smaller and more flexible than plain SQL dumps.

Scheduling with cron

# Run daily at 02:00 UTC
echo "0 2 * * * root /opt/app/scripts/backup-db.sh >> /var/log/db-backup.log 2>&1" \
  | sudo tee /etc/cron.d/db-backup

Off-site copies

A backup on the same server as the database is not a real backup. Copy dumps to an S3-compatible storage provider. Add this to the backup script after the dump:

# Upload to S3 (Hetzner Object Storage, Backblaze B2, etc.)
S3_BUCKET="s3://myapp-backups"
S3_ENDPOINT="https://fsn1.your-objectstorage.com"

aws s3 cp "${DUMP_FILE}" "${S3_BUCKET}/db/${TIMESTAMP}.dump" \
  --endpoint-url "${S3_ENDPOINT}"

# Remove remote backups older than 30 days
aws s3 ls "${S3_BUCKET}/db/" --endpoint-url "${S3_ENDPOINT}" \
  | awk '{print $4}' \
  | head -n -30 \
  | xargs -I{} aws s3 rm "${S3_BUCKET}/db/{}" --endpoint-url "${S3_ENDPOINT}"

Install the AWS CLI on the services VPS (apt install awscli) and configure it with your S3 credentials. The AWS CLI works with any S3-compatible provider.

Restoring from backup

# Stop the application first to prevent writes during restore
docker exec -i postgres pg_restore \
  -U myapp \
  -d myapp \
  --clean \
  --if-exists \
  < /opt/app/backups/myapp-20260227-020000.dump

--clean --if-exists drops existing objects before restoring, which handles the common case of restoring to an existing database. Test the restore process periodically. An untested backup is not a backup.

When pg_dump is not enough

pg_dump takes a full snapshot every time. For databases larger than a few GB, consider:

• pgBackRest: supports incremental and differential backups, parallel restore, backup verification, and direct archiving to S3. It is the standard tool for serious PostgreSQL backup infrastructure.
• WAL archiving with point-in-time recovery: continuous archiving of write-ahead logs enables recovery to any point in time, not just the last backup. This requires more configuration but provides the strongest recovery guarantees.

For most applications in this guide’s scope, daily pg_dump with off-site copies is sufficient.

Deploying updates

The simplest deployment workflow: pull the new image and recreate the container.

# On the app VPS
docker compose pull app
docker compose up -d app

docker compose up -d app recreates only the app service if its image has changed. There is a brief interruption (typically 1-3 seconds) while the old container stops and the new one starts and passes its health check. For most applications, this is acceptable.

Wrap this in a deployment script:

#!/usr/bin/env bash
# deploy.sh
set -euo pipefail

TAG="${1:?Usage: deploy.sh <tag>}"
export TAG

cd /opt/app
docker compose pull app
docker compose up -d app

echo "Deployed ${TAG}"
echo "Waiting for health check..."
docker compose exec app /usr/local/bin/myapp healthcheck
echo "Healthy."

Trigger the script from CI or run it manually over SSH (via Tailscale):

ssh services-1 "cd /opt/app && TAG=v1.2.3 ./deploy.sh v1.2.3"

Zero-downtime with Caddy

If a few seconds of downtime during deployment is not acceptable, use a blue-green approach with Caddy’s admin API. Run two instances of the application (blue and green). Deploy to the inactive one, verify it is healthy, then switch Caddy’s upstream.

Caddy exposes an admin API on localhost:2019 that can update routing without a restart. Switch the upstream from blue to green atomically:

curl -X PATCH http://localhost:2019/config/apps/http/servers/srv0/routes/0/handle/0/upstreams \
  -H "Content-Type: application/json" \
  -d '[{"dial": "green:3000"}]'

This is more complex to set up and maintain. Start with the simple recreate approach and add blue-green when you have a genuine need for zero-downtime deployments.

Scaling strategy

Docker Compose on a single VPS scales further than most people expect. A Rust application serving HTML is fast. PostgreSQL on a dedicated volume with proper indexes handles millions of rows. You will likely hit organisational complexity before you hit server capacity.

When to add servers

• Database contention: the application and database compete for CPU or memory on the same server. Move the database to the services VPS (stage 2).
• Backup impact: pg_dump on a busy database affects application latency. A separate services VPS isolates backup I/O.
• Independent scaling: the application needs more CPU but the database does not (or vice versa). Separate servers let you size each independently.
• Isolation requirements: security policy requires the database to be on a server with no public internet access.

When to consider Kubernetes

Stay on Docker Compose until you genuinely exhaust what a small number of VPS instances can provide. The signals that Kubernetes might be worth the operational cost:

• Multiple application instances across servers: you need horizontal scaling beyond what a single server provides, and a load balancer in front of multiple app servers.
• Auto-scaling: traffic is bursty enough that you need to add and remove capacity automatically.
• Many services with complex dependencies: once you pass 10-20 containers with interdependent deployment ordering, Docker Compose files become fragile.
• Multiple teams deploying independently: Kubernetes namespaces and RBAC provide isolation that Docker Compose does not.

If you reach this point, k3s is a lightweight Kubernetes distribution that runs on a single node or a small cluster. It installs in under a minute, uses roughly 500 MB of RAM, and provides full Kubernetes API compatibility. k3s on two or three Hetzner CX33 instances is a reasonable stepping stone before full managed Kubernetes.

Do not adopt Kubernetes because it seems like the professional choice. The operational complexity is real. For most Rust web applications, Docker Compose on one to three servers is the right answer for years.

Gotchas

Mount the Hetzner Volume before starting containers. If a volume is not mounted when Docker Compose starts, containers write to the server’s local disk. When the volume is later mounted, the data on local disk is hidden. Verify mount points with df -h before the first docker compose up.

Pin Docker image tags in production. Use myapp:v1.2.3, not myapp:latest. With latest, docker compose pull fetches whatever was most recently pushed, which makes deployments unpredictable and rollbacks impossible.

Caddy’s data volume is important. The caddy-data volume stores TLS certificates and ACME account keys. Losing this volume means Caddy must re-issue all certificates, which can hit Let’s Encrypt rate limits (50 certificates per registered domain per week). Keep the Caddy data volume on persistent storage.

Tailscale auth keys expire. Pre-authentication keys have a maximum lifetime of 90 days. Once a server joins the tailnet, the key is no longer needed, so this only matters for new server provisioning. Use tagged auth keys (--advertise-tags=tag:server) to disable node key expiry on the device itself.

Docker logging fills disks. Without max-size and max-file on the json-file logging driver, container logs grow without bound. Set these on every service in production Compose files.

Test your backups. Periodically restore a backup to a temporary database and verify the data. A backup that cannot be restored is not a backup.

Distroless has no shell. You cannot docker exec -it app bash into a distroless container. For debugging, run a temporary container with a full image (docker run -it --network container:app debian:bookworm-slim bash) that shares the application container’s network namespace. Or add a debug sidecar to the Compose file temporarily.