---
url: /sibyl/deployment/docker-compose.md
---
# Docker Compose Deployment

Local development using Docker Compose for database services with the Python/Node applications
running natively.

## Architecture

Docker Compose runs the database services while applications run natively for hot reload:

```
+------------------+     +------------------+
|   Native Apps    |     |   Docker Compose |
|------------------|     |------------------|
| Backend (:3334)  |---->| SurrealDB (:8000)|
| Frontend (:3337) |     | Redis* (:6381)   |
| Jobs + Schedules |     |                  |
+------------------+     +------------------+
```

`Redis` is opt-in for distributed or multi-process dev. The default local path only needs SurrealDB.

## Prerequisites

* Docker and Docker Compose
* Python 3.13+
* Node.js 24 and pnpm
* uv (Python package manager)

## Quick Start

```bash
# 1. Start database services
docker compose --env-file /dev/null up -d surrealdb

# 2. Install dependencies
uv sync                         # Python packages
cd apps/web && pnpm install     # Frontend packages

# 3. Configure your shell
export SIBYL_OPENAI_API_KEY=sk-...
# SIBYL_JWT_SECRET is auto-generated in dev.

# 4. Start all services
moon run dev
```

For Redis-backed coordination, opt into the `redis` profile explicitly:

```bash
docker compose --env-file /dev/null --profile redis up -d surrealdb redis
SIBYL_COORDINATION_BACKEND=redis moon run dev
```

## Service Definitions

The root `docker-compose.yml` defines a Surreal-first local stack plus an opt-in `redis` profile. It
runs only the data services; the API and web apps run natively for hot reload.

```yaml
services:
  surrealdb:
    image: ${SIBYL_SURREAL_IMAGE:-surrealdb/surrealdb:v3.2.4}
    container_name: ${COMPOSE_PROJECT_NAME:-sibyl}-surrealdb
    command:
      [
        "start",
        "--log",
        "info",
        "--user",
        "${SIBYL_SURREAL_USERNAME:-root}",
        "--pass",
        "${SIBYL_SURREAL_PASSWORD:-root}",
        "${SIBYL_SURREAL_PATH:-rocksdb:///data/sibyl.db}",
      ]
    ports:
      - "${SIBYL_SURREAL_PORT:-8000}:8000"
    volumes:
      # `:U` chowns the bind mount to the container UID under rootless podman
      - "${SURREAL_DATA_DIR:-./.moon/cache/surreal-dev}:/data:U"

  redis:
    image: valkey/valkey:8-alpine
    container_name: ${COMPOSE_PROJECT_NAME:-sibyl}-redis
    profiles: ["redis"]
    ports:
      - "${SIBYL_REDIS_PORT:-6381}:6379"
    command: ["valkey-server", "--save", "", "--appendonly", "no"]
```

The default image is pinned to SurrealDB server `v3.2.4` for reproducible local and CI behavior.
Override `SIBYL_SURREAL_IMAGE` when rehearsing a newer server patch. The root compose stores
SurrealDB data in a bind mount under `.moon/cache/surreal-dev` so it survives container churn and is
easy to inspect.

## Port Mappings

| Service   | Host Port | Container Port | Purpose                       |
| --------- | --------- | -------------- | ----------------------------- |
| SurrealDB | 8000      | 8000           | Default local graph runtime   |
| Redis     | 6381      | 6379           | Optional coordination backend |

Ports are offset from defaults to avoid conflicts with local services.

## Moonrepo Commands

```bash
# Start databases only
moon run docker-up

# Stop databases
moon run docker-down

# Start recommended Surreal local-dev stack
moon run dev

# Start API only with in-process jobs (no frontend)
moon run dev-api

# Start frontend only
moon run dev-web

# Start a standalone worker process (used with SIBYL_COORDINATION_BACKEND=redis)
moon run api:worker

# Stop all services
moon run stop
```

## Full Stack Compose

For a complete containerized deployment (backend + frontend + databases), use
`docker-compose.prod.yml`:

```bash
# Copy the production env-file example outside the repo
cp .env.example ~/.sibyl/prod.env

# Edit ~/.sibyl/prod.env with required secrets:
#   SIBYL_JWT_SECRET=<generate with: openssl rand -hex 32>
#   SIBYL_OPENAI_API_KEY=sk-...
#   SIBYL_ANTHROPIC_API_KEY=sk-ant-...

# Start the repo production compose directly
docker compose --env-file ~/.sibyl/prod.env -f docker-compose.prod.yml up -d

# Or generate ~/.sibyl/docker/.env and a pinned compose file
sibyl docker init

# Optional: run a separate worker with Redis/Valkey coordination
sibyl docker init --force --with-worker

# Or run the repo production compose with its worker: the backend and worker
# then share the redis service for jobs, locks, events and rate limits
SIBYL_COORDINATION_BACKEND=redis docker compose --env-file ~/.sibyl/prod.env \
  -f docker-compose.prod.yml --profile redis up -d

# Start, follow logs, and stop
sibyl docker up
sibyl docker logs
sibyl docker down
```

The backend and worker read the same `SIBYL_COORDINATION_BACKEND`, defaulting to `local`. Set it to
`redis` whenever the `redis` profile runs: a worker left on `local` exits with an error instead of
idling while the backend runs every job itself. Use `local` or `redis` in these files rather than
`auto`, because the compose files always set the Redis address and `auto` would resolve to `redis`.
The worker's healthcheck runs `sibyld worker --check`, which passes while a worker heartbeat is live
in Valkey (the heartbeat belongs to the queue, so with several workers it shows that one is live,
not which). `sibyl docker upgrade` brings an older `--with-worker` bundle up to the same wiring and
moves its settings key into the bundle's `.env`, keeping the key the API container generated,
whether that container is running or stopped. If a key exists but cannot be read, the upgrade stops
without changing anything.

The repo's `docker-compose.prod.yml` refuses to start without both `SIBYL_OPENAI_API_KEY` and
`SIBYL_ANTHROPIC_API_KEY`, and it pins Anthropic as the language model, so switching providers there
means editing the file. The bundle `sibyl docker init` generates sets no provider keys: an admin
enters keys and picks providers in the setup wizard and the admin AI settings.

Amazon Bedrock takes its settings (the Region, inference scope, API choice, and an optional Bedrock
API key) only from the environment; the settings UI has no field for them. AWS credentials resolve
through the standard AWS credential chain instead: environment keys, a shared profile
(`SIBYL_BEDROCK_PROFILE` or `AWS_PROFILE`), SSO, or an instance role. Every process that calls a
model reads these for itself, and a worker container has its own `environment` block, so on either
stack add the [Bedrock variables](./environment.md#amazon-bedrock) and credential access to the API
container and to every enabled worker. The [Helm chart](./helm-chart.md#amazon-bedrock) shares them
across pods for you.

### Production Compose Services

```yaml
services:
  backend:
    build:
      context: .
      dockerfile: apps/api/Dockerfile
    ports:
      - "3334:3334"
    environment:
      SIBYL_SURREAL_URL: ws://surrealdb:8000/rpc
      SIBYL_JWT_SECRET: ${SIBYL_JWT_SECRET}
      SIBYL_OPENAI_API_KEY: ${SIBYL_OPENAI_API_KEY}
    depends_on:
      surrealdb:
        condition: service_healthy

  worker:
    profiles: ["redis"]
    build:
      context: .
      dockerfile: apps/api/Dockerfile
    command: ["sibyld", "worker"]
    environment:
      SIBYL_COORDINATION_BACKEND: redis
      SIBYL_REDIS_HOST: redis
      SIBYL_REDIS_PORT: 6379
      SIBYL_SURREAL_URL: ws://surrealdb:8000/rpc
      SIBYL_JWT_SECRET: ${SIBYL_JWT_SECRET}
      SIBYL_OPENAI_API_KEY: ${SIBYL_OPENAI_API_KEY}
    depends_on:
      surrealdb:
        condition: service_healthy
      redis:
        condition: service_started

  redis:
    profiles: ["redis"]
    image: valkey/valkey:8-alpine
    ports:
      - "6381:6379"

  frontend:
    build:
      context: ./apps/web
      dockerfile: Dockerfile
    ports:
      - "3337:3337"
    environment:
      NEXT_PUBLIC_API_URL: ${SIBYL_SERVER_URL:-http://localhost:3334}
      SIBYL_API_URL: http://backend:3334/api
    depends_on:
      backend:
        condition: service_healthy

  surrealdb:
    image: ${SIBYL_SURREAL_IMAGE:-surrealdb/surrealdb:v3.2.4}
    ports:
      - "${SIBYL_SURREAL_PORT:-8000}:8000"
    volumes:
      - surreal_data:/data
    healthcheck:
      test: ["CMD", "/surreal", "is-ready", "--endpoint", "http://localhost:8000"]

volumes:
  surreal_data:
```

Behind nginx, Caddy, or another reverse proxy, set `SIBYL_FORWARDED_ALLOW_IPS` to the proxy's own
address so logins rate-limit per user instead of per proxy. Run the proxy as a container on the same
network and stop publishing port 3334; the
[trusted proxy reference](./environment.md#trusted-proxies) explains why a Docker bridge range is
never safe to list.

The production compose persists SurrealDB to a named Docker volume (`surreal_data`) rather than a
bind mount. `NEXT_PUBLIC_API_URL` is the browser-facing API URL; `SIBYL_API_URL` is the in-network
URL the Next.js server uses for SSR fetches.

## Quickstart Compose (Zero-Config)

For individual developers who want a running stack in minutes using pre-built images, use
`docker-compose.quickstart.yml`. No `.env` file is required; API keys are entered through the web UI
during onboarding and stored encrypted in the database.

```bash
# Start with pre-built images from ghcr.io
docker compose --env-file /dev/null -f docker-compose.quickstart.yml up -d

# Open http://localhost:3337 and complete onboarding

# Run alongside an existing dev setup on offset ports
docker compose --env-file /dev/null \
  -f docker-compose.quickstart.yml \
  -f docker-compose.quickstart.test.yml up -d
```

Differences from the production compose:

* Pulls `ghcr.io/hyperb1iss/sibyl-api` and `sibyl-web` images instead of building locally
* `SIBYL_JWT_SECRET` and `SIBYL_SETTINGS_KEY` auto-generate when unset (persisted in the
  `sibyl_secrets` volume mounted at `/home/sibyl/.sibyl`). The `secrets-init` service writes the
  settings key once, before the api and worker start, because a process on Redis coordination never
  generates its own: it reads that shared key, or `SIBYL_SETTINGS_KEY`, and refuses to start with
  neither
* Runs with `SIBYL_ENVIRONMENT=development` and a `sibyl_quickstart` default Surreal password
* The backend service is named `api` and the frontend `web`

When upgrading from a pre-1.0 RC quickstart compose file by hand, keep the `secrets-init` service
from the current compose file. It chowns the persisted `/home/sibyl/.sibyl` volume to the current
API service UID (`10001`) so older `1000:1000` volumes stay writable after the image upgrade. If you
intentionally pin an older API image, set `SIBYL_API_UID=1000` and `SIBYL_API_GID=1000` while that
older image is running.

## Volume Persistence

The root local-dev compose persists SurrealDB data in the bind mount configured by
`SURREAL_DATA_DIR`, defaulting to `.moon/cache/surreal-dev`.

```bash
# Inspect the default local Surreal data directory
ls .moon/cache/surreal-dev
```

The production and quickstart compose files persist SurrealDB to named Docker volumes
(`surreal_data` and `sibyl_surreal`):

```bash
docker volume ls | grep sibyl

# Remove volumes (DESTROYS DATA)
sibyl docker down --volumes
```

## Connecting to Databases

### SurrealDB Health

```bash
curl http://localhost:8000/health
```

### Redis CLI

```bash
# Only when the redis profile is running
docker exec -it sibyl-redis redis-cli
```

## Troubleshooting

### Port Conflicts

If ports 8000 or 6381 are in use:

```bash
# Check what's using the port
lsof -i :8000
lsof -i :6381

# Stop conflicting services or modify docker-compose.yml ports
```

### Database Not Starting

```bash
# Check container logs
docker compose --env-file /dev/null logs surrealdb
docker compose --env-file /dev/null logs redis

# Restart with clean state
docker compose --env-file /dev/null down -v
docker compose --env-file /dev/null up -d surrealdb
```

### Connection Refused

Ensure your shell environment uses the correct ports:

```bash
SIBYL_SURREAL_URL=ws://127.0.0.1:8000/rpc
SIBYL_REDIS_PORT=6381        # Only when coordination backend is redis
```

## Next Steps

* [Environment Variables](environment.md) - Full configuration options
* [Troubleshooting](troubleshooting.md) - Common issues