Docker Compose Deployment

Local development using Docker Compose for database services with the Python/Node applications running natively.

Architecture

Docker Compose runs the database services while applications run natively for hot reload:

+------------------+     +------------------+
|   Native Apps    |     |   Docker Compose |
|------------------|     |------------------|
| Backend (:3334)  |---->| SurrealDB (:8000)|
| Frontend (:3337) |     | Redis* (:6381)   |
| Jobs + Schedules |     |                  |
+------------------+     +------------------+

Redis is opt-in for distributed or multi-process dev. The default local path only needs SurrealDB.

Prerequisites

Docker and Docker Compose
Python 3.13+
Node.js 20+ and pnpm
uv (Python package manager)

Quick Start

bash

# 1. Start database services
docker compose up -d surrealdb

# 2. Install dependencies
uv sync                         # Python packages
cd apps/web && pnpm install     # Frontend packages

# 3. Configure environment
cp .env.example .env
# Edit .env and add:
#   SIBYL_JWT_SECRET=<random-secret>
#   SIBYL_OPENAI_API_KEY=sk-...

# 4. Start all services
moon run dev

For Redis-backed coordination, opt into the redis profile explicitly:

bash

docker compose --profile redis up -d surrealdb redis
SIBYL_COORDINATION_BACKEND=redis moon run dev

Service Definitions

The root docker-compose.yml defines a Surreal-first local stack plus an opt-in redis profile. It runs only the data services; the API and web apps run natively for hot reload.

yaml

services:
  surrealdb:
    image: ${SIBYL_SURREAL_IMAGE:-surrealdb/surrealdb:v3.1.0}
    container_name: ${COMPOSE_PROJECT_NAME:-sibyl}-surrealdb
    command:
      [
        "start",
        "--log",
        "info",
        "--user",
        "${SIBYL_SURREAL_USERNAME:-root}",
        "--pass",
        "${SIBYL_SURREAL_PASSWORD:-root}",
        "${SIBYL_SURREAL_PATH:-rocksdb:///data/sibyl.db}",
      ]
    ports:
      - "${SIBYL_SURREAL_PORT:-8000}:8000"
    volumes:
      # `:U` chowns the bind mount to the container UID under rootless podman
      - "${SURREAL_DATA_DIR:-./.moon/cache/surreal-dev}:/data:U"

  redis:
    image: valkey/valkey:8-alpine
    container_name: ${COMPOSE_PROJECT_NAME:-sibyl}-redis
    profiles: ["redis"]
    ports:
      - "${SIBYL_REDIS_PORT:-6381}:6379"
    command: ["valkey-server", "--save", "", "--appendonly", "no"]

The default image is pinned to SurrealDB server v3.1.0 for reproducible local and CI behavior. Override SIBYL_SURREAL_IMAGE when rehearsing a newer server patch. The root compose stores SurrealDB data in a bind mount under .moon/cache/surreal-dev so it survives container churn and is easy to inspect.

Port Mappings

Service	Host Port	Container Port	Purpose
SurrealDB	8000	8000	Default local graph runtime
Redis	6381	6379	Optional coordination backend

Ports are offset from defaults to avoid conflicts with local services.

Moonrepo Commands

bash

# Start databases only
moon run docker-up

# Stop databases
moon run docker-down

# Start recommended Surreal local-dev stack
moon run dev

# Start API + Worker only (no frontend)
moon run dev-api

# Start frontend only
moon run dev-web

# Start Redis worker when SIBYL_COORDINATION_BACKEND=redis
moon run api:worker

# Stop all services
moon run stop

Full Stack Compose

For a complete containerized deployment (backend + frontend + databases), use docker-compose.prod.yml:

bash

# Copy environment file
cp apps/api/.env.example .env

# Edit .env with required secrets:
#   SIBYL_JWT_SECRET=<generate with: openssl rand -hex 32>
#   SIBYL_OPENAI_API_KEY=sk-...

# Generate ~/.sibyl/docker/.env and a pinned compose file
sibyl docker init

# Optional: run a separate worker with Redis/Valkey coordination
sibyl docker init --force --with-worker

# Start, follow logs, and stop
sibyl docker up
sibyl docker logs
sibyl docker down

Production Compose Services

yaml

services:
  backend:
    build:
      context: .
      dockerfile: apps/api/Dockerfile
    ports:
      - "3334:3334"
    environment:
      SIBYL_SURREAL_URL: ws://surrealdb:8000/rpc
      SIBYL_JWT_SECRET: ${SIBYL_JWT_SECRET}
      SIBYL_OPENAI_API_KEY: ${SIBYL_OPENAI_API_KEY}
    depends_on:
      surrealdb:
        condition: service_healthy

  worker:
    profiles: ["redis"]
    build:
      context: .
      dockerfile: apps/api/Dockerfile
    command: ["sibyld", "worker"]
    environment:
      SIBYL_COORDINATION_BACKEND: redis
      SIBYL_REDIS_HOST: redis
      SIBYL_REDIS_PORT: 6379
      SIBYL_SURREAL_URL: ws://surrealdb:8000/rpc
      SIBYL_JWT_SECRET: ${SIBYL_JWT_SECRET}
      SIBYL_OPENAI_API_KEY: ${SIBYL_OPENAI_API_KEY}
    depends_on:
      surrealdb:
        condition: service_healthy
      redis:
        condition: service_started

  redis:
    profiles: ["redis"]
    image: valkey/valkey:8-alpine
    ports:
      - "6381:6379"

  frontend:
    build:
      context: ./apps/web
      dockerfile: Dockerfile
    ports:
      - "3337:3337"
    environment:
      NEXT_PUBLIC_API_URL: ${SIBYL_SERVER_URL:-http://localhost:3334}
      SIBYL_API_URL: http://backend:3334/api
    depends_on:
      backend:
        condition: service_healthy

  surrealdb:
    image: ${SIBYL_SURREAL_IMAGE:-surrealdb/surrealdb:v3.1.0}
    ports:
      - "${SIBYL_SURREAL_PORT:-8000}:8000"
    volumes:
      - surreal_data:/data
    healthcheck:
      test: ["CMD", "/surreal", "is-ready", "--conn", "http://localhost:8000"]

volumes:
  surreal_data:

The production compose persists SurrealDB to a named Docker volume (surreal_data) rather than a bind mount. NEXT_PUBLIC_API_URL is the browser-facing API URL; SIBYL_API_URL is the in-network URL the Next.js server uses for SSR fetches.

Quickstart Compose (Zero-Config)

For individual developers who want a running stack in minutes using pre-built images, use docker-compose.quickstart.yml. No .env file is required; API keys are entered through the web UI during onboarding and stored encrypted in the database.

bash

# Start with pre-built images from ghcr.io
docker compose -f docker-compose.quickstart.yml up -d

# Open http://localhost:3337 and complete onboarding

# Run alongside an existing dev setup on offset ports
SIBYL_SERVER_PORT=3344 SIBYL_WEB_PORT=3347 SIBYL_SURREAL_PORT=8010 \
  docker compose -f docker-compose.quickstart.yml -p sibyl-qs up -d

Differences from the production compose:

Pulls ghcr.io/hyperb1iss/sibyl-api and sibyl-web images instead of building locally
SIBYL_JWT_SECRET and SIBYL_SETTINGS_KEY auto-generate when unset (persisted in the sibyl_secrets volume mounted at /root/.sibyl)
Runs with SIBYL_ENVIRONMENT=development and a sibyl_quickstart default Surreal password
The backend service is named api and the frontend web

Volume Persistence

The root local-dev compose persists SurrealDB data in the bind mount configured by SURREAL_DATA_DIR, defaulting to .moon/cache/surreal-dev.

bash

# Inspect the default local Surreal data directory
ls .moon/cache/surreal-dev

The production and quickstart compose files persist SurrealDB to named Docker volumes (surreal_data and sibyl_surreal):

bash

docker volume ls | grep sibyl

# Remove volumes (DESTROYS DATA)
sibyl docker down --volumes

Connecting to Databases

SurrealDB Health

bash

curl http://localhost:8000/health

Redis CLI

bash

# Only when the redis profile is running
docker exec -it sibyl-redis redis-cli

Migration Archive Rehearsal

The default compose file no longer ships a PostgreSQL sidecar. Historical postgres.sql archive rehearsal must point at an explicitly managed external database, then run the migration command with --restore-database-dump --source-type legacy-archive --target-mode postgres-rehearsal.

Troubleshooting

Port Conflicts

If ports 8000 or 6381 are in use:

bash

# Check what's using the port
lsof -i :8000
lsof -i :6381

# Stop conflicting services or modify docker-compose.yml ports

Database Not Starting

bash

# Check container logs
docker compose logs surrealdb
docker compose logs redis

# Restart with clean state
docker compose down -v
docker compose up -d surrealdb

Connection Refused

Ensure your .env uses the correct ports:

bash

SIBYL_SURREAL_URL=ws://127.0.0.1:8000/rpc
SIBYL_REDIS_PORT=6381        # Only when coordination backend is redis

Next Steps

Environment Variables - Full configuration options
Troubleshooting - Common issues

Docker Compose Deployment ​

Architecture ​

Prerequisites ​

Quick Start ​

Service Definitions ​

Port Mappings ​

Moonrepo Commands ​

Full Stack Compose ​

Production Compose Services ​

Quickstart Compose (Zero-Config) ​

Volume Persistence ​

Connecting to Databases ​

SurrealDB Health ​

Redis CLI ​

Migration Archive Rehearsal ​

Troubleshooting ​

Port Conflicts ​

Database Not Starting ​

Connection Refused ​

Next Steps ​

Docker Compose Deployment

Architecture

Prerequisites

Quick Start

Service Definitions

Port Mappings

Moonrepo Commands

Full Stack Compose

Production Compose Services

Quickstart Compose (Zero-Config)

Volume Persistence

Connecting to Databases

SurrealDB Health

Redis CLI

Migration Archive Rehearsal

Troubleshooting

Port Conflicts

Database Not Starting

Connection Refused

Next Steps