Sibyl Phase 3 - Relational Sidecar Burn-Down

Source of truth: docs/research/rust-port/INVENTORY.md, generated by tools/inventory/runtime_surface.py. Do not hand-edit the inventory; update this plan when the generated inventory changes.

Phase 3 is open as of v0.6.0, the first compatibility release with SurrealDB auth and graph runtime as the default path.

---

Goal

Remove the remaining FalkorDB/PostgreSQL runtime dependencies without breaking the SurrealDB-first development path, archive-backed migration path, or rollback story.

Phase 3 is done when:

• Fully Surreal mode has no active SQLModel, asyncpg, Alembic, pgvector, PostgreSQL service, or FalkorDB dependency.
• Legacy auth/RBAC code is gone after the compatibility release.
• Remaining content, settings, backup, crawler, raw-capture, and RAG paths either run on SurrealDB or are explicitly deleted.
• docs/research/rust-port/INVENTORY.md shows no active relational runtime consumers outside historical migrations and archive readers retained by policy.

---

Current Inventory Snapshot

From the generated inventory:

• 0 SQLModel tables remain in the codebase.
• 0 files contain raw SQL query usage.
• 0 files show session-backed storage access outside direct query usage.
• No legacy transition dependencies remain in apps/api/pyproject.toml.
• graphiti-core[anthropic,google-genai] remains as the active Graphiti integration; the FalkorDB extra and Python client are gone.

This inventory now tracks the remaining active runtime surface, plus retained archive rehearsal paths that still need explicit policy decisions.

---

Lanes

Lane 0 - Release Gate Evidence

Status: satisfied for opening Phase 3 by v0.6.0. Keep adding evidence when the same checks find new migration or rollback gaps.

• Run a live rehearsal against a production-like legacy archive and a fresh Surreal runtime.
• Run the auth-flow replay against the Surreal-backed API.
• Validate local migration from a real single-org legacy install.
• Validate local migration from a real multi-org legacy install.
• Keep release notes current with the results and any deviations.

Use docs/architecture/SURREALDB_PHASE2_LIVE_GATES.md to execute and capture evidence for these gates.

Lane 1 - Delete Legacy Auth/RBAC

Status: complete after v0.6.0.

Start after one compatibility release on SIBYL_AUTH_STORE=surreal.

Delete or collapse:

• apps/api/src/sibyl/persistence/legacy/auth.py
• apps/api/src/sibyl/persistence/legacy/auth_runtime.py
• apps/api/src/sibyl/persistence/legacy/auth_managers/*
• apps/api/src/sibyl/persistence/legacy/orgs.py
• apps/api/src/sibyl/persistence/legacy/org_members.py
• apps/api/src/sibyl/persistence/legacy/org_invitations.py
• apps/api/src/sibyl/persistence/legacy/project_members.py
• The postgres branch in apps/api/src/sibyl/persistence/auth_runtime.py
• The postgres branch in apps/api/src/sibyl/persistence/organization_runtime.py

Shared enums/value objects now live outside the deleted SQLModel schema module.

Verification:

• Auth route tests pass with only Surreal auth enabled.
• auth-flow passes against the Surreal API.
• Inventory no longer lists legacy auth/RBAC modules as active SQL users.

Completed evidence:

• Legacy auth/RBAC adapters, setup/user persistence, and route tests were removed.
• moon run api:test passed with 1689 passed, 7 skipped.
• GitHub Actions CI run 25653206701 passed with SurrealDB as the only database service.

Lane 2 - Prove and Retire Content Sidecars

Status: complete for active runtime and Surreal archive export/restore.

Content has Surreal runtime and archive paths. Phase 3 needs proof before deletion.

Decide each legacy module:

• apps/api/src/sibyl/persistence/legacy/crawler.py
• apps/api/src/sibyl/persistence/legacy/entities.py
• apps/api/src/sibyl/persistence/legacy/rag.py
• apps/api/src/sibyl/persistence/content_archive.py

apps/api/src/sibyl/persistence/content_archive.py is now Surreal-only for structured content.json export and restore. Retained postgres.sql payloads are restored only by explicit historical migration commands that name the archive source and target mode before any write.

Verification:

• Crawler, raw capture, document search, RAG, memory, and session routes pass in fully Surreal mode.
• A real legacy content archive imports into Surreal content and verifies counts.
• Inventory no longer lists active legacy content modules once compatibility support ends.

Completed evidence:

• Content runtime dispatch now resolves directly to the Surreal implementation.
• Legacy crawler, document entity, and RAG persistence modules were deleted.
• Live relational content archive export was removed; structured archive export now reads Surreal.
• Focused content/RAG route tests passed with 32 passed.
• Runtime inventory now reports 0 raw SQL query usage files and 0 session-backed storage access files.

Lane 3 - Prove and Retire Settings and Backup Sidecars

Status: complete for active runtime and Surreal backup/archive records.

Settings and backups already dispatch by runtime. Phase 3 decides when the legacy branches are no longer shipped.

Decide each legacy module:

• apps/api/src/sibyl/persistence/legacy/settings.py
• apps/api/src/sibyl/persistence/legacy/system_settings.py
• apps/api/src/sibyl/persistence/legacy/backups.py

apps/api/src/sibyl/persistence/backups_common.py now serves shared Surreal backup DTOs and runtime options, not legacy persistence dispatch. New backups include structured Surreal auth/content snapshots; postgres.sql is not produced in fully Surreal mode.

Verification:

• Settings routes pass with SIBYL_STORE=surreal.
• Backup create/list/delete flows pass with Surreal content/auth snapshots.
• Backup archives no longer require postgres.sql unless the release policy explicitly keeps it.

Completed evidence:

• Settings and backup runtime dispatch now resolves directly to Surreal implementations.
• Legacy settings, system settings, and backup persistence modules were deleted.
• Live relational auth/content archive export was removed from backup snapshots.
• Focused API tests passed with 53 passed.
• Runtime inventory now reports 0 raw SQL query usage files and 0 session-backed storage access files.

Lane 4 - Remove Ambient Relational Infrastructure

Status: complete for active runtime. Historical Alembic revisions remain as archive policy artifacts only.

Do this only after lanes 1-3 remove active consumers.

Delete or retire:

• apps/api/src/sibyl/db/connection.py (deleted)
• apps/api/src/sibyl/db/project_sync.py (deleted)
• apps/api/src/sibyl/db/sync.py (deleted)
• SQLModel schemas, Alembic runtime config, and direct sibyld db PostgreSQL commands (deleted)

Preserve historical Alembic revisions as history. Do not rewrite old migrations.

Verification:

• Startup never initializes PostgreSQL in fully Surreal mode.
• No active route imports sibyl.persistence.legacy.session or sibyl.auth.rls.
• Inventory shows zero session-backed storage access files.

Completed evidence:

• settings.requires_relational_support now resolves to false for all runtime shapes.
• API and combined app startup no longer import or call relational sidecar bootstrap.
• Legacy relational session, RLS, and sidecar startup modules were deleted.
• sibyld up no longer starts PostgreSQL for SIBYL_STORE=legacy.
• Migration-era graph-to-Postgres project sync helpers and CLI commands were deleted.
• Live relational auth/content archive export was removed; JSON archive export now reads Surreal.
• Legacy async PostgreSQL session helpers were deleted.
• SQLModel schemas, Alembic runtime config, active Alembic/asyncpg/pgvector/sqlmodel dependencies, and direct sibyld db PostgreSQL commands were deleted.
• Runtime inventory now reports 0 SQLModel tables, 0 raw SQL query usage files, 0 session-backed storage access files, and no legacy transition dependencies.

Archive policy update:

• sibyld migrate import, rehearse, and cutover require explicit --source-type and --target-mode flags.
• --source-type surreal-archive --target-mode surreal is the Surreal-native restore path.
• Restoring retained postgres.sql payloads requires --restore-database-dump --source-type legacy-archive --target-mode postgres-rehearsal.
• Import and rehearsal commands support --dry-run restore review before writes.
• Unsupported or ignored archive payloads are reported before restore writes begin.

Lane 5 - Remove Legacy Graph and Dependencies

Status: complete for active runtime and deployment surfaces. Migration sidecars remain available only for archive rehearsal until the graph export/import policy is closed.

Do this after graph archive restore, metrics, search, relationship, and admin surfaces are proven on SurrealDB with live data.

Delete or retire:

• FalkorDB driver setup and compose profile.
• Legacy graph query adapters that exist only for Falkor.
• graphiti-core[falkordb,...] extras once the Surreal driver no longer needs them.

Verification:

• Graph restore, search, entity CRUD, relationship traversal, metrics, and MCP tools pass in fully Surreal mode.
• Inventory no longer lists Falkor-only graph runtime code as active.

Completed evidence:

• moon run dev-legacy was removed from the root moon tasks.
• sibyld up now coerces SIBYL_STORE=legacy to SurrealDB instead of starting FalkorDB.
• The root Docker Compose PostgreSQL service moved behind the migration profile so docker compose up -d starts only SurrealDB by default.
• GraphClient no longer imports or constructs the FalkorDB driver; it always connects through the SurrealDB driver.
• The FalkorDB Graphiti search adapter and package extra were removed from sibyl-core.
• The local FalkorDB compose service, app/core Falkor settings, setup service check, and live local migration wrapper were removed. Local recovery now uses archive import.
• The Helm chart no longer renders PostgreSQL auth migration jobs, FalkorDB secrets, or FalkorDB environment, and local Kubernetes infra no longer carries FalkorDB or CNPG manifests.
• Active guides now describe FalkorDB and PostgreSQL only as migration/archive source surfaces.

Lane 6 - Exit Graphiti from Default Memory Loops

Status: complete for the v0.7 default loop. Retained Graphiti imports are named compatibility, admin, migration, or test-only surfaces.

Graphiti remains as a transition dependency on top of the SurrealDB driver. Its remaining work is not FalkorDB removal; it is default-loop removal for native memory. The generated runtime inventory lists every active graphiti_core import, while the hand-authored docs/architecture/SURREALDB_GRAPHITI_EXIT_INVENTORY.md classifies each call site by behavior, default-loop usage, status, removal condition, owner, and verification command. The executable v0.7 cleanup sequence lives in docs/architecture/SURREALDB_V07_GRAPHITI_EXIT_AND_PURE_SURREAL_PLAN.md.

Verification:

• moon run inventory-check verifies both the generated inventory snapshot and hand-authored Graphiti exit coverage.
• moon run core:test keeps the native retrieval and native write contract tests green.
• moon run core:no-graphiti-smoke blocks graphiti_core imports while exercising remember, recall, context, wake, explore, temporal, health, manage, link_graph, reflect, default MCP imports, API job imports, crawler graph integration imports, and prompt-hook imports.

Completed evidence:

• Default native writes are enabled unless SIBYL_NATIVE_WRITE=disabled is set explicitly.
• The no-Graphiti smoke test passes with 2 passed.
• The generated inventory covers every remaining graphiti_core import and the hand-authored inventory classifies them with owners and removal conditions.
• graphiti-core is isolated to the sibyl-core[compatibility] extra and dev/test dependencies, not the default runtime dependency.

Exit criteria:

• No default remember, recall, context, wake, or reflect path constructs Graphiti.
• Remaining Graphiti imports are named compatibility, migration, or test-only surfaces with removal owners and conditions.
• Projectable pre-v0.7 Episodic and Entity records are documented for native projection, and unprojectable records are excluded rather than treated as public memory.

---

v0.8 A0 Baseline Receipt

Recorded on 2026-05-13 at local commit 1de0b408.

• moon run inventory-check inventory-typecheck inventory-test: generated inventory current, Graphiti exit inventory covers 21 import files, inventory typecheck passed, and 14 inventory tests passed.
• moon run core:no-graphiti-smoke: 2 passed.
• moon run :check: 33 tasks completed, including 5 executed tasks and 28 cache hits. Core reported 1327 passed and 15 skipped; API reported 1639 passed and 1 skipped; CLI reported 156 passed; web reported 88 passed.
• graphiti-core remains optional through sibyl-core[compatibility] and the sibyl-core dev dependency group, not default runtime dependencies.
• Remote CI receipts are green for origin/main at d2d3d926: CI run 25801942331, docs deploy run 25801942466, and scheduled nightly run 25791871706. Local main is ahead of origin/main, so these remote receipts do not cover local commits.

---

Dependency Removal Order

1. Remove auth/RBAC legacy code.
2. Remove legacy content/settings/backup branches after archive policy is settled.
3. Finish PostgreSQL archive and migration policy.
4. Remove FalkorDB compose/runtime support and Graphiti Falkor extras.

Each dependency removal requires a fresh inventory run and targeted route tests for the surface it used to support.

---

Completion Checklist

• moon run inventory updates docs/research/rust-port/INVENTORY.md.
• moon run inventory-check inventory-typecheck inventory-test passes.
• moon run api:check core:check cli:check web:check passes for touched projects.
• Live rehearsal evidence is appended to the Phase 2 migration plan or release notes.
• Archive rollback policy is explicit: Surreal-native restore uses surreal-archive to surreal, while retained postgres.sql support is historical legacy-archive to postgres-rehearsal evidence.