Back up and restore
Recover canonical database state and immutable artifact bytes, then reconcile workflows and providers.
Outcome
A named recovery point restores semantic state and verifies every referenced immutable artifact before work resumes.
Prerequisites
Define the recovery-point and recovery-time targets, database backup/PITR policy, independent logical export, object replication policy, encryption keys, and a clean restore environment.
Steps
- Stop new commands and schedules while preserving read access where safe.
- Restore PostgreSQL to an isolated environment from the selected recovery point.
- Restore or mount the immutable artifact namespace and run the database/object consistency scan.
- Validate migrations, tenant constraints, event order, and sampled evidence digests.
- Reconcile Temporal open work and external provider state against PostgreSQL intent.
- Resume a synthetic run before reopening real work.
The committed PostgreSQL gate exercises one logical backup against loopback PostgreSQL. Set
LUMEN_BACKUP_SOURCE_URL and LUMEN_BACKUP_ADMIN_URL in the command's process environment, then use a
fresh disposable database name with the required prefix:
scripts/verify-postgres-backup-restore --restore-database lumen_restore_release_gateThe gate dumps the public schema without ownership or privileges, restores it into the database it
just created, compares public table counts, runs Alembic upgrade and drift checks, emits a redacted JSON
report, and drops only that created database. It copies referenced auth.users IDs into a minimal
restore fixture so tenant foreign keys remain enforceable. It does not exercise a managed Supabase
backup or PITR restore.
Process restart, project archives, and this local logical gate are intentionally separate proofs. A production release still requires saved rehearsals against the selected PostgreSQL backup/PITR service and object-store recovery path. No object replication policy, retention setting, or architecture claim is evidence that restored bytes exist.
After restoring the database and object namespace, use the exact workspace-scoped scanner documented in Verify artifacts. A healthy exit proves that the selected restored database manifests resolve to verified bytes; it does not prove replication freshness or replace the database restore rehearsal.
Verify
The local logical gate exits 0 with matching public table counts and migration heads. The artifact
scanner exits 0 only when the object for every canonical artifact manifest in the selected workspace
streams through digest and size verification. For a production rehearsal, an exported project also
opens from the restored environment, worker readiness returns only after reconciliation succeeds, and
no workflow result mutates a stale generation.
Recover
If reconciliation finds missing bytes or ambiguous external operations, keep affected evidence unavailable and work paused. Restore from another replica or re-create only from a proven source; never invent a successful manifest.