Backup and restore

The platform database holds the tenant registry, the billing ledger, the audit trail and every sealed credential. With the bundled database and a backup destination set, Olympusx AI backs it up continuously to an S3-compatible bucket you own, and can restore a new installation to any point inside the retention window. Tenant vector databases are backed up the same way, to a destination set in the console.

What is covered

Data	Backed up by Olympusx AI	How
Platform database: tenants, projects, keys, budgets, usage, statements, audit trail, settings, sealed credentials	Yes, when a destination is set	A base backup on a schedule (03:00 daily by default), plus every write-ahead log segment archived continuously
Tenant vector databases: embeddings and collections	Yes, when a destination is set and the database’s backups are on	The same, per database, each under its own path

You back up the following yourself:

Uploaded documents in object storage. Turn on versioning and lifecycle rules on the bucket.
Model weights in your registry. Replicate the registry.
The platform’s master key. Keep a copy outside the cluster. See below.
A database you run yourself (external mode). Back it up where it lives.

Continuous archiving is what makes point-in-time recovery possible: a restore can stop at any moment inside the retention window, not only at the time of a base backup. The default retention is 30 days. The bundled database runs three instances with automatic failover, which covers a lost node. To undo a mistaken change such as a DELETE, restore a backup.

The master key

Every cluster token, registry password, provider key and bastion key is sealed with the platform’s master key. A restored database is useless without the original key: a new key leaves every sealed credential unreadable, and each must be entered again by hand. Store the master key Secret’s value somewhere that survives the loss of the cluster.

Before you install

A bucket. AWS S3 or an S3-compatible store on your premises. Not one in the same cluster: a backup that dies with the cluster it backs up protects only against a dropped table.

A credential, as a Secret you own, in the installation’s namespace:

kubectl -n <namespace> create secret generic olympus-backup-s3 \
  --from-literal=ACCESS_KEY_ID='<access-key-id>' \
  --from-literal=ACCESS_SECRET_KEY='<secret-access-key>'

Scope the key to that bucket and prefix.

Decide who expires old backups.
- The database expires them after the retention window. The credential then needs delete on the prefix. This is the default.
- The bucket expires them, through its own lifecycle rules, and the credential gets write and list only. A credential that cannot delete cannot be used to destroy the backups, which is the stronger posture. The database logs failures pruning old backups; that noise is the cost.
Not the middle: a retention set while delete is withheld fails quietly, and the bucket grows. Object lock or versioning on the bucket is worth having either way.

Turn on platform database backups

On the Olympus resource, set all of these together. A schedule without a destination does nothing.

spec:
  database:
    mode: managed
    backupDestination: s3://olympus-backups/platform
    # Omit for AWS S3. Any other S3-compatible store needs its endpoint here.
    backupEndpoint: https://s3.example.com
    backupCredentialsSecret: olympus-backup-s3
    # Six fields: the first is SECONDS. A five-field expression is refused.
    backupSchedule: '0 0 3 * * *'
    # archiveName: ""   # the folder this installation owns in the bucket

With the Helm chart, the managed database’s backup values do the same. See Configuration.

An on-premises store without backupEndpoint resolves against AWS S3 and fails to archive while the installation still reports ready. Set it.

Check that it works

Do this after turning it on, and after any change to the bucket, its credential or the platform version.

The archive is moving. The bucket holds a folder for the database with base backups and archived write-ahead log, and new log segments keep arriving.
The database is recoverable. It reports a first recoverability point. Judge by that, not by whether archiving reports success: a database with no backup configuration at all still reports its archiving as working.
A base backup completed after the scheduled time.

Restore

A restore creates a new database from the archive. It never rewrites the existing one in place, which is what you want at 03:00: the damaged database is still there while you work.

Onto a new installation

Set restoreFrom on the Olympus resource of a new installation, beside its own backup settings:

spec:
  database:
    mode: managed
    backupDestination: s3://olympus-backups/platform
    backupCredentialsSecret: olympus-backup-s3
    restoreFrom:
      # The database the archive was written BY: <name>-db, where <name> is
      # the metadata.name of the `Olympus` resource that wrote it.
      sourceServerName: olympus-db
      # destination: s3://other-bucket/platform   # when the archive is elsewhere
      # pointInTime: "2026-08-15T14:31:00Z"

Three things about it:

It is a one-time start-up, not a mode. The database chooses between starting empty and recovering when it is created, and never again. Setting restoreFrom on a running installation changes nothing: use a new installation, a new namespace or a new resource name.
sourceServerName names the source, not the new one. A name the bucket does not hold leaves the database never becoming ready, and the reason is in a job’s log.
The recovered database archives to its own folder, so it never writes over the archive it came from.

Use the original master key in the new installation’s master key Secret, and the same platform version that wrote the backup, or a newer one. A newer platform migrates an older database forward on start; an older platform cannot read a newer database. With an external database (mode: external), restoreFrom is refused: restore that database where it lives.

To a point in time

Someone deleted rows by mistake at 14:32. Set pointInTime to a moment before the damage, such as 14:31, not the moment of it: replay stops there, and everything after it stays gone. A restore with no pointInTime replays to the end of the archive, which brings the damage back too.

After restoring

Confirm the data is there: sign in and open Tenants. A platform that comes up healthy and empty means the restore read the wrong source.
Check System health, and Clusters for every registered cluster reconnecting.
Re-apply any erasure carried out after the restore point. See Erasure and backups.
Confirm the restored installation’s own backups: it is not recoverable itself until its first base backup completes.

Rehearse it

A backup you have never restored is a hypothesis. Run this before the first customer, and again after any platform upgrade or storage change. It takes about twenty minutes.

Note the time, and a fact you can check later, such as the number of tenants.
Create something recognizable: a tenant named dr-drill-<date>.
Wait a few minutes for the write-ahead log to be archived.
Note the time, then do damage on purpose: remove something the drill created.
Restore onto a new installation in another namespace, with pointInTime one minute before step 4.
Check the drill tenant is back and the count matches step 1.
Write down how long steps 5 and 6 took. That is your measured recovery time.
Remove the restored installation.

Tenant vector databases

Each tenant vector database streams its write-ahead log to the bucket and takes a daily base backup, under its own path beneath the destination.

The destination is set by a platform administrator in Settings › Backups, or at installation. See Settings.
Per database, a tenant admin or builder turns Back up this database on or off, sets Keep backups for (days), from 1 to 3,650, and the Daily base backup at (UTC) time, on the database’s page. New databases are on by default. The installation default is 30 days and 03:10 UTC.
The database’s page shows where it archives, the last archived and last failed log segment, its First recoverability point, the last base backup, and in words why archiving is failing when it is.
Turning a database’s backups on later is safe: archiving starts without a restart.
Turning them off stops archiving: the database cannot be restored to any later point. Backups already in the bucket stay, and nothing prunes them while backups are off.

Never point two installations at the same destination: two databases writing to one path overwrite each other’s catalog, and nothing reports it until a restore.

See Vector databases.

Erasure and backups

Erasure deletes a named person’s data from the live databases.

Backups Olympusx AI controls, the platform database archive and the vector database archives, still hold erased data in base backups and archived log taken before the erasure, until those age out of the retention window: 30 days by default. Set the retention to the period your erasure policy allows.
A restore to a point before an erasure brings the erased data back. The erasure’s receipt is in the same database, so it is rolled back too. Keep your own record of each erasure request outside the platform, and run any erasure later than the restore point again after restoring.
Backups you take elsewhere, such as bucket replication, volume snapshots or exports, are yours to handle under your own policy.