The backup tarred only /data/git, but CONTRACT_004 §4.2 requires the git
repos AND app.ini AND the host SSH keys — without app.ini a restored Forgejo
has no DB/S3 config and won't start. Discovered during the T13 DR rehearsal:
restore reached Forgejo and it had nothing to configure from.
Tar the whole /data volume (git/, gitea/conf/app.ini, ssh/ssh_host_*). It is
~1 MB at Layer 0 — the DB and LFS/packages are externalised to Postgres +
RustFS, so /data holds no large recreatable state. Restored end-to-end on a
fresh VM: Forgejo comes up fully configured against the restored PG + RustFS.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Close the known gap: backup bundles were uploaded unencrypted, relying
solely on destination access control. Now every data artifact is
age-encrypted on the VM before upload and decrypted on restore.
- backup-remote.sh: assemble rustfs blobs into rustfs-blobs.tar.zst (so the
whole bundle is one encrypted unit), then age -r <recipient> each artifact
to <name>.age and drop the plaintext. MANIFEST.json stays cleartext — it
is the inventory + integrity gate and carries no secrets; it records each
artifact's PLAINTEXT sha256 so restore verifies after decrypt.
- restore-remote.sh: materialise the age identity to a 0600 file, decrypt
each .age, then run the existing sha + scratch-restore asserts; add a
rustfs-blobs extract+assert.
- backup.sh / restore.sh: pass the public recipient (arg) / secret identity
(stdin, never argv) from passphrase-encrypted config.
- provision/index.ts: install age + zstd on the VM via cloud-init so a fresh
DR VM (T13) has the backup tools from first boot.
- Pulumi.foundation.yaml: seed backup.ageRecipient (public) + backup.ageIdentity
(secure:). The identity lives in config so {repo + passphrase} can decrypt a
bundle even after total Vault loss (CONTRACT_004 §4.3).
Validated live: encrypted backup + restore-verify PASS from both RustFS and
offsite; bucket shows only *.age + cleartext MANIFEST.json.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
backup/backup.sh (operator orchestrator) + backup-remote.sh (VM assembler) produce
a CONTRACT_004 bundle in RustFS foundation-backups/<TS>/ and replicate it to the
offsite olsitec-foundation bucket: pg_dumpall, forgejo git repos (tar.zst), vault
raft snapshot, pulumi state, rustfs blobs, MANIFEST.json (sha256 + restore order).
The timestamp is caller-supplied (§4.1); secrets travel on stdin (never argv,
ADR-007); mc runs containerized. restore.sh + restore-remote.sh are the §4.6
verifier: pull a bundle (rfs or offsite), check MANIFEST shas, then
NON-DESTRUCTIVELY reconstruct into scratch resources and assert (postgres users>0,
olsitec/foundation.git present, vault snapshot non-empty).
Live on cx33 Helsinki: bundle written to RustFS + offsite; restore-verify PASSES
from BOTH sources (forgejo.user rows=2, repo present, 16KB vault snapshot).
Known gap: at-rest age encryption (§4.3) not yet applied — both destinations are
private/access-controlled; adding age (generate key + encrypt-before-upload) is
the next hardening. Acceptance T12 met.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Repo topology, baseline overlay, planning docs (PLAN-001/002), ADR-004/005,
and the bootstrap/packages/documentation skeleton. Implementation (T00+) not started.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
