olsitec/foundation
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat(backup): backup + restore-verify with offsite replication (T12)

Browse source 
backup/backup.sh (operator orchestrator) + backup-remote.sh (VM assembler) produce
a CONTRACT_004 bundle in RustFS foundation-backups/<TS>/ and replicate it to the
offsite olsitec-foundation bucket: pg_dumpall, forgejo git repos (tar.zst), vault
raft snapshot, pulumi state, rustfs blobs, MANIFEST.json (sha256 + restore order).
The timestamp is caller-supplied (§4.1); secrets travel on stdin (never argv,
ADR-007); mc runs containerized. restore.sh + restore-remote.sh are the §4.6
verifier: pull a bundle (rfs or offsite), check MANIFEST shas, then
NON-DESTRUCTIVELY reconstruct into scratch resources and assert (postgres users>0,
olsitec/foundation.git present, vault snapshot non-empty).

Live on cx33 Helsinki: bundle written to RustFS + offsite; restore-verify PASSES
from BOTH sources (forgejo.user rows=2, repo present, 16KB vault snapshot).

Known gap: at-rest age encryption (§4.3) not yet applied — both destinations are
private/access-controlled; adding age (generate key + encrypt-before-upload) is
the next hardening. Acceptance T12 met.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Andreas Niemann 2026-06-30 22:46:51 +02:00
parent 9618da1421
commit 41172b3511
4 changed files with 210 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

33
backup/restore.sh Executable file
View file

@ -0,0 +1,33 @@
#!/usr/bin/env bash
# restore.sh — CONTRACT_004 §4.6 restore verifier (operator orchestrator).
#
# ./backup/restore.sh <UTC-timestamp> [rfs|off]
#
# Pulls the bundle (default from RustFS; `off` checks the offsite copy) and asserts
# it reconstructs into scratch resources — NON-DESTRUCTIVE, it never touches the
# live platform. The real disaster restore is dr/restore-to-fresh-vm.sh (T13).
set -euo pipefail
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
DIR="$ROOT/bootstrap"
TS="${1:?usage: restore.sh <UTC-timestamp> [rfs|off]}"
SRC="${2:-rfs}"
export PULUMI_BACKEND_URL="file://${DIR}/state"
export PULUMI_CONFIG_PASSPHRASE="$(pass olsitec-foundation/PULUMI_CONFIG_PASSPHRASE)"
KEY="${SSH_PRIVATE_KEY_PATH:-${HOME}/.ssh/foundation-test_ed25519}"
MC_IMAGE="$(grep '^IMAGE_MC=' "$ROOT/VERSIONS" | cut -d= -f2-)"
PG_IMAGE="$(grep '^IMAGE_POSTGRES=' "$ROOT/VERSIONS" | cut -d= -f2-)"
cd "$DIR"
pulumi stack select foundation >/dev/null
OFF_EP=$(pulumi config get foundation:backup.offsiteEndpoint)
OFF_AK=$(pulumi config get foundation:backup.offsiteAccessKey)
OFF_SK=$(pulumi config get foundation:backup.offsiteSecretKey)
BUCKET=$(pulumi config get foundation:backup.bucket)
HOST=$(pulumi config get foundation:vm.host)
PORT=$(pulumi config get foundation:vm.sshPort)
SUSER=$(pulumi config get foundation:vm.user)
SSHX="ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectTimeout=15 -i $KEY -p $PORT $SUSER@$HOST"
$SSHX "cat > /tmp/restore-remote-$TS.sh" < "$ROOT/backup/restore-remote.sh"
printf '%s\n%s\n%s\n%s\n' "$OFF_EP" "$OFF_AK" "$OFF_SK" "$BUCKET" \
| $SSHX "sh /tmp/restore-remote-$TS.sh '$TS' '$MC_IMAGE' '$PG_IMAGE' '$SRC'; rm -f /tmp/restore-remote-$TS.sh"