olsitec/foundation
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat(backup): backup + restore-verify with offsite replication (T12)

Browse source 
backup/backup.sh (operator orchestrator) + backup-remote.sh (VM assembler) produce
a CONTRACT_004 bundle in RustFS foundation-backups/<TS>/ and replicate it to the
offsite olsitec-foundation bucket: pg_dumpall, forgejo git repos (tar.zst), vault
raft snapshot, pulumi state, rustfs blobs, MANIFEST.json (sha256 + restore order).
The timestamp is caller-supplied (§4.1); secrets travel on stdin (never argv,
ADR-007); mc runs containerized. restore.sh + restore-remote.sh are the §4.6
verifier: pull a bundle (rfs or offsite), check MANIFEST shas, then
NON-DESTRUCTIVELY reconstruct into scratch resources and assert (postgres users>0,
olsitec/foundation.git present, vault snapshot non-empty).

Live on cx33 Helsinki: bundle written to RustFS + offsite; restore-verify PASSES
from BOTH sources (forgejo.user rows=2, repo present, 16KB vault snapshot).

Known gap: at-rest age encryption (§4.3) not yet applied — both destinations are
private/access-controlled; adding age (generate key + encrypt-before-upload) is
the next hardening. Acceptance T12 met.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Andreas Niemann 2026-06-30 22:46:51 +02:00
parent 9618da1421
commit 41172b3511
4 changed files with 210 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

40
backup/backup.sh Executable file
View file

@ -0,0 +1,40 @@
#!/usr/bin/env bash
# backup.sh — CONTRACT_004 backup producer (operator orchestrator).
#
# ./backup/backup.sh [UTC-timestamp]
#
# The timestamp is supplied by the caller (CI/cron) per CONTRACT_004 §4.1; it
# defaults to now for manual runs. The operator contributes the Pulumi state
# (local file backend) and the secrets (from passphrase-encrypted config); the
# heavy lifting runs on the VM via backup-remote.sh. Result: a bundle in RustFS
# foundation-backups/<TS>/ replicated to the offsite bucket.
set -euo pipefail
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
DIR="$ROOT/bootstrap"
TS="${1:-$(date -u +%Y%m%dT%H%M%SZ)}"
export PULUMI_BACKEND_URL="file://${DIR}/state"
export PULUMI_CONFIG_PASSPHRASE="$(pass olsitec-foundation/PULUMI_CONFIG_PASSPHRASE)"
KEY="${SSH_PRIVATE_KEY_PATH:-${HOME}/.ssh/foundation-test_ed25519}"
MC_IMAGE="$(grep '^IMAGE_MC=' "$ROOT/VERSIONS" | cut -d= -f2-)"
cd "$DIR"
pulumi stack select foundation >/dev/null
RT=$(pulumi config get vaultCredentials:rootToken)
OFF_EP=$(pulumi config get foundation:backup.offsiteEndpoint)
OFF_AK=$(pulumi config get foundation:backup.offsiteAccessKey)
OFF_SK=$(pulumi config get foundation:backup.offsiteSecretKey)
BUCKET=$(pulumi config get foundation:backup.bucket)
HOST=$(pulumi config get foundation:vm.host)
PORT=$(pulumi config get foundation:vm.sshPort)
SUSER=$(pulumi config get foundation:vm.user)
SSHX="ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectTimeout=15 -i $KEY -p $PORT $SUSER@$HOST"
W="/tmp/foundation-backup-$TS"
echo "backup: $TS -> rfs/$BUCKET/$TS (+ offsite)"
# Pulumi state + the assembler script onto the VM.
pulumi stack export | $SSHX "mkdir -p $W && cat > $W/pulumi-state.json"
$SSHX "cat > /tmp/backup-remote-$TS.sh" < "$ROOT/backup/backup-remote.sh"
# Run the assembler: secrets on stdin (never argv), TS + MC_IMAGE as args.
printf '%s\n%s\n%s\n%s\n%s\n' "$RT" "$OFF_EP" "$OFF_AK" "$OFF_SK" "$BUCKET" \
| $SSHX "sh /tmp/backup-remote-$TS.sh '$TS' '$MC_IMAGE'; rm -f /tmp/backup-remote-$TS.sh"
echo "backup: done ($TS)"