feat(backup): backup + restore-verify with offsite replication (T12)
backup/backup.sh (operator orchestrator) + backup-remote.sh (VM assembler) produce
a CONTRACT_004 bundle in RustFS foundation-backups/<TS>/ and replicate it to the
offsite olsitec-foundation bucket: pg_dumpall, forgejo git repos (tar.zst), vault
raft snapshot, pulumi state, rustfs blobs, MANIFEST.json (sha256 + restore order).
The timestamp is caller-supplied (§4.1); secrets travel on stdin (never argv,
ADR-007); mc runs containerized. restore.sh + restore-remote.sh are the §4.6
verifier: pull a bundle (rfs or offsite), check MANIFEST shas, then
NON-DESTRUCTIVELY reconstruct into scratch resources and assert (postgres users>0,
olsitec/foundation.git present, vault snapshot non-empty).
Live on cx33 Helsinki: bundle written to RustFS + offsite; restore-verify PASSES
from BOTH sources (forgejo.user rows=2, repo present, 16KB vault snapshot).
Known gap: at-rest age encryption (§4.3) not yet applied — both destinations are
private/access-controlled; adding age (generate key + encrypt-before-upload) is
the next hardening. Acceptance T12 met.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 22:46:51 +02:00
|
|
|
#!/usr/bin/env bash
|
|
|
|
|
# backup.sh — CONTRACT_004 backup producer (operator orchestrator).
|
|
|
|
|
#
|
|
|
|
|
# ./backup/backup.sh [UTC-timestamp]
|
|
|
|
|
#
|
|
|
|
|
# The timestamp is supplied by the caller (CI/cron) per CONTRACT_004 §4.1; it
|
|
|
|
|
# defaults to now for manual runs. The operator contributes the Pulumi state
|
|
|
|
|
# (local file backend) and the secrets (from passphrase-encrypted config); the
|
|
|
|
|
# heavy lifting runs on the VM via backup-remote.sh. Result: a bundle in RustFS
|
|
|
|
|
# foundation-backups/<TS>/ replicated to the offsite bucket.
|
|
|
|
|
set -euo pipefail
|
|
|
|
|
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
|
|
|
|
|
DIR="$ROOT/bootstrap"
|
|
|
|
|
TS="${1:-$(date -u +%Y%m%dT%H%M%SZ)}"
|
|
|
|
|
export PULUMI_BACKEND_URL="file://${DIR}/state"
|
2026-07-01 00:15:01 +02:00
|
|
|
export PULUMI_CONFIG_PASSPHRASE="${PULUMI_CONFIG_PASSPHRASE:-$(pass olsitec-foundation/PULUMI_CONFIG_PASSPHRASE)}"
|
feat(backup): backup + restore-verify with offsite replication (T12)
backup/backup.sh (operator orchestrator) + backup-remote.sh (VM assembler) produce
a CONTRACT_004 bundle in RustFS foundation-backups/<TS>/ and replicate it to the
offsite olsitec-foundation bucket: pg_dumpall, forgejo git repos (tar.zst), vault
raft snapshot, pulumi state, rustfs blobs, MANIFEST.json (sha256 + restore order).
The timestamp is caller-supplied (§4.1); secrets travel on stdin (never argv,
ADR-007); mc runs containerized. restore.sh + restore-remote.sh are the §4.6
verifier: pull a bundle (rfs or offsite), check MANIFEST shas, then
NON-DESTRUCTIVELY reconstruct into scratch resources and assert (postgres users>0,
olsitec/foundation.git present, vault snapshot non-empty).
Live on cx33 Helsinki: bundle written to RustFS + offsite; restore-verify PASSES
from BOTH sources (forgejo.user rows=2, repo present, 16KB vault snapshot).
Known gap: at-rest age encryption (§4.3) not yet applied — both destinations are
private/access-controlled; adding age (generate key + encrypt-before-upload) is
the next hardening. Acceptance T12 met.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 22:46:51 +02:00
|
|
|
KEY="${SSH_PRIVATE_KEY_PATH:-${HOME}/.ssh/foundation-test_ed25519}"
|
|
|
|
|
MC_IMAGE="$(grep '^IMAGE_MC=' "$ROOT/VERSIONS" | cut -d= -f2-)"
|
|
|
|
|
cd "$DIR"
|
|
|
|
|
pulumi stack select foundation >/dev/null
|
|
|
|
|
|
|
|
|
|
RT=$(pulumi config get vaultCredentials:rootToken)
|
|
|
|
|
OFF_EP=$(pulumi config get foundation:backup.offsiteEndpoint)
|
|
|
|
|
OFF_AK=$(pulumi config get foundation:backup.offsiteAccessKey)
|
|
|
|
|
OFF_SK=$(pulumi config get foundation:backup.offsiteSecretKey)
|
|
|
|
|
BUCKET=$(pulumi config get foundation:backup.bucket)
|
2026-06-30 23:23:38 +02:00
|
|
|
AGE_RECIPIENT=$(pulumi config get foundation:backup.ageRecipient) # public; CONTRACT_004 §4.3
|
feat(backup): backup + restore-verify with offsite replication (T12)
backup/backup.sh (operator orchestrator) + backup-remote.sh (VM assembler) produce
a CONTRACT_004 bundle in RustFS foundation-backups/<TS>/ and replicate it to the
offsite olsitec-foundation bucket: pg_dumpall, forgejo git repos (tar.zst), vault
raft snapshot, pulumi state, rustfs blobs, MANIFEST.json (sha256 + restore order).
The timestamp is caller-supplied (§4.1); secrets travel on stdin (never argv,
ADR-007); mc runs containerized. restore.sh + restore-remote.sh are the §4.6
verifier: pull a bundle (rfs or offsite), check MANIFEST shas, then
NON-DESTRUCTIVELY reconstruct into scratch resources and assert (postgres users>0,
olsitec/foundation.git present, vault snapshot non-empty).
Live on cx33 Helsinki: bundle written to RustFS + offsite; restore-verify PASSES
from BOTH sources (forgejo.user rows=2, repo present, 16KB vault snapshot).
Known gap: at-rest age encryption (§4.3) not yet applied — both destinations are
private/access-controlled; adding age (generate key + encrypt-before-upload) is
the next hardening. Acceptance T12 met.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 22:46:51 +02:00
|
|
|
HOST=$(pulumi config get foundation:vm.host)
|
|
|
|
|
PORT=$(pulumi config get foundation:vm.sshPort)
|
|
|
|
|
SUSER=$(pulumi config get foundation:vm.user)
|
|
|
|
|
SSHX="ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectTimeout=15 -i $KEY -p $PORT $SUSER@$HOST"
|
|
|
|
|
W="/tmp/foundation-backup-$TS"
|
|
|
|
|
|
|
|
|
|
echo "backup: $TS -> rfs/$BUCKET/$TS (+ offsite)"
|
|
|
|
|
# Pulumi state + the assembler script onto the VM.
|
|
|
|
|
pulumi stack export | $SSHX "mkdir -p $W && cat > $W/pulumi-state.json"
|
|
|
|
|
$SSHX "cat > /tmp/backup-remote-$TS.sh" < "$ROOT/backup/backup-remote.sh"
|
2026-06-30 23:23:38 +02:00
|
|
|
# Run the assembler: secrets on stdin (never argv); TS, MC_IMAGE, age recipient as args.
|
feat(backup): backup + restore-verify with offsite replication (T12)
backup/backup.sh (operator orchestrator) + backup-remote.sh (VM assembler) produce
a CONTRACT_004 bundle in RustFS foundation-backups/<TS>/ and replicate it to the
offsite olsitec-foundation bucket: pg_dumpall, forgejo git repos (tar.zst), vault
raft snapshot, pulumi state, rustfs blobs, MANIFEST.json (sha256 + restore order).
The timestamp is caller-supplied (§4.1); secrets travel on stdin (never argv,
ADR-007); mc runs containerized. restore.sh + restore-remote.sh are the §4.6
verifier: pull a bundle (rfs or offsite), check MANIFEST shas, then
NON-DESTRUCTIVELY reconstruct into scratch resources and assert (postgres users>0,
olsitec/foundation.git present, vault snapshot non-empty).
Live on cx33 Helsinki: bundle written to RustFS + offsite; restore-verify PASSES
from BOTH sources (forgejo.user rows=2, repo present, 16KB vault snapshot).
Known gap: at-rest age encryption (§4.3) not yet applied — both destinations are
private/access-controlled; adding age (generate key + encrypt-before-upload) is
the next hardening. Acceptance T12 met.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 22:46:51 +02:00
|
|
|
printf '%s\n%s\n%s\n%s\n%s\n' "$RT" "$OFF_EP" "$OFF_AK" "$OFF_SK" "$BUCKET" \
|
2026-06-30 23:23:38 +02:00
|
|
|
| $SSHX "sh /tmp/backup-remote-$TS.sh '$TS' '$MC_IMAGE' '$AGE_RECIPIENT'; rm -f /tmp/backup-remote-$TS.sh"
|
feat(backup): backup + restore-verify with offsite replication (T12)
backup/backup.sh (operator orchestrator) + backup-remote.sh (VM assembler) produce
a CONTRACT_004 bundle in RustFS foundation-backups/<TS>/ and replicate it to the
offsite olsitec-foundation bucket: pg_dumpall, forgejo git repos (tar.zst), vault
raft snapshot, pulumi state, rustfs blobs, MANIFEST.json (sha256 + restore order).
The timestamp is caller-supplied (§4.1); secrets travel on stdin (never argv,
ADR-007); mc runs containerized. restore.sh + restore-remote.sh are the §4.6
verifier: pull a bundle (rfs or offsite), check MANIFEST shas, then
NON-DESTRUCTIVELY reconstruct into scratch resources and assert (postgres users>0,
olsitec/foundation.git present, vault snapshot non-empty).
Live on cx33 Helsinki: bundle written to RustFS + offsite; restore-verify PASSES
from BOTH sources (forgejo.user rows=2, repo present, 16KB vault snapshot).
Known gap: at-rest age encryption (§4.3) not yet applied — both destinations are
private/access-controlled; adding age (generate key + encrypt-before-upload) is
the next hardening. Acceptance T12 met.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 22:46:51 +02:00
|
|
|
echo "backup: done ($TS)"
|