feat(ci): state-dependent pulumi-preview + backup-verify pipelines (T14)

Completes T14: the two CI pipelines that need Pulumi stack state, which
bootstrap/state/ is gitignored from. Solves the blocker by publishing a
fresh `pulumi stack export` to RustFS after every `up`, then having CI
pull + import it.

- state-publish.sh: ships the stack export to rfs/foundation-ci-state/
  foundation-stack.json via a throwaway mc container on foundation-net
  (ADR-007), exactly like backup.sh. Secrets inside the export stay
  passphrase-encrypted; config travels in the committed (encrypted)
  Pulumi.foundation.yaml. run.sh invokes it best-effort after `up`.
- rustfs.ts + Pulumi.foundation.yaml: declare the foundation-ci-state
  bucket (created belt-and-suspenders by state-publish on first run).
- pulumi-preview.yml (push/PR): read-only drift/PR check. Pulls + imports
  state, materializes the operator key from the SSH_PRIVATE_KEY secret
  (the provider + index.ts read it), `pulumi preview` — never `up`. A diff
  is informational so the job fails only on a program/preview error.
- backup-verify.yml (weekly + dispatch): reuses backup.sh/restore.sh
  unchanged to produce a bundle and restore-verify it from offsite
  (CONTRACT_004 §4.6). Imports real state so the bundle's pulumi-state.json
  is real, not an empty deployment.

Repo-scoped Actions secrets set via the admin API: PULUMI_CONFIG_PASSPHRASE,
SSH_PRIVATE_KEY, RUSTFS_ACCESS_KEY, RUSTFS_SECRET_KEY. Both pipelines
validated end-to-end in a foundation-ci container on the VM (preview exit 0;
backup-verify RESTORE VERIFY PASS from offsite).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

.forgejo/workflows/backup-verify.yml

@@ -0,0 +1,62 @@
+# backup-verify — weekly "a backup is not trusted until restored" (CONTRACT_004
+# §4.6, T14 state-dependent half). Produces a fresh bundle and asserts it restores
+# from the OFFSITE copy into scratch resources (NON-DESTRUCTIVE — restore.sh never
+# touches the live platform). Runs on the foundation's own runner in foundation-ci.
+#
+# It reuses the operator scripts UNCHANGED (backup/backup.sh + restore.sh), which
+# read everything from `pulumi config get` (vm coords, vault root token, offsite +
+# age creds — all in the committed, passphrase-encrypted Pulumi.foundation.yaml) and
+# orchestrate the heavy lifting on the VM over SSH (ADR-007). Two CI-specific needs:
+#   1. the operator SSH key (SSH_PRIVATE_KEY secret → /tmp/op_key) to reach the VM;
+#   2. real Pulumi STATE imported into a file backend, because backup.sh embeds a
+#      `pulumi stack export` (pulumi-state.json, CONTRACT_004 §4.2) in the bundle —
+#      a bare `stack init` would ship an EMPTY deployment. State is pulled from the
+#      object run.sh publishes (state-publish.sh), same as pulumi-preview.
+#
+# Schedule only (+ manual dispatch): never on push — it creates real backup bundles.
+name: backup-verify
+on:
+  schedule:
+    - cron: "17 3 * * 0" # weekly, Sunday 03:17 UTC
+  workflow_dispatch:
+
+jobs:
+  backup-verify:
+    runs-on: docker
+    container:
+      image: foundation-ci:latest
+    env:
+      PULUMI_CONFIG_PASSPHRASE: ${{ secrets.PULUMI_CONFIG_PASSPHRASE }}
+      SSH_PRIVATE_KEY_PATH: /tmp/op_key
+      PULUMI_SKIP_UPDATE_CHECK: "true"
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Materialize operator SSH key (reaches the VM over SSH)
+        env:
+          SSH_PRIVATE_KEY: ${{ secrets.SSH_PRIVATE_KEY }}
+        run: |
+          install -m 600 /dev/null /tmp/op_key
+          printf '%s' "$SSH_PRIVATE_KEY" > /tmp/op_key
+          [ -z "$(tail -c1 /tmp/op_key)" ] || printf '\n' >> /tmp/op_key
+
+      - name: Import Pulumi state (so the bundle's pulumi-state.json is real)
+        env:
+          RUSTFS_ACCESS_KEY: ${{ secrets.RUSTFS_ACCESS_KEY }}
+          RUSTFS_SECRET_KEY: ${{ secrets.RUSTFS_SECRET_KEY }}
+        working-directory: bootstrap
+        run: |
+          mc alias set rfs http://foundation-rustfs:9000 "$RUSTFS_ACCESS_KEY" "$RUSTFS_SECRET_KEY" >/dev/null
+          mc cp rfs/foundation-ci-state/foundation-stack.json /tmp/foundation-stack.json
+          mkdir -p state
+          export PULUMI_BACKEND_URL="file://$(pwd)/state"
+          pulumi stack select foundation 2>/dev/null || pulumi stack init foundation
+          pulumi stack import --file /tmp/foundation-stack.json
+
+      - name: Backup, then restore-verify from offsite (CONTRACT_004 §4.6)
+        run: |
+          TS=$(date -u +%Y%m%dT%H%M%SZ)
+          echo "backup-verify: bundle timestamp $TS"
+          ./backup/backup.sh "$TS"
+          ./backup/restore.sh "$TS" off
+          echo "backup-verify: OK ($TS restored from offsite)"

.forgejo/workflows/pulumi-preview.yml

@@ -0,0 +1,64 @@
+# pulumi-preview — read-only infra drift / PR check (T14, state-dependent half).
+#
+# Runs on the foundation's OWN runner in the baked foundation-ci image. Unlike
+# ci.yml (self-contained), this needs Pulumi STATE, which bootstrap/state/ is
+# gitignored from. It pulls the latest `pulumi stack export` that run.sh publishes
+# to rfs/foundation-ci-state/foundation-stack.json on every `up` (state-publish.sh),
+# imports it into a throwaway file backend, and previews. CONFIG + encrypted
+# secrets come from the committed Pulumi.foundation.yaml (the passphrase decrypts).
+#
+# READ-ONLY: `pulumi preview` only — NEVER `up`. It dials the VM docker over SSH
+# the same way the operator does (ADR-007), so the operator key is materialized
+# from the SSH_PRIVATE_KEY secret. A diff is informational (expected on infra PRs),
+# so the job does NOT fail on changes — it fails only if the program/preview errors.
+#
+# Secrets (repo-scoped, set via the admin API): PULUMI_CONFIG_PASSPHRASE,
+# SSH_PRIVATE_KEY (operator ed25519), RUSTFS_ACCESS_KEY/RUSTFS_SECRET_KEY (scoped
+# RustFS service account, to fetch the state object over foundation-net).
+name: pulumi-preview
+on:
+  push:
+  pull_request:
+  workflow_dispatch:
+
+jobs:
+  preview:
+    runs-on: docker
+    container:
+      image: foundation-ci:latest
+    env:
+      PULUMI_CONFIG_PASSPHRASE: ${{ secrets.PULUMI_CONFIG_PASSPHRASE }}
+      SSH_PRIVATE_KEY_PATH: /tmp/op_key
+      PULUMI_SKIP_UPDATE_CHECK: "true"
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Install workspace deps (bun workspace, root lockfile)
+        run: bun install --frozen-lockfile || bun install
+
+      - name: Materialize operator SSH key (provider + index.ts read it)
+        env:
+          SSH_PRIVATE_KEY: ${{ secrets.SSH_PRIVATE_KEY }}
+        run: |
+          install -m 600 /dev/null /tmp/op_key
+          printf '%s' "$SSH_PRIVATE_KEY" > /tmp/op_key
+          # OpenSSH keys must end in a newline; add one only if missing.
+          [ -z "$(tail -c1 /tmp/op_key)" ] || printf '\n' >> /tmp/op_key
+          ssh-keygen -y -f /tmp/op_key > /tmp/op_key.pub
+
+      - name: Fetch Pulumi state from RustFS
+        env:
+          RUSTFS_ACCESS_KEY: ${{ secrets.RUSTFS_ACCESS_KEY }}
+          RUSTFS_SECRET_KEY: ${{ secrets.RUSTFS_SECRET_KEY }}
+        run: |
+          mc alias set rfs http://foundation-rustfs:9000 "$RUSTFS_ACCESS_KEY" "$RUSTFS_SECRET_KEY" >/dev/null
+          mc cp rfs/foundation-ci-state/foundation-stack.json /tmp/foundation-stack.json
+
+      - name: Pulumi preview (read-only — never up)
+        working-directory: bootstrap
+        run: |
+          mkdir -p state   # bootstrap/state/ is gitignored — create the file backend dir
+          export PULUMI_BACKEND_URL="file://$(pwd)/state"
+          pulumi stack select foundation 2>/dev/null || pulumi stack init foundation
+          pulumi stack import --file /tmp/foundation-stack.json
+          pulumi preview --non-interactive --diff