feat(dr): kv-migrate.sh — logical KV export/import (Vault<->OpenBao); fix PLAN-004

The raft SNAPSHOT isn't portable to OpenBao (encrypted-storage change past Vault 1.15),
but the KV DATA is just JSON: walk every path, replay it. Carries everything over incl.
operator-supplied secrets (foundation/seaspots/minio) — nothing re-specified. Verified the
whole foundation KV is 5 leaf secrets, so migration is a few-KB round-trip. dr/kv-migrate.sh
is portable (SECRETS_BIN=vault|bao) and doubles as a DR asset. PLAN-004 §2/§3/§4 updated:
dump/load is now the canonical method (LOW risk), vault.olsitec.net endpoint stays.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

documentation/planning/PLAN-004-forgejo15-openbao-spike.md

@@ -57,30 +57,29 @@ OpenBao we can give each org (e.g. `seaspots`) a **real namespace**, not the OSS
 workaround. (Gaps still on OpenBao's roadmap: per-namespace storage backends, namespace sealing,
 non-hierarchical namespaces — none blocking for us.)
 
-**The gotcha — NOT a drop-in from Vault 1.18:**
-- OpenBao's in-place migration guide **explicitly will not work on Vault ≥ 1.15.0** — and we run
-  **1.18**. So you **cannot** simply point OpenBao at the existing Vault raft data / restore the Vault
-  raft snapshot into OpenBao. (Raft/API are compatible in shape, but the sealed-data compatibility
-  breaks past 1.15.)
-- **Therefore: re-seed, don't migrate.** Fresh OpenBao `operator init` (→ **new unseal keys + new root
-  token**, captured back into passphrase-encrypted Pulumi config exactly like `bootstrap/vault.ts`
-  does today), then **repopulate the KV**. Two viable ways:
-  - **(a) Re-run the credential provisioning** — the foundation is **Pulumi-owns-credentials**
-    (`bootstrap/components/credentials.ts` writes `@pulumi/random`-generated secrets into KV). Point it
-    at OpenBao and it re-writes them. Cleanest for *generated* secrets. ⚠ But **operator-supplied**
-    secrets that are NOT Pulumi-generated (e.g. the MinIO creds at `foundation/seaspots/minio` stored
-    this session, and anything else hand-entered) would be **lost** and must be re-put.
-  - **(b) Export → import the KV** — dump every `foundation/*` path from the live Vault
-    (`vault kv get -format=json` walk) and `bao kv put` into OpenBao. Preserves operator-supplied
-    values. A ~30-line script; the spike should write it and keep it (it's also a real DR asset).
-  - Recommendation: **(b) export/import** as the general mechanism, with (a) as the fallback/validation.
-- **API/provider/CLI compatibility (High):** OpenBao speaks the Vault HTTP API, so `@pulumi/vault`
-  (vendored in bootstrap) and our `vault kv …` calls work against it; the CLI is **`bao`** (Vault-compatible
-  env/flags). `bootstrap/vault.ts`'s init/unseal logic should port with an image swap + `vault`→`bao`
-  command rename. **Verify the `@pulumi/vault` provider talks to OpenBao cleanly** early in the spike.
+**The one gotcha — the raft SNAPSHOT isn't portable, but the DATA trivially is (High confidence):**
+- OpenBao's in-place migration guide **explicitly will not work on Vault ≥ 1.15.0** (we run **1.18**):
+  the **encrypted storage layer** (seal/barrier) changed, so you cannot restore Vault's **raft snapshot**
+  into OpenBao. **This is a storage-format limit, not a data limit.**
+- **The KV content is just JSON → dump it.** Walk every path (`kv get -format=json`) and replay it
+  (`kv put`) into a freshly-initialised OpenBao. This carries **everything** over — including
+  operator-supplied secrets like `foundation/seaspots/minio` — **nothing re-specified by hand.** Tool:
+  [`dr/kv-migrate.sh`](../../dr/kv-migrate.sh) (`export` from Vault → `import` into OpenBao; portable via
+  `SECRETS_BIN=vault|bao`, `SECRETS_CONTAINER=…`). **Verified this session:** the *entire* foundation KV
+  is just **5 leaf secrets** — `backup/`, `forgejo/`, `postgres/`, `rustfs/service-credentials` +
+  `seaspots/minio` — so the migration is a few-KB JSON round-trip, not a risk.
+- **What a fresh OpenBao still needs** (re-established by `bootstrap`, NOT carried by the KV dump): its
+  own `operator init` → **new unseal keys + root token** (captured into passphrase-encrypted Pulumi
+  config exactly like `bootstrap/vault.ts` does), plus any policies/auth methods. The foundation is
+  **KV-only** (CONTRACT_002) → nothing else to move. Version *history* isn't preserved (current value
+  only — fine here).
+- **Endpoint unchanged — `vault.olsitec.net` stays.** OpenBao speaks the Vault HTTP API, so every
+  consumer (`@pulumi/vault`, `vault kv …` calls, apps reading their creds) points at the same URL; only
+  the container image + the `vault`→`bao` CLI name change. **Verify `@pulumi/vault` drives OpenBao
+  cleanly** early — it gates the bootstrap credentials path.
 - **Action:** add `IMAGE_OPENBAO=openbao/openbao:<pinned ≥2.3.1>@sha256:…` to `VERSIONS` (confirm the
-  canonical image ref — `openbao/openbao` vs `quay.io/openbao/openbao`; pin the current stable, which as
-  of 2026-07 is well past the 2.3.1 namespaces-GA line).
+  canonical image ref — `openbao/openbao` vs `quay.io/openbao/openbao`; pin the current stable, well past
+  the 2.3.1 namespaces-GA line). `bootstrap/vault.ts` init/unseal ports with the image swap + `vault`→`bao`.
 
 ## 3. The spike procedure (proposed)
 
@@ -92,9 +91,9 @@ non-hierarchical namespaces — none blocking for us.)
 4. **Restore + migrate:**
    - Forgejo: restore repos + Postgres from the dump; start v15 → it auto-migrates the DB. Handle the
      two v15 breaking changes (§1). Confirm `forge.<host>` = 200, packages + Actions intact.
-   - OpenBao: fresh init/unseal, then **§2 re-seed** (export/import KV). Re-put operator secrets
-     (`foundation/seaspots/minio`). Confirm the app reads its creds (Forgejo/Postgres/RustFS came from
-     Vault via `credentials.ts`).
+   - OpenBao: fresh init/unseal, then **`dr/kv-migrate.sh import`** the KV JSON exported from the live
+     Vault (all 5 secrets incl. `foundation/seaspots/minio` — nothing re-typed). Confirm the app reads
+     its creds (Forgejo/Postgres/RustFS came from Vault via `credentials.ts`).
 5. **Re-run the test matrix** (the acceptance gate):
    - `999_testing` 5 candidates green; `ecosystem-selftest` green.
    - ci-bot registry push + **repo-link** still works (re-check quirks 4/5 under v15 — see §1).
@@ -104,12 +103,11 @@ non-hierarchical namespaces — none blocking for us.)
 
 ## 4. Risks / open questions
 
-- **OpenBao re-seed is the highest risk** — no drop-in from 1.18. De-risk with the export/import script
-  (§2b) built + tested against the backup before touching anything live. (Medium)
-- **Do generated-vs-supplied secrets round-trip?** Enumerate every KV path and classify
-  Pulumi-generated vs operator-supplied before the spike (the latter must be re-put). (Medium)
-- **Does `@pulumi/vault` + our `vault kv put -` stdin pattern work unmodified against OpenBao?** Verify
-  first — it gates the whole bootstrap credentials path. (Medium)
+- **OpenBao re-seed is LOW risk** — the KV is 5 entries and `dr/kv-migrate.sh` does the dump/load
+  (`export` verified this session; the `import` half runs in the spike). The generated-vs-supplied
+  distinction is **moot** — a logical dump preserves *every* value, operator-supplied included. (Low)
+- **Does `@pulumi/vault` + our `vault kv put -` stdin pattern drive OpenBao unmodified?** This is the
+  real check — it gates the bootstrap credentials path. Verify first. (Medium)
 - **Forgejo v15 root-image config path** — confirm `/data/gitea/conf/app.ini` still honoured (§1.2). (Low)
 - **Namespaces reshape PLAN-003** — if we adopt OpenBao, revisit PLAN-003 §2.4/§7.1 to use real
   per-org namespaces instead of `foundation/<org>/*` mount+policy. (design follow-up)

dr/kv-migrate.sh

@@ -0,0 +1,64 @@
+#!/usr/bin/env bash
+# dr/kv-migrate.sh — logical KV v2 export / import between Vault and OpenBao.
+#
+# WHY logical (not a raft snapshot): OpenBao's IN-PLACE raft migration is unsupported
+# from Vault >= 1.15 (the egg runs 1.18) — the encrypted STORAGE layer changed. But the
+# LOGICAL KV content is just JSON: walk every path and replay it. This carries EVERYTHING
+# over — including operator-supplied secrets like foundation/seaspots/minio — so nothing
+# is re-specified by hand. It is also a standalone DR asset (a portable KV backup).
+#
+# Runs the CLI INSIDE the secrets container over `docker exec` (so it needs no host CLI),
+# and needs `jq` on the host. Portable to both engines via env:
+#   Vault  (source): SECRETS_CONTAINER=foundation-vault   SECRETS_BIN=vault
+#   OpenBao (target): SECRETS_CONTAINER=foundation-openbao SECRETS_BIN=bao
+#
+# Usage (run on the VM that hosts the container, or wrap in ssh):
+#   SECRETS_TOKEN=<root/admin> ./kv-migrate.sh paths  [mount]     # list leaf secret paths
+#   SECRETS_TOKEN=<root/admin> ./kv-migrate.sh export [mount] > foundation-kv.json
+#   SECRETS_TOKEN=<root/admin> SECRETS_CONTAINER=foundation-openbao SECRETS_BIN=bao \
+#       ./kv-migrate.sh import [mount] < foundation-kv.json
+#
+# Scope/caveats: captures the CURRENT version of each secret (not full KV v2 version
+# history — fine for the foundation); covers the given KV mount only (the foundation is
+# KV-only per CONTRACT_002; policies/auth methods/root token are re-established by
+# bootstrap on the target, not migrated here).
+set -euo pipefail
+
+CMD="${1:?usage: paths|export|import [mount]}"
+MOUNT="${2:-foundation}"
+CONT="${SECRETS_CONTAINER:-foundation-vault}"
+BIN="${SECRETS_BIN:-vault}"
+TOKEN="${SECRETS_TOKEN:?set SECRETS_TOKEN to the target engine root/admin token}"
+
+ve() { docker exec -e VAULT_ADDR=http://127.0.0.1:8200 -e VAULT_TOKEN="$TOKEN" "$CONT" "$BIN" "$@"; }
+
+# Recursively print every leaf secret path (relative to the mount).
+walk() {
+  local p="$1" k
+  while IFS= read -r k; do
+    [ -z "$k" ] && continue
+    if [ "${k%/}" != "$k" ]; then walk "$p$k"; else echo "$p$k"; fi
+  done < <(ve kv list -format=json "$MOUNT/$p" 2>/dev/null | jq -r '.[]?' 2>/dev/null || true)
+}
+
+case "$CMD" in
+  paths)
+    walk ""
+    ;;
+  export)
+    # {"<path>": {<data>}, ...} on stdout
+    while IFS= read -r path; do
+      [ -z "$path" ] && continue
+      ve kv get -format=json "$MOUNT/$path" | jq -c --arg p "$path" '{($p): .data.data}'
+    done < <(walk "") | jq -s 'add // {}'
+    ;;
+  import)
+    # {"<path>": {<data>}} on stdin -> replay into the target engine
+    jq -c 'to_entries[]' | while IFS= read -r e; do
+      path=$(jq -r '.key' <<<"$e")
+      jq -c '.value' <<<"$e" | ve kv put "$MOUNT/$path" - >/dev/null
+      echo "put $MOUNT/$path"
+    done
+    ;;
+  *) echo "unknown command: $CMD (paths|export|import)" >&2; exit 2 ;;
+esac