docs(plan): PLAN-003 — Forgejo org & CI-config as code
Writes up the operator's org-management idea (from SESSION_005): an isolated Pulumi 'orgs/' project (Gitea + Vault providers, peer to runners/ and ci-bot/) that manages Forgejo orgs/repos/teams and CI secrets/variables declaratively, Vault-sourced, with a seaspots org. Names it distinctly to avoid colliding with the package-registry 'Stage-2'. Flags the one real unknown to verify first: whether the Gitea TF provider covers Forgejo Actions secrets/variables (else a command-shim fallback). Corrects 'own Vault namespace' (Enterprise-only) to the OSS mount+policy equivalent. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
95146009bb
commit
be974dd5a0
1 changed files with 159 additions and 0 deletions
159
documentation/planning/PLAN-003-forgejo-org-and-ci-config.md
Normal file
159
documentation/planning/PLAN-003-forgejo-org-and-ci-config.md
Normal file
|
|
@ -0,0 +1,159 @@
|
|||
# PLAN-003 — Forgejo org & CI-config as code
|
||||
|
||||
> **Companion to** [PLAN-002-foundation-implementation.md](PLAN-002-foundation-implementation.md)
|
||||
> (the layered-platform strategy) and the isolated Tier-1 steps it spawned:
|
||||
> [`ci-bot/`](../../ci-bot/), [`runners/`](../../runners/), [`runners-k8s/`](../../runners-k8s/).
|
||||
> **Status:** Draft for human ratification. **Mode at authoring:** EXPLORE (design only, no code).
|
||||
> **Author role:** Lead Agent, HIGH-RISK / INFRA. **Date:** 2026-07-01.
|
||||
> **Origin:** operator ask at the end of `SESSION_2026-07-01_005.md` (MinIO creds for the
|
||||
> seaspots data pipelines → "Stage-2 Pulumi-managed organizations, repos, groups and variables").
|
||||
> Confidence markers (High/Medium/Low) follow baseline PD-5.
|
||||
|
||||
---
|
||||
|
||||
## 0. Naming note (read first — avoids a collision)
|
||||
|
||||
The word **"Stage-2" is already taken** in this repo and means the **@olsitec npm package
|
||||
registry** (`999_testing.md`, `SESSION_2026-07-01_004.md §Stage-2`) and the `STAGE 1 VENDOR →
|
||||
STAGE 2 PUBLISH → STAGE 3 CONSUME` module lifecycle (`000_TOPOLOGY.md §5`). Both are about
|
||||
**packages**. This document is a **different** concern — declarative management of Forgejo
|
||||
**orgs / repos / teams / CI variables** — so it is named for what it does, not "Stage-2".
|
||||
|
||||
---
|
||||
|
||||
## 1. Problem
|
||||
|
||||
The forge now hosts real work (ecosystem CI, the package registry, the seaspots toolchain
|
||||
runners), and org/CI configuration is accreting **imperatively**:
|
||||
|
||||
- **`ci-bot`** (user, `ci` team, tokens, org Actions secrets) was provisioned this session by an
|
||||
idempotent shell script ([`ci-bot/provision.sh`](../../ci-bot/provision.sh)) — good, but it is a
|
||||
bespoke script, not a declared resource set.
|
||||
- The seaspots test repos live **ad-hoc under `olsitec`** (`olsitec/seaspots-homepage`,
|
||||
`olsitec/token-service`) — a staging convenience, not a decision.
|
||||
- The seaspots data pipelines need **MinIO/S3 creds** (`AWS_ENDPOINT_URL`, `AWS_ACCESS_KEY_ID`,
|
||||
`AWS_SECRET_ACCESS_KEY`) that today are GitLab **group** CI/CD variables. They are now captured
|
||||
in Vault at **`foundation/seaspots/minio`** (this session) but are **not yet wired** into any
|
||||
Forgejo org secret/variable, and there is no repeatable mechanism to do so.
|
||||
|
||||
As more GitLab groups/projects migrate to the forge, "click it in the UI" and "another bespoke
|
||||
script" do not scale and are not reviewable/reproducible. We want the **same codify-everything
|
||||
discipline** the rest of the foundation already has (`bootstrap`, `runners`, `ci-bot`).
|
||||
|
||||
## 2. Decision — an isolated Pulumi `orgs/` project (High confidence on shape)
|
||||
|
||||
Manage Forgejo org/CI topology **as code** in a new isolated Pulumi project **`orgs/`**, a peer to
|
||||
`runners/` / `runners-k8s/` / `ci-bot/` — **never imported by `bootstrap`** (ADR-004 layering: it
|
||||
dials the Forgejo API + Vault only on its own `up`/`refresh`, so foundation ops never depend on it,
|
||||
and it never blocks a `bootstrap up`).
|
||||
|
||||
### 2.1 Providers
|
||||
|
||||
| Provider | Role | Confidence |
|
||||
| -------- | ---- | ---------- |
|
||||
| **Gitea** (Terraform provider, bridged via `pulumi package add terraform-provider …`) | Declares orgs, repos, teams, memberships, deploy keys, webhooks. Forgejo is Gitea-API-compatible (the whole foundation relies on this). | Medium — bridging TF providers is routine; **Actions secrets/variables coverage must be verified** (see §6). |
|
||||
| **Vault** (`@pulumi/vault`, already vendored in `bootstrap`) | Reads secret VALUES from `foundation/…` and feeds them to the Gitea/CI resources. | High |
|
||||
| **`@pulumi/command`** (fallback) | For anything the Gitea provider doesn't cover yet — e.g. org Actions secrets via the Forgejo API, exactly as `ci-bot/provision.sh` does today. | High |
|
||||
|
||||
### 2.2 What it manages
|
||||
|
||||
- **Organizations** (e.g. `seaspots`, `olsitec`), their visibility, and **teams** (the GitLab
|
||||
"group role" equivalent — `ci`, `owners`, per-project teams) + memberships.
|
||||
- **Repositories** (created/adopted), settings, default branch, enabled units (incl. Packages,
|
||||
Actions).
|
||||
- **Org/repo Actions secrets + variables** — the migration target for GitLab group CI/CD variables.
|
||||
`AWS_ENDPOINT_URL` → org **variable** (not secret); `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY`
|
||||
→ org **secrets**. Values sourced from Vault, never inlined.
|
||||
|
||||
### 2.3 Vault is the source of truth (High confidence)
|
||||
|
||||
Secret VALUES live in Vault; the `orgs/` project reads them and writes Forgejo secrets. CI runtime
|
||||
reads **only** the Forgejo Actions secret — **never Vault**. Therefore Vault-unsealed is required
|
||||
**only when you run `pulumi up`** on this project (a deliberate provisioning act), **not** for every
|
||||
CI job. This is the exact decoupling we enforced for `ci-bot`: the idempotent provisioning step is
|
||||
the source of truth for the runtime secret; Vault going away breaks no running pipeline. (Vault
|
||||
re-seals on VM reboot → `bootstrap/vault-unseal.sh`; a hard runtime dependency would be the trap.)
|
||||
|
||||
### 2.4 The "own Vault namespace/tenant" — OSS reality (High confidence, corrects the ask)
|
||||
|
||||
**Vault namespaces are Enterprise-only.** The foundation runs **Vault OSS 1.18.5**. The OSS
|
||||
tenant-equivalent for per-org isolation is:
|
||||
|
||||
1. a **dedicated KV path** (already: `foundation/seaspots/*`) — or, for a harder boundary, a
|
||||
**separate KV mount** `seaspots/` distinct from `foundation/`; **plus**
|
||||
2. a **scoped policy** granting read on only that path; **plus**
|
||||
3. an **AppRole (or scoped token)** the `orgs/` provisioning uses for that org.
|
||||
|
||||
This gives the same isolation (seaspots creds separated + access-controlled) without Enterprise.
|
||||
Extends `CONTRACT_002` (Vault path layout) with an org-scoped convention. **Decision to ratify
|
||||
(§7):** stay on `foundation/<org>/*` sub-paths, or promote each org to its own mount.
|
||||
|
||||
## 3. seaspots org topology (Medium confidence — needs operator ratification)
|
||||
|
||||
GitLab **group** ≈ Forgejo **organization**; GitLab **project** ≈ Forgejo **repository**
|
||||
(Forgejo has no nested subgroups — model hierarchy via naming + teams).
|
||||
|
||||
- New org **`seaspots`** (peer to `olsitec`), holding the pipeline + tools repos
|
||||
(`seaspots-homepage`, the data-pipeline repos, `seaspots-*-utils` tool repos, …).
|
||||
- Org variable `AWS_ENDPOINT_URL` + org secrets `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY`
|
||||
(from Vault `foundation/seaspots/minio`) → visible to every seaspots pipeline.
|
||||
- The k8s toolchain runners (`seaspots-s57-utils`, `seaspots-postgis-utils`) are **org-agnostic**
|
||||
(they route by `runs-on:` label), so they already serve a `seaspots` org unchanged.
|
||||
|
||||
## 4. How `ci-bot` folds in (High confidence)
|
||||
|
||||
`ci-bot` is the **first instance** of this pattern, done imperatively under time pressure. Under
|
||||
`orgs/` it becomes declared resources: the `ci-bot` user, the `ci` team + membership, and the
|
||||
`FORGE_REGISTRY_*` org secrets become Gitea-provider / Vault-fed resources. Token **minting** may
|
||||
stay imperative (a `@pulumi/command` calling the Forgejo admin CLI, as `ci-bot` and the runners do)
|
||||
— Forgejo mints tokens server-side; the provider likely won't. Net: `ci-bot/provision.sh` is
|
||||
**absorbed**, not thrown away; its logic ports 1:1.
|
||||
|
||||
## 5. Migration path (incremental, no big bang)
|
||||
|
||||
1. **Adopt, don't recreate.** Import existing `olsitec` org + the live repos into `orgs/` state
|
||||
(`pulumi import`) so the first `up` is a no-op diff — same discipline as adopting the live s57
|
||||
runner into `runners-k8s`.
|
||||
2. Stand up the **`seaspots` org** + Vault mount/policy + org secrets/variables.
|
||||
3. Migrate the seaspots repos from `olsitec/*` to `seaspots/*` (or create fresh + push), repointing
|
||||
their `.forgejo/workflows`. The docker candidates' registry image refs move
|
||||
`olsitec/<repo>` → `seaspots/<repo>` (and the reusable-workflow `source-url`).
|
||||
4. Fold `ci-bot` into `orgs/`; retire the standalone script once parity is proven.
|
||||
5. Repeat per GitLab group as projects migrate.
|
||||
|
||||
## 6. Forgejo-11 constraints that shape this (verified this session — see `.forgejo/workflows/README.md`)
|
||||
|
||||
- **Actions secrets are NOT inherited by reusable workflows** → callers need `secrets: inherit`.
|
||||
Any org secret this project sets is subject to that quirk downstream.
|
||||
- **`@master` reusable refs are stale-parsed** → callers pin the SHA. Orthogonal to this project but
|
||||
relevant when it manages caller repos.
|
||||
- **⚠ Open verification (Medium/Low):** does the Gitea Terraform provider expose **org/repo Actions
|
||||
secrets + variables**? Forgejo Actions is recent; if the provider lacks it, those specific
|
||||
resources stay on the `@pulumi/command` + Forgejo-API fallback (§2.1) — the design still holds,
|
||||
just with a thin imperative shim for that one resource type. **This is the first thing to verify
|
||||
before building.**
|
||||
|
||||
## 7. Open decisions to ratify
|
||||
|
||||
1. **Vault layout:** `foundation/<org>/*` sub-paths (simplest, current) vs a dedicated `seaspots/`
|
||||
mount + policy (harder tenant boundary). Recommendation: sub-path now, mount later if a second
|
||||
tenant appears. (Medium)
|
||||
2. **Org boundary:** one `seaspots` org for all seaspots repos vs keeping them under `olsitec`.
|
||||
Recommendation: dedicated `seaspots` org (matches the GitLab group, cleaner secret scope). (Medium)
|
||||
3. **Provider choice** once §6 is verified: pure Gitea provider vs Gitea + command-shim for secrets.
|
||||
4. **Scope of first cut:** seaspots only, or model `olsitec` + `ci-bot` in the same pass.
|
||||
|
||||
## 8. Non-goals / risks
|
||||
|
||||
- **Not** in `bootstrap`; **not** a runtime dependency for CI (Vault only at provision time).
|
||||
- **Not** GitLab-subgroup fidelity (Forgejo has no nested orgs).
|
||||
- Risk: managing a live org declaratively can **delete** drift — first `up` MUST be import-to-no-op
|
||||
(§5.1); treat repo/secret deletions as guarded, review every `pulumi preview`.
|
||||
- Risk: the Gitea provider's Forgejo-Actions coverage (§6) — de-risk by verifying before committing
|
||||
to the pure-provider path.
|
||||
|
||||
---
|
||||
|
||||
**Next concrete step:** verify the Gitea Terraform provider's Actions-secret/variable coverage
|
||||
(§6), then scaffold `orgs/` (Gitea + Vault providers) and `pulumi import` the existing `olsitec`
|
||||
org as a no-op baseline before adding `seaspots`.
|
||||
Loading…
Add table
Add a link
Reference in a new issue