Wire schema v1 into a manual Supabase migration and vendor the real stack
deploy / deploy (push) Successful in 5m57s
check / flake-check (push) Failing after 35m52s

- Move db/schema.sql to supabase/migrations/ as the first supabase CLI
  migration (manual `db push` only, no automated runner); re-add the
  gebos_ingest role + grants there since init.sql never re-runs
- Add gebos-postgres-passwords oneshot on db-host: syncs role passwords
  from sops (LoadCredential, journal-safe), makes supabase_admin
  SUPERUSER and hands the auth schema to supabase_auth_admin to match
  the upstream supabase/postgres image; add pg_hba rule for 10.0.0.0/8
- Vendor the official docker-compose (studio/kong/auth/rest/meta only,
  external Postgres, loopback Studio with no Kong dashboard route) plus
  kong.yml (trimmed) and kong-entrypoint.sh (verbatim); tested: compose
  config, Kong config parse, migration applied on TimescaleDB pg17
- Document decisions as ADRs 0001-0004 (migrations, passwords, vendored
  stack, JWT API keys incl. verify/mint procedure)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Lars Nolden
2026-07-08 22:33:50 +02:00
parent b2a1056255
commit 2ce68cb098
17 changed files with 967 additions and 118 deletions
@@ -0,0 +1,51 @@
# ADR-0001: Database schema changes are manual Supabase CLI migrations
Date: 2026-07-08
Status: Accepted
## Context
Schema v1 (`db/schema.sql`, ~600 lines) was committed but wired to nothing.
The only SQL that ran automatically was `nix/supabase/init.sql` via
`services.postgresql.initialScript` — which executes **only at first initdb**,
so on the already-initialized db-host cluster neither it nor any schema change
would ever apply again. A delivery mechanism was needed.
Options considered:
1. **Automated runner** — a systemd oneshot on db-host applying migrations on
every deploy (dbmate/atlas or plain psql with a tracking table).
2. **Manual Supabase CLI migrations** — plain-SQL files in
`supabase/migrations/`, applied by a human with `supabase db push`.
3. **Extending initialScript** — rejected outright: never re-runs on an
existing cluster.
## Decision
Option 2. Schema changes are rare, deliberate, admin-level operations; a
human applies them on purpose, and no automation exists to break or to apply
a half-reviewed migration as a side effect of a deploy.
* Migrations live in `supabase/migrations/<timestamp>_<name>.sql`; schema v1
moved there as the first one. `supabase/config.toml` is the minimal CLI
project marker.
* Applied via SSH tunnel to db-host as the `postgres` superuser
(see `supabase/README.md` for the exact commands). The CLI records applied
files in `supabase_migrations.schema_migrations`, so pushes are incremental
and re-running is safe. Individual migrations need **not** be idempotent.
* Layering rule: `nix/supabase/init.sql` holds only cluster bootstrap
(Supabase schemas, admin/API roles, extensions needing
`shared_preload_libraries`); everything else — including the `gebos_ingest`
role and its grants, which are coupled to application tables and must reach
existing clusters — lives in migrations. Passwords live in neither
(see ADR-0002).
## Consequences
* Deploys (`deploy-rs`) never touch the schema; a deploy and a migration are
two separate, independently reversible acts.
* A human must remember to push after merging a migration. Accepted for a
pilot with one operator.
* The local `nix run .#dev` Postgres has no TimescaleDB yet, so the v1
migration's `create_hypertable` fails there until the dev stack gains the
extension.
@@ -0,0 +1,61 @@
# ADR-0002: Postgres role passwords are synced by a one-shot systemd unit
Date: 2026-07-08
Status: Accepted
## Context
The Supabase services on app-host and the ingester on mqtt-ingest connect to
db-host over TCP and need password auth, but every login role
(`supabase_admin`, `supabase_auth_admin`, `authenticator`, `gebos_ingest`)
was created passwordless.
NixOS deliberately offers **no declarative option** for Postgres passwords:
anything the configuration references lands world-readable in `/nix/store`,
and `initialScript` — the only built-in hook that could set one — runs solely
at first initdb. Passwords are data, not configuration.
Two supporting gaps surfaced at the same time:
* NixOS's default `pg_hba.conf` covers only local sockets and loopback, so
connections from 10.0.0.0/8 were rejected before password auth even
started, firewall rule notwithstanding.
* The upstream `supabase/postgres` image ships role *attributes* our vanilla
Postgres lacked: `supabase_admin` is a SUPERUSER there (Studio/postgres-meta
connect as it), and `supabase_auth_admin` owns the `auth` schema (GoTrue
runs its own migrations in it at startup).
## Decision
A oneshot unit, `gebos-postgres-passwords` (`nix/modules/gebos-postgres.nix`),
runs on db-host after `postgresql.service` on every boot/deploy:
* Runs as the `postgres` OS user (local peer auth); reads the sops-rendered
env file via systemd `LoadCredential`, so the root-owned 0400 file needs no
permission widening.
* Idempotently syncs: `postgres``postgres_admin_password`;
`supabase_admin` / `supabase_auth_admin` / `authenticator` ← the shared
`supabase_postgres_password` (mirroring upstream's single
`POSTGRES_PASSWORD` convention); `gebos_ingest`
`ingester_postgres_password`.
* Enforces the two upstream attributes: `supabase_admin SUPERUSER` and
`ALTER SCHEMA auth OWNER TO supabase_auth_admin`.
* Guards every role for existence (tolerates a cluster the schema-v1
migration hasn't reached) and discards query output
(`--output=/dev/null`) so passwords never reach the journal.
* db-host enables the `supabase` and `ingester` secret toggles solely to get
those keys into its env file; `.sops.yaml`'s single key group already
permits this.
* `pg_hba` gains `host all all 10.0.0.0/8 scram-sha-256`.
## Consequences
* No password ever exists in git, the nix store, or SQL files; rotation is
`sops nix/secrets/secrets.yaml` + deploy + `systemctl restart
gebos-postgres-passwords` (the unit does not auto-rerun on secret-content
changes alone).
* The three Supabase roles share one password by design — per-role passwords
would diverge from the vendored compose, which interpolates a single
`${POSTGRES_PASSWORD}` everywhere.
* db-host can decrypt the Supabase JWT/API-key secrets it doesn't strictly
need; acceptable until `.sops.yaml` moves to per-prefix rules (noted there).
@@ -0,0 +1,66 @@
# ADR-0003: Vendor a trimmed Supabase compose stack against external Postgres
Date: 2026-07-08
Status: Accepted
## Context
The Supabase stack on app-host was a placeholder skeleton. The official
self-hosting bundle assumes its own bundled `db` container and ships nine+
services (realtime, storage, imgproxy, edge functions, supavisor pooler,
Logflare analytics, …), most of which the UVI pilot does not use. Meanwhile
our Postgres lives on a separate NixOS host (db-host) with TimescaleDB.
Kong deserves a note, since "why is it even there" came up: Kong is the
single front door for `api.gebos.online` (Caddy → Kong on loopback :8000).
It routes paths to internal services (`/auth/v1` → GoTrue, `/rest/v1`
PostgREST) and enforces the perimeter (key-auth + ACLs), so unauthenticated
internet traffic never reaches the backends. Replacing it would mean
re-implementing routing/auth/CORS in Caddy by hand instead of inheriting
Supabase's tested config.
## Decision
Vendor the upstream `docker/docker-compose.yml` (and `kong.yml` +
`kong-entrypoint.sh`) into `nix/supabase/`, keeping only
**studio, kong, auth (GoTrue), rest (PostgREST), meta (postgres-meta)**, with
these deviations — each also documented in the files' header comments:
1. **No `db` service.** Everything connects to
`${POSTGRES_HOST}:${POSTGRES_PORT}` (db-host); `depends_on: db` dropped.
2. **No realtime / storage / imgproxy / functions / supavisor / analytics.**
Re-vendor from upstream when actually needed.
3. **Studio binds to loopback only, and Kong's catch-all dashboard route is
removed.** Upstream fronts Studio with Kong basic-auth on `/`; we expose
no route to it at all. Access is `ssh -L 3000:127.0.0.1:3000` — possession
of the SSH key is the credential. `supabase_dashboard_password` is
therefore currently unused. The public API surface is exactly
`/auth/v1`, `/rest/v1`, `/graphql/v1` and two `.well-known` endpoints.
4. **meta connects as `supabase_admin`, not `postgres`.** Upstream's
`postgres` role *is* its cluster superuser; on db-host `postgres` keeps a
separate admin password, so `supabase_admin` (made SUPERUSER by ADR-0002)
fills that role with the shared service password.
5. **`kong-entrypoint.sh` is vendored byte-identical.** Kong's declarative
YAML cannot read env vars, so the entrypoint substitutes `$VARS` (the API
keys) into the config at container start; keeping it unmodified makes
re-vendoring a plain copy.
Variable layering (the answer to "where does `POSTGRES_HOST` come from"):
* non-secret deployment config (`POSTGRES_HOST/PORT/DB`, `STUDIO_BIND`) —
systemd unit `environment` in `gebos-supabase.nix`, fed by module options;
* secrets (`POSTGRES_PASSWORD`, `JWT_SECRET`, `ANON_KEY`,
`SERVICE_ROLE_KEY`) — sops-rendered `EnvironmentFile`;
* gebos constants (public URLs, org name) — hardcoded in the vendored
compose.
## Consequences
* Deliberately small attack surface and dependency set; Studio reachable only
through SSH.
* Upstream updates are a re-copy plus re-applying the five listed deviations;
all drift is concentrated in `docker-compose.yml` and `kong.yml` headers.
* GoTrue has **no SMTP configured yet** — magic-link login cannot send email
until the `SMTP_*` secrets are added (TODO marked in the compose file).
* `anon`/`authenticated` still hold zero grants; PostgREST serves nothing
until the RLS/grants migration lands (future ADR).
+103
View File
@@ -0,0 +1,103 @@
# ADR-0004: Legacy JWT-based API keys, one signing secret
Date: 2026-07-08
Status: Accepted
## Context
Supabase's API tier knows two credentials, and they are easy to confuse
because in the legacy scheme **both are JWTs signed with the same
`JWT_SECRET`**:
| | `apikey` header | `Authorization: Bearer …` header |
|---|---|---|
| identifies | the application | a logged-in person |
| minted by | a human, once, at project setup | GoTrue, at every login |
| payload | `{"role":"anon"}` / `{"role":"service_role"}` | `{"role":"authenticated","sub":"<user uuid>",…}` |
| lifetime | ~10 years | 1 h (`GOTRUE_JWT_EXP`), refreshed |
| checked by | Kong: exact string match against its consumer list | PostgREST: signature verification, then `SET ROLE` on the claim |
The `role` claim is what binds tokens to the database: PostgREST executes
each request as the Postgres role the (verified) `Authorization` JWT names —
`anon` for pre-login traffic, `authenticated` for sessions (with `auth.uid()`
= the `sub` claim, feeding RLS), `service_role` for server-side admin work.
`service_role` exists because some work legitimately spans all tenants
(admin invites, cross-apartment aggregation jobs, backfills); it carries
`BYPASSRLS` and its key must never leave the server side, while the anon key
is public by design (it ships in the frontend bundle).
Supabase is migrating to opaque `sb_publishable_*`/`sb_secret_*` keys that
separate the two concepts; the vendored `kong-entrypoint.sh` contains the
translation shim for that scheme.
## Decision
Stay on the **legacy scheme** for the pilot: `supabase_anon_key` and
`supabase_service_role_key` are long-lived HS256 JWTs signed with
`supabase_jwt_secret`. The opaque-key shim stays dormant (its env vars are
empty, which switches the Kong entrypoint to plain apikey pass-through).
This has a non-obvious integrity requirement: **the three secrets form one
cryptographic family.** The API keys are only valid if they were signed with
the exact `JWT_SECRET` stored alongside them. They are minted offline by us —
no service issues them — and rotating `JWT_SECRET` silently invalidates both
API keys *and* every active user session; all three must always be rotated
together.
## Verifying / minting the keys
The keys currently in `nix/secrets/secrets.yaml` must be checked once against
the stored secret (they predate this ADR). From the repo root, with sops
access — `openssl` is not in the dev shell, so wrap in
`nix shell nixpkgs#openssl` if needed:
```sh
SECRET=$(sops -d --extract '["supabase_jwt_secret"]' nix/secrets/secrets.yaml)
verify() { # verify <jwt> — checks HS256 signature against $SECRET
local hp=${1%.*} sig=${1##*.}
local expect=$(printf '%s' "$hp" \
| openssl dgst -sha256 -hmac "$SECRET" -binary \
| basenc --base64url -w0 | tr -d '=')
if [ -n "$expect" ] && [ "$sig" = "$expect" ]; then
echo "valid ($(printf '%s' "${hp#*.}" | tr '_-' '/+' | base64 -d 2>/dev/null))"
else
echo "INVALID SIGNATURE"
fi
}
verify "$(sops -d --extract '["supabase_anon_key"]' nix/secrets/secrets.yaml)"
verify "$(sops -d --extract '["supabase_service_role_key"]' nix/secrets/secrets.yaml)"
```
Each should print `valid` with a payload naming the right role. If either
prints `INVALID SIGNATURE`, re-mint and store both:
```sh
mint() { # mint <role> — 10-year HS256 JWT signed with $SECRET
local iat=$(date +%s) b64='basenc --base64url -w0'
local h=$(printf '{"alg":"HS256","typ":"JWT"}' | $b64 | tr -d '=')
local p=$(printf '{"role":"%s","iss":"supabase","iat":%s,"exp":%s}' \
"$1" "$iat" $((iat + 315360000)) | $b64 | tr -d '=')
local s=$(printf '%s.%s' "$h" "$p" \
| openssl dgst -sha256 -hmac "$SECRET" -binary | $b64 | tr -d '=')
printf '%s.%s.%s\n' "$h" "$p" "$s"
}
mint anon # → sops set as supabase_anon_key
mint service_role # → sops set as supabase_service_role_key
```
(Equivalently: the generator on Supabase's self-hosting docs page produces
the same thing — paste in the stored `JWT_SECRET`, don't let it invent a new
one.)
## Consequences
* Matches what supabase-js sends by default; no client-side configuration
beyond URL + anon key.
* One secret to protect (`JWT_SECRET`) — and one blast radius: leak it and an
attacker can mint `service_role` tokens; rotate it and keys + sessions die
together.
* Moving to opaque keys later is config-only: fill the four `sb_*`/asymmetric
env vars and the already-vendored Kong entrypoint starts translating.
+12
View File
@@ -0,0 +1,12 @@
# Architecture Decision Records
Numbered, immutable-once-accepted records of the significant decisions in
this repo. Supersede by adding a new ADR and flipping the old one's status to
`Superseded by ADR-NNNN`, not by editing history.
| # | Title | Status |
|---|-------|--------|
| [0001](0001-manual-supabase-cli-migrations.md) | Database schema changes are manual Supabase CLI migrations | Accepted |
| [0002](0002-role-passwords-via-oneshot-unit.md) | Postgres role passwords are synced by a one-shot systemd unit | Accepted |
| [0003](0003-vendored-supabase-compose-subset.md) | Vendor a trimmed Supabase compose stack against external Postgres | Accepted |
| [0004](0004-legacy-jwt-api-keys.md) | Legacy JWT-based API keys, one signing secret | Accepted |