switch to emqx

README.md

@@ -11,7 +11,7 @@ Three hosts (see issue #21 for the full design discussion):
 
 | Host          | Role                                      | Public hostname        | Private IP |
 | ------------- | ----------------------------------------- | ---------------------- | ---------- |
-| `mqtt-ingest` | HiveMQ CE + Go MQTT→Postgres ingester     | `ingest.gebos.online`     | `10.0.0.4` |
+| `mqtt-ingest` | EMQX broker + Go MQTT→Postgres ingester   | `ingest.gebos.online`     | `10.0.0.4` |
 | `db-host`     | Postgres 17 + TimescaleDB (telemetry+auth)| `db.gebos.online` (SSH only) | `10.0.0.2` |
 | `app-host`    | Caddy + Kong + Supabase (compose) + SPA   | `app.gebos.online`, `api.gebos.online` | `10.0.0.3` |
 
@@ -38,27 +38,25 @@ choice. The device firmware is the fixed end of this contract, so the rest of
 the stack is built to match it rather than the other way around.
 
 ```
-device ──MQTTS:8883──▶ Caddy (layer4, TLS term) ──MQTT:1883──▶ HiveMQ CE ──▶ ingester ──▶ Postgres
-         acrios/<IMSI>/<metric>          loopback                file-RBAC      IMSI→tenant
+device ──MQTTS:8883──▶ EMQX (native TLS) ──MQTT:1883 loopback──▶ ingester ──▶ Postgres
+         acrios/<IMSI>/<metric>            authn + ACL              IMSI→tenant
 ```
 
-### Broker auth: file-RBAC, not allow-all
+### Broker: EMQX in the official container, deny-by-default auth
 
-HiveMQ CE ships the `hivemq-allow-all` extension — any client can publish or
-subscribe to anything. We replaced it with the vendored
-[`hivemq-file-rbac-extension`](https://github.com/hivemq/hivemq-file-rbac-extension)
-(`nix/modules/gebos-hivemq.nix`), which adds username/password auth and
-topic-level authorization. `gebos-hivemq.preStart` reconciles the writable
-extensions dir on every start: it installs the pinned extension and **removes
-allow-all**, so a redeploy fixes already-provisioned boxes too. Credentials
-(`credentials.xml`) are rendered by sops-nix (`services.gebos.secrets.hivemq`)
-and symlinked from the `/run/secrets` tmpfs into the extension's `conf/` — they
-never touch the Nix store or the state dir. `password-type` is `PLAIN` because
-the file is already an encrypted secret at rest and `0400`/`gebos-hivemq` at
-runtime; flip to `HASHED` for defence-in-depth.
+EMQX runs as the official `emqx/emqx` image via `oci-containers` with host
+networking (`nix/modules/gebos-emqx.nix`). Authentication uses the built-in
+database, seeded from a `users.csv` rendered by sops-nix
+(`services.gebos.secrets.emqx`) and bootstrapped on first start; the bootstrap
+only inserts users that don't exist yet, so a password rotation means deleting
+the user (dashboard or `emqx ctl`) and restarting to re-import. Authorization
+is a static `acl.conf` with `no_match = deny`. Passwords are plaintext in the
+CSV because the file is already an encrypted secret at rest and `0400` at
+runtime.
 
 There are three broker users: `ingester` (subscribes the device tree), `admin`
-(break-glass superuser), and `bender` (the shared device account, below).
+(break-glass, `is_superuser` — bypasses the ACL), and `bender` (the shared
+device account, below).
 
 ### Topic scheme: the device's `acrios/<IMSI>/…` is the source of truth
 
@@ -80,7 +78,7 @@ publish/subscribe under `acrios/#`. Tenant isolation is therefore enforced
 could publish under any IMSI. That's an accepted trade-off for a small trusted
 fleet, and it gets data flowing without per-device provisioning.
 
-The production answer (**Option B**, a TODO in `nix/modules/gebos-secrets.nix`)
+The production answer (**Option B**, a TODO in `nix/modules/gebos-emqx.nix`)
 is one broker user per device with `username == IMSI`, scoped to
 `acrios/${{username}}/#` so the broker itself prevents a device from spoofing
 another's IMSI. It's not wired up because it needs a firmware change
@@ -88,20 +86,18 @@ another's IMSI. It's not wired up because it needs a firmware change
 firmware's client-id (`acrcv-<IMSI>`) carries a prefix the topic doesn't, so
 per-device scoping must key on username, not `${{clientid}}`.
 
-### TLS: caddy-l4 terminates, HiveMQ stays loopback
+### TLS: EMQX terminates natively, certs via ACME HTTP-01
 
-HiveMQ binds `127.0.0.1:1883` only; it never faces the network. Senders connect
-MQTTS on `:8883`, and Caddy — rebuilt with the
-[`caddy-l4`](https://github.com/mholt/caddy-l4) layer4 module — terminates TLS
-and proxies cleartext to the broker (`nix/modules/gebos-caddy.nix`). We reuse
-Caddy here (rather than a native HiveMQ TLS listener + keystore) so there's one
-ACME story for the whole fleet. The wrinkle: caddy-l4's `tls` handler only
-*terminates* — it never obtains certs — so each `tcpProxy` site also emits a
-companion HTTPS block purely to make Caddy's automatic HTTPS provision the cert
-that the layer4 handler then serves by SNI. That's why `mqtt-ingest` opens
-**80/443** (ACME challenge + renewal) alongside **8883**. The custom Caddy build
-is only used on hosts that actually have a `tcpProxy` site; app-host's
-static/reverse-proxy sites stay on stock Caddy.
+EMQX terminates TLS itself on `:8883`; the plain MQTT listener stays on
+loopback for the co-located ingester. Certificates come from `security.acme`
+using the **HTTP-01** challenge: lego's built-in standalone server answers on
+`:80` (nothing else listens there — no Caddy on this host), so no DNS API
+secrets are needed. `mqtt-ingest` opens **80 + 8883** only. The cert's
+`postRun` hook installs `fullchain.pem`/`privkey.pem` at fixed paths in the
+broker state dir, and EMQX re-reads the PEMs from disk (~every 120 s), so
+renewals hot-reload without a restart or dropped connections. The key is RSA
+(`keyType = "rsa2048"`) because embedded sender TLS stacks often can't do
+ECDSA. Caddy remains only on app-host (stock build, HTTP sites).
 
 ## Repo layout