Compare commits

...

21 Commits

Author SHA1 Message Date
Lars Nolden b2a1056255 schema v1
check / flake-check (push) Successful in 32s
deploy / deploy (push) Successful in 37s
2026-07-03 17:05:43 +02:00
Lars Nolden d530f0f3e5 debugging gateway internet access 2026-07-03 16:50:42 +02:00
Lars Nolden 2d0ac1bc53 gateway script 2026-07-02 17:44:24 +02:00
Lars Nolden 58155dc737 switch to emqx
check / flake-check (push) Successful in 25s
deploy / deploy (push) Successful in 48s
2026-07-02 15:19:59 +02:00
Lars Nolden 08d9fc594e Fix invalid Caddyfile: put layer4 in globalConfig, not a 2nd global block
check / flake-check (push) Successful in 22s
deploy / deploy (push) Successful in 37s
The NixOS caddy module already emits one global options block (for `email`)
and appends extraConfig after it. gebos-caddy also prepended its own keyless
`{ layer4 { ... } }` block, so the rendered Caddyfile had two keyless blocks
and caddy rejected it: "server block without any key is global configuration,
and if used, it must be first" -> caddy.service failed -> mqtt-ingest
activation exited 4 and deploy-rs rolled back.

Move the layer4 directive into services.caddy.globalConfig so it lands inside
the module's single global block. Verified the rendered Caddyfile for
mqtt-ingest; app-host (http-only) output is unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-01 13:09:08 +02:00
Lars Nolden 5e2454f622 Inject runner-absolute SSH opts via deploy-rs --ssh-opts
check / flake-check (push) Successful in 19s
deploy / deploy (push) Failing after 56s
The gitea-runner is a systemd DynamicUser whose passwd home is /, so ssh
expands ~ to /.ssh/... (not $HOME). deploy-rs passes ssh opts to ssh with
no shell, so a tilde in deploy.nix sshOpts resolved to a missing key and
nix copy failed with 'failed to start SSH connection'.

Keep deploy.nix sshOpts path-free; build absolute -i/known_hosts paths in
deploy.yml from $HOME (bash-expanded) and append them via --ssh-opts, which
deploy-rs merges into NIX_SSHOPTS for the copy and the activation ssh.
Also removes the temporary SSH debug step.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-30 19:06:12 +02:00
Lars Nolden 16fd12173f Add temporary SSH debug step to diagnose db-host deploy auth
check / flake-check (push) Successful in 17s
deploy / deploy (push) Failing after 20s
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-30 18:21:54 +02:00
Lars Nolden e1b3017fb7 test deploy
check / flake-check (push) Successful in 18s
deploy / deploy (push) Failing after 18s
2026-06-30 18:11:23 +02:00
Lars Nolden d5a0be9e40 dont overwrite SSH key
check / flake-check (push) Successful in 16s
deploy / deploy (push) Failing after 19s
2026-06-29 18:04:48 +02:00
Lars Nolden 649dd737c7 Readme update
deploy / deploy (push) Failing after 3m20s
check / flake-check (push) Successful in 17s
2026-06-29 17:23:08 +02:00
Lars Nolden bf8b25f374 make caddy a L4 proxy to terminate TLS for HiveMQ 2026-06-29 17:23:02 +02:00
Lars Nolden 70c2ada986 add secrets and age pubkeys to sops 2026-06-29 17:22:43 +02:00
Lars Nolden 4f9c6086bd hivemq setup 2026-06-26 18:36:44 +02:00
Lars Nolden c526cf671f Add nixos-anywhere deploy instructions to README 2026-06-26 17:32:07 +02:00
Lars Nolden 30b325393a replace with real hardware configs 2026-06-26 17:01:28 +02:00
Lars Nolden 7579d4dcf5 change domains, ips and make flake check work 2026-06-26 16:12:57 +02:00
Lars Nolden 9a6a901dfe bump to nix 26.05 2026-06-26 14:26:54 +02:00
Lars Nolden b10d22bd57 change runner names
check / flake-check (push) Failing after 25s
deploy / deploy (push) Failing after 4s
2026-06-25 18:55:06 +02:00
Lars Nolden 19027e3b48 add data to gitignore
check / flake-check (push) Has been cancelled
deploy / deploy (push) Has been cancelled
2026-06-25 18:32:48 +02:00
Lars Nolden 08ae2764e0 update authorized keys 2026-06-25 18:32:39 +02:00
lars 1cf710fe5b Merge pull request 'Infrastructure skeleton — flake, three hosts, Supabase compose, ingester, deploy-rs' (#23) from klaus/skeleton-infra into main
check / flake-check (push) Has been cancelled
deploy / deploy (push) Has been cancelled
Reviewed-on: #23
2026-06-25 14:54:06 +00:00
34 changed files with 2570 additions and 111 deletions
+1 -1
View File
@@ -6,7 +6,7 @@ on:
jobs:
flake-check:
runs-on: nix
runs-on: nixos
steps:
- uses: actions/checkout@v4
- run: nix flake check
+13 -6
View File
@@ -5,15 +5,22 @@ on:
jobs:
deploy:
runs-on: nix
runs-on: nixos
steps:
- uses: actions/checkout@v4
- name: Setup deploy SSH key
run: |
mkdir -p ~/.ssh
install -m 600 /dev/stdin ~/.ssh/id_ed25519 <<< "${{ secrets.DEPLOY_SSH_KEY }}"
ssh-keyscan -H db-host.gebos.internal app-host.gebos.internal ingest.ge-bos.de >> ~/.ssh/known_hosts 2>/dev/null || true
# Dedicated key file so we never overwrite the runner user's own key.
install -m 600 /dev/stdin ~/.ssh/gebos_deploy <<< "${{ secrets.DEPLOY_SSH_KEY }}"
ssh-keyscan -H db.gebos.online app.gebos.online ingest.gebos.online >> ~/.ssh/known_hosts 2>/dev/null || true
# The gitea-runner is a systemd DynamicUser whose passwd home is `/`,
# so ssh expands `~` to `/.ssh/...` rather than $HOME. deploy-rs passes
# ssh opts with no shell, so it can't expand $HOME itself. Build the
# absolute paths here (bash expands $HOME) and hand them to deploy-rs
# via --ssh-opts; they append to nix/deploy.nix's path-free sshOpts.
echo "DEPLOY_SSH_OPTS=-i $HOME/.ssh/gebos_deploy -o UserKnownHostsFile=$HOME/.ssh/known_hosts" >> "$GITHUB_ENV"
- name: Build all closures
run: nix flake check --no-build
@@ -22,10 +29,10 @@ jobs:
# so it must land first. App-host's Supabase compose stack expects the
# schemas to exist. mqtt-ingest only needs the network paths to db-host.
- name: Deploy db-host
run: nix run github:serokell/deploy-rs -- .#db-host --skip-checks
run: nix run github:serokell/deploy-rs -- --ssh-opts "$DEPLOY_SSH_OPTS" .#db-host --skip-checks
- name: Deploy app-host
run: nix run github:serokell/deploy-rs -- .#app-host --skip-checks
run: nix run github:serokell/deploy-rs -- --ssh-opts "$DEPLOY_SSH_OPTS" .#app-host --skip-checks
- name: Deploy mqtt-ingest
run: nix run github:serokell/deploy-rs -- .#mqtt-ingest --skip-checks
run: nix run github:serokell/deploy-rs -- --ssh-opts "$DEPLOY_SSH_OPTS" .#mqtt-ingest --skip-checks
+1
View File
@@ -14,3 +14,4 @@ secrets.env
# age private keys — never commit (public keys live in .sops.yaml)
*.age
keys.txt
data
+5 -5
View File
@@ -10,17 +10,17 @@
keys:
# ---- developers (laptops, hardware keys) ----
- &dev_lars age1TODO_lars_personal_age_pubkey_replace_me_before_first_real_secret
- &dev_lars age1cx8ul285kjkzmnhw6skdstnzrxnnme4xkflknzn7yhv52fgxqevqkd66cn
# ---- hosts (derived from each host's ssh_host_ed25519_key.pub via ssh-to-age) ----
- &host_db age1TODO_db_host_age_pubkey_replace_me
- &host_app age1TODO_app_host_age_pubkey_replace_me
- &host_ingest age1TODO_mqtt_ingest_host_age_pubkey_replace_me
- &host_db age1wpl3vz60tlt880p5fmfedr8c59dm4kf0czsacv0w5my706twuf5q9p0x43
- &host_app age1g57pznep69nlyv3sltz6k0sml47v68m03gh68jkqwcq4jdx68vtsvnefq0
- &host_ingest age190gu75rf3ra89mhk27xe3tv87tad087altqhugjlhkerqwe2jfqsnu738d
creation_rules:
# Supabase compose stack → app-host only.
- path_regex: nix/secrets/secrets\.yaml$
encrypted_regex: ^(supabase_|ingester_|postgres_admin_)
encrypted_regex: ^(supabase_|ingester_|postgres_admin_|emqx_)
key_groups:
- age:
- *dev_lars
+157 -5
View File
@@ -9,15 +9,96 @@ and CI/CD pipelines for the project. Everything is one `flake.nix`.
Three hosts (see issue #21 for the full design discussion):
| Host | Role | Public hostname |
| ------------- | ----------------------------------------- | ---------------------- |
| `mqtt-ingest` | HiveMQ CE + Go MQTT→Postgres ingester | `ingest.ge-bos.de` |
| `db-host` | Postgres 17 + TimescaleDB (telemetry+auth)| internal only |
| `app-host` | Caddy + Kong + Supabase (compose) + SPA | `app.ge-bos.de`, `api.ge-bos.de` |
| Host | Role | Public hostname | Private IP |
| ------------- | ----------------------------------------- | ---------------------- | ---------- |
| `mqtt-ingest` | EMQX broker + Go MQTT→Postgres ingester | `ingest.gebos.online` | `10.0.0.4` |
| `db-host` | Postgres 17 + TimescaleDB (telemetry+auth)| `db.gebos.online` (SSH only) | `10.0.0.2` |
| `app-host` | Caddy + Kong + Supabase (compose) + SPA | `app.gebos.online`, `api.gebos.online` | `10.0.0.3` |
Public REST surface is PostgREST + SQL `/rpc/` functions, fronted by Kong, TLS-terminated by Caddy.
Supabase Studio is bound to `127.0.0.1` on `app-host` — reach it with `ssh -L 3000:127.0.0.1:3000 app-host`.
### Networking
There is no private DNS. Each host has a public hostname (used for `deploy-rs`
SSH access from the Gitea runner, which is **not** on the private network, plus
TLS ingress where applicable), and a static `10.0.0.0/8` IP used for all
host-to-host traffic:
- `db.gebos.online` is for SSH/deploy only — Postgres is never exposed publicly.
- `app-host` and `mqtt-ingest` reach Postgres at `10.0.0.2:5432` over the private
network.
- `db-host` only accepts Postgres (port 5432) from `10.0.0.0/8` (firewall rule
in `nix/hosts/db-host.nix`).
## MQTT ingest path
How a sender device's telemetry reaches Postgres, and the reasoning behind each
choice. The device firmware is the fixed end of this contract, so the rest of
the stack is built to match it rather than the other way around.
```
device ──MQTTS:8883──▶ EMQX (native TLS) ──MQTT:1883 loopback──▶ ingester ──▶ Postgres
acrios/<IMSI>/<metric> authn + ACL IMSI→tenant
```
### Broker: EMQX in the official container, deny-by-default auth
EMQX runs as the official `emqx/emqx` image via `oci-containers` with host
networking (`nix/modules/gebos-emqx.nix`). Authentication uses the built-in
database, seeded from a `users.csv` rendered by sops-nix
(`services.gebos.secrets.emqx`) and bootstrapped on first start; the bootstrap
only inserts users that don't exist yet, so a password rotation means deleting
the user (dashboard or `emqx ctl`) and restarting to re-import. Authorization
is a static `acl.conf` with `no_match = deny`. Passwords are plaintext in the
CSV because the file is already an encrypted secret at rest and `0400` at
runtime.
There are three broker users: `ingester` (subscribes the device tree), `admin`
(break-glass, `is_superuser` — bypasses the ACL), and `bender` (the shared
device account, below).
### Topic scheme: the device's `acrios/<IMSI>/…` is the source of truth
Senders publish to `acrios/<IMSI>/<metric>``mqtt_topic_base` plus the SIM's
IMSI plus the metric name. We adopted that namespace verbatim rather than
reshaping it into the `t/<tenant>/d/<device>/…` form the schema originally
imagined, because the firmware can't emit our `tenant_id`/`device_id` **UUIDs**
— it only knows its IMSI. So the IMSI is the natural device key, and the
ingester (subscribed to `acrios/#`) will resolve `IMSI → (tenant_id,
device_id)` via a **device registry** before inserting into `public.telemetry`.
That registry table and the ingester's topic parsing are still TODO — the
ingester is currently a stub.
### Device auth: shared user now (Option A), per-device later (Option B)
The sender logs in with a single shared account (`bender`), authorized to
publish/subscribe under `acrios/#`. Tenant isolation is therefore enforced
**downstream** by the ingester's IMSI registry, not at the broker — any device
could publish under any IMSI. That's an accepted trade-off for a small trusted
fleet, and it gets data flowing without per-device provisioning.
The production answer (**Option B**, a TODO in `nix/modules/gebos-emqx.nix`)
is one broker user per device with `username == IMSI`, scoped to
`acrios/${{username}}/#` so the broker itself prevents a device from spoofing
another's IMSI. It's not wired up because it needs a firmware change
(`mqtt_user = <IMSI>`) and a per-device password provisioning flow. Note the
firmware's client-id (`acrcv-<IMSI>`) carries a prefix the topic doesn't, so
per-device scoping must key on username, not `${{clientid}}`.
### TLS: EMQX terminates natively, certs via ACME HTTP-01
EMQX terminates TLS itself on `:8883`; the plain MQTT listener stays on
loopback for the co-located ingester. Certificates come from `security.acme`
using the **HTTP-01** challenge: lego's built-in standalone server answers on
`:80` (nothing else listens there — no Caddy on this host), so no DNS API
secrets are needed. `mqtt-ingest` opens **80 + 8883** only. The cert's
`postRun` hook installs `fullchain.pem`/`privkey.pem` at fixed paths in the
broker state dir, and EMQX re-reads the PEMs from disk (~every 120 s), so
renewals hot-reload without a restart or dropped connections. The key is RSA
(`keyType = "rsa2048"`) because embedded sender TLS stacks often can't do
ECDSA. Caddy remains only on app-host (stock build, HTTP sites).
## Repo layout
```
@@ -44,10 +125,81 @@ Supabase compose, Kong, Caddy, ingester, Vite dev server). NixOS required.
## Deployment
There are two distinct phases. **Initial provisioning** turns a blank box into a
NixOS host (`nixos-anywhere`, run once per machine). **Updates** push new
closures to a host that already runs NixOS (`deploy-rs`, run on every change).
### SSH key setup (do this first)
Both phases authenticate over SSH with `~/.ssh/larsnolden`, which is
passphrase-protected. Load it into an `ssh-agent` once so the deploy tools can
reuse it without prompting:
```fish
eval (ssh-agent -c) # bash/zsh: eval "$(ssh-agent -s)"
ssh-add ~/.ssh/larsnolden # enter the passphrase once
ssh-add -l # confirm the key is loaded
```
This is **required** for `deploy-rs`, not just a convenience: with
`magicRollback = true` (see `nix/deploy.nix`) activation opens two concurrent SSH
connections — the activation command and a rollback waiter. Without an agent,
both race to read the passphrase from the terminal, one loses, and the deploy
fails with `Permission denied (publickey,keyboard-interactive)` even though
manual SSH and the copy step work. The agent serves the key to every connection,
so no prompt is needed.
### Initial provisioning (`nixos-anywhere`)
`deploy-rs` only *updates* a machine that already runs NixOS — it copies a
prebuilt closure and activates it. A fresh box (e.g. a stock Debian image with
only a `root` user) has no Nix store and no NixOS generation to switch to, so
`deploy-rs` fails with `nix-store: command not found`. Use
[`nixos-anywhere`](https://github.com/nix-community/nixos-anywhere) to install
NixOS over SSH first; after that, `deploy-rs` takes over for all subsequent
deploys.
`nixos-anywhere` SSHes in as `root`, kexecs into an in-memory NixOS installer,
partitions and formats the disk per the host's [`disko`](https://github.com/nix-community/disko)
config, installs `nixosConfigurations.<host>`, and reboots into NixOS. **This
wipes the target disk.**
Prerequisites, per host, before running it:
1. **A real disk layout.** Hosts currently import the fictional
`nix/hosts/placeholder-hardware.nix` (it only exists so `nix flake check`
evaluates). Replace that import with a `disko` config describing the actual
disk device (`/dev/sda` vs `/dev/vda`/nvme) and firmware (UEFI vs legacy
BIOS). `disko` replaces the hand-generated `hardware-configuration.nix`.
2. **Root SSH access** to the box. The `deploy` user and its authorized keys are
created by `nix/hosts/common.nix` during the install, so deploy-rs access
works automatically once NixOS is up.
3. **Host secrets key** present so sops-nix can decrypt at first boot — see
[`nix/secrets/README.md`](nix/secrets/README.md). Otherwise services that
read `/run/secrets/*` (e.g. the ingester) fail to start after reboot.
Then, from the repo root:
```
# installs NixOS onto the target, wiping its disk
nix run github:nix-community/nixos-anywhere -- \
--flake .#mqtt-ingest root@ingest.gebos.online
```
Repeat with `.#db-host root@db.gebos.online` and `.#app-host root@app.gebos.online`.
Provision `db-host` first if you intend to deploy updates immediately afterward
(see the ordering note below). Once a host has rebooted into NixOS, never run
`nixos-anywhere` against it again — use `deploy-rs`.
### Updates (`deploy-rs`)
`deploy-rs` from a Gitea Actions runner on push to `main`. Closures are built
once, copied to each host, activated with auto-rollback. Order: `db-host`
`app-host``mqtt-ingest`.
Running it by hand needs the key loaded into an `ssh-agent` first — see
[SSH key setup](#ssh-key-setup-do-this-first) above.
```
nix run github:serokell/deploy-rs -- .#db-host # one host
nix run github:serokell/deploy-rs -- . # all hosts
+600
View File
@@ -0,0 +1,600 @@
-- ============================================================================
-- GebOS Schema V1
-- Target: Postgres 17 + TimescaleDB
--
-- Scope: UVI pilot per § 6a HeizkostenV, methodology per UBA-Leitfaden
-- CLIMATE CHANGE 69/2021 (all six modules). Annual billing is a separate
-- epic and deliberately not modeled here.
--
-- Requirement IDs (UVI-DAT-*, UVI-FUN-*, UVI-NFR-*, UVI-MET-*) refer to the
-- Notion page "UVI Anforderungen (Pilot)", which is the source of truth.
--
-- Conventions
-- * lowercase snake_case everywhere — no quoted identifiers, ever.
-- * Surrogate UUID primary keys, except time-series tables where the
-- composite natural key carries meaning (see measurement).
-- * timestamptz for all instants; date for civil dates (tenancy, prices).
-- * jsonb, never json (binary, indexable, deduplicated keys).
-- * created_at on every table for ops forensics.
-- * ON DELETE RESTRICT as default policy: telemetry and tenancy data must
-- never disappear as a side effect of deleting a parent row. Deletions
-- are rare, deliberate, admin-level operations.
-- ============================================================================
CREATE EXTENSION IF NOT EXISTS timescaledb;
CREATE EXTENSION IF NOT EXISTS pgcrypto; -- gen_random_uuid()
CREATE EXTENSION IF NOT EXISTS btree_gist; -- exclusion constraint on occupancy
-- ---------------------------------------------------------------------------
-- Enums
--
-- Enums are used where the value set is fixed by law or by the product
-- itself and changes require a deliberate migration (which is a feature:
-- a new fuel type SHOULD force someone to also add its CO2 factor and
-- price rows). Where the value set is operational content that admins
-- curate at runtime (saving tips), a table is used instead of an enum.
-- ---------------------------------------------------------------------------
-- Drives CO2 emission factors (GEG Anlage 9) and cost estimation (Module 5).
-- One fuel per building is the pilot assumption. District heating fuel-mix
-- disclosure (§ 6a Abs. 3) belongs to annual billing; if it later requires
-- per-network fuel compositions, promote this enum to a table with a
-- composition child table — the FK sites below are the only touch points.
CREATE TYPE fuel_type AS ENUM (
'natural_gas',
'heating_oil',
'district_heating',
'heat_pump_electricity',
'wood_pellets',
'other'
);
-- The SEMANTIC kind of a reading. This is deliberately separate from the
-- physical unit: heating energy and hot-water energy are both kWh, yet the
-- UVI must never mix them (Modules 1 and 2 render them separately, and
-- UVI-FUN-14 treats them differently outside the heating period). The unit
-- says how a number is measured; the kind says what it means.
--
-- Diagnostic kinds (temperatures, error flags) are included because OMS
-- telegrams carry them anyway and they are the raw material for device
-- health monitoring — cheap to store now, expensive to backfill later.
CREATE TYPE measurement_kind AS ENUM (
'heating_energy', -- kWh, heat cost allocator / heat meter
'hot_water_energy', -- kWh
'hot_water_volume', -- m3; some installations meter volume and
-- convert to energy downstream
'cold_water_volume', -- m3; not needed for the UVI but commonly on
-- the same radio and trivially captured
'flow_temperature', -- °C, diagnostics
'return_temperature', -- °C, diagnostics
'error_flags', -- OMS status/error register
'other'
);
-- Two roles are enough for the pilot: tenants see exactly one apartment
-- dashboard (UVI-NFR-03), admins see the read-only overview of everything
-- (UVI-FUN-17). A role enum rather than a boolean because the role set is
-- known to grow (landlord read-only view is an announced post-pilot epic)
-- and because UVI-NFR-07 already treats "admin" as a first-class concept
-- with its own audit requirements.
CREATE TYPE user_role AS ENUM ('tenant', 'admin');
CREATE TYPE saving_tip_category AS ENUM ('heating', 'hot_water');
-- ===========================================================================
-- 1. BUILDING & TENANCY
--
-- The physical and legal world: buildings contain apartments, apartments
-- are occupied by tenants over time periods. Everything the UVI displays
-- is ultimately scoped by this layer — a measurement only becomes a
-- "tenant's consumption" through sensor -> apartment -> occupancy.
-- ===========================================================================
CREATE TABLE building (
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
-- Address is structured (not one text blob) because the admin
-- dashboard groups by building address (UVI-FUN-17) and because
-- postal_code is the likely input for automatic weather-station
-- assignment if that open decision lands on "auto".
street text NOT NULL,
house_number text NOT NULL,
postal_code text NOT NULL,
city text NOT NULL,
-- Coordinates exist for exactly one consumer: nearest-weather-station
-- resolution. They are NOT used to locate devices — device placement
-- is human-readable installer text on the device rows themselves.
lat numeric(9,6),
lng numeric(9,6),
-- The building's energy carrier. Chosen at building level (not per
-- apartment, not per measurement) because the boiler / district
-- heating connection is a building-level fact, and both the CO2
-- module and the cost module resolve their factors through it.
fuel_type fuel_type NOT NULL,
-- UVI-MET-03. The UBA/GEG comparison metrics (Bandtacho thresholds,
-- in-house kWh/m²) are defined against Gebäudenutzfläche (AN), a
-- technical energy-balance quantity nobody in this business has on
-- file. The sanctioned approximation is AN ≈ Wohnfläche × 1.2.
-- Stored per building, not hardcoded, so that a building with a
-- known real AN (e.g. from an energy certificate) can override it:
-- set area_factor = real_AN / sum(living_area_m2). Every specific
-- consumption in the app divides by (living_area_m2 * area_factor);
-- omitting the factor would inflate every building's apparent
-- consumption ~20% and shift Bandtacho classes.
area_factor numeric(4,2) NOT NULL DEFAULT 1.20
CHECK (area_factor > 0),
-- Which DWD station's heating degree days contextualize this
-- building's consumption (Module 1 footnote, UVI-FUN-07). Nullable
-- and manually assigned in the pilot: the auto-vs-manual assignment
-- question is an explicitly open decision, and the schema must not
-- presume its answer. A NULL here means the weather footnote renders
-- as "not available" (UVI-FUN-15) — never as a silent default station.
weather_station_id uuid, -- FK added after weather_station is defined
-- Building-level operational remarks: key-box code, basement access,
-- contact person. Device mounting locations do NOT belong here —
-- they live on the device rows (see gateway/sensor.installation_note).
note text,
created_at timestamptz NOT NULL DEFAULT now()
);
CREATE TABLE apartment (
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
building_id uuid NOT NULL REFERENCES building(id),
-- Human-readable unit designation ("2. OG links", "WE 04"). Unique
-- within a building so admin views and installer workflows have an
-- unambiguous handle.
label text NOT NULL,
-- UVI-DAT-01, and NOT NULL on purpose: the in-house comparison
-- (Module 2) is kWh per m² — an apartment without an area cannot
-- participate in the product's core comparison at all, so it is
-- invalid data, not missing data.
living_area_m2 numeric(7,2) NOT NULL CHECK (living_area_m2 > 0),
created_at timestamptz NOT NULL DEFAULT now(),
UNIQUE (building_id, label)
);
-- Application-level user profile. Identity — email address, magic-link
-- issuance, session state — is owned entirely by Supabase Auth (GoTrue)
-- in the auth schema (UVI-NFR-08: passwordless only). This table stores
-- only what the application adds on top. The PK IS the auth.users id
-- (same uuid, no separate surrogate) so that auth.uid() in RLS policies
-- joins directly without a mapping table.
CREATE TABLE app_user (
id uuid PRIMARY KEY, -- = auth.users.id
role user_role NOT NULL DEFAULT 'tenant',
display_name text,
created_at timestamptz NOT NULL DEFAULT now()
);
-- Tenancy periods (UVI-DAT-04). This table is the legal heart of the
-- data model, for two reasons:
--
-- 1. Mieterwechsel correctness: § 6a comparisons to the previous month /
-- previous-year month are only permitted against the SAME tenant's
-- consumption. The dashboard decides whether to render a comparison
-- by checking that both months fall inside one occupancy row. Without
-- periods, a new tenant would be shown (and judged against) their
-- predecessor's behavior.
--
-- 2. Privacy scope: a tenant may only ever see measurements taken during
-- their own tenancy (UVI-NFR-03). The RLS sketch at the bottom of this
-- file enforces that by joining measurement timestamps against this
-- table's date range — history before move-in is structurally
-- invisible, not just filtered in application code.
--
-- valid_to IS NULL means "current tenant". The exclusion constraint makes
-- overlapping tenancies for one apartment a constraint violation rather
-- than a bug class: the database itself guarantees that any timestamp
-- maps to at most one responsible tenant.
CREATE TABLE occupancy (
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
apartment_id uuid NOT NULL REFERENCES apartment(id),
user_id uuid NOT NULL REFERENCES app_user(id),
valid_from date NOT NULL,
valid_to date,
created_at timestamptz NOT NULL DEFAULT now(),
CHECK (valid_to IS NULL OR valid_to > valid_from),
EXCLUDE USING gist (
apartment_id WITH =,
daterange(valid_from, COALESCE(valid_to, 'infinity'::date), '[)') WITH &&
)
);
-- ===========================================================================
-- 2. DEVICES & INGEST
--
-- The radio world: gateways listen, meters broadcast, frames arrive.
-- Design stance: wM-Bus at 868 MHz is a BROADCAST medium. A gateway does
-- not "own" the sensors it hears — two gateways in one building may both
-- receive the same meter, and every gateway will receive meters that are
-- not ours (neighbors, unprovisioned devices). The schema therefore:
-- * ties gateways to buildings (where they are installed), never to
-- sensors;
-- * records which gateway heard a frame on the frame itself, because
-- that is a fact about the reception event, not about the meter;
-- * accepts and stores frames from unknown senders instead of
-- dropping them at the door.
-- ===========================================================================
CREATE TABLE gateway (
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
-- IMEI identifies the cellular modem and is unique per device, but it
-- is a hardware serial, not an identity: a defective gateway swapped
-- in the field is, to the business, "the same listening post" with a
-- new IMEI. A surrogate PK keeps history intact across swaps; the
-- unique index still supports lookup-by-IMEI at ingest time.
imei varchar(15) NOT NULL UNIQUE,
imsi varchar(15), -- SIM identity; changes with
-- provider swaps, so informative
-- only, never a key
building_id uuid NOT NULL REFERENCES building(id),
label text, -- short admin handle, "GW Keller"
-- Free-text installer note answering "where exactly is this thing?":
-- e.g. "mounted above the first ceiling panel on level 1". Kept per
-- device (not on a shared location entity) so that editing one
-- device's note can never silently rewrite another's.
installation_note text,
created_at timestamptz NOT NULL DEFAULT now()
);
-- A device profile: one row per (manufacturer, model) of meter we know
-- how to decode. Exists so that parsing knowledge attaches to the KIND
-- of device rather than to each physical unit — provisioning meter #500
-- of a known model requires zero new parser configuration. This is the
-- schema-level expression of the product strategy "works with any open
-- OMS sensor": supporting a new vendor means inserting rows here and in
-- value_parser, not shipping code.
CREATE TABLE device_type (
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
manufacturer text NOT NULL, -- OMS M-field, e.g. 'QDS'
model text NOT NULL,
medium text, -- OMS medium byte, informative
created_at timestamptz NOT NULL DEFAULT now(),
UNIQUE (manufacturer, model)
);
-- Extraction rules: how to pull ONE value out of a decoded telegram of a
-- given device type. A telegram routinely carries several values we care
-- about (energy total, volume, temperatures, error register), hence
-- 1 device_type : N parsers. Each parser declares:
-- * where the value sits (json_path into the decoded telegram),
-- * how to normalize it (parser_instructions: scaling, offsets),
-- * what it MEANS (kind) and in what unit it is expressed.
-- The kind declared here is what stamps every measurement row with its
-- heating/hot-water semantics — the parser is the single place where
-- raw vendor bytes acquire domain meaning.
--
-- unit is plain text ('kWh', 'm3', '°C') rather than a lookup table:
-- a three-row table adds a join and an id without adding information.
-- Reintroduce a unit table only if units ever need behavior or metadata
-- (conversion factors, display localization).
CREATE TABLE value_parser (
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
device_type_id uuid NOT NULL REFERENCES device_type(id),
json_path text NOT NULL,
parser_instructions jsonb,
kind measurement_kind NOT NULL,
unit text NOT NULL,
created_at timestamptz NOT NULL DEFAULT now(),
UNIQUE (device_type_id, kind, json_path)
);
CREATE TABLE sensor (
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
-- The meter's own identity as broadcast in its telegrams (OMS
-- secondary address / serial). This is the value ingest matches on
-- to attribute an incoming frame to a known sensor.
oms_id text NOT NULL UNIQUE,
device_type_id uuid NOT NULL REFERENCES device_type(id),
-- The device-to-apartment assignment (UVI-FUN-02). Direction matters:
-- an apartment has MANY devices (a heat cost allocator per radiator
-- plus a hot-water meter is the normal case), a device sits in at
-- most one apartment. NULLable by design — in the pilot this
-- assignment is performed manually in the database after physical
-- installation, so a sensor legitimately exists in a provisioned-
-- but-unassigned state. Its measurements are stored regardless and
-- become tenant-visible once the assignment is made.
apartment_id uuid REFERENCES apartment(id),
-- AES-128 key for OMS telegram decryption. Plaintext here is a
-- pilot-grade decision to keep moving; before production, move key
-- material to a dedicated mechanism (pgsodium / Vault) — the column
-- is the interface, the storage hardening is an ops task.
decrypt_key text,
installed_at date,
-- Same rationale as gateway.installation_note: "which radiator,
-- which room, how to reach it" is per-device installer knowledge.
installation_note text,
created_at timestamptz NOT NULL DEFAULT now()
);
-- Note what sensor deliberately does NOT have: a gateway reference (the
-- radio is broadcast; reception routing lives on received_payload) and a
-- location/building reference (derived via apartment -> building; a
-- second, independent path would allow the two to contradict each other).
-- Raw reception log: every frame any of our gateways heard, verbatim.
-- This table exists for auditability and reprocessing — if a parser had
-- a bug, or a new device_type is added for meters we were already
-- hearing, history can be re-parsed from here. Storage is the price of
-- never losing data to a software mistake.
CREATE TABLE received_payload (
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
gateway_id uuid NOT NULL REFERENCES gateway(id),
-- Attribution to a known sensor happens at parse time and is
-- NULLABLE: on a shared radio band we routinely receive frames from
-- meters that are not ours. Those are stored (they may become ours
-- — e.g. a meter provisioned after installation) and flagged via
-- parse_status rather than rejected at ingest.
sensor_id uuid REFERENCES sensor(id),
-- Both ends of the transport hop. sent = gateway clock at upload,
-- received = server clock at arrival; their divergence is the
-- cheapest possible gateway-health signal (clock drift, buffering
-- backlog after connectivity loss).
timestamp_sent timestamptz,
timestamp_received timestamptz NOT NULL DEFAULT now(),
raw_payload jsonb NOT NULL,
-- Ingest pipeline state machine. 'unknown_sensor' is a normal,
-- expected state, not an error — see attribution note above.
parse_status text NOT NULL DEFAULT 'pending'
CHECK (parse_status IN ('pending','parsed','unknown_sensor','error')),
created_at timestamptz NOT NULL DEFAULT now()
);
CREATE INDEX ON received_payload (sensor_id, timestamp_received DESC);
-- Partial index: the parser worker's queue query ("give me unprocessed
-- frames") stays fast no matter how large the historical log grows.
CREATE INDEX ON received_payload (parse_status)
WHERE parse_status IN ('pending','error');
-- Raw-frame retention is an open decision. When made, implement it as a
-- scheduled purge (or convert this table to a hypertable and attach a
-- Timescale retention policy) — do NOT cascade-delete from measurement;
-- measurement.raw_payload_id is nullable precisely so lineage can be
-- severed by retention without touching the measurements themselves.
-- ===========================================================================
-- 3. MEASUREMENT (time series)
--
-- The product's ground truth: one row per (meter, kind, instant) reading.
-- Everything the UVI shows is computed from these rows at query time —
-- no persisted aggregates (UVI-DAT-06). That rule keeps a single source
-- of truth and makes bug fixes retroactive for free: correct a parser,
-- re-parse, and every dashboard number is correct.
-- ===========================================================================
CREATE TABLE measurement (
sensor_id uuid NOT NULL REFERENCES sensor(id),
-- The METER's own reading time, extracted from the telegram — not
-- the reception time. wM-Bus meters repeat frames and gateways
-- buffer during connectivity loss, so arrival time can trail reading
-- time by minutes to days; all consumption math (monthly windows,
-- year-over-year) must bind to when the meter measured.
measured_at timestamptz NOT NULL,
kind measurement_kind NOT NULL,
-- NOT NULL: a reading without a value is not a reading. Missing data
-- is represented by the ABSENCE of rows for a period, and the UI
-- renders that as "not available" (UVI-FUN-15) — never by storing 0
-- or NULL, both of which would poison aggregations silently.
value numeric NOT NULL,
-- Denormalized from value_parser at write time so a row is
-- self-describing even if parser configuration later changes.
unit text NOT NULL,
-- Lineage to the exact raw frame this value was parsed from.
-- Nullable so raw-frame retention can delete old frames without
-- invalidating measurements (see received_payload).
raw_payload_id uuid REFERENCES received_payload(id),
created_at timestamptz NOT NULL DEFAULT now(),
-- Composite natural key, doing three jobs at once:
-- 1. TimescaleDB requires the partitioning column in any unique
-- constraint — a bare uuid PK is incompatible with a hypertable.
-- 2. Dedup: meters broadcast the same reading repeatedly by design.
-- Ingest uses INSERT .. ON CONFLICT DO NOTHING against this key,
-- so repeats collapse to no-ops instead of duplicate rows that
-- would double-count consumption.
-- 3. It IS the dominant access path (this sensor, this kind, this
-- time range), so the PK index serves the dashboard queries and
-- no secondary index is needed for the pilot.
PRIMARY KEY (sensor_id, kind, measured_at)
);
-- Monthly chunks match the product's natural query grain (the UVI is a
-- monthly report over a 13-month window).
SELECT create_hypertable('measurement', by_range('measured_at', INTERVAL '1 month'));
-- If the 13-month dashboard aggregations ever get slow at scale, the
-- sanctioned escape hatch is a Timescale continuous aggregate: it is
-- machine-maintained derived state over this table, which arguably
-- honors the intent of "no persisted aggregates" (no hand-maintained
-- second truth). Adopt it when measured, not preemptively.
-- ===========================================================================
-- 4. REFERENCE DATA
--
-- External regulatory and environmental facts the UVI computes against
-- (UVI-DAT-07..11). Two shared design rules:
--
-- * Versioning by validity range wherever the source can change
-- (CO2 factors, prices): the question "what number was shown to the
-- tenant in March?" must remain answerable after an update, both for
-- tenant trust and for the § 7 Kürzungsrecht audit scenario.
-- * source columns record provenance, because every one of these
-- numbers is a claim about a legal document or an external dataset.
-- ===========================================================================
-- Bandtacho class boundaries (Module 3). GEG Anlage 10 defines ANNUAL
-- kWh/m² class limits; the UBA-Leitfaden (Tabelle 1) splits them into
-- per-calendar-month thresholds so a single month's consumption can be
-- classified. Hence one row per (class, month): 9 classes × 12 months.
CREATE TABLE efficiency_class_threshold (
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
class text NOT NULL CHECK (class IN
('A+','A','B','C','D','E','F','G','H')),
month smallint NOT NULL CHECK (month BETWEEN 1 AND 12),
-- Upper bound of the class for that month. 'H' is open-ended in the
-- source; store a sentinel high value rather than NULL so that
-- classification is a simple "first class whose bound >= value"
-- scan with no NULL special-casing.
max_kwh_per_m2 numeric(8,3) NOT NULL,
source text NOT NULL DEFAULT 'UBA CC 69/2021 Tab. 1 / GEG Anl. 10',
UNIQUE (class, month)
);
-- CO2 factors per fuel (Module 6), GEG Anlage 9. Versioned because the
-- annex gets amended; emissions shown for a given month use the factor
-- valid in that month.
CREATE TABLE co2_emission_factor (
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
fuel_type fuel_type NOT NULL,
g_co2_per_kwh numeric(8,2) NOT NULL,
valid_from date NOT NULL,
valid_to date, -- NULL = currently valid
source text NOT NULL DEFAULT 'GEG Anlage 9',
UNIQUE (fuel_type, valid_from)
);
-- Energy prices per fuel (Module 5: cost estimate = kWh × price).
-- Versioned for the same auditability reason, plus created_by: price
-- maintenance is a manual admin process (owner and cadence still an
-- open decision), and attributing each price row makes that process
-- accountable from day one regardless of how the decision lands.
CREATE TABLE energy_price (
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
fuel_type fuel_type NOT NULL,
ct_per_kwh numeric(8,3) NOT NULL,
valid_from date NOT NULL,
valid_to date,
source text,
created_by uuid REFERENCES app_user(id),
created_at timestamptz NOT NULL DEFAULT now(),
UNIQUE (fuel_type, valid_from)
);
-- DWD stations we import heating-degree-day data for. Modeled as its own
-- entity (not columns on building) because many buildings share a
-- station and the HDD import job iterates stations, not buildings.
CREATE TABLE weather_station (
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
dwd_id text NOT NULL UNIQUE, -- DWD's station identifier
name text NOT NULL,
lat numeric(9,6),
lng numeric(9,6),
created_at timestamptz NOT NULL DEFAULT now()
);
ALTER TABLE building
ADD CONSTRAINT building_weather_station_fk
FOREIGN KEY (weather_station_id) REFERENCES weather_station(id);
-- Monthly heating degree days per station (VDI 3807, DWD open data).
-- Consumed ONLY as display context — the Module 1 footnote "this month
-- was N% colder/warmer than the same month last year". Actual weather
-- normalization of consumption values is explicitly out of scope for
-- the UVI (it belongs to annual billing) and measurement values are
-- never adjusted by this data.
-- Operational note: a monthly DWD import fills this table; if a month
-- is missing, the footnote renders "not available" (UVI-FUN-15). The
-- import's timing sits inside the NFR-05 chain (data visible by the
-- 5th working day).
CREATE TABLE heating_degree_days (
station_id uuid NOT NULL REFERENCES weather_station(id),
year smallint NOT NULL,
month smallint NOT NULL CHECK (month BETWEEN 1 AND 12),
hdd numeric(7,1) NOT NULL,
PRIMARY KEY (station_id, year, month)
);
-- Savings tips library (Module 4, UVI-FUN-11). A table, not content in
-- code, because tips are editorial material admins curate. The category
-- drives seasonal selection (UVI-FUN-14): heating tips during the
-- heating period, hot-water tips outside it. active supports retiring
-- tips without deleting them (a tip already shown in a past month
-- should remain resolvable).
CREATE TABLE saving_tip (
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
category saving_tip_category NOT NULL,
title text NOT NULL,
body text NOT NULL,
link_url text, -- deep link to advisory content
-- (verbraucherzentrale, co2online)
active boolean NOT NULL DEFAULT true,
created_at timestamptz NOT NULL DEFAULT now()
);
-- ===========================================================================
-- 5. TENANT ISOLATION (UVI-NFR-03) — enforcement sketch
--
-- Isolation is enforced IN the database via row-level security, not only
-- in the Go backend: with Supabase Auth issuing JWTs, auth.uid() is
-- available inside policies, and a backend bug can then never leak a
-- neighbor's data — the worst case becomes an empty result.
--
-- Example policy shape for measurement (enable analogously per table):
--
-- ALTER TABLE measurement ENABLE ROW LEVEL SECURITY;
-- CREATE POLICY tenant_reads_own ON measurement FOR SELECT
-- USING (
-- sensor_id IN (
-- SELECT s.id
-- FROM sensor s
-- JOIN occupancy o ON o.apartment_id = s.apartment_id
-- WHERE o.user_id = auth.uid()
-- AND daterange(o.valid_from,
-- COALESCE(o.valid_to, 'infinity'::date),
-- '[)') @> measured_at::date
-- )
-- );
--
-- The occupancy-window predicate is the important part: a tenant sees
-- only measurements from their own tenancy period, so pre-move-in
-- history is structurally invisible (the Mieterwechsel rule enforced at
-- the storage layer). Admins get a separate permissive policy scoped to
-- role = 'admin'; admin reads are additionally logged at the application
-- layer (UVI-NFR-07).
--
-- Deliberately NOT in the schema:
-- * The small-building anonymity rule (UVI-FUN-09: with ≤ 4 units,
-- show the mean of the three thriftiest households instead of the
-- spectrum) — that is presentation logic in the in-house comparison
-- query, driven by count(apartment) per building at query time.
-- * Heating-period vs. off-season behavior (UVI-FUN-14) — pure display
-- logic; the data model is season-agnostic.
-- ===========================================================================
Generated
+47 -5
View File
@@ -20,6 +20,26 @@
"type": "github"
}
},
"disko": {
"inputs": {
"nixpkgs": [
"nixpkgs"
]
},
"locked": {
"lastModified": 1781152676,
"narHash": "sha256-RxWs5ND31KzTG7wvMM+PMfUjyNpmIEr999lqNARaM5o=",
"owner": "nix-community",
"repo": "disko",
"rev": "ff8702b4de27f72b4c78573dfb89ec74e36abdf1",
"type": "github"
},
"original": {
"owner": "nix-community",
"repo": "disko",
"type": "github"
}
},
"flake-compat": {
"flake": false,
"locked": {
@@ -87,16 +107,16 @@
},
"nixpkgs_2": {
"locked": {
"lastModified": 1779560665,
"narHash": "sha256-tpyBcxPpcQb8ukyNF7DoCwfSY3VPsxHoYwj00Cayv5o=",
"lastModified": 1782233679,
"narHash": "sha256-QyuGP5+QOtmXpy4i2X4DhBVBaySBdDKQEhqKcphcp34=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "64c08a7ca051951c8eae34e3e3cb1e202fe36786",
"rev": "667d5cf1c59585031d743c78b394b0a647537c35",
"type": "github"
},
"original": {
"owner": "NixOS",
"ref": "nixos-unstable",
"ref": "nixos-26.05",
"repo": "nixpkgs",
"type": "github"
}
@@ -119,10 +139,12 @@
"root": {
"inputs": {
"deploy-rs": "deploy-rs",
"disko": "disko",
"flake-parts": "flake-parts",
"nixpkgs": "nixpkgs_2",
"process-compose-flake": "process-compose-flake",
"services-flake": "services-flake"
"services-flake": "services-flake",
"sops-nix": "sops-nix"
}
},
"services-flake": {
@@ -140,6 +162,26 @@
"type": "github"
}
},
"sops-nix": {
"inputs": {
"nixpkgs": [
"nixpkgs"
]
},
"locked": {
"lastModified": 1782165805,
"narHash": "sha256-478kKQBvK6SYTOdN2h9jhKJv94nbXRbFMfuL1WshErg=",
"owner": "Mic92",
"repo": "sops-nix",
"rev": "56b24064fdcaedca53553b1a6d607fd23b613a24",
"type": "github"
},
"original": {
"owner": "Mic92",
"repo": "sops-nix",
"type": "github"
}
},
"systems": {
"locked": {
"lastModified": 1681028828,
+5 -1
View File
@@ -2,11 +2,15 @@
description = "Gebos IoT telemetry stack on bare-metal NixOS";
inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
nixpkgs.url = "github:NixOS/nixpkgs/nixos-26.05";
flake-parts.url = "github:hercules-ci/flake-parts";
deploy-rs.url = "github:serokell/deploy-rs";
process-compose-flake.url = "github:Platonic-Systems/process-compose-flake";
services-flake.url = "github:juspay/services-flake";
disko = {
url = "github:nix-community/disko";
inputs.nixpkgs.follows = "nixpkgs";
};
sops-nix = {
url = "github:Mic92/sops-nix";
inputs.nixpkgs.follows = "nixpkgs";
+2 -3
View File
@@ -1,12 +1,11 @@
{ buildNpmPackage, lib }:
{ buildNpmPackage }:
buildNpmPackage {
pname = "gebos-frontend";
version = "0.0.0";
src = ./.;
# Filled in once package-lock.json exists.
npmDepsHash = lib.fakeHash;
npmDepsHash = "sha256-GeXgNbf94QRLNaZP8ma2vx19Vzm8jDWPeOTvE9exwm0=";
installPhase = ''
runHook preInstall
+1 -1
View File
@@ -11,7 +11,7 @@ const DEV_SUPABASE_ANON_KEY =
const SUPABASE_URL =
import.meta.env.VITE_SUPABASE_URL ??
(import.meta.env.DEV ? DEV_SUPABASE_URL : "https://api.ge-bos.de");
(import.meta.env.DEV ? DEV_SUPABASE_URL : "https://api.gebos.online");
const SUPABASE_ANON_KEY =
import.meta.env.VITE_SUPABASE_ANON_KEY ??
+1
View File
@@ -0,0 +1 @@
/// <reference types="vite/client" />
+4
View File
@@ -8,6 +8,10 @@ buildGoModule {
# Set once `go mod tidy` has produced a real go.sum.
vendorHash = null;
# The main package lives in cmd/gebos-ingester/, so `go install` names the
# binary after that directory — matching mainProgram and the systemd unit.
subPackages = [ "cmd/gebos-ingester" ];
meta = {
description = "MQTT Postgres telemetry ingester for Gebos";
mainProgram = "gebos-ingester";
+106
View File
@@ -0,0 +1,106 @@
# Gateway MQTT connectivity — debugging findings (July 2026)
Why the ACRIOS `ACR_CV_101N_W_D2` gateway (SIM7022 NB-IoT modem) could not
connect to `ingest.gebos.online:8883`, how we proved it, and what to do next.
**TL;DR: the bundled SIM lives in a private ACRIOS APN (`acrios.iot`) with no
public-internet breakout. The broker, TLS setup, and gateway firmware are all
fine.** No packet from the gateway ever reached our host; every failure
happened inside the carrier/APN network.
## Root cause
The modem's own PDP-context diagnostics (see step 6) show:
```
+CGDCONT: 0,"IP","acrios.iot","10.10.111.184"
+CGPADDR: 0,"10.10.111.184"
+CGCONTRDP: 0,5,"acrios.iot","10.10.111.184.255.255.255.0",,"10.65.0.1","10.65.0.2",,,,,1430
```
- APN `acrios.iot` — a private APN; the SIM (IMSI prefix 901-28, a global
IoT-roaming range) is provisioned for ACRIOS's walled garden only.
- Private device IP (`10.10.111.184`) and private DNS servers (`10.65.0.x`).
- Result: DNS for public names fails (`+CMQTTCONNECT: 0,25`) and raw TCP to a
public IP fails (`+CMQTTCONNECT: 0,3`). Signalling-plane features (network
registration, NITZ time sync, CSQ) all work, which made the SIM *look* fine.
- `APN = "auto"` in the Lua config is not the bug: `acrios.iot` is almost
certainly the only APN this SIM subscription allows, so overriding the APN
string alone cannot fix it.
This also explains the firmware's `acrios/` topic prefix — these gateways are
normally sold talking to ACRIOS's own MQTT cloud inside that APN.
## Debugging steps (in order, with what each eliminated)
1. **Verified the broker from the outside.** From a dev machine:
`openssl s_client -connect ingest.gebos.online:8883` (valid Let's Encrypt
RSA-2048 chain, TLS 1.2 + 1.3 OK) and
`mosquitto_pub ... -u bender -P <device pw>``CONNACK 0`, publish
accepted. ⇒ EMQX, its ACME cert, and the device credentials all work; the
problem is specific to the gateway's path.
2. **Read the serial log symptom precisely.** `AT+CMQTTCONNECT` returned `OK`
(command accepted) but the result URC never arrived before the script's
20 s timeout — indistinguishable from a TLS stall at that point.
3. **Checked EMQX for handshake errors.** `docker logs` showed nothing — but a
deliberately failed cipher probe from the dev machine *also* logged nothing,
proving EMQX's default log level swallows TLS failures. Inconclusive, not
exonerating. (`emqx ctl listeners` shutdown counters were similarly
ambiguous; `emqx ctl log set-level debug` enables visibility.)
4. **tcpdump on the ingest host** (`tcpdump -i any 'tcp port 8883'`) while the
gateway retried: zero packets from the gateway, ever. ⇒ nothing above the
IP layer (TLS version, ciphers, cert chain, EMQX config) could be the cause.
5. **Script changes to force better diagnostics** (v1.8 → v1.10, see below):
- `mqtt_timeout` 20 s → 60 s: the modem finally had time to report real
result codes instead of being abandoned mid-attempt.
⇒ surfaced `+CMQTTCONNECT: 0,25` = **DNS error**.
- `mqtt_server` hostname → IP literal `178.104.178.108`: bypassed DNS.
⇒ error changed to `+CMQTTCONNECT: 0,3` = **socket connect fail**.
DNS *and* TCP both dead, registration fine → no data breakout.
- `authmode 1 → 0` (skip server-cert verification): ruled out the modem
rejecting our CA chain; made no difference (never reached TLS). Note the
stock ACRIOS script enables cert verification whenever an MQTT username
is set, yet configures no CA — with this SIM it never got far enough to
matter, but it could never have verified successfully anyway.
- Added PDP-context dump (`AT+CGDCONT?`, `AT+CGPADDR`, `AT+CGCONTRDP`)
before every connect attempt. ⇒ produced the root-cause evidence above.
6. **SIMCom CMQTT result codes seen** (for future reference):
`+CMQTTCONNECT: 0,25` = DNS error · `0,3` = socket connect fail ·
`+CMQTTPUB: 0,26` = socket closed / not connected (follow-on noise, not a
distinct fault). No URC before timeout usually means the command was still
waiting — raise the wait, don't guess.
## Current state of `mbus_mqtt_gebos.lua` (v1.10) — temporary changes to revert
| Change | Why it's there | Revert when |
| --- | --- | --- |
| `mqtt_server = "178.104.178.108"` | bypass private-APN DNS | breakout exists **and** public DNS resolves |
| `AT+CSSLCFG="authmode",0,0` | no CA provisioned on modem | ISRG Root X1 loaded (`AT+CCERTDOWN`) + `cacert` configured → set authmode 1 |
| `mqtt_timeout = 60000` | see real modem result codes | after measuring real handshake time on a working network |
| PDP-context dump in `MQTT_SSL_Start()` | root-cause evidence | once connectivity is stable (costs battery/airtime per connect) |
## Options to get data flowing
1. **Preferred: ask ACRIOS** (SIM contract holder) to enable internet breakout
on the APN, or whitelist `ingest.gebos.online` / `178.104.178.108:8883`
for our SIMs. Hardware untouched.
2. **Own SIM:** any IoT SIM with public breakout (1NCE, EMnify, local carrier
NB-IoT, …); set `APN = "<provider apn>"` explicitly in the Lua config.
Check provider NB-IoT coverage on bands 3/5/8/20 (what the firmware
configures).
3. Fallback: ACRIOS cloud bridge/forwarding — reintroduces the vendor
dependency the self-hosted ingest stack exists to avoid.
## Verified-good along the way
- EMQX 5.8.6 (oci-container, host network) serving MQTTS :8883 with the ACME
(HTTP-01) RSA-2048 cert; hot-reloads renewed PEMs.
- Broker auth: `bender` device login + ACL, external end-to-end publish test.
- Note for MQTTX/dashboard tests: EMQX default log level logs neither TLS
handshake failures nor connects; use `emqx ctl log set-level debug`
temporarily, and `emqx ctl listeners` for connection/shutdown counters.
File diff suppressed because it is too large Load Diff
+18 -4
View File
@@ -12,14 +12,28 @@ let
};
in
{
sshOpts = [ "-o" "StrictHostKeyChecking=accept-new" ];
# Path-free ssh options only. The key path and known_hosts are runner-specific
# and injected by the deploy.yml via deploy-rs `--ssh-opts` (which appends to
# this list for both `nix copy` and the activation SSH). They must be absolute:
# the gitea-runner is a systemd DynamicUser whose passwd home is `/`, so ssh
# expands `~` to `/.ssh/...` (NOT $HOME) and a tilde here finds no key. The
# workflow builds the absolute paths from $HOME, where bash expands correctly.
# `-F none` ignores the runner's ~/.ssh/config; IdentitiesOnly pins the key.
sshOpts = [
"-F" "none"
"-o" "IdentitiesOnly=yes"
"-o" "StrictHostKeyChecking=accept-new"
];
autoRollback = true;
magicRollback = true;
nodes = {
# Order matters: the Gitea Actions workflow invokes them in this sequence.
db-host = mkNode "db-host" "db-host.gebos.internal"; # TODO: real address
app-host = mkNode "app-host" "app-host.gebos.internal"; # TODO: real address
mqtt-ingest = mkNode "mqtt-ingest" "ingest.ge-bos.de";
# SSH/deploy targets use public hostnames because the Gitea runner is not on
# the private network. Host-to-host traffic still uses the 10.0.0.0/8 IPs.
# See the host address table in README.md.
db-host = mkNode "db-host" "db.gebos.online";
app-host = mkNode "app-host" "app.gebos.online";
mqtt-ingest = mkNode "mqtt-ingest" "ingest.gebos.online";
};
}
+17 -6
View File
@@ -38,6 +38,13 @@ let
VITE_SUPABASE_URL = dev.supabaseUrl;
VITE_SUPABASE_ANON_KEY = dev.anonKey;
};
# services-flake ships no MQTT broker, so run mosquitto as a plain
# process-compose process. Anonymous access on the dev listener only.
mosquittoConf = pkgs.writeText "mosquitto-dev.conf" ''
listener ${toString dev.mqttPort} ${dev.mqttHost}
allow_anonymous true
'';
in
{
imports = [ services-flake.processComposeModules.default ];
@@ -48,16 +55,20 @@ in
port = dev.pgPort;
initialDatabases = [{ name = dev.pgDB; }];
};
mosquitto."broker" = {
enable = true;
port = dev.mqttPort;
};
};
settings.processes = {
broker = {
command = "${pkgs.mosquitto}/bin/mosquitto -c ${mosquittoConf}";
readiness_probe = {
exec.command = "${pkgs.mosquitto}/bin/mosquitto_sub -h ${dev.mqttHost} -p ${toString dev.mqttPort} -t '$$SYS/#' -C 1 -W 2";
initial_delay_seconds = 1;
period_seconds = 2;
};
};
ingester = {
command = "${pkgs.go}/bin/go run ./ingester";
command = "${pkgs.go}/bin/go run ./ingester/cmd/gebos-ingester";
environment = ingesterEnv;
depends_on = {
"db".condition = "process_healthy";
+3 -3
View File
@@ -8,7 +8,7 @@
services.gebos.supabase = {
enable = true;
# Compose stack points at external db-host instead of the bundled `db` service.
postgresHost = "db-host.gebos.internal"; # TODO: real internal hostname
postgresHost = "10.0.0.2"; # db-host on the private network
postgresPort = 5432;
# Studio bound to loopback only — reach via `ssh -L 3000:127.0.0.1:3000 app-host`.
studioBindAddress = "127.0.0.1";
@@ -18,8 +18,8 @@
enable = true;
# Caddy terminates TLS, proxies to Kong on 127.0.0.1:8000.
sites = {
"api.ge-bos.de" = { upstream = "127.0.0.1:8000"; }; # → Kong → PostgREST/GoTrue
"app.ge-bos.de" = { staticRoot = "${config.services.gebos.frontend.package}/share/frontend"; };
"api.gebos.online" = { upstream = "127.0.0.1:8000"; }; # → Kong → PostgREST/GoTrue
"app.gebos.online" = { staticRoot = "${config.services.gebos.frontend.package}/share/frontend"; };
};
};
+17 -2
View File
@@ -1,7 +1,17 @@
{ config, lib, pkgs, ... }:
{
system.stateVersion = "25.05";
system.stateVersion = "26.05";
# Bootloader. The disko layout (hardware-configurations/disk-config.nix) lays
# down both a BIOS-boot (EF02) partition and an ESP mounted at /boot, so grub
# works whether the VM firmware is legacy BIOS or UEFI. disko fills in
# boot.loader.grub.devices from the EF02 partition; we just enable grub here.
boot.loader.grub = {
enable = true;
efiSupport = true;
efiInstallAsRemovable = true;
};
nix.settings = {
experimental-features = [ "nix-command" "flakes" ];
@@ -13,7 +23,8 @@
isNormalUser = true;
extraGroups = [ "wheel" ];
openssh.authorizedKeys.keys = [
# TODO: paste the Gitea runner's deploy ed25519 pubkey here
"ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJuB+MbW/YuXYGZ/lm6dp+aHJ1YqfaEkiqgj/Z6+cdd0 me@larsnolden.com" # developer machines key
"ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIB9LFaFbro5jU9jG0ZNn66FO61gniKfi865jS6TnZ+SU deploy" # deployment nodes key
];
};
security.sudo.wheelNeedsPassword = false;
@@ -22,6 +33,10 @@
enable = true;
settings.PasswordAuthentication = false;
};
# TODO: restrict inbound SSH (port 22) to the Gitea deployment runner's IP
# only, instead of leaving it open to the world. Likely via
# services.openssh.openFirewall = false + a firewall.extraInputRules rule
# accepting tcp dport 22 from the runner's source address.
networking.firewall.enable = true;
+4 -4
View File
@@ -1,14 +1,14 @@
{ config, lib, pkgs, ... }:
{
# Postgres data lives on the dedicated second disk (/dev/sdb), not the OS disk.
# The shared hardware-configuration.nix + sda disk layout come from ./default.nix.
imports = [ ./hardware-configurations/db-data-disk.nix ];
networking.hostName = "db-host";
services.gebos.secrets.postgresAdmin = true;
# Skeleton: real hardware-configuration.nix + boot/fs lives next to this file
# once we have the actual box. Marked here so the structure is visible.
# imports = [ ./db-host.hardware.nix ];
services.gebos.postgres = {
enable = true;
listenAddresses = [ "0.0.0.0" ];
+9 -1
View File
@@ -10,11 +10,19 @@ let
modules = [
./common.nix
inputs.sops-nix.nixosModules.sops
inputs.disko.nixosModules.disko
# All three boxes are identical qemu guests, so they share one
# hardware-configuration.nix and one disko disk layout (single
# /dev/sda → GPT → LVM → ext4 root). disko derives fileSystems +
# bootloader devices from disk-config.nix. db-host additionally
# imports the /dev/sdb data disk from ./db-host.nix.
./hardware-configurations/hardware-configuration.nix
./hardware-configurations/disk-config.nix
modules.gebos-secrets
modules.gebos-postgres
modules.gebos-supabase
modules.gebos-caddy
modules.gebos-hivemq
modules.gebos-emqx
modules.gebos-ingester
modules.gebos-frontend
./${name}.nix
@@ -0,0 +1,28 @@
{ lib, ... }:
# Second disk for db-host: the Postgres data directory lives on /dev/sdb, kept
# separate from the OS disk (/dev/sda). The whole disk is one GPT partition,
# ext4, mounted at /var/lib/postgresql — so the cluster data dir
# (/var/lib/postgresql/17, the upstream NixOS default) sits on this drive and
# survives an OS-disk rebuild.
{
disko.devices.disk.disk2 = {
device = lib.mkDefault "/dev/sdb";
type = "disk";
content = {
type = "gpt";
partitions = {
data = {
name = "data";
size = "100%";
content = {
type = "filesystem";
format = "ext4";
mountpoint = "/var/lib/postgresql";
mountOptions = [ "defaults" ];
};
};
};
};
};
}
@@ -0,0 +1,55 @@
{ lib, ... }:
{
disko.devices = {
disk.disk1 = {
device = lib.mkDefault "/dev/sda";
type = "disk";
content = {
type = "gpt";
partitions = {
boot = {
name = "boot";
size = "1M";
type = "EF02";
};
esp = {
name = "ESP";
size = "500M";
type = "EF00";
content = {
type = "filesystem";
format = "vfat";
mountpoint = "/boot";
};
};
root = {
name = "root";
size = "100%";
content = {
type = "lvm_pv";
vg = "pool";
};
};
};
};
};
lvm_vg = {
pool = {
type = "lvm_vg";
lvs = {
root = {
size = "100%FREE";
content = {
type = "filesystem";
format = "ext4";
mountpoint = "/";
mountOptions = [
"defaults"
];
};
};
};
};
};
};
}
@@ -0,0 +1,14 @@
{ config, lib, pkgs, modulesPath, ... }:
{
imports =
[ (modulesPath + "/profiles/qemu-guest.nix")
];
boot.initrd.availableKernelModules = [ "ahci" "xhci_pci" "virtio_pci" "virtio_scsi" "sd_mod" "sr_mod" ];
boot.initrd.kernelModules = [ "dm-snapshot" ];
boot.kernelModules = [ ];
boot.extraModulePackages = [ ];
nixpkgs.hostPlatform = lib.mkDefault "x86_64-linux";
}
+12 -12
View File
@@ -4,9 +4,15 @@
networking.hostName = "mqtt-ingest";
services.gebos.secrets.ingester = true;
# Broker authn users (users.csv).
services.gebos.secrets.emqx = true;
services.gebos.hivemq = {
# EMQX (official container image) terminates TLS on :8883 itself and serves
# the ingester on loopback :1883. Certs come from security.acme via HTTP-01
# (lego standalone on :80) — no Caddy on this host anymore.
services.gebos.emqx = {
enable = true;
domain = "ingest.gebos.online";
# Persistent sessions on disk — single-node, devices reconnect on restart.
persistentSessions = true;
tlsPort = 8883;
@@ -14,18 +20,12 @@
services.gebos.ingester = {
enable = true;
# mqttBroker defaults to tcp://127.0.0.1:1883 (the local HiveMQ).
postgresUrl = "postgres://gebos_ingest@db-host.gebos.internal:5432/postgres?sslmode=require";
# mqttBroker defaults to tcp://127.0.0.1:1883 (the local EMQX).
postgresUrl = "postgres://gebos_ingest@10.0.0.2:5432/postgres?sslmode=require";
# GEBOS_POSTGRES_PASSWORD sourced from /run/secrets/gebos-env (sops-nix).
};
# Caddy fronts TLS for `ingest.ge-bos.de` and forwards to HiveMQ.
services.gebos.caddy = {
enable = true;
sites = {
"ingest.ge-bos.de" = { tcpProxy = "127.0.0.1:8883"; };
};
};
networking.firewall.allowedTCPPorts = [ 8883 ];
# 8883: MQTTS for senders. 80: ACME HTTP-01 challenge (lego's standalone
# solver binds it during issuance/renewal; idle otherwise).
networking.firewall.allowedTCPPorts = [ 80 8883 ];
}
+1 -1
View File
@@ -4,6 +4,6 @@
gebos-supabase = import ./gebos-supabase.nix;
gebos-caddy = import ./gebos-caddy.nix;
gebos-postgres = import ./gebos-postgres.nix;
gebos-hivemq = import ./gebos-hivemq.nix;
gebos-emqx = import ./gebos-emqx.nix;
gebos-frontend = import ./gebos-frontend.nix;
}
+4 -11
View File
@@ -3,6 +3,7 @@
let
cfg = config.services.gebos.caddy;
# HTTP-style sites: static file server or reverse proxy.
siteBlock = host: site:
if site ? staticRoot then ''
${host} {
@@ -18,14 +19,7 @@ let
encode zstd gzip
}
''
else if site ? tcpProxy then ''
${host}:8883 {
# TLS termination for MQTT Caddy's `layer4` app would be ideal here;
# for now, document that the upstream HiveMQ port is ${site.tcpProxy}.
# TODO: switch to the caddy-l4 module or terminate TLS in HiveMQ directly.
}
''
else throw "site `${host}` needs one of: staticRoot, upstream, tcpProxy";
else throw "site `${host}` needs one of: staticRoot, upstream";
in
{
options.services.gebos.caddy = {
@@ -39,9 +33,8 @@ in
config = lib.mkIf cfg.enable {
services.caddy = {
enable = true;
email = "ops@ge-bos.de"; # TODO: confirm ACME contact
extraConfig = lib.concatStringsSep "\n"
(lib.mapAttrsToList siteBlock cfg.sites);
email = "ops@gebos.online"; # TODO: confirm ACME contact
extraConfig = lib.concatStringsSep "\n" (lib.mapAttrsToList siteBlock cfg.sites);
};
};
}
+238
View File
@@ -0,0 +1,238 @@
{ config, lib, pkgs, ... }:
let
cfg = config.services.gebos.emqx;
stateDir = "/var/lib/gebos-emqx";
# UID/GID of the `emqx` user baked into the official image. Host-side files
# the container must read or write (certs, bootstrap users, data dir) are
# chowned to this numeric id. On these hosts uid 1000 is the `deploy` user,
# which already has passwordless sudo, so this grants it nothing new.
emqxUid = 1000;
emqxGid = 1000;
# Full emqx.conf, replacing the image default. EMQX reads HOCON natively;
# container paths below are bind mounts declared on the oci-container.
emqxConf = pkgs.writeText "emqx.conf" ''
node {
name = "emqx@127.0.0.1"
# Erlang distribution cookie. Not treated as a secret: the dist/epmd
# ports are never opened in the firewall and the only local users are
# root and deploy (wheel).
cookie = "gebos-emqx"
data_dir = "data"
}
cluster {
name = gebos
# Single node, and required to be `singleton` for durable sessions on
# the open-source builtin_local storage backend.
discovery_strategy = singleton
}
# Dashboard on loopback only (the container uses host networking) reach
# it via `ssh -L 18083:127.0.0.1:18083`. First login is admin/public with a
# forced password change; the changed password persists in the data dir.
dashboard {
listeners.http.bind = "127.0.0.1:18083"
}
# Plain MQTT for the co-located ingester only.
listeners.tcp.default {
bind = "${cfg.bindAddress}:${toString cfg.port}"
}
# MQTTS for the senders. EMQX terminates TLS itself; certs are managed by
# security.acme on the host and installed into ${stateDir}/certs (mounted
# at /etc/emqx/certs). EMQX re-reads the PEM files from disk periodically
# (~120 s), so renewals need no restart paths stay fixed, contents swap.
listeners.ssl.default {
bind = "0.0.0.0:${toString cfg.tlsPort}"
ssl_options {
certfile = "/etc/emqx/certs/fullchain.pem"
keyfile = "/etc/emqx/certs/privkey.pem"
versions = ["tlsv1.2", "tlsv1.3"]
}
}
# The schema defines websocket listeners on 8083/8084 by default unused.
listeners.ws.default.enable = false
listeners.wss.default.enable = false
# Sessions and queued messages survive a broker restart (QoS > 0 with
# clean_start = false), mirroring HiveMQ's file persistence mode.
durable_sessions.enable = ${lib.boolToString cfg.persistentSessions}
# Username/password auth against the built-in database, seeded from the
# sops-rendered users.csv on first start (bootstrap only inserts users that
# do not exist yet to rotate a password, delete the user in the dashboard
# or via `emqx ctl`, then restart to re-import).
authentication = [
{
mechanism = password_based
backend = built_in_database
user_id_type = username
password_hash_algorithm { name = plain, salt_position = disable }
bootstrap_file = "/etc/emqx/auth/users.csv"
bootstrap_type = plain
}
]
# Topic-level authorization from the static acl.conf; anything not
# explicitly allowed is denied.
authorization {
no_match = deny
sources = [
{
type = file
enable = true
path = "/etc/emqx/auth/acl.conf"
}
]
}
'';
# Topic policy (non-secret — usernames and topics only, no credentials).
# Devices publish under `acrios/<IMSI>/<metric>` (the sender firmware's
# mqtt_topic_base), so that is the namespace the ingester consumes and the
# device user is scoped to. `admin` is is_superuser in users.csv and bypasses
# this ACL entirely (break-glass).
#
# Option A (current): one shared device user (`bender`) for the whole fleet.
# Tenant isolation is enforced downstream by the ingester's IMSI→tenant
# registry lookup, NOT at the broker — any device could publish under any
# IMSI. TODO(Option B): per-device users with username == IMSI and an ACL
# rule scoped to `acrios/''${username}/#` for real anti-spoofing.
aclConf = pkgs.writeText "emqx-acl.conf" ''
%% ingester: consume the whole device tree.
{allow, {username, "ingester"}, subscribe, ["acrios/#"]}.
%% device fleet: publish telemetry + LWT/status, subscribe downlinks.
{allow, {username, "bender"}, all, ["acrios/#"]}.
{deny, all}.
'';
in
{
options.services.gebos.emqx = {
enable = lib.mkEnableOption "EMQX broker (official container image) with native TLS";
image = lib.mkOption {
type = lib.types.str;
default = "emqx/emqx:5.8.6";
description = "Official EMQX image reference to run.";
};
domain = lib.mkOption {
type = lib.types.str;
example = "ingest.gebos.online";
description = "Public hostname the senders connect to; ACME cert is issued for it.";
};
tlsPort = lib.mkOption {
type = lib.types.port;
default = 8883;
description = "MQTTS listener port (TLS terminated by EMQX itself).";
};
port = lib.mkOption {
type = lib.types.port;
default = 1883;
description = "Plain MQTT TCP listener port (what the ingester connects to).";
};
bindAddress = lib.mkOption {
type = lib.types.str;
default = "127.0.0.1";
description = ''
Address the plain MQTT listener binds to. Defaults to loopback:
external clients only ever reach the TLS listener.
'';
};
persistentSessions = lib.mkOption {
type = lib.types.bool;
default = true;
description = ''
Whether client sessions and queued messages survive a restart
(EMQX durable sessions) or are kept only in memory.
'';
};
acmeEmail = lib.mkOption {
type = lib.types.str;
default = "ops@gebos.online";
description = "ACME account contact for the broker certificate.";
};
bootstrapUsersFile = lib.mkOption {
type = lib.types.path;
default = config.services.gebos.secrets.emqxBootstrapUsersFile;
defaultText = lib.literalExpression "config.services.gebos.secrets.emqxBootstrapUsersFile";
description = ''
CSV (user_id,password,is_superuser) seeding EMQX's built-in auth
database. Defaults to the sops-rendered secret; copied into the state
dir on service start so the container user can read it.
'';
};
};
config = lib.mkIf cfg.enable {
virtualisation.oci-containers = {
backend = "docker";
containers.emqx = {
image = cfg.image;
# Host networking: EMQX binds host ports directly, so loopback-only
# listeners (plain MQTT, dashboard) really are loopback-only and the
# TLS listener sees real client source addresses.
extraOptions = [ "--network=host" ];
volumes = [
"${emqxConf}:/opt/emqx/etc/emqx.conf:ro"
"${aclConf}:/etc/emqx/auth/acl.conf:ro"
"${stateDir}/auth/users.csv:/etc/emqx/auth/users.csv:ro"
"${stateDir}/certs:/etc/emqx/certs:ro"
"${stateDir}/data:/opt/emqx/data"
];
};
};
# The oci-containers unit is docker-emqx.service; extend it to stage the
# writable state the container mounts. The bootstrap users file is copied
# (not symlinked) out of the sops tmpfs so it can be owned by the
# container's emqx uid.
systemd.services.docker-emqx = {
# Don't start before the first cert issuance has been attempted. If ACME
# fails (e.g. Porkbun creds not provisioned yet), EMQX crash-loops on the
# missing certfile until the cert lands — Restart=always retries.
after = [ "acme-finished-${cfg.domain}.target" ];
wants = [ "acme-finished-${cfg.domain}.target" ];
preStart = ''
install -d -m 750 -o ${toString emqxUid} -g ${toString emqxGid} ${stateDir}/data
install -d -m 755 ${stateDir}/auth ${stateDir}/certs
install -m 400 -o ${toString emqxUid} -g ${toString emqxGid} \
${cfg.bootstrapUsersFile} ${stateDir}/auth/users.csv
'';
};
# Cert issuance + renewal via HTTP-01: lego's built-in standalone server
# answers the challenge on :80 (nothing else listens there — Caddy is gone
# from this host), so no DNS API secrets are needed; the host just keeps
# port 80 open alongside 8883. postRun is the deploy hook: install the
# PEMs at the fixed paths EMQX watches, owned by the container user. EMQX
# hot-reloads them; no restart or `emqx ctl` needed. RSA keys because
# embedded sender TLS stacks often lack ECDSA.
security.acme = {
acceptTerms = true;
certs.${cfg.domain} = {
email = cfg.acmeEmail;
listenHTTP = ":80";
keyType = "rsa2048";
postRun = ''
install -D -m 644 -o ${toString emqxUid} -g ${toString emqxGid} \
fullchain.pem ${stateDir}/certs/fullchain.pem
install -D -m 600 -o ${toString emqxUid} -g ${toString emqxGid} \
key.pem ${stateDir}/certs/privkey.pem
'';
};
};
};
}
-33
View File
@@ -1,33 +0,0 @@
{ config, lib, pkgs, ... }:
let
cfg = config.services.gebos.hivemq;
in
{
options.services.gebos.hivemq = {
enable = lib.mkEnableOption "HiveMQ CE single-node broker";
persistentSessions = lib.mkOption {
type = lib.types.bool;
default = true;
};
tlsPort = lib.mkOption {
type = lib.types.port;
default = 8883;
};
};
config = lib.mkIf cfg.enable {
# TODO: package HiveMQ CE (download tarball + JRE wrapper) and wire as a
# systemd unit. Until then this module only declares options so dependent
# hosts can be evaluated.
systemd.services.gebos-hivemq = {
description = "HiveMQ CE";
wantedBy = [ "multi-user.target" ];
serviceConfig = {
Type = "simple";
ExecStart = "${pkgs.coreutils}/bin/true"; # TODO: real HiveMQ command
Restart = "on-failure";
};
};
};
}
+10 -2
View File
@@ -13,12 +13,20 @@ in
};
config = lib.mkIf cfg.enable {
# TimescaleDB ships under the non-OSI "Timescale License" (TSL), which
# nixpkgs flags as unfree. Allow just this package rather than opening the
# whole config to unfree.
nixpkgs.config.allowUnfreePredicate = pkg:
lib.elem (lib.getName pkg) [ "timescaledb" ];
services.postgresql = {
enable = true;
package = pkgs.postgresql_17;
enableTCPIP = true;
# listen_addresses is managed explicitly below. The upstream postgresql
# module also defines it (via its own default / enableTCPIP), so mkForce
# makes this module the single source of truth and avoids the collision.
settings = {
listen_addresses = lib.concatStringsSep "," cfg.listenAddresses;
listen_addresses = lib.mkForce (lib.concatStringsSep "," cfg.listenAddresses);
shared_preload_libraries = "timescaledb,pg_stat_statements";
};
extensions = ps: with ps; [
+58 -1
View File
@@ -17,6 +17,31 @@ let
];
ingesterKeys = [ "ingester_postgres_password" ];
postgresAdminKeys = [ "postgres_admin_password" ];
emqxKeys = [
"emqx_ingester_password"
"emqx_admin_password"
"emqx_device_password"
];
# EMQX authn bootstrap CSV (bootstrap_type = plain). Seeds the broker's
# built-in auth database on first start. Broker users:
# ingester — the Go ingester; SUBSCRIBE the whole device tree (see the ACL
# in gebos-emqx.nix).
# admin — operational break-glass; is_superuser bypasses the ACL.
# bender — shared device user for the whole fleet (Option A). Tenant
# isolation is enforced downstream by the ingester's IMSI→tenant
# registry, NOT at the broker. TODO(Option B): per-device users
# (username == IMSI) — see the ACL comment in gebos-emqx.nix.
#
# Plaintext passwords are acceptable here because the file is sops-encrypted
# at rest and rendered 0400 into a tmpfs at runtime; passwords must not
# contain commas (CSV).
emqxUsersCsv = ingesterPassword: adminPassword: devicePassword: ''
user_id,password,is_superuser
ingester,${ingesterPassword},false
admin,${adminPassword},true
bender,${devicePassword},false
'';
# Render an env file body from a list of (key, envVarName) pairs.
envBody = pairs: lib.concatStringsSep "\n" (map
@@ -41,7 +66,8 @@ let
enabledKeys =
lib.optionals cfg.supabase supabaseKeys
++ lib.optionals cfg.ingester ingesterKeys
++ lib.optionals cfg.postgresAdmin postgresAdminKeys;
++ lib.optionals cfg.postgresAdmin postgresAdminKeys
++ lib.optionals cfg.emqx emqxKeys;
in
{
options.services.gebos.secrets = {
@@ -55,6 +81,26 @@ in
postgresAdmin = lib.mkEnableOption ''
Postgres bootstrap admin password applied via a one-shot ALTER ROLE unit.'';
emqx = lib.mkEnableOption ''
EMQX broker secrets: the authn bootstrap users.csv (ingester + admin +
device MQTT users).'';
emqxBootstrapUsersFile = lib.mkOption {
type = lib.types.path;
readOnly = true;
description = ''
Path of the rendered EMQX authn bootstrap users.csv consumed by the
gebos-emqx broker. Resolves to the sops-templated secret when sops
material is present and the emqx toggle is on, else to a baked-in
dev-defaults file (so first-boot eval and local VMs work).
'';
default =
if hasSecrets && cfg.emqx
then config.sops.templates."emqx-users.csv".path
else "${pkgs.writeText "emqx-users-dev.csv"
(emqxUsersCsv "dev-ingester" "dev-admin" "dev-device")}";
};
envFile = lib.mkOption {
type = lib.types.path;
readOnly = true;
@@ -90,6 +136,17 @@ in
mode = "0400";
owner = "root";
};
# Rendered as a standalone file — the gebos-emqx preStart copies it into
# the broker state dir, chowned to the container's emqx uid.
templates."emqx-users.csv" = lib.mkIf cfg.emqx {
content = emqxUsersCsv
config.sops.placeholder.emqx_ingester_password
config.sops.placeholder.emqx_admin_password
config.sops.placeholder.emqx_device_password;
mode = "0400";
owner = "root";
};
};
};
}
+76
View File
@@ -0,0 +1,76 @@
# Template for nix/secrets/secrets.yaml — the sops-encrypted store.
#
# To bootstrap:
# 1. cp nix/secrets/secrets.yaml.example nix/secrets/secrets.yaml
# 2. Replace the placeholder values below with real ones.
# (See nix/secrets/README.md for how to generate JWT_SECRET / ANON_KEY /
# SERVICE_ROLE_KEY etc.)
# 3. sops -e -i nix/secrets/secrets.yaml
# (requires sops + age + the recipients in .sops.yaml to be real keys)
#
# Once encrypted, edit in place with: sops nix/secrets/secrets.yaml
#
# Local dev does NOT need this file at all — process-compose.nix bakes in
# dev defaults and the NixOS modules fall back to a writeText env file when
# nix/secrets/secrets.yaml is absent. This file is only consumed at activation
# time on real hosts.
# Supabase compose stack (consumed on app-host by gebos-supabase.service)
supabase_postgres_password: ENC[AES256_GCM,data:OrjkfiumbB3uMD0tuDkgzaiLDgnt20wYOP9tBAyVwVUXQJxLiDypJoUb4wTY,iv:b4/Lep0Jzb6UtAhyOCpcetXgppul+V5O43zokMtpM2M=,tag:3fvi5xpverfSG+xc/4/HHg==,type:str]
supabase_jwt_secret: ENC[AES256_GCM,data:csu0Rj/DM07DCtYFieIe6eEca+eUPR6KGfQBqa6cqRe7bX4=,iv:OKYFa7Y1bbEAKh1ueM7K8JDLldrTJL/r83AGTtMx/IU=,tag:y6OGVofm4cUFPc0BBmkY+A==,type:str]
supabase_anon_key: ENC[AES256_GCM,data:ReE4g5pNUHqi9M9ejnR88XUjwgaibcgZRM5GCMl+Ds7pzKUY442nqEtc2xVO2vQ=,iv:+lp/a/JshLZfdkxm4uQrgOKxBFmIduLtu3aUAhWrKlE=,tag:ft9swD2SeFJ/jMyL07kmGQ==,type:str]
supabase_service_role_key: ENC[AES256_GCM,data:QZQbvJ6rkyoHIpxIvjVDFwxeZ4oIb7Bt2h2vgtW8/sp6swSxtW/oAeVfLkdeduir+2BC5mFAhA==,iv:/F5F3MRC5Yebn68g45F1sj4d18+60JZupneHa3ipExs=,tag:fuzDuSj2npxKQxP2Pz+cjQ==,type:str]
supabase_dashboard_password: ENC[AES256_GCM,data:6KHJCPJ66dW9n/uzb3mpGIYqGZABHeEew1H5kx3I8kqgABqXcnHi6w==,iv:87NBfRoNOwtEKgGC0CacY/0xyVE5DQJpuN0X+7j8dfQ=,tag:TcMmIS2GgUlBXlH0elC7Rg==,type:str]
# Ingester (consumed on mqtt-ingest by gebos-ingester.service)
ingester_postgres_password: ENC[AES256_GCM,data:6jy1G6zG4r0z3c9B6s7jPUtvn/Nr/J2HQN23ch6AGFigiVbefY3hCw==,iv:X/PWQjTW/++MVGsi+hcxbkOJZQMOUH0a4mRH5CbOw4A=,tag:TBKFIdbXZ2YzDLbjdnsyxg==,type:str]
# Postgres bootstrap (consumed on db-host by a one-shot ALTER ROLE unit)
postgres_admin_password: ENC[AES256_GCM,data:j5XHA2ZepUq6rJMmIrKr88xjNLiMsJ+ng7CeQxNO3fVwydQhMW0hl0KB,iv:ZXilU07B3aJRQeIM0GVypFcGe4K7y5BsEOa2cIYzAOo=,tag:CEdzYR7NElODPYmTWdVccQ==,type:str]
# EMQX broker users (consumed on mqtt-ingest by gebos-emqx as bootstrap users.csv).
# PLAIN passwords (password-type defaults to PLAIN; the whole file is sops-
# encrypted). The ingester will authenticate as `ingester`; `admin` is for ops.
emqx_ingester_password: ENC[AES256_GCM,data:pV1FIY5uNeZ2zkjB9dv3C5DP2AmeyaV55lqgizQoKGKq0w3PU1gV+xev9g==,iv:E+anGu88+fhBQRLp55tYghx5SvHVniolXaj6kE/3FrQ=,tag:iJpH+0zAowKq+t/xXR+NjA==,type:str]
emqx_admin_password: ENC[AES256_GCM,data:lrOUYSCkYFkXlelAoMUp9WFxo7COJWYkDSBVEr2RT0B2lfMnF3MzqhqJ,iv:uZi+oCw6esNuweIAFTGn3A6Ix5ctwn1Oj4GBchalXmA=,tag:84QigtqxRr1zddMtCsFu9w==,type:str]
# Shared device login (Option A) — must match mqtt_user/mqtt_password on the
# senders. Per-device users (Option B) are a TODO in gebos-secrets.nix.
emqx_device_password: ENC[AES256_GCM,data:pFp3RxJoDw==,iv:1FQzAHSr4VIyuuq2qgFIcuzOq487W4sKV1x8Opoa0jM=,tag:zwn7dOmZArUOtlZBv5G63w==,type:str]
sops:
age:
- enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSAzaVAwall2bTE0Y09ZeUJn
SFR2cHBDaFZtRmpodGV0ZHJsUzVuWGdNb213ClFFZ04xV0pTVzR5eFh0NjNsYWpZ
SU9IUzBoZWJaNjZmUW1Tc3NER2hpRDAKLS0tIG9BVnAzYW1jL1VZOHRLeTJvQzh0
Y3dtSnp5eGNnWnN0SWVYa0laNExtUWsKhiZOiy1otoGW+Y5HOvFz1nc549LTyvvl
hP0McG7/p0Awh9bQNzhvgQO88M4S0wIrDJ3P0Gu5/HKzKZXQ8EhSOA==
-----END AGE ENCRYPTED FILE-----
recipient: age1cx8ul285kjkzmnhw6skdstnzrxnnme4xkflknzn7yhv52fgxqevqkd66cn
- enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBzRHpzVWtpQnFrWFBNV1c5
UitrRVdQWGUyb0Z6WHVlMFhibjRhWXhFMmdzCnpyMW1QVUErMjBBUHJvMnN6RWJP
b0JselpJQkoxNDFtdU5TYlRBRmpUOGMKLS0tIDdQNGY3TUxhZGtkdzVZSlIwRXZK
NGJqWmJoSkFXMTVKOGNWMGlUN0o1czQKw3/naLb0zt5yHeITSJp9GKEOJfW5bSlL
CtQLx6Nn1WFOTcuoxgaf/xtUCBiSYSRg7pM9KwX/Pfu9gpvWjScUZA==
-----END AGE ENCRYPTED FILE-----
recipient: age1wpl3vz60tlt880p5fmfedr8c59dm4kf0czsacv0w5my706twuf5q9p0x43
- enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBHL3hhWG5WL2N5THRaUFBz
SEo0SE0vQUZIZ05ad0ZUN3lrU3hHSTRISm1zCmVEdWxRSU8zSE1vODNwbVVub2VZ
eWZqU3FiOG9mNHpBZjBQeEdhYkF6SmcKLS0tIEh1WkJIOXdDRmNMMHR2VE5jdXpU
WHRqTUFJQTJQTVFBSzBqVlVWY2ROU28KToql5OybiwITIS9GHieUGfgRILlhmUyM
abkD57fWwMzh1h5UNdUt/VMQEuYX9pK/QhLOrWNgZumk7JesbKRL3A==
-----END AGE ENCRYPTED FILE-----
recipient: age1g57pznep69nlyv3sltz6k0sml47v68m03gh68jkqwcq4jdx68vtsvnefq0
- enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBqUTN5NENOcm1WRnltbzBE
RFgwY3l5cXVYeVptZG9UenBqc25GbCs2NFFFCkNvSytuOVUzRkV2R0doMFpvNndu
a1NyYmVVRE9LRmpwdis4Sk5lcnNoMkEKLS0tIFhTZjFhZVhFZ2FIKzg4dzJ1a25y
cW84SytPUGZmMkVxdmFhMmQ2dy9XZlEK/S1drmtv8te8m2Zgpkm2l+RdOTTbhKCs
ziyUG3WkxBzihV7WpVyfKOu8XMJfLUsgiusjCeCsmkdq+UN+VBlZ/g==
-----END AGE ENCRYPTED FILE-----
recipient: age190gu75rf3ra89mhk27xe3tv87tad087altqhugjlhkerqwe2jfqsnu738d
encrypted_regex: ^(supabase_|ingester_|postgres_admin_|emqx_|porkbun_)
lastmodified: "2026-07-02T13:11:42Z"
mac: ENC[AES256_GCM,data:eJRwD6eGsmca+Gv3TP7JLSoCXlH34boilkxxEqRkbz8vNsCpEANZufet4Oes8vpNubU8yx509nrGo3SoC+MihvRaw/hfWqYOlXhlB0j6RsQuuG3LRQFVLJ1m1Q4zw5PYEcB/JHc4ORSIDrMb6nlemBc1dsDItIqyxlOqFnesUec=,iv:EBmK7XZ9ZDQT68FiItivu3QglhX0LfvMgw6n6qUhXUM=,tag:IqvQ4CVOrnWpQ3cE/mC/2w==,type:str]
version: 3.13.1
+10
View File
@@ -27,3 +27,13 @@ ingester_postgres_password: "replace-me-openssl-rand-hex-32"
# Postgres bootstrap (consumed on db-host by a one-shot ALTER ROLE unit)
postgres_admin_password: "replace-me-openssl-rand-hex-32"
# EMQX broker users (consumed on mqtt-ingest by gebos-emqx). Plaintext
# passwords in a bootstrap users.csv (the whole file is sops-encrypted at
# rest); no commas allowed. The ingester will authenticate as `ingester`;
# `admin` is the operational break-glass superuser.
emqx_ingester_password: "replace-me-openssl-rand-hex-32"
emqx_admin_password: "replace-me-openssl-rand-hex-32"
# Shared device login (Option A) — must match mqtt_user/mqtt_password on the
# senders. Per-device users (Option B) are a TODO in gebos-emqx.nix.
emqx_device_password: "replace-me-to-match-the-sender-mqtt_password"
+4 -4
View File
@@ -16,7 +16,7 @@ services:
image: kong:2.8.1
restart: unless-stopped
ports:
- "127.0.0.1:8000:8000/tcp" # Caddy proxies api.ge-bos.de here
- "127.0.0.1:8000:8000/tcp" # Caddy proxies api.gebos.online here
environment:
KONG_DATABASE: "off"
KONG_DECLARATIVE_CONFIG: /home/kong/kong.yml
@@ -29,10 +29,10 @@ services:
environment:
GOTRUE_DB_DRIVER: postgres
GOTRUE_DB_DATABASE_URL: postgres://supabase_auth_admin:${POSTGRES_PASSWORD}@${POSTGRES_HOST}:${POSTGRES_PORT}/${POSTGRES_DB}
GOTRUE_SITE_URL: https://app.ge-bos.de
GOTRUE_SITE_URL: https://app.gebos.online
GOTRUE_JWT_SECRET: ${JWT_SECRET}
GOTRUE_JWT_EXP: "3600"
API_EXTERNAL_URL: https://api.ge-bos.de
API_EXTERNAL_URL: https://api.gebos.online
rest:
image: postgrest/postgrest:v12.0.2
@@ -52,7 +52,7 @@ services:
STUDIO_PG_META_URL: http://meta:8080
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
DEFAULT_ORGANIZATION_NAME: Gebos
SUPABASE_URL: https://api.ge-bos.de
SUPABASE_URL: https://api.gebos.online
DASHBOARD_USERNAME: gebos
DASHBOARD_PASSWORD: ${DASHBOARD_PASSWORD}