Gebos

IoT telemetry stack on bare-metal NixOS.

This repo is a monorepo containing all code, NixOS modules, host configurations, and CI/CD pipelines for the project. Everything is one flake.nix.

Architecture

Three hosts (see issue #21 for the full design discussion):

Host	Role	Public hostname	Private IP
`mqtt-ingest`	HiveMQ CE + Go MQTT→Postgres ingester	`ingest.gebos.online`	`10.0.0.4`
`db-host`	Postgres 17 + TimescaleDB (telemetry+auth)	`db.gebos.online` (SSH only)	`10.0.0.2`
`app-host`	Caddy + Kong + Supabase (compose) + SPA	`app.gebos.online`, `api.gebos.online`	`10.0.0.3`

Public REST surface is PostgREST + SQL /rpc/ functions, fronted by Kong, TLS-terminated by Caddy. Supabase Studio is bound to 127.0.0.1 on app-host — reach it with ssh -L 3000:127.0.0.1:3000 app-host.

Networking

There is no private DNS. Each host has a public hostname (used for deploy-rs SSH access from the Gitea runner, which is not on the private network, plus TLS ingress where applicable), and a static 10.0.0.0/8 IP used for all host-to-host traffic:

db.gebos.online is for SSH/deploy only — Postgres is never exposed publicly.
app-host and mqtt-ingest reach Postgres at 10.0.0.2:5432 over the private network.
db-host only accepts Postgres (port 5432) from 10.0.0.0/8 (firewall rule in nix/hosts/db-host.nix).

Repo layout

flake.nix
frontend/                # Vite + React SPA
ingester/                # Go MQTT → Postgres
nix/
  modules/               # NixOS modules (one per service)
  hosts/                 # nixosConfigurations: mqtt-ingest, db-host, app-host
  supabase/              # vendored Supabase docker-compose, db init SQL
  dev/                   # process-compose for local development
  deploy.nix             # deploy-rs node map
.gitea/workflows/        # CI + CD

Local development

nix run .#dev

Brings the full stack up on one machine via process-compose-flake (Postgres, Supabase compose, Kong, Caddy, ingester, Vite dev server). NixOS required.

Deployment

There are two distinct phases. Initial provisioning turns a blank box into a NixOS host (nixos-anywhere, run once per machine). Updates push new closures to a host that already runs NixOS (deploy-rs, run on every change).

Initial provisioning (`nixos-anywhere`)

deploy-rs only updates a machine that already runs NixOS — it copies a prebuilt closure and activates it. A fresh box (e.g. a stock Debian image with only a root user) has no Nix store and no NixOS generation to switch to, so deploy-rs fails with nix-store: command not found. Use nixos-anywhere to install NixOS over SSH first; after that, deploy-rs takes over for all subsequent deploys.

nixos-anywhere SSHes in as root, kexecs into an in-memory NixOS installer, partitions and formats the disk per the host's disko config, installs nixosConfigurations.<host>, and reboots into NixOS. This wipes the target disk.

Prerequisites, per host, before running it:

A real disk layout. Hosts currently import the fictional nix/hosts/placeholder-hardware.nix (it only exists so nix flake check evaluates). Replace that import with a disko config describing the actual disk device (/dev/sda vs /dev/vda/nvme) and firmware (UEFI vs legacy BIOS). disko replaces the hand-generated hardware-configuration.nix.
Root SSH access to the box. The deploy user and its authorized keys are created by nix/hosts/common.nix during the install, so deploy-rs access works automatically once NixOS is up.
Host secrets key present so sops-nix can decrypt at first boot — see nix/secrets/README.md. Otherwise services that read /run/secrets/* (e.g. the ingester) fail to start after reboot.

Then, from the repo root:

# installs NixOS onto the target, wiping its disk
nix run github:nix-community/nixos-anywhere -- \
  --flake .#mqtt-ingest root@ingest.gebos.online

Repeat with .#db-host root@db.gebos.online and .#app-host root@app.gebos.online. Provision db-host first if you intend to deploy updates immediately afterward (see the ordering note below). Once a host has rebooted into NixOS, never run nixos-anywhere against it again — use deploy-rs.

Updates (`deploy-rs`)

deploy-rs from a Gitea Actions runner on push to main. Closures are built once, copied to each host, activated with auto-rollback. Order: db-host → app-host → mqtt-ingest.

nix run github:serokell/deploy-rs -- .#db-host       # one host
nix run github:serokell/deploy-rs -- .                # all hosts

Secrets

sops-nix + age. Single encrypted file at nix/secrets/secrets.yaml; each host decrypts only the keys it needs at activation, rendered into a tmpfs env file consumed by systemd EnvironmentFile=. Plaintext never enters the Nix store. See nix/secrets/README.md for bootstrap and rotation.

Local dev needs no secrets bootstrap — nix run .#dev, go run ./ingester, and npm --prefix frontend run dev all default to the local dev stack values defined in nix/dev/process-compose.nix.

5.5 KiB Raw Blame History