Gebos/README.md

# Gebos

IoT telemetry stack on bare-metal NixOS.

This repo is a monorepo containing all code, NixOS modules, host configurations,
and CI/CD pipelines for the project. Everything is one `flake.nix`.

## Architecture

Three hosts (see issue #21 for the full design discussion):

| Host          | Role                                      | Public hostname        | Private IP |
| ------------- | ----------------------------------------- | ---------------------- | ---------- |
| `mqtt-ingest` | HiveMQ CE + Go MQTT→Postgres ingester     | `ingest.gebos.online`     | `10.0.0.4` |
| `db-host`     | Postgres 17 + TimescaleDB (telemetry+auth)| `db.gebos.online` (SSH only) | `10.0.0.2` |
| `app-host`    | Caddy + Kong + Supabase (compose) + SPA   | `app.gebos.online`, `api.gebos.online` | `10.0.0.3` |

Public REST surface is PostgREST + SQL `/rpc/` functions, fronted by Kong, TLS-terminated by Caddy.
Supabase Studio is bound to `127.0.0.1` on `app-host` — reach it with `ssh -L 3000:127.0.0.1:3000 app-host`.

### Networking

There is no private DNS. Each host has a public hostname (used for `deploy-rs`
SSH access from the Gitea runner, which is **not** on the private network, plus
TLS ingress where applicable), and a static `10.0.0.0/8` IP used for all
host-to-host traffic:

- `db.gebos.online` is for SSH/deploy only — Postgres is never exposed publicly.
- `app-host` and `mqtt-ingest` reach Postgres at `10.0.0.2:5432` over the private
  network.
- `db-host` only accepts Postgres (port 5432) from `10.0.0.0/8` (firewall rule
  in `nix/hosts/db-host.nix`).

## Repo layout

```
flake.nix
frontend/                # Vite + React SPA
ingester/                # Go MQTT → Postgres
nix/
  modules/               # NixOS modules (one per service)
  hosts/                 # nixosConfigurations: mqtt-ingest, db-host, app-host
  supabase/              # vendored Supabase docker-compose, db init SQL
  dev/                   # process-compose for local development
  deploy.nix             # deploy-rs node map
.gitea/workflows/        # CI + CD
```

## Local development

```
nix run .#dev
```

Brings the full stack up on one machine via process-compose-flake (Postgres,
Supabase compose, Kong, Caddy, ingester, Vite dev server). NixOS required.

## Deployment

There are two distinct phases. **Initial provisioning** turns a blank box into a
NixOS host (`nixos-anywhere`, run once per machine). **Updates** push new
closures to a host that already runs NixOS (`deploy-rs`, run on every change).

### SSH key setup (do this first)

Both phases authenticate over SSH with `~/.ssh/larsnolden`, which is
passphrase-protected. Load it into an `ssh-agent` once so the deploy tools can
reuse it without prompting:

```fish
eval (ssh-agent -c)        # bash/zsh: eval "$(ssh-agent -s)"
ssh-add ~/.ssh/larsnolden  # enter the passphrase once
ssh-add -l                 # confirm the key is loaded
```

This is **required** for `deploy-rs`, not just a convenience: with
`magicRollback = true` (see `nix/deploy.nix`) activation opens two concurrent SSH
connections — the activation command and a rollback waiter. Without an agent,
both race to read the passphrase from the terminal, one loses, and the deploy
fails with `Permission denied (publickey,keyboard-interactive)` even though
manual SSH and the copy step work. The agent serves the key to every connection,
so no prompt is needed.

### Initial provisioning (`nixos-anywhere`)

`deploy-rs` only *updates* a machine that already runs NixOS — it copies a
prebuilt closure and activates it. A fresh box (e.g. a stock Debian image with
only a `root` user) has no Nix store and no NixOS generation to switch to, so
`deploy-rs` fails with `nix-store: command not found`. Use
[`nixos-anywhere`](https://github.com/nix-community/nixos-anywhere) to install
NixOS over SSH first; after that, `deploy-rs` takes over for all subsequent
deploys.

`nixos-anywhere` SSHes in as `root`, kexecs into an in-memory NixOS installer,
partitions and formats the disk per the host's [`disko`](https://github.com/nix-community/disko)
config, installs `nixosConfigurations.<host>`, and reboots into NixOS. **This
wipes the target disk.**

Prerequisites, per host, before running it:

1. **A real disk layout.** Hosts currently import the fictional
   `nix/hosts/placeholder-hardware.nix` (it only exists so `nix flake check`
   evaluates). Replace that import with a `disko` config describing the actual
   disk device (`/dev/sda` vs `/dev/vda`/nvme) and firmware (UEFI vs legacy
   BIOS). `disko` replaces the hand-generated `hardware-configuration.nix`.
2. **Root SSH access** to the box. The `deploy` user and its authorized keys are
   created by `nix/hosts/common.nix` during the install, so deploy-rs access
   works automatically once NixOS is up.
3. **Host secrets key** present so sops-nix can decrypt at first boot — see
   [`nix/secrets/README.md`](nix/secrets/README.md). Otherwise services that
   read `/run/secrets/*` (e.g. the ingester) fail to start after reboot.

Then, from the repo root:

```
# installs NixOS onto the target, wiping its disk
nix run github:nix-community/nixos-anywhere -- \
  --flake .#mqtt-ingest root@ingest.gebos.online
```

Repeat with `.#db-host root@db.gebos.online` and `.#app-host root@app.gebos.online`.
Provision `db-host` first if you intend to deploy updates immediately afterward
(see the ordering note below). Once a host has rebooted into NixOS, never run
`nixos-anywhere` against it again — use `deploy-rs`.

### Updates (`deploy-rs`)

`deploy-rs` from a Gitea Actions runner on push to `main`. Closures are built
once, copied to each host, activated with auto-rollback. Order: `db-host` →
`app-host` → `mqtt-ingest`.

Running it by hand needs the key loaded into an `ssh-agent` first — see
[SSH key setup](#ssh-key-setup-do-this-first) above.

```
nix run github:serokell/deploy-rs -- .#db-host       # one host
nix run github:serokell/deploy-rs -- .                # all hosts
```

## Secrets

[sops-nix](https://github.com/Mic92/sops-nix) + [age](https://github.com/FiloSottile/age).
Single encrypted file at `nix/secrets/secrets.yaml`; each host decrypts only the
keys it needs at activation, rendered into a tmpfs env file consumed by systemd
`EnvironmentFile=`. Plaintext never enters the Nix store. See
[`nix/secrets/README.md`](nix/secrets/README.md) for bootstrap and rotation.

Local dev needs **no** secrets bootstrap — `nix run .#dev`, `go run ./ingester`,
and `npm --prefix frontend run dev` all default to the local dev stack values
defined in `nix/dev/process-compose.nix`.