Add native NixOS deployment and UI-managed provider credentials

2026-09-10 14:25:37 +02:00
parent 9843fe0c50
commit 964b9dfc15
21 changed files with 2084 additions and 104 deletions
@@ -39,7 +39,7 @@ Use the address you open Finance Duck at through your VPN, followed by `/api/ban
 https://finance.example.internal/api/banking/callback
 ```

-Replace `finance.example.internal` throughout these examples with your actual hostname. Register the exact URL with Enable Banking and configure the same value on the server, including scheme, hostname, port if applicable, and path.
+Replace `finance.example.internal` throughout these examples with your actual hostname. **Settings → Enable Banking credentials** shows the exact callback URL to register, including scheme, hostname, port and path. On the deployed server it is `https://nixos.taile9e6d9.ts.net:8444/api/banking/callback`.

 For local use, the corresponding URL is:

@@ -51,7 +51,7 @@ http://localhost:8080/api/banking/callback

 **This is a browser redirect, not a webhook.** Your browser must be connected to the VPN and able to reach the callback. Finance Duck does not need to be publicly exposed. The server needs outbound HTTPS access to Enable Banking.

-The **Accounts** screen displays a copyable callback URL. Before configuration it suggests the current browser origin; afterward it displays `ENABLEBANKING_REDIRECT_URL`.
+**Settings** shows a copyable callback URL for the current browser origin and saves it with the application credentials. **Accounts** displays the configured callback URL, or suggests the current origin before configuration.

 ### 2. Generate an application key and certificate

@@ -74,10 +74,10 @@ nix-shell -p openssl --run '
 ```

 - Upload **`secrets/enablebanking.crt`**, the public certificate, to Enable Banking.
- Keep **`secrets/enablebanking.key`** private on your server and back it up securely.
+- Upload **`secrets/enablebanking.key`** through Finance Duck's Settings, and back up your original key securely. Never upload it to Enable Banking in place of the public certificate.
 - Do not rerun the generation command over a key already in use. A different private key will not authenticate against the existing registered certificate.

-The application accepts PEM RSA keys in PKCS#1 or PKCS#8 format, at least 2048 bits. It creates signed application JWTs itself; you do not need to generate daily tokens.
+The application accepts a single unencrypted PEM RSA key in PKCS#1 or PKCS#8 format, at least 2048 bits and no larger than 32 KiB. It creates signed application JWTs itself; you do not need to generate daily tokens.

 ### 3. Register and activate the Enable Banking application

@@ -92,9 +92,24 @@ Enable Banking documents restricted production access for individual, non-commer

 **Linking accounts in their control panel activates your application; it does not create Finance Duck's bank session.** You must also authorize from Finance Duck in step 5, even for the same account.

-### 4. Configure Finance Duck and restart it
+### 4. Configure Finance Duck in Settings

-For a native process, set all three banking variables in the environment of the process that starts Finance Duck:
+Open **Settings → Enable Banking credentials**:
+
+1. Enter the application ID from Enable Banking.
+2. Select the matching private-key `.pem` or `.key` file.
+3. Register the displayed callback URL in Enable Banking if you have not already.
+4. Click **Save configuration**, then proceed to **Accounts** to authorize your bank.
+
+Saving validates the local key and callback format and applies the provider immediately; it does not contact Enable Banking, register an application, or grant bank consent. **Configured** means locally configured, not externally verified. No SSH, environment file, Nix rebuild, or restart is required.
+
+For the same application ID, leave the upload empty to keep the saved key, or upload a replacement to rotate it. Existing bank sessions and sync cursors are preserved. Changing the application ID requires a new upload and invalidates old local bank connections, so reconnect afterward. **Remove configuration** disables banking and invalidates local connections after confirmation; accounts and transaction history stay intact. It does not revoke consent at your bank. Successful configuration changes also cancel pending authorization flows.
+
+The private key and application settings are stored atomically in `state/enablebanking.json` beneath the data directory, with permissions `0600`. The key is never returned by the API or saved in browser storage. Banking sessions are bound to that configuration's application generation so old sessions cannot be reused after switching applications or removal, even across restart. Protect backups of the entire data directory.
+
+#### Optional environment-managed setup
+
+For headless configuration, all three variables below remain a startup fallback **only when no saved banking configuration exists**. A UI-saved configuration, including explicit removal, overrides them. Environment changes require a restart; malformed saved credentials fail closed rather than reverting to the environment.

 ```sh
 export ENABLEBANKING_APP_ID="your-application-id"
@@ -111,7 +126,7 @@ This example assumes a VPN-accessible reverse proxy forwards to `127.0.0.1:8080`

 Stop the old process before restarting; only one process may write a finance directory. Native runs do **not** automatically load `.env` files. Shell exports also do not configure an already-running process or a systemd service—configure that service's environment separately.

-For Docker, use the container-side key path and the configuration in [Docker Compose](#docker-compose-recommended).
+For Docker, use the container-side key path and the configuration in [Docker Compose](#docker-compose).

 ### 5. Authorize from the UI

@@ -138,20 +153,94 @@ Reconnect → bank login and approval → callback code → new session

 Renewal preserves local account identities and existing transaction history. It does not create a second copy of the account or its transactions. Transient provider failures are displayed separately from expired consent.

-Reconnecting renews bank consent, not your application registration. Problems with the registered certificate/private key must be corrected in the server configuration and Enable Banking control panel.
+Reconnecting renews bank consent, not your application registration. Correct certificate registration in the Enable Banking control panel and update the matching private key in **Settings** (or your optional environment-managed setup).

 ## Deployment options

-| Option | Best fit | Tradeoff / current support |
-| --- | --- | --- |
-| **Docker Compose** | Straightforward unattended deployment on a private server or VM | Included Dockerfile and Compose configuration; packages the Go runtime dependencies and frontend together. |
-| **Native binary + systemd** | A small installation without a container daemon | Binary build is supported; configure your own service user, persistent data directory, credentials, and systemd unit. |
-| **NixOS declarative OCI container** | Managing the application alongside other services in your NixOS configuration | Reuse the Docker image through `virtualisation.oci-containers`; a ready-made NixOS module is not included. |
-| **Manual native run** | Development, evaluation, or troubleshooting | The quick-start command works, but it is not an unattended service and stops when the process is terminated. |
+| Option                   | Best fit                             | Included support                                                                             |
+| ------------------------ | ------------------------------------ | -------------------------------------------------------------------------------------------- |
+| **Native NixOS service** | Deployment on your own NixOS machine | Pinned flake package and one systemd service, with persistent state and runtime credentials. |
+| **Docker Compose**       | A private server using containers    | Dockerfile and Compose configuration package the binary and frontend together.               |
+| **Manual native run**    | Development or troubleshooting       | The quick-start command runs in your terminal.                                               |

-A VM or VPS can host either of the first two options; it does not require a separate application architecture. Kubernetes and multiple replicas add little here: **run one application instance per canonical finance directory**.
+**Run one application instance per canonical finance directory.** React is compiled and embedded in the Go binary: production needs neither Node.js nor a separate frontend service.

-### Docker Compose (recommended)
+### Native NixOS service (recommended on NixOS)
+
+Add this input to your host flake after committing and pushing the deployment files:
+
+```nix
+inputs.finance-duck.url = "git+https://git.larsnolden.com/lars/finance-duck.git";
+```
+
+Include `finance-duck` in your flake's `outputs` arguments, then add these entries to your existing `nixosSystem.modules` list:
+
+```nix
+finance-duck.nixosModules.default
+{
+  services.finance-duck = {
+    enable = true;
+    publicURL = "http://localhost:8080";
+    # Optional, once you have installed the runtime credential file:
+    # environmentFile = "/var/lib/finance-duck-secrets/environment";
+  };
+}
+```
+
+Keep your existing host modules, including OpenClaw, unchanged. **Do not make this input's `nixpkgs` follow the host's NixOS 24.11 input:** Finance Duck needs Go 1.24 or newer. The package uses its own pinned toolchain; the service module works on the older host.
+
+Deploy and inspect:
+
+```sh
+sudo nixos-rebuild switch --flake /etc/nixos#nixos
+systemctl status finance-duck
+journalctl -u finance-duck -f
+```
+
+The service runs as `finance-duck`, listens on **127.0.0.1:8080**, and stores all canonical data, settings, provider state, and the disposable DuckDB cache in **/var/lib/finance-duck**. It creates that directory with mode `0700`; rebuilds do not replace it. No firewall ports, VM, container, reverse proxy, or Tailscale configuration are created.
+
+For access from another machine, an SSH tunnel is sufficient: `ssh -L 8080:127.0.0.1:8080 your-server`, then open `http://localhost:8080`. For your existing Tailscale setup, instead set `publicURL` to the host's **actual full MagicDNS HTTPS origin**, for example `https://nixos.YOUR-TAILNET.ts.net:8444`, rebuild, and publish the loopback listener:
+
+```sh
+sudo tailscale serve --bg --yes --https=8444 http://127.0.0.1:8080
+```
+
+Choose an unused port; `8444` leaves the existing OpenClaw `8443` endpoint alone. Keep access restricted to trusted tailnet users. This application has no login; do not enable Funnel or expose it publicly. The Enable Banking callback must use that exact HTTPS origin plus `/api/banking/callback`.
+
+**Optional environment-managed credentials:** both Enable Banking and OpenRouter can be configured directly in **Settings**, without these steps. If you prefer a banking startup fallback, create a private runtime directory and install the key after the first rebuild creates the service account:
+
+```sh
+sudo install -d -o root -g finance-duck -m 0750 /var/lib/finance-duck-secrets
+sudo install -o root -g finance-duck -m 0640 secrets/enablebanking.key \
+  /var/lib/finance-duck-secrets/enablebanking.key
+sudo touch /var/lib/finance-duck-secrets/environment
+sudo chmod 0600 /var/lib/finance-duck-secrets/environment
+sudoedit /var/lib/finance-duck-secrets/environment
+```
+
+Put the three `ENABLEBANKING_*` variables from [bank setup](#optional-environment-managed-setup) in that file together, using `ENABLEBANKING_KEY_FILE=/var/lib/finance-duck-secrets/enablebanking.key`. Enable `environmentFile` in your Nix configuration and rebuild. Later environment-file changes require `sudo systemctl restart finance-duck`. An optional `OPENROUTER_API_KEY` provides the equivalent AI startup fallback. For each provider, saved Settings configuration or explicit removal takes precedence over its environment fallback. Never place private keys or credential values in Nix configuration or the Nix store.
+
+**Existing local data:** this service does not automatically adopt `./finance`. Stop the previous instance, back it up, stop `finance-duck.service`, and copy the entire data directory into `/var/lib/finance-duck`, owned by `finance-duck:finance-duck`. Never run two writers against the same directory.
+
+**Updates:** update the host's Finance Duck input with `nix flake update finance-duck` in `/etc/nixos`, then run `nixos-rebuild switch` again. Normal NixOS rollback restores the binary, not financial data; keep separate data backups. To build without installing a service, run `nix build` in this repository (the new flake and `nix/` files must be tracked by Git).
+
+#### Deployed server
+
+The server at `bender@100.87.224.3` serves Finance Duck at **https://nixos.taile9e6d9.ts.net:8444**, accessible only through Tailscale. Port 443's existing Funnel and OpenClaw on port 8443 are separate and unchanged.
+
+```sh
+ssh -t bender@100.87.224.3 'sudo finance-duck-update'
+# Deploy edits already in the server checkout without pulling:
+ssh -t bender@100.87.224.3 'sudo finance-duck-update --no-pull'
+```
+
+The command pulls `~/projects/finance-duck` with `git pull --ff-only`, snapshots its Git-tracked working files to `/var/lib/finance-duck-source`, updates only the host's `finance-duck` flake input, rebuilds NixOS, and checks backend health. New files must be tracked with Git to enter the snapshot. Ignored data, untracked secrets, and `.git` are excluded; never track credentials. The host's Nixpkgs and NixVirt pins are not updated.
+
+Host configuration lives in `/etc/nixos/flake.nix` and `/etc/nixos/modules/finance-duck.nix`; the latter defines the service settings, tailnet proxy, and updater. Application data lives separately in `/var/lib/finance-duck`, owned by `finance-duck` with mode `0700`. The initial deployment has no bank/AI credentials and does not import workstation data.
+
+Application and deployment source are tracked in the project repository. Commit and push changes before running the normal updater; `--no-pull` can deploy tracked local edits. Git refuses an update that would overwrite conflicting local edits. The updater never stashes, resets, commits, or pushes automatically. See `OPERATIONS.txt` for service inspection, backup, and rollback commands.
+
+### Docker Compose

 For an initial local deployment without bank credentials:

@@ -216,10 +305,10 @@ Configure a systemd service with:
 - An absolute binary path and `-data /var/lib/finance-duck`.
 - `-listen 127.0.0.1:8080` behind your VPN-only reverse proxy.
 - `-public-url https://finance.example.internal` matching the browser origin.
- Banking environment variables and a private key readable by that service user.
+- Provider credentials configured through **Settings**, or optional banking environment variables and a private key readable by that service user.
 - Automatic startup and a restart policy, such as `Restart=on-failure`.

-The binary uses CGO/native DuckDB dependencies; it is not a universally portable static Go executable. Build for your target OS/architecture and provide the required native runtime libraries. On NixOS, deploy it with its runtime closure retained—copying a Nix-linked binary alone to another machine, or garbage-collecting its unrooted runtime paths, is not a reliable deployment method. The repository currently provides `shell.nix`, not a production Nix package or service module.
+The binary uses CGO/native DuckDB dependencies; it is not a universally portable static Go executable. Build for your target OS/architecture and provide the required native runtime libraries. On NixOS, use the native flake package and service module above to retain its runtime closure—copying a Nix-linked binary alone to another machine, or garbage-collecting its unrooted runtime paths, is not a reliable deployment method.

 ### NixOS declarative OCI container

@@ -233,7 +322,7 @@ docker build -t finance-duck:local .

 Make it available to the daemon used by the NixOS service, or publish it to a private registry and pin a version/digest. A rootless Docker image is not automatically available to the system Docker daemon or Podman.

-Keep private keys and API keys in protected runtime files or a secrets manager. **Do not embed secret values in Nix expressions or copy them into the world-readable Nix store.** This is an available deployment approach, not an included ready-to-enable Finance Duck NixOS module.
+Keep private keys and API keys in protected runtime files or a secrets manager. **Do not embed secret values in Nix expressions or copy them into the world-readable Nix store.** The included NixOS module runs the native package; an OCI deployment requires your own container configuration.

 ### VPN and reverse-proxy requirements

@@ -243,7 +332,7 @@ For any deployment:
 - Point your chosen hostname at the VPN-accessible proxy and configure trusted HTTPS there. Finance Duck itself serves HTTP.
 - Forward to the application and preserve the public `Host` header. The application checks Host and mutation origins; forwarding only `X-Forwarded-Host` is insufficient.
 - For native runs, configure `-public-url`; for the supplied Compose configuration, set `FINANCE_PUBLIC_URL`.
- Set `ENABLEBANKING_REDIRECT_URL` to that same origin plus `/api/banking/callback`, and register it exactly with Enable Banking.
+- Register the callback shown in **Settings → Enable Banking credentials**: the browser origin plus `/api/banking/callback`. For optional environment-managed setup, set `ENABLEBANKING_REDIRECT_URL` to that same URL.
 - Allow outbound HTTPS to Enable Banking, and optionally OpenRouter. No public inbound bank webhook is required.
 - Allow sufficiently long proxy requests for sequential bulk AI previews.

@@ -251,7 +340,13 @@ A direct bind to a VPN interface is also supported with `-listen <VPN-IP>:8080`

 ## Optional OpenRouter setup

-Set `OPENROUTER_API_KEY` in the server environment, or add it to the Compose `.env`, then restart/recreate the service and choose a supported model in **Settings**.
+Open **Settings → OpenRouter credentials**, paste your API key, and click **Save key**. Then choose an exact OpenRouter `provider/model` identifier under **Classification preferences** and save those preferences. No SSH, Nix configuration changes, or service restart is needed.
+
+Use **Replace key** to rotate the credential or **Remove key** to disable AI. Changes apply to future classifications immediately and survive restart; an already-running classification keeps the key it started with. “Configured” means a key is present, not that OpenRouter has accepted it. A successful **AI classification → Analyse** request checks the key, model, and private routing together.
+
+The key is stored separately from preferences and journals in `state/openrouter.json` beneath your data directory (`/var/lib/finance-duck/state/openrouter.json` on the deployed server), as a private `0600` plaintext file. The API never returns it, and the UI clears the password field after saving rather than storing it in browser storage. Treat data-directory backups as containing credentials. Anyone able to access this no-login application can replace or remove the key: keep access restricted to trusted tailnet users.
+
+For administrator-managed startup configuration, `OPENROUTER_API_KEY` in the server environment or Compose `.env` remains an optional fallback **only while no saved credential file exists**. A UI-saved key takes precedence; **Remove key** persists an explicit disable, so a restart cannot silently restore the environment key. Environment-only changes require a restart/recreate.

 Bank imports do **not** require this key. Without AI, explicit merchant-default rules still work; unresolved transactions remain unclassified and editable.

@@ -259,7 +354,7 @@ Every AI classification requests `provider.data_collection = "deny"`, `provider.

 ## Data, backups, and recovery

-Back up the **entire canonical finance directory**, including registry files, journals, `config.toml` when present, and operational/recovery state, plus the separately stored secrets. Stop the service for a consistent filesystem backup. DuckDB under `cache/` can be excluded and rebuilt.
+Back up the **entire canonical finance directory**, including registry files, journals, `config.toml` when present, and operational/recovery state, plus any separately stored environment-managed secrets. `state/openrouter.json` and `state/enablebanking.json` contain UI-managed credentials: protect backups accordingly, including the matching banking session state. Stop the service for a consistent filesystem backup. DuckDB under `cache/` can be excluded and rebuilt.

 Use **Settings → Rebuild index** while the app runs. For an offline rebuild, stop the service first and run: