finance-duck/OPERATIONS.txt

FINANCE DUCK — GO + REACT

Local build on NixOS
-------------------
nix-shell
npm --prefix web ci
npm --prefix web run build
go build -o bin/finance-duck ./cmd/finance-duck
./bin/finance-duck -data ./finance

Open http://localhost:8080. Create an account, then import its N26 CSV from
Accounts. The application starts empty except for expense/income fallback
categories. Create your category tree, tags, and merchants in the UI. Enable a
merchant's default rule explicitly only when its category/tags are reliable;
leave it disabled for ambiguous merchants such as Amazon.

Tests: go test ./...
The Go build embeds web/dist, so build React first. CGO and a C++ linker are
required by the native DuckDB driver. shell.nix supplies the toolchain. React
and CSS are served locally; there are no CDN requests or tracking scripts.

Private deployment
------------------
There is NO application login. Default bind: 127.0.0.1:8080.
For a VPN/reverse-proxy hostname, use an exact browser origin:
  ./bin/finance-duck -data /srv/finance -listen 127.0.0.1:8080 \
    -public-url https://finance.example.internal
Preserve the public Host header at the reverse proxy. The application rejects
other Hosts and cross-origin writes. Configure a long proxy request timeout
for bulk AI previews. Do not expose the reverse proxy to the public Internet.
A VPN-address bind can be used instead, with the corresponding -public-url.

Docker: docker compose up --build -d
Compose publishes only 127.0.0.1:8080. Set FINANCE_PUBLIC_URL when proxying it.
The container is unprivileged with a read-only root and a persistent named
volume at /data. A host bind mount must be writable by UID/GID 10001.
Never change the port mapping to public 0.0.0.0 without VPN/firewall isolation.

Native NixOS: the flake exports nixosModules.default. Import it into your host,
then set services.finance-duck.enable=true and publicURL to the exact browser
origin. Deploy with your normal nixos-rebuild switch. The module runs ONE
systemd service; React is embedded in the Go binary, not a separate service.
Defaults: listen 127.0.0.1:8080, service user finance-duck, persistent data at
/var/lib/finance-duck (0700). It does not configure networking or Tailscale.
Optional services.finance-duck.environmentFile names an existing runtime file
read by systemd for optional provider startup fallbacks. Both Enable Banking
and OpenRouter can be configured directly in Settings without that file.
Keep private keys outside the Nix store and readable only by the service.
See README's native NixOS section for optional environment-file permissions.

Deployed host: bender@100.87.224.3
--------------------------------
URL: https://nixos.taile9e6d9.ts.net:8444 (Tailscale only, no application login).
The backend binds 127.0.0.1:8080. Existing Funnel on 443 and OpenClaw on 8443
are separate. Never enable Funnel for Finance Duck.

Update from your workstation:
  ssh -t bender@100.87.224.3 'sudo finance-duck-update'
Or on the host:
  sudo finance-duck-update
  sudo finance-duck-update --no-pull   # deploy existing checkout edits

The updater serializes updates, pulls ~/projects/finance-duck as bender with
git pull --ff-only, creates a Git-filtered source snapshot, updates ONLY the
finance-duck input in /etc/nixos/flake.lock, runs nixos-rebuild switch, then
checks /api/health with the correct Host. The app has its own pinned Nixpkgs;
the host's Nixpkgs and NixVirt pins stay unchanged. A build failure leaves the
running service untouched; an activation/health failure exits nonzero.

Application and deployment source are tracked in the project repository.
Commit and push changes before running the normal updater. New local source
files must be tracked with git add before a --no-pull deployment. Pull refuses
to overwrite conflicting local edits; resolve them before retrying. No
automatic stashing, resetting, committing or pushing occurs. Keep credentials
and live finance data out of Git.

Host files:
  /etc/nixos/flake.nix                   application input and module import
  /etc/nixos/modules/finance-duck.nix    service, tailnet proxy, update command
  /var/lib/finance-duck-source           replaceable Git-filtered source snapshot
  /var/lib/finance-duck                  persistent data, finance-duck:finance-duck 0700
The source snapshot excludes .git, ignored files and untracked files. It is
not a data backup. The initial deployment starts empty without provider
credentials; it does not copy the workstation's finance/ directory.

Inspect on the host:
  systemctl status finance-duck finance-duck-tailscale-serve
  sudo journalctl -u finance-duck -f
  tailscale serve status
  curl -f https://nixos.taile9e6d9.ts.net:8444/api/health

Before significant upgrades, stop the service and back up the entire data
directory to a protected location (including hidden recovery files). Restart
after the backup, and protect any separate credential files as well:
  sudo systemctl stop finance-duck
  sudo install -d -m 0700 /var/backups/finance-duck
  sudo sh -c 'umask 077; tar -C /var/lib -czf /var/backups/finance-duck/data-$(date +%Y%m%dT%H%M%S).tar.gz finance-duck'
  sudo systemctl start finance-duck
Copy backups to secure off-host storage; a backup on this disk cannot protect
against disk loss.

Binary/system rollback:
  sudo nixos-rebuild switch --rollback
This switches the previous NixOS generation, NOT financial data or config
source. The generation before the first deployment has no Finance Duck service.
For a lasting rollback, restore the intended source/config/lock before the
next update. Do not remove or roll back live data blindly.

OpenRouter
----------
Open Settings -> OpenRouter credentials, paste the API key, and Save key.
Choose the exact provider/model identifier in Classification preferences and
save preferences. Replace key rotates it; Remove key disables AI. No SSH,
Nix rebuild or restart is needed. Changes affect future classification
requests; in-flight requests retain their original key.

The key is stored as private 0600 plaintext in state/openrouter.json under
the data directory, never in config.toml or browser storage. API responses
never return the saved key. Backups of the data directory contain this secret.
Only allow trusted users to reach the application; there is no separate
administrator login for changing credentials.

OPENROUTER_API_KEY remains a startup fallback only when the saved credential
file is absent. A saved key overrides it. Removing the key in Settings saves
an explicit disable, which also overrides the environment after restart.
Malformed saved credentials fail startup rather than reverting to another key.

Configured means present, not verified with OpenRouter. A successful
AI classification -> Analyse request verifies access with the selected model.
The model and endpoint must support strict structured outputs and the
configured privacy routing. Every classification request sets:
  provider.data_collection = deny
  provider.zdr = true
  provider.require_parameters = true
No retry relaxes these requirements. OpenRouter must also have prompt logging
disabled in your account settings. The underlying provider processes prompts;
this is not local AI and cannot promise that a remote provider honors policy.

Amounts and currency are omitted by default. Include Amount in Settings is
explicit opt-in. Local account/provider IDs, known counterparty names, banking
identifiers and recognizable references are stripped; candidate identifiers
are per-request opaque tokens. Categories/tags and candidate merchant names
are deliberately sent as classification context. Free-form text can contain
unknown personal names, so automatic sanitization is not an anonymity guarantee.
Conservative redaction can reduce recognition quality. Inspect your descriptions
and do not configure an API key if no financial text may leave the server.

Classification failures do not discard imports: facts are committed first and
failed enrichment stays unclassified with an error visible in Transactions.
Classification requests use one transaction at a time, not batches. Known
merchant defaults can classify without any configured AI key.

Enable Banking
--------------
Register your application and public certificate with Enable Banking. For
personal production access follow its linked-own-accounts registration rules:
  https://enablebanking.com/docs/api/linked-accounts/
Configure the allowed redirect URL to the exact externally reachable URL:
  https://finance.example.internal/api/banking/callback
The browser must be able to reach this callback through your VPN.

Open Settings -> Enable Banking credentials. Enter the application ID, upload
its matching private-key PEM file, and Save configuration. Settings supplies
the exact browser-origin callback URL to register. Keys must be a single
unencrypted RSA PKCS#1 or PKCS#8 PEM, >=2048 bits and <=32 KiB.
No SSH, environment file, Nix rebuild or restart is needed.

Save validates local credentials without contacting Enable Banking. It does
not register/activate the app or authorize an account. Configured means local
setup is present; complete registration above and authorize under Accounts.

Leave the key upload empty when editing the same application to keep its key.
Same-app key/callback rotation preserves sessions and cursors. Changing the
application ID requires a new key upload and reconnecting your banks. Remove
configuration confirms before disabling banking and clearing local connection
state. Canonical accounts/history remain; upstream bank consent is not revoked.
Each successful change invalidates pending authorization flows.

Credentials persist privately (0600) in state/enablebanking.json beneath the
data directory. The API never returns the private key; browser storage never
contains it. Session state is bound to the application configuration generation,
so stale sessions cannot be revived by a restart after app changes/removal.
Back up the entire data directory securely, not isolated credential/state files.

Optional environment startup fallback, used ONLY when no saved config exists:
  ENABLEBANKING_APP_ID=<registered application ID>
  ENABLEBANKING_KEY_FILE=/run/secrets/enablebanking.key
  ENABLEBANKING_REDIRECT_URL=https://finance.example.internal/api/banking/callback
The referenced key must be readable by the service. Environment changes need
a restart. UI configuration or explicit removal overrides all three variables;
invalid saved configuration fails startup rather than using another key.

Accounts shows the configured callback URL. Register it exactly with Enable
Banking. Settings saves the current browser callback URL. The redirect carries a
one-time code and state, not a reusable API key. Go verifies state, exchanges the
code for a session_id, and persists session details locally with mode 0600.

Accounts -> Connect: enter the exact Enable Banking institution name and
country code (DE for Germany), then authorize through the bank. A successful
callback immediately wakes the synchronization worker; Sync now is also available.
The dashboard shows, for example, \"ING needs reconnection\" when consent expires
or is revoked. Reconnect ING starts the same approval flow with that bank and
country already selected. The new consent replaces the old account bindings
without duplicating local accounts or their financial history. Transient provider
errors are displayed separately from expired consent.

Only booked transactions are persisted. Daily sync deliberately overlaps each
account's last successful sync by 14 days; each new account first requests 90 days.
Per-account cursors prevent newly connected or reactivated accounts losing history
because another account synced recently. Older records can be imported using CSV.
A failed provider call retains local data and is retried by the daily scheduler;
Sync now can retry sooner. Balances are fetched
on demand, with exact amount/currency/type values, rather than inferred from an
incomplete historical journal.

CSV and identity
----------------
The initial real CSV adapter is N26, not generic ING/Kontist CSV autodetection.
It accepts comma/semicolon separators, German/English headers, UTF-8 BOM,
quoted multiline descriptions, ISO/German dates and decimal point/comma.
Required columns: Date / Datum / Booking Date / Buchungsdatum and
Amount (EUR) / Betrag (EUR) (or Amount/Betrag with a currency column/account).
Optional: Payee / Partner Name / Zahlungsempfaenger [with German umlaut],
Payment reference / Verwendungszweck, Account number / IBAN,
Value Date / Wertstellung, Transaction ID / Transaktions-ID.
Use the original export, not spreadsheet-reformatted dates/numbers. Foreign
original amounts/exchange-rate columns are not mistaken for account amounts.
The initial application preserves currency but never converts or sums currencies.

Stable provider entry references are preferred. Enable Banking transaction_id
is NOT guaranteed stable and is not used as the primary identity. Fallback
fingerprints retain identical-record occurrence counts: two identical rows
remain two transactions, and repeat imports do not add two more. Without stable
IDs, identical records from separately truncated exports are intrinsically
ambiguous. Import consistent overlapping/full exports. Uncertain cross-source
collisions are rejected rather than silently double counted; retain the error
and reconcile the input locally before retrying. Facts are never silently
replaced when upstream descriptions or amounts change for an existing identity.

Transfers use reciprocal records from different owned accounts, equal/opposite
exact amounts and matching currency, with own-IBAN evidence and unambiguous
matching. Ambiguous pairs are not guessed. The linked records remain separate
immutable facts; analytical double-entry postings balance and transfers do not
count as income/spending. Populate local account IBANs to support recognition.

Canonical files and recovery
----------------------------
finance/
  config.toml                 model/amount opt-in only, no API keys
  accounts.finance
  categories.finance
  tags.finance
  merchants.finance
  journal/YYYY/YYYY-MM.finance
  state/sync-state.json       sensitive local consent/session metadata
  state/openrouter.json       sensitive UI-managed OpenRouter key or explicit disable
  state/enablebanking.json    sensitive UI-managed banking key/configuration or disable
  cache/finance.duckdb        disposable analytical projection

The custom grammar is deliberately small:
  category {
    id: "cat_example"
    name: "Groceries"
    parent_id: "cat_expenses"
    kind: "expense"
  }
A transaction block has facts: {...} and enrichment: {...} JSON-valued fields.
Financial amounts are quoted decimal strings, never binary floating point.
Up to four fractional digits are supported; arithmetic uses exact ten-thousandths
with explicit overflow checks. DuckDB stores DECIMAL(24,4).

Each block starts with account/category/tag/merchant/transaction and '{' on its
own line; fields use name: JSON. Strings use JSON escaping (including \n for
multiline descriptions). JSON values may span lines. Blank lines and full-line
# or // comments are accepted between fields/blocks. Unknown fields, duplicate
keys, malformed records, invalid references and taxonomy cycles are rejected.
The grammar is version-one strict: extension/split fields are not accepted yet.
Future format extensions require an explicit parser migration.

Stable category IDs survive renaming and moving; assigned categories must remain
leaves. Built-in roots and fallback leaves are protected. Move assigned records
to another leaf before adding children to their former category. Category
merges migrate referenced transactions/defaults; tag merges deduplicate links;
tag deletion removes all affected links after UI confirmation. Merchant merging
migrates transactions and retains source names/aliases on the target merchant.

Unchanged blocks and comments retain their text. UI enrichment edits do not
rewrite imported facts. File hashes detect external edits; invalid files stop
loading/indexing with a file/line error, not a partially refreshed dashboard.
A process lock prevents multiple app writers; use one instance per finance dir.
Revision conflicts require refreshing/re-previewing, not blind overwriting.

Multi-file writes are recoverable and logically atomic within the application.
Do not run external writers during a commit; external tools do not participate
in the app's process lock. Stop the service for manual bulk edits or backups.
If a write is interrupted, keep all state files and restart for recovery before
editing the journal manually. Keep backups of the entire canonical directory,
including hidden/state recovery files, plus the separately stored secrets.
The cache can be excluded. Avoid exposing any financial directory through a
static file server, Git public remote, or unencrypted shared backup.

Rebuild from text:
  ./bin/finance-duck -data ./finance -rebuild
Stop the running app before using that command (single writer lock), or use
Settings -> Rebuild index while it runs. A broken/deleted DuckDB file can be
removed while stopped and regenerated; it never contains the only copy of
financial records. When an index rebuild fails, UI mutations still preserve
canonical data and the index error is surfaced rather than serving stale totals.

Reclassification
----------------
AI / Classification: choose dates, model and independent Merchant/Category/Tags
fields. Analyse produces a read-only preview. Apply all/selected writes all
selected changes in one canonical commit; financial facts never change. A
manual edit, external journal change or taxonomy change invalidates old previews.
Previews are kept in memory for up to one hour and disappear on restart. Cancel
writes nothing. Transfers are skipped, and unselected fields are preserved.
Failed rows remain unchanged and are listed separately from proposed changes.

Boundaries and verification
---------------------------
There are no splits, budgets, investments, tax/invoice/receipt processing,
login/multi-user support, arbitrary SQL or natural-language query execution.
Natural-language query DSL and Sankey exploration remain explicitly later work.
There is no browser-to-bank credential handling or payment initiation.

Automated tests exercise deterministic financial invariants and mock remote
provider HTTP behavior. They do not replace testing real consent renewals and
real booked transaction samples for your banks. No real user credentials or
financial history are included in the repository. Test fixtures are synthetic.

API documentation sources:
  https://enablebanking.com/docs/api/reference/
  https://openrouter.ai/docs/guides/features/structured-outputs
  https://openrouter.ai/docs/guides/features/zdr
  https://openrouter.ai/docs/guides/routing/provider-selection