finance-duck/OPERATIONS.txt

FINANCE DUCK — GO + REACT

Local build on NixOS
-------------------
nix-shell
npm --prefix web ci
npm --prefix web run build
go build -o bin/finance-duck ./cmd/finance-duck
./bin/finance-duck -data ./finance

Open http://localhost:8080. Create an account, then import its N26 CSV from
Accounts. The application starts empty except for expense/income fallback
categories. Create your category tree, tags, and merchants in the UI. Enable a
merchant's default rule explicitly only when its category/tags are reliable;
leave it disabled for ambiguous merchants such as Amazon.

Tests: go test ./...
The Go build embeds web/dist, so build React first. CGO and a C++ linker are
required by the native DuckDB driver. shell.nix supplies the toolchain. React
and CSS are served locally; there are no CDN requests or tracking scripts.

Private deployment
------------------
There is NO application login. Default bind: 127.0.0.1:8080.
For a VPN/reverse-proxy hostname, use an exact browser origin:
  ./bin/finance-duck -data /srv/finance -listen 127.0.0.1:8080 \
    -public-url https://finance.example.internal
Preserve the public Host header at the reverse proxy. The application rejects
other Hosts and cross-origin writes. Configure a long proxy request timeout
for bulk AI previews. Do not expose the reverse proxy to the public Internet.
A VPN-address bind can be used instead, with the corresponding -public-url.

Docker: docker compose up --build -d
Compose publishes only 127.0.0.1:8080. Set FINANCE_PUBLIC_URL when proxying it.
The container is unprivileged with a read-only root and a persistent named
volume at /data. A host bind mount must be writable by UID/GID 10001.
Never change the port mapping to public 0.0.0.0 without VPN/firewall isolation.

OpenRouter
----------
Set OPENROUTER_API_KEY in the process environment, then choose a model in
Settings. The model and chosen endpoint must support strict structured outputs
and the configured privacy routing. Every classification request sets:
  provider.data_collection = deny
  provider.zdr = true
  provider.require_parameters = true
No retry relaxes these requirements. OpenRouter must also have prompt logging
disabled in your account settings. The underlying provider processes prompts;
this is not local AI and cannot promise that a remote provider honors policy.

Amounts and currency are omitted by default. Include Amount in Settings is
explicit opt-in. Local account/provider IDs, known counterparty names, banking
identifiers and recognizable references are stripped; candidate identifiers
are per-request opaque tokens. Categories/tags and candidate merchant names
are deliberately sent as classification context. Free-form text can contain
unknown personal names, so automatic sanitization is not an anonymity guarantee.
Conservative redaction can reduce recognition quality. Inspect your descriptions
and do not configure an API key if no financial text may leave the server.

Classification failures do not discard imports: facts are committed first and
failed enrichment stays unclassified with an error visible in Transactions.
Classification requests use one transaction at a time, not batches. Known
merchant defaults can classify without any configured AI key.

Enable Banking
--------------
Register your application and public certificate with Enable Banking. For
personal production access follow its linked-own-accounts registration rules:
  https://enablebanking.com/docs/api/linked-accounts/
Configure the allowed redirect URL to the exact externally reachable URL:
  https://finance.example.internal/api/banking/callback
The browser must be able to reach this callback through your VPN.

Environment variables (all three required when enabling banking):
  ENABLEBANKING_APP_ID=<registered application ID>
  ENABLEBANKING_KEY_FILE=/run/secrets/enablebanking.key
  ENABLEBANKING_REDIRECT_URL=https://finance.example.internal/api/banking/callback
Use a PEM RSA private key (PKCS#1 or PKCS#8, at least 2048 bits). Mount it
read-only with permissions allowing the service user to read it. Never commit
it to Git. The Compose file contains a commented example key mount; set the
container-side KEY_FILE path when enabling that mount.

Accounts shows a copyable callback URL. Register it exactly with Enable Banking
and set ENABLEBANKING_REDIRECT_URL to the same value. The redirect carries a
one-time code and state, not a reusable API key. Go verifies state, exchanges the
code for a session_id, and persists session details locally with mode 0600.

Accounts -> Connect: enter the exact Enable Banking institution name and
country code (DE for Germany), then authorize through the bank. A successful
callback immediately wakes the synchronization worker; Sync now is also available.
The dashboard shows, for example, \"ING needs reconnection\" when consent expires
or is revoked. Reconnect ING starts the same approval flow with that bank and
country already selected. The new consent replaces the old account bindings
without duplicating local accounts or their financial history. Transient provider
errors are displayed separately from expired consent.

Only booked transactions are persisted. Daily sync deliberately overlaps each
account's last successful sync by 14 days; each new account first requests 90 days.
Per-account cursors prevent newly connected or reactivated accounts losing history
because another account synced recently. Older records can be imported using CSV.
A failed provider call retains local data and is retried by the daily scheduler;
Sync now can retry sooner. Balances are fetched
on demand, with exact amount/currency/type values, rather than inferred from an
incomplete historical journal.

CSV and identity
----------------
The initial real CSV adapter is N26, not generic ING/Kontist CSV autodetection.
It accepts comma/semicolon separators, German/English headers, UTF-8 BOM,
quoted multiline descriptions, ISO/German dates and decimal point/comma.
Required columns: Date / Datum / Booking Date / Buchungsdatum and
Amount (EUR) / Betrag (EUR) (or Amount/Betrag with a currency column/account).
Optional: Payee / Partner Name / Zahlungsempfaenger [with German umlaut],
Payment reference / Verwendungszweck, Account number / IBAN,
Value Date / Wertstellung, Transaction ID / Transaktions-ID.
Use the original export, not spreadsheet-reformatted dates/numbers. Foreign
original amounts/exchange-rate columns are not mistaken for account amounts.
The initial application preserves currency but never converts or sums currencies.

Stable provider entry references are preferred. Enable Banking transaction_id
is NOT guaranteed stable and is not used as the primary identity. Fallback
fingerprints retain identical-record occurrence counts: two identical rows
remain two transactions, and repeat imports do not add two more. Without stable
IDs, identical records from separately truncated exports are intrinsically
ambiguous. Import consistent overlapping/full exports. Uncertain cross-source
collisions are rejected rather than silently double counted; retain the error
and reconcile the input locally before retrying. Facts are never silently
replaced when upstream descriptions or amounts change for an existing identity.

Transfers use reciprocal records from different owned accounts, equal/opposite
exact amounts and matching currency, with own-IBAN evidence and unambiguous
matching. Ambiguous pairs are not guessed. The linked records remain separate
immutable facts; analytical double-entry postings balance and transfers do not
count as income/spending. Populate local account IBANs to support recognition.

Canonical files and recovery
----------------------------
finance/
  config.toml                 model/amount opt-in only, no API keys
  accounts.finance
  categories.finance
  tags.finance
  merchants.finance
  journal/YYYY/YYYY-MM.finance
  state/sync-state.json       sensitive local consent/session metadata
  cache/finance.duckdb        disposable analytical projection

The custom grammar is deliberately small:
  category {
    id: "cat_example"
    name: "Groceries"
    parent_id: "cat_expenses"
    kind: "expense"
  }
A transaction block has facts: {...} and enrichment: {...} JSON-valued fields.
Financial amounts are quoted decimal strings, never binary floating point.
Up to four fractional digits are supported; arithmetic uses exact ten-thousandths
with explicit overflow checks. DuckDB stores DECIMAL(24,4).

Each block starts with account/category/tag/merchant/transaction and '{' on its
own line; fields use name: JSON. Strings use JSON escaping (including \n for
multiline descriptions). JSON values may span lines. Blank lines and full-line
# or // comments are accepted between fields/blocks. Unknown fields, duplicate
keys, malformed records, invalid references and taxonomy cycles are rejected.
The grammar is version-one strict: extension/split fields are not accepted yet.
Future format extensions require an explicit parser migration.

Stable category IDs survive renaming and moving; assigned categories must remain
leaves. Built-in roots and fallback leaves are protected. Move assigned records
to another leaf before adding children to their former category. Category
merges migrate referenced transactions/defaults; tag merges deduplicate links;
tag deletion removes all affected links after UI confirmation. Merchant merging
migrates transactions and retains source names/aliases on the target merchant.

Unchanged blocks and comments retain their text. UI enrichment edits do not
rewrite imported facts. File hashes detect external edits; invalid files stop
loading/indexing with a file/line error, not a partially refreshed dashboard.
A process lock prevents multiple app writers; use one instance per finance dir.
Revision conflicts require refreshing/re-previewing, not blind overwriting.

Multi-file writes are recoverable and logically atomic within the application.
Do not run external writers during a commit; external tools do not participate
in the app's process lock. Stop the service for manual bulk edits or backups.
If a write is interrupted, keep all state files and restart for recovery before
editing the journal manually. Keep backups of the entire canonical directory,
including hidden/state recovery files, plus the separately stored secrets.
The cache can be excluded. Avoid exposing any financial directory through a
static file server, Git public remote, or unencrypted shared backup.

Rebuild from text:
  ./bin/finance-duck -data ./finance -rebuild
Stop the running app before using that command (single writer lock), or use
Settings -> Rebuild index while it runs. A broken/deleted DuckDB file can be
removed while stopped and regenerated; it never contains the only copy of
financial records. When an index rebuild fails, UI mutations still preserve
canonical data and the index error is surfaced rather than serving stale totals.

Reclassification
----------------
AI / Classification: choose dates, model and independent Merchant/Category/Tags
fields. Analyse produces a read-only preview. Apply all/selected writes all
selected changes in one canonical commit; financial facts never change. A
manual edit, external journal change or taxonomy change invalidates old previews.
Previews are kept in memory for up to one hour and disappear on restart. Cancel
writes nothing. Transfers are skipped, and unselected fields are preserved.
Failed rows remain unchanged and are listed separately from proposed changes.

Boundaries and verification
---------------------------
There are no splits, budgets, investments, tax/invoice/receipt processing,
login/multi-user support, arbitrary SQL or natural-language query execution.
Natural-language query DSL and Sankey exploration remain explicitly later work.
There is no browser-to-bank credential handling or payment initiation.

Automated tests exercise deterministic financial invariants and mock remote
provider HTTP behavior. They do not replace testing real consent renewals and
real booked transaction samples for your banks. No real user credentials or
financial history are included in the repository. Test fixtures are synthetic.

API documentation sources:
  https://enablebanking.com/docs/api/reference/
  https://openrouter.ai/docs/guides/features/structured-outputs
  https://openrouter.ai/docs/guides/features/zdr
  https://openrouter.ai/docs/guides/routing/provider-selection