init
This commit is contained in:
Lars Nolden
+224
@@ -0,0 +1,224 @@
|
||||
FINANCE DUCK — GO + REACT
|
||||
|
||||
Local build on NixOS
|
||||
-------------------
|
||||
nix-shell
|
||||
npm --prefix web ci
|
||||
npm --prefix web run build
|
||||
go build -o bin/finance-duck ./cmd/finance-duck
|
||||
./bin/finance-duck -data ./finance
|
||||
|
||||
Open http://localhost:8080. Create an account, then import its N26 CSV from
|
||||
Accounts. The application starts empty except for expense/income fallback
|
||||
categories. Create your category tree, tags, and merchants in the UI. Enable a
|
||||
merchant's default rule explicitly only when its category/tags are reliable;
|
||||
leave it disabled for ambiguous merchants such as Amazon.
|
||||
|
||||
Tests: go test ./...
|
||||
The Go build embeds web/dist, so build React first. CGO and a C++ linker are
|
||||
required by the native DuckDB driver. shell.nix supplies the toolchain. React
|
||||
and CSS are served locally; there are no CDN requests or tracking scripts.
|
||||
|
||||
Private deployment
|
||||
------------------
|
||||
There is NO application login. Default bind: 127.0.0.1:8080.
|
||||
For a VPN/reverse-proxy hostname, use an exact browser origin:
|
||||
./bin/finance-duck -data /srv/finance -listen 127.0.0.1:8080 \
|
||||
-public-url https://finance.example.internal
|
||||
Preserve the public Host header at the reverse proxy. The application rejects
|
||||
other Hosts and cross-origin writes. Configure a long proxy request timeout
|
||||
for bulk AI previews. Do not expose the reverse proxy to the public Internet.
|
||||
A VPN-address bind can be used instead, with the corresponding -public-url.
|
||||
|
||||
Docker: docker compose up --build -d
|
||||
Compose publishes only 127.0.0.1:8080. Set FINANCE_PUBLIC_URL when proxying it.
|
||||
The container is unprivileged with a read-only root and a persistent named
|
||||
volume at /data. A host bind mount must be writable by UID/GID 10001.
|
||||
Never change the port mapping to public 0.0.0.0 without VPN/firewall isolation.
|
||||
|
||||
OpenRouter
|
||||
----------
|
||||
Set OPENROUTER_API_KEY in the process environment, then choose a model in
|
||||
Settings. The model and chosen endpoint must support strict structured outputs
|
||||
and the configured privacy routing. Every classification request sets:
|
||||
provider.data_collection = deny
|
||||
provider.zdr = true
|
||||
provider.require_parameters = true
|
||||
No retry relaxes these requirements. OpenRouter must also have prompt logging
|
||||
disabled in your account settings. The underlying provider processes prompts;
|
||||
this is not local AI and cannot promise that a remote provider honors policy.
|
||||
|
||||
Amounts and currency are omitted by default. Include Amount in Settings is
|
||||
explicit opt-in. Local account/provider IDs, known counterparty names, banking
|
||||
identifiers and recognizable references are stripped; candidate identifiers
|
||||
are per-request opaque tokens. Categories/tags and candidate merchant names
|
||||
are deliberately sent as classification context. Free-form text can contain
|
||||
unknown personal names, so automatic sanitization is not an anonymity guarantee.
|
||||
Conservative redaction can reduce recognition quality. Inspect your descriptions
|
||||
and do not configure an API key if no financial text may leave the server.
|
||||
|
||||
Classification failures do not discard imports: facts are committed first and
|
||||
failed enrichment stays unclassified with an error visible in Transactions.
|
||||
Classification requests use one transaction at a time, not batches. Known
|
||||
merchant defaults can classify without any configured AI key.
|
||||
|
||||
Enable Banking
|
||||
--------------
|
||||
Register your application and public certificate with Enable Banking. For
|
||||
personal production access follow its linked-own-accounts registration rules:
|
||||
https://enablebanking.com/docs/api/linked-accounts/
|
||||
Configure the allowed redirect URL to the exact externally reachable URL:
|
||||
https://finance.example.internal/api/banking/callback
|
||||
The browser must be able to reach this callback through your VPN.
|
||||
|
||||
Environment variables (all three required when enabling banking):
|
||||
ENABLEBANKING_APP_ID=<registered application ID>
|
||||
ENABLEBANKING_KEY_FILE=/run/secrets/enablebanking.key
|
||||
ENABLEBANKING_REDIRECT_URL=https://finance.example.internal/api/banking/callback
|
||||
Use a PEM RSA private key (PKCS#1 or PKCS#8, at least 2048 bits). Mount it
|
||||
read-only with permissions allowing the service user to read it. Never commit
|
||||
it to Git. The Compose file contains a commented example key mount; set the
|
||||
container-side KEY_FILE path when enabling that mount.
|
||||
|
||||
Accounts shows a copyable callback URL. Register it exactly with Enable Banking
|
||||
and set ENABLEBANKING_REDIRECT_URL to the same value. The redirect carries a
|
||||
one-time code and state, not a reusable API key. Go verifies state, exchanges the
|
||||
code for a session_id, and persists session details locally with mode 0600.
|
||||
|
||||
Accounts -> Connect: enter the exact Enable Banking institution name and
|
||||
country code (DE for Germany), then authorize through the bank. A successful
|
||||
callback immediately wakes the synchronization worker; Sync now is also available.
|
||||
The dashboard shows, for example, \"ING needs reconnection\" when consent expires
|
||||
or is revoked. Reconnect ING starts the same approval flow with that bank and
|
||||
country already selected. The new consent replaces the old account bindings
|
||||
without duplicating local accounts or their financial history. Transient provider
|
||||
errors are displayed separately from expired consent.
|
||||
|
||||
Only booked transactions are persisted. Daily sync deliberately overlaps each
|
||||
account's last successful sync by 14 days; each new account first requests 90 days.
|
||||
Per-account cursors prevent newly connected or reactivated accounts losing history
|
||||
because another account synced recently. Older records can be imported using CSV.
|
||||
A failed provider call retains local data and is retried by the daily scheduler;
|
||||
Sync now can retry sooner. Balances are fetched
|
||||
on demand, with exact amount/currency/type values, rather than inferred from an
|
||||
incomplete historical journal.
|
||||
|
||||
CSV and identity
|
||||
----------------
|
||||
The initial real CSV adapter is N26, not generic ING/Kontist CSV autodetection.
|
||||
It accepts comma/semicolon separators, German/English headers, UTF-8 BOM,
|
||||
quoted multiline descriptions, ISO/German dates and decimal point/comma.
|
||||
Required columns: Date / Datum / Booking Date / Buchungsdatum and
|
||||
Amount (EUR) / Betrag (EUR) (or Amount/Betrag with a currency column/account).
|
||||
Optional: Payee / Partner Name / Zahlungsempfaenger [with German umlaut],
|
||||
Payment reference / Verwendungszweck, Account number / IBAN,
|
||||
Value Date / Wertstellung, Transaction ID / Transaktions-ID.
|
||||
Use the original export, not spreadsheet-reformatted dates/numbers. Foreign
|
||||
original amounts/exchange-rate columns are not mistaken for account amounts.
|
||||
The initial application preserves currency but never converts or sums currencies.
|
||||
|
||||
Stable provider entry references are preferred. Enable Banking transaction_id
|
||||
is NOT guaranteed stable and is not used as the primary identity. Fallback
|
||||
fingerprints retain identical-record occurrence counts: two identical rows
|
||||
remain two transactions, and repeat imports do not add two more. Without stable
|
||||
IDs, identical records from separately truncated exports are intrinsically
|
||||
ambiguous. Import consistent overlapping/full exports. Uncertain cross-source
|
||||
collisions are rejected rather than silently double counted; retain the error
|
||||
and reconcile the input locally before retrying. Facts are never silently
|
||||
replaced when upstream descriptions or amounts change for an existing identity.
|
||||
|
||||
Transfers use reciprocal records from different owned accounts, equal/opposite
|
||||
exact amounts and matching currency, with own-IBAN evidence and unambiguous
|
||||
matching. Ambiguous pairs are not guessed. The linked records remain separate
|
||||
immutable facts; analytical double-entry postings balance and transfers do not
|
||||
count as income/spending. Populate local account IBANs to support recognition.
|
||||
|
||||
Canonical files and recovery
|
||||
----------------------------
|
||||
finance/
|
||||
config.toml model/amount opt-in only, no API keys
|
||||
accounts.finance
|
||||
categories.finance
|
||||
tags.finance
|
||||
merchants.finance
|
||||
journal/YYYY/YYYY-MM.finance
|
||||
state/sync-state.json sensitive local consent/session metadata
|
||||
cache/finance.duckdb disposable analytical projection
|
||||
|
||||
The custom grammar is deliberately small:
|
||||
category {
|
||||
id: "cat_example"
|
||||
name: "Groceries"
|
||||
parent_id: "cat_expenses"
|
||||
kind: "expense"
|
||||
}
|
||||
A transaction block has facts: {...} and enrichment: {...} JSON-valued fields.
|
||||
Financial amounts are quoted decimal strings, never binary floating point.
|
||||
Up to four fractional digits are supported; arithmetic uses exact ten-thousandths
|
||||
with explicit overflow checks. DuckDB stores DECIMAL(24,4).
|
||||
|
||||
Each block starts with account/category/tag/merchant/transaction and '{' on its
|
||||
own line; fields use name: JSON. Strings use JSON escaping (including \n for
|
||||
multiline descriptions). JSON values may span lines. Blank lines and full-line
|
||||
# or // comments are accepted between fields/blocks. Unknown fields, duplicate
|
||||
keys, malformed records, invalid references and taxonomy cycles are rejected.
|
||||
The grammar is version-one strict: extension/split fields are not accepted yet.
|
||||
Future format extensions require an explicit parser migration.
|
||||
|
||||
Stable category IDs survive renaming and moving; assigned categories must remain
|
||||
leaves. Built-in roots and fallback leaves are protected. Move assigned records
|
||||
to another leaf before adding children to their former category. Category
|
||||
merges migrate referenced transactions/defaults; tag merges deduplicate links;
|
||||
tag deletion removes all affected links after UI confirmation. Merchant merging
|
||||
migrates transactions and retains source names/aliases on the target merchant.
|
||||
|
||||
Unchanged blocks and comments retain their text. UI enrichment edits do not
|
||||
rewrite imported facts. File hashes detect external edits; invalid files stop
|
||||
loading/indexing with a file/line error, not a partially refreshed dashboard.
|
||||
A process lock prevents multiple app writers; use one instance per finance dir.
|
||||
Revision conflicts require refreshing/re-previewing, not blind overwriting.
|
||||
|
||||
Multi-file writes are recoverable and logically atomic within the application.
|
||||
Do not run external writers during a commit; external tools do not participate
|
||||
in the app's process lock. Stop the service for manual bulk edits or backups.
|
||||
If a write is interrupted, keep all state files and restart for recovery before
|
||||
editing the journal manually. Keep backups of the entire canonical directory,
|
||||
including hidden/state recovery files, plus the separately stored secrets.
|
||||
The cache can be excluded. Avoid exposing any financial directory through a
|
||||
static file server, Git public remote, or unencrypted shared backup.
|
||||
|
||||
Rebuild from text:
|
||||
./bin/finance-duck -data ./finance -rebuild
|
||||
Stop the running app before using that command (single writer lock), or use
|
||||
Settings -> Rebuild index while it runs. A broken/deleted DuckDB file can be
|
||||
removed while stopped and regenerated; it never contains the only copy of
|
||||
financial records. When an index rebuild fails, UI mutations still preserve
|
||||
canonical data and the index error is surfaced rather than serving stale totals.
|
||||
|
||||
Reclassification
|
||||
----------------
|
||||
AI / Classification: choose dates, model and independent Merchant/Category/Tags
|
||||
fields. Analyse produces a read-only preview. Apply all/selected writes all
|
||||
selected changes in one canonical commit; financial facts never change. A
|
||||
manual edit, external journal change or taxonomy change invalidates old previews.
|
||||
Previews are kept in memory for up to one hour and disappear on restart. Cancel
|
||||
writes nothing. Transfers are skipped, and unselected fields are preserved.
|
||||
Failed rows remain unchanged and are listed separately from proposed changes.
|
||||
|
||||
Boundaries and verification
|
||||
---------------------------
|
||||
There are no splits, budgets, investments, tax/invoice/receipt processing,
|
||||
login/multi-user support, arbitrary SQL or natural-language query execution.
|
||||
Natural-language query DSL and Sankey exploration remain explicitly later work.
|
||||
There is no browser-to-bank credential handling or payment initiation.
|
||||
|
||||
Automated tests exercise deterministic financial invariants and mock remote
|
||||
provider HTTP behavior. They do not replace testing real consent renewals and
|
||||
real booked transaction samples for your banks. No real user credentials or
|
||||
financial history are included in the repository. Test fixtures are synthetic.
|
||||
|
||||
API documentation sources:
|
||||
https://enablebanking.com/docs/api/reference/
|
||||
https://openrouter.ai/docs/guides/features/structured-outputs
|
||||
https://openrouter.ai/docs/guides/features/zdr
|
||||
https://openrouter.ai/docs/guides/routing/provider-selection
|
||||
Reference in New Issue
Block a user