Import ING and Kontist statements behind a reviewed column mapping

CSV import is now mapping-driven: N26, ING (metadata preamble, Windows-1252,
German decimals) and Kontist exports are recognized locally, and any other
layout can have its columns proposed by the configured model from a sample in
which letters are replaced by x and digits by 0. Proposals are untrusted: every
column must name a supplied header, money must come from one signed column or
one debit/credit pair, and formats must be from a closed list.

Uploading no longer imports. /api/import is replaced by prepare/confirm/cancel:
prepare parses, deduplicates and previews the exact facts, and only confirming
at the reviewed revision writes them. ING and AI-mapped facts carry no
transaction reference, because repeating SEPA mandate references must never
become a transaction identity.
This commit is contained in:
Lars Nolden
2026-09-11 17:49:03 +02:00
parent 6f791b1277
commit dc767799bc
16 changed files with 2182 additions and 301 deletions
+45 -18
View File
@@ -8,11 +8,12 @@ npm --prefix web run build
go build -o bin/finance-duck ./cmd/finance-duck
./bin/finance-duck -data ./finance
Open http://localhost:8080. Create an account, then import its N26 CSV from
Accounts. The application starts empty except for expense/income fallback
categories. Create your category tree, tags, and merchants in the UI. Enable a
merchant's default rule explicitly only when its category/tags are reliable;
leave it disabled for ambiguous merchants such as Amazon.
Open http://localhost:8080. Create an account, then import its CSV statement
from Accounts and confirm the reviewed mapping. The application starts empty
except for expense/income fallback categories. Create your category tree, tags,
and merchants in the UI. Enable a merchant's default rule explicitly only when
its category/tags are reliable; leave it disabled for ambiguous merchants such
as Amazon.
Tests: go test ./...
The Go build embeds web/dist, so build React first. CGO and a C++ linker are
@@ -290,19 +291,45 @@ changed. Inactive connected accounts may import history manually. Expired or
revoked bank authorization must be reconnected first. On failure, the UI refreshes
the journal to reveal any records already committed; it does not retry the import.
CSV and identity
----------------
The initial real CSV adapter is N26, not generic ING/Kontist CSV autodetection.
It accepts comma/semicolon separators, German/English headers, UTF-8 BOM,
quoted multiline descriptions, ISO/German dates and decimal point/comma.
Required columns: Date / Datum / Booking Date / Buchungsdatum and
Amount (EUR) / Betrag (EUR) (or Amount/Betrag with a currency column/account).
Optional: Payee / Partner Name / Zahlungsempfaenger [with German umlaut],
Payment reference / Verwendungszweck, Account number / IBAN,
Value Date / Wertstellung, Transaction ID / Transaktions-ID.
Use the original export, not spreadsheet-reformatted dates/numbers. Foreign
original amounts/exchange-rate columns are not mistaken for account amounts.
The initial application preserves currency but never converts or sums currencies.
CSV import and identity
-----------------------
Import is a two-step flow: upload prepares a mapping and a preview, and nothing
reaches the journal until it is confirmed. Cancel, a browser reload, a failed
parse or a journal change between preview and confirmation all import nothing.
A prepared statement is held in memory only, expires after one hour, and at most
five may await confirmation. Uploads are limited to 2 MiB.
Recognized locally, without AI: N26 (English/German, old Date/Datum and new
Booking Date/Buchungsdatum schemas), ING Umsatzanzeige (Buchung,
Wertstellungsdatum, Auftraggeber/Empfaenger, Buchungstext, Verwendungszweck,
Betrag, Waehrung, below its metadata preamble), and Kontist's documented
transaction vocabulary (Payment Date/Buchungsdatum, Name, Amount/Betrag,
Purpose/Verwendungszweck). Accepted everywhere: comma/semicolon/tab separators,
UTF-8 with or without BOM, Windows-1252, CRLF, quoted multiline descriptions,
ISO and German dates, decimal point and comma, and thousands groups of exactly
three digits. Kontist date and decimal conventions are inferred from the file's
own first populated values; slash dates are read as month/day/year unless the
first component exceeds twelve. Foreign original amounts, exchange rates,
balances and categories are never used as account money. Currency comes from a
currency column, else an "Amount (EUR)"-style header, else the account; a
conflict with the account currency fails the whole statement. Use the original
export rather than spreadsheet-reformatted dates and numbers. Currency is
preserved but never converted or summed across currencies.
Any other layout requires a saved OpenRouter key and model: the columns are
mapped by the model, not the records. The request contains the delimiter, the
column names and up to four sample rows in which every letter is replaced by x
and every digit by 0, so descriptions, counterparties, references, IBANs and
amounts are not sent. The proposal is untrusted: each column must name a
supplied header exactly, money must come from one signed column or one
debit/credit pair, the date and decimal conventions must be from the supported
list, and no column may serve two fields. AI-mapped facts are recorded with
source "csv" and carry no transaction reference, because a repeating SEPA
mandate reference must never become a transaction identity. ING facts likewise
carry no reference. Review the previewed dates, amount signs and currency before
confirming; a wrong mapping is visible there, not after import.
Import sources: n26_csv, ing_csv, kontist_csv, csv (AI-mapped), enablebanking.
Stable provider entry references are scoped by account, source and debit/credit
direction: a debit and credit can share a reference without being collapsed.