Track investments as broker facts with a position leg

An account now has a kind, and an investment account holds positions as well as
cash. A broker row is not a new entity: it is a bank fact with an optional
position leg, so deduplication, the journal, fact immutability, the DuckDB
projection and the transactions view carry it unchanged. Facts.Amount stays the
cash leg and is zero on the rows that move only a position.

Scalable Capital exports are recognized locally as a fourth format, read by
their own parser because a column mapping cannot describe them: the amount
column is settled cash on a cash row, a gross to be netted on a trade, and a
position valuation that must never touch cash on a corporate action or a depot
transfer. A cash amount is already net of the tax the broker withheld or
refunded, so that tax is recorded on the fact and never subtracted a second
time; treating a corporate action's valuation as money conjures cash, and a
depot switch would do it once per instrument. The share column is signed only
for those two types, so buys and sells take their direction from the type. Every
security row is checked against shares times price at 128-bit width, because a
lost decimal separator survives every other check. An unknown status, type or
assetType, a foreign currency, a missing ISIN, or one failed check rejects the
whole file with the record number.

Instruments live in instruments.finance, keyed by ISIN with an ID derived from
it, so re-importing never registers a security twice. One ISIN appears under
several broker descriptions over the years and sometimes under the ISIN itself:
the most recent real description names it, and an import never renames one that
already exists. A broker also reuses a single reference across every leg of one
event, so transaction identity includes the event and its instrument.

domain.Fallback returns kind "investment" for any fact carrying a position leg,
so no broker row reaches the sign-based branch. That single rule is what stops
an unmatched deposit from counting as income and a broker fee from counting as
household spending; the monthly PRIME fee and its matching credit now cancel in
clearing:investments with no configuration at all. Investment rows are excluded
from spending analytics, from bulk reclassification and from the model, exactly
as transfers are.

Equal competing transfers are paired instead of skipped. Every connected
component of the candidate graph is a complete bipartite graph between two fixed
accounts at one amount and currency, so every pairing produces the same
accounts, kinds and postings and only the displayed counterpart differs.
Refusing to choose was the expensive option: both legs fell through to the
sign-based fallback and appeared as spending and income that never happened.
Pairing follows the nearest booking date, then the transaction ID, so iteration
order decides nothing. POST /api/transactions/{id}/transfer rewrites the old and
the new pair in one commit, because reciprocity is validated and a half-applied
link is an invalid dataset, and the matcher now skips any record classified
manually so a hand-made link or unlink outlives the next import.

Wealth reports each account's cash and positions from the journal rather than
the index, with named checks - row arithmetic, cash never negative, holdings
never negative - because it exists to be compared against the figures a broker
shows on its own screen. A negative holding means the imported history is
partial. Share counts are exact to eight places; a reinvested distribution
quoted to six is rounded to money's four and the residue is reported rather than
hidden. Market prices, market value, net worth over time, FIFO lot accounting,
realised gains and currency conversion are deliberately absent.
This commit is contained in:
Lars Nolden
2026-09-11 21:58:47 +02:00
parent 673cbf917b
commit 922ae507bd
27 changed files with 3071 additions and 157 deletions
+147 -13
View File
@@ -351,7 +351,8 @@ mandate reference must never become a transaction identity. ING facts likewise
carry no reference. Review the previewed dates, amount signs and currency before carry no reference. Review the previewed dates, amount signs and currency before
confirming; a wrong mapping is visible there, not after import. confirming; a wrong mapping is visible there, not after import.
Import sources: n26_csv, ing_csv, kontist_csv, csv (AI-mapped), enablebanking. Import sources: n26_csv, ing_csv, kontist_csv, scalable_csv, csv (AI-mapped),
enablebanking.
Stable provider entry references are scoped by account, source and debit/credit Stable provider entry references are scoped by account, source and debit/credit
direction: a debit and credit can share a reference without being collapsed. direction: a debit and credit can share a reference without being collapsed.
@@ -368,10 +369,131 @@ and reconcile the input locally before retrying. Facts are never silently
replaced when upstream descriptions or amounts change for an existing identity. replaced when upstream descriptions or amounts change for an existing identity.
Transfers use reciprocal records from different owned accounts, equal/opposite Transfers use reciprocal records from different owned accounts, equal/opposite
exact amounts and matching currency, with own-IBAN evidence and unambiguous exact amounts and matching currency, with own-IBAN evidence and booking dates
matching. Ambiguous pairs are not guessed. The linked records remain separate within three calendar days. Equal competing payments ARE paired, by nearest
immutable facts; analytical double-entry postings balance and transfers do not booking date and then by transaction ID: every candidate set is a complete
count as income/spending. Populate local account IBANs to support recognition. bipartite graph between two fixed accounts at one amount and currency, so every
pairing yields the same accounts, kinds and postings, and iteration order
decides nothing. Leaving them unpaired was the worse option, because both legs
then fell through to the sign-based fallback and appeared as spending and income
that never happened. An existing link is never revisited, and neither is a
record whose classification source is "manual": a hand-made link or unlink
outlives every later import. The linked records remain separate immutable
facts; analytical double-entry postings balance and transfers do not count as
income/spending. Populate local account IBANs to support recognition.
Investment accounts and broker imports
--------------------------------------
An account has a kind, "cash" (the default, and what an absent kind means) or
"investment". An investment account holds a cash balance and positions. It also
carries a settlement IBAN (reference_iban): a broker export has no counterparty
column, so deposits and withdrawals are stamped with that IBAN and pair with the
funding account through ordinary transfer matching. Leave it empty and those
rows simply stay unpaired, which costs accuracy in spending analysis but never
invents income.
Scalable Capital exports (scalable_csv) are recognized locally by their full
column set: date, time, status, reference, description, assetType, type, isin,
shares, price, amount, fee, tax, currency. The layout is matched whole, because
a row's meaning depends on the combination of status, assetType and type.
The booking date is the date column exactly as printed. Batch rows are stamped
midnight UTC rendered in local time, so their time column reads 01:00 in winter
and 02:00 in summer; reading date and time together would move half the year's
corporate actions and distributions to the previous day.
Only status "Executed" imports. A cancelled retry is all zeros, so it satisfies
every arithmetic check and would otherwise enter the journal as a phantom trade.
The ten row types, and what each settles:
type assetType cash position
Deposit Cash amount -
Withdrawal Cash amount -
Fee Cash amount -
Interest Cash amount -
Distribution Cash amount -
Buy Security amount - fee - tax +shares
Sell Security amount - fee - tax -shares
Reinvestment_Distribution Security amount - fee - tax +shares
Corporate action Security NONE shares as printed
Security transfer Security NONE shares as printed
A cash row's amount is the money that actually settled and is already net of
the tax the broker withheld or refunded, so its fee and tax columns are recorded
on the fact and never subtracted again. Subtracting them a second time
double-counts by exactly the tax figure. A security row's amount is a gross
pinned to shares times price. A corporate action or depot transfer quotes a
position valuation, not cash: treating it as money conjures or destroys it, and
a depot switch of a whole portfolio does that once per instrument.
The share column is signed only for corporate actions and depot transfers. Buys
and sells are unsigned and take their direction from the type. Both conventions
are resolved at import, once.
Every security row is checked against shares times price at full precision.
This is the only check that catches a lost decimal separator, and it cannot
catch one that was lost uniformly across a row: 1 x 25,795 and 1 x 25795 both
satisfy it. A price cross-check against an outside provider is the only remedy
and is deliberately not implemented.
Rejected whole, with the record number: an unknown status, an unknown type, an
assetType that disagrees with its type, a currency other than the account's, a
security row without an ISIN, an invalid ISIN, a signed buy or sell, a corporate
action or depot transfer carrying a fee or tax, and any failed arithmetic check.
A zero amount is accepted; it corrupts nothing, and a free share allocation is
legitimately priced at zero.
Money holds four decimal places and share counts hold eight. A reinvested
distribution is quoted to six, so its amount is rounded half away from zero and
the exact residue is reported in the import review and never hidden. A share
count beyond eight places is refused rather than truncated, because a holding is
verified against the broker's own figure.
Instruments are registered from the export, keyed by ISIN, with an ID derived
from the ISIN so re-importing never creates a second entry for one security. One
ISIN appears under several descriptions over the years and sometimes under the
ISIN itself; the most recent real description names it, and an import never
renames an instrument that already exists. The name is editable display text;
the ISIN is identity and cannot be changed.
A broker reuses one reference across every leg of an economic event: the cash
and position sides of a corporate action arrive with the same reference byte for
byte, and the position leg's zero amount does not even differ in direction.
Transaction identity therefore includes the event and its instrument. The
reference itself also embeds an account-level identifier that repeats across
unrelated events, so it is evidence of an event, never of a transaction.
Broker facts carry enrichment kind "investment". Like a transfer it has no
category and no merchant, it is excluded from spending and income analytics and
from bulk reclassification, and the AI never sees it. Crucially, a broker fact
never reaches the sign-based fallback, so an unmatched deposit is not income and
a broker fee is not household spending. Analytical postings route it to
clearing:investments, where the residue left behind is exactly the cash an
investment account has returned: distributions and interest received, less fees.
Wealth and reconciliation
-------------------------
The Wealth page reports, per account, the cash balance as every recorded
movement summed, the positions as every signed share count summed, and named
checks. It is computed from the journal, not from the DuckDB index, because it
exists to be compared with the figures a bank or broker shows on its own screen.
A cash balance equals the real balance only when the journal holds that
account's complete history. A broker export does; a date-windowed bank statement
does not.
Checks that fail mean the journal disagrees with itself: row arithmetic, cash
never negative, holdings never negative. A negative holding means a position was
closed that was never opened in the imported data, so the export is partial or a
sign is wrong. Checks that only note: fee and tax recorded but not applied, and
deposits or withdrawals with no counterpart in another account.
Out of scope, deliberately: market prices, market value, net worth over time,
FIFO lot accounting, realised gains, Vorabpauschale, and currency conversion. A
position's "invested" figure is cash in less cash out, not a cost basis: a depot
transfer moves a position with no cash at all, and a sale returns cash without
identifying which lot it closed.
Canonical files and recovery Canonical files and recovery
---------------------------- ----------------------------
@@ -381,6 +503,7 @@ finance/
categories.finance categories.finance
tags.finance tags.finance
merchants.finance merchants.finance
instruments.finance
journal/YYYY/YYYY-MM.finance journal/YYYY/YYYY-MM.finance
state/sync-state.json sensitive local consent/session metadata state/sync-state.json sensitive local consent/session metadata
state/openrouter.json sensitive UI-managed OpenRouter key or explicit disable state/openrouter.json sensitive UI-managed OpenRouter key or explicit disable
@@ -395,15 +518,22 @@ The custom grammar is deliberately small:
kind: "expense" kind: "expense"
} }
A transaction block has facts: {...} and enrichment: {...} JSON-valued fields. A transaction block has facts: {...} and enrichment: {...} JSON-valued fields.
A broker fact additionally carries an investment: {...} object holding the
event, instrument, signed quantity, price, gross, fee and tax; absent fields are
omitted, and its presence is what marks a fact as a broker fact.
Financial amounts are quoted decimal strings, never binary floating point. Financial amounts are quoted decimal strings, never binary floating point.
Up to four fractional digits are supported; arithmetic uses exact ten-thousandths Up to four fractional digits are supported; arithmetic uses exact ten-thousandths
with explicit overflow checks. DuckDB stores DECIMAL(24,4). with explicit overflow checks. DuckDB stores DECIMAL(24,4).
Share quantities are quoted decimal strings with up to eight fractional digits,
arithmetic uses exact hundred-millionths, and a quantity times a price is
multiplied at 128-bit width before rounding back to four places.
Each block starts with account/category/tag/merchant/transaction and '{' on its Each block starts with account/category/tag/merchant/instrument/transaction and
own line; fields use name: JSON. Strings use JSON escaping (including \n for '{' on its own line; fields use name: JSON. Strings use JSON escaping (including
multiline descriptions). JSON values may span lines. Blank lines and full-line \n for multiline descriptions). JSON values may span lines. Blank lines and
# or // comments are accepted between fields/blocks. Unknown fields, duplicate full-line # or // comments are accepted between fields/blocks. Unknown fields,
keys, malformed records, invalid references and taxonomy cycles are rejected. duplicate keys, malformed records, invalid references and taxonomy cycles are
rejected.
The grammar is version-one strict: extension/split fields are not accepted yet. The grammar is version-one strict: extension/split fields are not accepted yet.
Future format extensions require an explicit parser migration. Future format extensions require an explicit parser migration.
@@ -444,13 +574,17 @@ fields. Analyse produces a read-only preview. Apply all/selected writes all
selected changes in one canonical commit; financial facts never change. A selected changes in one canonical commit; financial facts never change. A
manual edit, external journal change or taxonomy change invalidates old previews. manual edit, external journal change or taxonomy change invalidates old previews.
Previews are kept in memory for up to one hour and disappear on restart. Cancel Previews are kept in memory for up to one hour and disappear on restart. Cancel
writes nothing. Transfers are skipped, and unselected fields are preserved. writes nothing. Transfers and broker facts are skipped, and unselected fields
are preserved.
Failed rows remain unchanged and are listed separately from proposed changes. Failed rows remain unchanged and are listed separately from proposed changes.
Boundaries and verification Boundaries and verification
--------------------------- ---------------------------
There are no splits, budgets, investments, tax/invoice/receipt processing, There are no splits, budgets, tax/invoice/receipt processing, login/multi-user
login/multi-user support, arbitrary SQL or natural-language query execution. support, arbitrary SQL or natural-language query execution. Investment support
covers positions and cash, not valuation: no market prices, market value,
net worth over time, FIFO lots, realised gains, Vorabpauschale or currency
conversion.
Natural-language query DSL and Sankey exploration remain explicitly later work. Natural-language query DSL and Sankey exploration remain explicitly later work.
There is no browser-to-bank credential handling or payment initiation. There is no browser-to-bank credential handling or payment initiation.
+24 -2
View File
@@ -2,7 +2,7 @@
A self-hosted personal finance dashboard with a **Go backend**, **React frontend**, and **DuckDB analytics**. Human-readable `.finance` journals are the source of truth; DuckDB is a disposable index. A self-hosted personal finance dashboard with a **Go backend**, **React frontend**, and **DuckDB analytics**. Human-readable `.finance` journals are the source of truth; DuckDB is a disposable index.
Imported bank facts are separate from editable merchant, category, and tag classifications. N26, ING, and Kontist CSV imports and bank synchronization work without AI. Optional OpenRouter enrichment uses restrictive provider routing and omits amounts by default, and can map the columns of an unrecognized CSV layout from a redacted sample. Imported bank facts are separate from editable merchant, category, and tag classifications. N26, ING, Kontist, and Scalable Capital CSV imports and bank synchronization work without AI. An investment account tracks positions by ISIN alongside its cash, and reconciles both against your broker's own figures. Optional OpenRouter enrichment uses restrictive provider routing and omits amounts by default, and can map the columns of an unrecognized CSV layout from a redacted sample.
> **There is no application login.** Keep Finance Duck behind your VPN. The default service and Docker Compose port bindings are loopback-only. Setting a hostname does not provide authentication or firewall protection. > **There is no application login.** Keep Finance Duck behind your VPN. The default service and Docker Compose port bindings are loopback-only. Setting a hostname does not provide authentication or firewall protection.
@@ -174,12 +174,34 @@ Reconnecting renews bank consent, not your application registration. Correct cer
Open **Accounts → Import a statement**, choose the account, select the export, and click **Review statement**. Uploading imports nothing: it parses the file and opens a review dialog showing the detected export, the column mapping, how many records are new or already imported, and a sample of the parsed transactions with their dates, descriptions, counterparties, and signed amounts. **Import N transactions** commits exactly those records; **Cancel**, a reload, or a journal change in between commits nothing. Open **Accounts → Import a statement**, choose the account, select the export, and click **Review statement**. Uploading imports nothing: it parses the file and opens a review dialog showing the detected export, the column mapping, how many records are new or already imported, and a sample of the parsed transactions with their dates, descriptions, counterparties, and signed amounts. **Import N transactions** commits exactly those records; **Cancel**, a reload, or a journal change in between commits nothing.
**N26**, **ING** (Umsatzanzeige, including its metadata preamble and Windows-1252 encoding), and **Kontist** exports are recognized on your own machine, with no AI involved. Comma, semicolon, and tab separators, UTF-8 with or without BOM, CRLF, quoted multiline descriptions, ISO and German dates, and both decimal separators are accepted. Use the bank's original export rather than a spreadsheet-reformatted copy. Uploads are limited to **2 MiB**, and a prepared statement expires after **one hour**. **N26**, **ING** (Umsatzanzeige, including its metadata preamble and Windows-1252 encoding), **Kontist**, and **Scalable Capital** exports are recognized on your own machine, with no AI involved. Comma, semicolon, and tab separators, UTF-8 with or without BOM, CRLF, quoted multiline descriptions, ISO and German dates, and both decimal separators are accepted. Use the bank's original export rather than a spreadsheet-reformatted copy — a spreadsheet round-trip is what drops a decimal comma. Uploads are limited to **2 MiB**, and a prepared statement expires after **one hour**.
Any other layout needs a saved OpenRouter key and model, which maps the **columns** rather than reading the transactions: the request carries the delimiter, the column names, and up to four sample rows in which every letter is replaced by `x` and every digit by `0`. Descriptions, counterparties, references, IBANs, and amounts are never sent. The proposal must name existing columns, choose exactly one money convention (one signed amount column, or a debit and credit pair), and use a supported date and decimal format; anything else is rejected instead of guessed. Because a proposed mapping can still be wrong, check the sample's dates, signs, and currency before confirming. Any other layout needs a saved OpenRouter key and model, which maps the **columns** rather than reading the transactions: the request carries the delimiter, the column names, and up to four sample rows in which every letter is replaced by `x` and every digit by `0`. Descriptions, counterparties, references, IBANs, and amounts are never sent. The proposal must name existing columns, choose exactly one money convention (one signed amount column, or a debit and credit pair), and use a supported date and decimal format; anything else is rejected instead of guessed. Because a proposed mapping can still be wrong, check the sample's dates, signs, and currency before confirming.
Reimporting the same statement adds nothing: the review dialog reports the overlap as already imported. A statement whose currency conflicts with the account, or whose records cannot be parsed, is rejected whole rather than imported in part. Reimporting the same statement adds nothing: the review dialog reports the overlap as already imported. A statement whose currency conflicts with the account, or whose records cannot be parsed, is rejected whole rather than imported in part.
## Track investments
Set an account's **kind** to **Investment** in **Accounts**, then import a **Scalable Capital** transaction export into it. The account then holds both a cash balance and positions, and **Wealth** reports them.
A broker export is not a list of interchangeable statement lines, so it is read by its own parser rather than by a column mapping. The same `amount` column means three different things:
| Row | `amount` is | Settles |
| --- | --- | --- |
| `Deposit`, `Withdrawal`, `Fee`, `Interest`, `Distribution` | the money that moved, **already net of tax** | cash only |
| `Buy`, `Sell`, `Reinvestment_Distribution` | a gross, pinned to shares × price | `amount fee tax`, plus the position |
| `Corporate action`, `Security transfer` | a **position valuation** | **no cash at all** |
Because a cash row's amount already includes the tax the broker withheld or refunded, that tax is recorded on the record and never subtracted again; the review dialog lists every such figure before you confirm. Corporate actions and depot transfers move a position without moving money — treating their amount as cash would invent or destroy it, and a depot switch does that once per instrument.
Only `Executed` rows import: a cancelled retry is all zeros, so it passes every arithmetic check and would otherwise become a phantom trade. Every security row is verified against shares × price at full precision. An unknown row type, an unknown status, a mismatched currency, a missing ISIN, or a failed check rejects the **whole file** with the record number, because each of those can move money that never moved.
Securities are registered by **ISIN** in **Wealth → Instruments**. The ISIN is the identity; the name is editable display text, because one ISIN appears under several broker descriptions over the years. Set the account's **settlement IBAN** so deposits from your bank pair with the funding account: a broker export has no counterparty column, and without it those rows stay unpaired. They never become income either way — a broker record is excluded from spending and income analytics, from bulk reclassification, and from the AI entirely.
**Verify it yourself.** **Wealth** shows each account's cash balance, its positions as exact share counts, and named checks — row arithmetic, cash never negative, holdings never negative. Compare the cash balance and the positions against your broker's own screen. The figures come from the journal, not from the DuckDB index, so they do not depend on the cache that the same journal derives. A negative holding means the imported history is partial: a position was closed that was never opened.
Deliberately **not** included: market prices, market value, net worth over time, FIFO lot accounting, realised gains, `Vorabpauschale`, and currency conversion. A position's *invested* figure is cash in less cash out, not a cost basis.
## Deployment options ## Deployment options
| Option | Best fit | Included support | | Option | Best fit | Included support |
+9 -4
View File
@@ -203,12 +203,14 @@ func (s *Store) Rebuild(ctx context.Context, data domain.Dataset) error {
} }
} }
// Each bank fact produces a balanced asset/counterpart pair. Own-account // Each bank fact produces a balanced asset/counterpart pair. Own-account
// transfers use one clearing ledger; both sides cancel there when linked. // transfers cancel through one clearing ledger; broker facts cancel through
// another, where the residue left behind is exactly the cash an investment
// account has returned: distributions and interest received, less fees.
if _, err := tx.ExecContext(ctx, `INSERT INTO postings if _, err := tx.ExecContext(ctx, `INSERT INTO postings
SELECT id, 1, 'asset:' || account_id, account_id, '', currency, amount FROM transactions SELECT id, 1, 'asset:' || account_id, account_id, '', currency, amount FROM transactions
UNION ALL UNION ALL
SELECT id, 2, CASE WHEN kind = 'transfer' THEN 'clearing:transfers' ELSE 'category:' || category_id END, SELECT id, 2, CASE kind WHEN 'transfer' THEN 'clearing:transfers' WHEN 'investment' THEN 'clearing:investments' ELSE 'category:' || category_id END,
'', CASE WHEN kind = 'transfer' THEN '' ELSE category_id END, currency, -amount FROM transactions`); err != nil { '', CASE WHEN kind IN ('transfer', 'investment') THEN '' ELSE category_id END, currency, -amount FROM transactions`); err != nil {
return fmt.Errorf("derive postings: %w", err) return fmt.Errorf("derive postings: %w", err)
} }
if err := tx.Commit(); err != nil { if err := tx.Commit(); err != nil {
@@ -233,8 +235,11 @@ func (f Filter) validate() error {
// where uses EXISTS for many-to-many filters so a transaction carrying several // where uses EXISTS for many-to-many filters so a transaction carrying several
// selected tags, or several matching ancestors, can never multiply totals. // selected tags, or several matching ancestors, can never multiply totals.
// Transfers and broker facts are excluded: moving your own money between your
// own cash and your own positions is neither spending nor income, and a broker
// history is large enough to swamp everything else if it leaked in.
func (f Filter) where() (string, []any) { func (f Filter) where() (string, []any) {
clauses := []string{"t.kind <> 'transfer'"} clauses := []string{"t.kind NOT IN ('transfer', 'investment')"}
args := []any{} args := []any{}
add := func(clause string, value string) { add := func(clause string, value string) {
if value != "" { if value != "" {
+3 -3
View File
@@ -50,7 +50,7 @@ func sampleFacts(description, date string, amount domain.Money) domain.Facts {
func seed(t *testing.T, a *App, s State) State { func seed(t *testing.T, a *App, s State) State {
t.Helper() t.Helper()
a.mu.Lock() a.mu.Lock()
result, err := a.importFacts(context.Background(), s, []domain.Facts{sampleFacts("REWE", "2026-09-08", "-42.80"), sampleFacts("EDEKA", "2026-09-09", "-19.30")}) result, err := a.importFacts(context.Background(), s, []domain.Facts{sampleFacts("REWE", "2026-09-08", "-42.80"), sampleFacts("EDEKA", "2026-09-09", "-19.30")}, nil)
a.mu.Unlock() a.mu.Unlock()
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
@@ -73,7 +73,7 @@ func TestFailedClassificationStillImportsAndRetryIsIdempotent(t *testing.T) {
} }
before := domain.Clone(s.Data) before := domain.Clone(s.Data)
a.mu.Lock() a.mu.Lock()
again, err := a.importFacts(context.Background(), s, []domain.Facts{sampleFacts("REWE", "2026-09-08", "-42.80"), sampleFacts("EDEKA", "2026-09-09", "-19.30")}) again, err := a.importFacts(context.Background(), s, []domain.Facts{sampleFacts("REWE", "2026-09-08", "-42.80"), sampleFacts("EDEKA", "2026-09-09", "-19.30")}, nil)
a.mu.Unlock() a.mu.Unlock()
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
@@ -126,7 +126,7 @@ func TestPreviewCooldownProtectsLaterPreviewsAndImports(t *testing.T) {
} }
a.mu.Lock() a.mu.Lock()
result, err := a.importFacts(ctx, unchanged, []domain.Facts{sampleFacts("ALDI", "2026-09-10", "-12.34")}) result, err := a.importFacts(ctx, unchanged, []domain.Facts{sampleFacts("ALDI", "2026-09-10", "-12.34")}, nil)
a.mu.Unlock() a.mu.Unlock()
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
+70 -6
View File
@@ -35,12 +35,26 @@ func addProposal(d *domain.Dataset, p classification.Proposal) error {
} }
return nil return nil
} }
func (a *App) importFacts(ctx context.Context, s State, facts []domain.Facts) (ImportResult, error) { func (a *App) importFacts(ctx context.Context, s State, facts []domain.Facts, instruments []domain.Instrument) (ImportResult, error) {
// Instruments first: a broker fact references one, and the canonical
// dataset is validated as a whole, so a trade cannot be committed before
// the security it trades exists.
known := make(map[string]bool, len(s.Data.Instruments))
for _, v := range s.Data.Instruments {
known[v.ID] = true
}
registered := false
for _, v := range instruments {
if !known[v.ID] {
known[v.ID], registered = true, true
s.Data.Instruments = append(s.Data.Instruments, v)
}
}
added, err := banking.NormalizeAndDedupe(s.Data, facts) added, err := banking.NormalizeAndDedupe(s.Data, facts)
if err != nil { if err != nil {
return ImportResult{}, err return ImportResult{}, err
} }
if len(added) == 0 { if len(added) == 0 && !registered {
return ImportResult{State: s}, nil return ImportResult{State: s}, nil
} }
s.Data.Transactions = append(s.Data.Transactions, added...) s.Data.Transactions = append(s.Data.Transactions, added...)
@@ -55,7 +69,7 @@ func (a *App) importFacts(ctx context.Context, s State, facts []domain.Facts) (I
ids[t.Facts.ID] = true ids[t.Facts.ID] = true
} }
for i, t := range s.Data.Transactions { for i, t := range s.Data.Transactions {
if !ids[t.Facts.ID] || t.Enrichment.Kind == "transfer" { if !ids[t.Facts.ID] || t.Enrichment.Kind == "transfer" || t.Enrichment.Kind == domain.KindInvestment {
continue continue
} }
// With AI classification off for imports, no provider is contacted at // With AI classification off for imports, no provider is contacted at
@@ -105,8 +119,15 @@ type CSVImport struct {
New int `json:"new"` New int `json:"new"`
Duplicates int `json:"duplicates"` Duplicates int `json:"duplicates"`
Samples []domain.Facts `json:"samples"` Samples []domain.Facts `json:"samples"`
// Broker is present when the statement is a broker export. Its rows carry
// positions as well as cash, so they are read by a dedicated parser rather
// than by a column mapping, and the review needs to show what that parser
// decided: which securities it would register, which rows it skipped, and
// which figures it deliberately did not apply.
Broker *banking.ScalableImport `json:"broker,omitempty"`
facts []domain.Facts facts []domain.Facts
instruments []domain.Instrument
created time.Time created time.Time
} }
@@ -141,6 +162,28 @@ func (a *App) PrepareCSVImport(ctx context.Context, rev, accountID string, r io.
return CSVImport{}, err return CSVImport{}, err
} }
prepared := CSVImport{ID: domain.NewID("csvimport"), Revision: s.Revision, AccountID: account.ID, MappedBy: "preset", created: time.Now()} prepared := CSVImport{ID: domain.NewID("csvimport"), Revision: s.Revision, AccountID: account.ID, MappedBy: "preset", created: time.Now()}
// A broker export is recognized before any column mapping is attempted. Its
// rows are not interchangeable statement lines: the same amount column is
// cash on one row, a gross to be netted on another, and a position
// valuation that must not touch cash on a third, so a column mapping cannot
// describe it.
if header, broker := banking.DetectScalableCSV(file); broker {
read, e := banking.ParseScalableCSV(file, account, s.Data.Instruments)
if e != nil {
return CSVImport{}, e
}
added, e := banking.NormalizeAndDedupe(s.Data, read.Facts)
if e != nil {
return CSVImport{}, e
}
prepared.Source, prepared.SourceLabel = banking.SourceScalable, "Scalable Capital"
prepared.Mapping = banking.CSVMapping{HeaderRow: header, DateFormat: "yyyy-mm-dd", DecimalFormat: "comma"}
prepared.Columns = brokerColumns()
prepared.Records, prepared.New, prepared.Duplicates = len(read.Facts), len(added), len(read.Facts)-len(added)
prepared.Samples, prepared.facts, prepared.instruments = csvSamples(read.Facts), read.Facts, read.Instruments
prepared.Broker = &read
return a.retain(prepared)
}
mapping, source, label, recognized := banking.DetectCSVMapping(file) mapping, source, label, recognized := banking.DetectCSVMapping(file)
if !recognized { if !recognized {
sample, e := file.Sample() sample, e := file.Sample()
@@ -188,6 +231,12 @@ func (a *App) PrepareCSVImport(ctx context.Context, rev, accountID string, r io.
prepared.Columns = csvColumns(mapping, account) prepared.Columns = csvColumns(mapping, account)
prepared.Records, prepared.New, prepared.Duplicates = len(facts), len(added), len(facts)-len(added) prepared.Records, prepared.New, prepared.Duplicates = len(facts), len(added), len(facts)-len(added)
prepared.Samples, prepared.facts = csvSamples(facts), facts prepared.Samples, prepared.facts = csvSamples(facts), facts
return a.retain(prepared)
}
// retain holds a reviewed statement until it is confirmed or expires. Nothing
// is written to the journal here.
func (a *App) retain(prepared CSVImport) (CSVImport, error) {
a.mu.Lock() a.mu.Lock()
defer a.mu.Unlock() defer a.mu.Unlock()
for id, old := range a.csvImports { for id, old := range a.csvImports {
@@ -202,6 +251,21 @@ func (a *App) PrepareCSVImport(ctx context.Context, rev, accountID string, r io.
return prepared, nil return prepared, nil
} }
// brokerColumns describes what the broker parser decided, in the same
// reviewable shape as a column mapping. The dispatch is the part that can be
// wrong in a way that moves money, so it is the part shown.
func brokerColumns() []CSVColumn {
return []CSVColumn{
{Field: "Booking date", Column: "date, exactly as printed; the time column is local and crosses midnight, so it is ignored"},
{Field: "Imported rows", Column: `status "Executed" only; cancelled retries are all zeros and would import as phantom trades`},
{Field: "Cash movement", Column: "cash rows: amount, already net of tax; trades: amount fee tax; corporate actions and depot transfers: none"},
{Field: "Position change", Column: "shares, signed by type for buys and sells and exactly as printed for corporate actions and depot transfers"},
{Field: "Instrument", Column: "isin; the description only names it"},
{Field: "Reference", Column: "reference, which the broker reuses across every leg of one event"},
{Field: "Decimals", Column: "German: comma decimal, and a dot only groups thousands in exact three-digit runs"},
}
}
// ConfirmCSVImport imports exactly the facts that were previewed, provided the // ConfirmCSVImport imports exactly the facts that were previewed, provided the
// journal has not changed since. // journal has not changed since.
func (a *App) ConfirmCSVImport(ctx context.Context, id, rev string) (ImportResult, error) { func (a *App) ConfirmCSVImport(ctx context.Context, id, rev string) (ImportResult, error) {
@@ -221,7 +285,7 @@ func (a *App) ConfirmCSVImport(ctx context.Context, id, rev string) (ImportResul
if s.Revision != prepared.Revision { if s.Revision != prepared.Revision {
return ImportResult{}, errors.New("revision conflict: data changed after the preview; upload the statement again") return ImportResult{}, errors.New("revision conflict: data changed after the preview; upload the statement again")
} }
result, err := a.importFacts(ctx, s, prepared.facts) result, err := a.importFacts(ctx, s, prepared.facts, prepared.instruments)
if err != nil { if err != nil {
return ImportResult{}, err return ImportResult{}, err
} }
@@ -414,7 +478,7 @@ func (a *App) Backfill(ctx context.Context, rev, accountID string, historyMonths
} }
// Use normal import processing without changing sync cursors or saved consent // Use normal import processing without changing sync cursors or saved consent
// settings, including when the requested range adds no transactions. // settings, including when the requested range adds no transactions.
result, err := a.importFacts(ctx, s, facts) result, err := a.importFacts(ctx, s, facts, nil)
if err != nil { if err != nil {
return ImportResult{}, err return ImportResult{}, err
} }
@@ -779,7 +843,7 @@ func (a *App) Sync(ctx context.Context) (State, error) {
failures = append(failures, account.DisplayName+": "+meta.Error) failures = append(failures, account.DisplayName+": "+meta.Error)
continue continue
} }
result, e := a.importFacts(ctx, s, facts) result, e := a.importFacts(ctx, s, facts, nil)
if e != nil { if e != nil {
failures = append(failures, account.DisplayName+": "+e.Error()) failures = append(failures, account.DisplayName+": "+e.Error())
waiting = false waiting = false
+41
View File
@@ -67,6 +67,33 @@ func SaveAccount(d *domain.Dataset, v domain.Account) error {
d.Accounts = append(d.Accounts, v) d.Accounts = append(d.Accounts, v)
return nil return nil
} }
// SaveInstrument registers or renames a security. The ISIN is the identity the
// facts were imported under, so it cannot be changed: pointing an existing
// instrument at a different security would silently relabel every trade that
// references it.
func SaveInstrument(d *domain.Dataset, v domain.Instrument) error {
v.Name = strings.TrimSpace(v.Name)
v.ISIN = strings.ToUpper(strings.Join(strings.Fields(v.ISIN), ""))
v.Currency = strings.ToUpper(strings.TrimSpace(v.Currency))
if v.ID == "" {
if !domain.ValidISIN(v.ISIN) {
return errors.New("an instrument needs a valid ISIN")
}
v.ID = domain.InstrumentID(v.ISIN)
}
for i, x := range d.Instruments {
if x.ID == v.ID {
if x.ISIN != v.ISIN {
return errors.New("an instrument's ISIN is its identity; register the other security separately")
}
d.Instruments[i] = v
return nil
}
}
d.Instruments = append(d.Instruments, v)
return nil
}
func SaveCategory(d *domain.Dataset, v domain.Category) error { func SaveCategory(d *domain.Dataset, v domain.Category) error {
v.Name = strings.TrimSpace(v.Name) v.Name = strings.TrimSpace(v.Name)
if v.ID == "" { if v.ID == "" {
@@ -146,6 +173,20 @@ func Manage(d *domain.Dataset, entity, action, id, target string) error {
if n == len(d.Accounts) { if n == len(d.Accounts) {
return errors.New("unknown account") return errors.New("unknown account")
} }
case "instrument":
if action != "delete" {
return errors.New("instrument merging is not supported; an ISIN identifies exactly one security")
}
for _, t := range d.Transactions {
if t.Facts.Investment != nil && t.Facts.Investment.InstrumentID == id {
return errors.New("instrument is referenced by immutable financial records")
}
}
n := len(d.Instruments)
d.Instruments = slices.DeleteFunc(d.Instruments, func(v domain.Instrument) bool { return v.ID == id })
if n == len(d.Instruments) {
return errors.New("unknown instrument")
}
case "tag": case "tag":
if !slices.ContainsFunc(d.Tags, func(v domain.Tag) bool { return v.ID == id }) { if !slices.ContainsFunc(d.Tags, func(v domain.Tag) bool { return v.ID == id }) {
return errors.New("unknown tag") return errors.New("unknown tag")
+1 -1
View File
@@ -81,7 +81,7 @@ func (a *App) Preview(ctx context.Context, r PreviewRequest) (Preview, error) {
p := Preview{ID: domain.NewID("preview"), Revision: s.Revision, Changes: []Change{}, Errors: []ClassificationError{}, created: time.Now()} p := Preview{ID: domain.NewID("preview"), Revision: s.Revision, Changes: []Change{}, Errors: []ClassificationError{}, created: time.Now()}
baseMerchants := len(s.Data.Merchants) baseMerchants := len(s.Data.Merchants)
for _, t := range s.Data.Transactions { for _, t := range s.Data.Transactions {
if t.Facts.BookingDate < r.From || t.Facts.BookingDate > r.To || t.Enrichment.Kind == "transfer" { if t.Facts.BookingDate < r.From || t.Facts.BookingDate > r.To || t.Enrichment.Kind == "transfer" || t.Enrichment.Kind == domain.KindInvestment {
continue continue
} }
if err = ctx.Err(); err != nil { if err = ctx.Err(); err != nil {
+77
View File
@@ -0,0 +1,77 @@
package app
import (
"context"
"errors"
"finance-duck/internal/domain"
)
// LinkTransfer links a transaction to its own-account counterpart, or unlinks it
// when peerID is empty.
//
// Reciprocity is a validated invariant: each side must name the other, with
// opposite money, one currency and different accounts. So relinking has to
// rewrite the old pair and the new pair in a single commit — applied one side
// at a time, the dataset is invalid halfway through and the commit is refused.
func (a *App) LinkTransfer(ctx context.Context, rev, id, peerID string) (State, error) {
return a.Mutate(ctx, rev, func(d *domain.Dataset) error { return Link(d, id, peerID) })
}
// Link rewrites both sides of a transfer decision at once.
func Link(d *domain.Dataset, id, peerID string) error {
if id == "" {
return errors.New("select a transaction to link")
}
if id == peerID {
return errors.New("a transaction cannot be its own counterpart")
}
index := make(map[string]int, len(d.Transactions))
for i, t := range d.Transactions {
index[t.Facts.ID] = i
}
self, ok := index[id]
if !ok {
return errors.New("unknown transaction")
}
// Releasing a side also releases whatever it currently names, or the old
// counterpart is left pointing at a transaction that no longer points back.
release := func(i int) {
peer := d.Transactions[i].Enrichment.TransferPeerID
d.Transactions[i].Enrichment = unlinked(d.Transactions[i])
if j, found := index[peer]; found && j != i {
d.Transactions[j].Enrichment = unlinked(d.Transactions[j])
}
}
release(self)
if peerID == "" {
return nil
}
other, ok := index[peerID]
if !ok {
return errors.New("unknown counterpart transaction")
}
release(other)
for _, ends := range [][2]int{{self, other}, {other, self}} {
t := &d.Transactions[ends[0]]
t.Enrichment = domain.Enrichment{
Kind: "transfer",
TagIDs: t.Enrichment.TagIDs,
TransferPeerID: d.Transactions[ends[1]].Facts.ID,
Classification: domain.Provenance{Source: "manual"},
}
}
return nil
}
// unlinked is what a transaction becomes when it stops being a transfer: a
// broker fact returns to the investment ledger, anything else to the sign-based
// fallback. Either way the decision is recorded as manual, because the import
// matcher skips manual rows — otherwise unlinking a pair that is not really a
// transfer would be undone by the next import, every time.
func unlinked(t domain.Transaction) domain.Enrichment {
e := domain.Fallback(t.Facts)
e.TagIDs = append([]string{}, t.Enrichment.TagIDs...)
e.Classification = domain.Provenance{Source: "manual"}
return e
}
+259
View File
@@ -0,0 +1,259 @@
package app
import (
"context"
"fmt"
"slices"
"strings"
"finance-duck/internal/domain"
)
// Wealth is a reconciliation report, computed from the journal rather than from
// the DuckDB index: it exists to be checked against the figures a bank or
// broker shows on its own screen, so it must not depend on the cache that the
// same journal derives.
type Wealth struct {
Accounts []WealthAccount `json:"accounts"`
// Totals is cash summed per currency across every account.
Totals []WealthTotal `json:"totals"`
}
type WealthTotal struct {
Currency string `json:"currency"`
Cash domain.Money `json:"cash"`
}
// WealthAccount is one account's position as the journal records it.
type WealthAccount struct {
AccountID string `json:"account_id"`
DisplayName string `json:"display_name"`
Institution string `json:"institution"`
Currency string `json:"currency"`
Kind string `json:"kind"`
Active bool `json:"active"`
Records int `json:"records"`
FirstBooking string `json:"first_booking,omitempty"`
LastBooking string `json:"last_booking,omitempty"`
// Cash is every recorded movement summed. It equals the account's real
// balance only when the journal holds that account's complete history,
// which a broker export does and a date-windowed bank statement does not.
Cash domain.Money `json:"cash"`
Holdings []WealthHolding `json:"holdings"`
Checks []WealthCheck `json:"checks"`
}
// WealthHolding is one instrument's position in one account.
type WealthHolding struct {
InstrumentID string `json:"instrument_id"`
ISIN string `json:"isin"`
Name string `json:"name"`
Quantity domain.Quantity `json:"quantity"`
// Invested is cash paid in less cash taken out through trades. It is not a
// cost basis: a depot transfer moves a position with no cash at all, and a
// sale returns cash without identifying which lot it closed.
Invested domain.Money `json:"invested"`
// Received is cash this instrument paid out without moving the position:
// distributions, and the cash side of a corporate action.
Received domain.Money `json:"received"`
Records int `json:"records"`
}
// WealthCheck is one named verification with its evidence. Failed marks a
// disagreement inside the journal; the rest are notes that explain a figure
// before it is compared with a broker's screen.
type WealthCheck struct {
Name string `json:"name"`
Detail string `json:"detail"`
Failed bool `json:"failed"`
}
// Wealth reports every account's cash and positions with the checks that decide
// whether those figures can be trusted.
func (a *App) Wealth(ctx context.Context) (Wealth, error) {
s, err := a.Snapshot(ctx)
if err != nil {
return Wealth{}, err
}
return WealthOf(s.Data), nil
}
// WealthOf derives the report from a dataset. Money is summed in exact
// ten-thousandths; 64 bits hold hundreds of trillions, far beyond any number a
// journal of personal accounts can reach.
func WealthOf(data domain.Dataset) Wealth {
instruments := map[string]domain.Instrument{}
for _, v := range data.Instruments {
instruments[v.ID] = v
}
accounts := map[string]domain.Account{}
for _, v := range data.Accounts {
accounts[v.ID] = v
}
ordered := slices.Clone(data.Transactions)
slices.SortStableFunc(ordered, func(x, y domain.Transaction) int {
if c := strings.Compare(x.Facts.BookingDate, y.Facts.BookingDate); c != 0 {
return c
}
return strings.Compare(x.Facts.ID, y.Facts.ID)
})
type holdingState struct {
units, invested, received int64
records int
lowest int64
lowestDate string
}
type accountState struct {
cash, lowestCash int64
lowestCashDate string
records int
first, last string
holdings map[string]*holdingState
order []string
broken []string
unappliedFee, unappliedTax int64
unappliedRows int
unmatchedCash, unmatchedRows int64
}
states := map[string]*accountState{}
state := func(id string) *accountState {
if states[id] == nil {
states[id] = &accountState{holdings: map[string]*holdingState{}}
}
return states[id]
}
for _, t := range ordered {
f := t.Facts
account := accounts[f.AccountID]
st := state(f.AccountID)
st.records++
if st.first == "" {
st.first = f.BookingDate
}
st.last = f.BookingDate
minor, err := f.Amount.Minor()
if err != nil {
st.broken = append(st.broken, fmt.Sprintf("%s: unreadable amount %q", f.BookingDate, f.Amount))
continue
}
st.cash += minor
if st.cash < st.lowestCash {
st.lowestCash, st.lowestCashDate = st.cash, f.BookingDate
}
inv := f.Investment
if inv == nil {
continue
}
if err := domain.ValidateInvestment(f, account, instruments); err != nil {
st.broken = append(st.broken, fmt.Sprintf("%s %s: %v", f.BookingDate, f.ID, err))
}
if inv.CashOnly() {
fee, _ := inv.Fee.Minor()
tax, _ := inv.Tax.Minor()
if fee != 0 || tax != 0 {
st.unappliedRows++
st.unappliedFee += fee
st.unappliedTax += tax
}
if (inv.Event == domain.EventDeposit || inv.Event == domain.EventWithdrawal) && t.Enrichment.Kind != "transfer" {
st.unmatchedRows++
st.unmatchedCash += minor
}
}
if inv.InstrumentID == "" {
continue
}
held := st.holdings[inv.InstrumentID]
if held == nil {
held = &holdingState{}
st.holdings[inv.InstrumentID] = held
st.order = append(st.order, inv.InstrumentID)
}
held.records++
if inv.Settling() {
held.invested -= minor
} else {
held.received += minor
}
units, err := inv.Quantity.Units()
if inv.Quantity == "" {
units, err = 0, nil
}
if err != nil {
st.broken = append(st.broken, fmt.Sprintf("%s %s: unreadable quantity %q", f.BookingDate, f.ID, inv.Quantity))
continue
}
held.units += units
if held.units < held.lowest {
held.lowest, held.lowestDate = held.units, f.BookingDate
}
}
report := Wealth{Accounts: []WealthAccount{}, Totals: []WealthTotal{}}
totals := map[string]int64{}
currencies := []string{}
for _, account := range data.Accounts {
st := state(account.ID)
kind := account.Kind
if kind == "" {
kind = domain.AccountCash
}
entry := WealthAccount{
AccountID: account.ID, DisplayName: account.DisplayName, Institution: account.Institution,
Currency: account.Currency, Kind: kind, Active: account.Active,
Records: st.records, FirstBooking: st.first, LastBooking: st.last,
Cash: domain.FormatMoney(st.cash), Holdings: []WealthHolding{}, Checks: []WealthCheck{},
}
if _, seen := totals[account.Currency]; !seen {
currencies = append(currencies, account.Currency)
}
totals[account.Currency] += st.cash
for _, id := range st.order {
held := st.holdings[id]
instrument := instruments[id]
entry.Holdings = append(entry.Holdings, WealthHolding{
InstrumentID: id, ISIN: instrument.ISIN, Name: instrument.Name,
Quantity: domain.FormatQuantity(held.units), Invested: domain.FormatMoney(held.invested),
Received: domain.FormatMoney(held.received), Records: held.records,
})
}
slices.SortFunc(entry.Holdings, func(x, y WealthHolding) int { return strings.Compare(x.Name, y.Name) })
check := func(name, detail string, failed bool) {
entry.Checks = append(entry.Checks, WealthCheck{Name: name, Detail: detail, Failed: failed})
}
if len(st.broken) > 0 {
check("Row arithmetic", fmt.Sprintf("%d record(s) disagree with their own figures: %s", len(st.broken), strings.Join(st.broken, "; ")), true)
} else {
check("Row arithmetic", "every record agrees with its own gross, fee, tax, quantity and price", false)
}
if st.lowestCash < 0 {
check("Cash never negative", fmt.Sprintf("balance reached %s on %s, so the history is incomplete or a movement is misread", domain.FormatMoney(st.lowestCash), st.lowestCashDate), true)
} else {
check("Cash never negative", "the running balance stays at or above zero throughout", false)
}
negative := []string{}
for _, id := range st.order {
if held := st.holdings[id]; held.lowest < 0 {
negative = append(negative, fmt.Sprintf("%s reached %s on %s", instruments[id].ISIN, domain.FormatQuantity(held.lowest), held.lowestDate))
}
}
if len(negative) > 0 {
check("Holdings never negative", fmt.Sprintf("%s — a sale before its purchase means the export is partial or a sign is wrong", strings.Join(negative, "; ")), true)
} else if len(st.order) > 0 {
check("Holdings never negative", "every position stays at or above zero throughout", false)
}
if st.unappliedRows > 0 {
check("Fee and tax recorded, not applied", fmt.Sprintf("%d cash record(s) carry fee %s and tax %s. A broker cash amount is already net of them, so they are recorded and not subtracted again. If the balance above is wrong by one of these figures, this is why", st.unappliedRows, domain.FormatMoney(st.unappliedFee), domain.FormatMoney(st.unappliedTax)), false)
}
if st.unmatchedRows > 0 {
check("Deposits and withdrawals unmatched", fmt.Sprintf("%d transfer(s) totalling %s have no counterpart in another account. They stay out of spending either way; set this account's IBAN and settlement IBAN to pair them", st.unmatchedRows, domain.FormatMoney(st.unmatchedCash)), false)
}
report.Accounts = append(report.Accounts, entry)
}
for _, currency := range currencies {
report.Totals = append(report.Totals, WealthTotal{Currency: currency, Cash: domain.FormatMoney(totals[currency])})
}
return report
}
+233
View File
@@ -0,0 +1,233 @@
package app
import (
"context"
"strings"
"testing"
"finance-duck/internal/analytics"
"finance-duck/internal/domain"
)
const brokerHeader = "date;time;status;reference;description;assetType;type;isin;shares;price;amount;fee;tax;currency\n"
// A broker history end to end: money in, three purchases averaging down, the
// distribution that came with a knock-out, the position row that closed it, and
// a reinvested fraction of a share. Cash and holdings are what the user
// compares against the broker's own screen, so they are asserted exactly.
var brokerRows = []string{
`2025-05-06;02:00:00;Executed;DEP1;Scalable Capital Broker Einzahlung;Cash;Deposit;;;;800,00;;;EUR`,
`2025-05-07;09:02:13;Cancelled;SCAL9RdFWnYpi5T;Rheinmetall Long 10x Faktor-Zertifikat HVB;Security;Buy;DE000UG4V0Z7;0;0,00;0,00;0,00;0,00;EUR`,
`2025-05-07;09:02:29;Executed;SCALTThBbxx6z5Z;Rheinmetall Long 10x Faktor-Zertifikat HVB;Security;Buy;DE000UG4V0Z7;14;26,45;-370,30;0,00;0,00;EUR`,
`2025-09-17;15:14:49;Executed;SCALwBaNVPpjf8p;Rheinmetall Long 10x Factor HVB;Security;Buy;DE000UG4V0Z7;203;1,23;-249,69;0,99;0,00;EUR`,
`2025-09-18;13:38:08;Executed;SCALSVuyHibZT4w;Rheinmetall Long 10x Factor HVB;Security;Buy;DE000UG4V0Z7;6;1,10;-6,60;0,99;0,00;EUR`,
`2025-10-28;01:00:00;Executed;48231_rrCjP4EcbpefpNiVQeD495;Rheinmetall Long 10x Factor HVB;Cash;Distribution;DE000UG4V0Z7;;;32,64;;-1,42;EUR`,
`2025-10-28;01:00:00;Executed;48231_rrCjP4EcbpefpNiVQeD495;Rheinmetall Long 10x Factor HVB;Security;Corporate action;DE000UG4V0Z7;-223;0,14;-31,22;;;EUR`,
`2026-01-20;01:00:00;Executed;429776_rrCjP4EcbpefpNiVQeD495;Taiwan Semiconductor Manufact. ADR;Security;Reinvestment_Distribution;US8740391003;0,076494;388,00;-29,679672;0,00;0,00;EUR`,
}
func brokerApp(t *testing.T, rows []string) (*App, State, string) {
t.Helper()
a, s := testApp(t)
s, err := a.Mutate(context.Background(), s.Revision, func(d *domain.Dataset) error {
return SaveAccount(d, domain.Account{
ID: "broker", DisplayName: "Scalable", Institution: "Scalable Capital",
Currency: "EUR", Kind: domain.AccountInvestment, Active: true,
})
})
if err != nil {
t.Fatal(err)
}
statement := brokerHeader + strings.Join(rows, "\n") + "\n"
prepared, err := a.PrepareCSVImport(context.Background(), s.Revision, "broker", strings.NewReader(statement))
if err != nil {
t.Fatal(err)
}
result, err := a.ConfirmCSVImport(context.Background(), prepared.ID, prepared.Revision)
if err != nil {
t.Fatal(err)
}
return a, result.State, prepared.ID
}
func TestBrokerImportReconcilesCashAndHoldings(t *testing.T) {
a, s, _ := brokerApp(t, brokerRows)
// Seven executed rows; the cancelled retry is all zeros and must not
// import as a phantom trade.
broker := WealthOf(s.Data).Accounts[1]
if broker.Records != 7 {
t.Fatalf("imported %d records, want 7", broker.Records)
}
// 800.00 370.30 250.68 7.59 + 32.64 29.6797
if broker.Cash != "174.3903" {
t.Errorf("cash %s, want 174.3903", broker.Cash)
}
if broker.FirstBooking != "2025-05-06" || broker.LastBooking != "2026-01-20" {
t.Errorf("history spans %s..%s", broker.FirstBooking, broker.LastBooking)
}
holdings := map[string]domain.Quantity{}
for _, h := range broker.Holdings {
holdings[h.ISIN] = h.Quantity
}
// 14 + 203 + 6 223, the knock-out closing the position exactly.
if holdings["DE000UG4V0Z7"] != "0" {
t.Errorf("certificate holds %s, want 0", holdings["DE000UG4V0Z7"])
}
if holdings["US8740391003"] != "0.076494" {
t.Errorf("reinvested fraction holds %s, want 0.076494", holdings["US8740391003"])
}
for _, check := range broker.Checks {
if check.Failed {
t.Errorf("check %q failed: %s", check.Name, check.Detail)
}
}
// The distribution's refunded tax is recorded and not applied, because the
// broker's cash amount already includes it.
note := false
for _, check := range broker.Checks {
if strings.HasPrefix(check.Name, "Fee and tax") {
note = true
if !strings.Contains(check.Detail, "-1.42") {
t.Errorf("unapplied tax not reported: %s", check.Detail)
}
}
}
if !note {
t.Error("no note about the tax that was recorded but not applied")
}
// Instruments are registered from the export, and the latest description
// names one whose text changed between May and October.
names := map[string]string{}
for _, v := range s.Data.Instruments {
names[v.ISIN] = v.Name
}
if names["DE000UG4V0Z7"] != "Rheinmetall Long 10x Factor HVB" {
t.Errorf("certificate named %q", names["DE000UG4V0Z7"])
}
// The broker history must not reach spending analytics: a closed position
// and a reinvested dividend are neither income nor expenditure.
dashboard, err := a.Dashboard(context.Background(), analytics.Filter{})
if err != nil {
t.Fatal(err)
}
for _, total := range dashboard.Totals {
if total.Expenses != "0.0000" || total.Income != "0.0000" {
t.Errorf("broker rows leaked into spending: %+v", total)
}
}
// Re-importing the same export changes nothing, including the two legs
// that share one reference.
again, err := a.PrepareCSVImport(context.Background(), s.Revision, "broker", strings.NewReader(brokerHeader+strings.Join(brokerRows, "\n")+"\n"))
if err != nil {
t.Fatal(err)
}
if again.New != 0 || again.Duplicates != 7 {
t.Fatalf("re-import proposed %d new and %d duplicate records", again.New, again.Duplicates)
}
}
// A partial export sells or closes a position that was never opened in it. The
// journal accepts the facts, because they are facts, and the report says so.
func TestPartialBrokerExportReportsNegativeHolding(t *testing.T) {
partial := []string{brokerRows[0], brokerRows[5], brokerRows[6]}
_, s, _ := brokerApp(t, partial)
broker := WealthOf(s.Data).Accounts[1]
failed := map[string]string{}
for _, check := range broker.Checks {
if check.Failed {
failed[check.Name] = check.Detail
}
}
detail, found := failed["Holdings never negative"]
if !found {
t.Fatalf("a position closed without ever being opened passed every check: %+v", broker.Checks)
}
if !strings.Contains(detail, "DE000UG4V0Z7") || !strings.Contains(detail, "2025-10-28") {
t.Errorf("negative holding not located: %s", detail)
}
if len(failed) != 1 {
t.Errorf("unexpected additional failures: %+v", failed)
}
}
// A broker fact never reaches the sign-based fallback. This is the single rule
// that stops an unmatched deposit from being counted as income and a broker fee
// from being counted as household spending.
func TestBrokerFactsNeverClassifyBySign(t *testing.T) {
_, s, _ := brokerApp(t, brokerRows)
for _, tx := range s.Data.Transactions {
if tx.Facts.Investment == nil {
continue
}
if tx.Enrichment.Kind != domain.KindInvestment {
t.Fatalf("%s classified as %q", tx.Facts.ID, tx.Enrichment.Kind)
}
if tx.Enrichment.CategoryID != "" || tx.Enrichment.MerchantID != "" {
t.Fatalf("%s acquired a category or merchant: %+v", tx.Facts.ID, tx.Enrichment)
}
}
}
// Linking is one commit over both pairs, because reciprocity is validated: a
// half-applied relink is an invalid dataset.
func TestManualTransferLinkRewritesBothPairsAtOnce(t *testing.T) {
a, s, _ := brokerApp(t, brokerRows)
s, err := a.Mutate(context.Background(), s.Revision, func(d *domain.Dataset) error {
facts := domain.Facts{
Source: "test", AccountID: "n26", BookingDate: "2025-05-06", Amount: "-800.00",
Currency: "EUR", RawDescription: "Uberweisung Scalable", Fingerprint: "manual_fixture", ID: "tx_bank_out",
}
d.Transactions = append(d.Transactions, domain.Transaction{Facts: facts, Enrichment: domain.Fallback(facts)})
return nil
})
if err != nil {
t.Fatal(err)
}
deposit := ""
for _, tx := range s.Data.Transactions {
if tx.Facts.Investment != nil && tx.Facts.Investment.Event == domain.EventDeposit {
deposit = tx.Facts.ID
}
}
if deposit == "" {
t.Fatal("no broker deposit to link")
}
s, err = a.LinkTransfer(context.Background(), s.Revision, "tx_bank_out", deposit)
if err != nil {
t.Fatal(err)
}
linked := map[string]domain.Enrichment{}
for _, tx := range s.Data.Transactions {
linked[tx.Facts.ID] = tx.Enrichment
}
if linked["tx_bank_out"].TransferPeerID != deposit || linked[deposit].TransferPeerID != "tx_bank_out" {
t.Fatalf("link is not reciprocal: %+v", linked)
}
if linked["tx_bank_out"].Kind != "transfer" || linked[deposit].Kind != "transfer" {
t.Fatalf("linked pair is not a transfer: %+v", linked)
}
// Unlinking returns the broker leg to the investment ledger and the bank
// leg to the fallback, both stamped manual so the next import's matcher
// leaves the decision alone.
s, err = a.LinkTransfer(context.Background(), s.Revision, "tx_bank_out", "")
if err != nil {
t.Fatal(err)
}
for _, tx := range s.Data.Transactions {
switch tx.Facts.ID {
case "tx_bank_out":
if tx.Enrichment.Kind != "expense" || tx.Enrichment.Classification.Source != "manual" {
t.Errorf("bank leg after unlink: %+v", tx.Enrichment)
}
case deposit:
if tx.Enrichment.Kind != domain.KindInvestment || tx.Enrichment.Classification.Source != "manual" {
t.Errorf("broker leg after unlink: %+v", tx.Enrichment)
}
}
}
}
+32 -6
View File
@@ -671,13 +671,11 @@ func parseMappedCSVDecimal(value, format string) (domain.Money, error) {
case "dot-or-comma": case "dot-or-comma":
return parseCSVAmount(value) return parseCSVAmount(value)
case "comma": case "comma":
// A dot can only be grouping here, and only in exact thousands groups. plain, err := germanDecimal(value)
if !strings.Contains(value, ",") && strings.Contains(value, ".") { if err != nil {
if digits, ok := ungroup(value, "."); ok { return "", err
value = digits
} }
} return domain.ParseMoney(plain)
return parseCSVAmount(value)
case "dot": case "dot":
value = strings.TrimPrefix(value, "+") value = strings.TrimPrefix(value, "+")
if strings.Contains(value, ",") { if strings.Contains(value, ",") {
@@ -693,6 +691,34 @@ func parseMappedCSVDecimal(value, format string) (domain.Money, error) {
} }
} }
// germanDecimal rewrites a German-formatted number as a plain decimal string
// without parsing it, so a caller can choose its own precision. A dot is only
// grouping when every group is exactly three digits: "1.014" is 1014 while
// "1.14" stays 1.14. Broker exports carry both shapes in one share column.
func germanDecimal(value string) (string, error) {
value = strings.TrimPrefix(strings.TrimSpace(value), "+")
if strings.Contains(value, ",") {
if strings.Count(value, ",") != 1 {
return "", errors.New("invalid decimal separator")
}
whole, fraction, _ := strings.Cut(value, ",")
if strings.Contains(whole, ".") {
digits, ok := ungroup(whole, ".")
if !ok {
return "", errors.New("invalid grouping")
}
whole = digits
}
return whole + "." + fraction, nil
}
if strings.Contains(value, ".") {
if digits, ok := ungroup(value, "."); ok {
return digits, nil
}
}
return value, nil
}
// ungroup removes thousands separators, and only when every group is exactly // ungroup removes thousands separators, and only when every group is exactly
// three digits: "1.234" is 1234, while "1.23" stays a decimal value. // three digits: "1.234" is 1234, while "1.23" stays a decimal value.
func ungroup(value, separator string) (string, bool) { func ungroup(value, separator string) (string, bool) {
+82 -23
View File
@@ -25,13 +25,31 @@ func identity(f domain.Facts) string {
if strings.HasPrefix(string(f.Amount), "-") { if strings.HasPrefix(string(f.Amount), "-") {
direction = "debit" direction = "debit"
} }
return digest(f.AccountID, f.Source, f.ExternalID, direction) return digest(f.AccountID, f.Source, f.ExternalID, direction, leg(f))
}
// leg distinguishes the records of one broker event. A broker reuses a single
// reference across every leg: the cash side of a corporate action and its
// position side arrive with the same reference byte for byte, and a position
// leg's zero amount does not even differ in direction. The event and its
// instrument separate them without making money part of an identity, so a
// corrected upstream figure is still reported rather than imported twice.
func leg(f domain.Facts) string {
if f.Investment == nil {
return ""
}
return f.Investment.Event + "\x00" + f.Investment.InstrumentID
} }
func fingerprint(f domain.Facts) string { func fingerprint(f domain.Facts) string {
return digest(f.AccountID, f.BookingDate, f.ValueDate, f.Amount.String(), f.Currency, strings.Join(strings.Fields(f.RawDescription), " "), strings.ToLower(strings.Join(strings.Fields(f.Counterparty), " ")), f.CounterpartyIBAN) inv := domain.Investment{}
if f.Investment != nil {
inv = *f.Investment
}
return digest(f.AccountID, f.BookingDate, f.ValueDate, f.Amount.String(), f.Currency, strings.Join(strings.Fields(f.RawDescription), " "), strings.ToLower(strings.Join(strings.Fields(f.Counterparty), " ")), f.CounterpartyIBAN,
inv.Event, inv.InstrumentID, string(inv.Quantity), string(inv.Price), string(inv.Gross), string(inv.Fee), string(inv.Tax))
} }
func looseFingerprint(f domain.Facts) string { func looseFingerprint(f domain.Facts) string {
return digest(f.AccountID, f.BookingDate, f.Amount.String(), f.Currency) return digest(f.AccountID, f.BookingDate, f.Amount.String(), f.Currency, leg(f))
} }
func sameBookedMoney(a, b domain.Facts) bool { func sameBookedMoney(a, b domain.Facts) bool {
return a.AccountID == b.AccountID && a.BookingDate == b.BookingDate && a.Amount == b.Amount && a.Currency == b.Currency return a.AccountID == b.AccountID && a.BookingDate == b.BookingDate && a.Amount == b.Amount && a.Currency == b.Currency
@@ -46,6 +64,8 @@ func sourceLabel(source string) string {
return "ING CSV" return "ING CSV"
case "kontist_csv": case "kontist_csv":
return "Kontist CSV" return "Kontist CSV"
case SourceScalable:
return "Scalable CSV"
case "csv": case "csv":
return "mapped CSV" return "mapped CSV"
default: default:
@@ -310,10 +330,21 @@ func normalizeFacts(f domain.Facts, accounts map[string]bool) (domain.Facts, err
return f, nil return f, nil
} }
// MatchTransfers links only mutually unique candidates, with reciprocal own // MatchTransfers links own-account pairs with reciprocal own IBANs, inverse
// IBANs, inverse exact money in one currency, and booking dates within 3 calendar // exact money in one currency, and booking dates within 3 calendar days.
// days. Existing manual links are retained. Ambiguous equal payments stay ordinary //
// transactions: iteration order must never decide which transfer gets linked. // Equal competing payments are paired by nearest booking date rather than left
// alone. Every connected component of the candidate graph is a complete
// bipartite graph between two fixed accounts at one amount and one currency:
// an edge needs exactly inverse money, and a record's own counterparty IBAN
// names exactly one other account. So every perfect matching produces the same
// accounts, amounts, kinds and postings, and the only thing a choice decides is
// which row displays as which one's counterpart. Refusing to choose is the
// expensive option: both legs then fall through to the sign-based fallback and
// show up as spending and income that never happened.
//
// Ordering is by date gap, then by transaction ID, so iteration order cannot
// decide anything. Existing links and hand-made decisions are never revisited.
func MatchTransfers(data *domain.Dataset) { func MatchTransfers(data *domain.Dataset) {
if data == nil { if data == nil {
return return
@@ -336,10 +367,28 @@ func MatchTransfers(data *domain.Dataset) {
byAccount[id] = iban byAccount[id] = iban
} }
} }
candidates := make([][]int, len(data.Transactions)) matchable := func(t domain.Transaction) bool {
if t.Enrichment.Kind == "transfer" || t.Enrichment.TransferPeerID != "" {
return false
}
// A hand-made decision outlives the next import. Without this, an
// operator who unlinks a pair that is not really a transfer watches the
// matcher relink it on the following import, forever.
if t.Enrichment.Classification.Source == "manual" {
return false
}
// Only a broker cash movement can be a transfer leg; a trade's cash
// side settles against a position, not against another account.
return t.Facts.Investment == nil || t.Facts.Investment.CashOnly()
}
type candidate struct {
i, j int
gap time.Duration
}
candidates := []candidate{}
for i := range data.Transactions { for i := range data.Transactions {
a := data.Transactions[i] a := data.Transactions[i]
if a.Enrichment.Kind == "transfer" || a.Enrichment.TransferPeerID != "" { if !matchable(a) {
continue continue
} }
ai := byAccount[a.Facts.AccountID] ai := byAccount[a.Facts.AccountID]
@@ -357,7 +406,7 @@ func MatchTransfers(data *domain.Dataset) {
} }
for j := i + 1; j < len(data.Transactions); j++ { for j := i + 1; j < len(data.Transactions); j++ {
b := data.Transactions[j] b := data.Transactions[j]
if b.Enrichment.Kind == "transfer" || b.Enrichment.TransferPeerID != "" || b.Facts.AccountID != own[target] || normalizeIBAN(b.Facts.CounterpartyIBAN) != ai || a.Facts.Currency != b.Facts.Currency { if !matchable(b) || b.Facts.AccountID != own[target] || normalizeIBAN(b.Facts.CounterpartyIBAN) != ai || a.Facts.Currency != b.Facts.Currency {
continue continue
} }
bm, err := b.Facts.Amount.Minor() bm, err := b.Facts.Amount.Minor()
@@ -368,29 +417,39 @@ func MatchTransfers(data *domain.Dataset) {
if err != nil { if err != nil {
continue continue
} }
delta := ad.Sub(bd) gap := ad.Sub(bd)
if delta < -72*time.Hour || delta > 72*time.Hour { if gap < 0 {
gap = -gap
}
if gap > 72*time.Hour {
continue continue
} }
candidates[i] = append(candidates[i], j) candidates = append(candidates, candidate{i: i, j: j, gap: gap})
candidates[j] = append(candidates[j], i)
} }
} }
for i, matches := range candidates { sort.Slice(candidates, func(x, y int) bool {
if len(matches) != 1 { if candidates[x].gap != candidates[y].gap {
return candidates[x].gap < candidates[y].gap
}
left, right := data.Transactions[candidates[x].i].Facts.ID, data.Transactions[candidates[y].i].Facts.ID
if left != right {
return left < right
}
return data.Transactions[candidates[x].j].Facts.ID < data.Transactions[candidates[y].j].Facts.ID
})
linked := make([]bool, len(data.Transactions))
for _, c := range candidates {
if linked[c.i] || linked[c.j] {
continue continue
} }
j := matches[0] linked[c.i], linked[c.j] = true, true
if j <= i || len(candidates[j]) != 1 { for _, ends := range [][2]int{{c.i, c.j}, {c.j, c.i}} {
continue t := &data.Transactions[ends[0]]
}
for _, pair := range [][2]int{{i, j}, {j, i}} {
t := &data.Transactions[pair[0]]
tags := t.Enrichment.TagIDs tags := t.Enrichment.TagIDs
if tags == nil { if tags == nil {
tags = []string{} tags = []string{}
} }
t.Enrichment = domain.Enrichment{Kind: "transfer", TagIDs: tags, TransferPeerID: data.Transactions[pair[1]].Facts.ID, Classification: domain.Provenance{Source: "transfer_match"}} t.Enrichment = domain.Enrichment{Kind: "transfer", TagIDs: tags, TransferPeerID: data.Transactions[ends[1]].Facts.ID, Classification: domain.Provenance{Source: "transfer_match"}}
} }
} }
} }
+54 -6
View File
@@ -249,11 +249,6 @@ func TestTransfersRequireUniqueReciprocalOwnBankEvidence(t *testing.T) {
t.Fatal(err) t.Fatal(err)
} }
for _, change := range []func(*domain.Dataset){ for _, change := range []func(*domain.Dataset){
func(d *domain.Dataset) {
copy := d.Transactions[1]
copy.Facts.ID = "tx_c"
d.Transactions = append(d.Transactions, copy)
},
func(d *domain.Dataset) { d.Transactions[1].Facts.CounterpartyIBAN = "" }, func(d *domain.Dataset) { d.Transactions[1].Facts.CounterpartyIBAN = "" },
func(d *domain.Dataset) { d.Transactions[1].Facts.Currency = "USD" }, func(d *domain.Dataset) { d.Transactions[1].Facts.Currency = "USD" },
func(d *domain.Dataset) { d.Transactions[1].Facts.Amount = "9.99" }, func(d *domain.Dataset) { d.Transactions[1].Facts.Amount = "9.99" },
@@ -267,11 +262,64 @@ func TestTransfersRequireUniqueReciprocalOwnBankEvidence(t *testing.T) {
before := domain.Clone(d) before := domain.Clone(d)
MatchTransfers(&d) MatchTransfers(&d)
if !reflect.DeepEqual(d, before) { if !reflect.DeepEqual(d, before) {
t.Fatalf("ambiguous or unsupported transfer evidence linked: %+v", d.Transactions) t.Fatalf("unsupported transfer evidence linked: %+v", d.Transactions)
} }
} }
} }
// Two equal top-ups in one week give every leg two candidates. Refusing to
// pair them is what turned both legs into spending and income that never
// happened, so the pairing must happen, must follow the nearest booking date,
// and must not depend on the order the records arrive in.
func TestEqualCompetingTransfersPairByNearestDate(t *testing.T) {
build := func(reverse bool) domain.Dataset {
d := fixtureDataset()
leg := func(id, account, amount, date, peerIBAN string) domain.Transaction {
f := fixtureFacts()
f.ID, f.AccountID, f.Amount, f.BookingDate, f.CounterpartyIBAN = id, account, domain.Money(amount), date, peerIBAN
return domain.Transaction{Facts: f, Enrichment: domain.Fallback(f)}
}
a, b := d.Accounts[0].IBAN, d.Accounts[1].IBAN
d.Transactions = []domain.Transaction{
leg("tx_out_mon", "account_a", "-800.00", "2026-09-05", b),
leg("tx_out_wed", "account_a", "-800.00", "2026-09-07", b),
leg("tx_in_tue", "account_b", "800.00", "2026-09-06", a),
leg("tx_in_thu", "account_b", "800.00", "2026-09-08", a),
}
if reverse {
for i, j := 0, len(d.Transactions)-1; i < j; i, j = i+1, j-1 {
d.Transactions[i], d.Transactions[j] = d.Transactions[j], d.Transactions[i]
}
}
return d
}
want := map[string]string{"tx_out_mon": "tx_in_tue", "tx_in_tue": "tx_out_mon", "tx_out_wed": "tx_in_thu", "tx_in_thu": "tx_out_wed"}
for _, reverse := range []bool{false, true} {
d := build(reverse)
MatchTransfers(&d)
for _, tx := range d.Transactions {
if tx.Enrichment.Kind != "transfer" || tx.Enrichment.TransferPeerID != want[tx.Facts.ID] {
t.Fatalf("reverse=%v: %s linked to %q as %q, want %q as transfer", reverse, tx.Facts.ID, tx.Enrichment.TransferPeerID, tx.Enrichment.Kind, want[tx.Facts.ID])
}
}
if err := domain.Validate(d); err != nil {
t.Fatal(err)
}
}
}
// A hand-made decision must outlive the next import, or unlinking a pair that
// is not really a transfer is undone the moment anything is imported again.
func TestManualClassificationSurvivesMatching(t *testing.T) {
d := transferDataset()
d.Transactions[0].Enrichment.Classification.Source = "manual"
before := domain.Clone(d)
MatchTransfers(&d)
if !reflect.DeepEqual(d, before) {
t.Fatalf("matcher overrode a manual decision: %+v", d.Transactions)
}
}
func TestMixedReferencedAndAnonymousMultiplicity(t *testing.T) { func TestMixedReferencedAndAnonymousMultiplicity(t *testing.T) {
d := fixtureDataset() d := fixtureDataset()
anonymous := fixtureFacts() anonymous := fixtureFacts()
+396
View File
@@ -0,0 +1,396 @@
package banking
import (
"errors"
"fmt"
"strings"
"finance-duck/internal/domain"
)
// SourceScalable identifies facts imported from a Scalable Capital broker
// export.
const SourceScalable = "scalable_csv"
// scalableColumns are the exact normalized headers of a Scalable Capital
// transaction export. The layout is matched in full rather than column by
// column: a row's meaning depends on the combination of status, assetType and
// type, so a partial match would be a different file wearing the same names.
var scalableColumns = []string{
"date", "time", "status", "reference", "description",
"assettype", "type", "isin", "shares", "price", "amount", "fee", "tax", "currency",
}
// scalableEvents maps the export's complete type vocabulary to journal events.
// The set is closed on purpose: two of the ten types move a position without
// moving money, so an unrecognized type cannot be defaulted either way without
// risking a silent balance error. Keys are lowercased with collapsed spaces.
var scalableEvents = map[string]string{
"deposit": domain.EventDeposit,
"withdrawal": domain.EventWithdrawal,
"fee": domain.EventFee,
"interest": domain.EventInterest,
"distribution": domain.EventDistribution,
"buy": domain.EventBuy,
"sell": domain.EventSell,
"reinvestment_distribution": domain.EventReinvest,
"corporate action": domain.EventCorporateAction,
"security transfer": domain.EventPositionTransfer,
}
// ScalableNote records a figure the export carried that the import deliberately
// did not apply, so it can be reviewed before confirming and recognized later
// if a balance disagrees.
type ScalableNote struct {
Record int `json:"record"`
Date string `json:"date"`
Description string `json:"description"`
Fee domain.Money `json:"fee,omitempty"`
Tax domain.Money `json:"tax,omitempty"`
}
// ScalableImport is a read broker export awaiting review.
type ScalableImport struct {
Facts []domain.Facts `json:"-"`
// Instruments are securities the export named that the registry does not
// hold yet. An import never renames an existing instrument: the name is
// editable display text, and the export's own description for one ISIN
// changes over time.
Instruments []domain.Instrument `json:"instruments"`
// Cancelled counts rows the broker did not execute. Their money and share
// columns are all zeros, so they satisfy every arithmetic check and would
// otherwise import as phantom trades.
Cancelled int `json:"cancelled"`
// Rounded counts rows whose money carried more than four decimal places,
// and Rounding is the exact total adjustment that rounding applied.
Rounded int `json:"rounded"`
Rounding domain.Quantity `json:"rounding"`
// Unapplied lists cash rows carrying a fee or tax. A broker cash amount is
// already net of them, so subtracting them again would double-count; they
// are recorded on the fact and reported here.
Unapplied []ScalableNote `json:"unapplied"`
}
// DetectScalableCSV reports whether a document is a Scalable Capital export and
// which 1-based record holds its header.
func DetectScalableCSV(f CSVFile) (header int, ok bool) {
for i, row := range f.rows {
if i >= maxCSVPreambleRows {
break
}
columns, usable := csvColumnIndex(row)
if !usable || len(columns) != len(scalableColumns) {
continue
}
matched := true
for _, name := range scalableColumns {
if _, exists := columns[name]; !exists {
matched = false
break
}
}
if matched {
return i + 1, true
}
}
return 0, false
}
// ParseScalableCSV converts a broker export into bank facts carrying position
// legs.
//
// The amount column means a different thing per row class, and reading it
// wrongly moves money that never moved:
//
// - a cash row's amount is the money that actually settled, already net of
// the tax the broker withheld or refunded, so its tax is recorded and not
// applied;
// - a buy, sell or reinvestment quotes gross shares times price and settles
// gross minus fee minus tax;
// - a corporate action or depot transfer quotes a position valuation and
// settles no cash at all.
//
// The share column is signed only for those last two types; buys and sells are
// unsigned and take their direction from the type. Both conventions are
// resolved here, once.
//
// The booking date is the date column exactly as printed. Batch rows are
// stamped midnight UTC rendered in local time, so the time column crosses
// midnight for part of the year and reading date and time together would move
// those rows to the previous day.
//
// A single unrecognized status, type or assetType, or one failed arithmetic
// check, rejects the whole file. Every one of those cases can move money, and a
// partially imported broker history cannot be told from a truncated export
// afterwards.
func ParseScalableCSV(f CSVFile, account domain.Account, registry []domain.Instrument) (ScalableImport, error) {
// Empty rather than nil: these are arrays in the reviewed JSON, and a null
// where a caller expects a list is a bug waiting on a different machine.
result := ScalableImport{Instruments: []domain.Instrument{}, Unapplied: []ScalableNote{}}
if account.ID == "" {
return result, errors.New("broker import requires a selected account")
}
if !account.Investing() {
return result, fmt.Errorf("account %q must be an investment account to hold a broker export", account.DisplayName)
}
header, ok := DetectScalableCSV(f)
if !ok {
return result, errors.New("not a Scalable Capital export")
}
headers := f.rows[header-1]
index := map[string]int{}
for i, raw := range headers {
index[headerName(raw)] = i
}
cell := func(row []string, name string) string { return strings.TrimSpace(row[index[name]]) }
instruments := map[string]domain.Instrument{}
byISIN := map[string]domain.Instrument{}
for _, v := range registry {
instruments[v.ID] = v
byISIN[v.ISIN] = v
}
created := map[string]int{}
named := map[string]string{}
drift := int64(0)
for offset, row := range f.rows[header:] {
record := header + offset + 1
if blankCSVRow(row) {
continue
}
if len(row) != len(headers) {
return result, fmt.Errorf("broker record %d has %d columns, expected %d", record, len(row), len(headers))
}
switch status := cell(row, "status"); {
case strings.EqualFold(status, "executed"):
case strings.EqualFold(status, "cancelled"), strings.EqualFold(status, "canceled"):
result.Cancelled++
continue
default:
return result, fmt.Errorf("broker record %d has unknown status %q: only executed and cancelled rows are understood", record, status)
}
rawType := cell(row, "type")
event, known := scalableEvents[strings.ToLower(strings.Join(strings.Fields(rawType), " "))]
if !known {
return result, fmt.Errorf("broker record %d has unknown type %q: it may or may not move cash, so nothing was imported", record, rawType)
}
investment := domain.Investment{Event: event}
asset, wanted := cell(row, "assettype"), "Security"
if investment.CashOnly() {
wanted = "Cash"
}
if !strings.EqualFold(asset, wanted) {
return result, fmt.Errorf("broker record %d pairs type %q with assetType %q, expected %q", record, rawType, asset, wanted)
}
currency := strings.ToUpper(cell(row, "currency"))
if currency != strings.ToUpper(account.Currency) {
return result, fmt.Errorf("broker record %d settles in %q but account %q holds %s: currency conversion is not supported", record, currency, account.DisplayName, account.Currency)
}
booking, err := parseMappedCSVDate(cell(row, "date"), "yyyy-mm-dd")
if err != nil {
return result, fmt.Errorf("broker record %d has an invalid date %q", record, cell(row, "date"))
}
description := cell(row, "description")
isin := strings.ToUpper(strings.Join(strings.Fields(cell(row, "isin")), ""))
if isin != "" && !domain.ValidISIN(isin) {
return result, fmt.Errorf("broker record %d has an invalid ISIN %q", record, isin)
}
if isin != "" {
held, exists := byISIN[isin]
if !exists {
held = domain.Instrument{ID: domain.InstrumentID(isin), ISIN: isin, Name: isin, Currency: currency}
byISIN[isin] = held
instruments[held.ID] = held
created[isin] = len(result.Instruments)
result.Instruments = append(result.Instruments, held)
}
investment.InstrumentID = held.ID
// One ISIN appears under several descriptions over the years, and
// once under the ISIN itself. The most recent real description
// names it, and only when this import is the one creating it.
slot, mine := created[isin]
if mine && description != "" && description != isin && booking >= named[isin] {
named[isin] = booking
result.Instruments[slot].Name = description
}
}
amount, amountDrift, err := scalableMoney(cell(row, "amount"))
if err != nil {
return result, fmt.Errorf("broker record %d has an invalid amount %q: %w", record, cell(row, "amount"), err)
}
fee, feeDrift, err := scalableMoney(cell(row, "fee"))
if err != nil {
return result, fmt.Errorf("broker record %d has an invalid fee %q: %w", record, cell(row, "fee"), err)
}
tax, taxDrift, err := scalableMoney(cell(row, "tax"))
if err != nil {
return result, fmt.Errorf("broker record %d has an invalid tax %q: %w", record, cell(row, "tax"), err)
}
if amountDrift|feeDrift|taxDrift != 0 {
result.Rounded++
drift += amountDrift + feeDrift + taxDrift
}
cash := amount
if investment.CashOnly() {
if nonzeroMoney(fee) || nonzeroMoney(tax) {
result.Unapplied = append(result.Unapplied, ScalableNote{Record: record, Date: booking, Description: description, Fee: fee, Tax: tax})
}
investment.Fee, investment.Tax = fee, tax
} else {
if isin == "" {
return result, fmt.Errorf("broker record %d moves a position without an ISIN", record)
}
shares, err := scalableQuantity(cell(row, "shares"))
if err != nil {
return result, fmt.Errorf("broker record %d has an invalid share count %q: %w", record, cell(row, "shares"), err)
}
price, priceDrift, err := scalableMoney(cell(row, "price"))
if err != nil {
return result, fmt.Errorf("broker record %d has an invalid price %q: %w", record, cell(row, "price"), err)
}
if priceDrift != 0 {
return result, fmt.Errorf("broker record %d has a price %q beyond four decimal places", record, cell(row, "price"))
}
signed, err := scalableSignedShares(event, shares)
if err != nil {
return result, fmt.Errorf("broker record %d: %w", record, err)
}
investment.Quantity, investment.Price, investment.Gross = signed, price, amount
if investment.PositionOnly() {
if fee != "" || tax != "" {
return result, fmt.Errorf("broker record %d is a %s carrying fee %q and tax %q, which have no settled cash to apply to", record, rawType, fee, tax)
}
cash = "0.00"
} else {
investment.Fee, investment.Tax = fee, tax
if cash, err = scalableSettlement(amount, fee, tax); err != nil {
return result, fmt.Errorf("broker record %d: %w", record, err)
}
}
}
facts := domain.Facts{
Source: SourceScalable, AccountID: account.ID, BookingDate: booking,
Amount: cash, Currency: currency, RawDescription: description,
ExternalID: cell(row, "reference"), Investment: &investment,
}
// A broker export has no counterparty column, so a deposit or
// withdrawal takes the account's configured settlement IBAN. That is
// what lets the ordinary transfer matcher pair it with the funding
// account instead of leaving it to look like income.
if investment.Event == domain.EventDeposit || investment.Event == domain.EventWithdrawal {
facts.CounterpartyIBAN = normalizeIBAN(account.ReferenceIBAN)
}
if err := domain.ValidateInvestment(facts, account, instruments); err != nil {
return result, fmt.Errorf("broker record %d: %w", record, err)
}
result.Facts = append(result.Facts, facts)
}
if len(result.Facts) == 0 {
return result, errors.New("broker export contains no executed records")
}
result.Rounding = domain.FormatQuantity(drift)
return result, nil
}
// nonzeroMoney reports a figure that could change a balance. The export leaves
// a column blank where it does not apply and writes an explicit zero where it
// applies but is nil; only the second kind is worth putting in front of
// someone before they confirm an import.
func nonzeroMoney(m domain.Money) bool {
minor, err := m.Minor()
return err == nil && minor != 0
}
// scalableMoney reads one German-formatted money cell and returns the exact
// remainder that rounding discarded, in hundred-millionths. The export quotes a
// reinvested distribution to six decimal places, which money's four cannot
// hold; the residue is reported rather than hidden. An empty cell is empty
// money, not zero: blank marks a column that does not apply to the row.
func scalableMoney(value string) (domain.Money, int64, error) {
plain, ok, err := scalablePlain(value)
if !ok || err != nil {
return "", 0, err
}
exact, err := domain.ParseQuantity(plain)
if err != nil {
return "", 0, err
}
units, err := exact.Units()
if err != nil {
return "", 0, err
}
rounded := units / 10000
switch remainder := units % 10000; {
case remainder >= 5000:
rounded++
case remainder <= -5000:
rounded--
}
return domain.FormatMoney(rounded), units - rounded*10000, nil
}
// scalableQuantity reads one German-formatted share count. Nothing is rounded:
// a holding is verified against the broker's own figure, so a count beyond
// eight decimal places is refused instead of silently truncated.
func scalableQuantity(value string) (domain.Quantity, error) {
plain, ok, err := scalablePlain(value)
if !ok || err != nil {
return "", err
}
return domain.ParseQuantity(plain)
}
// scalablePlain normalizes one numeric cell to a plain decimal string, or
// reports that the cell was blank.
func scalablePlain(value string) (string, bool, error) {
value = strings.NewReplacer("\u00a0", "", "\u202f", "", "'", "").Replace(strings.TrimSpace(value))
if value == "" {
return "", false, nil
}
plain, err := germanDecimal(value)
return plain, err == nil, err
}
// scalableSignedShares resolves the export's two sign conventions. A buy, sell
// or reinvestment carries an unsigned count and takes its direction from the
// type; a corporate action or depot transfer is already signed.
func scalableSignedShares(event string, shares domain.Quantity) (domain.Quantity, error) {
units, err := shares.Units()
if err != nil {
return "", err
}
if units == 0 {
return "", fmt.Errorf("%s requires a nonzero share count", event)
}
switch event {
case domain.EventBuy, domain.EventReinvest, domain.EventSell:
if units < 0 {
return "", fmt.Errorf("%s carries a signed share count %s; only corporate actions and depot transfers are signed", event, shares)
}
if event == domain.EventSell {
units = -units
}
}
return domain.FormatQuantity(units), nil
}
// scalableSettlement is gross minus fee minus tax: the cash a trade moved. The
// broker states fee and tax as positive deductions whichever way the trade
// went, so both are subtracted from a signed gross.
func scalableSettlement(gross, fee, tax domain.Money) (domain.Money, error) {
total := int64(0)
for _, deduction := range []struct {
sign int64
money domain.Money
}{{1, gross}, {-1, fee}, {-1, tax}} {
if deduction.money == "" {
continue
}
minor, err := deduction.money.Minor()
if err != nil {
return "", err
}
total += deduction.sign * minor
}
return domain.FormatMoney(total), nil
}
+243
View File
@@ -0,0 +1,243 @@
package banking
import (
"strings"
"testing"
"finance-duck/internal/domain"
)
const scalableHeader = "date;time;status;reference;description;assetType;type;isin;shares;price;amount;fee;tax;currency\n"
// Every row below is a real Scalable Capital export line. Together they cover
// all ten row types, both sign conventions, a reference shared by two legs of
// one event, a six-decimal reinvestment, a zero-price corporate action, a
// depot switch, a cancelled retry, and one ISIN whose description changes over
// time and whose latest description is the one that names it.
var scalableRows = []string{
`2024-11-10;02:00:00;Executed;ABCDEF012345;Scalable Instant Cash Deposit;Cash;Deposit;;;;800,00;;;EUR`,
`2026-08-18;02:00:00;Executed;ITLLRRVPRK11ZGNPAD2VNC;Scalable Broker PRIME bis 16.09.2026;Cash;Deposit;;;;4,99;0,00;;EUR`,
`2026-08-18;02:00:00;Executed;LZLVVJYLNJY9ARAK;Entgelt PRIME+ Broker;Cash;Fee;;;;-4,99;0,00;;EUR`,
`2026-07-16;14:19:06;Executed;O9HNT63GYQVUNPXEXQMSCJ;Scalable Capital Broker Auszahlung;Cash;Withdrawal;;;;-4.458,19;0,00;0,00;EUR`,
`2026-01-02;01:00:00;Executed;INTEREST0001;Zinsen;Cash;Interest;;;;12,34;;1,23;EUR`,
`2026-01-20;01:00:00;Executed;429776_rrCjP4EcbpefpNiVQeD495;Taiwan Semiconductor Manufact. ADR;Cash;Distribution;US8740391003;;;29,68;0,00;7,43;EUR`,
`2026-01-20;01:00:00;Executed;429776_rrCjP4EcbpefpNiVQeD495;Taiwan Semiconductor Manufact. ADR;Security;Reinvestment_Distribution;US8740391003;0,076494;388,00;-29,679672;0,00;0,00;EUR`,
`2025-05-07;09:02:29;Executed;SCALTThBbxx6z5Z;Rheinmetall Long 10x Faktor-Zertifikat HVB;Security;Buy;DE000UG4V0Z7;14;26,45;-370,30;0,00;0,00;EUR`,
`2025-09-17;15:14:49;Executed;SCALwBaNVPpjf8p;Rheinmetall Long 10x Factor HVB;Security;Buy;DE000UG4V0Z7;203;1,23;-249,69;0,99;0,00;EUR`,
`2025-09-18;13:38:08;Executed;SCALSVuyHibZT4w;Rheinmetall Long 10x Factor HVB;Security;Buy;DE000UG4V0Z7;6;1,10;-6,60;0,99;0,00;EUR`,
`2025-10-28;01:00:00;Executed;48231_rrCjP4EcbpefpNiVQeD495;Rheinmetall Long 10x Factor HVB;Cash;Distribution;DE000UG4V0Z7;;;32,64;;-1,42;EUR`,
`2025-10-28;01:00:00;Executed;48231_rrCjP4EcbpefpNiVQeD495;Rheinmetall Long 10x Factor HVB;Security;Corporate action;DE000UG4V0Z7;-223;0,14;-31,22;;;EUR`,
`2025-10-21;02:00:00;Executed;WWUM 00566579567;FR0014012ZX8;Security;Corporate action;FR0014012ZX8;1,14;0,00;0,00;;;EUR`,
`2025-12-05;01:00:00;Executed;WWUM 00590038089;Amundi MSCI USA Daily (2x) Leveraged (Acc);Security;Security transfer;FR0010755611;-65;25,235;-1.640,275;;;EUR`,
`2025-12-06;01:00:00;Executed;SWITCH-101-rrCjP4EcbpefpNiVQeD495-FR0010755611-WDP;Amundi MSCI USA Daily (2x) Leveraged (Acc);Security;Security transfer;FR0010755611;65;25,59;1.663,35;;;EUR`,
`2026-03-17;01:00:00;Executed;SCALmNYgdoA58V;Amundi Core MSCI World (Acc);Security;Sell;IE000BI8OT95;61;158,385;9.661,485;0,00;220,47;EUR`,
`2025-01-27;16:26:31;Cancelled;SCALCRFHbTWXN9h;Amundi Leveraged MSCI USA Daily (Acc);Security;Buy;FR0010755611;0;0,00;0,00;0,00;0,00;EUR`,
}
func brokerAccount() domain.Account {
return domain.Account{
ID: "acct_broker", DisplayName: "Scalable", Institution: "Scalable Capital",
Currency: "EUR", Kind: domain.AccountInvestment,
IBAN: "DE02120300000000202051", ReferenceIBAN: "DE89370400440532013000", Active: true,
}
}
func readBroker(t *testing.T, rows ...string) ScalableImport {
t.Helper()
file, err := ReadCSV(strings.NewReader(scalableHeader + strings.Join(rows, "\n") + "\n"))
if err != nil {
t.Fatal(err)
}
result, err := ParseScalableCSV(file, brokerAccount(), nil)
if err != nil {
t.Fatal(err)
}
return result
}
func TestScalableExportSettlesCashAndPositionsSeparately(t *testing.T) {
result := readBroker(t, scalableRows...)
if result.Cancelled != 1 {
t.Fatalf("cancelled rows imported: %d skipped", result.Cancelled)
}
if len(result.Facts) != len(scalableRows)-1 {
t.Fatalf("imported %d of %d executed rows", len(result.Facts), len(scalableRows)-1)
}
// The cash a row settles, per row class. A cash row's amount is already
// net; a trade settles gross minus fee minus tax; a corporate action or
// depot transfer settles nothing at all.
wantCash := map[string]string{
"ABCDEF012345": "800.00",
"ITLLRRVPRK11ZGNPAD2VNC": "4.99",
"LZLVVJYLNJY9ARAK": "-4.99",
"O9HNT63GYQVUNPXEXQMSCJ": "-4458.19",
"INTEREST0001": "12.34",
"SCALTThBbxx6z5Z": "-370.30",
"SCALwBaNVPpjf8p": "-250.68",
"SCALSVuyHibZT4w": "-7.59",
"WWUM 00566579567": "0.00",
"WWUM 00590038089": "0.00",
"SWITCH-101-rrCjP4EcbpefpNiVQeD495-FR0010755611-WDP": "0.00",
"SCALmNYgdoA58V": "9441.015",
}
total := int64(0)
for _, f := range result.Facts {
minor, err := f.Amount.Minor()
if err != nil {
t.Fatal(err)
}
total += minor
if want, ok := wantCash[f.ExternalID]; ok && string(f.Amount) != want {
t.Errorf("%s settled %s, want %s", f.ExternalID, f.Amount, want)
}
}
if got := string(domain.FormatMoney(total)); got != "5199.2353" {
t.Errorf("cash balance %s, want 5199.2353", got)
}
// Signs: a buy and a reinvestment add, a sell removes, and a corporate
// action or depot transfer keeps the sign the export printed.
holdings := map[string]int64{}
for _, f := range result.Facts {
if f.Investment.InstrumentID == "" || f.Investment.Quantity == "" {
continue
}
units, err := f.Investment.Quantity.Units()
if err != nil {
t.Fatal(err)
}
holdings[f.Investment.InstrumentID] += units
}
for isin, want := range map[string]int64{
"DE000UG4V0Z7": 0, // 14 + 203 + 6 - 223, the knock-out closing the position
"FR0010755611": 0, // a depot switch out and back
"FR0014012ZX8": 114000000, // 1.14 free units at no price
"US8740391003": 7649400, // 0.076494 reinvested
"IE000BI8OT95": -6100000000,
} {
if got := holdings[domain.InstrumentID(isin)]; got != want {
t.Errorf("%s holds %d hundred-millionths, want %d", isin, got, want)
}
}
// One ISIN, several descriptions over the years, and one row that carries
// the ISIN as its own description.
names := map[string]string{}
for _, v := range result.Instruments {
names[v.ISIN] = v.Name
}
for isin, want := range map[string]string{
"DE000UG4V0Z7": "Rheinmetall Long 10x Factor HVB",
"FR0014012ZX8": "FR0014012ZX8",
"US8740391003": "Taiwan Semiconductor Manufact. ADR",
} {
if names[isin] != want {
t.Errorf("%s named %q, want %q", isin, names[isin], want)
}
}
// Six decimal places do not fit in money. The residue is reported, not hidden.
if result.Rounded != 1 || result.Rounding != "0.000028" {
t.Errorf("rounding reported as %d rows and %s, want 1 row and 0.000028", result.Rounded, result.Rounding)
}
// A broker cash amount is already net of tax, so the tax column is
// recorded and never subtracted again.
if len(result.Unapplied) != 3 {
t.Fatalf("unapplied fee/tax notes: %+v", result.Unapplied)
}
for _, note := range result.Unapplied {
if note.Tax == "" {
t.Errorf("note without the figure that was not applied: %+v", note)
}
}
}
// The broker reuses one reference for every leg of an economic event, so
// dedupe on the reference alone silently drops half of each corporate action.
func TestSharedBrokerReferenceKeepsEveryLeg(t *testing.T) {
result := readBroker(t, scalableRows...)
data := domain.NewDataset()
data.Accounts = []domain.Account{brokerAccount()}
data.Instruments = result.Instruments
added, err := NormalizeAndDedupe(data, result.Facts)
if err != nil {
t.Fatal(err)
}
if len(added) != len(result.Facts) {
t.Fatalf("dedupe kept %d of %d legs", len(added), len(result.Facts))
}
data.Transactions = added
if err := domain.Validate(data); err != nil {
t.Fatal(err)
}
again, err := NormalizeAndDedupe(data, result.Facts)
if err != nil || len(again) != 0 {
t.Fatalf("re-import was not idempotent: %v %+v", err, again)
}
}
// Every one of these can move money that never moved, so each rejects the
// whole file rather than importing the rest.
func TestScalableRejectsRowsItCannotAccountFor(t *testing.T) {
for name, row := range map[string]string{
"unknown type": `2026-01-05;01:00:00;Executed;R1;Something;Cash;Vorabpauschale;;;;-12,00;;;EUR`,
"unknown status": `2026-01-05;01:00:00;Pending;R1;Something;Cash;Deposit;;;;12,00;;;EUR`,
"asset type mismatch": `2026-01-05;01:00:00;Executed;R1;Something;Security;Deposit;;;;12,00;;;EUR`,
"foreign currency": `2026-01-05;01:00:00;Executed;R1;Something;Cash;Deposit;;;;12,00;;;USD`,
"mismatched gross": `2026-01-05;01:00:00;Executed;R1;Something;Security;Buy;DE000UG4V0Z7;10;2,00;-25,00;0,00;0,00;EUR`,
"signed buy": `2026-01-05;01:00:00;Executed;R1;Something;Security;Buy;DE000UG4V0Z7;-10;2,00;20,00;0,00;0,00;EUR`,
"paid corporate": `2026-01-05;01:00:00;Executed;R1;Something;Security;Corporate action;DE000UG4V0Z7;-5;2,00;-10,00;1,00;0,00;EUR`,
"security without ISIN": `2026-01-05;01:00:00;Executed;R1;Something;Security;Buy;;10;2,00;-20,00;0,00;0,00;EUR`,
"invalid ISIN": `2026-01-05;01:00:00;Executed;R1;Something;Security;Buy;NOTANISIN;10;2,00;-20,00;0,00;0,00;EUR`,
} {
file, err := ReadCSV(strings.NewReader(scalableHeader + row + "\n"))
if err != nil {
t.Fatalf("%s: %v", name, err)
}
if _, err := ParseScalableCSV(file, brokerAccount(), nil); err == nil {
t.Errorf("%s: accepted a row that can move money it should not", name)
}
}
}
// An inconsistent row is caught, and a uniformly mangled one is not. Where the
// whole row lost its separator together, shares times price still equals the
// amount at every scale, so no check inside the file can see it. This is a
// known limit, not an oversight: only a price cross-check against an outside
// provider distinguishes 1 x 25,795 from 1 x 25795, and that is deliberately
// out of scope. The test exists so nobody claims coverage that is not here.
func TestSingleShareRowCatchesOnlyInconsistentArithmetic(t *testing.T) {
valid := `2024-12-09;10:48:44;Executed;SCALfhSXRbGWKno;Amundi Leveraged MSCI USA Daily (Acc);Security;Buy;FR0010755611;1;25,795;-25,795;0,99;0,00;EUR`
result := readBroker(t, valid)
if got := result.Facts[0].Amount; got != "-26.785" {
t.Fatalf("one-share buy settled %s, want -26.785", got)
}
inconsistent := strings.Replace(valid, "25,795;-25,795", "25,795;-257,95", 1)
file, err := ReadCSV(strings.NewReader(scalableHeader + inconsistent + "\n"))
if err != nil {
t.Fatal(err)
}
if _, err := ParseScalableCSV(file, brokerAccount(), nil); err == nil {
t.Fatal("accepted a one-share row whose amount is off by a factor of ten")
}
uniform := readBroker(t, strings.Replace(valid, "1;25,795;-25,795", "1;25795;-25795", 1))
if got := uniform.Facts[0].Investment.Gross; got != "-25795.00" {
t.Fatalf("uniformly mangled row read as %s: the file-internal identity cannot see it, and that must stay visible here", got)
}
}
// A thousands dot and a decimal dot are both present in one share column.
func TestBrokerShareColumnDistinguishesGroupingFromDecimals(t *testing.T) {
result := readBroker(t,
`2026-02-24;17:11:29;Executed;G1;iShares Global Clean Energy Transition (Dist);Security;Buy;IE00B1XNHC34;1.014;9,408;-9.539,712;0,00;0,00;EUR`,
`2026-02-25;17:11:29;Executed;G2;iShares Global Clean Energy Transition (Dist);Security;Buy;IE00B1XNHC34;1.14;9,408;-10,7251;0,00;0,00;EUR`,
)
if got := result.Facts[0].Investment.Quantity; got != "1014" {
t.Errorf("grouped share count read as %s, want 1014", got)
}
if got := result.Facts[1].Investment.Quantity; got != "1.14" {
t.Errorf("fractional share count read as %s, want 1.14", got)
}
}
+9 -4
View File
@@ -77,12 +77,17 @@ type Proposal struct {
} }
// ruleProposal applies deterministic local classification: an existing transfer // ruleProposal applies deterministic local classification: an existing transfer
// keeps its enrichment, and a matching merchant alias contributes that merchant // or broker fact keeps its enrichment, and a matching merchant alias
// plus, only when the merchant opts in, its default category and tags. done // contributes that merchant plus, only when the merchant opts in, its default
// reports that no provider call can improve the result. // category and tags. done reports that no provider call can improve the result.
func ruleProposal(facts domain.Facts, data domain.Dataset, forceAI bool) (Proposal, bool, error) { func ruleProposal(facts domain.Facts, data domain.Dataset, forceAI bool) (Proposal, bool, error) {
for _, tx := range data.Transactions { for _, tx := range data.Transactions {
if tx.Facts.ID == facts.ID && tx.Enrichment.Kind == "transfer" { if tx.Facts.ID != facts.ID {
continue
}
// Moving your own money between your own cash and your own positions
// has no merchant and no category, and the model must never see it.
if tx.Enrichment.Kind == "transfer" || tx.Enrichment.Kind == domain.KindInvestment {
e := tx.Enrichment e := tx.Enrichment
e.TagIDs = append([]string{}, e.TagIDs...) e.TagIDs = append([]string{}, e.TagIDs...)
return Proposal{Enrichment: e}, true, nil return Proposal{Enrichment: e}, true, nil
+259 -17
View File
@@ -2,9 +2,11 @@ package domain
import ( import (
"crypto/rand" "crypto/rand"
"crypto/sha256"
"encoding/hex" "encoding/hex"
"fmt" "fmt"
"math" "math"
"math/big"
"regexp" "regexp"
"strconv" "strconv"
"strings" "strings"
@@ -14,19 +16,34 @@ import (
var idPattern = regexp.MustCompile(`^[A-Za-z][A-Za-z0-9_-]{0,127}$`) var idPattern = regexp.MustCompile(`^[A-Za-z][A-Za-z0-9_-]{0,127}$`)
var currencyPattern = regexp.MustCompile(`^[A-Z]{3}$`) var currencyPattern = regexp.MustCompile(`^[A-Z]{3}$`)
var isinPattern = regexp.MustCompile(`^[A-Z]{2}[A-Z0-9]{9}[0-9]$`)
const moneyScale = 4
const quantityScale = 8
// ParseMoney accepts exact decimal values representable as signed 64-bit ten-thousandths. // ParseMoney accepts exact decimal values representable as signed 64-bit ten-thousandths.
// This intentionally bounds the otherwise larger DECIMAL(24,4) database domain. // This intentionally bounds the otherwise larger DECIMAL(24,4) database domain.
func ParseMoney(s string) (Money, error) { func ParseMoney(s string) (Money, error) {
n, err := parseMinor(s) n, err := parseScaled(s, moneyScale, "money", "four")
if err != nil { if err != nil {
return "", err return "", err
} }
return Money(formatMinor(n)), nil return Money(formatScaled(n, moneyScale, 2)), nil
} }
func parseMinor(s string) (int64, error) {
// ParseQuantity accepts exact share counts representable as signed 64-bit
// hundred-millionths. Money's four places cannot hold a reinvested fraction of
// a share, and a truncated share count silently misstates a holding.
func ParseQuantity(s string) (Quantity, error) {
n, err := parseScaled(s, quantityScale, "quantity", "eight")
if err != nil {
return "", err
}
return Quantity(formatScaled(n, quantityScale, 0)), nil
}
func parseScaled(s string, scale int, noun, places string) (int64, error) {
invalid := func() (int64, error) { invalid := func() (int64, error) {
return 0, fmt.Errorf("invalid or out-of-range money %q: require signed 64-bit ten-thousandths, at most four fractional digits", s) return 0, fmt.Errorf("invalid or out-of-range %s %q: require signed 64-bit value with at most %s fractional digits", noun, s, places)
} }
if s == "" { if s == "" {
return invalid() return invalid()
@@ -65,7 +82,7 @@ func parseMinor(s string) (int64, error) {
} }
if fraction >= 0 { if fraction >= 0 {
fraction++ fraction++
if fraction > 4 { if fraction > scale {
return invalid() return invalid()
} }
} }
@@ -78,7 +95,7 @@ func parseMinor(s string) (int64, error) {
if fraction < 0 { if fraction < 0 {
fraction = 0 fraction = 0
} }
for range 4 - fraction { for range scale - fraction {
if magnitude > limit/10 { if magnitude > limit/10 {
return invalid() return invalid()
} }
@@ -92,22 +109,28 @@ func parseMinor(s string) (int64, error) {
} }
return int64(magnitude), nil return int64(magnitude), nil
} }
func formatMinor(n int64) string {
// formatScaled renders exact units. minFraction keeps money at two places for
// display while letting a whole share count render without eight zeros.
func formatScaled(n int64, scale, minFraction int) string {
s := strconv.FormatInt(n, 10) s := strconv.FormatInt(n, 10)
sign := "" sign := ""
if strings.HasPrefix(s, "-") { if strings.HasPrefix(s, "-") {
sign, s = "-", s[1:] sign, s = "-", s[1:]
} }
if len(s) < 5 { if len(s) < scale+1 {
s = strings.Repeat("0", 5-len(s)) + s s = strings.Repeat("0", scale+1-len(s)) + s
} }
whole, fraction := s[:len(s)-4], strings.TrimRight(s[len(s)-4:], "0") whole, fraction := s[:len(s)-scale], strings.TrimRight(s[len(s)-scale:], "0")
if len(fraction) < 2 { if len(fraction) < minFraction {
fraction += strings.Repeat("0", 2-len(fraction)) fraction += strings.Repeat("0", minFraction-len(fraction))
}
if fraction == "" {
return sign + whole
} }
return sign + whole + "." + fraction return sign + whole + "." + fraction
} }
func (m Money) Minor() (int64, error) { return parseMinor(string(m)) } func (m Money) Minor() (int64, error) { return parseScaled(string(m), moneyScale, "money", "four") }
func (m Money) String() string { func (m Money) String() string {
parsed, err := ParseMoney(string(m)) parsed, err := ParseMoney(string(m))
if err != nil { if err != nil {
@@ -115,6 +138,25 @@ func (m Money) String() string {
} }
return string(parsed) return string(parsed)
} }
func (q Quantity) Units() (int64, error) {
return parseScaled(string(q), quantityScale, "quantity", "eight")
}
func (q Quantity) String() string {
parsed, err := ParseQuantity(string(q))
if err != nil {
return string(q)
}
return string(parsed)
}
// FormatMoney renders exact ten-thousandths as money, and FormatQuantity
// renders exact hundred-millionths as a share count. Exact units are the only
// safe currency for arithmetic, and these are how a computed total re-enters
// the journal without a float ever being involved.
func FormatMoney(minor int64) Money { return Money(formatScaled(minor, moneyScale, 2)) }
func FormatQuantity(units int64) Quantity {
return Quantity(formatScaled(units, quantityScale, 0))
}
func NewID(prefix string) string { func NewID(prefix string) string {
if !idPattern.MatchString(prefix) || len(prefix) > 94 { if !idPattern.MatchString(prefix) || len(prefix) > 94 {
panic("invalid ID prefix") panic("invalid ID prefix")
@@ -129,20 +171,40 @@ func NewDataset() Dataset {
return Dataset{Accounts: []Account{}, Categories: []Category{ return Dataset{Accounts: []Account{}, Categories: []Category{
{ID: "cat_expenses", Name: "Expenses", Kind: "expense"}, {ID: ExpenseFallback, Name: "Unclassified", ParentID: "cat_expenses", Kind: "expense"}, {ID: "cat_expenses", Name: "Expenses", Kind: "expense"}, {ID: ExpenseFallback, Name: "Unclassified", ParentID: "cat_expenses", Kind: "expense"},
{ID: "cat_income", Name: "Income", Kind: "income"}, {ID: IncomeFallback, Name: "Unclassified", ParentID: "cat_income", Kind: "income"}, {ID: "cat_income", Name: "Income", Kind: "income"}, {ID: IncomeFallback, Name: "Unclassified", ParentID: "cat_income", Kind: "income"},
}, Tags: []Tag{}, Merchants: []Merchant{}, Transactions: []Transaction{}} }, Tags: []Tag{}, Merchants: []Merchant{}, Instruments: []Instrument{}, Transactions: []Transaction{}}
}
// InstrumentID derives a stable registry ID from an ISIN so re-importing the
// same export never creates a second instrument for one security.
func InstrumentID(isin string) string {
sum := sha256.Sum256([]byte("instrument\x00" + strings.ToUpper(strings.TrimSpace(isin))))
return "ins_" + hex.EncodeToString(sum[:16])
} }
func Clone(d Dataset) Dataset { func Clone(d Dataset) Dataset {
c := Dataset{Accounts: append([]Account{}, d.Accounts...), Categories: append([]Category{}, d.Categories...), Tags: append([]Tag{}, d.Tags...), Merchants: append([]Merchant{}, d.Merchants...), Transactions: append([]Transaction{}, d.Transactions...)} c := Dataset{Accounts: append([]Account{}, d.Accounts...), Categories: append([]Category{}, d.Categories...), Tags: append([]Tag{}, d.Tags...), Merchants: append([]Merchant{}, d.Merchants...), Instruments: append([]Instrument{}, d.Instruments...), Transactions: append([]Transaction{}, d.Transactions...)}
for i := range c.Merchants { for i := range c.Merchants {
c.Merchants[i].Aliases = append([]string{}, d.Merchants[i].Aliases...) c.Merchants[i].Aliases = append([]string{}, d.Merchants[i].Aliases...)
c.Merchants[i].DefaultTagIDs = append([]string{}, d.Merchants[i].DefaultTagIDs...) c.Merchants[i].DefaultTagIDs = append([]string{}, d.Merchants[i].DefaultTagIDs...)
} }
for i := range c.Transactions { for i := range c.Transactions {
c.Transactions[i].Enrichment.TagIDs = append([]string{}, d.Transactions[i].Enrichment.TagIDs...) c.Transactions[i].Enrichment.TagIDs = append([]string{}, d.Transactions[i].Enrichment.TagIDs...)
// Facts are immutable, but a shared pointer would let one dataset's
// edit reach another's copy.
if inv := d.Transactions[i].Facts.Investment; inv != nil {
copied := *inv
c.Transactions[i].Facts.Investment = &copied
}
} }
return c return c
} }
// Fallback classifies a fact that no rule or model claimed. Broker facts never
// take the sign-based branch: an unmatched deposit is not income and a broker
// fee paid out of an investment account is not household spending.
func Fallback(f Facts) Enrichment { func Fallback(f Facts) Enrichment {
if f.Investment != nil {
return Enrichment{Kind: KindInvestment, TagIDs: []string{}, Classification: Provenance{Source: "fallback"}}
}
kind, category := "expense", ExpenseFallback kind, category := "expense", ExpenseFallback
n, err := f.Amount.Minor() n, err := f.Amount.Minor()
if err == nil && n > 0 { if err == nil && n > 0 {
@@ -185,6 +247,16 @@ func validText(values ...string) bool {
return true return true
} }
// ValidISIN reports a syntactically valid ISIN: two country letters, nine
// alphanumerics and a check digit.
func ValidISIN(s string) bool { return isinPattern.MatchString(s) }
// ValidateInvestment checks one broker fact against the investment model. An
// importer calls it per record so a malformed export is refused with the
// record that caused it, rather than at commit with only an ID.
func ValidateInvestment(f Facts, account Account, instruments map[string]Instrument) error {
return validateInvestment(f, account, instruments)
}
func Validate(d Dataset) error { func Validate(d Dataset) error {
ids := map[string]string{} ids := map[string]string{}
register := func(id, kind string) error { register := func(id, kind string) error {
@@ -205,9 +277,12 @@ func Validate(d Dataset) error {
if err := register(a.ID, "account"); err != nil { if err := register(a.ID, "account"); err != nil {
return err return err
} }
if !nonblank(a.DisplayName) || !currencyPattern.MatchString(a.Currency) || !validText(a.Institution, a.ExternalAccountID, a.IBAN) { if !nonblank(a.DisplayName) || !currencyPattern.MatchString(a.Currency) || !validText(a.Institution, a.ExternalAccountID, a.IBAN, a.ReferenceIBAN) {
return fmt.Errorf("account %q: valid UTF-8 name and three-letter uppercase currency required", a.ID) return fmt.Errorf("account %q: valid UTF-8 name and three-letter uppercase currency required", a.ID)
} }
if a.Kind != "" && a.Kind != AccountCash && a.Kind != AccountInvestment {
return fmt.Errorf("account %q: kind must be %q or %q", a.ID, AccountCash, AccountInvestment)
}
accounts[a.ID] = a accounts[a.ID] = a
} }
for _, c := range d.Categories { for _, c := range d.Categories {
@@ -285,6 +360,24 @@ func Validate(d Dataset) error {
aliases[key] = true aliases[key] = true
} }
} }
instruments := map[string]Instrument{}
isins := map[string]string{}
for _, v := range d.Instruments {
if err := register(v.ID, "instrument"); err != nil {
return err
}
if !isinPattern.MatchString(v.ISIN) {
return fmt.Errorf("instrument %q: ISIN must be two letters, nine alphanumerics and a check digit", v.ID)
}
if other, ok := isins[v.ISIN]; ok {
return fmt.Errorf("instrument %q: ISIN %s already held by %q", v.ID, v.ISIN, other)
}
if !nonblank(v.Name) || !currencyPattern.MatchString(v.Currency) {
return fmt.Errorf("instrument %q: valid UTF-8 name and three-letter uppercase currency required", v.ID)
}
isins[v.ISIN] = v.ID
instruments[v.ID] = v
}
for _, t := range d.Transactions { for _, t := range d.Transactions {
f := t.Facts f := t.Facts
if err := register(f.ID, "transaction"); err != nil { if err := register(f.ID, "transaction"); err != nil {
@@ -309,6 +402,9 @@ func Validate(d Dataset) error {
if !validText(f.RawDescription, f.ExternalID, f.Counterparty, f.CounterpartyIBAN) { if !validText(f.RawDescription, f.ExternalID, f.Counterparty, f.CounterpartyIBAN) {
return fmt.Errorf("transaction %q: bank facts must be valid UTF-8", f.ID) return fmt.Errorf("transaction %q: bank facts must be valid UTF-8", f.ID)
} }
if err := validateInvestment(f, a, instruments); err != nil {
return fmt.Errorf("transaction %q: %w", f.ID, err)
}
} }
index := enrichmentIndex{categories: categories, children: children, tags: tags, merchants: map[string]bool{}, transactions: map[string]Transaction{}} index := enrichmentIndex{categories: categories, children: children, tags: tags, merchants: map[string]bool{}, transactions: map[string]Transaction{}}
for _, m := range d.Merchants { for _, m := range d.Merchants {
@@ -351,9 +447,12 @@ func ValidateEnrichment(d Dataset, f Facts, e Enrichment) error {
return index.validate(f, e) return index.validate(f, e)
} }
func (index enrichmentIndex) validate(f Facts, e Enrichment) error { func (index enrichmentIndex) validate(f Facts, e Enrichment) error {
if e.Kind != "expense" && e.Kind != "income" && e.Kind != "transfer" { if e.Kind != "expense" && e.Kind != "income" && e.Kind != "transfer" && e.Kind != KindInvestment {
return fmt.Errorf("invalid enrichment kind %q", e.Kind) return fmt.Errorf("invalid enrichment kind %q", e.Kind)
} }
if (e.Kind == KindInvestment) != (f.Investment != nil && e.Kind != "transfer") {
return fmt.Errorf("only broker facts carry kind %q, and every unlinked broker fact must", KindInvestment)
}
seen := map[string]bool{} seen := map[string]bool{}
for _, id := range e.TagIDs { for _, id := range e.TagIDs {
if !index.tags[id] || seen[id] { if !index.tags[id] || seen[id] {
@@ -372,6 +471,15 @@ func (index enrichmentIndex) validate(f Facts, e Enrichment) error {
return fmt.Errorf("invalid classification timestamp") return fmt.Errorf("invalid classification timestamp")
} }
} }
if e.Kind == KindInvestment {
if e.CategoryID != "" || e.MerchantID != "" || e.TransferPeerID != "" {
return fmt.Errorf("investment must not have category, merchant or transfer peer")
}
if e.Classification.Source == "ai" || e.Classification.Source == "openrouter" {
return fmt.Errorf("AI cannot classify investments")
}
return nil
}
if e.Kind == "transfer" { if e.Kind == "transfer" {
if e.CategoryID != "" || e.MerchantID != "" { if e.CategoryID != "" || e.MerchantID != "" {
return fmt.Errorf("transfer must not have category or merchant") return fmt.Errorf("transfer must not have category or merchant")
@@ -379,6 +487,9 @@ func (index enrichmentIndex) validate(f Facts, e Enrichment) error {
if e.Classification.Source == "ai" || e.Classification.Source == "openrouter" { if e.Classification.Source == "ai" || e.Classification.Source == "openrouter" {
return fmt.Errorf("AI cannot classify transfers") return fmt.Errorf("AI cannot classify transfers")
} }
if f.Investment != nil && !f.Investment.CashOnly() {
return fmt.Errorf("only a broker cash movement can be linked as a transfer, not %q", f.Investment.Event)
}
amount, err := f.Amount.Minor() amount, err := f.Amount.Minor()
if err != nil { if err != nil {
return err return err
@@ -413,3 +524,134 @@ func (index enrichmentIndex) validate(f Facts, e Enrichment) error {
} }
return nil return nil
} }
func optionalMoney(m Money) (int64, error) {
if m == "" {
return 0, nil
}
return m.Minor()
}
func optionalQuantity(q Quantity) (int64, error) {
if q == "" {
return 0, nil
}
return q.Units()
}
// RoundedProduct multiplies an exact share count by an exact price and rounds
// to money's four places, half away from zero. Quantity is 1e-8 units and
// price is 1e-4 units, so the product is 1e-12 and needs 128-bit width.
func RoundedProduct(quantity, price int64) (int64, bool) {
product := new(big.Int).Mul(big.NewInt(quantity), big.NewInt(price))
half := big.NewInt(50_000_000)
if product.Sign() < 0 {
product.Sub(product, half)
} else {
product.Add(product, half)
}
rounded := product.Quo(product, big.NewInt(100_000_000))
if !rounded.IsInt64() {
return 0, false
}
return rounded.Int64(), true
}
// validateInvestment enforces the broker row model.
//
// A cash row's amount is the money that actually moved and is already net of
// the tax the broker withheld or refunded, so its tax column is recorded but
// never applied. A buy, sell or reinvestment quotes a gross of shares times
// price and settles gross minus fee minus tax. A corporate action or depot
// transfer moves a position at a valuation and must never touch cash: treating
// its amount as money conjures or destroys it.
//
// Every security row is checked against shares times price. That is the only
// check that catches a lost decimal separator, and it is worthless without it:
// a one-share row satisfies every other invariant at any scale.
func validateInvestment(f Facts, a Account, instruments map[string]Instrument) error {
inv := f.Investment
if inv == nil {
return nil
}
if !a.Investing() {
return fmt.Errorf("investment leg requires an account of kind %q", AccountInvestment)
}
if !inv.CashOnly() && !inv.Settling() && !inv.PositionOnly() {
return fmt.Errorf("unknown investment event %q", inv.Event)
}
if inv.InstrumentID != "" {
v, ok := instruments[inv.InstrumentID]
if !ok {
return fmt.Errorf("unknown instrument %q", inv.InstrumentID)
}
if v.Currency != f.Currency {
return fmt.Errorf("instrument %s trades in %s but this fact settles in %s", v.ISIN, v.Currency, f.Currency)
}
}
quantity, err := optionalQuantity(inv.Quantity)
if err != nil {
return err
}
price, err := optionalMoney(inv.Price)
if err != nil {
return err
}
gross, err := optionalMoney(inv.Gross)
if err != nil {
return err
}
fee, err := optionalMoney(inv.Fee)
if err != nil {
return err
}
tax, err := optionalMoney(inv.Tax)
if err != nil {
return err
}
amount, err := f.Amount.Minor()
if err != nil {
return err
}
if inv.CashOnly() {
if quantity != 0 || inv.Price != "" || inv.Gross != "" {
return fmt.Errorf("%s moves cash only: it carries no quantity, price or gross", inv.Event)
}
return nil
}
if inv.InstrumentID == "" {
return fmt.Errorf("%s requires an instrument", inv.Event)
}
if quantity == 0 {
return fmt.Errorf("%s requires a nonzero quantity", inv.Event)
}
// A position-only valuation carries the sign of the position change; a
// settled trade carries the sign of the cash, which is the opposite.
expected, ok := RoundedProduct(quantity, price)
if !ok {
return fmt.Errorf("%s quantity times price is out of range", inv.Event)
}
if inv.Settling() {
expected = -expected
}
if gross != expected {
return fmt.Errorf("%s gross %s does not equal quantity %s times price %s", inv.Event, Money(formatScaled(gross, moneyScale, 2)), inv.Quantity.String(), inv.Price.String())
}
if inv.PositionOnly() {
if amount != 0 {
return fmt.Errorf("%s moves position only, but this fact carries cash %s", inv.Event, f.Amount.String())
}
if fee != 0 || tax != 0 {
return fmt.Errorf("%s cannot carry a fee or tax", inv.Event)
}
return nil
}
if (inv.Event == EventSell) != (quantity < 0) {
return fmt.Errorf("%s must %s the position", inv.Event, map[bool]string{true: "reduce", false: "increase"}[inv.Event == EventSell])
}
settled := new(big.Int).Sub(big.NewInt(gross), big.NewInt(fee))
settled.Sub(settled, big.NewInt(tax))
if !settled.IsInt64() || settled.Int64() != amount {
return fmt.Errorf("%s cash %s does not equal gross %s minus fee %s minus tax %s", inv.Event, f.Amount.String(), inv.Gross.String(), inv.Fee.String(), inv.Tax.String())
}
return nil
}
+99
View File
@@ -3,15 +3,104 @@ package domain
// Money is an exact decimal string bounded to signed 64-bit ten-thousandths. // Money is an exact decimal string bounded to signed 64-bit ten-thousandths.
type Money string type Money string
// Quantity is an exact decimal string bounded to signed 64-bit hundred-millionths.
// Broker share counts are fractional: savings plans and reinvested distributions
// settle in eight decimal places, which Money cannot represent.
type Quantity string
// Account kinds. An empty kind is a cash account: the field was added after the
// journal format, and absent means the original behaviour.
const (
AccountCash = "cash"
AccountInvestment = "investment"
)
type Account struct { type Account struct {
ID string `json:"id"` ID string `json:"id"`
DisplayName string `json:"display_name"` DisplayName string `json:"display_name"`
Institution string `json:"institution"` Institution string `json:"institution"`
Currency string `json:"currency"` Currency string `json:"currency"`
// Kind is "cash" or "investment". An investment account also holds
// positions, and its facts never reach the sign-based classification
// fallback.
Kind string `json:"kind,omitempty"`
ExternalAccountID string `json:"external_account_id,omitempty"` ExternalAccountID string `json:"external_account_id,omitempty"`
IBAN string `json:"iban,omitempty"` IBAN string `json:"iban,omitempty"`
// ReferenceIBAN is the counterpart this account settles cash against: a
// broker exports no counterparty column, so deposits and withdrawals carry
// this IBAN instead and pair with the funding account like any transfer.
ReferenceIBAN string `json:"reference_iban,omitempty"`
Active bool `json:"active"` Active bool `json:"active"`
} }
func (a Account) Investing() bool { return a.Kind == AccountInvestment }
// Investment events. Cash events move money only; buy, sell and reinvest move
// both money and position; corporate actions and position transfers move
// position only and must never touch cash.
const (
EventDeposit = "deposit"
EventWithdrawal = "withdrawal"
EventFee = "fee"
EventInterest = "interest"
EventDistribution = "distribution"
EventBuy = "buy"
EventSell = "sell"
EventReinvest = "reinvest"
EventCorporateAction = "corporate_action"
EventPositionTransfer = "position_transfer"
)
// Investment is the broker-native leg of an imported fact. Cash movement always
// stays in Facts.Amount, so a position-only event has a zero amount; Gross,
// Fee and Tax record the broker's own figures the amount was derived from.
//
// Quantity is signed: positive adds to the holding, negative removes it. The
// export signs corporate actions and position transfers in its share column but
// leaves buys and sells unsigned, so the sign is resolved at import, once.
type Investment struct {
Event string `json:"event"`
InstrumentID string `json:"instrument_id,omitempty"`
Quantity Quantity `json:"quantity,omitempty"`
Price Money `json:"price,omitempty"`
Gross Money `json:"gross,omitempty"`
Fee Money `json:"fee,omitempty"`
Tax Money `json:"tax,omitempty"`
}
// CashOnly reports an event that moves money without moving a position.
func (i Investment) CashOnly() bool {
switch i.Event {
case EventDeposit, EventWithdrawal, EventFee, EventInterest, EventDistribution:
return true
}
return false
}
// PositionOnly reports an event that moves a position without moving money.
func (i Investment) PositionOnly() bool {
return i.Event == EventCorporateAction || i.Event == EventPositionTransfer
}
// Settling reports an event that moves money and position together.
func (i Investment) Settling() bool {
switch i.Event {
case EventBuy, EventSell, EventReinvest:
return true
}
return false
}
// Instrument is a security held in an investment account, identified by ISIN.
// The broker's description for one ISIN changes over time, so Name is editable
// display text and never an identity.
type Instrument struct {
ID string `json:"id"`
ISIN string `json:"isin"`
Name string `json:"name"`
Currency string `json:"currency"`
}
type Facts struct { type Facts struct {
ID string `json:"id"` ID string `json:"id"`
Source string `json:"source"` Source string `json:"source"`
@@ -25,6 +114,9 @@ type Facts struct {
Fingerprint string `json:"fingerprint"` Fingerprint string `json:"fingerprint"`
Counterparty string `json:"counterparty,omitempty"` Counterparty string `json:"counterparty,omitempty"`
CounterpartyIBAN string `json:"counterparty_iban,omitempty"` CounterpartyIBAN string `json:"counterparty_iban,omitempty"`
// Investment is present exactly on facts imported from an investment
// account. It is bank fact data and therefore immutable.
Investment *Investment `json:"investment,omitempty"`
} }
type Provenance struct { type Provenance struct {
Source string `json:"source"` Source string `json:"source"`
@@ -67,8 +159,15 @@ type Dataset struct {
Categories []Category `json:"categories"` Categories []Category `json:"categories"`
Tags []Tag `json:"tags"` Tags []Tag `json:"tags"`
Merchants []Merchant `json:"merchants"` Merchants []Merchant `json:"merchants"`
Instruments []Instrument `json:"instruments"`
Transactions []Transaction `json:"transactions"` Transactions []Transaction `json:"transactions"`
} }
const ExpenseFallback = "cat_expenses_unclassified" const ExpenseFallback = "cat_expenses_unclassified"
const IncomeFallback = "cat_income_unclassified" const IncomeFallback = "cat_income_unclassified"
// KindInvestment is the enrichment kind for broker facts. Like a transfer it
// carries no category or merchant and never reaches spending analytics: money
// moving between your own cash and your own positions is not income or
// spending, and the AI must never see it.
const KindInvestment = "investment"
+15 -2
View File
@@ -16,6 +16,11 @@ import (
// Grammar: kind { on its own line, followed by field: JSON values, then }. // Grammar: kind { on its own line, followed by field: JSON values, then }.
// JSON values may span lines. Blank lines and full-line # or // comments are // JSON values may span lines. Blank lines and full-line # or // comments are
// permitted between fields and blocks. Strings use JSON escaping, including \n. // permitted between fields and blocks. Strings use JSON escaping, including \n.
// registryFiles are the non-monthly journal files, in the order they are read
// and written. A block's file is its kind pluralized, so this list and the
// kinds accepted by parseDocument must stay in step.
var registryFiles = []string{"accounts.finance", "categories.finance", "tags.finance", "merchants.finance", "instruments.finance"}
type fieldSpan struct{ start, end int } type fieldSpan struct{ start, end int }
type block struct { type block struct {
kind, id string kind, id string
@@ -135,7 +140,7 @@ func parseDocument(path string, raw []byte) (*document, error) {
} }
header := strings.Fields(trimmed) header := strings.Fields(trimmed)
if len(header) != 2 || header[1] != "{" { if len(header) != 2 || header[1] != "{" {
return fail(i+1, "expected 'account|category|tag|merchant|transaction {'") return fail(i+1, "expected 'account|category|tag|merchant|instrument|transaction {'")
} }
kind := header[0] kind := header[0]
var value any var value any
@@ -148,6 +153,8 @@ func parseDocument(path string, raw []byte) (*document, error) {
value = &domain.Tag{} value = &domain.Tag{}
case "merchant": case "merchant":
value = &domain.Merchant{} value = &domain.Merchant{}
case "instrument":
value = &domain.Instrument{}
case "transaction": case "transaction":
value = &domain.Transaction{} value = &domain.Transaction{}
default: default:
@@ -227,6 +234,9 @@ func parseDocument(path string, raw []byte) (*document, error) {
case *domain.Tag: case *domain.Tag:
b.id = v.ID b.id = v.ID
b.value = *v b.value = *v
case *domain.Instrument:
b.id = v.ID
b.value = *v
case *domain.Merchant: case *domain.Merchant:
if v.Aliases == nil { if v.Aliases == nil {
v.Aliases = []string{} v.Aliases = []string{}
@@ -308,7 +318,7 @@ func (b *block) render(value any) ([]byte, error) {
} }
func datasetFiles(d domain.Dataset) map[string]map[string]piece { func datasetFiles(d domain.Dataset) map[string]map[string]piece {
files := map[string]map[string]piece{} files := map[string]map[string]piece{}
for _, p := range []string{"accounts.finance", "categories.finance", "tags.finance", "merchants.finance"} { for _, p := range registryFiles {
files[p] = map[string]piece{} files[p] = map[string]piece{}
} }
add := func(path, kind, id string, value any) { add := func(path, kind, id string, value any) {
@@ -329,6 +339,9 @@ func datasetFiles(d domain.Dataset) map[string]map[string]piece {
for _, v := range d.Merchants { for _, v := range d.Merchants {
add("merchants.finance", "merchant", v.ID, v) add("merchants.finance", "merchant", v.ID, v)
} }
for _, v := range d.Instruments {
add("instruments.finance", "instrument", v.ID, v)
}
for _, v := range d.Transactions { for _, v := range d.Transactions {
month := v.Facts.BookingDate[:7] month := v.Facts.BookingDate[:7]
add("journal/"+month[:4]+"/"+month+".finance", "transaction", v.Facts.ID, v) add("journal/"+month[:4]+"/"+month+".finance", "transaction", v.Facts.ID, v)
+6 -4
View File
@@ -12,6 +12,7 @@ import (
"path/filepath" "path/filepath"
"reflect" "reflect"
"regexp" "regexp"
"slices"
"sort" "sort"
"strings" "strings"
"sync" "sync"
@@ -257,8 +258,7 @@ func revisionHashes(hashes map[string]string) string {
return hex.EncodeToString(h.Sum(nil)) return hex.EncodeToString(h.Sum(nil))
} }
func validPath(path string) bool { func validPath(path string) bool {
switch path { if slices.Contains(registryFiles, path) {
case "accounts.finance", "categories.finance", "tags.finance", "merchants.finance":
return true return true
} }
parts := monthlyPath.FindStringSubmatch(path) parts := monthlyPath.FindStringSubmatch(path)
@@ -363,7 +363,7 @@ func (s *Store) readFiles() (map[string][]byte, error) {
} }
} }
raw := map[string][]byte{} raw := map[string][]byte{}
for _, path := range []string{"accounts.finance", "categories.finance", "tags.finance", "merchants.finance"} { for _, path := range registryFiles {
b, err := readSecure(filepath.Join(s.dir, path)) b, err := readSecure(filepath.Join(s.dir, path))
if errors.Is(err, os.ErrNotExist) { if errors.Is(err, os.ErrNotExist) {
continue continue
@@ -435,7 +435,7 @@ func (s *Store) snapshot() (*snapshot, error) {
return snap, nil return snap, nil
} }
func decodeSnapshot(raw map[string][]byte) (*snapshot, error) { func decodeSnapshot(raw map[string][]byte) (*snapshot, error) {
snap := &snapshot{raw: raw, docs: map[string]*document{}, revision: revision(raw), data: domain.Dataset{Accounts: []domain.Account{}, Categories: []domain.Category{}, Tags: []domain.Tag{}, Merchants: []domain.Merchant{}, Transactions: []domain.Transaction{}}} snap := &snapshot{raw: raw, docs: map[string]*document{}, revision: revision(raw), data: domain.Dataset{Accounts: []domain.Account{}, Categories: []domain.Category{}, Tags: []domain.Tag{}, Merchants: []domain.Merchant{}, Instruments: []domain.Instrument{}, Transactions: []domain.Transaction{}}}
if len(raw) == 0 { if len(raw) == 0 {
snap.data = domain.NewDataset() snap.data = domain.NewDataset()
return snap, nil return snap, nil
@@ -481,6 +481,8 @@ func decodeSnapshot(raw map[string][]byte) (*snapshot, error) {
snap.data.Tags = append(snap.data.Tags, v) snap.data.Tags = append(snap.data.Tags, v)
case domain.Merchant: case domain.Merchant:
snap.data.Merchants = append(snap.data.Merchants, v) snap.data.Merchants = append(snap.data.Merchants, v)
case domain.Instrument:
snap.data.Instruments = append(snap.data.Instruments, v)
case domain.Transaction: case domain.Transaction:
snap.data.Transactions = append(snap.data.Transactions, v) snap.data.Transactions = append(snap.data.Transactions, v)
} }
+29
View File
@@ -38,10 +38,13 @@ func New(a *app.App, assets fs.FS, publicURL string) (http.Handler, error) {
} }
s.mux.HandleFunc("GET /api/state", s.state) s.mux.HandleFunc("GET /api/state", s.state)
s.mux.HandleFunc("GET /api/dashboard", s.dashboard) s.mux.HandleFunc("GET /api/dashboard", s.dashboard)
s.mux.HandleFunc("GET /api/wealth", func(w http.ResponseWriter, r *http.Request) { v, e := a.Wealth(r.Context()); respond(w, v, e) })
s.mux.HandleFunc("POST /api/accounts", s.account) s.mux.HandleFunc("POST /api/accounts", s.account)
s.mux.HandleFunc("POST /api/categories", s.category) s.mux.HandleFunc("POST /api/categories", s.category)
s.mux.HandleFunc("POST /api/tags", s.tag) s.mux.HandleFunc("POST /api/tags", s.tag)
s.mux.HandleFunc("POST /api/merchants", s.merchant) s.mux.HandleFunc("POST /api/merchants", s.merchant)
s.mux.HandleFunc("POST /api/instruments", s.instrument)
s.mux.HandleFunc("POST /api/transactions/{id}/transfer", s.transfer)
s.mux.HandleFunc("POST /api/transactions/{id}", s.transaction) s.mux.HandleFunc("POST /api/transactions/{id}", s.transaction)
s.mux.HandleFunc("POST /api/manage", s.manage) s.mux.HandleFunc("POST /api/manage", s.manage)
s.mux.HandleFunc("POST /api/import/prepare", s.importPrepare) s.mux.HandleFunc("POST /api/import/prepare", s.importPrepare)
@@ -274,6 +277,32 @@ func (s *Server) merchant(w http.ResponseWriter, r *http.Request) {
v, e := s.app.Mutate(r.Context(), b.Revision, func(d *domain.Dataset) error { return app.SaveMerchant(d, b.Merchant) }) v, e := s.app.Mutate(r.Context(), b.Revision, func(d *domain.Dataset) error { return app.SaveMerchant(d, b.Merchant) })
respond(w, v, e) respond(w, v, e)
} }
func (s *Server) instrument(w http.ResponseWriter, r *http.Request) {
var b struct {
Revision string `json:"revision"`
Instrument domain.Instrument `json:"instrument"`
}
if !decode(w, r, &b) {
return
}
v, e := s.app.Mutate(r.Context(), b.Revision, func(d *domain.Dataset) error { return app.SaveInstrument(d, b.Instrument) })
respond(w, v, e)
}
// transfer links or unlinks one transaction's own-account counterpart. It is a
// separate endpoint because both sides change together: the transaction editor
// cannot express it, and validation refuses a half-applied link.
func (s *Server) transfer(w http.ResponseWriter, r *http.Request) {
var b struct {
Revision string `json:"revision"`
PeerID string `json:"peer_id"`
}
if !decode(w, r, &b) {
return
}
v, e := s.app.LinkTransfer(r.Context(), b.Revision, r.PathValue("id"), b.PeerID)
respond(w, v, e)
}
func (s *Server) transaction(w http.ResponseWriter, r *http.Request) { func (s *Server) transaction(w http.ResponseWriter, r *http.Request) {
var b struct { var b struct {
Revision string `json:"revision"` Revision string `json:"revision"`
+147
View File
@@ -268,8 +268,14 @@ function AccountCard({
<h3>{account.display_name}</h3> <h3>{account.display_name}</h3>
<p> <p>
{account.institution} · {account.currency} {account.institution} · {account.currency}
{account.kind === "investment" ? " · Investment" : ""}
</p> </p>
{account.iban && <small className="account-iban">{account.iban}</small>} {account.iban && <small className="account-iban">{account.iban}</small>}
{account.reference_iban && (
<small className="account-iban">
Settles against {account.reference_iban}
</small>
)}
<div className="connection-status"> <div className="connection-status">
<span <span
className={`badge ${needsReconnect || connection?.status === "error" || connection?.status === "rate_limited" ? "connection-warning" : "neutral"}`} className={`badge ${needsReconnect || connection?.status === "error" || connection?.status === "rate_limited" ? "connection-warning" : "neutral"}`}
@@ -667,6 +673,13 @@ function ImportReview({
const [busy, setBusy] = useState(false); const [busy, setBusy] = useState(false);
const classifying = const classifying =
state.settings.classify_on_import && state.status.ai_configured; state.settings.classify_on_import && state.status.ai_configured;
const broker = prepared.broker;
// Broker figures are money in the account's own currency; the export carries
// no second currency and the samples are drawn from the same rows.
const currency =
state.data.accounts.find((a) => a.id === prepared.account_id)?.currency ||
prepared.samples[0]?.currency ||
"EUR";
const discard = () => { const discard = () => {
// Free the server's prepared statement; an expiring one is harmless. // Free the server's prepared statement; an expiring one is harmless.
void request("/api/import/cancel", { id: prepared.id }).catch(() => {}); void request("/api/import/cancel", { id: prepared.id }).catch(() => {});
@@ -716,6 +729,116 @@ function ImportReview({
))} ))}
</dl> </dl>
</details> </details>
{broker && (
<>
<div className="preview-summary">
<span>
<strong>{broker.instruments.length}</strong>{" "}
{broker.instruments.length === 1 ? "security" : "securities"} to
register
</span>
<span>
<strong>{broker.cancelled}</strong> cancelled{" "}
{broker.cancelled === 1 ? "row" : "rows"} skipped
</span>
<span>
<strong>{broker.rounded}</strong>{" "}
{broker.rounded === 1 ? "row" : "rows"} rounded
</span>
</div>
<details open>
<summary>Securities this import registers</summary>
{broker.instruments.length ? (
<dl className="facts">
{broker.instruments.map((instrument) => (
<div key={instrument.isin}>
<dt>{instrument.isin}</dt>
<dd>
{instrument.name} · {instrument.currency}
</dd>
</div>
))}
</dl>
) : (
<p className="muted small">
Every security this export names is already in your registry.
No new instrument is created.
</p>
)}
<p className="muted small">
A security is identified by its ISIN. An import never renames
one you already hold: the broker's description for an ISIN
changes over time, so the name stays yours to correct under
Instruments.
</p>
</details>
<p className="muted small">
{broker.cancelled
? `${broker.cancelled} ${broker.cancelled === 1 ? "row the broker did not execute is" : "rows the broker did not execute are"} skipped: a cancelled row's money and share columns are all zeros, so it would import as a phantom trade that every arithmetic check accepts.`
: "Every row in this export was executed; none were skipped."}
</p>
{broker.rounded > 0 && (
<p className="muted small">
{broker.rounded}{" "}
{broker.rounded === 1 ? "row carried" : "rows carried"} more
than four decimal places and{" "}
{broker.rounded === 1 ? "was" : "were"} rounded to the precision
the journal stores. The exact total adjustment across this
import is {broker.rounding} {currency}.
</p>
)}
{broker.unapplied.length > 0 && (
<details open>
<summary>
Fees and taxes recorded but not subtracted{" "}
<span className="badge neutral">
{broker.unapplied.length}
</span>
</summary>
<div className="table-scroll">
<table>
<thead>
<tr>
<th>Booking date</th>
<th>Description</th>
<th className="numeric">Fee</th>
<th className="numeric">Tax</th>
</tr>
</thead>
<tbody>
{broker.unapplied.map((note) => (
<tr key={note.record}>
<td className="nowrap">
{note.date}
<small>record {note.record}</small>
</td>
<td>
{note.description || (
<span className="muted">no description</span>
)}
</td>
<td className="numeric money">
{note.fee ? money(note.fee, currency) : "—"}
</td>
<td className="numeric money">
{note.tax ? money(note.tax, currency) : "—"}
</td>
</tr>
))}
</tbody>
</table>
</div>
<p className="muted small">
A broker cash amount is already net of its fee and tax, so
these figures are recorded on the transaction and deliberately
not subtracted a second time. Subtracting them again would
make your cash balance disagree with the broker's by exactly
these amounts.
</p>
</details>
)}
</>
)}
<div className="table-scroll"> <div className="table-scroll">
<table> <table>
<thead> <thead>
@@ -1139,6 +1262,7 @@ function AccountEditor({
display_name: value.display_name.trim(), display_name: value.display_name.trim(),
institution: value.institution.trim(), institution: value.institution.trim(),
iban: value.iban?.replaceAll(" ", ""), iban: value.iban?.replaceAll(" ", ""),
reference_iban: value.reference_iban?.replaceAll(" ", ""),
}, },
}, },
"Account saved", "Account saved",
@@ -1186,6 +1310,18 @@ function AccountEditor({
/> />
</Field> </Field>
</div> </div>
<Field
label="Account kind"
hint="An investment account holds securities. Its imported rows carry the broker ledger and stay out of spending and income analytics."
>
<select
value={value.kind || "cash"}
onChange={(e) => setValue({ ...value, kind: e.target.value })}
>
<option value="cash">Cash</option>
<option value="investment">Investment</option>
</select>
</Field>
<Field <Field
label="IBAN (optional)" label="IBAN (optional)"
hint="Used to recognize transfers between your own accounts." hint="Used to recognize transfers between your own accounts."
@@ -1195,6 +1331,17 @@ function AccountEditor({
onChange={(e) => setValue({ ...value, iban: e.target.value })} onChange={(e) => setValue({ ...value, iban: e.target.value })}
/> />
</Field> </Field>
<Field
label="Reference IBAN (optional)"
hint="The account this one settles cash against. A broker export names no counterparty, so this IBAN is what lets a deposit pair with the funding account instead of looking like income."
>
<input
value={value.reference_iban || ""}
onChange={(e) =>
setValue({ ...value, reference_iban: e.target.value })
}
/>
</Field>
<Field <Field
label="External account ID (optional)" label="External account ID (optional)"
hint="The provider account identifier used for connected-bank sync." hint="The provider account identifier used for connected-bank sync."
+81 -9
View File
@@ -7,9 +7,10 @@ import {
FolderTree, FolderTree,
Tag as TagIcon, Tag as TagIcon,
Store, Store,
CandlestickChart,
ChevronRight, ChevronRight,
} from "lucide-react"; } from "lucide-react";
import type { Category, Dataset, Merchant, Tag } from "./api"; import type { Category, Dataset, Instrument, Merchant, Tag } from "./api";
import { categoryPath } from "./api"; import { categoryPath } from "./api";
import { import {
CategoryOptions, CategoryOptions,
@@ -21,10 +22,21 @@ import {
TagPicker, TagPicker,
} from "./ui"; } from "./ui";
import type { Mutate } from "./ui"; import type { Mutate } from "./ui";
type Entity = "category" | "tag" | "merchant"; type Entity = "category" | "tag" | "merchant" | "instrument";
type Item = Category | Tag | Merchant; type Item = Category | Tag | Merchant | Instrument;
const titles = { category: "Categories", tag: "Tags", merchant: "Merchants" }; const titles = {
const plurals = { category: "categories", tag: "tags", merchant: "merchants" }; category: "Categories",
tag: "Tags",
merchant: "Merchants",
instrument: "Instruments",
};
const plurals = {
category: "categories",
tag: "tags",
merchant: "merchants",
instrument: "instruments",
};
type Plural = "categories" | "tags" | "merchants" | "instruments";
export function Registry({ export function Registry({
entity, entity,
data, data,
@@ -39,8 +51,7 @@ export function Registry({
item: Item; item: Item;
action: "merge" | "delete"; action: "merge" | "delete";
} | null>(null); } | null>(null);
const items: Item[] = const items: Item[] = data[plurals[entity] as Plural];
data[plurals[entity] as "categories" | "tags" | "merchants"];
const create = () => const create = () =>
setEditing( setEditing(
entity === "category" entity === "category"
@@ -53,6 +64,8 @@ export function Registry({
default_tag_ids: [], default_tag_ids: [],
use_defaults: false, use_defaults: false,
} }
: entity === "instrument"
? { id: "", isin: "", name: "", currency: "EUR" }
: { id: "", name: "" }, : { id: "", name: "" },
); );
const row = (item: Item, depth = 0) => ( const row = (item: Item, depth = 0) => (
@@ -65,6 +78,8 @@ export function Registry({
<FolderTree size={18} /> <FolderTree size={18} />
) : entity === "tag" ? ( ) : entity === "tag" ? (
<TagIcon size={18} /> <TagIcon size={18} />
) : entity === "instrument" ? (
<CandlestickChart size={18} />
) : ( ) : (
<Store size={18} /> <Store size={18} />
)} )}
@@ -82,6 +97,11 @@ export function Registry({
{item.use_defaults ? " · Defaults enabled" : ""} {item.use_defaults ? " · Defaults enabled" : ""}
</small> </small>
)} )}
{"isin" in item && (
<small>
{item.isin} · {item.currency}
</small>
)}
</div> </div>
</div> </div>
{"default_category_id" in item && item.default_category_id && ( {"default_category_id" in item && item.default_category_id && (
@@ -98,6 +118,7 @@ export function Registry({
> >
<Pencil size={16} /> <Pencil size={16} />
</button> </button>
{entity !== "instrument" && (
<button <button
className="icon-button" className="icon-button"
title={`Merge ${item.name}`} title={`Merge ${item.name}`}
@@ -106,6 +127,7 @@ export function Registry({
> >
<GitMerge size={16} /> <GitMerge size={16} />
</button> </button>
)}
<button <button
className="icon-button danger" className="icon-button danger"
title={`Delete ${item.name}`} title={`Delete ${item.name}`}
@@ -142,6 +164,8 @@ export function Registry({
? "A clear home for every transaction. Parent categories roll up their children." ? "A clear home for every transaction. Parent categories roll up their children."
: entity === "tag" : entity === "tag"
? "Flexible labels that work across your accounts and categories." ? "Flexible labels that work across your accounts and categories."
: entity === "instrument"
? "The securities your broker rows trade. The ISIN is the identity; the name is yours to correct."
: "Recognize familiar names and choose explicit classification defaults."} : "Recognize familiar names and choose explicit classification defaults."}
</p> </p>
</div> </div>
@@ -209,6 +233,9 @@ function RegistryEditor({
const [category, setCategory] = useState(merchant?.default_category_id || ""); const [category, setCategory] = useState(merchant?.default_category_id || "");
const [tags, setTags] = useState(merchant?.default_tag_ids || []); const [tags, setTags] = useState(merchant?.default_tag_ids || []);
const [defaults, setDefaults] = useState(merchant?.use_defaults || false); const [defaults, setDefaults] = useState(merchant?.use_defaults || false);
const instrument = "isin" in item ? item : null;
const [isin, setIsin] = useState(instrument?.isin || "");
const [currency, setCurrency] = useState(instrument?.currency || "EUR");
const [error, setError] = useState(""); const [error, setError] = useState("");
const [busy, setBusy] = useState(false); const [busy, setBusy] = useState(false);
const descendants = new Set([item.id]); const descendants = new Set([item.id]);
@@ -252,6 +279,13 @@ function RegistryEditor({
default_tag_ids: tags, default_tag_ids: tags,
use_defaults: defaults, use_defaults: defaults,
} }
: entity === "instrument"
? {
id: item.id,
isin: isin.replaceAll(" ", "").toUpperCase(),
name: name.trim(),
currency: currency.toUpperCase(),
}
: { id: item.id, name: name.trim() }; : { id: item.id, name: name.trim() };
await mutate( await mutate(
`/api/${plurals[entity]}`, `/api/${plurals[entity]}`,
@@ -347,6 +381,43 @@ function RegistryEditor({
</p> </p>
</> </>
)} )}
{entity === "instrument" && (
<>
<Field
label="ISIN"
hint={
item.id
? "An instrument's ISIN is its identity: the trades were imported under it and the server refuses to change it. Register a different security separately."
: "Twelve characters: two country letters, nine alphanumerics and a check digit."
}
>
<input
required
readOnly={!!item.id}
maxLength={12}
value={isin}
onChange={(e) => setIsin(e.target.value.toUpperCase())}
/>
</Field>
<Field
label="Currency"
hint="The currency the broker prices this security in."
>
<input
required
pattern="[A-Z]{3}"
maxLength={3}
value={currency}
onChange={(e) => setCurrency(e.target.value.toUpperCase())}
/>
</Field>
<p className="muted">
The broker's own description for one ISIN changes over time, so
the name is display text you can correct. Renaming does not
touch a single imported trade.
</p>
</>
)}
</div> </div>
<FormActions busy={busy} close={close} /> <FormActions busy={busy} close={close} />
</form> </form>
@@ -372,8 +443,7 @@ function ManageDialog({
const [confirm, setConfirm] = useState(false); const [confirm, setConfirm] = useState(false);
const [error, setError] = useState(""); const [error, setError] = useState("");
const [busy, setBusy] = useState(false); const [busy, setBusy] = useState(false);
const items: Item[] = const items: Item[] = data[plurals[entity] as Plural];
data[plurals[entity] as "categories" | "tags" | "merchants"];
return ( return (
<Modal <Modal
title={`${action === "merge" ? "Merge" : "Delete"} ${item.name}`} title={`${action === "merge" ? "Merge" : "Delete"} ${item.name}`}
@@ -407,6 +477,8 @@ function ManageDialog({
? "This tag will be removed from every transaction and merchant default. The original bank facts will not change." ? "This tag will be removed from every transaction and merchant default. The original bank facts will not change."
: entity === "category" : entity === "category"
? "Referenced categories need a replacement. Protected roots and unsafe tree changes cannot be deleted." ? "Referenced categories need a replacement. Protected roots and unsafe tree changes cannot be deleted."
: entity === "instrument"
? "Remove this security from your registry. An instrument any imported trade still references cannot be deleted: the server refuses it and says so."
: "Remove this merchant from your registry. Referenced merchants may require a merge instead."} : "Remove this merchant from your registry. Referenced merchants may require a merge instead."}
</p> </p>
{(action === "merge" || entity === "category") && ( {(action === "merge" || entity === "category") && (
+299 -19
View File
@@ -6,9 +6,16 @@ import {
ArrowLeftRight, ArrowLeftRight,
ChevronLeft, ChevronLeft,
ChevronRight, ChevronRight,
Layers,
SlidersHorizontal, SlidersHorizontal,
} from "lucide-react"; } from "lucide-react";
import type { Dataset, Enrichment, Filter, Transaction } from "./api"; import type {
Dataset,
Enrichment,
Filter,
Investment,
Transaction,
} from "./api";
import { categoryPath, money } from "./api"; import { categoryPath, money } from "./api";
import { import {
CategoryOptions, CategoryOptions,
@@ -21,6 +28,56 @@ import {
TagPicker, TagPicker,
} from "./ui"; } from "./ui";
import type { Mutate } from "./ui"; import type { Mutate } from "./ui";
const EVENTS: Record<string, string> = {
deposit: "Deposit",
withdrawal: "Withdrawal",
fee: "Fee",
interest: "Interest",
distribution: "Distribution",
buy: "Buy",
sell: "Sell",
reinvest: "Reinvestment",
corporate_action: "Corporate action",
position_transfer: "Position transfer",
};
// A corporate action or a position transfer moves shares between holdings and
// settles no money at all, so its zero amount is a fact and not a gap.
function positionOnly(investment?: Investment): boolean {
return (
investment?.event === "corporate_action" ||
investment?.event === "position_transfer"
);
}
// A broker cash movement settles money without moving a position, and it is
// the only broker row the server accepts as one side of a transfer.
function cashOnly(investment: Investment): boolean {
return ["deposit", "withdrawal", "fee", "interest", "distribution"].includes(
investment.event,
);
}
// Quantities are exact decimals, never money: they are shown as the broker
// wrote them, with the sign that says whether the holding grew or shrank.
function signedQuantity(quantity: string): string {
return quantity.startsWith("-") ? quantity : `+${quantity}`;
}
// Transfer candidates are compared as exact decimals: "10.00" and "10" are the
// same money, and a float round-trip is never allowed to decide a link.
function decimalKey(value: string): string {
const negative = value.startsWith("-");
const [whole, fraction = ""] = (negative ? value.slice(1) : value).split(".");
const digits = `${whole.replace(/^0+(?=\d)/, "")}.${fraction.replace(/0+$/, "")}`;
const body = digits.endsWith(".") ? digits.slice(0, -1) : digits;
return body === "0" ? "0" : `${negative ? "-" : ""}${body}`;
}
// Booking dates are calendar days, so the window is counted in whole days from
// the ISO string itself and no browser time zone can widen or narrow it.
function daysApart(a: string, b: string): number {
const left = Date.parse(`${a}T00:00:00Z`);
const right = Date.parse(`${b}T00:00:00Z`);
if (Number.isNaN(left) || Number.isNaN(right))
return Number.POSITIVE_INFINITY;
return Math.abs(left - right) / 86400000;
}
export function Transactions({ export function Transactions({
data, data,
filter, filter,
@@ -128,6 +185,11 @@ export function Transactions({
.slice(currentPage * 40, currentPage * 40 + 40) .slice(currentPage * 40, currentPage * 40 + 40)
.map((tx) => { .map((tx) => {
const { facts: f, enrichment: e } = tx; const { facts: f, enrichment: e } = tx;
const investment = f.investment;
const moves = positionOnly(investment);
const security = data.instruments.find(
(i) => i.id === investment?.instrument_id,
);
return ( return (
<tr key={f.id}> <tr key={f.id}>
<td> <td>
@@ -143,7 +205,9 @@ export function Transactions({
onClick={() => setEditing(tx)} onClick={() => setEditing(tx)}
> >
<span className={`transaction-icon ${e.kind}`}> <span className={`transaction-icon ${e.kind}`}>
{e.kind === "transfer" ? ( {moves ? (
<Layers size={17} />
) : e.kind === "transfer" ? (
<ArrowLeftRight size={17} /> <ArrowLeftRight size={17} />
) : f.amount.startsWith("-") ? ( ) : f.amount.startsWith("-") ? (
<ArrowUpRight size={17} /> <ArrowUpRight size={17} />
@@ -157,11 +221,29 @@ export function Transactions({
(m) => m.id === e.merchant_id, (m) => m.id === e.merchant_id,
)?.name || )?.name ||
f.counterparty || f.counterparty ||
"Bank transaction"} (investment
? security?.name || f.raw_description
: "Bank transaction")}
</strong> </strong>
{(!investment ||
f.raw_description !== security?.name) && (
<small className="description"> <small className="description">
{f.raw_description} {f.raw_description}
</small> </small>
)}
{investment && (
<small className="description">
{EVENTS[investment.event] ||
investment.event}
{investment.quantity
? ` · ${signedQuantity(investment.quantity)} shares`
: ""}
{investment.price
? ` @ ${investment.price} ${f.currency}`
: ""}
{moves ? " · position only, no cash" : ""}
</small>
)}
</span> </span>
</button> </button>
</td> </td>
@@ -169,6 +251,8 @@ export function Transactions({
<span> <span>
{e.kind === "transfer" {e.kind === "transfer"
? "Own-account transfer" ? "Own-account transfer"
: e.kind === "investment"
? "Investment ledger"
: categoryPath(data, e.category_id)} : categoryPath(data, e.category_id)}
</span> </span>
<div className="chips"> <div className="chips">
@@ -270,9 +354,10 @@ function TransactionEditor({
}); });
const [error, setError] = useState(""); const [error, setError] = useState("");
const [busy, setBusy] = useState(false); const [busy, setBusy] = useState(false);
const { investment, ...plain } = transaction.facts;
const f = transaction.facts; const f = transaction.facts;
const peer = data.transactions.find( const instrument = data.instruments.find(
(t) => t.facts.id === value.transfer_peer_id, (i) => i.id === investment?.instrument_id,
); );
return ( return (
<Modal title="Transaction details" close={close} wide> <Modal title="Transaction details" close={close} wide>
@@ -315,10 +400,22 @@ function TransactionEditor({
<div className="two-columns"> <div className="two-columns">
<Field <Field
label="Kind" label="Kind"
hint="Derived from the bank amount and verified transfer links." hint="Derived from the bank amount, the broker ledger and verified transfer links."
> >
<input value={value.kind} readOnly /> <input value={value.kind} readOnly />
</Field> </Field>
{value.kind === "transfer" || value.kind === "investment" ? (
<Field
label="Merchant"
hint={
value.kind === "investment"
? "An investment ledger row carries no merchant and no category: it never reaches spending or income analytics."
: "An own-account transfer carries no merchant and no category: it never reaches spending or income analytics."
}
>
<input value="Not applicable" readOnly />
</Field>
) : (
<Field label="Merchant"> <Field label="Merchant">
<select <select
value={value.merchant_id || ""} value={value.merchant_id || ""}
@@ -334,19 +431,9 @@ function TransactionEditor({
))} ))}
</select> </select>
</Field> </Field>
)}
</div> </div>
{value.kind === "transfer" ? ( {value.kind !== "transfer" && value.kind !== "investment" && (
<Field label="Linked opposite transaction">
<input
readOnly
value={
peer
? `${peer.facts.booking_date} · ${money(peer.facts.amount, peer.facts.currency)} · ${peer.facts.raw_description}`
: value.transfer_peer_id || "No counterpart supplied"
}
/>
</Field>
) : (
<Field label="Category"> <Field label="Category">
<select <select
required required
@@ -360,6 +447,12 @@ function TransactionEditor({
</select> </select>
</Field> </Field>
)} )}
<TransferLink
data={data}
transaction={transaction}
mutate={mutate}
close={close}
/>
<TagPicker <TagPicker
data={data} data={data}
value={value.tag_ids} value={value.tag_ids}
@@ -371,7 +464,7 @@ function TransactionEditor({
<span className="badge neutral">Read only</span> <span className="badge neutral">Read only</span>
</summary> </summary>
<dl className="facts"> <dl className="facts">
{Object.entries(f).map(([key, text]) => ( {Object.entries(plain).map(([key, text]) => (
<div key={key}> <div key={key}>
<dt>{key.replaceAll("_", " ")}</dt> <dt>{key.replaceAll("_", " ")}</dt>
<dd>{text || "—"}</dd> <dd>{text || "—"}</dd>
@@ -379,6 +472,71 @@ function TransactionEditor({
))} ))}
</dl> </dl>
</details> </details>
{investment && (
<details open>
<summary>
Broker ledger <span className="badge neutral">Read only</span>
</summary>
<dl className="facts">
<div>
<dt>event</dt>
<dd>{EVENTS[investment.event] || investment.event}</dd>
</div>
<div>
<dt>instrument</dt>
<dd>
{investment.instrument_id
? `${instrument?.name || investment.instrument_id}${instrument ? ` · ${instrument.isin}` : ""}`
: "—"}
</dd>
</div>
<div>
<dt>quantity</dt>
<dd>
{investment.quantity
? `${signedQuantity(investment.quantity)} shares`
: "—"}
</dd>
</div>
<div>
<dt>price</dt>
<dd>
{investment.price
? money(
investment.price,
instrument?.currency || f.currency,
)
: "—"}
</dd>
</div>
<div>
<dt>gross</dt>
<dd>
{investment.gross
? money(investment.gross, f.currency)
: "—"}
</dd>
</div>
<div>
<dt>fee</dt>
<dd>
{investment.fee ? money(investment.fee, f.currency) : "—"}
</dd>
</div>
<div>
<dt>tax</dt>
<dd>
{investment.tax ? money(investment.tax, f.currency) : "—"}
</dd>
</div>
</dl>
<p className="muted small">
{positionOnly(investment)
? "A corporate action and a position transfer move shares only: the amount above is zero because no cash settled."
: "The amount above is the broker's own cash figure, already net of any fee and tax shown here. Those figures are recorded, never subtracted a second time."}
</p>
</details>
)}
<details open> <details open>
<summary>Classification provenance</summary> <summary>Classification provenance</summary>
<dl className="facts"> <dl className="facts">
@@ -402,3 +560,125 @@ function TransactionEditor({
</Modal> </Modal>
); );
} }
// TransferLink is its own control because a transfer is a decision about two
// transactions: the plain transaction endpoint refuses a changed kind or peer,
// and the server rewrites both sides of the old and the new pair in one commit.
function TransferLink({
data,
transaction,
mutate,
close,
}: {
data: Dataset;
transaction: Transaction;
mutate: Mutate;
close: () => void;
}) {
const f = transaction.facts;
const linked = transaction.enrichment.transfer_peer_id || "";
const [peerId, setPeerId] = useState(linked);
const [error, setError] = useState("");
const [busy, setBusy] = useState(false);
const candidates = useMemo(() => {
const wanted = decimalKey(
f.amount.startsWith("-") ? f.amount.slice(1) : `-${f.amount}`,
);
return data.transactions
.filter(
({ facts: c }) =>
c.id !== f.id &&
(c.id === linked ||
(c.account_id !== f.account_id &&
c.currency === f.currency &&
decimalKey(c.amount) === wanted &&
daysApart(c.booking_date, f.booking_date) <= 3)) &&
// Only a broker cash movement can be a transfer: a trade or a
// position move is refused by the server, so it is never offered.
(!c.investment || cashOnly(c.investment)),
)
.sort((a, b) => a.facts.booking_date.localeCompare(b.facts.booking_date));
}, [data, f, linked]);
if (f.investment && !cashOnly(f.investment))
return (
<Field
label="Own-account counterpart"
hint="Only a broker cash movement — a deposit, withdrawal, fee, interest or distribution — can be paired with the other side of the transfer."
>
<input
readOnly
value={`A ${(EVENTS[f.investment.event] || f.investment.event).toLowerCase()} row stays in the investment ledger.`}
/>
</Field>
);
return (
<>
<Field
label="Own-account counterpart"
hint="Linking rewrites both sides in one commit: each becomes an own-account transfer and leaves spending and income analytics. Candidates are the exactly opposite amount, in the same currency, in another account, booked within three days."
>
<select
value={peerId}
disabled={busy || (!candidates.length && !linked)}
onChange={(event) => setPeerId(event.target.value)}
>
<option value="">Not a transfer no counterpart</option>
{candidates.map(({ facts: c, enrichment: e }) => (
<option key={c.id} value={c.id}>
{`${data.accounts.find((a) => a.id === c.account_id)?.display_name || c.account_id} · ${c.booking_date} · ${money(c.amount, c.currency)}`}
{e.transfer_peer_id && e.transfer_peer_id !== f.id
? " · already linked elsewhere"
: ""}
</option>
))}
</select>
</Field>
{!candidates.length && !linked && (
<p className="muted small">
No transaction in another account carries exactly{" "}
{money(
f.amount.startsWith("-") ? f.amount.slice(1) : `-${f.amount}`,
f.currency,
)}{" "}
within three days of {f.booking_date}.
</p>
)}
<ErrorMessage error={error} />
<div className="form-actions">
<button
type="button"
className={
!peerId && linked ? "button destructive" : "button secondary"
}
disabled={busy || peerId === linked}
onClick={async () => {
setBusy(true);
setError("");
try {
await mutate(
`/api/transactions/${encodeURIComponent(f.id)}/transfer`,
{ peer_id: peerId },
peerId ? "Transfer linked" : "Transfer link removed",
);
close();
} catch (err) {
setError(err instanceof Error ? err.message : String(err));
setBusy(false);
}
}}
>
<ArrowLeftRight size={16} />
{busy
? "Working…"
: !peerId && linked
? "Remove transfer link"
: peerId === linked
? "Counterpart linked"
: linked
? "Relink counterpart"
: "Link as own-account transfer"}
</button>
</div>
</>
);
}
+309
View File
@@ -0,0 +1,309 @@
import { useEffect, useState } from "react";
import {
AlertTriangle,
CandlestickChart,
CheckCircle2,
Landmark,
PiggyBank,
} from "lucide-react";
import type { Wealth, WealthAccount } from "./api";
import { money, request } from "./api";
import { Empty, ErrorMessage } from "./ui";
// The report is recomputed from the journal, so it is keyed on the revision and
// never cached: it exists to be compared with a bank or broker's own screen.
// Renaming a security lives in the Instruments registry, beside every other
// registry entity, rather than being a second editor here.
export default function WealthPage({ revision }: { revision: string }) {
const [wealth, setWealth] = useState<Wealth | null>(null);
const [error, setError] = useState("");
const [loading, setLoading] = useState(true);
const [retry, setRetry] = useState(0);
useEffect(() => {
const controller = new AbortController();
setLoading(true);
setError("");
request<Wealth>("/api/wealth", undefined, controller.signal)
.then((value) => {
for (const key of ["accounts", "totals"] as const) {
if (!(key in value))
throw new Error(`Wealth response is missing ${key}.`);
if (value[key] === null) Object.assign(value, { [key]: [] });
}
for (const account of value.accounts) {
account.holdings ??= [];
account.checks ??= [];
}
setWealth(value);
})
.catch((err) => {
if (!controller.signal.aborted) {
setError(err instanceof Error ? err.message : String(err));
setWealth(null);
}
})
.finally(() => {
if (!controller.signal.aborted) setLoading(false);
});
return () => controller.abort();
}, [revision, retry]);
const failures =
wealth?.accounts.reduce(
(total, account) => total + account.checks.filter((c) => c.failed).length,
0,
) || 0;
const failingAccounts =
wealth?.accounts.filter((account) => account.checks.some((c) => c.failed))
.length || 0;
return (
<>
<div className="section-heading">
<div>
<h2>Wealth</h2>
<p>
Cash and positions recomputed from your journal, with the checks
that decide whether the figures can be trusted.
</p>
</div>
<button
className="button secondary"
onClick={() => setRetry(retry + 1)}
disabled={loading}
>
Recheck figures
</button>
</div>
<ErrorMessage error={error} />
{loading ? (
<div className="loading-block" role="status">
<span className="spinner" />
Recomputing cash and positions
</div>
) : (
wealth && (
<>
{failures > 0 && (
<div className="alert error" role="alert">
<AlertTriangle size={19} />
<div>
<strong>
{failures} check{failures === 1 ? "" : "s"} failed across{" "}
{failingAccounts} account
{failingAccounts === 1 ? "" : "s"}.
</strong>
<p>
A failed check means the journal disagrees with itself, so
the balance below will not match your bank or broker. The
details sit with the account that failed.
</p>
</div>
</div>
)}
{wealth.totals.length > 0 && (
<section className="panel">
<div className="panel-heading">
<div>
<h3>
<PiggyBank size={17} />
Total cash
</h3>
<p>
Every recorded movement summed per currency, across all{" "}
{wealth.accounts.length} account
{wealth.accounts.length === 1 ? "" : "s"}.
</p>
</div>
</div>
<div className="registry">
<div className="preview-summary">
{wealth.totals.map((total) => (
<span key={total.currency}>
<strong className="money">
{money(total.cash, total.currency)}
</strong>{" "}
in cash
</span>
))}
</div>
</div>
</section>
)}
{wealth.accounts.length === 0 ? (
<section className="panel">
<Empty title="No accounts to report on yet">
Add an account and import a statement or broker export to see
its cash balance, positions and checks here.
</Empty>
</section>
) : (
wealth.accounts.map((account) => (
<AccountReport key={account.account_id} account={account} />
))
)}
<section className="panel">
<div className="panel-heading">
<div>
<h3>Reading these figures</h3>
<p>
The three rules that decide what a broker export does and
does not move.
</p>
</div>
</div>
<div className="registry">
<dl className="facts">
<div>
<dt>Tax on broker cash</dt>
<dd>
A broker cash amount is already net of tax. The tax is
recorded on the transaction and deliberately not
subtracted a second time.
</dd>
</div>
<div>
<dt>Position-only events</dt>
<dd>
Corporate actions and position transfers move a position
and settle zero cash, so they change a holding without
touching the balance.
</dd>
</div>
<div>
<dt>Investment transactions</dt>
<dd>
Transactions classified as investment are excluded from
every spending and income figure, exactly like transfers.
</dd>
</div>
<div>
<dt>Completeness</dt>
<dd>
Cash equals the real balance only when the journal holds
that account's full history: a broker export does, a
date-windowed bank statement does not.
</dd>
</div>
</dl>
</div>
</section>
</>
)
)}
</>
);
}
function AccountReport({ account }: { account: WealthAccount }) {
const range =
account.first_booking && account.last_booking
? `${account.first_booking} ${account.last_booking}`
: account.first_booking || account.last_booking || "";
const failed = account.checks.filter((check) => check.failed);
const notes = account.checks.filter((check) => !check.failed);
const investing = account.kind === "investment";
return (
<section className="panel">
<div className="panel-heading">
<div>
<h3>
{investing ? (
<CandlestickChart size={17} />
) : (
<Landmark size={17} />
)}
{account.display_name}
</h3>
<p>
{account.institution} · {investing ? "Investment" : "Cash"} account
· {account.records} record{account.records === 1 ? "" : "s"}
{range ? ` · ${range}` : " · no bookings"}
{!account.active && " · archived"}
</p>
</div>
<div>
<span className="eyebrow">Cash balance</span>
<span className="large-money money">
{money(account.cash, account.currency)}
</span>
</div>
</div>
{account.holdings.length > 0 && (
<div className="table-scroll">
<table>
<thead>
<tr>
<th>Instrument</th>
<th>ISIN</th>
<th className="numeric">Quantity</th>
<th className="numeric">Invested</th>
<th className="numeric">Received</th>
<th className="numeric">Records</th>
</tr>
</thead>
<tbody>
{account.holdings.map((holding) => {
// A quantity is an exact decimal string and stays one: the sign
// is its first character and a digit above zero is what makes
// the position non-empty, with no number parsing in between.
// A negative holding means more units left the account than
// entered it, which is always worth seeing.
const negative = holding.quantity.startsWith("-");
const empty = !/[1-9]/.test(holding.quantity);
return (
<tr key={holding.instrument_id}>
<td>{holding.name}</td>
<td className="nowrap muted">{holding.isin}</td>
<td
className={`numeric money ${negative ? "text-danger" : empty ? "muted" : "positive"}`}
>
{holding.quantity}
{negative && (
<small className="text-danger">
more units left than entered
</small>
)}
</td>
<td className="numeric money">
{money(holding.invested, account.currency)}
</td>
<td className="numeric money">
{money(holding.received, account.currency)}
</td>
<td className="numeric">{holding.records}</td>
</tr>
);
})}
</tbody>
</table>
</div>
)}
{failed.length > 0 && (
<div className="registry">
{failed.map((check) => (
<div className="alert error" role="alert" key={check.name}>
<AlertTriangle size={19} />
<div>
<strong>{check.name}</strong>
<p>{check.detail}</p>
</div>
</div>
))}
</div>
)}
{notes.length > 0 && (
<div className="health-grid">
{notes.map((check) => (
<div className="health" key={check.name}>
<span className="positive">
<CheckCircle2 size={18} />
</span>
<div>
<strong>{check.name}</strong>
<p>{check.detail}</p>
</div>
</div>
))}
</div>
)}
</section>
);
}
+95
View File
@@ -3,10 +3,36 @@ export interface Account {
display_name: string; display_name: string;
institution: string; institution: string;
currency: string; currency: string;
// kind is "cash" or "investment"; an absent kind is a cash account.
kind?: string;
external_account_id?: string; external_account_id?: string;
iban?: string; iban?: string;
// reference_iban is the counterpart an investment account settles cash
// against: a broker export carries no counterparty, so its deposits and
// withdrawals pair with the funding account through this IBAN.
reference_iban?: string;
active: boolean; active: boolean;
} }
// Instrument is a security held in an investment account. The ISIN is the
// identity; the name is editable display text.
export interface Instrument {
id: string;
isin: string;
name: string;
currency: string;
}
// Investment is the broker-native leg of a fact. Cash movement always stays in
// Facts.amount, so a position-only event carries a zero amount. Quantity is an
// exact signed decimal, not money: negative removes from the holding.
export interface Investment {
event: string;
instrument_id?: string;
quantity?: string;
price?: string;
gross?: string;
fee?: string;
tax?: string;
}
export interface Facts { export interface Facts {
id: string; id: string;
source: string; source: string;
@@ -20,6 +46,7 @@ export interface Facts {
fingerprint: string; fingerprint: string;
counterparty?: string; counterparty?: string;
counterparty_iban?: string; counterparty_iban?: string;
investment?: Investment;
} }
export interface Provenance { export interface Provenance {
source: string; source: string;
@@ -62,6 +89,7 @@ export interface Dataset {
categories: Category[]; categories: Category[];
tags: Tag[]; tags: Tag[];
merchants: Merchant[]; merchants: Merchant[];
instruments: Instrument[];
transactions: Transaction[]; transactions: Transaction[];
} }
export interface Connection { export interface Connection {
@@ -161,6 +189,27 @@ export interface CSVColumn {
field: string; field: string;
column: string; column: string;
} }
// BrokerReview is what the broker parser decided about an export that has not
// been imported yet: which securities it would register, which rows it skipped
// and which figures it deliberately did not apply.
export interface BrokerReview {
instruments: Instrument[];
// cancelled counts rows the broker did not execute.
cancelled: number;
// rounded counts rows whose money carried more than four decimal places;
// rounding is the exact total adjustment, to eight places.
rounded: number;
rounding: string;
// unapplied lists cash rows carrying a fee or tax. A broker cash amount is
// already net of both, so subtracting them again would double-count.
unapplied: {
record: number;
date: string;
description: string;
fee?: string;
tax?: string;
}[];
}
// PreparedImport is a parsed statement that has not been imported yet: the // PreparedImport is a parsed statement that has not been imported yet: the
// mapping and sample must be confirmed before any transaction is written. // mapping and sample must be confirmed before any transaction is written.
export interface PreparedImport { export interface PreparedImport {
@@ -176,6 +225,51 @@ export interface PreparedImport {
new: number; new: number;
duplicates: number; duplicates: number;
samples: Facts[]; samples: Facts[];
broker?: BrokerReview;
}
// WealthHolding is one instrument's position in one account. quantity is an
// exact signed decimal and never money; invested and received are money.
export interface WealthHolding {
instrument_id: string;
isin: string;
name: string;
quantity: string;
invested: string;
received: string;
records: number;
}
// WealthCheck is one named verification with its evidence. failed marks a
// disagreement inside the journal; the rest are notes that explain a figure.
export interface WealthCheck {
name: string;
detail: string;
failed: boolean;
}
export interface WealthAccount {
account_id: string;
display_name: string;
institution: string;
currency: string;
kind: string;
active: boolean;
records: number;
first_booking?: string;
last_booking?: string;
// cash is every recorded movement summed. It equals the real balance only
// when the journal holds that account's complete history.
cash: string;
holdings: WealthHolding[];
checks: WealthCheck[];
}
export interface WealthTotal {
currency: string;
cash: string;
}
// Wealth is a reconciliation report computed from the journal rather than the
// analytics index, so it can be checked against a bank or broker's own screen.
export interface Wealth {
accounts: WealthAccount[];
totals: WealthTotal[];
} }
export class APIError extends Error { export class APIError extends Error {
constructor( constructor(
@@ -244,6 +338,7 @@ export function normalizeState(state: State): State {
"categories", "categories",
"tags", "tags",
"merchants", "merchants",
"instruments",
"transactions", "transactions",
] as const) { ] as const) {
if (!(key in state.data)) if (!(key in state.data))
+16 -2
View File
@@ -6,7 +6,9 @@ import {
FolderTree, FolderTree,
Tags, Tags,
Store, Store,
CandlestickChart,
Wallet, Wallet,
PiggyBank,
Sparkles, Sparkles,
Settings as SettingsIcon, Settings as SettingsIcon,
RefreshCw, RefreshCw,
@@ -30,6 +32,7 @@ import { Registry } from "./Registry";
import { Accounts } from "./Accounts"; import { Accounts } from "./Accounts";
import { Classification } from "./Classification"; import { Classification } from "./Classification";
import { Settings } from "./Settings"; import { Settings } from "./Settings";
import Wealth from "./Wealth";
import { ErrorMessage } from "./ui"; import { ErrorMessage } from "./ui";
// Montserrat carries the wordmark. The subsets are bundled rather than fetched // Montserrat carries the wordmark. The subsets are bundled rather than fetched
// from Google Fonts: the Content-Security-Policy serves fonts from 'self' only, // from Google Fonts: the Content-Security-Policy serves fonts from 'self' only,
@@ -43,7 +46,9 @@ const navigation = [
{ id: "categories", label: "Categories", icon: FolderTree }, { id: "categories", label: "Categories", icon: FolderTree },
{ id: "tags", label: "Tags", icon: Tags }, { id: "tags", label: "Tags", icon: Tags },
{ id: "merchants", label: "Merchants", icon: Store }, { id: "merchants", label: "Merchants", icon: Store },
{ id: "instruments", label: "Instruments", icon: CandlestickChart },
{ id: "accounts", label: "Accounts", icon: Wallet }, { id: "accounts", label: "Accounts", icon: Wallet },
{ id: "wealth", label: "Wealth", icon: PiggyBank },
{ id: "classification", label: "AI classification", icon: Sparkles }, { id: "classification", label: "AI classification", icon: Sparkles },
{ id: "settings", label: "Settings", icon: SettingsIcon }, { id: "settings", label: "Settings", icon: SettingsIcon },
]; ];
@@ -178,10 +183,10 @@ function App() {
</a> </a>
<span className="nav-label">WORKSPACE</span> <span className="nav-label">WORKSPACE</span>
<nav aria-label="Main navigation"> <nav aria-label="Main navigation">
{navigation.map(({ id, label, icon: Icon }, i) => ( {navigation.map(({ id, label, icon: Icon }) => (
<button <button
key={id} key={id}
className={`nav-item ${page === id ? "active" : ""} ${i === 7 ? "nav-settings" : ""}`} className={`nav-item ${page === id ? "active" : ""} ${id === "settings" ? "nav-settings" : ""}`}
aria-current={page === id ? "page" : undefined} aria-current={page === id ? "page" : undefined}
onClick={() => navigate(id)} onClick={() => navigate(id)}
> >
@@ -378,6 +383,14 @@ function App() {
mutate={mutate} mutate={mutate}
/> />
)} )}
{page === "instruments" && (
<Registry
key={`instruments-${state.revision}`}
entity="instrument"
data={state.data}
mutate={mutate}
/>
)}
{page === "accounts" && ( {page === "accounts" && (
<Accounts <Accounts
state={state} state={state}
@@ -385,6 +398,7 @@ function App() {
acceptState={acceptState} acceptState={acceptState}
/> />
)} )}
{page === "wealth" && <Wealth revision={state.revision} />}
{page === "classification" && ( {page === "classification" && (
<Classification state={state} acceptState={acceptState} /> <Classification state={state} acceptState={acceptState} />
)} )}