Track investments as broker facts with a position leg
An account now has a kind, and an investment account holds positions as well as
cash. A broker row is not a new entity: it is a bank fact with an optional
position leg, so deduplication, the journal, fact immutability, the DuckDB
projection and the transactions view carry it unchanged. Facts.Amount stays the
cash leg and is zero on the rows that move only a position.
Scalable Capital exports are recognized locally as a fourth format, read by
their own parser because a column mapping cannot describe them: the amount
column is settled cash on a cash row, a gross to be netted on a trade, and a
position valuation that must never touch cash on a corporate action or a depot
transfer. A cash amount is already net of the tax the broker withheld or
refunded, so that tax is recorded on the fact and never subtracted a second
time; treating a corporate action's valuation as money conjures cash, and a
depot switch would do it once per instrument. The share column is signed only
for those two types, so buys and sells take their direction from the type. Every
security row is checked against shares times price at 128-bit width, because a
lost decimal separator survives every other check. An unknown status, type or
assetType, a foreign currency, a missing ISIN, or one failed check rejects the
whole file with the record number.
Instruments live in instruments.finance, keyed by ISIN with an ID derived from
it, so re-importing never registers a security twice. One ISIN appears under
several broker descriptions over the years and sometimes under the ISIN itself:
the most recent real description names it, and an import never renames one that
already exists. A broker also reuses a single reference across every leg of one
event, so transaction identity includes the event and its instrument.
domain.Fallback returns kind "investment" for any fact carrying a position leg,
so no broker row reaches the sign-based branch. That single rule is what stops
an unmatched deposit from counting as income and a broker fee from counting as
household spending; the monthly PRIME fee and its matching credit now cancel in
clearing:investments with no configuration at all. Investment rows are excluded
from spending analytics, from bulk reclassification and from the model, exactly
as transfers are.
Equal competing transfers are paired instead of skipped. Every connected
component of the candidate graph is a complete bipartite graph between two fixed
accounts at one amount and currency, so every pairing produces the same
accounts, kinds and postings and only the displayed counterpart differs.
Refusing to choose was the expensive option: both legs fell through to the
sign-based fallback and appeared as spending and income that never happened.
Pairing follows the nearest booking date, then the transaction ID, so iteration
order decides nothing. POST /api/transactions/{id}/transfer rewrites the old and
the new pair in one commit, because reciprocity is validated and a half-applied
link is an invalid dataset, and the matcher now skips any record classified
manually so a hand-made link or unlink outlives the next import.
Wealth reports each account's cash and positions from the journal rather than
the index, with named checks - row arithmetic, cash never negative, holdings
never negative - because it exists to be compared against the figures a broker
shows on its own screen. A negative holding means the imported history is
partial. Share counts are exact to eight places; a reinvested distribution
quoted to six is rounded to money's four and the residue is reported rather than
hidden. Market prices, market value, net worth over time, FIFO lot accounting,
realised gains and currency conversion are deliberately absent.
This commit is contained in:
Lars Nolden
@@ -0,0 +1,259 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"slices"
|
||||
"strings"
|
||||
|
||||
"finance-duck/internal/domain"
|
||||
)
|
||||
|
||||
// Wealth is a reconciliation report, computed from the journal rather than from
|
||||
// the DuckDB index: it exists to be checked against the figures a bank or
|
||||
// broker shows on its own screen, so it must not depend on the cache that the
|
||||
// same journal derives.
|
||||
type Wealth struct {
|
||||
Accounts []WealthAccount `json:"accounts"`
|
||||
// Totals is cash summed per currency across every account.
|
||||
Totals []WealthTotal `json:"totals"`
|
||||
}
|
||||
|
||||
type WealthTotal struct {
|
||||
Currency string `json:"currency"`
|
||||
Cash domain.Money `json:"cash"`
|
||||
}
|
||||
|
||||
// WealthAccount is one account's position as the journal records it.
|
||||
type WealthAccount struct {
|
||||
AccountID string `json:"account_id"`
|
||||
DisplayName string `json:"display_name"`
|
||||
Institution string `json:"institution"`
|
||||
Currency string `json:"currency"`
|
||||
Kind string `json:"kind"`
|
||||
Active bool `json:"active"`
|
||||
Records int `json:"records"`
|
||||
FirstBooking string `json:"first_booking,omitempty"`
|
||||
LastBooking string `json:"last_booking,omitempty"`
|
||||
// Cash is every recorded movement summed. It equals the account's real
|
||||
// balance only when the journal holds that account's complete history,
|
||||
// which a broker export does and a date-windowed bank statement does not.
|
||||
Cash domain.Money `json:"cash"`
|
||||
Holdings []WealthHolding `json:"holdings"`
|
||||
Checks []WealthCheck `json:"checks"`
|
||||
}
|
||||
|
||||
// WealthHolding is one instrument's position in one account.
|
||||
type WealthHolding struct {
|
||||
InstrumentID string `json:"instrument_id"`
|
||||
ISIN string `json:"isin"`
|
||||
Name string `json:"name"`
|
||||
Quantity domain.Quantity `json:"quantity"`
|
||||
// Invested is cash paid in less cash taken out through trades. It is not a
|
||||
// cost basis: a depot transfer moves a position with no cash at all, and a
|
||||
// sale returns cash without identifying which lot it closed.
|
||||
Invested domain.Money `json:"invested"`
|
||||
// Received is cash this instrument paid out without moving the position:
|
||||
// distributions, and the cash side of a corporate action.
|
||||
Received domain.Money `json:"received"`
|
||||
Records int `json:"records"`
|
||||
}
|
||||
|
||||
// WealthCheck is one named verification with its evidence. Failed marks a
|
||||
// disagreement inside the journal; the rest are notes that explain a figure
|
||||
// before it is compared with a broker's screen.
|
||||
type WealthCheck struct {
|
||||
Name string `json:"name"`
|
||||
Detail string `json:"detail"`
|
||||
Failed bool `json:"failed"`
|
||||
}
|
||||
|
||||
// Wealth reports every account's cash and positions with the checks that decide
|
||||
// whether those figures can be trusted.
|
||||
func (a *App) Wealth(ctx context.Context) (Wealth, error) {
|
||||
s, err := a.Snapshot(ctx)
|
||||
if err != nil {
|
||||
return Wealth{}, err
|
||||
}
|
||||
return WealthOf(s.Data), nil
|
||||
}
|
||||
|
||||
// WealthOf derives the report from a dataset. Money is summed in exact
|
||||
// ten-thousandths; 64 bits hold hundreds of trillions, far beyond any number a
|
||||
// journal of personal accounts can reach.
|
||||
func WealthOf(data domain.Dataset) Wealth {
|
||||
instruments := map[string]domain.Instrument{}
|
||||
for _, v := range data.Instruments {
|
||||
instruments[v.ID] = v
|
||||
}
|
||||
accounts := map[string]domain.Account{}
|
||||
for _, v := range data.Accounts {
|
||||
accounts[v.ID] = v
|
||||
}
|
||||
ordered := slices.Clone(data.Transactions)
|
||||
slices.SortStableFunc(ordered, func(x, y domain.Transaction) int {
|
||||
if c := strings.Compare(x.Facts.BookingDate, y.Facts.BookingDate); c != 0 {
|
||||
return c
|
||||
}
|
||||
return strings.Compare(x.Facts.ID, y.Facts.ID)
|
||||
})
|
||||
|
||||
type holdingState struct {
|
||||
units, invested, received int64
|
||||
records int
|
||||
lowest int64
|
||||
lowestDate string
|
||||
}
|
||||
type accountState struct {
|
||||
cash, lowestCash int64
|
||||
lowestCashDate string
|
||||
records int
|
||||
first, last string
|
||||
holdings map[string]*holdingState
|
||||
order []string
|
||||
broken []string
|
||||
unappliedFee, unappliedTax int64
|
||||
unappliedRows int
|
||||
unmatchedCash, unmatchedRows int64
|
||||
}
|
||||
states := map[string]*accountState{}
|
||||
state := func(id string) *accountState {
|
||||
if states[id] == nil {
|
||||
states[id] = &accountState{holdings: map[string]*holdingState{}}
|
||||
}
|
||||
return states[id]
|
||||
}
|
||||
for _, t := range ordered {
|
||||
f := t.Facts
|
||||
account := accounts[f.AccountID]
|
||||
st := state(f.AccountID)
|
||||
st.records++
|
||||
if st.first == "" {
|
||||
st.first = f.BookingDate
|
||||
}
|
||||
st.last = f.BookingDate
|
||||
minor, err := f.Amount.Minor()
|
||||
if err != nil {
|
||||
st.broken = append(st.broken, fmt.Sprintf("%s: unreadable amount %q", f.BookingDate, f.Amount))
|
||||
continue
|
||||
}
|
||||
st.cash += minor
|
||||
if st.cash < st.lowestCash {
|
||||
st.lowestCash, st.lowestCashDate = st.cash, f.BookingDate
|
||||
}
|
||||
inv := f.Investment
|
||||
if inv == nil {
|
||||
continue
|
||||
}
|
||||
if err := domain.ValidateInvestment(f, account, instruments); err != nil {
|
||||
st.broken = append(st.broken, fmt.Sprintf("%s %s: %v", f.BookingDate, f.ID, err))
|
||||
}
|
||||
if inv.CashOnly() {
|
||||
fee, _ := inv.Fee.Minor()
|
||||
tax, _ := inv.Tax.Minor()
|
||||
if fee != 0 || tax != 0 {
|
||||
st.unappliedRows++
|
||||
st.unappliedFee += fee
|
||||
st.unappliedTax += tax
|
||||
}
|
||||
if (inv.Event == domain.EventDeposit || inv.Event == domain.EventWithdrawal) && t.Enrichment.Kind != "transfer" {
|
||||
st.unmatchedRows++
|
||||
st.unmatchedCash += minor
|
||||
}
|
||||
}
|
||||
if inv.InstrumentID == "" {
|
||||
continue
|
||||
}
|
||||
held := st.holdings[inv.InstrumentID]
|
||||
if held == nil {
|
||||
held = &holdingState{}
|
||||
st.holdings[inv.InstrumentID] = held
|
||||
st.order = append(st.order, inv.InstrumentID)
|
||||
}
|
||||
held.records++
|
||||
if inv.Settling() {
|
||||
held.invested -= minor
|
||||
} else {
|
||||
held.received += minor
|
||||
}
|
||||
units, err := inv.Quantity.Units()
|
||||
if inv.Quantity == "" {
|
||||
units, err = 0, nil
|
||||
}
|
||||
if err != nil {
|
||||
st.broken = append(st.broken, fmt.Sprintf("%s %s: unreadable quantity %q", f.BookingDate, f.ID, inv.Quantity))
|
||||
continue
|
||||
}
|
||||
held.units += units
|
||||
if held.units < held.lowest {
|
||||
held.lowest, held.lowestDate = held.units, f.BookingDate
|
||||
}
|
||||
}
|
||||
|
||||
report := Wealth{Accounts: []WealthAccount{}, Totals: []WealthTotal{}}
|
||||
totals := map[string]int64{}
|
||||
currencies := []string{}
|
||||
for _, account := range data.Accounts {
|
||||
st := state(account.ID)
|
||||
kind := account.Kind
|
||||
if kind == "" {
|
||||
kind = domain.AccountCash
|
||||
}
|
||||
entry := WealthAccount{
|
||||
AccountID: account.ID, DisplayName: account.DisplayName, Institution: account.Institution,
|
||||
Currency: account.Currency, Kind: kind, Active: account.Active,
|
||||
Records: st.records, FirstBooking: st.first, LastBooking: st.last,
|
||||
Cash: domain.FormatMoney(st.cash), Holdings: []WealthHolding{}, Checks: []WealthCheck{},
|
||||
}
|
||||
if _, seen := totals[account.Currency]; !seen {
|
||||
currencies = append(currencies, account.Currency)
|
||||
}
|
||||
totals[account.Currency] += st.cash
|
||||
for _, id := range st.order {
|
||||
held := st.holdings[id]
|
||||
instrument := instruments[id]
|
||||
entry.Holdings = append(entry.Holdings, WealthHolding{
|
||||
InstrumentID: id, ISIN: instrument.ISIN, Name: instrument.Name,
|
||||
Quantity: domain.FormatQuantity(held.units), Invested: domain.FormatMoney(held.invested),
|
||||
Received: domain.FormatMoney(held.received), Records: held.records,
|
||||
})
|
||||
}
|
||||
slices.SortFunc(entry.Holdings, func(x, y WealthHolding) int { return strings.Compare(x.Name, y.Name) })
|
||||
|
||||
check := func(name, detail string, failed bool) {
|
||||
entry.Checks = append(entry.Checks, WealthCheck{Name: name, Detail: detail, Failed: failed})
|
||||
}
|
||||
if len(st.broken) > 0 {
|
||||
check("Row arithmetic", fmt.Sprintf("%d record(s) disagree with their own figures: %s", len(st.broken), strings.Join(st.broken, "; ")), true)
|
||||
} else {
|
||||
check("Row arithmetic", "every record agrees with its own gross, fee, tax, quantity and price", false)
|
||||
}
|
||||
if st.lowestCash < 0 {
|
||||
check("Cash never negative", fmt.Sprintf("balance reached %s on %s, so the history is incomplete or a movement is misread", domain.FormatMoney(st.lowestCash), st.lowestCashDate), true)
|
||||
} else {
|
||||
check("Cash never negative", "the running balance stays at or above zero throughout", false)
|
||||
}
|
||||
negative := []string{}
|
||||
for _, id := range st.order {
|
||||
if held := st.holdings[id]; held.lowest < 0 {
|
||||
negative = append(negative, fmt.Sprintf("%s reached %s on %s", instruments[id].ISIN, domain.FormatQuantity(held.lowest), held.lowestDate))
|
||||
}
|
||||
}
|
||||
if len(negative) > 0 {
|
||||
check("Holdings never negative", fmt.Sprintf("%s — a sale before its purchase means the export is partial or a sign is wrong", strings.Join(negative, "; ")), true)
|
||||
} else if len(st.order) > 0 {
|
||||
check("Holdings never negative", "every position stays at or above zero throughout", false)
|
||||
}
|
||||
if st.unappliedRows > 0 {
|
||||
check("Fee and tax recorded, not applied", fmt.Sprintf("%d cash record(s) carry fee %s and tax %s. A broker cash amount is already net of them, so they are recorded and not subtracted again. If the balance above is wrong by one of these figures, this is why", st.unappliedRows, domain.FormatMoney(st.unappliedFee), domain.FormatMoney(st.unappliedTax)), false)
|
||||
}
|
||||
if st.unmatchedRows > 0 {
|
||||
check("Deposits and withdrawals unmatched", fmt.Sprintf("%d transfer(s) totalling %s have no counterpart in another account. They stay out of spending either way; set this account's IBAN and settlement IBAN to pair them", st.unmatchedRows, domain.FormatMoney(st.unmatchedCash)), false)
|
||||
}
|
||||
report.Accounts = append(report.Accounts, entry)
|
||||
}
|
||||
for _, currency := range currencies {
|
||||
report.Totals = append(report.Totals, WealthTotal{Currency: currency, Cash: domain.FormatMoney(totals[currency])})
|
||||
}
|
||||
return report
|
||||
}
|
||||
Reference in New Issue
Block a user