Track investments as broker facts with a position leg

An account now has a kind, and an investment account holds positions as well as cash. A broker row is not a new entity: it is a bank fact with an optional position leg, so deduplication, the journal, fact immutability, the DuckDB projection and the transactions view carry it unchanged. Facts.Amount stays the cash leg and is zero on the rows that move only a position. Scalable Capital exports are recognized locally as a fourth format, read by their own parser because a column mapping cannot describe them: the amount column is settled cash on a cash row, a gross to be netted on a trade, and a position valuation that must never touch cash on a corporate action or a depot transfer. A cash amount is already net of the tax the broker withheld or refunded, so that tax is recorded on the fact and never subtracted a second time; treating a corporate action's valuation as money conjures cash, and a depot switch would do it once per instrument. The share column is signed only for those two types, so buys and sells take their direction from the type. Every security row is checked against shares times price at 128-bit width, because a lost decimal separator survives every other check. An unknown status, type or assetType, a foreign currency, a missing ISIN, or one failed check rejects the whole file with the record number. Instruments live in instruments.finance, keyed by ISIN with an ID derived from it, so re-importing never registers a security twice. One ISIN appears under several broker descriptions over the years and sometimes under the ISIN itself: the most recent real description names it, and an import never renames one that already exists. A broker also reuses a single reference across every leg of one event, so transaction identity includes the event and its instrument. domain.Fallback returns kind "investment" for any fact carrying a position leg, so no broker row reaches the sign-based branch. That single rule is what stops an unmatched deposit from counting as income and a broker fee from counting as household spending; the monthly PRIME fee and its matching credit now cancel in clearing:investments with no configuration at all. Investment rows are excluded from spending analytics, from bulk reclassification and from the model, exactly as transfers are. Equal competing transfers are paired instead of skipped. Every connected component of the candidate graph is a complete bipartite graph between two fixed accounts at one amount and currency, so every pairing produces the same accounts, kinds and postings and only the displayed counterpart differs. Refusing to choose was the expensive option: both legs fell through to the sign-based fallback and appeared as spending and income that never happened. Pairing follows the nearest booking date, then the transaction ID, so iteration order decides nothing. POST /api/transactions/{id}/transfer rewrites the old and the new pair in one commit, because reciprocity is validated and a half-applied link is an invalid dataset, and the matcher now skips any record classified manually so a hand-made link or unlink outlives the next import. Wealth reports each account's cash and positions from the journal rather than the index, with named checks - row arithmetic, cash never negative, holdings never negative - because it exists to be compared against the figures a broker shows on its own screen. A negative holding means the imported history is partial. Share counts are exact to eight places; a reinvested distribution quoted to six is rounded to money's four and the residue is reported rather than hidden. Market prices, market value, net worth over time, FIFO lot accounting, realised gains and currency conversion are deliberately absent.
2026-09-11 21:58:47 +02:00
parent 673cbf917b
commit 922ae507bd
27 changed files with 3071 additions and 157 deletions
@@ -2,9 +2,11 @@ package domain

 import (
 	"crypto/rand"
+	"crypto/sha256"
 	"encoding/hex"
 	"fmt"
 	"math"
+	"math/big"
 	"regexp"
 	"strconv"
 	"strings"
@@ -14,19 +16,34 @@ import (

 var idPattern = regexp.MustCompile(`^[A-Za-z][A-Za-z0-9_-]{0,127}$`)
 var currencyPattern = regexp.MustCompile(`^[A-Z]{3}$`)
+var isinPattern = regexp.MustCompile(`^[A-Z]{2}[A-Z0-9]{9}[0-9]$`)
+
+const moneyScale = 4
+const quantityScale = 8

 // ParseMoney accepts exact decimal values representable as signed 64-bit ten-thousandths.
 // This intentionally bounds the otherwise larger DECIMAL(24,4) database domain.
 func ParseMoney(s string) (Money, error) {
-	n, err := parseMinor(s)
+	n, err := parseScaled(s, moneyScale, "money", "four")
 	if err != nil {
 		return "", err
 	}
-	return Money(formatMinor(n)), nil
+	return Money(formatScaled(n, moneyScale, 2)), nil
 }
-func parseMinor(s string) (int64, error) {
+
+// ParseQuantity accepts exact share counts representable as signed 64-bit
+// hundred-millionths. Money's four places cannot hold a reinvested fraction of
+// a share, and a truncated share count silently misstates a holding.
+func ParseQuantity(s string) (Quantity, error) {
+	n, err := parseScaled(s, quantityScale, "quantity", "eight")
+	if err != nil {
+		return "", err
+	}
+	return Quantity(formatScaled(n, quantityScale, 0)), nil
+}
+func parseScaled(s string, scale int, noun, places string) (int64, error) {
 	invalid := func() (int64, error) {
-		return 0, fmt.Errorf("invalid or out-of-range money %q: require signed 64-bit ten-thousandths, at most four fractional digits", s)
+		return 0, fmt.Errorf("invalid or out-of-range %s %q: require signed 64-bit value with at most %s fractional digits", noun, s, places)
 	}
 	if s == "" {
 		return invalid()
@@ -65,7 +82,7 @@ func parseMinor(s string) (int64, error) {
 		}
 		if fraction >= 0 {
 			fraction++
-			if fraction > 4 {
+			if fraction > scale {
 				return invalid()
 			}
 		}
@@ -78,7 +95,7 @@ func parseMinor(s string) (int64, error) {
 	if fraction < 0 {
 		fraction = 0
 	}
-	for range 4 - fraction {
+	for range scale - fraction {
 		if magnitude > limit/10 {
 			return invalid()
 		}
@@ -92,22 +109,28 @@ func parseMinor(s string) (int64, error) {
 	}
 	return int64(magnitude), nil
 }
-func formatMinor(n int64) string {
+
+// formatScaled renders exact units. minFraction keeps money at two places for
+// display while letting a whole share count render without eight zeros.
+func formatScaled(n int64, scale, minFraction int) string {
 	s := strconv.FormatInt(n, 10)
 	sign := ""
 	if strings.HasPrefix(s, "-") {
 		sign, s = "-", s[1:]
 	}
-	if len(s) < 5 {
-		s = strings.Repeat("0", 5-len(s)) + s
+	if len(s) < scale+1 {
+		s = strings.Repeat("0", scale+1-len(s)) + s
 	}
-	whole, fraction := s[:len(s)-4], strings.TrimRight(s[len(s)-4:], "0")
-	if len(fraction) < 2 {
-		fraction += strings.Repeat("0", 2-len(fraction))
+	whole, fraction := s[:len(s)-scale], strings.TrimRight(s[len(s)-scale:], "0")
+	if len(fraction) < minFraction {
+		fraction += strings.Repeat("0", minFraction-len(fraction))
+	}
+	if fraction == "" {
+		return sign + whole
 	}
 	return sign + whole + "." + fraction
 }
-func (m Money) Minor() (int64, error) { return parseMinor(string(m)) }
+func (m Money) Minor() (int64, error) { return parseScaled(string(m), moneyScale, "money", "four") }
 func (m Money) String() string {
 	parsed, err := ParseMoney(string(m))
 	if err != nil {
@@ -115,6 +138,25 @@ func (m Money) String() string {
 	}
 	return string(parsed)
 }
+func (q Quantity) Units() (int64, error) {
+	return parseScaled(string(q), quantityScale, "quantity", "eight")
+}
+func (q Quantity) String() string {
+	parsed, err := ParseQuantity(string(q))
+	if err != nil {
+		return string(q)
+	}
+	return string(parsed)
+}
+
+// FormatMoney renders exact ten-thousandths as money, and FormatQuantity
+// renders exact hundred-millionths as a share count. Exact units are the only
+// safe currency for arithmetic, and these are how a computed total re-enters
+// the journal without a float ever being involved.
+func FormatMoney(minor int64) Money { return Money(formatScaled(minor, moneyScale, 2)) }
+func FormatQuantity(units int64) Quantity {
+	return Quantity(formatScaled(units, quantityScale, 0))
+}
 func NewID(prefix string) string {
 	if !idPattern.MatchString(prefix) || len(prefix) > 94 {
 		panic("invalid ID prefix")
@@ -129,20 +171,40 @@ func NewDataset() Dataset {
 	return Dataset{Accounts: []Account{}, Categories: []Category{
 		{ID: "cat_expenses", Name: "Expenses", Kind: "expense"}, {ID: ExpenseFallback, Name: "Unclassified", ParentID: "cat_expenses", Kind: "expense"},
 		{ID: "cat_income", Name: "Income", Kind: "income"}, {ID: IncomeFallback, Name: "Unclassified", ParentID: "cat_income", Kind: "income"},
-	}, Tags: []Tag{}, Merchants: []Merchant{}, Transactions: []Transaction{}}
+	}, Tags: []Tag{}, Merchants: []Merchant{}, Instruments: []Instrument{}, Transactions: []Transaction{}}
+}
+
+// InstrumentID derives a stable registry ID from an ISIN so re-importing the
+// same export never creates a second instrument for one security.
+func InstrumentID(isin string) string {
+	sum := sha256.Sum256([]byte("instrument\x00" + strings.ToUpper(strings.TrimSpace(isin))))
+	return "ins_" + hex.EncodeToString(sum[:16])
 }
 func Clone(d Dataset) Dataset {
-	c := Dataset{Accounts: append([]Account{}, d.Accounts...), Categories: append([]Category{}, d.Categories...), Tags: append([]Tag{}, d.Tags...), Merchants: append([]Merchant{}, d.Merchants...), Transactions: append([]Transaction{}, d.Transactions...)}
+	c := Dataset{Accounts: append([]Account{}, d.Accounts...), Categories: append([]Category{}, d.Categories...), Tags: append([]Tag{}, d.Tags...), Merchants: append([]Merchant{}, d.Merchants...), Instruments: append([]Instrument{}, d.Instruments...), Transactions: append([]Transaction{}, d.Transactions...)}
 	for i := range c.Merchants {
 		c.Merchants[i].Aliases = append([]string{}, d.Merchants[i].Aliases...)
 		c.Merchants[i].DefaultTagIDs = append([]string{}, d.Merchants[i].DefaultTagIDs...)
 	}
 	for i := range c.Transactions {
 		c.Transactions[i].Enrichment.TagIDs = append([]string{}, d.Transactions[i].Enrichment.TagIDs...)
+		// Facts are immutable, but a shared pointer would let one dataset's
+		// edit reach another's copy.
+		if inv := d.Transactions[i].Facts.Investment; inv != nil {
+			copied := *inv
+			c.Transactions[i].Facts.Investment = &copied
+		}
 	}
 	return c
 }
+
+// Fallback classifies a fact that no rule or model claimed. Broker facts never
+// take the sign-based branch: an unmatched deposit is not income and a broker
+// fee paid out of an investment account is not household spending.
 func Fallback(f Facts) Enrichment {
+	if f.Investment != nil {
+		return Enrichment{Kind: KindInvestment, TagIDs: []string{}, Classification: Provenance{Source: "fallback"}}
+	}
 	kind, category := "expense", ExpenseFallback
 	n, err := f.Amount.Minor()
 	if err == nil && n > 0 {
@@ -185,6 +247,16 @@ func validText(values ...string) bool {
 	return true
 }

+// ValidISIN reports a syntactically valid ISIN: two country letters, nine
+// alphanumerics and a check digit.
+func ValidISIN(s string) bool { return isinPattern.MatchString(s) }
+
+// ValidateInvestment checks one broker fact against the investment model. An
+// importer calls it per record so a malformed export is refused with the
+// record that caused it, rather than at commit with only an ID.
+func ValidateInvestment(f Facts, account Account, instruments map[string]Instrument) error {
+	return validateInvestment(f, account, instruments)
+}
 func Validate(d Dataset) error {
 	ids := map[string]string{}
 	register := func(id, kind string) error {
@@ -205,9 +277,12 @@ func Validate(d Dataset) error {
 		if err := register(a.ID, "account"); err != nil {
 			return err
 		}
-		if !nonblank(a.DisplayName) || !currencyPattern.MatchString(a.Currency) || !validText(a.Institution, a.ExternalAccountID, a.IBAN) {
+		if !nonblank(a.DisplayName) || !currencyPattern.MatchString(a.Currency) || !validText(a.Institution, a.ExternalAccountID, a.IBAN, a.ReferenceIBAN) {
 			return fmt.Errorf("account %q: valid UTF-8 name and three-letter uppercase currency required", a.ID)
 		}
+		if a.Kind != "" && a.Kind != AccountCash && a.Kind != AccountInvestment {
+			return fmt.Errorf("account %q: kind must be %q or %q", a.ID, AccountCash, AccountInvestment)
+		}
 		accounts[a.ID] = a
 	}
 	for _, c := range d.Categories {
@@ -285,6 +360,24 @@ func Validate(d Dataset) error {
 			aliases[key] = true
 		}
 	}
+	instruments := map[string]Instrument{}
+	isins := map[string]string{}
+	for _, v := range d.Instruments {
+		if err := register(v.ID, "instrument"); err != nil {
+			return err
+		}
+		if !isinPattern.MatchString(v.ISIN) {
+			return fmt.Errorf("instrument %q: ISIN must be two letters, nine alphanumerics and a check digit", v.ID)
+		}
+		if other, ok := isins[v.ISIN]; ok {
+			return fmt.Errorf("instrument %q: ISIN %s already held by %q", v.ID, v.ISIN, other)
+		}
+		if !nonblank(v.Name) || !currencyPattern.MatchString(v.Currency) {
+			return fmt.Errorf("instrument %q: valid UTF-8 name and three-letter uppercase currency required", v.ID)
+		}
+		isins[v.ISIN] = v.ID
+		instruments[v.ID] = v
+	}
 	for _, t := range d.Transactions {
 		f := t.Facts
 		if err := register(f.ID, "transaction"); err != nil {
@@ -309,6 +402,9 @@ func Validate(d Dataset) error {
 		if !validText(f.RawDescription, f.ExternalID, f.Counterparty, f.CounterpartyIBAN) {
 			return fmt.Errorf("transaction %q: bank facts must be valid UTF-8", f.ID)
 		}
+		if err := validateInvestment(f, a, instruments); err != nil {
+			return fmt.Errorf("transaction %q: %w", f.ID, err)
+		}
 	}
 	index := enrichmentIndex{categories: categories, children: children, tags: tags, merchants: map[string]bool{}, transactions: map[string]Transaction{}}
 	for _, m := range d.Merchants {
@@ -351,9 +447,12 @@ func ValidateEnrichment(d Dataset, f Facts, e Enrichment) error {
 	return index.validate(f, e)
 }
 func (index enrichmentIndex) validate(f Facts, e Enrichment) error {
-	if e.Kind != "expense" && e.Kind != "income" && e.Kind != "transfer" {
+	if e.Kind != "expense" && e.Kind != "income" && e.Kind != "transfer" && e.Kind != KindInvestment {
 		return fmt.Errorf("invalid enrichment kind %q", e.Kind)
 	}
+	if (e.Kind == KindInvestment) != (f.Investment != nil && e.Kind != "transfer") {
+		return fmt.Errorf("only broker facts carry kind %q, and every unlinked broker fact must", KindInvestment)
+	}
 	seen := map[string]bool{}
 	for _, id := range e.TagIDs {
 		if !index.tags[id] || seen[id] {
@@ -372,6 +471,15 @@ func (index enrichmentIndex) validate(f Facts, e Enrichment) error {
 			return fmt.Errorf("invalid classification timestamp")
 		}
 	}
+	if e.Kind == KindInvestment {
+		if e.CategoryID != "" || e.MerchantID != "" || e.TransferPeerID != "" {
+			return fmt.Errorf("investment must not have category, merchant or transfer peer")
+		}
+		if e.Classification.Source == "ai" || e.Classification.Source == "openrouter" {
+			return fmt.Errorf("AI cannot classify investments")
+		}
+		return nil
+	}
 	if e.Kind == "transfer" {
 		if e.CategoryID != "" || e.MerchantID != "" {
 			return fmt.Errorf("transfer must not have category or merchant")
@@ -379,6 +487,9 @@ func (index enrichmentIndex) validate(f Facts, e Enrichment) error {
 		if e.Classification.Source == "ai" || e.Classification.Source == "openrouter" {
 			return fmt.Errorf("AI cannot classify transfers")
 		}
+		if f.Investment != nil && !f.Investment.CashOnly() {
+			return fmt.Errorf("only a broker cash movement can be linked as a transfer, not %q", f.Investment.Event)
+		}
 		amount, err := f.Amount.Minor()
 		if err != nil {
 			return err
@@ -413,3 +524,134 @@ func (index enrichmentIndex) validate(f Facts, e Enrichment) error {
 	}
 	return nil
 }
+
+func optionalMoney(m Money) (int64, error) {
+	if m == "" {
+		return 0, nil
+	}
+	return m.Minor()
+}
+func optionalQuantity(q Quantity) (int64, error) {
+	if q == "" {
+		return 0, nil
+	}
+	return q.Units()
+}
+
+// RoundedProduct multiplies an exact share count by an exact price and rounds
+// to money's four places, half away from zero. Quantity is 1e-8 units and
+// price is 1e-4 units, so the product is 1e-12 and needs 128-bit width.
+func RoundedProduct(quantity, price int64) (int64, bool) {
+	product := new(big.Int).Mul(big.NewInt(quantity), big.NewInt(price))
+	half := big.NewInt(50_000_000)
+	if product.Sign() < 0 {
+		product.Sub(product, half)
+	} else {
+		product.Add(product, half)
+	}
+	rounded := product.Quo(product, big.NewInt(100_000_000))
+	if !rounded.IsInt64() {
+		return 0, false
+	}
+	return rounded.Int64(), true
+}
+
+// validateInvestment enforces the broker row model.
+//
+// A cash row's amount is the money that actually moved and is already net of
+// the tax the broker withheld or refunded, so its tax column is recorded but
+// never applied. A buy, sell or reinvestment quotes a gross of shares times
+// price and settles gross minus fee minus tax. A corporate action or depot
+// transfer moves a position at a valuation and must never touch cash: treating
+// its amount as money conjures or destroys it.
+//
+// Every security row is checked against shares times price. That is the only
+// check that catches a lost decimal separator, and it is worthless without it:
+// a one-share row satisfies every other invariant at any scale.
+func validateInvestment(f Facts, a Account, instruments map[string]Instrument) error {
+	inv := f.Investment
+	if inv == nil {
+		return nil
+	}
+	if !a.Investing() {
+		return fmt.Errorf("investment leg requires an account of kind %q", AccountInvestment)
+	}
+	if !inv.CashOnly() && !inv.Settling() && !inv.PositionOnly() {
+		return fmt.Errorf("unknown investment event %q", inv.Event)
+	}
+	if inv.InstrumentID != "" {
+		v, ok := instruments[inv.InstrumentID]
+		if !ok {
+			return fmt.Errorf("unknown instrument %q", inv.InstrumentID)
+		}
+		if v.Currency != f.Currency {
+			return fmt.Errorf("instrument %s trades in %s but this fact settles in %s", v.ISIN, v.Currency, f.Currency)
+		}
+	}
+	quantity, err := optionalQuantity(inv.Quantity)
+	if err != nil {
+		return err
+	}
+	price, err := optionalMoney(inv.Price)
+	if err != nil {
+		return err
+	}
+	gross, err := optionalMoney(inv.Gross)
+	if err != nil {
+		return err
+	}
+	fee, err := optionalMoney(inv.Fee)
+	if err != nil {
+		return err
+	}
+	tax, err := optionalMoney(inv.Tax)
+	if err != nil {
+		return err
+	}
+	amount, err := f.Amount.Minor()
+	if err != nil {
+		return err
+	}
+	if inv.CashOnly() {
+		if quantity != 0 || inv.Price != "" || inv.Gross != "" {
+			return fmt.Errorf("%s moves cash only: it carries no quantity, price or gross", inv.Event)
+		}
+		return nil
+	}
+	if inv.InstrumentID == "" {
+		return fmt.Errorf("%s requires an instrument", inv.Event)
+	}
+	if quantity == 0 {
+		return fmt.Errorf("%s requires a nonzero quantity", inv.Event)
+	}
+	// A position-only valuation carries the sign of the position change; a
+	// settled trade carries the sign of the cash, which is the opposite.
+	expected, ok := RoundedProduct(quantity, price)
+	if !ok {
+		return fmt.Errorf("%s quantity times price is out of range", inv.Event)
+	}
+	if inv.Settling() {
+		expected = -expected
+	}
+	if gross != expected {
+		return fmt.Errorf("%s gross %s does not equal quantity %s times price %s", inv.Event, Money(formatScaled(gross, moneyScale, 2)), inv.Quantity.String(), inv.Price.String())
+	}
+	if inv.PositionOnly() {
+		if amount != 0 {
+			return fmt.Errorf("%s moves position only, but this fact carries cash %s", inv.Event, f.Amount.String())
+		}
+		if fee != 0 || tax != 0 {
+			return fmt.Errorf("%s cannot carry a fee or tax", inv.Event)
+		}
+		return nil
+	}
+	if (inv.Event == EventSell) != (quantity < 0) {
+		return fmt.Errorf("%s must %s the position", inv.Event, map[bool]string{true: "reduce", false: "increase"}[inv.Event == EventSell])
+	}
+	settled := new(big.Int).Sub(big.NewInt(gross), big.NewInt(fee))
+	settled.Sub(settled, big.NewInt(tax))
+	if !settled.IsInt64() || settled.Int64() != amount {
+		return fmt.Errorf("%s cash %s does not equal gross %s minus fee %s minus tax %s", inv.Event, f.Amount.String(), inv.Gross.String(), inv.Fee.String(), inv.Tax.String())
+	}
+	return nil
+}