Allow the rounding a broker's own printed figures propagate

A real Scalable export refused to import at record 148: "buy gross -808.5599
does not equal quantity 6 times price 134.76, which is -808.56". Six NVIDIA
shares settled at 808.5599 against a printed price of 134.76, because the fill
was 134.759983 and the export printed the price to two places. One
ten-thousandth out, and the whole file was rejected.

The check held a gross to its own stated precision, which is only half the
story: the price is rounded too, and the file never says by how much. So the
allowance is now half a unit of the gross's stated precision plus one part in a
hundred thousand of the gross, compared against a product kept exact at 1e-16
rather than rounded first.

Measured over the complete export - 88 security rows - exactly one deviates at
all, by one part in eight million, eighty times inside the new bound. What the
bound still refuses is unchanged in kind: a price taken from the wrong share
class, and the misplaced decimal separator the check exists for, which misses
by four orders of magnitude. What it now accepts is the broker's own rounding,
including a whole cent once a gross stated to the cent passes about five hundred
euro, where a genuine one-cent error cannot be told from that rounding anyway.

The row is kept as a regression test alongside four grosses that must still be
refused: a cent, a euro, a wrong instrument's price, and a factor of ten.
This commit is contained in:
Lars Nolden
2026-09-12 12:20:19 +02:00
parent da817078f4
commit 635c11be56
4 changed files with 99 additions and 36 deletions
+17 -8
View File
@@ -443,17 +443,26 @@ The share column is signed only for corporate actions and depot transfers. Buys
and sells are unsigned and take their direction from the type. Both conventions
are resolved at import, once.
Every security row is checked against shares times price, to the precision the
export stated the amount at and no further. One export prints the exact product
to nine places, and the check is then exact. Another prints the notional rounded
to cents, where demanding exactness rejects every trade whose product does not
land on a whole cent - measured on a real export, 29 of 59 of them. One unit of
the stated precision is still four orders of magnitude tighter than the
misplaced decimal separator this check exists to catch.
Every security row is checked against shares times price, allowing for the
rounding the export's own printed figures propagate. Both ends are rounded and
neither states by how much: one export prints the notional to the cent, so
0,426581 shares at 63,06 settle as 26,90 where the product is 26,90019786;
another prints a price to fewer places than the fill actually had, settling six
NVIDIA shares at 808,5599 against a printed 134,76 whose product is 808,56.
The allowance is half a unit of the gross's stated precision plus one part in a
hundred thousand of the gross. Measured over a complete real export of 88
security rows, exactly one deviates at all, by one part in eight million.
What that still refuses: a price taken from the wrong share class, and the lost
decimal separator the check exists for, four orders of magnitude out. What it
accepts: the broker's own rounding, including a whole cent once a gross stated
to the cent passes about five hundred euro, where a real one-cent error cannot
be told from that rounding.
It cannot catch a separator lost uniformly across a row: 1 x 25,795 and
1 x 25795 both satisfy it. A price cross-check against an outside provider is
the only remedy and is deliberately not implemented.
the only remedy and is deliberately not implemented. A spreadsheet round-trip
is what strips those separators, so import the broker's original file.
Rejected whole, with the record number: an unknown status, an unknown type, a
classifying column that disagrees with its type, an account type other than the
+1 -1
View File
@@ -202,7 +202,7 @@ Because a cash row's amount already includes the tax the broker withheld or refu
Two more traps there: a `DIVIDEND` row fills the share column with **the holding the dividend was paid on**, so adding it would double the position; and crypto carries a bare ticker like `DOGE` in `symbol`, with its real identifier only in the description. Both are handled, and a position row that resolves to neither is refused.
Only `Executed` rows import from Scalable: a cancelled retry is all zeros, so it passes every arithmetic check and would otherwise become a phantom trade. Every security row is verified against shares × price **to the precision the broker stated the amount at** — exactly, where the export prints the full product; to within a cent, where it prints the notional rounded. An unknown row type, a mismatched classifying column, a foreign settlement currency, an unresolvable security, or a failed check rejects the **whole file** with the record number, because each of those can move money that never moved.
Only `Executed` rows import from Scalable: a cancelled retry is all zeros, so it passes every arithmetic check and would otherwise become a phantom trade. Every security row is verified against shares × price, **allowing for the rounding the export's own figures propagate** — both the gross and the price are printed rounded, and neither says by how much. Across a complete real export of 88 security rows exactly one deviates at all, by one part in eight million; a misplaced decimal separator is four orders of magnitude outside the allowance. An unknown row type, a mismatched classifying column, a foreign settlement currency, an unresolvable security, or a failed check rejects the **whole file** with the record number, because each of those can move money that never moved.
Securities are registered by **ISIN** in **Instruments**. The ISIN is the identity; the name is editable display text, because one ISIN appears under several broker names over the years. Crypto is held under the ISIN-shaped identifier the broker issues for it. Set the account's **settlement IBAN** for an export that names no counterparty of its own, so deposits from your bank pair with the funding account instead of staying unpaired. They never become income either way — a broker record is excluded from spending and income analytics, from bulk reclassification, and from the AI entirely.
+32
View File
@@ -228,6 +228,38 @@ func TestSingleShareRowCatchesOnlyInconsistentArithmetic(t *testing.T) {
}
}
// A broker's own gross can disagree with its own printed shares times price,
// because the price is printed to fewer places than the fill actually had.
// Six NVIDIA shares settled at 808.5599 against a printed 134.76, whose
// product is 808.56: one ten-thousandth out, and the whole file was refused.
// The rounding the printed figures propagate is allowed; anything above one
// part in a hundred thousand still is not.
func TestRoundedPriceDoesNotRejectTheBrokersOwnGross(t *testing.T) {
const row = `2025-01-09;10:37:32;Executed;SCALixkS3TomjQv;NVIDIA;Security;Buy;US67066G1040;6;134,76;-808,5599;0,00;0,00;EUR`
result := readBroker(t, row)
inv := result.Facts[0].Investment
if inv.Gross != "-808.5599" || inv.Price != "134.76" || inv.Quantity != "6" {
t.Fatalf("trade read as %+v", inv)
}
if got := result.Facts[0].Amount; got != "-808.5599" {
t.Errorf("settled %s, want -808.5599", got)
}
for name, gross := range map[string]string{
"one cent out": "-808,5699",
"factor of ten": "-8.085,599",
"a euro out": "-809,5599",
"wrong instrument": "-908,5599",
} {
file, err := ReadCSV(strings.NewReader(scalableHeader + strings.Replace(row, ";-808,5599;", ";"+gross+";", 1) + "\n"))
if err != nil {
t.Fatalf("%s: %v", name, err)
}
if _, err := ParseScalableCSV(file, brokerAccount(), nil); err == nil {
t.Errorf("%s: accepted a gross its own shares times price does not support", name)
}
}
}
// A reinvested distribution settles shares times price, so it carries as many
// decimal places as the two together need. A real export reinvests to nine,
// which is past what money holds and past what a share count holds, so reading
+44 -22
View File
@@ -639,7 +639,14 @@ func validateInvestment(f Facts, a Account, instruments map[string]Instrument) e
return fmt.Errorf("%s requires a nonzero quantity", inv.Event)
}
// A position-only valuation carries the sign of the position change; a
// settled trade carries the sign of the cash, which is the opposite.
// settled trade carries the sign of the cash, which is the opposite. The
// product is kept exact at 1e-16 so the comparison never rounds first.
product := new(big.Int).Mul(big.NewInt(quantity), big.NewInt(price))
if inv.Settling() {
product.Neg(product)
}
difference := new(big.Int).Sub(product, new(big.Int).Mul(big.NewInt(gross), productPerMoney))
if difference.Abs(difference).Cmp(grossSlack(gross, inv.Gross)) > 0 {
expected, ok := RoundedProduct(quantity, price)
if !ok {
return fmt.Errorf("%s quantity times price is out of range", inv.Event)
@@ -647,18 +654,6 @@ func validateInvestment(f Facts, a Account, instruments map[string]Instrument) e
if inv.Settling() {
expected = -expected
}
// The gross is checked to the precision the broker stated it at, and no
// further. One broker prints the exact product to nine places, and the
// check is then exact. Another prints the notional rounded to cents, where
// demanding exactness rejects every trade whose product does not land on a
// whole cent - measured on a real export, 29 of 59 of them. One unit of
// the stated precision is still four orders of magnitude tighter than the
// misplaced decimal separator this check exists to catch.
difference := expected - gross
if difference < 0 {
difference = -difference
}
if difference >= statedUnit(inv.Gross) {
return fmt.Errorf("%s gross %s does not equal quantity %s times price %s, which is %s", inv.Event, inv.Gross.String(), inv.Quantity.String(), inv.Price.String(), Money(formatScaled(expected, moneyScale, 2)))
}
if inv.PositionOnly() {
@@ -689,14 +684,41 @@ func settles(inv *Investment, gross, fee, tax, amount int64) error {
return nil
}
// statedUnit is one unit of the last decimal place a money figure was written
// with, in exact ten-thousandths. Money always renders at least two places, so
// a whole-euro figure counts as stated to the cent.
func statedUnit(m Money) int64 {
_, fraction, _ := strings.Cut(string(m), ".")
unit := int64(1)
for range moneyScale - len(strings.TrimRight(fraction, "0")) {
unit *= 10
// productPerMoney converts money's ten-thousandths to the 1e-16 units a
// quantity times a price lands in.
var productPerMoney = new(big.Int).Exp(big.NewInt(10), big.NewInt(productScale-moneyScale), nil)
const productScale = quantityScale * 2
// grossSlack is how far a printed gross may sit from the product of the printed
// quantity and price before the row is refused. Both ends are rounded, and
// neither states by how much.
//
// The gross is rounded to its own last decimal place: one broker prints the
// notional to the cent, so 0.426581 shares at 63.06 settle as 26.90 where the
// product is 26.90019786, and demanding exactness there rejects half a
// portfolio. The price is rounded to a precision the file does not state: the
// same export settles six NVIDIA shares at 808.5599 while printing the price
// as 134.76, whose product is 808.56, because the real fill was 134.759983.
// So the slack is half a unit of the gross's stated precision, plus one part
// in a hundred thousand of the gross itself.
//
// Measured over a complete real export of 88 security rows, exactly one
// deviates at all, by one part in eight million - eighty times inside this
// bound. What it refuses: any deviation above one part in a hundred thousand,
// which covers a price taken from the wrong share class and the lost decimal
// separator this check exists for, four orders of magnitude out. What it
// accepts: the broker's own rounding. On a gross stated to the cent the slack
// reaches a whole cent at around five hundred euro, above which a genuine
// one-cent error is indistinguishable from that rounding and is allowed.
func grossSlack(gross int64, printed Money) *big.Int {
_, fraction, _ := strings.Cut(string(printed), ".")
places := len(fraction)
if places > moneyScale {
places = moneyScale
}
return unit
half := new(big.Int).Exp(big.NewInt(10), big.NewInt(int64(productScale-places)), nil)
half.Quo(half, big.NewInt(2))
relative := new(big.Int).Abs(new(big.Int).Mul(big.NewInt(gross), productPerMoney))
return half.Add(half, relative.Quo(relative, big.NewInt(100_000)))
}