lars/finance-duck
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Import Trade Republic exports, whose conventions invert Scalable's

Browse Source 
A second broker export is recognized locally, by its full column set, and read
through the same pipeline: detection and parsing now dispatch on the format, so
the upload path, the review dialog, deduplication, the journal and the Wealth
report are unchanged. Its nine row types cover cash transfers, interest,
dividends, tax settlements and trades in funds, shares and crypto; none of them
moves a position without moving cash, so the cash-neutral class that Scalable's
corporate actions belong to does not arise here.

Three of its conventions are the opposite of the export already supported, and
reading any of them the other way round moves money. Fee and tax are the signed
adjustments it made to the cash rather than deductions from a gross, so a one
euro order fee arrives as -1.00 and is negated at import; the journal keeps one
convention and the domain never learns that two exist. A cash row's amount is
the gross, not the net, so interest of 16.46 with -4.33 of tax credits 12.13 -
where the other export states its cash already net and its tax is recorded and
never applied. Whether a cash row carries a gross now decides which of those it
was, which also makes the first kind's settlement checkable and stops the Wealth
report from claiming a figure was left unapplied when it was not. And a
TAX_OPTIMIZATION row puts zero in the amount column and its money in the tax
column, signed both ways: read as cash, all six in a real export move nothing.

Two more rows lie about their own columns. A dividend fills the share column
with the holding the dividend was paid on, not with a position change, so adding
it would double the holding. Crypto carries a bare ticker in the symbol column
and its ISIN-shaped identifier only in the description, so the identifier is
taken from the symbol when that is an ISIN and otherwise from the one the
description names; a position row resolving to neither is refused rather than
attached to a guess.

The shares-times-price check now holds a gross to the precision the export
stated it at rather than to four places. This export prints the notional rounded
to cents, and 29 of 59 real trades do not land on a whole cent: demanding
exactness rejected half a portfolio. One unit of the stated precision is still
four orders of magnitude tighter than the misplaced separator the check exists
to catch, and where an export prints the full product the check stays exact.

A unit price moves from money to the eight-place quantity type, because a crypto
price is quoted to six and rounding it would break the check the amount is
verified against. Trailing zeros are dropped before any precision test: this
export pads a six-place price to ten, and the padding would otherwise exhaust
the precision the value needs.

A transfer's counterparty comes from the export's own IBAN column when it has
one, from the IBAN the description names in parentheses when it does not, and
from the account's configured settlement IBAN when neither names anything. Free
text contributes only a value shaped like an IBAN. Without this, 108 transfers
stay unpaired and their bank-side counterparts read as spending and income.

Verified end to end against a real export: 26 rows import to a cash balance of
32187.02 matching the figure computed by hand from the source rows, all four
positions close at exactly zero, and every trade satisfies its own arithmetic.
This commit is contained in:
Lars Nolden
2026-09-11 23:04:22 +02:00
parent 87f052a3ea
commit 762ad3fae5
12 changed files with 1059 additions and 288 deletions
Download Patch File Download Diff File Expand all files Collapse all files
OPERATIONS.txt
+115 -31
View File
@@ -357,8 +357,8 @@ mandate reference must never become a transaction identity. ING facts likewise
carry no reference. Review the previewed dates, amount signs and currency before
confirming; a wrong mapping is visible there, not after import.
Import sources: n26_csv, ing_csv, kontist_csv, scalable_csv, csv (AI-mapped),
enablebanking.
Import sources: n26_csv, ing_csv, kontist_csv, scalable_csv, traderepublic_csv,
csv (AI-mapped), enablebanking.
Stable provider entry references are scoped by account, source and debit/credit
direction: a debit and credit can share a reference without being collapsed.
@@ -392,11 +392,17 @@ Investment accounts and broker imports
--------------------------------------
An account has a kind, "cash" (the default, and what an absent kind means) or
"investment". An investment account holds a cash balance and positions. It also
carries a settlement IBAN (reference_iban): a broker export has no counterparty
column, so deposits and withdrawals are stamped with that IBAN and pair with the
funding account through ordinary transfer matching. Leave it empty and those
rows simply stay unpaired, which costs accuracy in spending analysis but never
invents income.
carries a settlement IBAN (reference_iban), used when an export names no
counterparty of its own, so deposits and withdrawals pair with the funding
account through ordinary transfer matching. Leave it empty and those rows simply
stay unpaired, which costs accuracy in spending analysis but never invents
income.
Two broker exports are recognized locally, by their complete column set. A
layout is matched whole because a row's meaning depends on a combination of its
classifying columns, so a partial match is a different file wearing the same
names. Everything below about events, instruments, precision and the checks
applies to both; the per-export differences are listed under each.
Scalable Capital exports (scalable_csv) are recognized locally by their full
column set: date, time, status, reference, description, assetType, type, isin,
@@ -437,35 +443,50 @@ The share column is signed only for corporate actions and depot transfers. Buys
and sells are unsigned and take their direction from the type. Both conventions
are resolved at import, once.
Every security row is checked against shares times price at full precision.
This is the only check that catches a lost decimal separator, and it cannot
catch one that was lost uniformly across a row: 1 x 25,795 and 1 x 25795 both
satisfy it. A price cross-check against an outside provider is the only remedy
and is deliberately not implemented.
Every security row is checked against shares times price, to the precision the
export stated the amount at and no further. One export prints the exact product
to nine places, and the check is then exact. Another prints the notional rounded
to cents, where demanding exactness rejects every trade whose product does not
land on a whole cent - measured on a real export, 29 of 59 of them. One unit of
the stated precision is still four orders of magnitude tighter than the
misplaced decimal separator this check exists to catch.
Rejected whole, with the record number: an unknown status, an unknown type, an
assetType that disagrees with its type, a currency other than the account's, a
security row without an ISIN, an invalid ISIN, a signed buy or sell, a corporate
action or depot transfer carrying a fee or tax, and any failed arithmetic check.
A zero amount is accepted; it corrupts nothing, and a free share allocation is
legitimately priced at zero.
It cannot catch a separator lost uniformly across a row: 1 x 25,795 and
1 x 25795 both satisfy it. A price cross-check against an outside provider is
the only remedy and is deliberately not implemented.
Money holds four decimal places and share counts hold eight. An amount is the
row's share count times its price, so it carries as many decimal places as the
two together need: a reinvested distribution in a real export reaches nine,
past both. Amounts are therefore read at arbitrary precision, rounded to four
places half away from zero, and the exact discarded residue is summed and
reported in the import review rather than hidden. A share count or a price
beyond its own precision is refused instead of truncated: rounding a share
count misstates a holding, and rounding a price would break the shares-times-
price check that the amount is verified against.
Rejected whole, with the record number: an unknown status, an unknown type, a
classifying column that disagrees with its type, an account type other than the
one the import targets, a currency other than the account's, a security row
without a resolvable identifier, an invalid ISIN, a signed buy or sell where the
export leaves them unsigned, a corporate action or depot transfer carrying a fee
or tax, and any failed arithmetic check. A zero amount is accepted; it corrupts
nothing, and a free share allocation is legitimately priced at zero.
Money holds four decimal places; share counts and unit prices hold eight. An
amount is the row's share count times its price, so it carries as many decimal
places as the two together need: a reinvested distribution in a real export
reaches nine, past both. Amounts are therefore read at arbitrary precision,
rounded to four places half away from zero, and the exact discarded residue is
summed and reported in the import review rather than hidden. Trailing zeros are
padding, not precision: an export that writes a six-place price to ten places is
read at six. A share count or a price beyond eight places is refused instead of
truncated: rounding a share count misstates a holding, and rounding a price
would break the check the amount is verified against.
Fee and tax are always stored as deductions from a gross, so a refunded tax is
a negative deduction, and an export that writes its fee as the negative
adjustment it made to the cash is normalized once, at import. Whether a cash
row's amount is already net of its tax, or a gross the deductions still apply
to, is a fact about the source and is decided there too.
Instruments are registered from the export, keyed by ISIN, with an ID derived
from the ISIN so re-importing never creates a second entry for one security. One
ISIN appears under several descriptions over the years and sometimes under the
ISIN itself; the most recent real description names it, and an import never
renames an instrument that already exists. The name is editable display text;
the ISIN is identity and cannot be changed.
ISIN appears under several names over the years and sometimes under the ISIN
itself; the most recent real name wins, and an import never renames an
instrument that already exists. The name is editable display text; the ISIN is
identity and cannot be changed. Crypto is held under the ISIN-shaped identifier
the broker issues for it, so it needs no separate identity scheme.
A broker reuses one reference across every leg of an economic event: the cash
and position sides of a corporate action arrive with the same reference byte for
@@ -474,6 +495,69 @@ Transaction identity therefore includes the event and its instrument. The
reference itself also embeds an account-level identifier that repeats across
unrelated events, so it is evidence of an event, never of a transaction.
Trade Republic exports and their differences
-------------------------------------------
Trade Republic exports (traderepublic_csv) are recognized by their full column
set: datetime, date, account_type, category, type, asset_class, name, symbol,
shares, price, amount, fee, tax, currency, original_amount, original_currency,
fx_rate, description, transaction_id, counterparty_name, counterparty_iban,
payment_reference, mcc_code.
Nine row types, classified by category and type:
category type cash position
CASH TRANSFER_INBOUND amount -
CASH TRANSFER_INSTANT_INBOUND amount -
CASH TRANSFER_OUTBOUND amount -
CASH TRANSFER_INSTANT_OUTBOUND amount -
CASH INTEREST_PAYMENT amount -
CASH DIVIDEND amount -
CASH TAX_OPTIMIZATION amount -
TRADING BUY amount +shares
TRADING SELL amount -shares
where cash is in every case amount minus the fee and tax deducted from it.
No row type moves a position without moving cash, so the cash-neutral class
that Scalable's corporate actions and depot transfers belong to does not arise.
Three conventions are the opposite of Scalable's, and each one moves money if
read the other way round:
- fee and tax are signed adjustments to cash, not deductions. A one euro
order fee is written -1.00 and withheld tax -4.33, so both are negated at
import and the journal keeps its single convention.
- a cash row's amount is the gross, not the net. Interest of 16.46 with -4.33
of tax credits 12.13.
- a TAX_OPTIMIZATION row carries zero in the amount column and its money in
the tax column, signed both ways. Read as cash, all of them move nothing;
read correctly, they are the loss-offset pot settling, in either direction.
A DIVIDEND row populates the share column with the holding the dividend was
paid on, not with a position change. Adding it would double the holding, so it
is read as the attribution it is and otherwise discarded.
The security identifier is the symbol column when that is an ISIN, and
otherwise the one ISIN the description names: crypto carries a bare ticker in
the column and its identifier only in the text. A row that moves a position
and resolves to neither is refused.
The counterparty of a transfer is the counterparty_iban column when populated,
else the IBAN the description carries in parentheses, else the account's
configured settlement IBAN. Free text contributes only a value shaped like an
IBAN, so a description naming no account contributes nothing.
The booking date is the date column exactly as printed. The datetime column is
UTC while the date column is local, so they disagree for rows booked late in
the evening; deriving the date from the timestamp moves those rows a day back.
Only account_type DEFAULT imports. One export covers one account, and a second
account type in the same file would merge two cash balances into one.
original_amount, original_currency and fx_rate are informational: settlement is
in the currency column, which must match the account's. payment_reference and
mcc_code are unused - no card rows appear in this export type, and if they ever
do they are spending with a merchant, not broker activity.
Broker facts carry enrichment kind "investment". Like a transfer it has no
category and no merchant, it is excluded from spending and income analytics and
from bulk reclassification, and the AI never sees it. Crucially, a broker fact