The preview endpoint held one HTTP request open while classifying
serially at three-second pacing, so any real range meant minutes of a
grayed-out button and per-row errors were invisible until the loop
ended. Analyse now starts a single background run against its own
snapshot; a progress endpoint reports analysed counts, proposed
changes and errors as they happen, and the page polls it with a
progress bar, pace-based estimate and a Stop button. Navigating away
no longer orphans the run: the page re-attaches to it on return.
A run that has produced no successful proposal and fails three times
in a row with the identical error stops early and reports that error,
so a wrong key or unsupported model surfaces in seconds instead of
repeating across the whole paced range.
Also normalize a null settings.private_names, which crashed the whole
UI on a workspace that had never saved preferences.
The deployed service crash-looped 83 times on "config.toml:3: unknown setting
\"include_amount\"". The classification redesign retired that preference from
both the reader and the writer, but /var/lib/finance-duck/config.toml was
written by the previous binary and still names it, and Open refuses any key its
switch does not recognise. So the new binary would not start against its own
settings file: nixos-rebuild switched successfully, systemd restarted the unit
until it gave up, and the updater's health check failed - a deployment error
whose cause was neither the build nor the code that was deployed.
Retired settings are now read and discarded, and the next SaveSettings rewrites
the file without them. An unrecognised key is still refused, because a
misspelled preference that loads silently is a preference the user believes is
in force. Every future removal adds its key to the same list rather than
stranding the deployments that already hold it.
Verified by running the built binary against a config.toml carrying exactly the
line the host has: it starts and /api/health answers 200, where the previous
binary exited 1. The regression test fails with "retired setting must not stop
startup" before the change, and it still requires classify_on_imports to be
rejected.
A second broker export is recognized locally, by its full column set, and read
through the same pipeline: detection and parsing now dispatch on the format, so
the upload path, the review dialog, deduplication, the journal and the Wealth
report are unchanged. Its nine row types cover cash transfers, interest,
dividends, tax settlements and trades in funds, shares and crypto; none of them
moves a position without moving cash, so the cash-neutral class that Scalable's
corporate actions belong to does not arise here.
Three of its conventions are the opposite of the export already supported, and
reading any of them the other way round moves money. Fee and tax are the signed
adjustments it made to the cash rather than deductions from a gross, so a one
euro order fee arrives as -1.00 and is negated at import; the journal keeps one
convention and the domain never learns that two exist. A cash row's amount is
the gross, not the net, so interest of 16.46 with -4.33 of tax credits 12.13 -
where the other export states its cash already net and its tax is recorded and
never applied. Whether a cash row carries a gross now decides which of those it
was, which also makes the first kind's settlement checkable and stops the Wealth
report from claiming a figure was left unapplied when it was not. And a
TAX_OPTIMIZATION row puts zero in the amount column and its money in the tax
column, signed both ways: read as cash, all six in a real export move nothing.
Two more rows lie about their own columns. A dividend fills the share column
with the holding the dividend was paid on, not with a position change, so adding
it would double the holding. Crypto carries a bare ticker in the symbol column
and its ISIN-shaped identifier only in the description, so the identifier is
taken from the symbol when that is an ISIN and otherwise from the one the
description names; a position row resolving to neither is refused rather than
attached to a guess.
The shares-times-price check now holds a gross to the precision the export
stated it at rather than to four places. This export prints the notional rounded
to cents, and 29 of 59 real trades do not land on a whole cent: demanding
exactness rejected half a portfolio. One unit of the stated precision is still
four orders of magnitude tighter than the misplaced separator the check exists
to catch, and where an export prints the full product the check stays exact.
A unit price moves from money to the eight-place quantity type, because a crypto
price is quoted to six and rounding it would break the check the amount is
verified against. Trailing zeros are dropped before any precision test: this
export pads a six-place price to ten, and the padding would otherwise exhaust
the precision the value needs.
A transfer's counterparty comes from the export's own IBAN column when it has
one, from the IBAN the description names in parentheses when it does not, and
from the account's configured settlement IBAN when neither names anything. Free
text contributes only a value shaped like an IBAN. Without this, 108 transfers
stay unpaired and their bank-side counterparts read as spending and income.
Verified end to end against a real export: 26 rows import to a cash balance of
32187.02 matching the figure computed by hand from the source rows, all four
positions close at exactly zero, and every trade satisfies its own arithmetic.
An account now has a kind, and an investment account holds positions as well as
cash. A broker row is not a new entity: it is a bank fact with an optional
position leg, so deduplication, the journal, fact immutability, the DuckDB
projection and the transactions view carry it unchanged. Facts.Amount stays the
cash leg and is zero on the rows that move only a position.
Scalable Capital exports are recognized locally as a fourth format, read by
their own parser because a column mapping cannot describe them: the amount
column is settled cash on a cash row, a gross to be netted on a trade, and a
position valuation that must never touch cash on a corporate action or a depot
transfer. A cash amount is already net of the tax the broker withheld or
refunded, so that tax is recorded on the fact and never subtracted a second
time; treating a corporate action's valuation as money conjures cash, and a
depot switch would do it once per instrument. The share column is signed only
for those two types, so buys and sells take their direction from the type. Every
security row is checked against shares times price at 128-bit width, because a
lost decimal separator survives every other check. An unknown status, type or
assetType, a foreign currency, a missing ISIN, or one failed check rejects the
whole file with the record number.
Instruments live in instruments.finance, keyed by ISIN with an ID derived from
it, so re-importing never registers a security twice. One ISIN appears under
several broker descriptions over the years and sometimes under the ISIN itself:
the most recent real description names it, and an import never renames one that
already exists. A broker also reuses a single reference across every leg of one
event, so transaction identity includes the event and its instrument.
domain.Fallback returns kind "investment" for any fact carrying a position leg,
so no broker row reaches the sign-based branch. That single rule is what stops
an unmatched deposit from counting as income and a broker fee from counting as
household spending; the monthly PRIME fee and its matching credit now cancel in
clearing:investments with no configuration at all. Investment rows are excluded
from spending analytics, from bulk reclassification and from the model, exactly
as transfers are.
Equal competing transfers are paired instead of skipped. Every connected
component of the candidate graph is a complete bipartite graph between two fixed
accounts at one amount and currency, so every pairing produces the same
accounts, kinds and postings and only the displayed counterpart differs.
Refusing to choose was the expensive option: both legs fell through to the
sign-based fallback and appeared as spending and income that never happened.
Pairing follows the nearest booking date, then the transaction ID, so iteration
order decides nothing. POST /api/transactions/{id}/transfer rewrites the old and
the new pair in one commit, because reciprocity is validated and a half-applied
link is an invalid dataset, and the matcher now skips any record classified
manually so a hand-made link or unlink outlives the next import.
Wealth reports each account's cash and positions from the journal rather than
the index, with named checks - row arithmetic, cash never negative, holdings
never negative - because it exists to be compared against the figures a broker
shows on its own screen. A negative holding means the imported history is
partial. Share counts are exact to eight places; a reinvested distribution
quoted to six is rounded to money's four and the residue is reported rather than
hidden. Market prices, market value, net worth over time, FIFO lot accounting,
realised gains and currency conversion are deliberately absent.
One named syncInterval replaces the two 24-hour literals in the scheduler: a
successful run now waits 12 hours, and an account becomes due again 12 hours
after its last successful sync. Two background fetches a day stay inside Enable
Banking's documented allowance of roughly four per account per day, which a
failing sync's hourly retries also draw from, so the bank rate limits that
prompted this are no more likely than before.
Two of three banks were only pacing us, yet the dashboard demanded attention,
printed four nested wrappers and a nanosecond UTC deadline, and the scheduler
retried hourly into a refusal whose end time the bank had already given.
A rate limit now carries its retry time as data: Status.SyncRetryAt is set when
every failure is self-clearing, the connection reports rate_limited with that
deadline, the dashboard says synchronization resumes by itself and renders the
time in the browser's zone, and the scheduler sleeps until the deadline instead
of spending hourly session checks. Sync now still tries immediately.
The third bank's "transaction retrieval failed" hid its cause. Provider
failures Finance Duck determines itself are typed as banking.ProviderError,
so an unreachable provider, a timeout or an unusable response, such as a booked
transaction without a booking date, is reported instead of the opaque fallback.
Provider response text still never reaches the message.
Classification preferences gains "Classify newly imported transactions with
AI", stored as classify_on_import in config.toml and on by default, so existing
configurations keep their behaviour. It covers CSV imports and bank
synchronization alike.
With it off, no import path contacts the provider: classification falls to the
new provider-free rules path, where an opted-in merchant rule still applies its
category and tags, an alias match still attaches its merchant, and everything
else arrives on the editable fallback without a provenance error that would
suggest the provider had failed. Analyse remains available on demand.
CSV import is now mapping-driven: N26, ING (metadata preamble, Windows-1252,
German decimals) and Kontist exports are recognized locally, and any other
layout can have its columns proposed by the configured model from a sample in
which letters are replaced by x and digits by 0. Proposals are untrusted: every
column must name a supplied header, money must come from one signed column or
one debit/credit pair, and formats must be from a closed list.
Uploading no longer imports. /api/import is replaced by prepare/confirm/cancel:
prepare parses, deduplicates and previews the exact facts, and only confirming
at the reviewed revision writes them. ING and AI-mapped facts carry no
transaction reference, because repeating SEPA mandate references must never
become a transaction identity.
Kontist authorized but shared no accounts: psu_type was hardcoded to
personal, and Enable Banking documents that a psu_type mismatch can
yield a consent without the expected accounts. The bank listing now
reports each institution's supported psu_types, the connect form offers
only those, the chosen type reaches POST /auth, and an unsupported
combination is refused before the user is sent to a bank. The choice is
stored per consent so reconnecting reuses it; consents predating the
choice stay personal.
Also repairs the frontend derivation, which the Montserrat dependency
broke: npmDepsHash was stale and web/public was missing from the
fileset, so the traced duck icon never reached the built assets.
Three defects made new connections silently vanish while removed
accounts returned:
- A single shared account the journal cannot represent (securities or
card entries without IBAN, stable identification or currency) aborted
the entire consent. Usable accounts are now linked and the rest
counted and reported.
- A consent that linked nothing was stored, redirected as success and
later reaped by session recovery. It now fails with the reason.
- Callback failures rendered a bare JSON error page and were never
logged. They now log and redirect into the app with the reason shown.
- Deleting an account left its session binding, so the next connect or
sync recovered the binding and re-added the account. Account deletion
now releases bindings, consents and cursors before committing.
GET /api/banking/institutions lists the banks Enable Banking can connect
for a country (personal AIS, connectable consents only), with logos
restricted to https Enable Banking hosts to match the CSP image
allowlist. The connect form offers a filterable dropdown with bank
logos, falling back to the previous free-text input when the list is
unavailable or banking is not configured.
Manual history imports failed opaquely once a bank capped lookback on an
established consent (N26 rejects date_from beyond ~90 days with
WRONG_TRANSACTIONS_PERIOD). Backfill now requests the documented longest
fetching strategy, reports the coverage the bank actually provided, and
non-2xx responses surface allowlisted documented error codes instead of a
generic fallback. Dead-session codes map to reconnection. Failed syncs
retry hourly so a stale sync banner no longer persists for a day.
Lars Nolden