Compare commits
2
Commits
dece0d5b79
...
673cbf917b
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
673cbf917b | ||
|
|
b3e1c65a82 |
+16
-2
@@ -275,13 +275,27 @@ errors are displayed separately from expired consent.
|
||||
|
||||
Only booked transactions are persisted. Each new account initially requests the
|
||||
selected number of calendar months (12 by default); the bank may provide less.
|
||||
Daily sync deliberately overlaps each account's last successful sync by 14 days.
|
||||
Automatic sync runs twice a day, every 12 hours from the last successful run,
|
||||
and deliberately overlaps each account's last successful sync by 14 days.
|
||||
Per-account cursors prevent another account's recent sync from skipping a new
|
||||
account's history. Reconnection preserves the saved history choice and existing
|
||||
cursors. Changing the choice or reconnecting does not backfill already-synced
|
||||
accounts. Older records can be imported using CSV. Older saved consents without
|
||||
a history choice use 12 months for accounts that have no successful-sync cursor.
|
||||
A failed provider call retains local data and is retried by the daily scheduler;
|
||||
A failed provider call retains local data. Failures whose cause Finance Duck
|
||||
determines locally are named: a bank rate limit with its retry time, an expired
|
||||
consent, an HTTP status, an unreachable provider, or a response the journal
|
||||
cannot use (for example a booked transaction without a booking date). Only an
|
||||
unrecognized cause falls back to "transaction retrieval failed". Provider
|
||||
response text is never shown.
|
||||
|
||||
While every failing bank has named its own retry time, the account is reported
|
||||
as waiting, not broken: the dashboard says synchronization retries by itself,
|
||||
the account card shows a rate-limit badge with that time, and the scheduler
|
||||
sleeps until the deadline instead of spending hourly session checks on a
|
||||
refusal it already knows about. Sync now still tries immediately. Any failure
|
||||
without such a deadline keeps the hourly retry. Retry times are persisted with
|
||||
the sync state in whole seconds and rendered in the browser's time zone.
|
||||
Sync now can retry sooner. Balances are fetched
|
||||
on demand, with exact amount/currency/type values, rather than inferred from an
|
||||
incomplete historical journal.
|
||||
|
||||
@@ -142,10 +142,12 @@ Open **Accounts → Connect your bank**:
|
||||
|
||||
Finance Duck verifies the callback state, exchanges the returned code for a `session_id`, stores the session locally with restrictive permissions, and wakes the synchronization worker immediately. You do not need to copy the code or session ID manually.
|
||||
|
||||
Initial synchronization requests the selected number of **calendar months of booked transactions per account**, defaulting to **12 months**. The bank may provide less history. The choice is saved with the bank connection and reused on reconnection. Subsequent daily synchronization overlaps each account's last successful sync by **14 days**. **Sync now** starts a manual synchronization. Existing accounts keep their successful-sync cursors: changing the history choice or reconnecting does **not** backfill them. Older history can be imported with CSV.
|
||||
Initial synchronization requests the selected number of **calendar months of booked transactions per account**, defaulting to **12 months**. The bank may provide less history. The choice is saved with the bank connection and reused on reconnection. Automatic synchronization then runs **twice a day**, every **12 hours** after the last successful run, overlapping each account's last successful sync by **14 days**. **Sync now** starts a manual synchronization at any time. Existing accounts keep their successful-sync cursors: changing the history choice or reconnecting does **not** backfill them. Older history can be imported with CSV.
|
||||
|
||||
**HTTP 429 is a provider rate limit, not evidence that bank consent has expired.** Bank reads honor `Retry-After` and use bounded exponential retries. A longer or exhausted limit pauses further requests until the reported retry time; failed accounts keep their previous sync cursors and imported data. Session checks use the saved account metadata rather than fetching every account's details again. A failed session is reported once instead of also marking each of its accounts unavailable. After the cooldown, **Sync now** can retry; the warning clears after a successful sync. One-time authorization and code-exchange requests are never automatically replayed.
|
||||
|
||||
**A rate-limited sync is a wait, not a fault.** While every failing bank has supplied a retry time, the dashboard reports that synchronization retries by itself after that moment, the account card shows a rate-limit badge instead of a connection error, and the background scheduler sleeps until the deadline rather than retrying hourly into a refusal it already knows about. **Sync now** still tries immediately. Any failure without a supplied deadline keeps the hourly retry, and its cause is named where Finance Duck can determine it locally: an expired consent, an HTTP status, an unreachable provider, or a response it cannot use, such as a booked transaction without a booking date. Provider response text is never displayed.
|
||||
|
||||
Manual **Sync now**, **Import older history**, and balance requests forward the requesting user's IP, browser User-Agent, and available Accept headers to the bank as PSU metadata. Scheduled syncs never claim a user is present. This distinction matters: [Enable Banking documents background limits of roughly four fetches per day at many banks](https://enablebanking.com/docs/faq/#why-am-i-getting-429-response-code-are-there-rate-limits-for-the-api). A confirmed background `ASPSP_RATE_LIMIT_EXCEEDED` defers that account's affected endpoint for at least six hours, preserving longer provider hints; it does not block an eligible user-initiated fetch. General provider limits still apply to both. Bank requests are spaced by at least one second, with longer learned spacing after throttling.
|
||||
|
||||
### Import older history for a connected account
|
||||
|
||||
+9
-4
@@ -29,9 +29,12 @@ type Settings struct {
|
||||
ClassifyOnImport bool `json:"classify_on_import"`
|
||||
}
|
||||
type Status struct {
|
||||
SyncError string `json:"sync_error"`
|
||||
IndexError string `json:"index_error"`
|
||||
LastSync string `json:"last_sync"`
|
||||
SyncError string `json:"sync_error"`
|
||||
IndexError string `json:"index_error"`
|
||||
LastSync string `json:"last_sync"`
|
||||
// SyncRetryAt is set only when every sync failure is a bank rate limit that
|
||||
// clears on its own; it is the earliest time an automatic retry is allowed.
|
||||
SyncRetryAt string `json:"sync_retry_at,omitempty"`
|
||||
BankingConfigured bool `json:"banking_configured"`
|
||||
AIConfigured bool `json:"ai_configured"`
|
||||
}
|
||||
@@ -49,6 +52,7 @@ type operational struct {
|
||||
Sessions []banking.Session `json:"sessions"`
|
||||
LastSync string `json:"last_sync"`
|
||||
SyncError string `json:"sync_error"`
|
||||
SyncRetryAt string `json:"sync_retry_at,omitempty"`
|
||||
Consents map[string]Consent `json:"consents"`
|
||||
AccountSync map[string]string `json:"account_sync"`
|
||||
BankingScope string `json:"banking_scope"`
|
||||
@@ -166,7 +170,8 @@ func (a *App) snapshot(ctx context.Context) (State, error) {
|
||||
a.indexError = ""
|
||||
}
|
||||
}
|
||||
return State{Data: d, Revision: rev, Settings: a.settings, Sessions: copySessions(a.ops.Sessions), CallbackURL: a.callbackURL, BankingAppID: a.bankingSettings.AppID, Connections: a.connections(d), Status: Status{SyncError: a.ops.SyncError, LastSync: a.ops.LastSync, IndexError: a.indexError, BankingConfigured: a.bank != nil, AIConfigured: a.classifier.APIKey != ""}}, nil
|
||||
status := Status{SyncError: a.ops.SyncError, SyncRetryAt: a.ops.SyncRetryAt, LastSync: a.ops.LastSync, IndexError: a.indexError, BankingConfigured: a.bank != nil, AIConfigured: a.classifier.APIKey != ""}
|
||||
return State{Data: d, Revision: rev, Settings: a.settings, Sessions: copySessions(a.ops.Sessions), CallbackURL: a.callbackURL, BankingAppID: a.bankingSettings.AppID, Connections: a.connections(d), Status: status}, nil
|
||||
}
|
||||
func (a *App) Snapshot(ctx context.Context) (State, error) {
|
||||
a.mu.Lock()
|
||||
|
||||
@@ -31,6 +31,7 @@ func (a *App) clearBankingSessions(scope string) {
|
||||
a.ops.AccountSync = make(map[string]string)
|
||||
a.ops.LastSync = ""
|
||||
a.ops.SyncError = ""
|
||||
a.ops.SyncRetryAt = ""
|
||||
a.ops.BankingScope = scope
|
||||
}
|
||||
|
||||
|
||||
@@ -27,6 +27,8 @@ type Consent struct {
|
||||
HistoryMonths int `json:"history_months"`
|
||||
Error string `json:"error,omitempty"`
|
||||
NeedsReconnect bool `json:"needs_reconnect"`
|
||||
// RetryAt is the bank's own retry time while it rate limits this consent.
|
||||
RetryAt string `json:"retry_at,omitempty"`
|
||||
}
|
||||
type Connection struct {
|
||||
AccountID string `json:"account_id"`
|
||||
@@ -37,6 +39,7 @@ type Connection struct {
|
||||
Status string `json:"status"`
|
||||
ValidUntil string `json:"valid_until"`
|
||||
Error string `json:"error"`
|
||||
RetryAt string `json:"retry_at,omitempty"`
|
||||
}
|
||||
|
||||
// psuType keeps legacy consents, which predate the choice, on the personal
|
||||
@@ -79,6 +82,7 @@ func (a *App) connections(d domain.Dataset) []Connection {
|
||||
}
|
||||
c.ValidUntil = session.ValidUntil
|
||||
c.Error = meta.Error
|
||||
c.RetryAt = meta.RetryAt
|
||||
c.Status = "connected"
|
||||
expiry, err := time.Parse(time.RFC3339, session.ValidUntil)
|
||||
if meta.NeedsReconnect || err != nil || !expiry.After(time.Now()) {
|
||||
@@ -86,6 +90,9 @@ func (a *App) connections(d domain.Dataset) []Connection {
|
||||
if c.Error == "" {
|
||||
c.Error = "Bank consent expired; reconnect to resume automatic imports"
|
||||
}
|
||||
} else if meta.RetryAt != "" {
|
||||
// A rate limit is the bank pacing us, not a broken connection.
|
||||
c.Status = "rate_limited"
|
||||
} else if meta.Error != "" {
|
||||
c.Status = "error"
|
||||
}
|
||||
|
||||
+92
-14
@@ -605,10 +605,12 @@ func (a *App) Balances(ctx context.Context, id string) ([]banking.Balance, error
|
||||
|
||||
// Only typed, locally generated errors are safe to expose; provider errors may
|
||||
// wrap private response data even when their underlying cause is recognizable.
|
||||
// The fallback is a last resort: an unnamed cause leaves an operator with
|
||||
// nothing to act on.
|
||||
func bankFailure(err error, fallback string) error {
|
||||
var background *banking.BackgroundQuotaError
|
||||
if errors.As(err, &background) {
|
||||
return fmt.Errorf("Enable Banking: %w", background)
|
||||
return background
|
||||
}
|
||||
var limited *ratelimit.RateLimitError
|
||||
if errors.As(err, &limited) {
|
||||
@@ -625,9 +627,25 @@ func bankFailure(err error, fallback string) error {
|
||||
if errors.As(err, &consent) {
|
||||
return consent
|
||||
}
|
||||
var provider *banking.ProviderError
|
||||
if errors.As(err, &provider) {
|
||||
return provider
|
||||
}
|
||||
return errors.New(fallback)
|
||||
}
|
||||
|
||||
// syncRetryAt reports the bank's own retry time for a failure that clears
|
||||
// itself. A rate limit without a usable deadline is not treated as waiting:
|
||||
// nothing would ever announce that it had expired.
|
||||
func syncRetryAt(err error) (time.Time, bool) {
|
||||
var limited *ratelimit.RateLimitError
|
||||
if !errors.As(err, &limited) {
|
||||
return time.Time{}, false
|
||||
}
|
||||
at := limited.RetryAt()
|
||||
return at, !at.IsZero()
|
||||
}
|
||||
|
||||
func (a *App) Sync(ctx context.Context) (State, error) {
|
||||
a.mu.Lock()
|
||||
defer a.mu.Unlock()
|
||||
@@ -639,6 +657,10 @@ func (a *App) Sync(ctx context.Context) (State, error) {
|
||||
return State{}, err
|
||||
}
|
||||
var failures []string
|
||||
// waitUntil is the earliest time the banks themselves allow a retry, used
|
||||
// only while every failure is such a self-clearing rate limit.
|
||||
var waitUntil time.Time
|
||||
waiting := true
|
||||
for i := range a.ops.Sessions {
|
||||
connectAccounts(&s.Data, &a.ops.Sessions[i], false)
|
||||
}
|
||||
@@ -684,6 +706,15 @@ func (a *App) Sync(ctx context.Context) (State, error) {
|
||||
}
|
||||
if e != nil {
|
||||
meta.Error = bankFailure(e, "bank connection unavailable; retry synchronization").Error()
|
||||
meta.RetryAt = ""
|
||||
if at, ok := syncRetryAt(e); ok {
|
||||
meta.RetryAt = at.UTC().Format(time.RFC3339)
|
||||
if waitUntil.IsZero() || at.Before(waitUntil) {
|
||||
waitUntil = at
|
||||
}
|
||||
} else {
|
||||
waiting = false
|
||||
}
|
||||
meta.NeedsReconnect = errors.Is(e, banking.ErrReconnect)
|
||||
a.ops.Consents[session.ID] = meta
|
||||
failures = append(failures, meta.Institution+": "+meta.Error)
|
||||
@@ -691,6 +722,7 @@ func (a *App) Sync(ctx context.Context) (State, error) {
|
||||
continue
|
||||
}
|
||||
meta.Error = ""
|
||||
meta.RetryAt = ""
|
||||
meta.NeedsReconnect = false
|
||||
a.ops.Consents[session.ID] = meta
|
||||
a.ops.Sessions[i].ValidUntil = current.ValidUntil
|
||||
@@ -712,9 +744,11 @@ func (a *App) Sync(ctx context.Context) (State, error) {
|
||||
}
|
||||
if !validAccounts[account.ID] {
|
||||
failures = append(failures, account.DisplayName+": bank connection unavailable")
|
||||
waiting = false
|
||||
if sessionID != "" {
|
||||
meta := a.ops.Consents[sessionID]
|
||||
meta.Error = banking.ErrReconnect.Error()
|
||||
meta.RetryAt = ""
|
||||
meta.NeedsReconnect = true
|
||||
a.ops.Consents[sessionID] = meta
|
||||
}
|
||||
@@ -731,6 +765,15 @@ func (a *App) Sync(ctx context.Context) (State, error) {
|
||||
if e != nil {
|
||||
meta := a.ops.Consents[sessionID]
|
||||
meta.Error = bankFailure(e, "transaction retrieval failed; retry synchronization").Error()
|
||||
meta.RetryAt = ""
|
||||
if at, ok := syncRetryAt(e); ok {
|
||||
meta.RetryAt = at.UTC().Format(time.RFC3339)
|
||||
if waitUntil.IsZero() || at.Before(waitUntil) {
|
||||
waitUntil = at
|
||||
}
|
||||
} else {
|
||||
waiting = false
|
||||
}
|
||||
meta.NeedsReconnect = meta.NeedsReconnect || errors.Is(e, banking.ErrReconnect)
|
||||
a.ops.Consents[sessionID] = meta
|
||||
failures = append(failures, account.DisplayName+": "+meta.Error)
|
||||
@@ -739,6 +782,7 @@ func (a *App) Sync(ctx context.Context) (State, error) {
|
||||
result, e := a.importFacts(ctx, s, facts)
|
||||
if e != nil {
|
||||
failures = append(failures, account.DisplayName+": "+e.Error())
|
||||
waiting = false
|
||||
s, err = a.snapshot(ctx)
|
||||
if err != nil {
|
||||
return State{}, err
|
||||
@@ -749,14 +793,54 @@ func (a *App) Sync(ctx context.Context) (State, error) {
|
||||
a.ops.AccountSync[account.ID] = now.Format(time.RFC3339)
|
||||
}
|
||||
a.ops.SyncError = strings.Join(failures, "; ")
|
||||
a.ops.SyncRetryAt = ""
|
||||
if len(failures) == 0 {
|
||||
a.ops.LastSync = now.Format(time.RFC3339)
|
||||
} else if waiting && !waitUntil.IsZero() {
|
||||
// Every bank named its own retry time: this is a wait, not a fault.
|
||||
a.ops.SyncRetryAt = waitUntil.UTC().Format(time.RFC3339)
|
||||
}
|
||||
if err = a.saveOps(); err != nil {
|
||||
return State{}, err
|
||||
}
|
||||
return a.snapshot(ctx)
|
||||
}
|
||||
|
||||
// syncInterval is how often connected accounts synchronize on their own. Twice
|
||||
// a day halves how long a booking can sit unseen while staying inside Enable
|
||||
// Banking's documented background allowance of roughly four fetches per day per
|
||||
// account, which a failing sync's hourly retries also draw from.
|
||||
const syncInterval = 12 * time.Hour
|
||||
|
||||
// syncSchedule decides whether an automatic sync may run now, and how long to
|
||||
// wait otherwise. While a bank has named its own retry time, waiting is the
|
||||
// only useful action: retrying earlier spends session-status calls on a refusal
|
||||
// that is already known. A manual request always proceeds.
|
||||
func syncSchedule(now time.Time, ops operational, force bool) (time.Duration, bool) {
|
||||
if retry, err := time.Parse(time.RFC3339, ops.SyncRetryAt); err == nil && !force && now.Before(retry) {
|
||||
return min(retry.Sub(now)+time.Minute, time.Hour), false
|
||||
}
|
||||
last, err := time.Parse(time.RFC3339, ops.LastSync)
|
||||
if force || err != nil || ops.SyncError != "" || now.Sub(last) >= syncInterval {
|
||||
return 0, true
|
||||
}
|
||||
return time.Minute, false
|
||||
}
|
||||
|
||||
// syncBackoff spaces the next attempt after a sync. A failure with a known bank
|
||||
// retry time waits for it; other failures retry hourly so transient provider
|
||||
// problems clear without waiting for the next scheduled run, while bounding
|
||||
// unattended traffic.
|
||||
func syncBackoff(now time.Time, ops operational) time.Duration {
|
||||
if ops.SyncError == "" {
|
||||
return syncInterval
|
||||
}
|
||||
if retry, err := time.Parse(time.RFC3339, ops.SyncRetryAt); err == nil && now.Before(retry) {
|
||||
return min(retry.Sub(now)+time.Minute, syncInterval)
|
||||
}
|
||||
return time.Hour
|
||||
}
|
||||
|
||||
func (a *App) RunScheduler(ctx context.Context) {
|
||||
timer := time.NewTimer(time.Minute)
|
||||
defer timer.Stop()
|
||||
@@ -771,25 +855,19 @@ func (a *App) RunScheduler(ctx context.Context) {
|
||||
}
|
||||
a.mu.Lock()
|
||||
configured := a.bank != nil
|
||||
last, err := time.Parse(time.RFC3339, a.ops.LastSync)
|
||||
failed := a.ops.SyncError != ""
|
||||
wait, due := syncSchedule(time.Now(), a.ops, force)
|
||||
a.mu.Unlock()
|
||||
due := force || err != nil || failed || time.Since(last) >= 24*time.Hour
|
||||
if !configured || !due {
|
||||
timer.Reset(time.Minute)
|
||||
if wait <= 0 {
|
||||
wait = time.Minute
|
||||
}
|
||||
timer.Reset(wait)
|
||||
continue
|
||||
}
|
||||
a.Sync(ctx)
|
||||
a.mu.Lock()
|
||||
failed = a.ops.SyncError != ""
|
||||
wait = syncBackoff(time.Now(), a.ops)
|
||||
a.mu.Unlock()
|
||||
if failed {
|
||||
// A failed sync leaves its persisted error banner behind. Retry
|
||||
// hourly so transient provider failures clear without waiting a
|
||||
// day, while bounding unattended traffic toward the provider.
|
||||
timer.Reset(time.Hour)
|
||||
} else {
|
||||
timer.Reset(24 * time.Hour)
|
||||
}
|
||||
timer.Reset(wait)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -273,7 +273,7 @@ func TestSyncMissingMembershipStillRejectsAccount(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestSyncTransactionFailuresPreserveProgressAndSafeErrors(t *testing.T) {
|
||||
for _, scenario := range []string{"rate limit", "background rate limit", "reconnect", "private response"} {
|
||||
for _, scenario := range []string{"rate limit", "background rate limit", "reconnect", "private response", "unusable response"} {
|
||||
t.Run(scenario, func(t *testing.T) {
|
||||
a, s, b := backfillApp(t)
|
||||
s = seed(t, a, s)
|
||||
@@ -292,6 +292,9 @@ func TestSyncTransactionFailuresPreserveProgressAndSafeErrors(t *testing.T) {
|
||||
b.fetchErr = fmt.Errorf("private provider response: %w", banking.ErrReconnect)
|
||||
case "private response":
|
||||
b.fetchErr = errors.New("private provider response")
|
||||
case "unusable response":
|
||||
unusable := &banking.ProviderError{Detail: "a booked transaction has no valid booking date"}
|
||||
b.fetchErr = fmt.Errorf("private provider response: %w", unusable)
|
||||
}
|
||||
failed, err := a.Sync(context.Background())
|
||||
if err != nil {
|
||||
@@ -313,6 +316,43 @@ func TestSyncTransactionFailuresPreserveProgressAndSafeErrors(t *testing.T) {
|
||||
if !reflect.DeepEqual(s.Data, failed.Data) || !reflect.DeepEqual(cursors, a.ops.AccountSync) || a.ops.LastSync != old {
|
||||
t.Fatal("failed retrieval imported partial data or advanced synchronization")
|
||||
}
|
||||
rateLimited := strings.Contains(scenario, "rate limit")
|
||||
// A bank that named its own retry time is a wait, not a fault: the
|
||||
// UI and the scheduler both rely on this distinction.
|
||||
if (failed.Status.SyncRetryAt != "") != rateLimited {
|
||||
t.Fatalf("waiting state is wrong for %s: retry at %q", scenario, failed.Status.SyncRetryAt)
|
||||
}
|
||||
for _, connection := range failed.Connections {
|
||||
if connection.Status == "local" {
|
||||
continue
|
||||
}
|
||||
want := "error"
|
||||
switch {
|
||||
case rateLimited:
|
||||
want = "rate_limited"
|
||||
case scenario == "reconnect":
|
||||
want = "reconnect_required"
|
||||
}
|
||||
if connection.Status != want || (connection.RetryAt != "") != rateLimited {
|
||||
t.Fatalf("connection reported %q with retry %q, want %q", connection.Status, connection.RetryAt, want)
|
||||
}
|
||||
}
|
||||
if rateLimited {
|
||||
at, e := time.Parse(time.RFC3339, failed.Status.SyncRetryAt)
|
||||
if e != nil || !at.After(time.Now()) {
|
||||
t.Fatalf("unusable retry deadline %q: %v", failed.Status.SyncRetryAt, e)
|
||||
}
|
||||
// Whole seconds, once: operators read this message.
|
||||
if !strings.Contains(meta.Error, at.UTC().Format(time.RFC3339)) || strings.Count(meta.Error, "429") != 1 {
|
||||
t.Fatalf("rate limit message is not legible: %q", meta.Error)
|
||||
}
|
||||
}
|
||||
if scenario == "unusable response" && (!strings.Contains(meta.Error, "cannot use") || !strings.Contains(meta.Error, "booking date")) {
|
||||
t.Fatalf("unusable provider response was reduced to an opaque failure: %q", meta.Error)
|
||||
}
|
||||
if scenario == "private response" && !strings.Contains(meta.Error, "transaction retrieval failed") {
|
||||
t.Fatalf("unrecognized failure lost its fallback: %q", meta.Error)
|
||||
}
|
||||
b.fetchErr = nil
|
||||
recovered, err := a.Sync(context.Background())
|
||||
if err != nil || recovered.Status.SyncError != "" || a.ops.Consents["current"].Error != "" || a.ops.Consents["current"].NeedsReconnect {
|
||||
@@ -321,3 +361,50 @@ func TestSyncTransactionFailuresPreserveProgressAndSafeErrors(t *testing.T) {
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// The scheduler must not spend session-status calls on a refusal the bank has
|
||||
// already scheduled, and must not sit on a deadline that has passed.
|
||||
func TestSyncSchedulingRespectsTheBanksOwnRetryTime(t *testing.T) {
|
||||
// Persisted deadlines carry whole seconds; compare against the same grid.
|
||||
now := time.Now().UTC().Truncate(time.Second)
|
||||
stale := now.Add(-48 * time.Hour).Format(time.RFC3339)
|
||||
waiting := operational{LastSync: stale, SyncError: "N26: rate limited", SyncRetryAt: now.Add(3 * time.Hour).Format(time.RFC3339)}
|
||||
broken := operational{LastSync: stale, SyncError: "Trade Republic: retrieval failed"}
|
||||
healthy := operational{LastSync: now.Format(time.RFC3339)}
|
||||
// Twice daily: still fresh at six hours, due again after thirteen.
|
||||
fresh := operational{LastSync: now.Add(-6 * time.Hour).Format(time.RFC3339)}
|
||||
overdue := operational{LastSync: now.Add(-13 * time.Hour).Format(time.RFC3339)}
|
||||
elapsed := operational{LastSync: stale, SyncError: waiting.SyncError, SyncRetryAt: now.Add(-time.Minute).Format(time.RFC3339)}
|
||||
cases := []struct {
|
||||
name string
|
||||
ops operational
|
||||
force bool
|
||||
wait time.Duration
|
||||
due bool
|
||||
}{
|
||||
{"waiting for the bank", waiting, false, time.Hour, false},
|
||||
{"manual sync during a wait", waiting, true, 0, true},
|
||||
{"deadline elapsed", elapsed, false, 0, true},
|
||||
{"failure without a deadline", broken, false, 0, true},
|
||||
{"recent success", healthy, false, time.Minute, false},
|
||||
{"halfway through the interval", fresh, false, time.Minute, false},
|
||||
{"interval elapsed", overdue, false, 0, true},
|
||||
}
|
||||
for _, tt := range cases {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
wait, due := syncSchedule(now, tt.ops, tt.force)
|
||||
if wait != tt.wait || due != tt.due {
|
||||
t.Fatalf("schedule = (%s, %t), want (%s, %t)", wait, due, tt.wait, tt.due)
|
||||
}
|
||||
})
|
||||
}
|
||||
if backoff := syncBackoff(now, waiting); backoff != 3*time.Hour+time.Minute {
|
||||
t.Fatalf("rate-limited backoff = %s, want the bank's own deadline", backoff)
|
||||
}
|
||||
if backoff := syncBackoff(now, broken); backoff != time.Hour {
|
||||
t.Fatalf("failure backoff = %s, want hourly retries", backoff)
|
||||
}
|
||||
if backoff := syncBackoff(now, healthy); backoff != 12*time.Hour {
|
||||
t.Fatalf("successful backoff = %s, want twice-daily synchronization", backoff)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -204,14 +204,43 @@ type BackgroundQuotaError struct {
|
||||
*ratelimit.RateLimitError
|
||||
}
|
||||
|
||||
// Error describes the pause in one line: operators need the bank's own retry
|
||||
// time, not a stack of nested rate-limit wrappers.
|
||||
func (e *BackgroundQuotaError) Error() string {
|
||||
return "background bank retrieval quota (normally six hours): " + e.RateLimitError.Error()
|
||||
if e.RetryAt().IsZero() {
|
||||
return "the bank is rate limiting background retrieval (HTTP 429); automatic retry is disabled, synchronize manually later"
|
||||
}
|
||||
return "the bank is rate limiting background retrieval (HTTP 429, normally six hours); automatic retry at " + e.RetryAt().UTC().Format(time.RFC3339)
|
||||
}
|
||||
|
||||
func (e *BackgroundQuotaError) Unwrap() error {
|
||||
return e.RateLimitError
|
||||
}
|
||||
|
||||
// ProviderError reports a provider interaction that failed for a reason Finance
|
||||
// Duck determined itself: the provider could not be reached, or its response
|
||||
// could not be used. Detail is written here and never taken from provider
|
||||
// response text, so callers may show the whole message to the user.
|
||||
type ProviderError struct {
|
||||
Unreachable bool
|
||||
Detail string
|
||||
cause error
|
||||
}
|
||||
|
||||
func (e *ProviderError) Error() string {
|
||||
message := "the bank's provider returned a response Finance Duck cannot use"
|
||||
if e.Unreachable {
|
||||
message = "the bank's provider could not be reached"
|
||||
}
|
||||
if e.Detail == "" {
|
||||
return message
|
||||
}
|
||||
return message + ": " + e.Detail
|
||||
}
|
||||
|
||||
// Unwrap keeps cancellation and deadline identity for callers that retry.
|
||||
func (e *ProviderError) Unwrap() error { return e.cause }
|
||||
|
||||
// APIError reports a failed Enable Banking call. Its message is built only
|
||||
// from the HTTP status and, when the response envelope's error code exactly
|
||||
// matches the documented enumeration, that code with a locally written hint.
|
||||
@@ -365,9 +394,9 @@ func (p *EnableBanking) request(ctx context.Context, method, path string, input,
|
||||
return nil, context.Canceled
|
||||
}
|
||||
if errors.Is(err, context.DeadlineExceeded) {
|
||||
return nil, fmt.Errorf("request timed out: %w", context.DeadlineExceeded)
|
||||
return nil, &ProviderError{Unreachable: true, Detail: "the request timed out", cause: context.DeadlineExceeded}
|
||||
}
|
||||
return nil, errors.New("connection failed")
|
||||
return nil, &ProviderError{Unreachable: true, Detail: "the connection failed"}
|
||||
}
|
||||
if response.StatusCode == http.StatusTooManyRequests && scope != "" && !foreground {
|
||||
// Inspect only a small structured error envelope, never exposing its
|
||||
@@ -404,15 +433,15 @@ func (p *EnableBanking) request(ctx context.Context, method, path string, input,
|
||||
return context.Canceled
|
||||
}
|
||||
if errors.Is(err, context.DeadlineExceeded) {
|
||||
return fmt.Errorf("Enable Banking response timed out: %w", context.DeadlineExceeded)
|
||||
return &ProviderError{Unreachable: true, Detail: "reading its response timed out", cause: context.DeadlineExceeded}
|
||||
}
|
||||
return fmt.Errorf("read Enable Banking response")
|
||||
return &ProviderError{Detail: "its response could not be read"}
|
||||
}
|
||||
if len(b) > limit {
|
||||
return fmt.Errorf("Enable Banking response exceeded size limit")
|
||||
return &ProviderError{Detail: "its response exceeded the size limit"}
|
||||
}
|
||||
if err = json.Unmarshal(b, output); err != nil {
|
||||
return fmt.Errorf("invalid Enable Banking response")
|
||||
return &ProviderError{Detail: "its response was not the expected JSON"}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -633,10 +662,10 @@ func (p *EnableBanking) Exchange(ctx context.Context, code string) (Session, err
|
||||
return Session{}, err
|
||||
}
|
||||
if response.ID == "" {
|
||||
return Session{}, fmt.Errorf("Enable Banking returned no session ID")
|
||||
return Session{}, &ProviderError{Detail: "it returned no session identifier"}
|
||||
}
|
||||
if _, err := time.Parse(time.RFC3339, response.Access.ValidUntil); err != nil {
|
||||
return Session{}, fmt.Errorf("Enable Banking returned invalid session expiry")
|
||||
return Session{}, &ProviderError{Detail: "it returned an invalid session expiry"}
|
||||
}
|
||||
result := Session{ID: response.ID, ValidUntil: response.Access.ValidUntil, Accounts: []domain.Account{}}
|
||||
// One account the journal cannot represent (securities or card entries
|
||||
@@ -669,7 +698,7 @@ func (p *EnableBanking) Status(ctx context.Context, sessionID string) (SessionSt
|
||||
}
|
||||
expires, err := time.Parse(time.RFC3339, response.Access.ValidUntil)
|
||||
if err != nil {
|
||||
return SessionStatus{}, fmt.Errorf("Enable Banking returned invalid session expiry")
|
||||
return SessionStatus{}, &ProviderError{Detail: "it returned an invalid session expiry"}
|
||||
}
|
||||
if !expires.After(time.Now()) {
|
||||
return SessionStatus{}, fmt.Errorf("Enable Banking session expired: %w", ErrReconnect)
|
||||
@@ -705,7 +734,7 @@ func (p *EnableBanking) Balances(ctx context.Context, externalAccountID string)
|
||||
for _, b := range response.Balances {
|
||||
amount, err := domain.ParseMoney(b.Amount.Amount)
|
||||
if err != nil || !validCurrency(b.Amount.Currency) {
|
||||
return nil, fmt.Errorf("Enable Banking returned invalid balance amount")
|
||||
return nil, &ProviderError{Detail: "it returned an invalid balance amount"}
|
||||
}
|
||||
result = append(result, Balance{Amount: amount, Currency: b.Amount.Currency, Type: b.Type, ReferenceDate: b.ReferenceDate})
|
||||
}
|
||||
@@ -776,30 +805,30 @@ func (p *EnableBanking) Transactions(ctx context.Context, account domain.Account
|
||||
}
|
||||
amount, err := domain.ParseMoney(t.Amount.Amount)
|
||||
if err != nil || strings.HasPrefix(amount.String(), "-") || !validCurrency(t.Amount.Currency) {
|
||||
return nil, fmt.Errorf("Enable Banking returned invalid transaction amount")
|
||||
return nil, &ProviderError{Detail: "a booked transaction has no usable amount or currency"}
|
||||
}
|
||||
party, iban := t.Debtor.Name, t.DebtorAccount.IBAN
|
||||
switch t.Indicator {
|
||||
case "DBIT":
|
||||
amount, err = domain.ParseMoney("-" + amount.String())
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("invalid debit amount")
|
||||
return nil, &ProviderError{Detail: "a booked debit has no usable amount"}
|
||||
}
|
||||
party, iban = t.Creditor.Name, t.CreditorAccount.IBAN
|
||||
case "CRDT":
|
||||
default:
|
||||
return nil, fmt.Errorf("Enable Banking returned invalid credit/debit indicator")
|
||||
return nil, &ProviderError{Detail: "a booked transaction has no credit/debit indicator"}
|
||||
}
|
||||
// Booked records without a booking date cannot be placed truthfully in the journal.
|
||||
if _, err := time.Parse("2006-01-02", t.BookingDate); err != nil {
|
||||
return nil, fmt.Errorf("Enable Banking booked transaction has no valid booking date")
|
||||
return nil, &ProviderError{Detail: "a booked transaction has no valid booking date"}
|
||||
}
|
||||
if (from != "" && t.BookingDate < from) || (to != "" && t.BookingDate > to) {
|
||||
continue
|
||||
}
|
||||
if t.ValueDate != "" {
|
||||
if _, err := time.Parse("2006-01-02", t.ValueDate); err != nil {
|
||||
return nil, fmt.Errorf("Enable Banking returned invalid value date")
|
||||
return nil, &ProviderError{Detail: "a booked transaction has an invalid value date"}
|
||||
}
|
||||
}
|
||||
description := strings.Join(t.Remittance, "\n")
|
||||
@@ -812,10 +841,10 @@ func (p *EnableBanking) Transactions(ctx context.Context, account domain.Account
|
||||
return result, nil
|
||||
}
|
||||
if seen[response.ContinuationKey] {
|
||||
return nil, fmt.Errorf("Enable Banking repeated a pagination key")
|
||||
return nil, &ProviderError{Detail: "it repeated a transaction pagination key"}
|
||||
}
|
||||
seen[response.ContinuationKey] = true
|
||||
query.Set("continuation_key", response.ContinuationKey)
|
||||
}
|
||||
return nil, fmt.Errorf("Enable Banking transaction pagination exceeded limit")
|
||||
return nil, &ProviderError{Detail: "its transaction pagination exceeded the supported page count"}
|
||||
}
|
||||
|
||||
@@ -57,7 +57,9 @@ func (r *RateLimitError) Error() string {
|
||||
if r.unbounded {
|
||||
return "provider rate limit (HTTP 429): retry time exceeds the supported range; automatic retry disabled"
|
||||
}
|
||||
return "provider rate limit (HTTP 429): retry allowed at " + r.next.UTC().Format(time.RFC3339Nano)
|
||||
// Second precision: this message is read by operators, not machines. Use
|
||||
// RetryAt for scheduling.
|
||||
return "provider rate limit (HTTP 429): automatic retry at " + r.next.UTC().Format(time.RFC3339)
|
||||
}
|
||||
|
||||
// RetryAt returns the earliest allowed retry time. Zero means the provider's
|
||||
|
||||
+15
-10
@@ -11,7 +11,7 @@ import {
|
||||
Sparkles,
|
||||
} from "lucide-react";
|
||||
import type { Account, Institution, PreparedImport, State } from "./api";
|
||||
import { money, request } from "./api";
|
||||
import { localInstant, money, request } from "./api";
|
||||
import { Empty, ErrorMessage, Field, FormActions, Modal } from "./ui";
|
||||
import type { Mutate } from "./ui";
|
||||
interface Balance {
|
||||
@@ -209,6 +209,8 @@ function backfillUnavailable(account: Account, state: State): string {
|
||||
const connection = state.connections.find((c) => c.account_id === account.id);
|
||||
if (connection?.status === "reconnect_required")
|
||||
return "Reconnect this account before importing older history.";
|
||||
if (connection?.status === "rate_limited")
|
||||
return `The bank is rate limiting this account until ${localInstant(connection.retry_at ?? "")}; import older history after that.`;
|
||||
if (
|
||||
!account.id ||
|
||||
!account.external_account_id ||
|
||||
@@ -270,17 +272,19 @@ function AccountCard({
|
||||
{account.iban && <small className="account-iban">{account.iban}</small>}
|
||||
<div className="connection-status">
|
||||
<span
|
||||
className={`badge ${needsReconnect || connection?.status === "error" ? "connection-warning" : "neutral"}`}
|
||||
className={`badge ${needsReconnect || connection?.status === "error" || connection?.status === "rate_limited" ? "connection-warning" : "neutral"}`}
|
||||
>
|
||||
{needsReconnect
|
||||
? `${institution} needs reconnection`
|
||||
: connection?.status === "connected"
|
||||
? "Bank connected"
|
||||
: connection?.status === "error"
|
||||
? "Connection error"
|
||||
: connection?.status === "local"
|
||||
? "Local account"
|
||||
: "Connection status unavailable"}
|
||||
: connection?.status === "rate_limited"
|
||||
? `${institution} rate limit until ${localInstant(connection.retry_at ?? "")}`
|
||||
: connection?.status === "connected"
|
||||
? "Bank connected"
|
||||
: connection?.status === "error"
|
||||
? "Connection error"
|
||||
: connection?.status === "local"
|
||||
? "Local account"
|
||||
: "Connection status unavailable"}
|
||||
</span>
|
||||
{connection?.valid_until && (
|
||||
<small>Authorization expires {connection.valid_until}</small>
|
||||
@@ -291,7 +295,8 @@ function AccountCard({
|
||||
{connection.history_months === 1 ? "month" : "months"}
|
||||
</small>
|
||||
)}
|
||||
{connection?.error && (
|
||||
{/* A rate limit is already stated, in local time, by the badge above. */}
|
||||
{connection?.error && connection.status !== "rate_limited" && (
|
||||
<small className="text-danger">{connection.error}</small>
|
||||
)}
|
||||
{connection && connection.status !== "local" && (
|
||||
|
||||
+29
-1
@@ -70,9 +70,16 @@ export interface Connection {
|
||||
country: string;
|
||||
psu_type: string;
|
||||
history_months: number;
|
||||
status: "local" | "connected" | "reconnect_required" | "error";
|
||||
status:
|
||||
| "local"
|
||||
| "connected"
|
||||
| "reconnect_required"
|
||||
| "rate_limited"
|
||||
| "error";
|
||||
valid_until: string;
|
||||
error: string;
|
||||
// retry_at is set while the bank rate limits this connection.
|
||||
retry_at?: string;
|
||||
}
|
||||
export interface Institution {
|
||||
name: string;
|
||||
@@ -88,6 +95,9 @@ export interface State {
|
||||
connections: Connection[];
|
||||
status: {
|
||||
sync_error: string;
|
||||
// sync_retry_at is set only when every failure is a bank rate limit that
|
||||
// clears by itself.
|
||||
sync_retry_at?: string;
|
||||
index_error: string;
|
||||
last_sync: string;
|
||||
banking_configured: boolean;
|
||||
@@ -252,6 +262,24 @@ export function normalizeState(state: State): State {
|
||||
for (const session of state.sessions) session.accounts ??= [];
|
||||
return state;
|
||||
}
|
||||
// Retry deadlines are instants, not calendar days: show them in the viewer's
|
||||
// own time zone rather than the server's UTC string.
|
||||
export function localInstant(iso: string): string {
|
||||
const at = new Date(iso);
|
||||
if (!iso || Number.isNaN(at.getTime())) return iso;
|
||||
const sameDay = at.toDateString() === new Date().toDateString();
|
||||
return at.toLocaleString(
|
||||
undefined,
|
||||
sameDay
|
||||
? { hour: "2-digit", minute: "2-digit" }
|
||||
: {
|
||||
day: "2-digit",
|
||||
month: "short",
|
||||
hour: "2-digit",
|
||||
minute: "2-digit",
|
||||
},
|
||||
);
|
||||
}
|
||||
export function money(value: string, currency: string): string {
|
||||
// Keep all financial values as decimal strings, including display formatting.
|
||||
const match = /^(-?)(\d+)(?:\.(\d+))?$/.exec(value);
|
||||
|
||||
+10
-2
@@ -17,7 +17,13 @@ import {
|
||||
CircleHelp,
|
||||
} from "lucide-react";
|
||||
import type { State } from "./api";
|
||||
import { APIError, emptyFilter, normalizeState, request } from "./api";
|
||||
import {
|
||||
APIError,
|
||||
emptyFilter,
|
||||
localInstant,
|
||||
normalizeState,
|
||||
request,
|
||||
} from "./api";
|
||||
import { Overview } from "./Overview";
|
||||
import { Transactions } from "./Transactions";
|
||||
import { Registry } from "./Registry";
|
||||
@@ -300,7 +306,9 @@ function App() {
|
||||
<span>
|
||||
{state.status.index_error
|
||||
? `Analytics needs attention: ${state.status.index_error}`
|
||||
: `Bank sync needs attention: ${state.status.sync_error}`}
|
||||
: state.status.sync_retry_at
|
||||
? `Bank sync is waiting for your bank's rate limit and retries by itself after ${localInstant(state.status.sync_retry_at)} — ${state.status.sync_error}`
|
||||
: `Bank sync needs attention: ${state.status.sync_error}`}
|
||||
</span>
|
||||
<button
|
||||
className="button subtle"
|
||||
|
||||
Reference in New Issue
Block a user