Report a rate-limited bank sync as a wait, and name real failures
Two of three banks were only pacing us, yet the dashboard demanded attention, printed four nested wrappers and a nanosecond UTC deadline, and the scheduler retried hourly into a refusal whose end time the bank had already given. A rate limit now carries its retry time as data: Status.SyncRetryAt is set when every failure is self-clearing, the connection reports rate_limited with that deadline, the dashboard says synchronization resumes by itself and renders the time in the browser's zone, and the scheduler sleeps until the deadline instead of spending hourly session checks. Sync now still tries immediately. The third bank's "transaction retrieval failed" hid its cause. Provider failures Finance Duck determines itself are typed as banking.ProviderError, so an unreachable provider, a timeout or an unusable response, such as a booked transaction without a booking date, is reported instead of the opaque fallback. Provider response text still never reaches the message.
This commit is contained in:
+14
-1
@@ -281,7 +281,20 @@ account's history. Reconnection preserves the saved history choice and existing
|
||||
cursors. Changing the choice or reconnecting does not backfill already-synced
|
||||
accounts. Older records can be imported using CSV. Older saved consents without
|
||||
a history choice use 12 months for accounts that have no successful-sync cursor.
|
||||
A failed provider call retains local data and is retried by the daily scheduler;
|
||||
A failed provider call retains local data. Failures whose cause Finance Duck
|
||||
determines locally are named: a bank rate limit with its retry time, an expired
|
||||
consent, an HTTP status, an unreachable provider, or a response the journal
|
||||
cannot use (for example a booked transaction without a booking date). Only an
|
||||
unrecognized cause falls back to "transaction retrieval failed". Provider
|
||||
response text is never shown.
|
||||
|
||||
While every failing bank has named its own retry time, the account is reported
|
||||
as waiting, not broken: the dashboard says synchronization retries by itself,
|
||||
the account card shows a rate-limit badge with that time, and the scheduler
|
||||
sleeps until the deadline instead of spending hourly session checks on a
|
||||
refusal it already knows about. Sync now still tries immediately. Any failure
|
||||
without such a deadline keeps the hourly retry. Retry times are persisted with
|
||||
the sync state in whole seconds and rendered in the browser's time zone.
|
||||
Sync now can retry sooner. Balances are fetched
|
||||
on demand, with exact amount/currency/type values, rather than inferred from an
|
||||
incomplete historical journal.
|
||||
|
||||
@@ -146,6 +146,8 @@ Initial synchronization requests the selected number of **calendar months of boo
|
||||
|
||||
**HTTP 429 is a provider rate limit, not evidence that bank consent has expired.** Bank reads honor `Retry-After` and use bounded exponential retries. A longer or exhausted limit pauses further requests until the reported retry time; failed accounts keep their previous sync cursors and imported data. Session checks use the saved account metadata rather than fetching every account's details again. A failed session is reported once instead of also marking each of its accounts unavailable. After the cooldown, **Sync now** can retry; the warning clears after a successful sync. One-time authorization and code-exchange requests are never automatically replayed.
|
||||
|
||||
**A rate-limited sync is a wait, not a fault.** While every failing bank has supplied a retry time, the dashboard reports that synchronization retries by itself after that moment, the account card shows a rate-limit badge instead of a connection error, and the background scheduler sleeps until the deadline rather than retrying hourly into a refusal it already knows about. **Sync now** still tries immediately. Any failure without a supplied deadline keeps the hourly retry, and its cause is named where Finance Duck can determine it locally: an expired consent, an HTTP status, an unreachable provider, or a response it cannot use, such as a booked transaction without a booking date. Provider response text is never displayed.
|
||||
|
||||
Manual **Sync now**, **Import older history**, and balance requests forward the requesting user's IP, browser User-Agent, and available Accept headers to the bank as PSU metadata. Scheduled syncs never claim a user is present. This distinction matters: [Enable Banking documents background limits of roughly four fetches per day at many banks](https://enablebanking.com/docs/faq/#why-am-i-getting-429-response-code-are-there-rate-limits-for-the-api). A confirmed background `ASPSP_RATE_LIMIT_EXCEEDED` defers that account's affected endpoint for at least six hours, preserving longer provider hints; it does not block an eligible user-initiated fetch. General provider limits still apply to both. Bank requests are spaced by at least one second, with longer learned spacing after throttling.
|
||||
|
||||
### Import older history for a connected account
|
||||
|
||||
+6
-1
@@ -32,6 +32,9 @@ type Status struct {
|
||||
SyncError string `json:"sync_error"`
|
||||
IndexError string `json:"index_error"`
|
||||
LastSync string `json:"last_sync"`
|
||||
// SyncRetryAt is set only when every sync failure is a bank rate limit that
|
||||
// clears on its own; it is the earliest time an automatic retry is allowed.
|
||||
SyncRetryAt string `json:"sync_retry_at,omitempty"`
|
||||
BankingConfigured bool `json:"banking_configured"`
|
||||
AIConfigured bool `json:"ai_configured"`
|
||||
}
|
||||
@@ -49,6 +52,7 @@ type operational struct {
|
||||
Sessions []banking.Session `json:"sessions"`
|
||||
LastSync string `json:"last_sync"`
|
||||
SyncError string `json:"sync_error"`
|
||||
SyncRetryAt string `json:"sync_retry_at,omitempty"`
|
||||
Consents map[string]Consent `json:"consents"`
|
||||
AccountSync map[string]string `json:"account_sync"`
|
||||
BankingScope string `json:"banking_scope"`
|
||||
@@ -166,7 +170,8 @@ func (a *App) snapshot(ctx context.Context) (State, error) {
|
||||
a.indexError = ""
|
||||
}
|
||||
}
|
||||
return State{Data: d, Revision: rev, Settings: a.settings, Sessions: copySessions(a.ops.Sessions), CallbackURL: a.callbackURL, BankingAppID: a.bankingSettings.AppID, Connections: a.connections(d), Status: Status{SyncError: a.ops.SyncError, LastSync: a.ops.LastSync, IndexError: a.indexError, BankingConfigured: a.bank != nil, AIConfigured: a.classifier.APIKey != ""}}, nil
|
||||
status := Status{SyncError: a.ops.SyncError, SyncRetryAt: a.ops.SyncRetryAt, LastSync: a.ops.LastSync, IndexError: a.indexError, BankingConfigured: a.bank != nil, AIConfigured: a.classifier.APIKey != ""}
|
||||
return State{Data: d, Revision: rev, Settings: a.settings, Sessions: copySessions(a.ops.Sessions), CallbackURL: a.callbackURL, BankingAppID: a.bankingSettings.AppID, Connections: a.connections(d), Status: status}, nil
|
||||
}
|
||||
func (a *App) Snapshot(ctx context.Context) (State, error) {
|
||||
a.mu.Lock()
|
||||
|
||||
@@ -31,6 +31,7 @@ func (a *App) clearBankingSessions(scope string) {
|
||||
a.ops.AccountSync = make(map[string]string)
|
||||
a.ops.LastSync = ""
|
||||
a.ops.SyncError = ""
|
||||
a.ops.SyncRetryAt = ""
|
||||
a.ops.BankingScope = scope
|
||||
}
|
||||
|
||||
|
||||
@@ -27,6 +27,8 @@ type Consent struct {
|
||||
HistoryMonths int `json:"history_months"`
|
||||
Error string `json:"error,omitempty"`
|
||||
NeedsReconnect bool `json:"needs_reconnect"`
|
||||
// RetryAt is the bank's own retry time while it rate limits this consent.
|
||||
RetryAt string `json:"retry_at,omitempty"`
|
||||
}
|
||||
type Connection struct {
|
||||
AccountID string `json:"account_id"`
|
||||
@@ -37,6 +39,7 @@ type Connection struct {
|
||||
Status string `json:"status"`
|
||||
ValidUntil string `json:"valid_until"`
|
||||
Error string `json:"error"`
|
||||
RetryAt string `json:"retry_at,omitempty"`
|
||||
}
|
||||
|
||||
// psuType keeps legacy consents, which predate the choice, on the personal
|
||||
@@ -79,6 +82,7 @@ func (a *App) connections(d domain.Dataset) []Connection {
|
||||
}
|
||||
c.ValidUntil = session.ValidUntil
|
||||
c.Error = meta.Error
|
||||
c.RetryAt = meta.RetryAt
|
||||
c.Status = "connected"
|
||||
expiry, err := time.Parse(time.RFC3339, session.ValidUntil)
|
||||
if meta.NeedsReconnect || err != nil || !expiry.After(time.Now()) {
|
||||
@@ -86,6 +90,9 @@ func (a *App) connections(d domain.Dataset) []Connection {
|
||||
if c.Error == "" {
|
||||
c.Error = "Bank consent expired; reconnect to resume automatic imports"
|
||||
}
|
||||
} else if meta.RetryAt != "" {
|
||||
// A rate limit is the bank pacing us, not a broken connection.
|
||||
c.Status = "rate_limited"
|
||||
} else if meta.Error != "" {
|
||||
c.Status = "error"
|
||||
}
|
||||
|
||||
+85
-14
@@ -605,10 +605,12 @@ func (a *App) Balances(ctx context.Context, id string) ([]banking.Balance, error
|
||||
|
||||
// Only typed, locally generated errors are safe to expose; provider errors may
|
||||
// wrap private response data even when their underlying cause is recognizable.
|
||||
// The fallback is a last resort: an unnamed cause leaves an operator with
|
||||
// nothing to act on.
|
||||
func bankFailure(err error, fallback string) error {
|
||||
var background *banking.BackgroundQuotaError
|
||||
if errors.As(err, &background) {
|
||||
return fmt.Errorf("Enable Banking: %w", background)
|
||||
return background
|
||||
}
|
||||
var limited *ratelimit.RateLimitError
|
||||
if errors.As(err, &limited) {
|
||||
@@ -625,9 +627,25 @@ func bankFailure(err error, fallback string) error {
|
||||
if errors.As(err, &consent) {
|
||||
return consent
|
||||
}
|
||||
var provider *banking.ProviderError
|
||||
if errors.As(err, &provider) {
|
||||
return provider
|
||||
}
|
||||
return errors.New(fallback)
|
||||
}
|
||||
|
||||
// syncRetryAt reports the bank's own retry time for a failure that clears
|
||||
// itself. A rate limit without a usable deadline is not treated as waiting:
|
||||
// nothing would ever announce that it had expired.
|
||||
func syncRetryAt(err error) (time.Time, bool) {
|
||||
var limited *ratelimit.RateLimitError
|
||||
if !errors.As(err, &limited) {
|
||||
return time.Time{}, false
|
||||
}
|
||||
at := limited.RetryAt()
|
||||
return at, !at.IsZero()
|
||||
}
|
||||
|
||||
func (a *App) Sync(ctx context.Context) (State, error) {
|
||||
a.mu.Lock()
|
||||
defer a.mu.Unlock()
|
||||
@@ -639,6 +657,10 @@ func (a *App) Sync(ctx context.Context) (State, error) {
|
||||
return State{}, err
|
||||
}
|
||||
var failures []string
|
||||
// waitUntil is the earliest time the banks themselves allow a retry, used
|
||||
// only while every failure is such a self-clearing rate limit.
|
||||
var waitUntil time.Time
|
||||
waiting := true
|
||||
for i := range a.ops.Sessions {
|
||||
connectAccounts(&s.Data, &a.ops.Sessions[i], false)
|
||||
}
|
||||
@@ -684,6 +706,15 @@ func (a *App) Sync(ctx context.Context) (State, error) {
|
||||
}
|
||||
if e != nil {
|
||||
meta.Error = bankFailure(e, "bank connection unavailable; retry synchronization").Error()
|
||||
meta.RetryAt = ""
|
||||
if at, ok := syncRetryAt(e); ok {
|
||||
meta.RetryAt = at.UTC().Format(time.RFC3339)
|
||||
if waitUntil.IsZero() || at.Before(waitUntil) {
|
||||
waitUntil = at
|
||||
}
|
||||
} else {
|
||||
waiting = false
|
||||
}
|
||||
meta.NeedsReconnect = errors.Is(e, banking.ErrReconnect)
|
||||
a.ops.Consents[session.ID] = meta
|
||||
failures = append(failures, meta.Institution+": "+meta.Error)
|
||||
@@ -691,6 +722,7 @@ func (a *App) Sync(ctx context.Context) (State, error) {
|
||||
continue
|
||||
}
|
||||
meta.Error = ""
|
||||
meta.RetryAt = ""
|
||||
meta.NeedsReconnect = false
|
||||
a.ops.Consents[session.ID] = meta
|
||||
a.ops.Sessions[i].ValidUntil = current.ValidUntil
|
||||
@@ -712,9 +744,11 @@ func (a *App) Sync(ctx context.Context) (State, error) {
|
||||
}
|
||||
if !validAccounts[account.ID] {
|
||||
failures = append(failures, account.DisplayName+": bank connection unavailable")
|
||||
waiting = false
|
||||
if sessionID != "" {
|
||||
meta := a.ops.Consents[sessionID]
|
||||
meta.Error = banking.ErrReconnect.Error()
|
||||
meta.RetryAt = ""
|
||||
meta.NeedsReconnect = true
|
||||
a.ops.Consents[sessionID] = meta
|
||||
}
|
||||
@@ -731,6 +765,15 @@ func (a *App) Sync(ctx context.Context) (State, error) {
|
||||
if e != nil {
|
||||
meta := a.ops.Consents[sessionID]
|
||||
meta.Error = bankFailure(e, "transaction retrieval failed; retry synchronization").Error()
|
||||
meta.RetryAt = ""
|
||||
if at, ok := syncRetryAt(e); ok {
|
||||
meta.RetryAt = at.UTC().Format(time.RFC3339)
|
||||
if waitUntil.IsZero() || at.Before(waitUntil) {
|
||||
waitUntil = at
|
||||
}
|
||||
} else {
|
||||
waiting = false
|
||||
}
|
||||
meta.NeedsReconnect = meta.NeedsReconnect || errors.Is(e, banking.ErrReconnect)
|
||||
a.ops.Consents[sessionID] = meta
|
||||
failures = append(failures, account.DisplayName+": "+meta.Error)
|
||||
@@ -739,6 +782,7 @@ func (a *App) Sync(ctx context.Context) (State, error) {
|
||||
result, e := a.importFacts(ctx, s, facts)
|
||||
if e != nil {
|
||||
failures = append(failures, account.DisplayName+": "+e.Error())
|
||||
waiting = false
|
||||
s, err = a.snapshot(ctx)
|
||||
if err != nil {
|
||||
return State{}, err
|
||||
@@ -749,14 +793,47 @@ func (a *App) Sync(ctx context.Context) (State, error) {
|
||||
a.ops.AccountSync[account.ID] = now.Format(time.RFC3339)
|
||||
}
|
||||
a.ops.SyncError = strings.Join(failures, "; ")
|
||||
a.ops.SyncRetryAt = ""
|
||||
if len(failures) == 0 {
|
||||
a.ops.LastSync = now.Format(time.RFC3339)
|
||||
} else if waiting && !waitUntil.IsZero() {
|
||||
// Every bank named its own retry time: this is a wait, not a fault.
|
||||
a.ops.SyncRetryAt = waitUntil.UTC().Format(time.RFC3339)
|
||||
}
|
||||
if err = a.saveOps(); err != nil {
|
||||
return State{}, err
|
||||
}
|
||||
return a.snapshot(ctx)
|
||||
}
|
||||
|
||||
// syncSchedule decides whether an automatic sync may run now, and how long to
|
||||
// wait otherwise. While a bank has named its own retry time, waiting is the
|
||||
// only useful action: retrying earlier spends session-status calls on a refusal
|
||||
// that is already known. A manual request always proceeds.
|
||||
func syncSchedule(now time.Time, ops operational, force bool) (time.Duration, bool) {
|
||||
if retry, err := time.Parse(time.RFC3339, ops.SyncRetryAt); err == nil && !force && now.Before(retry) {
|
||||
return min(retry.Sub(now)+time.Minute, time.Hour), false
|
||||
}
|
||||
last, err := time.Parse(time.RFC3339, ops.LastSync)
|
||||
if force || err != nil || ops.SyncError != "" || now.Sub(last) >= 24*time.Hour {
|
||||
return 0, true
|
||||
}
|
||||
return time.Minute, false
|
||||
}
|
||||
|
||||
// syncBackoff spaces the next attempt after a sync. A failure with a known bank
|
||||
// retry time waits for it; other failures retry hourly so transient provider
|
||||
// problems clear without waiting a day, while bounding unattended traffic.
|
||||
func syncBackoff(now time.Time, ops operational) time.Duration {
|
||||
if ops.SyncError == "" {
|
||||
return 24 * time.Hour
|
||||
}
|
||||
if retry, err := time.Parse(time.RFC3339, ops.SyncRetryAt); err == nil && now.Before(retry) {
|
||||
return min(retry.Sub(now)+time.Minute, 24*time.Hour)
|
||||
}
|
||||
return time.Hour
|
||||
}
|
||||
|
||||
func (a *App) RunScheduler(ctx context.Context) {
|
||||
timer := time.NewTimer(time.Minute)
|
||||
defer timer.Stop()
|
||||
@@ -771,25 +848,19 @@ func (a *App) RunScheduler(ctx context.Context) {
|
||||
}
|
||||
a.mu.Lock()
|
||||
configured := a.bank != nil
|
||||
last, err := time.Parse(time.RFC3339, a.ops.LastSync)
|
||||
failed := a.ops.SyncError != ""
|
||||
wait, due := syncSchedule(time.Now(), a.ops, force)
|
||||
a.mu.Unlock()
|
||||
due := force || err != nil || failed || time.Since(last) >= 24*time.Hour
|
||||
if !configured || !due {
|
||||
timer.Reset(time.Minute)
|
||||
if wait <= 0 {
|
||||
wait = time.Minute
|
||||
}
|
||||
timer.Reset(wait)
|
||||
continue
|
||||
}
|
||||
a.Sync(ctx)
|
||||
a.mu.Lock()
|
||||
failed = a.ops.SyncError != ""
|
||||
wait = syncBackoff(time.Now(), a.ops)
|
||||
a.mu.Unlock()
|
||||
if failed {
|
||||
// A failed sync leaves its persisted error banner behind. Retry
|
||||
// hourly so transient provider failures clear without waiting a
|
||||
// day, while bounding unattended traffic toward the provider.
|
||||
timer.Reset(time.Hour)
|
||||
} else {
|
||||
timer.Reset(24 * time.Hour)
|
||||
}
|
||||
timer.Reset(wait)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -273,7 +273,7 @@ func TestSyncMissingMembershipStillRejectsAccount(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestSyncTransactionFailuresPreserveProgressAndSafeErrors(t *testing.T) {
|
||||
for _, scenario := range []string{"rate limit", "background rate limit", "reconnect", "private response"} {
|
||||
for _, scenario := range []string{"rate limit", "background rate limit", "reconnect", "private response", "unusable response"} {
|
||||
t.Run(scenario, func(t *testing.T) {
|
||||
a, s, b := backfillApp(t)
|
||||
s = seed(t, a, s)
|
||||
@@ -292,6 +292,9 @@ func TestSyncTransactionFailuresPreserveProgressAndSafeErrors(t *testing.T) {
|
||||
b.fetchErr = fmt.Errorf("private provider response: %w", banking.ErrReconnect)
|
||||
case "private response":
|
||||
b.fetchErr = errors.New("private provider response")
|
||||
case "unusable response":
|
||||
unusable := &banking.ProviderError{Detail: "a booked transaction has no valid booking date"}
|
||||
b.fetchErr = fmt.Errorf("private provider response: %w", unusable)
|
||||
}
|
||||
failed, err := a.Sync(context.Background())
|
||||
if err != nil {
|
||||
@@ -313,6 +316,43 @@ func TestSyncTransactionFailuresPreserveProgressAndSafeErrors(t *testing.T) {
|
||||
if !reflect.DeepEqual(s.Data, failed.Data) || !reflect.DeepEqual(cursors, a.ops.AccountSync) || a.ops.LastSync != old {
|
||||
t.Fatal("failed retrieval imported partial data or advanced synchronization")
|
||||
}
|
||||
rateLimited := strings.Contains(scenario, "rate limit")
|
||||
// A bank that named its own retry time is a wait, not a fault: the
|
||||
// UI and the scheduler both rely on this distinction.
|
||||
if (failed.Status.SyncRetryAt != "") != rateLimited {
|
||||
t.Fatalf("waiting state is wrong for %s: retry at %q", scenario, failed.Status.SyncRetryAt)
|
||||
}
|
||||
for _, connection := range failed.Connections {
|
||||
if connection.Status == "local" {
|
||||
continue
|
||||
}
|
||||
want := "error"
|
||||
switch {
|
||||
case rateLimited:
|
||||
want = "rate_limited"
|
||||
case scenario == "reconnect":
|
||||
want = "reconnect_required"
|
||||
}
|
||||
if connection.Status != want || (connection.RetryAt != "") != rateLimited {
|
||||
t.Fatalf("connection reported %q with retry %q, want %q", connection.Status, connection.RetryAt, want)
|
||||
}
|
||||
}
|
||||
if rateLimited {
|
||||
at, e := time.Parse(time.RFC3339, failed.Status.SyncRetryAt)
|
||||
if e != nil || !at.After(time.Now()) {
|
||||
t.Fatalf("unusable retry deadline %q: %v", failed.Status.SyncRetryAt, e)
|
||||
}
|
||||
// Whole seconds, once: operators read this message.
|
||||
if !strings.Contains(meta.Error, at.UTC().Format(time.RFC3339)) || strings.Count(meta.Error, "429") != 1 {
|
||||
t.Fatalf("rate limit message is not legible: %q", meta.Error)
|
||||
}
|
||||
}
|
||||
if scenario == "unusable response" && (!strings.Contains(meta.Error, "cannot use") || !strings.Contains(meta.Error, "booking date")) {
|
||||
t.Fatalf("unusable provider response was reduced to an opaque failure: %q", meta.Error)
|
||||
}
|
||||
if scenario == "private response" && !strings.Contains(meta.Error, "transaction retrieval failed") {
|
||||
t.Fatalf("unrecognized failure lost its fallback: %q", meta.Error)
|
||||
}
|
||||
b.fetchErr = nil
|
||||
recovered, err := a.Sync(context.Background())
|
||||
if err != nil || recovered.Status.SyncError != "" || a.ops.Consents["current"].Error != "" || a.ops.Consents["current"].NeedsReconnect {
|
||||
@@ -321,3 +361,45 @@ func TestSyncTransactionFailuresPreserveProgressAndSafeErrors(t *testing.T) {
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// The scheduler must not spend session-status calls on a refusal the bank has
|
||||
// already scheduled, and must not sit on a deadline that has passed.
|
||||
func TestSyncSchedulingRespectsTheBanksOwnRetryTime(t *testing.T) {
|
||||
// Persisted deadlines carry whole seconds; compare against the same grid.
|
||||
now := time.Now().UTC().Truncate(time.Second)
|
||||
stale := now.Add(-48 * time.Hour).Format(time.RFC3339)
|
||||
waiting := operational{LastSync: stale, SyncError: "N26: rate limited", SyncRetryAt: now.Add(3 * time.Hour).Format(time.RFC3339)}
|
||||
broken := operational{LastSync: stale, SyncError: "Trade Republic: retrieval failed"}
|
||||
healthy := operational{LastSync: now.Format(time.RFC3339)}
|
||||
elapsed := operational{LastSync: stale, SyncError: waiting.SyncError, SyncRetryAt: now.Add(-time.Minute).Format(time.RFC3339)}
|
||||
cases := []struct {
|
||||
name string
|
||||
ops operational
|
||||
force bool
|
||||
wait time.Duration
|
||||
due bool
|
||||
}{
|
||||
{"waiting for the bank", waiting, false, time.Hour, false},
|
||||
{"manual sync during a wait", waiting, true, 0, true},
|
||||
{"deadline elapsed", elapsed, false, 0, true},
|
||||
{"failure without a deadline", broken, false, 0, true},
|
||||
{"recent success", healthy, false, time.Minute, false},
|
||||
}
|
||||
for _, tt := range cases {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
wait, due := syncSchedule(now, tt.ops, tt.force)
|
||||
if wait != tt.wait || due != tt.due {
|
||||
t.Fatalf("schedule = (%s, %t), want (%s, %t)", wait, due, tt.wait, tt.due)
|
||||
}
|
||||
})
|
||||
}
|
||||
if backoff := syncBackoff(now, waiting); backoff != 3*time.Hour+time.Minute {
|
||||
t.Fatalf("rate-limited backoff = %s, want the bank's own deadline", backoff)
|
||||
}
|
||||
if backoff := syncBackoff(now, broken); backoff != time.Hour {
|
||||
t.Fatalf("failure backoff = %s, want hourly retries", backoff)
|
||||
}
|
||||
if backoff := syncBackoff(now, healthy); backoff != 24*time.Hour {
|
||||
t.Fatalf("successful backoff = %s, want daily synchronization", backoff)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -204,14 +204,43 @@ type BackgroundQuotaError struct {
|
||||
*ratelimit.RateLimitError
|
||||
}
|
||||
|
||||
// Error describes the pause in one line: operators need the bank's own retry
|
||||
// time, not a stack of nested rate-limit wrappers.
|
||||
func (e *BackgroundQuotaError) Error() string {
|
||||
return "background bank retrieval quota (normally six hours): " + e.RateLimitError.Error()
|
||||
if e.RetryAt().IsZero() {
|
||||
return "the bank is rate limiting background retrieval (HTTP 429); automatic retry is disabled, synchronize manually later"
|
||||
}
|
||||
return "the bank is rate limiting background retrieval (HTTP 429, normally six hours); automatic retry at " + e.RetryAt().UTC().Format(time.RFC3339)
|
||||
}
|
||||
|
||||
func (e *BackgroundQuotaError) Unwrap() error {
|
||||
return e.RateLimitError
|
||||
}
|
||||
|
||||
// ProviderError reports a provider interaction that failed for a reason Finance
|
||||
// Duck determined itself: the provider could not be reached, or its response
|
||||
// could not be used. Detail is written here and never taken from provider
|
||||
// response text, so callers may show the whole message to the user.
|
||||
type ProviderError struct {
|
||||
Unreachable bool
|
||||
Detail string
|
||||
cause error
|
||||
}
|
||||
|
||||
func (e *ProviderError) Error() string {
|
||||
message := "the bank's provider returned a response Finance Duck cannot use"
|
||||
if e.Unreachable {
|
||||
message = "the bank's provider could not be reached"
|
||||
}
|
||||
if e.Detail == "" {
|
||||
return message
|
||||
}
|
||||
return message + ": " + e.Detail
|
||||
}
|
||||
|
||||
// Unwrap keeps cancellation and deadline identity for callers that retry.
|
||||
func (e *ProviderError) Unwrap() error { return e.cause }
|
||||
|
||||
// APIError reports a failed Enable Banking call. Its message is built only
|
||||
// from the HTTP status and, when the response envelope's error code exactly
|
||||
// matches the documented enumeration, that code with a locally written hint.
|
||||
@@ -365,9 +394,9 @@ func (p *EnableBanking) request(ctx context.Context, method, path string, input,
|
||||
return nil, context.Canceled
|
||||
}
|
||||
if errors.Is(err, context.DeadlineExceeded) {
|
||||
return nil, fmt.Errorf("request timed out: %w", context.DeadlineExceeded)
|
||||
return nil, &ProviderError{Unreachable: true, Detail: "the request timed out", cause: context.DeadlineExceeded}
|
||||
}
|
||||
return nil, errors.New("connection failed")
|
||||
return nil, &ProviderError{Unreachable: true, Detail: "the connection failed"}
|
||||
}
|
||||
if response.StatusCode == http.StatusTooManyRequests && scope != "" && !foreground {
|
||||
// Inspect only a small structured error envelope, never exposing its
|
||||
@@ -404,15 +433,15 @@ func (p *EnableBanking) request(ctx context.Context, method, path string, input,
|
||||
return context.Canceled
|
||||
}
|
||||
if errors.Is(err, context.DeadlineExceeded) {
|
||||
return fmt.Errorf("Enable Banking response timed out: %w", context.DeadlineExceeded)
|
||||
return &ProviderError{Unreachable: true, Detail: "reading its response timed out", cause: context.DeadlineExceeded}
|
||||
}
|
||||
return fmt.Errorf("read Enable Banking response")
|
||||
return &ProviderError{Detail: "its response could not be read"}
|
||||
}
|
||||
if len(b) > limit {
|
||||
return fmt.Errorf("Enable Banking response exceeded size limit")
|
||||
return &ProviderError{Detail: "its response exceeded the size limit"}
|
||||
}
|
||||
if err = json.Unmarshal(b, output); err != nil {
|
||||
return fmt.Errorf("invalid Enable Banking response")
|
||||
return &ProviderError{Detail: "its response was not the expected JSON"}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -633,10 +662,10 @@ func (p *EnableBanking) Exchange(ctx context.Context, code string) (Session, err
|
||||
return Session{}, err
|
||||
}
|
||||
if response.ID == "" {
|
||||
return Session{}, fmt.Errorf("Enable Banking returned no session ID")
|
||||
return Session{}, &ProviderError{Detail: "it returned no session identifier"}
|
||||
}
|
||||
if _, err := time.Parse(time.RFC3339, response.Access.ValidUntil); err != nil {
|
||||
return Session{}, fmt.Errorf("Enable Banking returned invalid session expiry")
|
||||
return Session{}, &ProviderError{Detail: "it returned an invalid session expiry"}
|
||||
}
|
||||
result := Session{ID: response.ID, ValidUntil: response.Access.ValidUntil, Accounts: []domain.Account{}}
|
||||
// One account the journal cannot represent (securities or card entries
|
||||
@@ -669,7 +698,7 @@ func (p *EnableBanking) Status(ctx context.Context, sessionID string) (SessionSt
|
||||
}
|
||||
expires, err := time.Parse(time.RFC3339, response.Access.ValidUntil)
|
||||
if err != nil {
|
||||
return SessionStatus{}, fmt.Errorf("Enable Banking returned invalid session expiry")
|
||||
return SessionStatus{}, &ProviderError{Detail: "it returned an invalid session expiry"}
|
||||
}
|
||||
if !expires.After(time.Now()) {
|
||||
return SessionStatus{}, fmt.Errorf("Enable Banking session expired: %w", ErrReconnect)
|
||||
@@ -705,7 +734,7 @@ func (p *EnableBanking) Balances(ctx context.Context, externalAccountID string)
|
||||
for _, b := range response.Balances {
|
||||
amount, err := domain.ParseMoney(b.Amount.Amount)
|
||||
if err != nil || !validCurrency(b.Amount.Currency) {
|
||||
return nil, fmt.Errorf("Enable Banking returned invalid balance amount")
|
||||
return nil, &ProviderError{Detail: "it returned an invalid balance amount"}
|
||||
}
|
||||
result = append(result, Balance{Amount: amount, Currency: b.Amount.Currency, Type: b.Type, ReferenceDate: b.ReferenceDate})
|
||||
}
|
||||
@@ -776,30 +805,30 @@ func (p *EnableBanking) Transactions(ctx context.Context, account domain.Account
|
||||
}
|
||||
amount, err := domain.ParseMoney(t.Amount.Amount)
|
||||
if err != nil || strings.HasPrefix(amount.String(), "-") || !validCurrency(t.Amount.Currency) {
|
||||
return nil, fmt.Errorf("Enable Banking returned invalid transaction amount")
|
||||
return nil, &ProviderError{Detail: "a booked transaction has no usable amount or currency"}
|
||||
}
|
||||
party, iban := t.Debtor.Name, t.DebtorAccount.IBAN
|
||||
switch t.Indicator {
|
||||
case "DBIT":
|
||||
amount, err = domain.ParseMoney("-" + amount.String())
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("invalid debit amount")
|
||||
return nil, &ProviderError{Detail: "a booked debit has no usable amount"}
|
||||
}
|
||||
party, iban = t.Creditor.Name, t.CreditorAccount.IBAN
|
||||
case "CRDT":
|
||||
default:
|
||||
return nil, fmt.Errorf("Enable Banking returned invalid credit/debit indicator")
|
||||
return nil, &ProviderError{Detail: "a booked transaction has no credit/debit indicator"}
|
||||
}
|
||||
// Booked records without a booking date cannot be placed truthfully in the journal.
|
||||
if _, err := time.Parse("2006-01-02", t.BookingDate); err != nil {
|
||||
return nil, fmt.Errorf("Enable Banking booked transaction has no valid booking date")
|
||||
return nil, &ProviderError{Detail: "a booked transaction has no valid booking date"}
|
||||
}
|
||||
if (from != "" && t.BookingDate < from) || (to != "" && t.BookingDate > to) {
|
||||
continue
|
||||
}
|
||||
if t.ValueDate != "" {
|
||||
if _, err := time.Parse("2006-01-02", t.ValueDate); err != nil {
|
||||
return nil, fmt.Errorf("Enable Banking returned invalid value date")
|
||||
return nil, &ProviderError{Detail: "a booked transaction has an invalid value date"}
|
||||
}
|
||||
}
|
||||
description := strings.Join(t.Remittance, "\n")
|
||||
@@ -812,10 +841,10 @@ func (p *EnableBanking) Transactions(ctx context.Context, account domain.Account
|
||||
return result, nil
|
||||
}
|
||||
if seen[response.ContinuationKey] {
|
||||
return nil, fmt.Errorf("Enable Banking repeated a pagination key")
|
||||
return nil, &ProviderError{Detail: "it repeated a transaction pagination key"}
|
||||
}
|
||||
seen[response.ContinuationKey] = true
|
||||
query.Set("continuation_key", response.ContinuationKey)
|
||||
}
|
||||
return nil, fmt.Errorf("Enable Banking transaction pagination exceeded limit")
|
||||
return nil, &ProviderError{Detail: "its transaction pagination exceeded the supported page count"}
|
||||
}
|
||||
|
||||
@@ -57,7 +57,9 @@ func (r *RateLimitError) Error() string {
|
||||
if r.unbounded {
|
||||
return "provider rate limit (HTTP 429): retry time exceeds the supported range; automatic retry disabled"
|
||||
}
|
||||
return "provider rate limit (HTTP 429): retry allowed at " + r.next.UTC().Format(time.RFC3339Nano)
|
||||
// Second precision: this message is read by operators, not machines. Use
|
||||
// RetryAt for scheduling.
|
||||
return "provider rate limit (HTTP 429): automatic retry at " + r.next.UTC().Format(time.RFC3339)
|
||||
}
|
||||
|
||||
// RetryAt returns the earliest allowed retry time. Zero means the provider's
|
||||
|
||||
@@ -11,7 +11,7 @@ import {
|
||||
Sparkles,
|
||||
} from "lucide-react";
|
||||
import type { Account, Institution, PreparedImport, State } from "./api";
|
||||
import { money, request } from "./api";
|
||||
import { localInstant, money, request } from "./api";
|
||||
import { Empty, ErrorMessage, Field, FormActions, Modal } from "./ui";
|
||||
import type { Mutate } from "./ui";
|
||||
interface Balance {
|
||||
@@ -209,6 +209,8 @@ function backfillUnavailable(account: Account, state: State): string {
|
||||
const connection = state.connections.find((c) => c.account_id === account.id);
|
||||
if (connection?.status === "reconnect_required")
|
||||
return "Reconnect this account before importing older history.";
|
||||
if (connection?.status === "rate_limited")
|
||||
return `The bank is rate limiting this account until ${localInstant(connection.retry_at ?? "")}; import older history after that.`;
|
||||
if (
|
||||
!account.id ||
|
||||
!account.external_account_id ||
|
||||
@@ -270,10 +272,12 @@ function AccountCard({
|
||||
{account.iban && <small className="account-iban">{account.iban}</small>}
|
||||
<div className="connection-status">
|
||||
<span
|
||||
className={`badge ${needsReconnect || connection?.status === "error" ? "connection-warning" : "neutral"}`}
|
||||
className={`badge ${needsReconnect || connection?.status === "error" || connection?.status === "rate_limited" ? "connection-warning" : "neutral"}`}
|
||||
>
|
||||
{needsReconnect
|
||||
? `${institution} needs reconnection`
|
||||
: connection?.status === "rate_limited"
|
||||
? `${institution} rate limit until ${localInstant(connection.retry_at ?? "")}`
|
||||
: connection?.status === "connected"
|
||||
? "Bank connected"
|
||||
: connection?.status === "error"
|
||||
@@ -291,7 +295,8 @@ function AccountCard({
|
||||
{connection.history_months === 1 ? "month" : "months"}
|
||||
</small>
|
||||
)}
|
||||
{connection?.error && (
|
||||
{/* A rate limit is already stated, in local time, by the badge above. */}
|
||||
{connection?.error && connection.status !== "rate_limited" && (
|
||||
<small className="text-danger">{connection.error}</small>
|
||||
)}
|
||||
{connection && connection.status !== "local" && (
|
||||
|
||||
+29
-1
@@ -70,9 +70,16 @@ export interface Connection {
|
||||
country: string;
|
||||
psu_type: string;
|
||||
history_months: number;
|
||||
status: "local" | "connected" | "reconnect_required" | "error";
|
||||
status:
|
||||
| "local"
|
||||
| "connected"
|
||||
| "reconnect_required"
|
||||
| "rate_limited"
|
||||
| "error";
|
||||
valid_until: string;
|
||||
error: string;
|
||||
// retry_at is set while the bank rate limits this connection.
|
||||
retry_at?: string;
|
||||
}
|
||||
export interface Institution {
|
||||
name: string;
|
||||
@@ -88,6 +95,9 @@ export interface State {
|
||||
connections: Connection[];
|
||||
status: {
|
||||
sync_error: string;
|
||||
// sync_retry_at is set only when every failure is a bank rate limit that
|
||||
// clears by itself.
|
||||
sync_retry_at?: string;
|
||||
index_error: string;
|
||||
last_sync: string;
|
||||
banking_configured: boolean;
|
||||
@@ -252,6 +262,24 @@ export function normalizeState(state: State): State {
|
||||
for (const session of state.sessions) session.accounts ??= [];
|
||||
return state;
|
||||
}
|
||||
// Retry deadlines are instants, not calendar days: show them in the viewer's
|
||||
// own time zone rather than the server's UTC string.
|
||||
export function localInstant(iso: string): string {
|
||||
const at = new Date(iso);
|
||||
if (!iso || Number.isNaN(at.getTime())) return iso;
|
||||
const sameDay = at.toDateString() === new Date().toDateString();
|
||||
return at.toLocaleString(
|
||||
undefined,
|
||||
sameDay
|
||||
? { hour: "2-digit", minute: "2-digit" }
|
||||
: {
|
||||
day: "2-digit",
|
||||
month: "short",
|
||||
hour: "2-digit",
|
||||
minute: "2-digit",
|
||||
},
|
||||
);
|
||||
}
|
||||
export function money(value: string, currency: string): string {
|
||||
// Keep all financial values as decimal strings, including display formatting.
|
||||
const match = /^(-?)(\d+)(?:\.(\d+))?$/.exec(value);
|
||||
|
||||
+9
-1
@@ -17,7 +17,13 @@ import {
|
||||
CircleHelp,
|
||||
} from "lucide-react";
|
||||
import type { State } from "./api";
|
||||
import { APIError, emptyFilter, normalizeState, request } from "./api";
|
||||
import {
|
||||
APIError,
|
||||
emptyFilter,
|
||||
localInstant,
|
||||
normalizeState,
|
||||
request,
|
||||
} from "./api";
|
||||
import { Overview } from "./Overview";
|
||||
import { Transactions } from "./Transactions";
|
||||
import { Registry } from "./Registry";
|
||||
@@ -300,6 +306,8 @@ function App() {
|
||||
<span>
|
||||
{state.status.index_error
|
||||
? `Analytics needs attention: ${state.status.index_error}`
|
||||
: state.status.sync_retry_at
|
||||
? `Bank sync is waiting for your bank's rate limit and retries by itself after ${localInstant(state.status.sync_retry_at)} — ${state.status.sync_error}`
|
||||
: `Bank sync needs attention: ${state.status.sync_error}`}
|
||||
</span>
|
||||
<button
|
||||
|
||||
Reference in New Issue
Block a user