Batch Analyse requests and survive opaque provider schema budgets
Analyse now classifies up to ten same-kind transactions per provider request: the registry and history travel once per batch, so a thousand-row backfill costs about a hundred paced requests instead of a thousand. The answer schema appears once — an array item carrying an enum-bound ref — because providers meter strict schemas by token cost: duplicating registry enums per row, or bounding arrays with minItems/maxItems that Gemini expands per element, rejects real registries with a bare HTTP 400. Row count, duplicate refs, duplicate tags and taxonomy bounds are all enforced server-side instead, and a request still rejected outright halves until accepted, remembering the working size for the run. Batch requests scale the HTTP budget by row count, chunk failures cannot abort a run whose later rows succeeded, and rows resolved against one snapshot share one minted merchant. Measured on a real 165-row month over a zero-data-retention route: 165 analysed, 152 proposals, 0 errors, 17 requests, under 8 minutes. Fresh installs default to google/gemini-3.8-flash, the model that demonstrably honors strict structured outputs over a ZDR route. Preview changes now carry counterparty, amount and currency, and the review list shows the amount with a counterparty fallback for banks that leave descriptions empty.
This commit is contained in:
@@ -59,13 +59,14 @@ func ledgerFixture() (domain.Dataset, domain.Facts) {
|
||||
return d, facts
|
||||
}
|
||||
|
||||
// strictKeywords is what OpenAI-family strict structured-output mode accepts.
|
||||
// uniqueItems is specifically rejected ("'uniqueItems' is not permitted") and
|
||||
// took every zero-data-retention route for those models down with HTTP 400;
|
||||
// duplicates are rejected server-side by decodeAnswer instead.
|
||||
// strictKeywords is what every targeted provider accepts in strict
|
||||
// structured-output mode. uniqueItems is rejected outright by OpenAI-family
|
||||
// endpoints ("'uniqueItems' is not permitted"); minItems/maxItems make Gemini
|
||||
// expand array item schemas per element and reject real registries with a
|
||||
// bare HTTP 400. Counts and duplicates are enforced server-side instead.
|
||||
var strictKeywords = map[string]bool{
|
||||
"type": true, "properties": true, "required": true, "additionalProperties": true,
|
||||
"items": true, "enum": true, "maxItems": true, "maxLength": true, "minLength": true,
|
||||
"items": true, "enum": true, "maxLength": true, "minLength": true,
|
||||
}
|
||||
|
||||
func checkStrict(t *testing.T, path string, value any) {
|
||||
@@ -92,6 +93,7 @@ func TestWireSchemasUseOnlyStrictModeKeywords(t *testing.T) {
|
||||
set := retrieve(facts.RawDescription, "expense", d, nil, nil)
|
||||
for name, schema := range map[string]map[string]any{
|
||||
"classification": set.schema(),
|
||||
"batch": set.batchSchema([]string{"r1", "r2", "r3"}),
|
||||
"taxonomy": taxonomySchema(),
|
||||
"csv": csvMappingSchema(CSVMappingRequest{Headers: []string{"Buchung", "Betrag"}}),
|
||||
} {
|
||||
|
||||
Reference in New Issue
Block a user