Lars Nolden
ec99434002
The gpt-5.6 family's zero-data-retention endpoints declare max_completion_tokens, so sending max_tokens under require_parameters excluded every ZDR route and returned HTTP 404 for the whole family. The cap is retired: the strict schema, the finish_reason check and the 64 KiB read cap already bound the response. The model fields now offer the provider's public ZDR catalog filtered by the exact conditions completions are routed under (live endpoint, strict structured outputs), fetched server-side, cached for an hour, and served at GET /api/models; the inputs stay free text so an unlisted model remains usable when the catalog is unreachable.
36 lines
1.3 KiB
Go
36 lines
1.3 KiB
Go
package app
|
|
|
|
import (
|
|
"context"
|
|
"time"
|
|
|
|
"finance-duck/internal/classification"
|
|
)
|
|
|
|
// VerifiedModels lists provider models that currently satisfy the fail-closed
|
|
// routing controls every classification request carries (a live
|
|
// zero-data-retention endpoint with strict structured outputs). Anything else
|
|
// routes to zero providers, so the UI offers only these. The public catalog
|
|
// changes slowly; an hour of caching keeps the settings screen instant without
|
|
// hiding newly usable models for long.
|
|
func (a *App) VerifiedModels(ctx context.Context) ([]classification.VerifiedModel, error) {
|
|
a.mu.Lock()
|
|
if a.verifiedModels != nil && time.Since(a.verifiedModelsAt) < time.Hour {
|
|
cached := append([]classification.VerifiedModel{}, a.verifiedModels...)
|
|
a.mu.Unlock()
|
|
return cached, nil
|
|
}
|
|
// The catalog fetch must not hold a.mu: it is a network call, and the
|
|
// probe client shares only immutable configuration with the classifier.
|
|
probe := &classification.Client{BaseURL: a.classifier.BaseURL, HTTPClient: a.classifier.HTTPClient}
|
|
a.mu.Unlock()
|
|
models, err := probe.VerifiedModels(ctx)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
a.mu.Lock()
|
|
a.verifiedModels, a.verifiedModelsAt = models, time.Now()
|
|
a.mu.Unlock()
|
|
return append([]classification.VerifiedModel{}, models...), nil
|
|
}
|