32 KiB
Query tiles
Status: rough draft, no code changes applied. §4 and §6 SQL are sketches that have not been executed; everything in §2 and §3 was measured.
Decisions taken, on evidence in §2: the ad-hoc surface runs against an engine-enforced read-only snapshot in a second DuckDB instance, never the existing handle; every submitted statement passes a parse-only single-SELECT gate before it reaches the driver; the natural-language model is loopback-only, which removes redaction and lets the real taxonomy into the prompt; chart choice is deterministic from result shape, with the model's hint as a tiebreak only.
Goal: answer "How much did I spend on my hobbies the last 2 weeks?" and "What are my fixed recurring costs for the past two months?" as pinnable tiles, without giving up the journal-is-truth architecture or adding a browser dependency.
1. Three surfaces, one pipeline
| Surface | Input | Who writes the SQL | Useful without the model |
|---|---|---|---|
| Raw query | SQL | you | yes |
| Ask | a question | local model, reviewed by you | — |
| Pinned tile | saved plan | whoever wrote it, once | yes |
question ──► /api/ask ──► {sql, title, chart, assumptions} ──┐
├─► review ──► gate ──► read-only snapshot ──► shape ──► tile ──► queries.finance
raw SQL ─────────────────────────────────────────────────────┘
The model never executes anything. /api/ask returns a plan; /api/query runs one. Keeping them apart means a slow or wrong model degrades one surface instead of all three, and the raw path carries no AI dependency at all.
2. Measured: the obvious implementation is unsafe
A throwaway probe (internal/analytics/probe_test.go, deleted) exercised the sandbox primitives against duckdb-go v2.5.6.
Multi-statement SQL is executed by database/sql. The driver's prepareStmts extracts every statement, then loops for i := 0; i < count-1 preparing and executing each leading statement, returning only the last one prepared (connection.go:215-256 in github.com/duckdb/duckdb-go/v2@v2.5.6). Submitting
SELECT * FROM (SELECT 1) AS q LIMIT 1; DROP TABLE t
dropped the table. Wrapping user SQL in SELECT * FROM ( … ) LIMIT n is therefore not a sandbox: the user closes the paren and appends statements. The wrapper still rejects a bare DROP/SET/COPY/PRAGMA at parse time, so it is a usability filter, not a boundary.
A second instance with a read-only attachment is a boundary. In-process, alongside the existing read-write handle:
-- instance 2, dsn ":memory:"
ATTACH '<data>/cache/finance.duckdb' AS fd (READ_ONLY); -- succeeds while instance 1 holds it RW
USE fd;
SET threads = 2; SET memory_limit = '256MB';
SET enable_external_access = false;
SET lock_configuration = true;
| Probe | Result |
|---|---|
DROP TABLE fd.t / INSERT / UPDATE / CREATE on fd |
Invalid Input Error: Cannot execute statement of type "DROP" on database "fd" which is attached in read-only mode! |
SELECT 1; DROP TABLE fd.t |
same rejection — multi-statement does not help |
read_csv('/etc/passwd') |
Permission Error: … file system operations are disabled by configuration |
COPY fd.t TO '/tmp/x.csv' |
same |
ATTACH '/tmp/evil.db' AS evil |
same |
INSTALL httpfs |
same |
SET enable_external_access = true |
Cannot change configuration option … the configuration has been locked |
CREATE TABLE memory.evil (x INT) |
allowed — scratch in the throwaway in-memory catalog |
SELECT count(*) FROM fd.t, information_schema.columns |
allowed |
10¹³-row cross join, 400 ms context deadline |
interrupted after 900 ms; pool usable afterwards |
| read-write cache after all of the above | intact |
Residual write surface is the ad-hoc instance's own memory catalog. That is harmless — it is discarded when the instance is recreated — and arguably useful for materialising an intermediate result. It is not free of cost: see the temp-directory item in §3.
The attachment is a frozen snapshot. After instance 1 committed an INSERT, the attached instance still reported the old row count. The ad-hoc instance must be recreated whenever the projection is rebuilt. Treat this as a feature: every tile in one page render reads one consistent snapshot, labelled with the existing State.Revision (internal/app/app.go:44).
json_serialize_sql is a free parse-only gate. User SQL is passed as a bound parameter, so there is no injection surface and nothing executes:
SELECT json_serialize_sql(CAST(? AS VARCHAR))
| Submitted | Gate result |
|---|---|
SELECT 1 |
statements=1 nodes=[SELECT_NODE] |
WITH x AS (SELECT 1) SELECT * FROM x |
statements=1 nodes=[SELECT_NODE] |
SELECT 1 -- ; DROP TABLE t |
statements=1 nodes=[SELECT_NODE] — comment handled |
SELECT '; DROP TABLE t' |
statements=1 nodes=[SELECT_NODE] — literal handled |
SELECT 1; DROP TABLE t |
error_type=not implemented, Only SELECT statements can be serialized to json! |
DROP TABLE t / SET … / COPY … TO / PRAGMA version |
same rejection |
this is not sql |
error_type=parser, syntax error at or near "this" |
SELECT * FROM nope |
statements=1 nodes=[SELECT_NODE] — parse only, no catalog binding |
Errors come back as JSON data, not as a raised exception, so the gate yields a clean message instead of a stray engine error. Unknown tables and columns are not caught here; they surface at execution with DuckDB's own "Did you mean" hint, which is the better message anyway.
DECIMAL(24,4) scans as duckdb.Decimal with exact String() → -12345678901234.5678 (types.go:371-386). Money must be serialised through that, never through Float64(); domain.Money (internal/domain/model.go:4) is an exact decimal string for the same reason.
lock_configuration is missing from the existing Open. internal/analytics/store.go:66-73 sets enable_external_access = false and disables extension autoload but never locks the configuration, so any future SQL path on that handle could re-enable filesystem access. Add it there regardless of this feature; the probe confirms DDL still works after the lock, and that a later SET on the same instance is rejected.
3. Execution layer
3.1 Two instances, one process
analytics.Store gains a second, disposable instance:
type Store struct {
db *sql.DB // read-write projection, as today
mu sync.Mutex // guards snapshot swap
snap *snapshot // read-only attachment, nil until first use
path string
}
type snapshot struct {
connector *duckdb.Connector // dsn ":memory:"
db *sql.DB // MaxOpenConns(1)
revision string
}
Lifecycle: created lazily on the first ad-hoc query, closed and recreated when revision differs. internal/app/app.go:167-173 already knows the moment the projection changes (a.indexed = rev); the snapshot is invalidated there.
ATTACH must precede SET enable_external_access = false, because attaching is itself a filesystem operation. The path is interpolated, not bound — ATTACH ? is a parser error, and the path is ours, not user input.
3.2 Do not hold the app mutex
App.Dashboard (app.go:207-217) holds a.mu for the whole query, and store.go:63 pins SetMaxOpenConns(1) deliberately so dashboard snapshots serialise against rebuilds. A five-second ad-hoc scan on either of those would stall every read and write in the process. The ad-hoc path therefore:
- takes
a.mu, callsa.snapshot(ctx)to make the projection current, readsrevision, obtains the read-only handle, releasesa.mu; - runs the query on the snapshot pool with its own deadline.
A rebuild that lands mid-query is harmless: the old attachment stays valid until its last reader is done.
3.3 Limits
| Control | Value | Why |
|---|---|---|
| statements | exactly 1, SELECT_NODE |
§2 gate |
| context deadline | 5 s default, 30 s ceiling | verified to interrupt; pool survives |
| row cap | LIMIT 501, report truncated when 501 come back |
one page of table, bounded JSON |
memory_limit |
256MB, matching store.go:68 |
bounded native memory |
temp_directory |
disabled, or max_temp_directory_size small |
unverified — the probe created a 20 M-row scratch table under a 256 MB memory_limit without error, but that table compresses to almost nothing, so whether it spilled at all was not established. Whether enable_external_access = false covers spill files needs its own probe; the safe default is to fail fast rather than risk filling the data disk |
| concurrency | one ad-hoc query at a time, others get a clear "busy" | single connection; queueing is worse than refusing |
The LIMIT is appended by wrapping the gated statement — SELECT * FROM ( … ) AS q LIMIT 501. The wrapper is safe after the gate has proven the text is one SELECT.
3.4 Result encoding
{
"columns": [{"name": "month", "type": "DATE"}, {"name": "outflow", "type": "DECIMAL(24,4)"}],
"rows": [["2026-08-01", "412.7300"]],
"truncated": false,
"ms": 14,
"revision": "…"
}
Rules: DECIMAL → exact string via duckdb.Decimal.String(); DATE/TIMESTAMP → ISO text, matching the calendar-day discipline in web/src/ui.tsx:73-75; NULL → JSON null; LIST/STRUCT → JSON; everything else by its natural JSON type. columns[].type carries DatabaseTypeName() so the frontend can right-align numerics and pick a chart without guessing.
4. Semantic views — the accuracy lever
Both example questions are four-table joins with three traps:
amountis signed net movement, not an expense (store.go:34-36);kind IN ('transfer', 'investment')must be excluded — the canonical clause isstore.go:242, explained atstore.go:238-241;- category rollups need
category_ancestors, and ancestor groups overlap, so they must never be summed together (store.go:34-36).
A small model will get all three wrong against the base tables. Views make them unrepresentable. They also shrink the task from a BIRD-style multi-join to a Spider-easy single-table query, which is exactly where small models are strong (§11).
Sketches, not yet executed:
-- Every fact, denormalised, with names instead of registry ids.
CREATE VIEW v_tx AS
SELECT t.id, t.booking_date, t.value_date, t.kind, t.currency,
t.account_id, t.category_id, t.merchant_id, -- kept for joins back to base tables
t.amount, -- signed: negative is money out
-t.amount AS outflow, -- positive is money out
a.display_name AS account,
a.institution,
c.name AS category,
c.kind AS category_kind,
COALESCE(m.name, '') AS merchant,
t.counterparty,
t.raw_description AS description,
(SELECT string_agg(c2.name, ' / ' ORDER BY ca.depth DESC)
FROM category_ancestors ca JOIN categories c2 ON c2.id = ca.ancestor_id
WHERE ca.category_id = t.category_id) AS category_path,
(SELECT list(g.name ORDER BY g.name)
FROM transaction_tags tt JOIN tags g ON g.id = tt.tag_id
WHERE tt.transaction_id = t.id) AS tags
FROM transactions t
LEFT JOIN accounts a ON a.id = t.account_id
LEFT JOIN categories c ON c.id = t.category_id
LEFT JOIN merchants m ON m.id = t.merchant_id;
-- Spending and income analytics. Mirrors store.go:242 exactly.
CREATE VIEW v_flow AS SELECT * FROM v_tx WHERE kind NOT IN ('transfer', 'investment');
CREATE VIEW v_spending AS SELECT * FROM v_flow WHERE amount < 0;
CREATE VIEW v_income AS SELECT * FROM v_flow WHERE amount > 0;
-- Correct rollups. One row per transaction × ancestor: never sum across ancestors.
CREATE VIEW v_spending_by_ancestor AS
SELECT s.id, s.booking_date, s.currency, s.outflow,
ca.ancestor_id, c.name AS ancestor, ca.depth
FROM v_spending s
JOIN category_ancestors ca ON ca.category_id = s.category_id
JOIN categories c ON c.id = ca.ancestor_id;
category_path is the cheap win: "hobbies" becomes WHERE category_path ILIKE '%Hobbies%' — no join, no fanout, no ancestor arithmetic. v_spending_by_ancestor exists for when a correct grouped total is needed.
A calendar spine, so empty months render as zero bars rather than vanishing:
CREATE VIEW v_month AS
SELECT month FROM (
SELECT DISTINCT date_trunc('month', booking_date)::DATE AS month FROM transactions
); -- draft: a gap-free generate_series between min and max is what is actually wanted,
-- and must behave on an empty dataset where min(booking_date) is NULL
Recurring costs — this is the second example question, and it is worth owning as a view rather than hoping the model derives it:
CREATE VIEW v_recurring AS
WITH s AS (
SELECT COALESCE(NULLIF(merchant, ''), counterparty) AS payee, booking_date, outflow, category_path
FROM v_spending
WHERE COALESCE(NULLIF(merchant, ''), counterparty) <> ''
), d AS (
SELECT *, date_diff('day', lag(booking_date) OVER (PARTITION BY payee ORDER BY booking_date),
booking_date) AS gap
FROM s
), g AS (
SELECT payee,
count(*) AS hits,
median(gap) AS gap_days,
median(outflow) AS typical_amount,
stddev_pop(outflow) / nullif(avg(outflow), 0) AS amount_variation,
max(booking_date) AS last_seen,
any_value(category_path) AS category_path
FROM d GROUP BY payee
)
SELECT *,
CASE WHEN gap_days BETWEEN 6 AND 8 THEN 'weekly'
WHEN gap_days BETWEEN 13 AND 16 THEN 'biweekly'
WHEN gap_days BETWEEN 25 AND 35 THEN 'monthly'
WHEN gap_days BETWEEN 85 AND 95 THEN 'quarterly'
WHEN gap_days BETWEEN 350 AND 380 THEN 'yearly'
END AS cadence,
typical_amount * 30.44 / gap_days AS monthly_equivalent
FROM g
WHERE hits >= 3 AND amount_variation < 0.15;
Cadence classification rather than a hardcoded 28–31 day window, so a yearly insurance premium and a weekly grocery standing order both land correctly, normalised to monthly_equivalent. The amount_variation filter is what separates a subscription from a coffee habit; grouping by payee alone (rather than payee × rounded amount) survives a price increase. Both choices are open — §14.
4.1 Mechanics and the stability contract
Views are part of schema (store.go:97-106) and must be created after postings is populated (store.go:209-215). Rebuild drops tables by name (store.go:119); DuckDB refuses to drop a table a view depends on, so the view names must join that list, dropped before the tables.
Pinned SQL in queries.finance outlives every rebuild, so view names and columns become a public interface. Base tables stay reachable and undocumented: power users may use them, nothing promises they are stable. A tile whose SQL no longer binds shows the engine error in place of its chart — visible breakage, never a silent zero.
5. Statement gate
POST /api/query {sql}
├─ reject empty / >8 KiB
├─ SELECT json_serialize_sql(CAST(? AS VARCHAR)) ── parse only, user SQL bound as a value
│ ├─ error:true → 400 with error_message
│ └─ len(statements) != 1 → 400 "one SELECT statement at a time"
│ └─ node.type != SELECT_NODE → 400 same
├─ wrap: SELECT * FROM ( <sql> ) AS q LIMIT 501
└─ run on the read-only snapshot with a deadline
The gate runs on the read-write handle (it is a pure function of a string) or on the snapshot — either works; the snapshot avoids touching the serialised handle at all.
Engine errors are returned verbatim: it is the user's own SQL, and DuckDB's messages are good. Nothing about a query or its results is logged, consistent with the no-provider-logging discipline in internal/classification/client.go.
6. Ask: question → plan
6.1 Why a new package, not classification.Client
| Concern | classification |
ask |
|---|---|---|
| Endpoint | OpenRouter by default; http:// allowed only for loopback (client.go:321) |
loopback required |
| Request body | hardcodes provider: {data_collection: deny, zdr: true, require_parameters: true} (client.go:302) |
no provider block; a local server may reject unknown fields |
| Privacy | redactor() strips IBANs, own names, digit-bearing tokens (privacy.go:53-107) |
no redaction — redacting the schema destroys the query |
| Sent content | one transaction | schema, taxonomy, dates; no transaction rows |
| Failure | falls back to unclassified enrichment with provenance | returns an error; the raw SQL path is unaffected |
The inversion is the point: because the model is loopback-only, redaction is unnecessary, and the real category names, tag names and account names can go into the prompt — which is precisely what NL→SQL needs to write a correct WHERE. A redacted schema yields useless SQL, so routing this through OpenRouter is not a smaller version of the feature; it is a different, broken one.
New package internal/askql. classification is untouched.
6.2 Transport
Target llama-server (llama.cpp). Ollama's /v1/chat/completions shim mistranslates or ignores response_format.json_schema; grammar-constrained decoding is what makes a small model emit parseable JSON every time, so it is not optional. llama-server honours the OpenAI shape and compiles the schema to a grammar.
Deployment: nixpkgs ships services.llama-cpp with model, host, port, extraFlags — one systemd unit next to nix/service.nix, loopback-bound, firewall untouched.
Configuration follows the existing hand-rolled reader (app.go:100-127, writer at app.go:376-380), which rejects unknown keys, so both sides need the new pair:
ask_base_url = "http://127.0.0.1:8081/v1"
ask_model = "qwen-coder"
Empty ask_base_url disables the Ask surface; the query and tile surfaces keep working. A non-loopback host is a startup error, not a warning.
server.go:114 sets CSP connect-src 'self', so the browser cannot reach the model even on loopback. The proxy through Go is mandatory, and is the right place for it anyway.
6.3 Prompt
Everything the model needs, nothing it does not:
- the view DDL from §4, column types and one-line comments — views only, base tables omitted;
- the taxonomy: every category with its
category_path, every tag, the top ~50 merchants by transaction count.finance/categories.financeis 364 B today; the whole taxonomy fits with room to spare; - today's date, plus
min(booking_date)andmax(booking_date). The model has no clock; without this, "last 2 weeks" silently returns zero rows; - a handful of curated question → SQL exemplars. Cheapest remaining accuracy gain (§11);
- the framing already used in
classification: user content is untrusted data, never instructions.
No transaction rows. Registry names are user-influenced (classification writes merchant names from bank text), so a hostile merchant name could in principle carry instructions — bounded by the fact that the only thing the model can produce is a read-only SELECT that you see before it runs. Worst case is a confusing query you decline.
6.4 Response
Strict JSON schema, one round trip, no tool loop:
{
"type": "object", "additionalProperties": false,
"required": ["sql", "title", "chart", "assumptions"],
"properties": {
"sql": {"type": "string"},
"title": {"type": "string"},
"chart": {"enum": ["kpi", "bar", "line", "table"]},
"x": {"type": ["string", "null"]},
"y": {"type": "array", "items": {"type": "string"}},
"assumptions": {"type": "string"}
}
}
assumptions is load-bearing, not decoration:
hobbies → categories under
Expenses / Leisure / Hobbies; last 2 weeks →2026-08-29…2026-09-11
finance/categories.finance currently holds only the four seed categories, so "hobbies" may not exist as a category at all and may resolve to a tag or a merchant. Making the model state its mapping is what lets you correct it in one edit instead of mistrusting every answer.
Repair: on a bind error, one retry with the engine message appended, then stop. Never an unattended loop. Both attempts are shown.
Serialisation: one inference at a time, 60 s deadline, Abort in the UI. No rate controller — a loopback model has no quota, and ratelimit.Controller exists for provider cooldowns.
7. Visualization
Deterministic from the result shape; the model's chart is a tiebreak, never an instruction.
| Shape | Render |
|---|---|
| 1 row × 1 numeric column | KPI, with the previous-period comparison idiom already in Overview |
DATE-like column + 1 numeric |
line/area |
| text column + 1 numeric, ≤ 24 rows | horizontal bars |
| anything else | table |
Reuse what exists: .bar-chart, .bar-column, .bar-track, .bar, .bar-value, .bar-label (web/src/styles.css:548-599), MonthlyChart (Overview.tsx:366-410) and GroupPanel (Overview.tsx:411-480). A line chart needs a small hand-rolled SVG. No charting dependency: web/package.json has four runtime deps and the CSP serves scripts from 'self' only; a self-hosted workspace that bundles its own fonts rather than fetching them (main.tsx:37-39) should not grow a chart library for four chart types.
Explicitly refused: letting the model emit chart code or a Vega-style spec it invents. That is arbitrary JS in a service with no application authentication.
8. Persistence and UI
8.1 queries.finance
Pinned tiles are user content, so they belong in the journal, not in state/. Add queries.finance to registryFiles (internal/journal/codec.go:22); the codec is json-tag driven, so the block grammar follows from the struct:
query {
id: "qry_hobbies_2w"
title: "Hobby spend, last 2 weeks"
question: "How much did I spend on my hobbies the last 2 weeks?"
sql: "SELECT sum(outflow) AS spent FROM v_spending WHERE category_path ILIKE '%Hobbies%' AND booking_date >= current_date - INTERVAL 14 DAY"
chart: "kpi"
position: 1
}
Diffable, hand-editable, backed up with everything else, survives a cache wipe. Writes go through the existing App.mutate revision-conflict path (app.go:199-201) and the s.account/s.category handler idiom (server.go:239-293).
domain.Dataset gains Queries []Query, and domain.Validate gains id/title/SQL checks. Whether a stored query is gated at write time or only at run time is an open item — gating at write time means a view rename can make the journal unloadable, which is worse than a broken tile.
8.2 The page
A new nav entry in navigation (main.tsx:43-54), between Wealth and Settings. Name open (§14).
Layout: a tile grid. Each tile shows title, chart, and a footer with row count, elapsed ms, the snapshot revision, and a disclosure that reveals the SQL. Tiles re-run when revision changes, exactly like Overview's effect (Overview.tsx:29-64).
Composer: a question box and a SQL editor, side by side rather than staged, so the two paths are visibly the same pipeline. Ask fills the SQL box and shows assumptions above it; Run executes; Pin writes to queries.finance. Nothing auto-runs and nothing auto-pins.
Optional, once tiles exist: render the first few pinned tiles on Overview. That is the "tile" framing in its most useful form, and it costs one component reuse.
9. API
| Route | Body | Returns |
|---|---|---|
POST /api/query |
{sql, limit?} |
{columns, rows, truncated, ms, revision} |
POST /api/ask |
{question} |
{sql, title, chart, x, y, assumptions, model, ms} |
POST /api/queries |
{id?, title, question, sql, chart, position, delete?} |
State |
Registered in New (server.go:40-74), decoded with decode (server.go:192-204), answered with respond (server.go:205-216). All three are POST, so they pick up the existing Sec-Fetch-Site, Origin and application/json guards at server.go:130-153.
10. Phasing
- Sandbox, gate,
/api/query, result table,queries.financetiles. No model. Independently useful, and it is where all the engineering risk lives. - Deterministic chart inference and the chart picker.
- Views (§4) and a schema-browser panel. Makes hand-written SQL pleasant and is the prerequisite for step 4 being any good.
internal/askql, llama-server,/api/ask, assumptions, one repair.
Doing 4 before 3 is the main way this disappoints.
11. Work breakdown
| File | Change |
|---|---|
internal/analytics/store.go |
add SET lock_configuration = true to Open; add views to schema and to the drop list at :119; create views after postings |
internal/analytics/adhoc.go (new) |
snapshot lifecycle, ATTACH … (READ_ONLY), lockdown, gate, wrap, run, result encoding incl. duckdb.Decimal |
internal/analytics/query.go |
unchanged |
internal/app/app.go |
invalidate the snapshot where a.indexed = rev (:171); AdHoc entry point that releases a.mu before running; ask_base_url / ask_model config keys and writer |
internal/askql/ (new) |
llama-server client, schema card, prompt, strict schema, one repair |
internal/domain/model.go, domain.go |
Query struct, Dataset.Queries, validation |
internal/journal/codec.go |
queries.finance in registryFiles |
internal/server/server.go |
three routes and handlers |
web/src/api.ts |
QueryResult, Plan, Query types |
web/src/Queries.tsx (new) |
page, tile grid, composer, chart inference, table |
web/src/main.tsx |
nav entry, route |
web/src/styles.css |
tile grid, table, line-chart svg |
nix/service.nix, README.md, OPERATIONS.txt |
llama-cpp unit, setup, operational notes |
12. Calibration
Published execution accuracy for general small coder models on realistic multi-table schemas: ~39 % at 7B, 47 % at 14B, 50 % at 32B on BIRD (cross-family size × technique frontier). SQL-specialised models do markedly better — Arctic-Text2SQL-R1 14B reaches 64.9 % BIRD-dev / 86.8 % Spider-test; XiYanSQL-QwenCoder-32B reaches 69 % BIRD-test, with 3B/7B/14B variants.
Two consequences already built into this design: the schema must be small and denormalised (§4 turns a BIRD-hard join into a Spider-easy single-table query), and the SQL must always be visible and one click from editable. Plan for "wrong a third of the time, obviously wrong when it is".
Latency, inferred not measured: a 7B Q4_K_M emitting ~200 constrained JSON tokens on a 7840U-class CPU lands around 15–25 s end to end. Tolerable for Ask, annoying for iteration — another reason the raw SQL path must stand alone. An SQL-specialised 3B is worth benchmarking against your own questions before committing to 7B.
13. Risks
| Risk | Mitigation |
|---|---|
| Driver executes leading statements of a multi-statement string | read-only attachment (engine-enforced) and the parse-only gate; neither alone |
| Ad-hoc query stalls the whole app | never hold a.mu during execution; separate pool; one query at a time |
| Stale snapshot answers with pre-rebuild data | invalidate at app.go:171; every result carries revision; tiles re-run on change |
| Spilling fills the data disk | disable temp_directory or cap max_temp_directory_size on the ad-hoc instance — needs a probe, enable_external_access = false does not cover spill |
| Plausible but wrong SQL believed | assumptions shown, SQL shown, row count and revision in the footer, previous-period comparison for KPIs |
| Pinned SQL breaks when views change | views are a versioned interface; broken tile shows the engine error, never a silent zero |
| Registry names carry prompt injection | model output is only a read-only SELECT you approve before it runs |
/api/query is a new privilege class on a service with no auth |
the sandbox is the mitigation; without §2 and §3 this endpoint is an arbitrary file read |
| Model unavailable or slow | Ask degrades independently; query and tile surfaces have no AI dependency |
14. Open decisions
- Page name and placement —
Ask,Lab,Query? A new nav page, tiles embedded inOverview, or both? - Loopback-only for the ask model — accept as a hard invariant, or is an OpenRouter fallback wanted (which forces redaction back in and, per §6.1, breaks the feature)?
- Views as the documented query surface, base tables explicitly unstable — accept?
v_recurringgrouping — payee alone with anamount_variationfilter (survives price rises, as drafted), or payee × rounded amount (splits on a price rise, but separates two different subscriptions to the same payee)?- Where inference runs and the RAM budget — decides 3B vs 7B vs 14B, and whether it shares the service host.
- Stored-query validation timing — gate SQL at write time (a view rename can make the journal unloadable) or only at run time (a broken tile, loadable journal)?
- Should
v_monthbe a gap-free spine generated betweenminandmax, and what should it do on an empty dataset?
15. Implementation notes
Only the parts where the obvious implementation is wrong.
15.1 Attach before locking down
// ATTACH is itself a filesystem operation, so external access must stay enabled
// until the snapshot is attached. The path is interpolated because ATTACH takes
// no parameters; it is our own path, never user input.
for _, stmt := range []string{
"ATTACH '" + path + "' AS fd (READ_ONLY)",
"USE fd",
"SET threads = 2",
"SET memory_limit = '256MB'",
"SET enable_external_access = false",
"SET lock_configuration = true",
} { … }
Reversing the last two lines, or setting enable_external_access = false before the ATTACH, fails with Permission Error: Cannot access file ….
15.2 The gate must bind, not format
// The submitted text is a value, not code: json_serialize_sql parses it without
// executing it, and a bound parameter leaves no injection surface. The cast is
// required — an untyped parameter yields
// "json_serialize_sql first argument must be a VARCHAR".
row := db.QueryRowContext(ctx, "SELECT json_serialize_sql(CAST(? AS VARCHAR))", sql)
The result scans as map[string]any through this driver, not as a string. {"error": true, "error_type": …, "error_message": …} on rejection; {"statements": [{"node": {"type": "SELECT_NODE"}}]} on acceptance.
15.3 Money must not become a float
// duckdb.Decimal.String() is exact; Float64() is not. domain.Money is a decimal
// string for the same reason, and the dashboard already renders exact strings.
case duckdb.Decimal:
cell = v.String()
15.4 Views join the drop list
Rebuild drops tables by name at store.go:119. DuckDB refuses to drop a table a view depends on, so a view left behind breaks the next rebuild — the failure surfaces as a.indexError and takes the whole dashboard down, not just the tiles. Drop views first, in dependency order, or drop with CASCADE.
15.5 The local request body is not the OpenRouter one
classification.complete sends provider: {data_collection: "deny", zdr: true, require_parameters: true} (client.go:302). Those keys are OpenRouter routing directives; a local server has no providers, and a strict OpenAI-compatible server may reject unknown fields. askql sends model, messages, stream: false, max_tokens, response_format and nothing else.
15.6 The config reader rejects unknown keys
app.go:112 is a switch over known keys with a hard error on anything else, and the writer at app.go:376-380 rewrites the whole file. Adding ask_base_url / ask_model to one side only makes an existing config.toml unreadable after the first save.