# Query tiles Status: rough draft, no code changes applied. §4 and §6 SQL are sketches that have not been executed; everything in §2 and §3 was measured. Decisions taken, on evidence in §2: the ad-hoc surface runs against an **engine-enforced read-only snapshot** in a second DuckDB instance, never the existing handle; every submitted statement passes a **parse-only single-`SELECT` gate** before it reaches the driver; the natural-language model is **loopback-only**, which removes redaction and lets the real taxonomy into the prompt; chart choice is **deterministic from result shape**, with the model's hint as a tiebreak only. Goal: answer *"How much did I spend on my hobbies the last 2 weeks?"* and *"What are my fixed recurring costs for the past two months?"* as pinnable tiles, without giving up the journal-is-truth architecture or adding a browser dependency. ## 1. Three surfaces, one pipeline | Surface | Input | Who writes the SQL | Useful without the model | | --- | --- | --- | --- | | Raw query | SQL | you | yes | | Ask | a question | local model, reviewed by you | — | | Pinned tile | saved plan | whoever wrote it, once | yes | ``` question ──► /api/ask ──► {sql, title, chart, assumptions} ──┐ ├─► review ──► gate ──► read-only snapshot ──► shape ──► tile ──► queries.finance raw SQL ─────────────────────────────────────────────────────┘ ``` The model never executes anything. `/api/ask` returns a plan; `/api/query` runs one. Keeping them apart means a slow or wrong model degrades one surface instead of all three, and the raw path carries no AI dependency at all. ## 2. Measured: the obvious implementation is unsafe A throwaway probe (`internal/analytics/probe_test.go`, deleted) exercised the sandbox primitives against duckdb-go v2.5.6. **Multi-statement SQL is executed by `database/sql`.** The driver's `prepareStmts` extracts every statement, then loops `for i := 0; i < count-1` **preparing and executing each leading statement**, returning only the last one prepared (`connection.go:215-256` in `github.com/duckdb/duckdb-go/v2@v2.5.6`). Submitting ```sql SELECT * FROM (SELECT 1) AS q LIMIT 1; DROP TABLE t ``` dropped the table. Wrapping user SQL in `SELECT * FROM ( … ) LIMIT n` is therefore **not** a sandbox: the user closes the paren and appends statements. The wrapper still rejects a bare `DROP`/`SET`/`COPY`/`PRAGMA` at parse time, so it is a usability filter, not a boundary. **A second instance with a read-only attachment is a boundary.** In-process, alongside the existing read-write handle: ```sql -- instance 2, dsn ":memory:" ATTACH '/cache/finance.duckdb' AS fd (READ_ONLY); -- succeeds while instance 1 holds it RW USE fd; SET threads = 2; SET memory_limit = '256MB'; SET enable_external_access = false; SET lock_configuration = true; ``` | Probe | Result | | --- | --- | | `DROP TABLE fd.t` / `INSERT` / `UPDATE` / `CREATE` on `fd` | `Invalid Input Error: Cannot execute statement of type "DROP" on database "fd" which is attached in read-only mode!` | | `SELECT 1; DROP TABLE fd.t` | same rejection — multi-statement does not help | | `read_csv('/etc/passwd')` | `Permission Error: … file system operations are disabled by configuration` | | `COPY fd.t TO '/tmp/x.csv'` | same | | `ATTACH '/tmp/evil.db' AS evil` | same | | `INSTALL httpfs` | same | | `SET enable_external_access = true` | `Cannot change configuration option … the configuration has been locked` | | `CREATE TABLE memory.evil (x INT)` | **allowed** — scratch in the throwaway in-memory catalog | | `SELECT count(*) FROM fd.t`, `information_schema.columns` | allowed | | 10¹³-row cross join, 400 ms `context` deadline | interrupted after 900 ms; pool usable afterwards | | read-write cache after all of the above | intact | Residual write surface is the ad-hoc instance's own `memory` catalog. That is harmless — it is discarded when the instance is recreated — and arguably useful for materialising an intermediate result. It is *not* free of cost: see the temp-directory item in §3. **The attachment is a frozen snapshot.** After instance 1 committed an `INSERT`, the attached instance still reported the old row count. The ad-hoc instance must be recreated whenever the projection is rebuilt. Treat this as a feature: every tile in one page render reads one consistent snapshot, labelled with the existing `State.Revision` (`internal/app/app.go:44`). **`json_serialize_sql` is a free parse-only gate.** User SQL is passed as a *bound parameter*, so there is no injection surface and nothing executes: ```sql SELECT json_serialize_sql(CAST(? AS VARCHAR)) ``` | Submitted | Gate result | | --- | --- | | `SELECT 1` | `statements=1 nodes=[SELECT_NODE]` | | `WITH x AS (SELECT 1) SELECT * FROM x` | `statements=1 nodes=[SELECT_NODE]` | | `SELECT 1 -- ; DROP TABLE t` | `statements=1 nodes=[SELECT_NODE]` — comment handled | | `SELECT '; DROP TABLE t'` | `statements=1 nodes=[SELECT_NODE]` — literal handled | | `SELECT 1; DROP TABLE t` | `error_type=not implemented`, `Only SELECT statements can be serialized to json!` | | `DROP TABLE t` / `SET …` / `COPY … TO` / `PRAGMA version` | same rejection | | `this is not sql` | `error_type=parser`, `syntax error at or near "this"` | | `SELECT * FROM nope` | `statements=1 nodes=[SELECT_NODE]` — parse only, no catalog binding | Errors come back as JSON data, not as a raised exception, so the gate yields a clean message instead of a stray engine error. Unknown tables and columns are *not* caught here; they surface at execution with DuckDB's own "Did you mean" hint, which is the better message anyway. **`DECIMAL(24,4)` scans as `duckdb.Decimal`** with exact `String()` → `-12345678901234.5678` (`types.go:371-386`). Money must be serialised through that, never through `Float64()`; `domain.Money` (`internal/domain/model.go:4`) is an exact decimal string for the same reason. **`lock_configuration` is missing from the existing `Open`.** `internal/analytics/store.go:66-73` sets `enable_external_access = false` and disables extension autoload but never locks the configuration, so any future SQL path on that handle could re-enable filesystem access. Add it there regardless of this feature; the probe confirms DDL still works after the lock, and that a later `SET` on the same instance is rejected. ## 3. Execution layer ### 3.1 Two instances, one process `analytics.Store` gains a second, disposable instance: ```go type Store struct { db *sql.DB // read-write projection, as today mu sync.Mutex // guards snapshot swap snap *snapshot // read-only attachment, nil until first use path string } type snapshot struct { connector *duckdb.Connector // dsn ":memory:" db *sql.DB // MaxOpenConns(1) revision string } ``` Lifecycle: created lazily on the first ad-hoc query, closed and recreated when `revision` differs. `internal/app/app.go:167-173` already knows the moment the projection changes (`a.indexed = rev`); the snapshot is invalidated there. `ATTACH` must precede `SET enable_external_access = false`, because attaching is itself a filesystem operation. The path is interpolated, not bound — `ATTACH ?` is a parser error, and the path is ours, not user input. ### 3.2 Do not hold the app mutex `App.Dashboard` (`app.go:207-217`) holds `a.mu` for the whole query, and `store.go:63` pins `SetMaxOpenConns(1)` deliberately so dashboard snapshots serialise against rebuilds. A five-second ad-hoc scan on either of those would stall every read and write in the process. The ad-hoc path therefore: 1. takes `a.mu`, calls `a.snapshot(ctx)` to make the projection current, reads `revision`, obtains the read-only handle, **releases `a.mu`**; 2. runs the query on the snapshot pool with its own deadline. A rebuild that lands mid-query is harmless: the old attachment stays valid until its last reader is done. ### 3.3 Limits | Control | Value | Why | | --- | --- | --- | | statements | exactly 1, `SELECT_NODE` | §2 gate | | context deadline | 5 s default, 30 s ceiling | verified to interrupt; pool survives | | row cap | `LIMIT 501`, report `truncated` when 501 come back | one page of table, bounded JSON | | `memory_limit` | `256MB`, matching `store.go:68` | bounded native memory | | `temp_directory` | disabled, or `max_temp_directory_size` small | **unverified** — the probe created a 20 M-row scratch table under a 256 MB `memory_limit` without error, but that table compresses to almost nothing, so whether it spilled at all was not established. Whether `enable_external_access = false` covers spill files needs its own probe; the safe default is to fail fast rather than risk filling the data disk | | concurrency | one ad-hoc query at a time, others get a clear "busy" | single connection; queueing is worse than refusing | The `LIMIT` is appended by wrapping the gated statement — `SELECT * FROM ( … ) AS q LIMIT 501`. The wrapper is safe *after* the gate has proven the text is one `SELECT`. ### 3.4 Result encoding ```json { "columns": [{"name": "month", "type": "DATE"}, {"name": "outflow", "type": "DECIMAL(24,4)"}], "rows": [["2026-08-01", "412.7300"]], "truncated": false, "ms": 14, "revision": "…" } ``` Rules: `DECIMAL` → exact string via `duckdb.Decimal.String()`; `DATE`/`TIMESTAMP` → ISO text, matching the calendar-day discipline in `web/src/ui.tsx:73-75`; `NULL` → JSON `null`; `LIST`/`STRUCT` → JSON; everything else by its natural JSON type. `columns[].type` carries `DatabaseTypeName()` so the frontend can right-align numerics and pick a chart without guessing. ## 4. Semantic views — the accuracy lever Both example questions are four-table joins with three traps: 1. `amount` is **signed net movement**, not an expense (`store.go:34-36`); 2. `kind IN ('transfer', 'investment')` must be excluded — the canonical clause is `store.go:242`, explained at `store.go:238-241`; 3. category rollups need `category_ancestors`, and ancestor groups **overlap**, so they must never be summed together (`store.go:34-36`). A small model will get all three wrong against the base tables. Views make them unrepresentable. They also shrink the task from a BIRD-style multi-join to a Spider-easy single-table query, which is exactly where small models are strong (§11). Sketches, not yet executed: ```sql -- Every fact, denormalised, with names instead of registry ids. CREATE VIEW v_tx AS SELECT t.id, t.booking_date, t.value_date, t.kind, t.currency, t.account_id, t.category_id, t.merchant_id, -- kept for joins back to base tables t.amount, -- signed: negative is money out -t.amount AS outflow, -- positive is money out a.display_name AS account, a.institution, c.name AS category, c.kind AS category_kind, COALESCE(m.name, '') AS merchant, t.counterparty, t.raw_description AS description, (SELECT string_agg(c2.name, ' / ' ORDER BY ca.depth DESC) FROM category_ancestors ca JOIN categories c2 ON c2.id = ca.ancestor_id WHERE ca.category_id = t.category_id) AS category_path, (SELECT list(g.name ORDER BY g.name) FROM transaction_tags tt JOIN tags g ON g.id = tt.tag_id WHERE tt.transaction_id = t.id) AS tags FROM transactions t LEFT JOIN accounts a ON a.id = t.account_id LEFT JOIN categories c ON c.id = t.category_id LEFT JOIN merchants m ON m.id = t.merchant_id; -- Spending and income analytics. Mirrors store.go:242 exactly. CREATE VIEW v_flow AS SELECT * FROM v_tx WHERE kind NOT IN ('transfer', 'investment'); CREATE VIEW v_spending AS SELECT * FROM v_flow WHERE amount < 0; CREATE VIEW v_income AS SELECT * FROM v_flow WHERE amount > 0; -- Correct rollups. One row per transaction × ancestor: never sum across ancestors. CREATE VIEW v_spending_by_ancestor AS SELECT s.id, s.booking_date, s.currency, s.outflow, ca.ancestor_id, c.name AS ancestor, ca.depth FROM v_spending s JOIN category_ancestors ca ON ca.category_id = s.category_id JOIN categories c ON c.id = ca.ancestor_id; ``` `category_path` is the cheap win: *"hobbies"* becomes `WHERE category_path ILIKE '%Hobbies%'` — no join, no fanout, no ancestor arithmetic. `v_spending_by_ancestor` exists for when a correct grouped total is needed. A calendar spine, so empty months render as zero bars rather than vanishing: ```sql CREATE VIEW v_month AS SELECT month FROM ( SELECT DISTINCT date_trunc('month', booking_date)::DATE AS month FROM transactions ); -- draft: a gap-free generate_series between min and max is what is actually wanted, -- and must behave on an empty dataset where min(booking_date) is NULL ``` Recurring costs — this *is* the second example question, and it is worth owning as a view rather than hoping the model derives it: ```sql CREATE VIEW v_recurring AS WITH s AS ( SELECT COALESCE(NULLIF(merchant, ''), counterparty) AS payee, booking_date, outflow, category_path FROM v_spending WHERE COALESCE(NULLIF(merchant, ''), counterparty) <> '' ), d AS ( SELECT *, date_diff('day', lag(booking_date) OVER (PARTITION BY payee ORDER BY booking_date), booking_date) AS gap FROM s ), g AS ( SELECT payee, count(*) AS hits, median(gap) AS gap_days, median(outflow) AS typical_amount, stddev_pop(outflow) / nullif(avg(outflow), 0) AS amount_variation, max(booking_date) AS last_seen, any_value(category_path) AS category_path FROM d GROUP BY payee ) SELECT *, CASE WHEN gap_days BETWEEN 6 AND 8 THEN 'weekly' WHEN gap_days BETWEEN 13 AND 16 THEN 'biweekly' WHEN gap_days BETWEEN 25 AND 35 THEN 'monthly' WHEN gap_days BETWEEN 85 AND 95 THEN 'quarterly' WHEN gap_days BETWEEN 350 AND 380 THEN 'yearly' END AS cadence, typical_amount * 30.44 / gap_days AS monthly_equivalent FROM g WHERE hits >= 3 AND amount_variation < 0.15; ``` Cadence classification rather than a hardcoded 28–31 day window, so a yearly insurance premium and a weekly grocery standing order both land correctly, normalised to `monthly_equivalent`. The `amount_variation` filter is what separates a subscription from a coffee habit; grouping by payee alone (rather than payee × rounded amount) survives a price increase. Both choices are open — §14. ### 4.1 Mechanics and the stability contract Views are part of `schema` (`store.go:97-106`) and must be created after `postings` is populated (`store.go:209-215`). `Rebuild` drops tables by name (`store.go:119`); DuckDB refuses to drop a table a view depends on, so **the view names must join that list, dropped before the tables**. Pinned SQL in `queries.finance` outlives every rebuild, so view names and columns become a **public interface**. Base tables stay reachable and undocumented: power users may use them, nothing promises they are stable. A tile whose SQL no longer binds shows the engine error in place of its chart — visible breakage, never a silent zero. ## 5. Statement gate ``` POST /api/query {sql} ├─ reject empty / >8 KiB ├─ SELECT json_serialize_sql(CAST(? AS VARCHAR)) ── parse only, user SQL bound as a value │ ├─ error:true → 400 with error_message │ └─ len(statements) != 1 → 400 "one SELECT statement at a time" │ └─ node.type != SELECT_NODE → 400 same ├─ wrap: SELECT * FROM ( ) AS q LIMIT 501 └─ run on the read-only snapshot with a deadline ``` The gate runs on the read-write handle (it is a pure function of a string) or on the snapshot — either works; the snapshot avoids touching the serialised handle at all. Engine errors are returned verbatim: it is the user's own SQL, and DuckDB's messages are good. Nothing about a query or its results is logged, consistent with the no-provider-logging discipline in `internal/classification/client.go`. ## 6. Ask: question → plan ### 6.1 Why a new package, not `classification.Client` | Concern | `classification` | ask | | --- | --- | --- | | Endpoint | OpenRouter by default; `http://` allowed only for loopback (`client.go:321`) | loopback **required** | | Request body | hardcodes `provider: {data_collection: deny, zdr: true, require_parameters: true}` (`client.go:302`) | no provider block; a local server may reject unknown fields | | Privacy | `redactor()` strips IBANs, own names, digit-bearing tokens (`privacy.go:53-107`) | no redaction — redacting the schema destroys the query | | Sent content | one transaction | schema, taxonomy, dates; **no transaction rows** | | Failure | falls back to unclassified enrichment with provenance | returns an error; the raw SQL path is unaffected | The inversion is the point: **because the model is loopback-only, redaction is unnecessary**, and the real category names, tag names and account names can go into the prompt — which is precisely what NL→SQL needs to write a correct `WHERE`. A redacted schema yields useless SQL, so routing this through OpenRouter is not a smaller version of the feature; it is a different, broken one. New package `internal/askql`. `classification` is untouched. ### 6.2 Transport Target **llama-server** (llama.cpp). Ollama's `/v1/chat/completions` shim [mistranslates or ignores](https://github.com/ollama/ollama/issues/10001) `response_format.json_schema`; grammar-constrained decoding is what makes a small model emit parseable JSON every time, so it is not optional. llama-server honours the OpenAI shape and compiles the schema to a grammar. Deployment: nixpkgs ships [`services.llama-cpp`](https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/misc/llama-cpp.nix) with `model`, `host`, `port`, `extraFlags` — one systemd unit next to `nix/service.nix`, loopback-bound, firewall untouched. Configuration follows the existing hand-rolled reader (`app.go:100-127`, writer at `app.go:376-380`), which rejects unknown keys, so both sides need the new pair: ```toml ask_base_url = "http://127.0.0.1:8081/v1" ask_model = "qwen-coder" ``` Empty `ask_base_url` disables the Ask surface; the query and tile surfaces keep working. A non-loopback host is a startup error, not a warning. `server.go:114` sets CSP `connect-src 'self'`, so the browser cannot reach the model even on loopback. The proxy through Go is mandatory, and is the right place for it anyway. ### 6.3 Prompt Everything the model needs, nothing it does not: - the view DDL from §4, column types and one-line comments — **views only**, base tables omitted; - the taxonomy: every category with its `category_path`, every tag, the top ~50 merchants by transaction count. `finance/categories.finance` is 364 B today; the whole taxonomy fits with room to spare; - **today's date**, plus `min(booking_date)` and `max(booking_date)`. The model has no clock; without this, "last 2 weeks" silently returns zero rows; - a handful of curated question → SQL exemplars. Cheapest remaining accuracy gain (§11); - the framing already used in `classification`: user content is untrusted data, never instructions. No transaction rows. Registry names are user-influenced (classification writes merchant names from bank text), so a hostile merchant name could in principle carry instructions — bounded by the fact that the only thing the model can produce is a read-only `SELECT` that **you see before it runs**. Worst case is a confusing query you decline. ### 6.4 Response Strict JSON schema, one round trip, no tool loop: ```json { "type": "object", "additionalProperties": false, "required": ["sql", "title", "chart", "assumptions"], "properties": { "sql": {"type": "string"}, "title": {"type": "string"}, "chart": {"enum": ["kpi", "bar", "line", "table"]}, "x": {"type": ["string", "null"]}, "y": {"type": "array", "items": {"type": "string"}}, "assumptions": {"type": "string"} } } ``` `assumptions` is load-bearing, not decoration: > hobbies → categories under `Expenses / Leisure / Hobbies`; last 2 weeks → `2026-08-29` … `2026-09-11` `finance/categories.finance` currently holds only the four seed categories, so *"hobbies"* may not exist as a category at all and may resolve to a tag or a merchant. Making the model state its mapping is what lets you correct it in one edit instead of mistrusting every answer. Repair: on a bind error, one retry with the engine message appended, then stop. Never an unattended loop. Both attempts are shown. Serialisation: one inference at a time, 60 s deadline, `Abort` in the UI. No rate controller — a loopback model has no quota, and `ratelimit.Controller` exists for provider cooldowns. ## 7. Visualization Deterministic from the result shape; the model's `chart` is a tiebreak, never an instruction. | Shape | Render | | --- | --- | | 1 row × 1 numeric column | KPI, with the previous-period comparison idiom already in `Overview` | | `DATE`-like column + 1 numeric | line/area | | text column + 1 numeric, ≤ 24 rows | horizontal bars | | anything else | table | Reuse what exists: `.bar-chart`, `.bar-column`, `.bar-track`, `.bar`, `.bar-value`, `.bar-label` (`web/src/styles.css:548-599`), `MonthlyChart` (`Overview.tsx:366-410`) and `GroupPanel` (`Overview.tsx:411-480`). A line chart needs a small hand-rolled SVG. **No charting dependency**: `web/package.json` has four runtime deps and the CSP serves scripts from `'self'` only; a self-hosted workspace that bundles its own fonts rather than fetching them (`main.tsx:37-39`) should not grow a chart library for four chart types. Explicitly refused: letting the model emit chart code or a Vega-style spec it invents. That is arbitrary JS in a service with no application authentication. ## 8. Persistence and UI ### 8.1 `queries.finance` Pinned tiles are user content, so they belong in the journal, not in `state/`. Add `queries.finance` to `registryFiles` (`internal/journal/codec.go:22`); the codec is json-tag driven, so the block grammar follows from the struct: ``` query { id: "qry_hobbies_2w" title: "Hobby spend, last 2 weeks" question: "How much did I spend on my hobbies the last 2 weeks?" sql: "SELECT sum(outflow) AS spent FROM v_spending WHERE category_path ILIKE '%Hobbies%' AND booking_date >= current_date - INTERVAL 14 DAY" chart: "kpi" position: 1 } ``` Diffable, hand-editable, backed up with everything else, survives a cache wipe. Writes go through the existing `App.mutate` revision-conflict path (`app.go:199-201`) and the `s.account`/`s.category` handler idiom (`server.go:239-293`). `domain.Dataset` gains `Queries []Query`, and `domain.Validate` gains id/title/SQL checks. Whether a stored query is gated at write time or only at run time is an open item — gating at write time means a view rename can make the journal unloadable, which is worse than a broken tile. ### 8.2 The page A new nav entry in `navigation` (`main.tsx:43-54`), between Wealth and Settings. Name open (§14). Layout: a tile grid. Each tile shows title, chart, and a footer with row count, elapsed ms, the snapshot revision, and a disclosure that reveals the SQL. Tiles re-run when `revision` changes, exactly like `Overview`'s effect (`Overview.tsx:29-64`). Composer: a question box and a SQL editor, side by side rather than staged, so the two paths are visibly the same pipeline. `Ask` fills the SQL box and shows `assumptions` above it; `Run` executes; `Pin` writes to `queries.finance`. Nothing auto-runs and nothing auto-pins. Optional, once tiles exist: render the first few pinned tiles on `Overview`. That is the "tile" framing in its most useful form, and it costs one component reuse. ## 9. API | Route | Body | Returns | | --- | --- | --- | | `POST /api/query` | `{sql, limit?}` | `{columns, rows, truncated, ms, revision}` | | `POST /api/ask` | `{question}` | `{sql, title, chart, x, y, assumptions, model, ms}` | | `POST /api/queries` | `{id?, title, question, sql, chart, position, delete?}` | `State` | Registered in `New` (`server.go:40-74`), decoded with `decode` (`server.go:192-204`), answered with `respond` (`server.go:205-216`). All three are `POST`, so they pick up the existing `Sec-Fetch-Site`, `Origin` and `application/json` guards at `server.go:130-153`. ## 10. Phasing 1. **Sandbox, gate, `/api/query`, result table, `queries.finance` tiles.** No model. Independently useful, and it is where all the engineering risk lives. 2. **Deterministic chart inference and the chart picker.** 3. **Views (§4) and a schema-browser panel.** Makes hand-written SQL pleasant *and* is the prerequisite for step 4 being any good. 4. **`internal/askql`, llama-server, `/api/ask`, assumptions, one repair.** Doing 4 before 3 is the main way this disappoints. ## 11. Work breakdown | File | Change | | --- | --- | | `internal/analytics/store.go` | add `SET lock_configuration = true` to `Open`; add views to `schema` and to the drop list at `:119`; create views after `postings` | | `internal/analytics/adhoc.go` (new) | `snapshot` lifecycle, `ATTACH … (READ_ONLY)`, lockdown, gate, wrap, run, result encoding incl. `duckdb.Decimal` | | `internal/analytics/query.go` | unchanged | | `internal/app/app.go` | invalidate the snapshot where `a.indexed = rev` (`:171`); `AdHoc` entry point that releases `a.mu` before running; `ask_base_url` / `ask_model` config keys and writer | | `internal/askql/` (new) | llama-server client, schema card, prompt, strict schema, one repair | | `internal/domain/model.go`, `domain.go` | `Query` struct, `Dataset.Queries`, validation | | `internal/journal/codec.go` | `queries.finance` in `registryFiles` | | `internal/server/server.go` | three routes and handlers | | `web/src/api.ts` | `QueryResult`, `Plan`, `Query` types | | `web/src/Queries.tsx` (new) | page, tile grid, composer, chart inference, table | | `web/src/main.tsx` | nav entry, route | | `web/src/styles.css` | tile grid, table, line-chart svg | | `nix/service.nix`, `README.md`, `OPERATIONS.txt` | llama-cpp unit, setup, operational notes | ## 12. Calibration Published execution accuracy for general small coder models on realistic multi-table schemas: **~39 % at 7B, 47 % at 14B, 50 % at 32B** on BIRD ([cross-family size × technique frontier](https://arxiv.org/pdf/2606.29733)). SQL-specialised models do markedly better — [Arctic-Text2SQL-R1](https://www.snowflake.com/en/blog/engineering/arctic-text2sql-r1-sql-generation-benchmark/) 14B reaches 64.9 % BIRD-dev / 86.8 % Spider-test; [XiYanSQL-QwenCoder](https://github.com/XGenerationLab/XiYanSQL-QwenCoder)-32B reaches 69 % BIRD-test, with 3B/7B/14B variants. Two consequences already built into this design: the schema must be small and denormalised (§4 turns a BIRD-hard join into a Spider-easy single-table query), and the SQL must always be visible and one click from editable. Plan for "wrong a third of the time, obviously wrong when it is". Latency, inferred not measured: a 7B Q4_K_M emitting ~200 constrained JSON tokens on a 7840U-class CPU lands around 15–25 s end to end. Tolerable for Ask, annoying for iteration — another reason the raw SQL path must stand alone. An SQL-specialised 3B is worth benchmarking against your own questions before committing to 7B. ## 13. Risks | Risk | Mitigation | | --- | --- | | Driver executes leading statements of a multi-statement string | read-only attachment (engine-enforced) **and** the parse-only gate; neither alone | | Ad-hoc query stalls the whole app | never hold `a.mu` during execution; separate pool; one query at a time | | Stale snapshot answers with pre-rebuild data | invalidate at `app.go:171`; every result carries `revision`; tiles re-run on change | | Spilling fills the data disk | disable `temp_directory` or cap `max_temp_directory_size` on the ad-hoc instance — **needs a probe**, `enable_external_access = false` does not cover spill | | Plausible but wrong SQL believed | `assumptions` shown, SQL shown, row count and revision in the footer, previous-period comparison for KPIs | | Pinned SQL breaks when views change | views are a versioned interface; broken tile shows the engine error, never a silent zero | | Registry names carry prompt injection | model output is only a read-only `SELECT` you approve before it runs | | `/api/query` is a new privilege class on a service with no auth | the sandbox *is* the mitigation; without §2 and §3 this endpoint is an arbitrary file read | | Model unavailable or slow | Ask degrades independently; query and tile surfaces have no AI dependency | ## 14. Open decisions 1. **Page name and placement** — `Ask`, `Lab`, `Query`? A new nav page, tiles embedded in `Overview`, or both? 2. **Loopback-only for the ask model** — accept as a hard invariant, or is an OpenRouter fallback wanted (which forces redaction back in and, per §6.1, breaks the feature)? 3. **Views as the documented query surface**, base tables explicitly unstable — accept? 4. **`v_recurring` grouping** — payee alone with an `amount_variation` filter (survives price rises, as drafted), or payee × rounded amount (splits on a price rise, but separates two different subscriptions to the same payee)? 5. **Where inference runs** and the RAM budget — decides 3B vs 7B vs 14B, and whether it shares the service host. 6. **Stored-query validation timing** — gate SQL at write time (a view rename can make the journal unloadable) or only at run time (a broken tile, loadable journal)? 7. Should `v_month` be a gap-free spine generated between `min` and `max`, and what should it do on an empty dataset? ## 15. Implementation notes Only the parts where the obvious implementation is wrong. ### 15.1 Attach before locking down ```go // ATTACH is itself a filesystem operation, so external access must stay enabled // until the snapshot is attached. The path is interpolated because ATTACH takes // no parameters; it is our own path, never user input. for _, stmt := range []string{ "ATTACH '" + path + "' AS fd (READ_ONLY)", "USE fd", "SET threads = 2", "SET memory_limit = '256MB'", "SET enable_external_access = false", "SET lock_configuration = true", } { … } ``` Reversing the last two lines, or setting `enable_external_access = false` before the `ATTACH`, fails with `Permission Error: Cannot access file …`. ### 15.2 The gate must bind, not format ```go // The submitted text is a value, not code: json_serialize_sql parses it without // executing it, and a bound parameter leaves no injection surface. The cast is // required — an untyped parameter yields // "json_serialize_sql first argument must be a VARCHAR". row := db.QueryRowContext(ctx, "SELECT json_serialize_sql(CAST(? AS VARCHAR))", sql) ``` The result scans as `map[string]any` through this driver, not as a string. `{"error": true, "error_type": …, "error_message": …}` on rejection; `{"statements": [{"node": {"type": "SELECT_NODE"}}]}` on acceptance. ### 15.3 Money must not become a float ```go // duckdb.Decimal.String() is exact; Float64() is not. domain.Money is a decimal // string for the same reason, and the dashboard already renders exact strings. case duckdb.Decimal: cell = v.String() ``` ### 15.4 Views join the drop list `Rebuild` drops tables by name at `store.go:119`. DuckDB refuses to drop a table a view depends on, so a view left behind breaks the next rebuild — the failure surfaces as `a.indexError` and takes the whole dashboard down, not just the tiles. Drop views first, in dependency order, or drop with `CASCADE`. ### 15.5 The local request body is not the OpenRouter one `classification.complete` sends `provider: {data_collection: "deny", zdr: true, require_parameters: true}` (`client.go:302`). Those keys are OpenRouter routing directives; a local server has no providers, and a strict OpenAI-compatible server may reject unknown fields. `askql` sends `model`, `messages`, `stream: false`, `max_tokens`, `response_format` and nothing else. ### 15.6 The config reader rejects unknown keys `app.go:112` is a `switch` over known keys with a hard error on anything else, and the writer at `app.go:376-380` rewrites the whole file. Adding `ask_base_url` / `ask_model` to one side only makes an existing `config.toml` unreadable after the first save.