Ekho-Labs / ton-app · pull request #5

M4 Forecast screen
cached by revision, narrated by proof

The measured forecasts finally reach a human: SBC quadrant cards, a hand-rolled SVG chart, the exception queue and a per-series drill-down. Behind them, two contracts — payloads served revision-keyed and immutable behind the ownership check, and narratives that fail closed if they quote a number the series JSON cannot back.

branchmilestone/m4-forecast-screen
basemilestone/m3-forecast
headf8ab35e
commits4
files22 changed
diff+2,208 −34
merged2026-08-14
M4 gate PASS · 4/4 in Chromium
00

What M4 actually built

M3 produces measured forecasts that nothing renders. M4 is the screen — and the two contracts that make the screen safe to trust: how a payload is served and cached, and what a generated sentence is allowed to say.

01 · contract

Revision-keyed, immutable payloads

Every payload read carries ?v={job_id} and comes back private, max-age=86400, immutable with the revision as its ETag. Nothing is ever invalidated because nothing is ever mutated.

ETag / 304edge cache post-auth only404, never 403
02 · honesty

Narratives that cannot invent numbers

One sentence per series, generated with the series JSON in the prompt and then checked: every numeric token must be traceable to that series' own numbers. A single stray figure fails the sentence closed to a deterministic template.

fail-closedKV 30 d per revisionre-checked in the gate
03 · screen

The forecast cycle, rendered

SBC quadrant cards, a hand-rolled SVG chart for the featured series, the demo's exception queue running off real payloads, the full series table, and a keyboard-accessible drill-down drawer.

4 quadrant cardsSVG chart 5 exception rulesdrawer + narrative
payloads
forecasts · series · dq
one revisioned route serves all three
cache
86,400 s
client private, edge public
kv
narrative cache
30 days, per user / dataset / revision
gate
m4.spec.ts
4/4 in real Chromium
benchmark
0.745 vs 1.043
median MASE after the honesty fix
01

Two request paths, both fail-closed

The payload path answers from the cheapest layer that is allowed to answer. The narrative path always answers, but never with a number it cannot prove. Hover any node to isolate its edges.

call chain fail-closed path response cache / state hover a node to isolate · scroll horizontally if clipped
tier 1Browser
screen
ForecastPage.tsx
Fetches all three payloads with the dataset's revision, then renders quadrant cards, the featured chart, the exception queue and the series table from them.
drill-down
SeriesDrawer.tsx
Keyboard-accessible dialog: champion, backtest MASE, the candidates it beat, the interval method — and one sentence fetched per series.
tier 2 · payloadCloudflare Worker
guard
requireAuth → ownedDataset
Session cookie, then a user-scoped row lookup. A dataset belonging to someone else is 404, never 403 — existence itself is not disclosed.
404
revalidation
ETag: "{revision}"
The revision is the ETag. A matching If-None-Match returns 304 with the same cache headers and no body — before any storage is touched.
edge cache
caches.default
Consulted only after the ownership check. The edge copy is stored public; the client is always told private.
origin
R2.get(payload)
Miss path only. The body is written back to the edge with waitUntil, so the fill never delays the response.
tier 2 · narrativeCloudflare Worker
route
GET …/narrative
Requires sku, customer and v. Loads that one series out of forecasts.json and answers no-store.
generate
llmNarrative()
One sentence, 40 words maximum, with the series JSON in the prompt and a hard rule that every number must come from it.
check
numbersAreTraceable
Every numeric token in the sentence must be in the allowed set built from that series' own JSON. One stray number fails the whole sentence.
fail closed
templateNarrative()
Deterministic sentence assembled from the same real numbers. The user still gets a summary; it just cannot be one the model invented.
The order of the first three nodes is the whole security argument

The edge cache is keyed by URL, and the URL carries no identity — only a dataset id and a revision. Checking the session and the ownership before touching caches.default is what keeps a shared edge object from ever becoming a cross-tenant read. It is a latency optimisation that had to be placed exactly once, correctly.

02

The payload caching contract

Forecast payloads are large, read repeatedly and never change. The contract makes all three facts explicit instead of hoping a default gets it right.

// worker/cache/payload.ts — serveRevisionedPayload GET /api/datasets/{id}/forecasts?v={job_id} 1. requireAuth session cookie or 401 2. ownedDataset(user) someone else's id -> 404, never 403 3. v matches /^[0-9a-f-]{1,64}$/i else 400 — bounded cache-key surface 4. If-None-Match == "{job_id}" -> 304, no body, no storage read 5. caches.default.match(url) -> edge hit 6. R2.get(datasets/{uid}/{id}/forecasts.json) executionCtx.waitUntil(cache.put(…, "public, max-age=86400")) ETag: "{job_id}" Cache-Control: private, max-age=86400, immutable The client is always told private so no shared proxy keeps a copy; the edge copy is stored public deliberately, because reaching it already required proving ownership.
No invalidation path exists. A re-forecast is a new job id, a new URL and a new entry; the old one is never requested again.
The ETag cannot disagree with the body — the revision is the entity tag, so there is no hash to recompute.
304 is answered before storage. A revalidation costs a D1 lookup and nothing else.
Deletion outranks the cache. M5's purge removes the dataset row, so the route 404s at step 2 before any cached object can be consulted.
03

The traceability rule

The plan allows a generated sentence per series. It also states the rule that makes that safe: every number in it must be traceable to a number the user can see. The Worker enforces it as code, not as a prompt instruction.

What counts as traceable

The allowed set, built per series before the sentence is judged.
every value
Each finite number anywhere in that series' JSON — raw, rounded, to 1 decimal, to 2 decimals, and ×100 for percentages.
on-screen derived
The horizon (6), the interval level (80), the candidate count and count − 1 (“beat N candidates”), and the rounded 6-month total.
identifiers
Digits embedded in the sku, customer, name or fallback reason — part numbers are not hallucinations.
periods
The year and month components of every forecast period, so “March 2027” is sayable.
the test
Every numeric token in the sentence must be in that set. One failure rejects the whole sentence — there is no partial acceptance.
illustration — how the check reads a sentence SKU-4471 · Northgate: intermittent demand; CrostonSBA won the backtest (MASE 0.68) and projects 1,240 units over the next 6 months, about 18% above last year. 18 appears nowhere in the series JSON → sentence rejected → template used
the fail-closed template, verbatim from the code {sku} · {customer}: {cls} demand; {champion} won the backtest (backtest MASE {mase}, beat {n} candidates) and projects {total} units over the next 6 months. source: "template" · numbers_checked: true — the user still gets a sentence
Cached per revision, not per session

The KV key is narr:{user}:{dataset}:{sha256(sku, customer, revision, prompt version)} with a 30-day TTL, so the same series never costs two completions and a re-forecast produces a new key rather than a stale sentence. Bumping PROMPT_VERSION retires every cached narrative at once. LLM_MODE=off or stub skips OpenAI entirely and answers from the template, which is what preview environments run.

04

What actually renders

The demo's forecast-cycle layout, ported to run entirely off the M3 payloads — no derived state on the server, no second source of truth.

The five exception rules

First match wins, per series. Rows are sorted by severity, then horizon volume, then key — then dealt round-robin across kinds so eight visible rows are never eight of the same thing.
kindfires whenseverity
quietSilent for more than 3× the typical order gap, and a 3-month moving average of zero.high
dyingLumpy history, but every forecast month is ≤ 0.5 units — reads dead, not quiet.medium
thinA named baseline stood in for a fitted model; the reason is printed verbatim.medium
seasonNext month forecast above 2.5× the typical order size.medium
fillInvoiced below 80% of ordered across the history — a fill-rate problem, not a forecasting one. Inert unless an invoiced column was mapped.low

The rest of the screen

Every element is a projection of one of the three payloads.
quadrant cardsFour SBC counts, asserted by the gate to equal both the payload summary and the per-series classes.
featured chartHand-rolled SVG: monthly history bars, then the forecast mean inside its 80% band. 136 lines, no charting dependency.
exception queueThe demo's queue, running off real payloads instead of generated demo data.
series tableEvery series with its pattern, champion, backtest score or fallback reason, and horizon total.
drill-down drawerKeyboard-accessible dialog: champion, backtest MASE, the candidate list it beat, the interval method, and the narrative. Escape closes it.
invoiced historyseries.json carries invoiced alongside ordered, which is what makes the fill-rate rule possible at all.
05

The gate, and its evidence

scripts/qa/m4.spec.ts is the plan's M4 gate: four Playwright assertions against a real local stack in real Chromium, each one comparing what a human sees against what the API returned.

assertion 1pass

Rendered quadrant counts equal the payload

Each of the four cards is read from the DOM and compared against both summary.quadrants and an independent count of the per-series cls values — so a rendering bug and a summary bug cannot cancel out.

assertion 2pass

The drawer names the champion and its score

drawer-champion must equal the series' champion exactly, drawer-mase must equal mase.toFixed(2), the dialog must contain “Backtest MASE”, and Escape must close it — the accessibility path, not just the visual one.

assertion 3pass

Every narrative number exists in the series JSON

The spec re-implements the traceability rule independently and applies it to the sentence that actually rendered. A regression in the Worker's checker therefore cannot hide behind the Worker's own checker.

assertion 4pass

A revalidated payload returns 304

First read is 200 and must carry an ETag; the same request with If-None-Match must come back 304. The caching contract is asserted over HTTP rather than assumed from the header string.

quoted from the PR body QA: scripts/qa/m4.spec.ts (the plan's M4 gate) PASSES 4/4 against a real local stack in real Chromium — quadrant counts equal the API payload, drawer shows champion+backtest, narrative numbers all exist in the series JSON, revalidation returns 304. Visual pass done (fixed a champion-name overflow found by screenshot review). Benchmark after the engine fix: median champion MASE 0.745 (band ≤0.9) vs seasonal-naive 1.043.
The assertion the automated gate could not make

Every spec test was green while a long champion name was overflowing its box in the drawer — toHaveText passes whether or not the glyphs fit. The defect was found by looking at a screenshot and fixed before merge. See finding R2.

06

What landed, by area

22 files, +2,208 −34. Roughly half is the screen itself; the two contracts are 180 lines together.

forecast screen4 files · 950
  • forecast/ForecastPage.tsx507
  • forecast/SeriesDrawer.tsx223
  • forecast/exceptions.ts184
  • forecast/SeriesChart.tsx136
worker contracts4 files · 279
  • llm/narrative.ts134
  • routes/datasets.ts45
  • llm/narrative.test.ts54
  • cache/payload.ts46
gate4 files · 294
  • scripts/qa/m4.spec.ts268
  • playwright.config.ts12
  • .github/workflows/ci.yml10
  • .github/workflows/deploy-dev.yml10
spa wiring4 files · 543
  • src/styles.css453
  • api/client.ts74
  • dashboard/DashboardPage.tsx12
  • src/App.tsx2
engine honesty fix3 files · 24
  • ton_app/jobs.py13
  • ton_app/forecast.py10
  • tests/test_forecast.py1

Short-history batteries drop season-12 models, and an unscored seasonal-naive reports null instead of 0.00. See finding R1.

misc3 files · 14
  • bun.lock9
  • scripts/qa/m3.sh4
  • package.json1
07

Findings & design notes

Two defects fixed inside the milestone — one a fabricated number, one only a screenshot could catch — plus the five decisions that hold the screen together. Click any row to expand.

resolved open notes
08

Where this leaves the plan

M4 is where the product becomes readable — and where honesty gets enforced twice

The measurements M3 produced now reach a human: a quadrant overview, a chart, a queue of things worth acting on, and a per-series drawer that names the winning model and the score it won with. Both of the milestone's contracts exist to stop the screen from saying something the data does not support — the cache never serves another tenant's payload, and the narrative never quotes a number it cannot prove. The milestone also went back into the engine to delete a flattering number it had inherited.

deliberately not here
  • No export — the screen holds everything, but nothing leaves it. M5.
  • No deletion UI and no retention sweeper — M5.
  • No production environment; still dev and per-PR previews only.
  • Narratives are generated on demand, never precomputed for a whole dataset.
what M5 inherits
  • Payloads already loaded in the browser — the export needs no new server route.
  • A deterministic exception queue whose row count the export can be asserted against.
  • KV keys under a per-dataset prefix, which the purge deletes wholesale.
  • A cache contract that a 404 at the route level already outranks.