The measured result
Measured locally—not extrapolated—using a quantized 4B model on Apple Metal. The sheet prefill is about 0.6 seconds and is paid once per state. A fresh one-question request is not 47 ms.
Same transformer. Different exit.
Normal LLM
state + question
↓
generate()
↓
token → token → token
↓
text → parser → maybe valid
System-One-style
state + question sheet
↓
shared KV prefill
↓
legal-label logits
↓
softmax → typed decision
How 400 ms became 47 ms
Independent full prompt
Every decision re-reads the state and question.
Cache the state
Reuse the shared-state KV prefix; run only each question suffix.
Cache the questions
Prefill state and all questions. Decisions become two-token pointers.
Batch single-position steps
One row per question. No attention mask construction on the hot path.
Speed did not waive the checks
What held
- 4/4 fast-sheet labelled spot-check
- 7/7 option-order rotations
- 5/5 deterministic distributions
- 4/5 decisions match the slow path
What did not
- The sheet path reads
retry_riskas 1.08 where the full prompt reads 3.00—low risk versus high risk. - A one-token pointer reached 27 ms—and collapsed accuracy to 1/4.
- Long-suffix batching was slower than serial.
- The full-prompt spot-check is 3/4, not 4/4.
- Raw softmax remains uncalibrated.
- The model still wrongly recommends an unchanged retry.
231 decisions. No victory lap.
Official JevBench v1.3 adapter, every redistributable public task, 231/231 valid responses. On those exact task IDs: Laya 58.4%, kev-4B 66.2%, Jev 1.13.0 86.6%. Prompted Gemma is a useful local baseline, not a trained decision model. This public-only run is not an official leaderboard rank.
Fresh-state API latency was p50 593 ms and p95 16.6 s; long-context tasks dominate the tail. That workload is deliberately separate from the 47 ms shared-state batch above.
Put it before the expensive model.
Local sidecar
agent event
↓
POST /v1/systemone
↓
route · policy · escalate
Remote fallback
uncertain or complex
↓
remote LLM
↓
reason · generate · act
The server speaks TypeSafe's request/response shape, so JevBench and an agent harness use the same API. Route only application-validated cases locally; raw softmax confidence is not a safe escalation policy.
Run the instrument
git clone https://github.com/zozo123/gemma-to-jev cd gemma-to-jev cargo run --release # conference demo cargo run --release -- bench # latency + proof suite cargo run --release -- serve # local TypeSafe-compatible API ./bench/run_jevbench.sh # official 231-task public run
The first run downloads about 2.3 GB of weights. Metal is selected automatically; pass --cpu to force CPU.