rune/docs/inference-trace.md

Inference Traces (P0-07, #23)

One durable record per operator run answering two questions at once:

• Why did this call cost this much? — estimated vs served prompt tokens, output tokens, wall latency, model time, cost basis.
• What exactly did the model see? — per-block token contributions, the tools allowlist, and which memory blocks were exposed.

Store

<rune_home>/state/inference_traces.jsonl — one JSON object per line (O_APPEND line-atomic; model calls run on exclusive workers). Rotates at 10 MB × 3 backups (inference_traces.jsonl.1 … .3), mirroring the _log_token_usage size-cap approach. The file is runtime state (state/ is gitignored, P0-05 M6) — never committed. Consumers: #25, #28, #29. Read back with rune.inference_trace.read_traces() or python3 -m rune.inference_trace [n] (tail n records).

Record schema (version 4)

Field	Source	Meaning
schema_version	constant (4)	Record shape version for consumers
provider	OllamaChatTransport.provider	"ollama" (class constant)
model	operator	Model id the run executed on
task_id / channel	operator (TaskState #18)	Joins the trace to TaskState
resolved_context_window	broker trace window	Discovered trained window
input_budget	broker trace budget	Input budget admission used
model_calls	transport usage_calls	chat() invocations this run
prompt_tokens_estimated	broker trace admitted_total	Broker's measured estimate
prompt_tokens_served	transport usage_prompt_tokens	Ollama prompt_eval_count sum (0 = unknown)
output_tokens	transport usage_output_tokens	Ollama eval_count sum
latency_ms	operator wall clock	Around the whole runtime.run attempt (incl. tier escalation)
model_time_ms	transport usage_total_duration_ns / 1e6	Ollama total_duration sum
per_block_tokens	broker trace blocks	{name: tokens} for admitted blocks
dropped_blocks	broker trace blocks	[{name, tokens}] for blocks admission dropped
tools_exposed	runtime.tool_names	The runtime's tool allowlist
memories_exposed	broker blocks ∩ MEMORY_BLOCK_NAMES	Memory blocks with tokens > 0
memory_tokens	broker trace memory_tokens	Summed memory-block tokens
degraded	operator (P0-06)	Subsystems that degraded this run
cache	constant None	Ollama exposes no KV-cache stats (cloud comparison field)
cost_usd	price_for(model)	None until the price map gains entries
cost_basis	derived	"local-unpriced" or "price-map"
success / error	final OperatorResponse	Run outcome; error truncated to 300 chars
routing_reason / routing_mode / task_family	router (P1-06, #29, v2)	Start-tier decision evidence; empty on pre-router rows
selected_tools / capability_reason / find_capability_calls	broker selection (P1-01, #24, v3)	Capability-authority evidence
capsule_messages	capsule capture (P1-02, #25, v4)	The EXACT message list runtime.run received — equality with the logged capsule is the acceptance invariant
capsule_content_hashes	operator (P1-02, #25, v4)	{block_name: sha256(content)} — integrity without full JSONL bloat
capsule_token_count	capsule re-measure (P1-02, #25, v4)	Post-safety-override re-measured total

Schema history: v1 (P0-07 #23), v2 additive routing fields (P1-06 #29), v3 additive capability fields (P1-01 #24), v4 additive capsule-fidelity fields (P1-02 #25). All bumps are additive with defaults — older rows without the new keys still parse (read_traces consumers must tolerate missing keys).

Authority and single-source rule

The trace builds from the broker trace (#19) + transport counters — it never re-measures. There is no parallel authority path: the broker stays the single source for window/budget/blocks; the transport stays the single source for served/output/duration. Calibration (#19 Task 2) remains log-only; the trace persists what calibration only logged, and nothing feeds back into admission.

Failure semantics (P0-06)

Recording is degrade-not-crash (tier=2 → storage.degraded): a trace failure is observable via rune.subsystem_health.record_failure but never fails the operator turn.

Cloud/local comparison

Local records carry provider: "ollama", cache: null, cost_usd: null, cost_basis: "local-unpriced". Cloud providers (#29) fill the price map (rune.inference_trace.price_for) and may report cache stats — the record shape is identical, so #25/#28/#29 compare providers on one schema.

Wiring

• Transport accumulators: OllamaChatTransport.chat() sums envelope usage (prompt_eval_count, eval_count, total_duration) across every chat; last_prompt_eval_count behavior is unchanged (calibration pin).
• Operator: RuneOperator._execute_on_model stamps run_started before the tier-escalation loop and records exactly one trace per run on the final response.
• Canary: src/rune/canary_p0_07.py (registered in .gitea/workflows/ci.yml's canary loop) re-proves the contract with no live model. Tests: tests/test_p0_07_inference_trace.py.