Model/Provider Routing (P1-06, issue #29)

One authoritative ModelRouter + ProviderAdapter interface. Deterministic → tiny local → strong local → cloud escalation is driven by verification evidence, with the routing reason and estimated cost recorded on every run's durable trace.

Authority-map migration rule (model_routing): "One router selects the cheapest verified-capable model from measured profiles and records the reason."

The capability ladder

ModelRouter.CAPABILITY_FLOOR encodes the static tier ladder (from model_registry.ModelProfile):

Tier	Model	Provider	Tools	Context	Cost
T0	deterministic	local	no	—	0
T1	qwen2.5:3b	ollama	no	32k	0
T2	qwen3:14b	ollama	no	32k	0
T3	qwen3.6:27b	ollama	no	32k	0
T4	mimo-v2.5-pro	openai	yes	128k	paid (declared, not live)

Measured registry

MeasuredModelProfile rows (per (provider, model)) aggregate per-task-family success from inference_traces.jsonl and persist to <rune_home>/state/model_routing_profiles.json (append-safe JSON, same durability pattern as model profiles). update_profiles_from_traces() folds trace rows in; load_profiles()/save_profiles() own the I/O so rank() stays pure.

Provider removal never erases state: disconnecting a provider removes only the adapter instance. Profiles and outcomes live on disk keyed by (provider, model), independent of any adapter object.

Router semantics

ModelRouter.rank(task_family, need_tools, min_context) -> list[RoutingDecision] is advisory — it never mutates inputs and never executes model calls (orchestration/router.py idiom). The operator tier loop is the final decider; the router's first candidate is the run's start-tier decision.

Reason strings name the evidence:

no-measured-profiles tier<N> — pure capability-floor ordering (no measured profiles exist at all);
measured <ok>/<n> success task_family=<family> — verified-capable candidates with measured stats;
a candidate with measured stats but zero successes in the family is not verified-capable and is excluded;
when profiles exist, unmeasured candidates are not ranked (no evidence).

Escalation across runs is a start-tier decision: when a run fails (success=False or tool failure) on a local model, the next run for that task family may rank at a higher tier. The router never burns cloud tokens mid-run.

Trace visibility (schema v2)

InferenceTrace is schema v2 with additive optional fields — v1 rows and v1 readers keep working unchanged:

routing_reason — the start-tier decision reason (e.g. router: measured 12/13 success task_family=code);
routing_mode — local | cloud;
task_family — the measured-registry family key.

RoutingDecision.estimated_cost lands with the reason, so routing cost is always visible alongside the run's cost_usd/cost_basis.

Provider adaptation

ProviderAdapter is a chat-only, runtime-checkable Protocol (forge idiom, forge_client.py): name, available(), chat(model, messages, tools, options, think). OllamaAdapter wraps OllamaChatTransport (availability pings /api/tags, 2s timeout, 30s cache). Cloud adapters are declared, not live: CloudStubAdapter.chat raises ProviderUnavailable — no credentials enter this issue, and no provider ever silently fakes a response.

Composition registers exactly one routing authority per root: composition.compose_model_routing(root, state_dir) → root.get("model_routing").

Out of scope (unchanged)

Safety-override in-run tier semantics: untouched.
num_ctx/admission: ContextBroker + #28.
Tool execution authority: LocalToolRuntime (#32); the provider_registry._execute_tool_calls parallel path stays dead.
Archive escalation engines (elastic_cognition, tiered_executor, progressive_escalation): not integrated.

4.0 KiB Raw Permalink Blame History