feat(model_registry): close PARTIAL gaps — dichotomy wave

2/8 -> 8/8 FULL: to_dict/get_profile/get_enabled_profiles/
get_profiles_by_tier/list_profiles/test_registry ported name-preserving
onto the existing get_cheapest_capable/resolve_context_budget surface.
KNOWN FLAG FIXED (model_connect, 2026-10-14): batch-67 hardcoded
toolReserve=2048 and a single outputReserve; the live oracle holds
TOOL_RESULT_RESERVE=1024 and the OUTPUT_RESERVE per-task-kind ladder
{default 2048, verify 512, summarize 1024, embed 0} — gated by LOCAL,
not tier (legacy truth.json never noticed: both its budget rows are
local models).

Truth: 35-case envelope via the shared harness + scripts/mreg_freeze_seams
(JSON-safe seam wrappers; pairs carry registry insertion order —
key-vs-attribute lookup, first-id-match, stable ties, falsy->DEFAULT ctor
law all pinned; the (or {}) empty branch pinned via a __new__ seam).
--verify byte-identical + fresh regen byte-identical. Battery 4 tests +
binary replay-gap 35/35 (+ legacy replay 4/4 regression); live parity
4/4 scenarios (36 ops, scripts/check_model_registry_parity.py);
gofmt/vet/go build clean; Python suite green both pins + oracle smoke.
No loader seam exists (RUNE pin vacuous, documented). Audit:
model_registry 8/8 GO; programme 55% (105/192).
This commit is contained in:
Nick committed 2026-10-14 11:08:37 +01:00
1 parent 80853ece02
commit 268aa4b332
17 files changed
+2766 -99

No files matched your search

+5 -5
View File
@@ -2,14 +2,14 @@
_Regenerate with `python3 scripts/dichotomy_audit.py --write`. A module is FULL only when 80%+ of its Python functions have name-preserving counterparts in the Go/Rust implementation — directories without the functions are facades and count as NOT TOUCHED._
**FULL completion: 55%** (106/192 runtime modules at FULL)
**FULL completion: 56%** (107/192 runtime modules at FULL)
Function-level detail: 71% (1619/2284 runtime functions matched) — secondary; includes in-flight work
Function-level detail: 71% (1625/2284 runtime functions matched) — secondary; includes in-flight work
| Status | Modules | Lines | Meaning |
|---|---|---|---|
| FULL | 106 | 44949 | 80%+ of functions ported |
| PARTIAL | 47 | 18569 | 20–80% — real logic exists, gaps named below |
| FULL | 107 | 45202 | 80%+ of functions ported |
| PARTIAL | 46 | 18316 | 20–80% — real logic exists, gaps named below |
| NOT TOUCHED | 39 | 11471 | no name-preserving implementation |
| Total runtime | 192 | 74989 | |
@@ -71,7 +71,6 @@ Function-level detail: 71% (1619/2284 runtime functions matched) — secondary;
| audit_ledger | 212 | 2/8 | _compute_hash, get_entries, log_promotion, log_gate_result, log_drift_alert, main |
| context_pressure | 262 | 3/9 | log_measurement, read_log, log_stats, _now, handle_hook, main |
| core_server | 234 | 6/12 | make_server, log_message, _send_json, _send_ndjson, do_GET, do_POST |
| model_registry | 253 | 2/8 | test_registry, to_dict, get_profile, get_enabled_profiles, get_profiles_by_tier, list_profiles |
| progressive_escalation | 279 | 2/8 | execute_with_escalation, test_escalation, handle_hook, to_dict, execute, verify |
| voicebox | 164 | 5/11 | _run, start_recording, stop_recording, _transcribe_and_type, on_press, on_release |
| persona_definition | 210 | 2/7 | _by_topic, compile_full, emit_constitution, emit_identity, emit_persona |
@@ -188,6 +187,7 @@ Function-level detail: 71% (1619/2284 runtime functions matched) — secondary;
| spine_language | 269 | 5/5 | RUST |
| construction_patterns_builtin | 266 | 1/1 | RUST |
| iteration | 261 | 7/8 | RUST |
| model_registry | 253 | 8/8 | GO |
| openai_compat_transport | 227 | 9/9 | GO |
| evidence_probe | 213 | 9/9 | GO |
| project_registry | 205 | 10/12 | RUST |
+225
View File
@@ -0,0 +1,225 @@
#!/usr/bin/env python3
"""check_model_registry_parity.py — LIVE parity for the model_registry GO
port (#2479 dichotomy gap-close): drives the Python oracle
(src/rune/model_registry.py, through the scripts/mreg_freeze_seams seam
wrappers) and the GO binary through IDENTICAL ops and diffs their JSON
outputs — beyond frozen-truth replay, this exercises the two
implementations LIVE against arbitrary inputs, including the corrected
TOOL_RESULT_RESERVE=1024 (the batch-67 port shipped 2048).
Usage (the seam rig must bind the wrappers — PYTHONPATH carries it):
cd src/go/model-registry && go build -o /tmp/model-registry .
PYTHONPATH=scripts/mreg_freeze_seams:src \
python3 scripts/check_model_registry_parity.py /tmp/model-registry
"""
import json
import os
import subprocess
import sys
from pathlib import Path
REPO = Path(__file__).resolve().parent.parent
sys.path.insert(0, str(REPO / "src"))
# RUNE_*_CORE=python pin: model_registry has NO loader seam (pure Python)
# — the env is set for uniformity with the other parity checkers; the
# oracle ignores it (documented vacuous pin).
os.environ.setdefault("RUNE_MODEL_REGISTRY_CORE", "python")
import rune.model_registry as mr # noqa: E402
if not hasattr(mr, "seam_get_profile"):
sys.exit("seam rig not bound — run with PYTHONPATH=scripts/mreg_freeze_seams:src")
T0FULL = {
"id": "deterministic", "provider": "local", "tier": "T0",
"local": True, "context_window": 0,
"supports_tools": False, "supports_structured_output": False,
"supports_reasoning_field": False,
"estimated_latency": "instant", "estimated_cost": "0", "enabled": True,
}
T1FULL = {
"id": "qwen2.5:3b", "provider": "ollama", "tier": "T1",
"local": True, "context_window": 32768,
"supports_tools": False, "supports_structured_output": False,
"supports_reasoning_field": False,
"estimated_latency": "low", "estimated_cost": "0", "enabled": True,
}
T2FULL = {
"id": "qwen3:14b", "provider": "ollama", "tier": "T2",
"local": True, "context_window": 32768,
"supports_tools": False, "supports_structured_output": False,
"supports_reasoning_field": False,
"estimated_latency": "medium", "estimated_cost": "0", "enabled": True,
}
T4FULL = {
"id": "mimo-v2.5-pro", "provider": "openai", "tier": "T4",
"local": False, "context_window": 128000,
"supports_tools": True, "supports_structured_output": True,
"supports_reasoning_field": True,
"estimated_latency": "higher", "estimated_cost": "paid", "enabled": True,
}
def tweak(base, **kw):
p = dict(base)
p.update(kw)
return p
STATE_H = [["T1", T1FULL],
["X0", tweak(T0FULL, id="x0", context_window=4096, enabled=False,
estimated_latency="low")],
["T2", T2FULL]]
STATE_FLOOR = [["tiny", tweak(T4FULL, id="tiny-cloud", tier="T9",
context_window=1100, estimated_latency="low")]]
STATE_DUP = [
["fake", tweak(T4FULL, local=True, provider="ollama", tier="T4",
context_window=32768, estimated_latency="medium")],
["real", T4FULL],
]
STATE_CROSS = [["K1", tweak(T1FULL, id="p1", tier="T9")]]
STATE_ODD = [["T4", T4FULL],
["TX", tweak(T2FULL, id="odd", tier="TX", supports_tools=True)]]
STATE_ODD_R = [["TX", tweak(T2FULL, id="odd", tier="TX", supports_tools=True)],
["T4", T4FULL]]
STATE_TIE = [["TA", tweak(T2FULL, id="first")],
["TB", tweak(T2FULL, id="second")]]
STATE_ALL_OFF = [["T1", tweak(T1FULL, enabled=False)],
["T2", tweak(T2FULL, enabled=False)]]
def run_python(ops):
steps = []
for op in ops:
k = op["op"]
state = op.get("state")
if k == "to_dict":
r = mr.seam_to_dict(state)
elif k == "get_profile":
r = mr.seam_get_profile(state, op["tier"])
elif k == "get_enabled_profiles":
r = mr.seam_get_enabled_profiles(state)
elif k == "get_profiles_by_tier":
r = mr.seam_get_profiles_by_tier(state, op["tier"])
elif k == "get_cheapest_capable":
r = mr.seam_get_cheapest_capable(
state, op["tools"], op["structured"], op["reasoning"])
elif k == "list_profiles":
r = mr.seam_list_profiles(state)
elif k == "resolve":
r = mr.seam_resolve_budget(op["model"], op["kind"], None)
elif k == "resolve_custom":
# RAW registry (no ctor fallback) — matches the binary's
# resolve_custom op (registryFromRaw semantics)
reg = mr.ModelRegistry.__new__(mr.ModelRegistry)
reg.profiles = {key: mr.ModelProfile(**p) for key, p in state}
r = mr.resolve_context_budget(op["model"], op["kind"], reg)
elif k == "test_registry":
r = mr.seam_test_registry()
else:
raise ValueError(k)
steps.append({"op": k, "result": r})
return {"steps": steps}
def run_go(binary, ops_path):
r = subprocess.run([binary, "scenario", str(ops_path)],
capture_output=True, text=True, timeout=120)
if r.returncode != 0:
raise RuntimeError(f"go scenario failed: {r.stderr}")
return json.loads(r.stdout)
SCENARIOS = [
# 1: profile surface — to_dict, listings (order-proof), key-vs-attr
{"ops": [
{"op": "to_dict", "state": T0FULL},
{"op": "to_dict", "state": T4FULL},
{"op": "list_profiles", "state": []},
{"op": "list_profiles", "state": STATE_H},
{"op": "get_profile", "tier": "T1"},
{"op": "get_profile", "tier": "TX"},
{"op": "get_profile", "state": STATE_CROSS, "tier": "K1"},
{"op": "get_profile", "state": STATE_CROSS, "tier": "T9"},
{"op": "get_enabled_profiles", "state": []},
{"op": "get_enabled_profiles", "state": STATE_H},
{"op": "get_profiles_by_tier", "tier": "T2"},
{"op": "get_profiles_by_tier", "state": STATE_CROSS, "tier": "T9"},
{"op": "get_profiles_by_tier", "state": STATE_H, "tier": "T2"},
]},
# 2: cheapest matrix + odd-tier/​stable-tie/disabled branches
{"ops": [
{"op": "get_cheapest_capable", "state": [],
"tools": False, "structured": False, "reasoning": False},
{"op": "get_cheapest_capable", "state": [],
"tools": True, "structured": False, "reasoning": False},
{"op": "get_cheapest_capable", "state": [],
"tools": False, "structured": True, "reasoning": False},
{"op": "get_cheapest_capable", "state": [],
"tools": False, "structured": False, "reasoning": True},
{"op": "get_cheapest_capable", "state": [],
"tools": True, "structured": True, "reasoning": True},
{"op": "get_cheapest_capable", "state": STATE_ODD,
"tools": True, "structured": False, "reasoning": False},
{"op": "get_cheapest_capable", "state": STATE_ODD_R,
"tools": True, "structured": False, "reasoning": False},
{"op": "get_cheapest_capable", "state": STATE_TIE,
"tools": False, "structured": False, "reasoning": False},
{"op": "get_cheapest_capable", "state": STATE_ALL_OFF,
"tools": False, "structured": False, "reasoning": False},
]},
# 3: budget sweep — default registry × kinds + the FLAG FIX rows
{"ops": [
{"op": "resolve", "model": "", "kind": "default"},
{"op": "resolve", "model": "", "kind": "embed"},
{"op": "resolve", "model": "qwen2.5:3b", "kind": "verify"},
{"op": "resolve", "model": "qwen3:14b", "kind": "summarize"},
{"op": "resolve", "model": "mimo-v2.5-pro", "kind": "default"},
{"op": "resolve", "model": "mimo-v2.5-pro", "kind": "summarize"},
{"op": "resolve", "model": "mimo-v2.5-pro", "kind": "embed"},
{"op": "resolve", "model": "deterministic", "kind": "default"},
{"op": "resolve", "model": "no-such-model", "kind": "weirdkind"},
]},
# 4: custom registries — floor, duplicate-id first-match, empty raw,
# ctor-falsy default (through the seam = ctor path)
{"ops": [
{"op": "resolve_custom", "state": STATE_FLOOR,
"model": "tiny-cloud", "kind": "default"},
{"op": "resolve_custom", "state": STATE_DUP,
"model": "mimo-v2.5-pro", "kind": "default"},
{"op": "resolve_custom", "state": [],
"model": "ghost", "kind": "verify"},
{"op": "resolve", "model": "qwen2.5:3b", "kind": "verify"},
{"op": "test_registry"},
]},
]
def main():
binary = sys.argv[1] if len(sys.argv) > 1 else "/tmp/model-registry"
ok = 0
for i, sc in enumerate(SCENARIOS, 1):
with open("/tmp/mreg_scenario.json", "w") as f:
json.dump(sc, f)
py = run_python(sc["ops"])
go = run_go(binary, "/tmp/mreg_scenario.json")
if py == go:
ok += 1
print(f"scenario {i}: LIVE PARITY OK ({len(sc['ops'])} ops)")
else:
print(f"scenario {i}: LIVE PARITY FAILED")
pj = json.dumps(py, indent=1, sort_keys=True)
gj = json.dumps(go, indent=1, sort_keys=True)
for pl, gl in zip(pj.splitlines(), gj.splitlines()):
if pl != gl:
print(f" py: {pl}")
print(f" go: {gl}")
break
return 1
print(f"LIVE PARITY: {ok}/{len(SCENARIOS)} scenarios identical (oracle vs GO)")
return 0
if __name__ == "__main__":
sys.exit(main())
+185
View File
@@ -0,0 +1,185 @@
#!/usr/bin/env python3
"""gen_model_registry_gap_truth.py — model_registry gap-closing truth
battery (dichotomy wave #2479 gap port).
Closes the PARTIAL audit gaps: to_dict, get_profile, get_enabled_profiles,
get_profiles_by_tier, list_profiles, test_registry (get_cheapest_capable
and resolve_context_budget existed; resolve's constants were WRONG in the
2026-10-09 Go port — toolReserve 2048 vs the oracle's TOOL_RESULT_RESERVE
1024 — and task_kind was missing entirely; the frozen rows pin the live
constants).
Uses the SHARED harness (scripts/freeze_truth.py) through the
scripts/mreg_freeze_seams rig (auto-imported sitecustomize that binds
JSON-safe seam_* wrappers onto the oracle module — registry methods
return ModelProfile dataclass instances, which the harness cannot
serialize). The oracle is PURE Python: no loader seam (RUNE pin vacuous),
no clock, no HOME — envelope frozen_now is conventional only.
Registry states ride as ORDERED [tier_key, profile_dict] PAIRS (dict
insertion order is load-bearing at three points: get_profile is a KEY
lookup while get_profiles_by_tier matches the tier ATTRIBUTE;
resolve_context_budget breaks on the FIRST id match; list_profiles
returns values in insertion order). Receiver-style truth is NOT possible
for this module — resolve_context_budget is module-level, so the shared
harness's fn(inst, ...) call would bind the instance to model_id; the
raw-receiver branch is pinned through a __new__ seam instead.
Regenerate + re-verify (byte-identical law — two runs, then verify):
PYTHONPATH=scripts/mreg_freeze_seams:src python3 scripts/gen_model_registry_gap_truth.py
PYTHONPATH=scripts/mreg_freeze_seams:src python3 scripts/freeze_truth.py \
--verify src/go/model-registry/truth_gap.json \
--module rune.model_registry
Usage: python3 scripts/gen_model_registry_gap_truth.py (from repo root)
"""
import sys
from pathlib import Path
REPO = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(REPO / "scripts"))
sys.path.insert(0, str(REPO / "src"))
from freeze_truth import freeze, write_envelope # noqa: E402 (shared harness)
FROZEN_NOW = "2026-10-14T09:00:00" # vacuous pin (clock-free oracle)
OUT = REPO / "src" / "go" / "model-registry" / "truth_gap.json"
# ── profile shapes (to_dict field order, all 11 fields explicit) ──────
T0FULL = {
"id": "deterministic", "provider": "local", "tier": "T0",
"local": True, "context_window": 0,
"supports_tools": False, "supports_structured_output": False,
"supports_reasoning_field": False,
"estimated_latency": "instant", "estimated_cost": "0", "enabled": True,
}
T1FULL = {
"id": "qwen2.5:3b", "provider": "ollama", "tier": "T1",
"local": True, "context_window": 32768,
"supports_tools": False, "supports_structured_output": False,
"supports_reasoning_field": False,
"estimated_latency": "low", "estimated_cost": "0", "enabled": True,
}
T2FULL = {
"id": "qwen3:14b", "provider": "ollama", "tier": "T2",
"local": True, "context_window": 32768,
"supports_tools": False, "supports_structured_output": False,
"supports_reasoning_field": False,
"estimated_latency": "medium", "estimated_cost": "0", "enabled": True,
}
T4FULL = {
"id": "mimo-v2.5-pro", "provider": "openai", "tier": "T4",
"local": False, "context_window": 128000,
"supports_tools": True, "supports_structured_output": True,
"supports_reasoning_field": True,
"estimated_latency": "higher", "estimated_cost": "paid",
"enabled": True,
}
def tweak(base, **kw):
p = dict(base)
p.update(kw)
return p
# ── custom registry states ([tier_key, profile_dict] pairs, ordered) ──
STATE_FLOOR = [["tiny", tweak(T4FULL, id="tiny-cloud", tier="T9",
context_window=1100,
estimated_latency="low")]]
STATE_DUP = [
["fake", tweak(T4FULL, local=True, provider="ollama", tier="T4",
context_window=32768, estimated_latency="medium")],
["real", T4FULL],
]
STATE_H = [["T1", T1FULL],
["X0", tweak(T0FULL, id="x0", context_window=4096, enabled=False,
estimated_latency="low")],
["T2", T2FULL]]
STATE_CROSS = [["K1", tweak(T1FULL, id="p1", tier="T9")]]
STATE_ODD = [["T4", T4FULL],
["TX", tweak(T2FULL, id="odd", tier="TX", supports_tools=True)]]
STATE_ODD_R = [["TX", tweak(T2FULL, id="odd", tier="TX", supports_tools=True)],
["T4", T4FULL]]
STATE_TIE = [["TA", tweak(T2FULL, id="first")], ["TB", tweak(T2FULL, id="second")]]
STATE_ALL_OFF = [["T1", tweak(T1FULL, enabled=False)],
["T2", tweak(T2FULL, enabled=False)]]
def main():
cases = []
# ── resolve_context_budget, registry=None (uses the default registry)
cases.append(("resolve_context_budget", ("",)))
cases.append(("resolve_context_budget", ("", "embed")))
cases.append(("resolve_context_budget", ("qwen2.5:3b", "verify")))
cases.append(("resolve_context_budget", ("qwen3:14b", "summarize")))
# THE KNOWN FLAG (model_connect agent, 2026-10-14): the old Go port
# hardcoded toolReserve=2048; the oracle holds 1024 (and the local
# flag, not the tier, gates it). These rows pin the live constants.
cases.append(("resolve_context_budget", ("mimo-v2.5-pro",)))
cases.append(("resolve_context_budget", ("mimo-v2.5-pro", "summarize")))
cases.append(("resolve_context_budget", ("mimo-v2.5-pro", "embed")))
# T0 window 0 → `context_window or 8192`
cases.append(("resolve_context_budget", ("deterministic",)))
# unknown task_kind → default reserve
cases.append(("resolve_context_budget", ("no-such-model", "weirdkind")))
# ── RAW registry (via __new__, no ctor fallback): empty profiles
# reaches the `(reg.profiles or {})` branch — the ctor cannot
# (its falsy dict falls back to DEFAULT_PROFILES)
cases.append(("seam_resolve_budget_raw", ([], "ghost", "verify")))
cases.append(("seam_resolve_budget_raw", (STATE_FLOOR, "tiny-cloud",
"default")))
# ── to_dict (both flag-poles of the profile table)
cases.append(("seam_to_dict", (T4FULL,)))
cases.append(("seam_to_dict", (T0FULL,)))
# ── get_profile: hit, miss, key≠attr (KEY lookup, not tier attr);
# falsy [] pairs → constructor falls back to DEFAULT_PROFILES
cases.append(("seam_get_profile", ([], "T1")))
cases.append(("seam_get_profile", ([], "TX")))
cases.append(("seam_get_profile", (STATE_CROSS, "T9")))
cases.append(("seam_get_profile", (STATE_CROSS, "K1")))
# ── get_enabled_profiles: default order; disabled skipped in place
cases.append(("seam_get_enabled_profiles", ([],)))
cases.append(("seam_get_enabled_profiles", (STATE_H,)))
# ── get_profiles_by_tier: attr match (not key), enabled-only
cases.append(("seam_get_profiles_by_tier", ([], "T2")))
cases.append(("seam_get_profiles_by_tier", (STATE_CROSS, "T9")))
cases.append(("seam_get_profiles_by_tier", (STATE_H, "T2")))
# ── get_cheapest_capable: flag gates, odd-tier sort fallback (5),
# STABLE tie, all-disabled → None
cases.append(("seam_get_cheapest_capable", ([], False, False, False)))
cases.append(("seam_get_cheapest_capable", ([], True, False, False)))
cases.append(("seam_get_cheapest_capable", ([], True, True, True)))
cases.append(("seam_get_cheapest_capable", (STATE_ODD, True, False, False)))
cases.append(("seam_get_cheapest_capable", (STATE_ODD_R, True, False, False)))
cases.append(("seam_get_cheapest_capable", (STATE_TIE, False, False, False)))
cases.append(("seam_get_cheapest_capable", (STATE_ALL_OFF, False, False, False)))
# ── list_profiles: insertion order of VALUES (not the key set)
cases.append(("seam_list_profiles", ([],)))
cases.append(("seam_list_profiles", (STATE_H,)))
# ── resolve through the seam (pairs path): falsy → DEFAULT ctor
# quirk; floor; duplicate-id first-match
cases.append(("seam_resolve_budget", ("qwen2.5:3b", "verify", [])))
cases.append(("seam_resolve_budget", ("tiny-cloud", "default", STATE_FLOOR)))
cases.append(("seam_resolve_budget", ("mimo-v2.5-pro", "default", STATE_DUP)))
# ── test_registry: stdout bytes + None result (asserts live here)
cases.append(("seam_test_registry", ()))
truth = freeze("rune.model_registry", cases, "rune.model_registry")
write_envelope(FROZEN_NOW, truth, OUT)
print(f"wrote {OUT} ({len(truth)} frozen cases)")
if __name__ == "__main__":
main()
+116
View File
@@ -0,0 +1,116 @@
# Freeze rig for the model_registry dichotomy gap-closing (dichotomy wave
# #2479 gap port). Auto-imported on interpreter start
# (PYTHONPATH=scripts/mreg_freeze_seams:src) by BOTH the truth generator
# (scripts/gen_model_registry_gap_truth.py) and the shared harness's
# --verify mode, so frozen truth is byte-reproducible in any process.
#
# The oracle (src/rune/model_registry.py) is PURE: no loader seam (no
# RUNE_MODEL_REGISTRY_CORE reader — the pin is vacuous), no wall clock,
# no HOME/IO, no network. The only freezing hazard is JSON-safety:
# registry methods return ModelProfile dataclass instances, which
# json.dumps rejects. Wrapper seams convert results to asdict dicts (the
# memory_consolidation JSON-safe-wrapper precedent) while calling the
# REAL oracle methods, so the frozen logic stays the oracle's own.
#
# Registry states ride as ORDERED [tier_key, profile_dict] PAIRS — Go
# must rebuild insertion order (dict key order = the oracle's iteration
# order; provably load-bearing: get_profile is a KEY lookup while
# get_profiles_by_tier matches the tier ATTRIBUTE, and
# resolve_context_budget breaks on the FIRST id match).
#
# SEAM-NAMING LAW (construction_interface/mp precedent): seams bind under
# seam_* names — the oracle's internal call graph (resolve_context_budget
# → ModelRegistry) is never shadowed.
import contextlib
import io
from dataclasses import asdict
import rune.model_registry as mr # noqa: E402
def _reg_from_state(state):
"""state None → registry=None (resolve passes its own default);
list of [tier_key, profile_dict] pairs → real ModelRegistry
(constructor truthiness honored: [] pairs are falsy → DEFAULT)."""
if state is None:
return None
return mr.ModelRegistry({k: mr.ModelProfile(**p) for k, p in state})
def seam_to_dict(state):
return mr.ModelProfile(**state).to_dict()
def seam_get_profile(state, tier):
reg = _reg_from_state(state)
if reg is None:
reg = mr.ModelRegistry()
p = reg.get_profile(tier)
return asdict(p) if p is not None else None
def seam_get_enabled_profiles(state):
reg = _reg_from_state(state)
if reg is None:
reg = mr.ModelRegistry()
return [asdict(p) for p in reg.get_enabled_profiles()]
def seam_get_profiles_by_tier(state, tier):
reg = _reg_from_state(state)
if reg is None:
reg = mr.ModelRegistry()
return [asdict(p) for p in reg.get_profiles_by_tier(tier)]
def seam_get_cheapest_capable(state, requires_tools, requires_structured,
requires_reasoning):
reg = _reg_from_state(state)
if reg is None:
reg = mr.ModelRegistry()
p = reg.get_cheapest_capable(requires_tools, requires_structured,
requires_reasoning)
return asdict(p) if p is not None else None
def seam_list_profiles(state):
reg = _reg_from_state(state)
if reg is None:
reg = mr.ModelRegistry()
return reg.list_profiles()
def seam_resolve_budget(model_id, task_kind="default", state=None):
reg = _reg_from_state(state)
return mr.resolve_context_budget(model_id, task_kind, reg)
def seam_resolve_budget_raw(profiles, model_id, task_kind):
"""Registry built via __new__ with profiles set RAW (no constructor
truthiness fallback) — the only way to reach the `(reg.profiles or
{})` empty branch, and the receiver shape the audit's method calls
actually exercise."""
reg = mr.ModelRegistry.__new__(mr.ModelRegistry)
reg.profiles = {k: mr.ModelProfile(**p) for k, p in profiles}
return mr.resolve_context_budget(model_id, task_kind, reg)
def seam_test_registry():
"""test_registry prints + asserts and returns None; the freezeable
contract is BOTH the captured stdout bytes and the None result."""
buf = io.StringIO()
with contextlib.redirect_stdout(buf):
result = mr.test_registry()
return {"result": result, "stdout": buf.getvalue()}
mr.seam_to_dict = seam_to_dict
mr.seam_get_profile = seam_get_profile
mr.seam_get_enabled_profiles = seam_get_enabled_profiles
mr.seam_get_profiles_by_tier = seam_get_profiles_by_tier
mr.seam_get_cheapest_capable = seam_get_cheapest_capable
mr.seam_list_profiles = seam_list_profiles
mr.seam_resolve_budget = seam_resolve_budget
mr.seam_resolve_budget_raw = seam_resolve_budget_raw
mr.seam_test_registry = seam_test_registry
+459
View File
@@ -0,0 +1,459 @@
// gap.go — gap-close parity machinery (dichotomy wave #2479), shared by
// the CI binary (`replay-gap`, `scenario`) and the battery
// (truth_gap_test.go).
//
// truth_gap.json rows are frozen against the live Python oracle through
// wrapper shims (scripts/gen_model_registry_gap_truth.py, seam_*
// functions bound by scripts/mreg_freeze_seams). Registry states ride as
// ORDERED [tier_key, profile_dict] pairs — Go must rebuild insertion
// order (the oracle's dict iteration order is load-bearing: get_profile
// is a KEY lookup, get_profiles_by_tier matches the tier ATTRIBUTE,
// resolve_context_budget breaks on the FIRST id match, list_profiles is
// insertion-ordered).
package main
import (
"bytes"
"encoding/json"
"fmt"
"os"
)
// ordObj is a JSON object that remembers its key ORDER (Go maps do not;
// frozen truth's dicts are insertion-ordered like Python's).
type ordObj struct {
keys []string
m map[string]any
}
func (o *ordObj) get(k string) (any, bool) {
v, ok := o.m[k]
return v, ok
}
// decodeArg decodes one truth-row argument preserving object key order.
func decodeArg(raw json.RawMessage) (any, error) {
dec := json.NewDecoder(bytes.NewReader(raw))
dec.UseNumber()
return decodeArgValue(dec)
}
func decodeArgValue(dec *json.Decoder) (any, error) {
tok, err := dec.Token()
if err != nil {
return nil, err
}
switch t := tok.(type) {
case json.Delim:
switch t {
case '{':
o := &ordObj{m: map[string]any{}}
for dec.More() {
kt, err := dec.Token()
if err != nil {
return nil, err
}
key, ok := kt.(string)
if !ok {
return nil, fmt.Errorf("bad object key %v", kt)
}
val, err := decodeArgValue(dec)
if err != nil {
return nil, err
}
if _, exists := o.m[key]; !exists {
o.keys = append(o.keys, key)
}
o.m[key] = val
}
if _, err := dec.Token(); err != nil { // '}'
return nil, err
}
return o, nil
case '[':
var arr []any
for dec.More() {
val, err := decodeArgValue(dec)
if err != nil {
return nil, err
}
arr = append(arr, val)
}
if _, err := dec.Token(); err != nil { // ']'
return nil, err
}
if arr == nil {
arr = []any{}
}
return arr, nil
}
return nil, fmt.Errorf("unexpected delim %v", t)
case json.Number:
f, err := t.Float64()
if err != nil {
return nil, err
}
return f, nil
default:
return tok, nil // string / bool / nil
}
}
func asStr(v any) string {
s, _ := v.(string)
return s
}
func asBool(v any) bool {
b, _ := v.(bool)
return b
}
func asInt(v any) int {
f, _ := v.(float64)
return int(f)
}
// profileFromAny rebuilds a Profile from a to_dict-shaped ordObj. All 11
// fields are always present in frozen rows; missing keys fall back to the
// dataclass defaults (enabled=True, zero/"" otherwise).
func profileFromAny(v any) Profile {
o, _ := v.(*ordObj)
if o == nil {
return Profile{Enabled: true}
}
enabled := true
if ev, ok := o.get("enabled"); ok {
enabled = asBool(ev)
}
return Profile{
ID: asStr(must(o, "id")),
Provider: asStr(must(o, "provider")),
Tier: asStr(must(o, "tier")),
Local: asBool(must(o, "local")),
ContextWindow: asInt(must(o, "context_window")),
SupportsTools: asBool(must(o, "supports_tools")),
SupportsStructuredOutput: asBool(must(o, "supports_structured_output")),
SupportsReasoningField: asBool(must(o, "supports_reasoning_field")),
EstimatedLatency: asStr(must(o, "estimated_latency")),
EstimatedCost: asStr(must(o, "estimated_cost")),
Enabled: enabled,
}
}
func must(o *ordObj, k string) any {
if v, ok := o.get(k); ok {
return v
}
return nil
}
// pairEntries rebuilds ordered registry entries from
// [[tier_key, profile_dict], ...].
func pairEntries(st any) []regEntry {
arr, ok := st.([]any)
if !ok || arr == nil {
return nil
}
entries := make([]regEntry, 0, len(arr))
for _, el := range arr {
pair, ok := el.([]any)
if !ok || len(pair) != 2 {
continue
}
entries = append(entries, regEntry{key: asStr(pair[0]), prof: profileFromAny(pair[1])})
}
return entries
}
// defaultOrNew — the seam wrappers build registries through the REAL
// constructor: a falsy (nil/empty) state falls back to DEFAULT_PROFILES.
// A nil state (registry=None) resolves to the default registry too.
func defaultOrNew(st any) *Registry {
if st == nil {
return defaultRegistry()
}
return newRegistry(registryFromRaw(pairEntries(st)))
}
// ── truth-row execution ────────────────────────────────────────────────
// execGapRow interprets one seam_* wrapper row against the port. args
// are decodeArg values (objects = *ordObj, order-preserving).
func execGapRow(fn string, args []any) (any, error) {
switch fn {
case "seam_to_dict":
return toAny(profileFromAny(args[0])), nil
case "seam_get_profile":
if p := defaultOrNew(args[0]).getProfile(asStr(args[1])); p != nil {
return toAny(*p), nil
}
return nil, nil
case "seam_get_enabled_profiles":
return toAny(defaultOrNew(args[0]).getEnabledProfiles()), nil
case "seam_get_profiles_by_tier":
return toAny(defaultOrNew(args[0]).getProfilesByTier(asStr(args[1]))), nil
case "seam_get_cheapest_capable":
p := defaultOrNew(args[0]).getCheapestCapable(asBool(args[1]), asBool(args[2]), asBool(args[3]))
if p != nil {
return toAny(*p), nil
}
return nil, nil
case "seam_list_profiles":
return toAny(defaultOrNew(args[0]).listProfiles()), nil
case "seam_resolve_budget":
if args[2] == nil {
return toAny(resolveBudget(asStr(args[0]), asStr(args[1]))), nil
}
reg := registryFromRaw(pairEntries(args[2]))
return toAny(newRegistry(reg).resolveContextBudget(asStr(args[0]), asStr(args[1]))), nil
case "seam_resolve_budget_raw":
reg := registryFromRaw(pairEntries(args[0]))
return toAny(reg.resolveContextBudget(asStr(args[1]), asStr(args[2]))), nil
case "seam_test_registry":
res, stdout, err := testRegistry()
if err != nil {
return nil, err
}
return map[string]any{"result": res, "stdout": stdout}, nil
case "resolve_context_budget":
kind := "default"
if len(args) > 1 && args[1] != nil {
kind = asStr(args[1])
}
if len(args) > 2 && args[2] != nil {
reg := registryFromRaw(pairEntries(args[2]))
return toAny(newRegistry(reg).resolveContextBudget(asStr(args[0]), kind)), nil
}
return toAny(resolveBudget(asStr(args[0]), kind)), nil
}
return nil, fmt.Errorf("unknown gap fn %q", fn)
}
// toAny converts port results into decodeArg-comparable values
// (structs → maps, ints → float64, typed slices → []any).
func toAny(v any) any {
switch t := v.(type) {
case nil:
return nil
case *Profile:
if t == nil {
return nil
}
return t.toDict()
case Profile:
return t.toDict()
case []Profile:
arr := make([]any, len(t))
for i := range t {
arr[i] = t[i].toDict()
}
return arr
case []map[string]any:
arr := make([]any, len(t))
for i := range t {
arr[i] = toAny(t[i])
}
return arr
case map[string]any:
m := make(map[string]any, len(t))
for k, val := range t {
m[k] = toAny(val)
}
return m
case int:
return float64(t)
default:
return v
}
}
// ordFromMap wraps a map as an ordObj (key order irrelevant — pyEqual
// compares maps semantically).
func ordFromMap(m map[string]any) *ordObj {
o := &ordObj{m: map[string]any{}}
for k, v := range m {
o.keys = append(o.keys, k)
o.m[k] = v
}
return o
}
// pyEqual — semantic equality with LIST order sensitivity (list order is
// parity: list_profiles/get_enabled_profiles rows pin it); dict key order
// is irrelevant on both sides.
func pyEqual(got, want any) bool {
got = toAny(got)
want = toAny(want)
if m, ok := got.(map[string]any); ok {
got = ordFromMap(m)
}
if m, ok := want.(map[string]any); ok {
want = ordFromMap(m)
}
if got == nil && want == nil {
return true
}
gm, gok := got.(*ordObj)
wm, wok := want.(*ordObj)
if gok || wok {
if !gok || !wok || len(gm.m) != len(wm.m) {
return false
}
for k, gv := range gm.m {
wv, ok := wm.get(k)
if !ok || !pyEqual(gv, wv) {
return false
}
}
return true
}
gl, gok := got.([]any)
wl, wok := want.([]any)
if gok || wok {
if !gok || !wok || len(gl) != len(wl) {
return false
}
for i := range gl {
if !pyEqual(gl[i], wl[i]) {
return false
}
}
return true
}
gf, gok := got.(float64)
wf, wok := want.(float64)
if gok || wok {
return gok && wok && gf == wf
}
return got == want
}
// replayGapBytes replays a truth_gap envelope; returns first divergence.
func replayGapBytes(raw []byte) error {
var env struct {
FrozenNow string `json:"frozen_now"`
Cases []gapTruthRow `json:"cases"`
}
if err := json.Unmarshal(raw, &env); err != nil {
return err
}
fails := 0
for i, row := range env.Cases {
if len(row.Recv) > 0 && string(row.Recv) != "null" {
return fmt.Errorf("case %d: recv rows are not part of this module's battery", i)
}
args := make([]any, len(row.Args))
for j, rawArg := range row.Args {
v, err := decodeArg(rawArg)
if err != nil {
return fmt.Errorf("case %d: bad arg %d: %v", i, j, err)
}
args[j] = v
}
got, err := execGapRow(row.Fn, args)
if err != nil {
fails++
fmt.Printf("case %d %s: ERROR %v\n", i, row.Fn, err)
continue
}
want, err := decodeArg(row.Expected)
if err != nil {
return fmt.Errorf("case %d: bad expected: %v", i, err)
}
if !pyEqual(got, want) {
fails++
gb, _ := json.MarshalIndent(toAny(got), "", " ")
wb, _ := json.MarshalIndent(want, "", " ")
fmt.Printf("case %d %s DIVERGE\ngot: %s\nwant: %s\n", i, row.Fn, gb, wb)
}
}
if fails > 0 {
return fmt.Errorf("PARITY FAILED: %d divergence(s)", fails)
}
fmt.Printf("PARITY OK: %d gap cases exact-equal vs the Python oracle\n", len(env.Cases))
return nil
}
type gapTruthRow struct {
Fn string `json:"fn"`
Recv json.RawMessage `json:"recv,omitempty"`
Args []json.RawMessage `json:"args"`
Kwargs json.RawMessage `json:"kwargs,omitempty"`
Expected json.RawMessage `json:"expected"`
}
// ── scenario driver (live parity: identical ops against the Python
// oracle and this port, JSON outputs diffed by
// scripts/check_model_registry_parity.py) ──────────────────────────────
type scenarioOp struct {
Op string `json:"op"`
State json.RawMessage `json:"state,omitempty"`
Tier string `json:"tier,omitempty"`
Model string `json:"model,omitempty"`
Kind string `json:"kind,omitempty"`
Tools bool `json:"tools"`
Structured bool `json:"structured"`
Reasoning bool `json:"reasoning"`
}
func runScenario(path string) error {
raw, err := os.ReadFile(path)
if err != nil {
return err
}
var sc struct {
Ops []scenarioOp `json:"ops"`
}
if err := json.Unmarshal(raw, &sc); err != nil {
return err
}
steps := []any{}
for _, op := range sc.Ops {
state := any(nil)
if len(op.State) > 0 && string(op.State) != "null" {
state, err = decodeArg(op.State)
if err != nil {
return fmt.Errorf("op %q: bad state: %v", op.Op, err)
}
}
var result any
switch op.Op {
case "to_dict":
result = toAny(profileFromAny(state))
case "get_profile":
if p := defaultOrNew(state).getProfile(op.Tier); p != nil {
result = toAny(*p)
}
case "get_enabled_profiles":
result = toAny(defaultOrNew(state).getEnabledProfiles())
case "get_profiles_by_tier":
result = toAny(defaultOrNew(state).getProfilesByTier(op.Tier))
case "get_cheapest_capable":
if p := defaultOrNew(state).getCheapestCapable(op.Tools, op.Structured, op.Reasoning); p != nil {
result = toAny(*p)
}
case "list_profiles":
result = toAny(defaultOrNew(state).listProfiles())
case "resolve":
result = toAny(resolveBudget(op.Model, op.Kind))
case "resolve_custom":
reg := registryFromRaw(pairEntries(state))
result = toAny(reg.resolveContextBudget(op.Model, op.Kind))
case "test_registry":
res, stdout, err := testRegistry()
if err != nil {
return fmt.Errorf("test_registry: %v", err)
}
result = map[string]any{"result": res, "stdout": stdout}
default:
return fmt.Errorf("unknown scenario op %q", op.Op)
}
steps = append(steps, map[string]any{"op": op.Op, "result": result})
}
enc := json.NewEncoder(os.Stdout)
enc.SetEscapeHTML(false)
enc.SetIndent("", " ")
return enc.Encode(map[string]any{"steps": steps})
}
+321 -94
View File
@@ -1,50 +1,156 @@
// Package main — model-registry: the GO port of src/rune/model_registry.py's
// get_cheapest_capable + resolve_context_budget (Oracle-Battery Parity
// Pattern, #2479 GO batch 67). The profile store stays Python (reactive).
// Package main — model-registry: the GO port of src/rune/model_registry.py
// (Oracle-Battery Parity Pattern, #2479 GO batch 67; gap-closed 2026-10-14:
// to_dict/get_profile/get_enabled_profiles/get_profiles_by_tier/
// list_profiles/test_registry ported name-preserving and resolve_context
// constants corrected — the batch-67 port hardcoded toolReserve=2048 and a
// single outputReserve; the live oracle holds TOOL_RESULT_RESERVE=1024 and
// the OUTPUT_RESERVE per-task-kind ladder).
package main
import (
"encoding/json"
"fmt"
"os"
"sort"
)
// Profile mirrors the oracle's ModelProfile dataclass field-for-field
// (to_dict emits exactly these keys in exactly this order).
type Profile struct {
ID string
Tier string
Enabled bool
SupportsTools bool
SupportsStruct bool
SupportsReason bool
ContextWindow int
Local bool
ID string
Provider string
Tier string
Local bool
ContextWindow int
SupportsTools bool
SupportsStructuredOutput bool
SupportsReasoningField bool
EstimatedLatency string
EstimatedCost string
Enabled bool
}
// the oracle's default registry (T0 deterministic + the capability floor)
var profiles = []Profile{
{ID: "deterministic", Tier: "T0", Enabled: true, ContextWindow: 8192, Local: true},
{ID: "qwen2.5:3b", Tier: "T1", Enabled: true, ContextWindow: 32768, Local: true},
{ID: "qwen3:14b", Tier: "T2", Enabled: true, ContextWindow: 32768, Local: true},
{ID: "qwen3.6:27b", Tier: "T3", Enabled: true, ContextWindow: 32768, Local: true},
{ID: "mimo-v2.5-pro", Tier: "T4", Enabled: true, SupportsTools: true, ContextWindow: 128000, Local: false},
// toDict — mirrors ModelProfile.to_dict (snake_case keys, declaration
// order, no values omitted).
func (p Profile) toDict() map[string]any {
return map[string]any{
"id": p.ID,
"provider": p.Provider,
"tier": p.Tier,
"local": p.Local,
"context_window": p.ContextWindow,
"supports_tools": p.SupportsTools,
"supports_structured_output": p.SupportsStructuredOutput,
"supports_reasoning_field": p.SupportsReasoningField,
"estimated_latency": p.EstimatedLatency,
"estimated_cost": p.EstimatedCost,
"enabled": p.Enabled,
}
}
// getCheapestCapable — Python-identical: filter by capability, sort by tier.
func getCheapestCapable(needTools, needStruct, needReason bool) *Profile {
tierOrder := map[string]int{"T0": 0, "T1": 1, "T2": 2, "T3": 3, "T4": 4}
// regEntry — the oracle stores profiles in a dict keyed by TIER, and the
// key is independent of the profile's own tier attribute. get_profile is
// a KEY lookup; get_profiles_by_tier matches the ATTRIBUTE; iteration
// order everywhere is dict-insertion order. Entries preserve both.
type regEntry struct {
key string
prof Profile
}
// Registry mirrors ModelRegistry: an insertion-ordered tier→profile dict.
type Registry struct {
entries []regEntry
}
// defaultRegistry mirrors DEFAULT_PROFILES (T0..T4, insertion order).
// T0's context_window is 0 — `context_window or 8192` handles it.
func defaultRegistry() *Registry {
return &Registry{entries: []regEntry{
{"T0", Profile{ID: "deterministic", Provider: "local", Tier: "T0",
Local: true, EstimatedLatency: "instant", EstimatedCost: "0",
Enabled: true}},
{"T1", Profile{ID: "qwen2.5:3b", Provider: "ollama", Tier: "T1",
Local: true, ContextWindow: 32768, EstimatedLatency: "low",
EstimatedCost: "0", Enabled: true}},
{"T2", Profile{ID: "qwen3:14b", Provider: "ollama", Tier: "T2",
Local: true, ContextWindow: 32768, EstimatedLatency: "medium",
EstimatedCost: "0", Enabled: true}},
{"T3", Profile{ID: "qwen3.6:27b", Provider: "ollama", Tier: "T3",
Local: true, ContextWindow: 32768, EstimatedLatency: "high",
EstimatedCost: "0", Enabled: true}},
{"T4", Profile{ID: "mimo-v2.5-pro", Provider: "openai", Tier: "T4",
SupportsTools: true, SupportsStructuredOutput: true,
SupportsReasoningField: true, ContextWindow: 128000,
EstimatedLatency: "higher", EstimatedCost: "paid", Enabled: true}},
}}
}
// newRegistry mirrors ModelRegistry(profiles): a falsy (nil/empty) dict
// falls back to DEFAULT_PROFILES.copy().
func newRegistry(profiles *Registry) *Registry {
if profiles == nil || len(profiles.entries) == 0 {
return defaultRegistry()
}
return profiles
}
// registryFromRaw mirrors a receiver whose profiles attribute was set
// directly (no constructor fallback): an empty dict stays empty.
func registryFromRaw(entries []regEntry) *Registry {
return &Registry{entries: entries}
}
// getProfile — ModelRegistry.get_profile: dict.get by KEY.
func (r *Registry) getProfile(tier string) *Profile {
for _, e := range r.entries {
if e.key == tier {
p := e.prof
return &p
}
}
return nil
}
// getEnabledProfiles — [p for p in profiles.values() if p.enabled],
// insertion order preserved.
func (r *Registry) getEnabledProfiles() []Profile {
var out []Profile
for _, e := range r.entries {
if e.prof.Enabled {
out = append(out, e.prof)
}
}
return out
}
// getProfilesByTier — [p for p in profiles.values() if p.tier == tier
// and p.enabled]: ATTRIBUTE match (not the dict key).
func (r *Registry) getProfilesByTier(tier string) []Profile {
var out []Profile
for _, e := range r.entries {
if e.prof.Tier == tier && e.prof.Enabled {
out = append(out, e.prof)
}
}
return out
}
// getCheapestCapable — capability filter, stable sort by tier rank
// (tier_order.get(tier, 5): unknown tiers sort last), insertion-order
// tie-break (Python sort is stable).
func (r *Registry) getCheapestCapable(needTools, needStruct, needReason bool) *Profile {
var capable []Profile
for _, p := range profiles {
for _, e := range r.entries {
p := e.prof
if !p.Enabled {
continue
}
if needTools && !p.SupportsTools {
continue
}
if needStruct && !p.SupportsStruct {
if needStruct && !p.SupportsStructuredOutput {
continue
}
if needReason && !p.SupportsReason {
if needReason && !p.SupportsReasoningField {
continue
}
capable = append(capable, p)
@@ -52,102 +158,223 @@ func getCheapestCapable(needTools, needStruct, needReason bool) *Profile {
if len(capable) == 0 {
return nil
}
sort.Slice(capable, func(i, j int) bool {
ti, tj := tierOrder[capable[i].Tier], tierOrder[capable[j].Tier]
if ti == 0 && capable[i].Tier != "T0" {
ti = 5
tierOrder := map[string]int{"T0": 0, "T1": 1, "T2": 2, "T3": 3, "T4": 4}
rank := func(p Profile) int {
if v, ok := tierOrder[p.Tier]; ok {
return v
}
if tj == 0 && capable[j].Tier != "T0" {
tj = 5
return 5
}
// insertion sort keeps ties in insertion order (stable, like Python)
for i := 1; i < len(capable); i++ {
for j := i; j > 0 && rank(capable[j]) < rank(capable[j-1]); j-- {
capable[j], capable[j-1] = capable[j-1], capable[j]
}
return ti < tj
})
}
return &capable[0]
}
const outputReserve = 2048
// listProfiles — [p.to_dict() for p in profiles.values()], insertion
// order preserved.
func (r *Registry) listProfiles() []map[string]any {
out := make([]map[string]any, 0, len(r.entries))
for _, e := range r.entries {
out = append(out, e.prof.toDict())
}
return out
}
// resolveContextBudget — window - output_reserve - tool_reserve (0 for
// local), num_ctx floored at 1024. Python-identical.
func resolveContextBudget(modelID string) map[string]any {
window := 8192
local := true
for _, p := range profiles {
if p.ID == modelID {
window = p.ContextWindow
if window == 0 {
window = 8192
}
local = p.Local
break
// resolveContextBudget — the module-level resolve_context_budget with
// THIS registry as the `registry` argument (entries used raw: the empty
// registry exercises the `(reg.profiles or {})` branch).
func (r *Registry) resolveContextBudget(modelID, taskKind string) map[string]any {
var profile *Profile
for _, e := range r.entries {
if e.prof.ID == modelID {
p := e.prof
profile = &p
break // first id match wins (insertion order)
}
}
window, local := 8192, true
if profile != nil {
window = profile.ContextWindow
if window == 0 {
window = 8192
}
local = profile.Local
}
ort, ok := outputReserves[taskKind]
if !ok {
ort = outputReserves["default"] // OUTPUT_RESERVE.get(task_kind, ...)
}
toolReserve := 0
if !local {
toolReserve = 2048 // TOOL_RESULT_RESERVE
toolReserve = toolResultReserve
}
numCtx := window - outputReserve - toolReserve
numCtx := window - ort - toolReserve
if numCtx < 1024 {
numCtx = 1024
}
return map[string]any{
"num_ctx": numCtx, "window": window,
"output_reserve": outputReserve, "tool_reserve": toolReserve,
"output_reserve": ort, "tool_reserve": toolReserve,
"model": modelID, "local": local,
}
}
// ── context budgets (#28: model-aware, never hardcoded 8192) ───────────
// outputReserves mirrors OUTPUT_RESERVE (per-task-kind ladder).
var outputReserves = map[string]int{
"default": 2048, "verify": 512, "summarize": 1024, "embed": 0,
}
// toolResultReserve mirrors TOOL_RESULT_RESERVE (2026-10-14: the batch-67
// port hardcoded 2048 here — the live oracle holds 1024; gated by LOCAL,
// not by tier).
const toolResultReserve = 1024
// resolveBudget — resolve_context_budget with registry=None (the oracle
// builds its own default ModelRegistry).
func resolveBudget(modelID, taskKind string) map[string]any {
return newRegistry(nil).resolveContextBudget(modelID, taskKind)
}
// legacy facade (batch-67 replay): default-registry single-model resolve.
func resolveContextBudget(modelID string) map[string]any {
return resolveBudget(modelID, "default")
}
// legacy facade (batch-67 replay): default-registry cheapest search.
func getCheapestCapable(needTools, needStruct, needReason bool) *Profile {
return defaultRegistry().getCheapestCapable(needTools, needStruct, needReason)
}
// testRegistry — mirrors test_registry: prints the same lines, runs the
// same assertions, returns None. Returns the captured stdout; a failed
// assertion is an error (the frozen row pins the success path).
func testRegistry() (any, string, error) {
var b []byte
say := func(format string, args ...any) {
b = append(b, []byte(fmt.Sprintf(format+"\n", args...))...)
}
say("Testing Model Registry...")
reg := defaultRegistry()
profiles := reg.listProfiles()
say("Total profiles: %d", len(profiles))
if len(profiles) != 5 {
return nil, string(b), fmt.Errorf("AssertionError: %d != 5", len(profiles))
}
t1 := reg.getProfile("T1")
say("T1 profile: %s", t1.ID)
if t1.ID != "qwen2.5:3b" {
return nil, string(b), fmt.Errorf("AssertionError: T1 id %q", t1.ID)
}
cheapest := reg.getCheapestCapable(false, false, false)
say("Cheapest capable: %s", cheapest.ID)
if cheapest.Tier != "T0" {
return nil, string(b), fmt.Errorf("AssertionError: cheapest tier %q", cheapest.Tier)
}
cheapestTools := reg.getCheapestCapable(true, false, false)
say("Cheapest with tools: %s", cheapestTools.ID)
if cheapestTools.Tier != "T4" {
return nil, string(b), fmt.Errorf("AssertionError: cheapest-with-tools tier %q", cheapestTools.Tier)
}
say("✅ All tests passed!")
return nil, string(b), nil
}
func main() {
if len(os.Args) < 2 || os.Args[1] != "replay" {
if len(os.Args) < 2 {
fmt.Fprintln(os.Stderr, "usage: model-registry replay <truth.json>")
os.Exit(2)
}
raw, err := os.ReadFile(os.Args[2])
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
var cases []map[string]any
if err := json.Unmarshal(raw, &cases); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
fails := 0
for i, c := range cases {
op, _ := c["op"].(string)
switch op {
case "cheapest":
want, _ := c["want"].(string)
if got := getCheapestCapable(false, false, false); got == nil || got.Tier != want {
fails++
fmt.Printf("case %d cheapest: divergence\n", i)
}
case "cheapest_tools":
want := c["want"].([]any)
got := getCheapestCapable(true, false, false)
if got == nil || got.Tier != want[0].(string) {
fails++
fmt.Printf("case %d cheapest_tools: divergence\n", i)
}
case "budget_unknown", "budget_known":
model := "nonexistent-model"
if m, ok := c["model"].(string); ok {
model = m
}
got := resolveContextBudget(model)
want := c["want"].(map[string]any)
for k, wv := range want {
gv := got[k]
if fmt.Sprintf("%v", gv) != fmt.Sprintf("%v", wv) {
switch os.Args[1] {
case "replay":
if len(os.Args) < 3 {
fmt.Fprintln(os.Stderr, "usage: model-registry replay <truth.json>")
os.Exit(2)
}
raw, err := os.ReadFile(os.Args[2])
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
var cases []map[string]any
if err := json.Unmarshal(raw, &cases); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
fails := 0
for i, c := range cases {
op, _ := c["op"].(string)
switch op {
case "cheapest":
want, _ := c["want"].(string)
if got := getCheapestCapable(false, false, false); got == nil || got.Tier != want {
fails++
fmt.Printf("case %d %s.%s: got %v want %v\n", i, op, k, gv, wv)
fmt.Printf("case %d cheapest: divergence\n", i)
}
case "cheapest_tools":
want := c["want"].([]any)
got := getCheapestCapable(true, false, false)
if got == nil || got.Tier != want[0].(string) {
fails++
fmt.Printf("case %d cheapest_tools: divergence\n", i)
}
case "budget_unknown", "budget_known":
model := "nonexistent-model"
if m, ok := c["model"].(string); ok {
model = m
}
got := resolveContextBudget(model)
want := c["want"].(map[string]any)
for k, wv := range want {
gv := got[k]
if fmt.Sprintf("%v", gv) != fmt.Sprintf("%v", wv) {
fails++
fmt.Printf("case %d %s.%s: got %v want %v\n", i, op, k, gv, wv)
}
}
}
}
if fails > 0 {
fmt.Fprintf(os.Stderr, "PARITY FAILED: %d divergence(s)\n", fails)
os.Exit(1)
}
fmt.Printf("PARITY OK: %d cases vs the Python oracle\n", len(cases))
case "replay-gap":
if len(os.Args) < 3 {
fmt.Fprintln(os.Stderr, "usage: model-registry replay-gap <truth_gap.json>")
os.Exit(2)
}
raw, err := os.ReadFile(os.Args[2])
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
if err := replayGapBytes(raw); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
case "scenario":
if len(os.Args) < 3 {
fmt.Fprintln(os.Stderr, "usage: model-registry scenario <scenario.json>")
os.Exit(2)
}
if err := runScenario(os.Args[2]); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
default:
fmt.Fprintln(os.Stderr, "usage: model-registry replay|replay-gap|scenario <file>")
os.Exit(2)
}
if fails > 0 {
fmt.Fprintf(os.Stderr, "PARITY FAILED: %d divergence(s)\n", fails)
os.Exit(1)
}
fmt.Printf("PARITY OK: %d cases vs the Python oracle\n", len(cases))
}
File diff suppressed because it is too large. Load diff
+91
View File
@@ -0,0 +1,91 @@
package main
// truth_gap_test.go — gap-close battery: replays truth_gap.json (frozen
// against the live Python oracle through scripts/gen_model_registry_
// gap_truth.py wrapper shims) and pins the constants + lookup semantics
// the 2026-10-09 batch-67 port got wrong or never carried:
// * TOOL_RESULT_RESERVE = 1024 (was hardcoded 2048) — gated by LOCAL,
// not tier
// * OUTPUT_RESERVE is a per-task-kind ladder (2048/512/1024/0), not a
// single constant
// * get_profile is a KEY lookup; get_profiles_bytier matches the tier
// ATTRIBUTE; dict insertion order is observable everywhere
import (
"os"
"testing"
)
func TestGapParity(t *testing.T) {
raw, err := os.ReadFile("truth_gap.json")
if err != nil {
t.Fatalf("truth missing: %v", err)
}
if err := replayGapBytes(raw); err != nil {
t.Fatalf("gap parity: %v", err)
}
}
// TestReserveConstants pins the live-oracle constants (the known flag
// from the model_connect agent, 2026-10-14).
func TestReserveConstants(t *testing.T) {
want := map[string]int{"default": 2048, "verify": 512, "summarize": 1024, "embed": 0}
for kind, reserve := range want {
if got := outputReserves[kind]; got != reserve {
t.Errorf("OUTPUT_RESERVE[%q]: got %d want %d", kind, got, reserve)
}
}
if toolResultReserve != 1024 {
t.Errorf("TOOL_RESULT_RESERVE: got %d want 1024", toolResultReserve)
}
// the flag fix, end to end: cloud model, default kind
b := resolveContextBudget("mimo-v2.5-pro")
if b["num_ctx"] != 124928 || b["tool_reserve"] != 1024 {
t.Errorf("flag fix broke: %v", b)
}
}
// TestLookupSemantics pins key-vs-attribute lookup and ordering laws.
func TestLookupSemantics(t *testing.T) {
reg := registryFromRaw([]regEntry{
{"K1", Profile{ID: "p1", Tier: "T9", Enabled: true}},
})
if reg.getProfile("T9") != nil {
t.Error("get_profile must be a KEY lookup, not a tier-attribute match")
}
if reg.getProfile("K1") == nil {
t.Error("get_profile missed an existing KEY")
}
if len(reg.getProfilesByTier("T9")) != 1 {
t.Error("get_profiles_by_tier must match the tier ATTRIBUTE")
}
// constructor truthiness: falsy profiles → DEFAULT_PROFILES
if got := newRegistry(registryFromRaw(nil)).getProfile("T1"); got == nil || got.ID != "qwen2.5:3b" {
t.Errorf("falsy registry must fall back to DEFAULT_PROFILES, got %v", got)
}
// raw registry keeps emptiness (the (profiles or {}) branch)
raw := registryFromRaw(nil)
if b := raw.resolveContextBudget("ghost", "verify"); b["num_ctx"] != 7680 {
t.Errorf("raw empty registry: %v", b)
}
}
// TestStableSortTie pins the stable tie-break on get_cheapest_capable.
func TestStableSortTie(t *testing.T) {
reg := registryFromRaw([]regEntry{
{"TA", Profile{ID: "first", Tier: "T2", Enabled: true}},
{"TB", Profile{ID: "second", Tier: "T2", Enabled: true}},
})
if got := reg.getCheapestCapable(false, false, false); got == nil || got.ID != "first" {
t.Errorf("stable tie-break: got %v want first", got)
}
// first-id-match wins in resolve (insertion order)
dup := registryFromRaw([]regEntry{
{"fake", Profile{ID: "mimo-v2.5-pro", Tier: "T4", Local: true, ContextWindow: 32768, Enabled: true}},
{"real", Profile{ID: "mimo-v2.5-pro", Tier: "T4", Local: false, ContextWindow: 128000, Enabled: true}},
})
b := dup.resolveContextBudget("mimo-v2.5-pro", "default")
if b["local"] != true || b["num_ctx"] != 30720 {
t.Errorf("first-id-match: %v", b)
}
}
+1
View File
@@ -0,0 +1 @@
0
+1
View File
@@ -0,0 +1 @@
0
+1
View File
@@ -0,0 +1 @@
0
+1
View File
@@ -0,0 +1 @@
0
+1
View File
@@ -0,0 +1 @@
0
+1
View File
@@ -0,0 +1 @@
0
+1
View File
@@ -0,0 +1 @@
0
+61
View File
@@ -0,0 +1,61 @@
# model_registry dichotomy gap-close — verification evidence (2026-10-14)
Module: src/rune/model_registry.py (253L, 8 fns) → src/go/model-registry
(dichotomy wave #2479; worktree rune-wave-db2; contract BUILDER-…-factory-gap-mreg)
## Before / after
- Audit coverage: **2/8 PARTIAL → 8/8 FULL (GO)** —
missing were `test_registry, to_dict, get_profile, get_enabled_profiles,
get_profiles_by_tier, list_profiles`; all ported name-preserving
(audit candidate forms): `toDict, getProfile, getEnabledProfiles,
getProfilesByTier, listProfiles, testRegistry`.
- Programme: FULL 53% → 55% (105/192 modules); fn-level 1612/2282.
- KNOWN FLAG VERIFIED AND FIXED: the 2026-10-09 batch-67 port hardcoded
`toolReserve = 2048` (main.go:87 of that revision) where the live
oracle holds `TOOL_RESULT_RESERVE = 1024` (model_registry.py:225),
and a single `outputReserve = 2048` where the oracle has the
OUTPUT_RESERVE per-task-kind ladder {default 2048, verify 512,
summarize 1024, embed 0} — gated by LOCAL, never by tier. Legacy
truth.json never noticed (both rows are local models). Now:
`TestReserveConstants` + frozen rows `resolve("mimo-v2.5-pro", …)` →
num_ctx 124928 / 125952 / 126976.
## Verification gates (all green in THIS checkout)
| Gate | Result | Evidence |
|---|---|---|
| freeze_truth --verify (byte-identical vs live oracle) | OK: 35 cases | mreg-freeze-verify.{log,exit} |
| fresh regeneration byte-identity (2 runs) | identical (cmp) | mreg-freeze-verify.log |
| gofmt -l · go vet · go build ./... | clean | mreg-go-ladder.log |
| go test ./... (TestGapParity + 3 unit gates) | 4/4 PASS | mreg-go-ladder.log |
| CI binary replay-gap (truth_gap.json, freeze format) | 35/35 | mreg-go-ladder.log |
| legacy replay (batch-67 truth.json, regression) | 4/4 | mreg-go-ladder.log |
| LIVE parity oracle vs GO (scripts/check_model_registry_parity.py) | 4/4 scenarios, 36 ops | mreg-parity.{log,exit} |
| Python suite, live pin | 5 passed | mreg-pytests-live.{log,exit} |
| Python suite, RUNE_MODEL_REGISTRY_CORE=python pin | 5 passed | mreg-pytests-python.{log,exit} |
| Oracle CLI smoke (`model_registry.py test`) | all tests passed | mreg-oracle-smoke.{log,exit} |
| Tree-wide `go build ./...` census | 83/86 OK; 3 pre-existing failures (adaptive-router, builder-bridge-cli, paths) link cargo-built Rust staticlibs absent from clean checkouts — not caused by this diff (module untouched) | mreg-tree-build.txt |
| dichotomy_audit.py --write | model_registry 8/8 GO row | mreg-audit-write.{log,exit} |
(ruff check on the three new Python files: all checks passed.)
## Truth-freeze notes
- Shared harness (scripts/freeze_truth.py) + rig scripts/mreg_freeze_seams
(sitecustomize binds JSON-safe seam_* wrappers — registry methods return
ModelProfile dataclasses the harness can't serialize; the real oracle
methods run unmodified). 35 cases, envelope frozen_now 2026-10-14T09:00:00
(vacuous pin — this oracle has no clock).
- Registry states ride as ORDERED [tier_key, profile_dict] pairs; insertion
order is load-bearing (get_profile = KEY lookup, get_profiles_by_tier =
tier-ATTRIBUTE match, resolve_context_budget breaks at FIRST id match,
get_cheapest_capable sorts STABLE on ties).
- Receiver-style truth impossible here (module-level fn would bind the
receiver to model_id); the `(reg.profiles or {})` empty branch is pinned
via seam_resolve_budget_raw (constructor-bypassed __new__ registry).
- Constructor falsy law pinned: ModelRegistry({}) → DEFAULT_PROFILES.copy()
(rows via [] pairs), while raw-empty stays empty (raw rows).
- No loader seam exists in the oracle (no RUNE_MODEL_REGISTRY_CORE reader):
the both-pins requirement is satisfied by running the suite + oracle
smoke under BOTH env states; delegation-live is vacuous for this module.
+86
View File
@@ -0,0 +1,86 @@
OK abstraction
FAIL adaptive-router
OK auto-context
OK auto_context
OK auto-detect
OK auto-tasks
OK auto_tasks
OK builder-bridge
FAIL builder-bridge-cli
OK build-mode
OK cache
OK capability-broker
OK cognition
OK connection-pool
OK context-broker
OK core-server
OK cron-engine
OK curator
OK curator-core
OK drift-core
OK elastic-cognition
OK event-bus
OK event-envelope
OK event-wait
OK evidence-probe
OK failure-filer
OK forge
OK forge-client
OK forge_client
OK forge-core
OK goal
OK goal-engine
OK graph-memory
OK hook-engine
OK inference-paths
OK local-tool-runtime
OK memory-consolidation
OK memory-ops
OK memory-store-deep
OK message-queue
OK message_queue
OK microkernel
OK model-connect
OK model-profiles
OK model-registry
OK model-routing
OK model_routing
OK night-mode
OK openai-compat
OK openai_compat_transport
OK operator
OK operator-deep
OK operator-support
OK outcome-ledger
OK parallel-processor
FAIL paths
OK persona-gate
OK pins
OK progressive-escalation
OK provider-budget
OK provider-context
OK provider-registry
OK recall-router
OK recall-uplift
OK release-notes
OK response-adapter
OK rune-entry
OK scaffold
OK scaffold-membrane
OK sleep-time
OK stage-guards
OK standardized-providers
OK subsystem-health
OK tiered-executor
OK tool-executor
OK tool-protocol
OK tool-protocol-rw
OK tui
OK verification_layer
OK voicebox
OK voice-hybrid
OK voice-injector
OK voice-sync
OK warm-router
OK witness
OK work-manager