feat(model_connect): close PARTIAL gaps — dichotomy wave

Oracle src/rune/model_connect.py 471 lines / 11 fns: 7/11 -> 11/11 per
scripts/dichotomy_audit.py (name-preserving match). The four audit-named
gaps land as ported Go functions: CtxBudget (_ctx_budget — vendored
model_registry.resolve_context_budget num_ctx kernel with the live
oracle's constants), Now (_now — behind the injectable clock seam),
GetCloudModels (_get_cloud_models — ordered-JSON JSONC reader with the
oracle's naive quirks), HandleHook (handle_hook).

Oracle quirks preserved and frozen as truth rows: string-blind JSONC //
strip (URLs corrupt the parse -> []), trailing-comma strip, partial
cloud list on mid-provider shape errors (try/except wraps the whole
loop), insertion-order provider/model iteration (json.Decoder token
walk), null models dict -> items() raises, OUTPUT_RESERVE['default']/
TOOL_RESULT_RESERVE=2048/1024 from the live registry oracle.

Truth: scripts/gen_model_connect_truth.py via the shared freeze_truth.py
harness (40-case envelope, pinned clock 2026-10-14T08:00:00+00:00 +
pinned HOME via scripts/mconn_freeze_seams); freeze_truth --verify
byte-identical; truth.json embedded VERBATIM in truth_test.go
TestFrozenTruthReplay (3/3). Live parity 44/44
(scripts/check_model_connect_parity.py). gofmt/vet/go build clean.
Python oracle smoke green both pins (module has no loader seam; the
byte-identity gate replays the live oracle itself).
This commit is contained in:
Nick committed 2026-10-13 23:09:01 +01:00
1 parent 70f8775838
commit 290ed7341e
18 files changed
+1543 -255

No files matched your search

+197
View File
@@ -0,0 +1,197 @@
#!/usr/bin/env python3
"""check_model_connect_parity.py — live oracle-vs-Go parity sweep for the
model_connect gap closure (#2479, contract factory-gap-mconn).
The frozen envelope (truth.json) pins the ported surface byte-identically.
THIS script drives the LIVE Python oracle and the Go binary (stdin op
dispatch) across every op family — clean, moe, route, ctx_budget, now
(clock pinned on both sides), hook, and the cloud-model config sweep
(per-fixture hermetic HOME shared by oracle and binary) — comparing
sort_keys JSON. Exit 0 on 100% parity, 1 with the first divergences.
Usage: python3 scripts/check_model_connect_parity.py
[--bin src/go/model-connect/model-connect]
"""
import argparse
import json
import os
import subprocess
import sys
import tempfile
from datetime import datetime, timezone
from pathlib import Path
REPO = Path(__file__).resolve().parent.parent
sys.path.insert(0, str(REPO / "src"))
import rune.model_connect as mc # noqa: E402
FROZEN_NOW = "2026-10-14T08:00:00+00:00"
CLEAN = [
"hf.co/DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF:IQ2_M",
"hf.co/org/Llama-3-8B-Instruct:latest",
"hf.co/short/gemma-2-9b",
"hf.co/user/nano-model",
"hf.co/user/ab",
"qwen2.5:3b",
"mimo-v2.5-pro",
"llava:7b",
"huggingface.co/x/y/Llama-3-8B",
]
MOE = [
"mixtral:8x7b", "qwen2.5:3b", "deepseek-moe-16b", "My-Mixture-Model",
"expert-v1", "4x22b-tiny", "plain-conversational", "GPT-XL",
]
MODELS_ALL = [
"qwen2.5:3b", "qwen2.5:7b", "qwen3:14b", "qwen2.5-coder:14b",
"hf.co/DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF:IQ2_M",
"mixtral:8x7b-instruct-v0.1-q4_0", "qwen2.5vl:3b", "llava:7b",
"qwen3-embedding:0.6b", "nomic-embed-text:latest", "mxbai-embed-large:latest",
]
ROUTE = [
("coding", MODELS_ALL), ("research", MODELS_ALL), ("conversation", MODELS_ALL),
("unrestricted", MODELS_ALL), ("unknown-type", MODELS_ALL),
("coding", ["mystery-model:x"]), ("coding", []),
]
CTX = ["deterministic", "qwen2.5:3b", "qwen3:14b", "qwen3.6:27b",
"mimo-v2.5-pro", "totally-unknown-model", ""]
HOOKS = [
("pre_tool_use", {"tool": "bash", "dry": False}),
("", None),
("post_response", {"ok": True}),
("unicode-évent", {}),
]
FIXTURES = {
"main": """{
// MiMoCode config — JSONC: comments and trailing commas allowed
"provider": {
"ollama": { // skipped: local models handled by list_models
"models": {
"qwen3:14b": {},
},
},
"openai": {
"models": {
"gpt-5.2": {"contextLength": 400000},
"o4-mini": "not-even-a-dict",
},
},
"kimi": {
"models": {
"kimi-for-coding": {},
},
},
},
}
""",
"only_ollama": '{\n "provider": {\n "ollama": {"models": {"qwen2.5:3b": {}}},\n },\n}\n',
"no_models_key": '{\n "provider": {\n "anthropic": {"apiKey": "sk-test"},\n },\n}\n',
"no_provider_key": '{\n "theme": "dark",\n}\n',
"broken_late": '{\n "provider": {\n "a": {"models": {"m1": {}}},\n "b": {"models": "not-a-dict"},\n },\n}\n',
"null_models": '{\n "provider": {\n "z": {"models": null},\n },\n}\n',
"url_corruption": '{\n "provider": {\n "gateway": {\n "baseUrl": "http://api.example.com/v1",\n "models": {"gw-model": {}},\n },\n },\n}\n',
}
def run_go(binpath, req, env=None):
e = dict(os.environ)
if env:
e.update(env)
p = subprocess.run([str(binpath)], input=json.dumps(req), capture_output=True,
text=True, env=e)
if p.returncode != 0:
raise RuntimeError(f"go op {req.get('op')} exited {p.returncode}: {p.stdout} {p.stderr}")
return json.loads(p.stdout)
def frozen_clock():
base = datetime.fromisoformat(FROZEN_NOW)
class FrozenDatetime(datetime):
@classmethod
def now(cls, tz=None):
return base if tz is not None else base.replace(tzinfo=None)
return FrozenDatetime
def main():
ap = argparse.ArgumentParser()
ap.add_argument("--bin", default=str(REPO / "src/go/model-connect/model-connect"))
args = ap.parse_args()
binpath = Path(args.bin)
if not binpath.exists():
subprocess.run(["go", "build", "-o", str(binpath), str(binpath.parent)],
check=True, cwd=binpath.parent)
fails = total = 0
def check(name, oracle_val, go_val):
nonlocal fails, total
total += 1
if json.dumps(oracle_val, sort_keys=True) != json.dumps(go_val, sort_keys=True):
fails += 1
print(f"DIVERGE {name}:\n oracle: {json.dumps(oracle_val)[:300]}\n go: {json.dumps(go_val)[:300]}")
for name in CLEAN:
g = run_go(binpath, {"op": "clean", "name": name})
check(f"clean:{name[:40]}", {"result": mc._clean_model_name(name)}, g)
for name in MOE:
g = run_go(binpath, {"op": "moe", "name": name})
check(f"moe:{name}", {"result": mc.is_moe_model(name)}, g)
for task, models in ROUTE:
g = run_go(binpath, {"op": "route", "task": task, "models": models})
check(f"route:{task}", mc.route_model(task, models), g)
for mid in CTX:
g = run_go(binpath, {"op": "ctx_budget", "model": mid})
check(f"ctx_budget:{mid}", {"result": mc._ctx_budget(mid)}, g)
saved_dt = mc.datetime
mc.datetime = frozen_clock()
try:
g = run_go(binpath, {"op": "now"}, env={"MCONN_CLOCK": FROZEN_NOW})
check("now", {"result": mc._now()}, g)
finally:
mc.datetime = saved_dt
for event, data in HOOKS:
g = run_go(binpath, {"op": "hook", "event": event, "data": data})
check(f"hook:{event}", {"result": mc.handle_hook(event, data)}, g)
saved_home = os.environ.get("HOME")
try:
for label, fixture in FIXTURES.items():
with tempfile.TemporaryDirectory() as td:
cfg = os.path.join(td, ".config", "mimocode")
os.makedirs(cfg)
with open(os.path.join(cfg, "mimocode.jsonc"), "w") as f:
f.write(fixture)
os.environ["HOME"] = td
oracle_val = mc._get_cloud_models()
g = run_go(binpath, {"op": "cloud_models"}, env={"HOME": td})
check(f"cloud:{label}", {"result": oracle_val}, g)
with tempfile.TemporaryDirectory() as td:
os.environ["HOME"] = td
oracle_val = mc._get_cloud_models()
g = run_go(binpath, {"op": "cloud_models"}, env={"HOME": td})
check("cloud:missing", {"result": oracle_val}, g)
finally:
if saved_home is not None:
os.environ["HOME"] = saved_home
print(f"parity: {total - fails}/{total} match")
return 1 if fails else 0
if __name__ == "__main__":
sys.exit(main())
+177
View File
@@ -0,0 +1,177 @@
#!/usr/bin/env python3
"""gen_model_connect_truth.py — freeze the model_connect gap-close truth
envelope via the shared freeze_truth.py harness (#2479 wave 4).
Covers the full ported surface, exercising every branch:
* _clean_model_name — HF repo paths (family+size / family+version+size /
short-part / no-match truncation) and clean names (passthrough)
* is_moe_model — every MOE pattern family + negatives
* route_model — exact matches per tier, unknown task type, fallback,
empty list (null model/tier)
* _ctx_budget — every DEFAULT_PROFILES id (incl. deterministic's 0-window
→ 8192 floor), the cloud (non-local) tool reserve, unknown + empty ids
* _now — the pinned-clock isoformat
* handle_hook — constant envelope over varied events/data
* _get_cloud_models (seeded/missing seams) — JSONC comments + trailing
commas, ollama skip, multi-provider ordering, provider without models,
config without provider, the PARTIAL-return mid-loop shape error, the
naive comment-strip corrupting URLs, and the missing-file branch
Usage (from the repo root; the seams dir is mandatory — it pins HOME and
binds the cloud seams):
PYTHONPATH=scripts/mconn_freeze_seams:src python3 scripts/gen_model_connect_truth.py
"""
import os
import sys
from pathlib import Path
REPO = Path(os.path.abspath(__file__)).resolve().parents[1]
sys.path.insert(0, str(REPO / "src"))
sys.path.insert(0, str(REPO / "scripts"))
FROZEN_NOW = "2026-10-14T08:00:00+00:00" # microsecond=0: isoformat == the pin
os.environ["RUNE_FROZEN_NOW"] = FROZEN_NOW
import freeze_truth # noqa: E402
MODELS_ALL = [
"qwen2.5:3b",
"qwen2.5:7b",
"qwen3:14b",
"qwen2.5-coder:14b",
"hf.co/DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF:IQ2_M",
"mixtral:8x7b-instruct-v0.1-q4_0",
"qwen2.5vl:3b",
"llava:7b",
"qwen3-embedding:0.6b",
"nomic-embed-text:latest",
"mxbai-embed-large:latest",
]
FIXTURE_MAIN = """{
// MiMoCode config — JSONC: comments and trailing commas allowed
"provider": {
"ollama": { // skipped: local models handled by list_models
"models": {
"qwen3:14b": {},
},
},
"openai": {
"models": {
"gpt-5.2": {"contextLength": 400000},
"o4-mini": "not-even-a-dict", // model_config is unused by the oracle
},
},
"kimi": {
"models": {
"kimi-for-coding": {},
},
},
},
}
"""
FIXTURE_ONLY_OLLAMA = """{
"provider": {
"ollama": {
"models": {"qwen2.5:3b": {}},
},
},
}
"""
FIXTURE_NO_MODELS_KEY = """{
"provider": {
"anthropic": {
"apiKey": "sk-test",
},
},
}
"""
FIXTURE_NO_PROVIDER_KEY = """{
// no providers configured
"theme": "dark",
}
"""
FIXTURE_BROKEN_LATE = """{
"provider": {
"a": {"models": {"m1": {}}},
"b": {"models": "not-a-dict"},
},
}
"""
FIXTURE_URL_CORRUPTION = """{
"provider": {
"gateway": {
"baseUrl": "http://api.example.com/v1",
"models": {"gw-model": {}},
},
},
}
"""
cases = []
# _clean_model_name
for name in [
"hf.co/DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF:IQ2_M",
"hf.co/org/Llama-3-8B-Instruct:latest",
"hf.co/short/gemma-2-9b",
"hf.co/user/nano-model",
"hf.co/user/ab",
"qwen2.5:3b",
"mimo-v2.5-pro",
"llava:7b",
]:
cases.append(("_clean_model_name", (name,), {}))
# is_moe_model
for name, want in [
("mixtral:8x7b", True),
("qwen2.5:3b", False),
("deepseek-moe-16b", True),
("My-Mixture-Model", True),
("expert-v1", True),
("4x22b-tiny", True),
("plain-conversational", False),
("GPT-XL", False),
]:
cases.append(("is_moe_model", (name,), {}))
# route_model (available_models always explicit — the None default would
# fall through to the networked list_models, which is reactive-side only)
for task, models in [
("coding", MODELS_ALL),
("research", MODELS_ALL),
("conversation", MODELS_ALL),
("unknown-type", MODELS_ALL),
("coding", ["mystery-model:x"]),
("coding", []),
]:
cases.append(("route_model", (task,), {"available_models": models}))
# _ctx_budget
for mid in ["deterministic", "qwen2.5:3b", "qwen3:14b", "qwen3.6:27b",
"mimo-v2.5-pro", "totally-unknown-model", ""]:
cases.append(("_ctx_budget", (mid,), {}))
# _now (pinned clock)
cases.append(("_now", (), {}))
# handle_hook
cases.append(("handle_hook", ("pre_tool_use", {"tool": "bash", "dry": False}), {}))
cases.append(("handle_hook", ("", None), {}))
cases.append(("handle_hook", ("post_response", {"ok": True}), {}))
# _get_cloud_models via the pinned-HOME seams
for fixture in [FIXTURE_MAIN, FIXTURE_ONLY_OLLAMA, FIXTURE_NO_MODELS_KEY,
FIXTURE_NO_PROVIDER_KEY, FIXTURE_BROKEN_LATE,
FIXTURE_URL_CORRUPTION]:
cases.append(("_get_cloud_models_seeded", (fixture,), {}))
cases.append(("_get_cloud_models_missing", (), {}))
truth = freeze_truth.freeze("model_connect", cases, oracle_module="rune.model_connect")
out = REPO / "src" / "go" / "model-connect" / "truth.json"
freeze_truth.write_envelope(FROZEN_NOW, truth, out)
print(f"wrote {len(truth)} cases -> {out}")
# sanity: the pinned clock must render exactly as the envelope pin
now_row = [c for c in truth if c["fn"] == "_now"][0]
assert now_row["expected"] == FROZEN_NOW, now_row
print("_now pinned-clock row:", now_row["expected"])
@@ -0,0 +1,59 @@
# Freeze rig for the model_connect dichotomy gap-closing (dichotomy wave
# #2479 gap port, contract factory-gap-mconn). Auto-imported on interpreter
# start (PYTHONPATH=scripts/mconn_freeze_seams:src) by BOTH the truth
# generator (scripts/gen_model_connect_truth.py), the shared harness's
# --verify mode, and the live-parity sweep, so frozen truth is
# byte-reproducible in any process:
#
# * HOME is pinned to a fresh throwaway dir (freeze_rigs.pin_home) —
# _get_cloud_models derives ~/.config/mimocode/mimocode.jsonc via
# expanduser at call time; hermetic HOME keeps --verify off the real
# ~/.config and makes every row deterministic.
# * The per-case cloud-model seams (_get_cloud_models_seeded /
# _get_cloud_models_missing) are bound onto the oracle module so the
# shared harness's getattr(module, fn) can freeze them: seeded writes the
# raw JSONC fixture into the pinned HOME then calls the REAL
# _get_cloud_models; missing removes the file. The battery reproduces the
# same rig from the case args (fixture text rides in the truth rows).
# * The clock needs no rig here: the shared harness pins the module-global
# datetime from the envelope's frozen_now (RUNE_FROZEN_NOW), which is the
# only clock site in the ported surface (_now).
import os
import sys
_REPO = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
sys.path.insert(0, os.path.join(_REPO, "scripts"))
sys.path.insert(0, os.path.join(_REPO, "src"))
from freeze_rigs import pin_home # noqa: E402
pin_home("mconn")
import rune.model_connect as _mc # noqa: E402 (after the HOME pin)
def _config_path():
return os.path.join(os.environ["HOME"], ".config", "mimocode", "mimocode.jsonc")
def _get_cloud_models_seeded(fixture):
"""Write the raw JSONC fixture into the pinned HOME, then read it
through the oracle's real _get_cloud_models."""
path = _config_path()
os.makedirs(os.path.dirname(path), exist_ok=True)
with open(path, "w") as f:
f.write(fixture)
return _mc._get_cloud_models()
def _get_cloud_models_missing():
"""Remove the config from the pinned HOME (fresh-home error branch)."""
path = _config_path()
if os.path.exists(path):
os.remove(path)
return _mc._get_cloud_models()
_mc._get_cloud_models_seeded = _get_cloud_models_seeded
_mc._get_cloud_models_missing = _get_cloud_models_missing
+441 -61
View File
@@ -1,22 +1,33 @@
// Package main — model-connect: the GO port of src/rune/model_connect.py's
// deterministic cores (batch 113 of the dichotomy tranche, #2479).
// deterministic cores (batch 113 of the dichotomy tranche, #2479; gap-close
// wave 4).
//
// Dichotomy verdict: GO (reactive/entry surface). Network I/O (list_models,
// health_check, generate) stays on the Python reactive side — only the pure
// transforms are ported: _clean_model_name, is_moe_model, route_model.
// health_check, generate) stays on the Python reactive side — the ported
// surface is the pure/config-derived transforms: _clean_model_name,
// is_moe_model, route_model, and (gap-close) _ctx_budget, _now,
// _get_cloud_models, handle_hook.
//
// Parity: truth.json (20 cases) generated oracle-first against the Python
// module; replayed hard in CI (non-swallowable step). Parity is the floor —
// improvements: none required; oracle verified correct on all probed edges
// (empty-list route returns reason "no-models-available" with null model).
// Parity: truth.json (shared freeze_truth.py envelope, pinned clock + pinned
// HOME) generated oracle-first against the Python module via
// scripts/gen_model_connect_truth.py; byte-verified with freeze_truth.py
// --verify and embedded VERBATIM in truth_test.go TestFrozenTruthReplay.
// Live parity sweep: scripts/check_model_connect_parity.py (op dispatch over
// stdin). Known oracle quirks preserved: the JSONC // strip is naive (it
// eats URLs), trailing commas are stripped anywhere before } or ], and a
// mid-provider shape error returns the PARTIAL model list (the oracle's
// try/except wraps the whole loop).
package main
import (
"encoding/json"
"fmt"
"io"
"os"
"path/filepath"
"regexp"
"strings"
"time"
)
// moePatterns mirrors MOE_MODEL_PATTERNS.
@@ -128,74 +139,443 @@ func routeModel(taskType string, availableModels []string) routeResult {
return routeResult{Reason: "no-models-available"}
}
func main() {
if len(os.Args) > 1 && os.Args[1] != "replay" {
fmt.Fprintln(os.Stderr, "usage: model-connect [replay [truth.json]]")
os.Exit(2)
// ── gap-close surface (#2479 wave 4) ────────────────────────────────────────
// ctxProfile is the projection of model_registry's DEFAULT_PROFILES that
// resolve_context_budget reads (the oracle's _ctx_budget imports
// model_registry at call time; the values below are the oracle's registry).
type ctxProfile struct {
ID string
ContextWindow int
Local bool
}
var ctxBudgetProfiles = []ctxProfile{
{ID: "deterministic", ContextWindow: 0, Local: true}, // 0 → 8192 via the `or` floor
{ID: "qwen2.5:3b", ContextWindow: 32768, Local: true},
{ID: "qwen3:14b", ContextWindow: 32768, Local: true},
{ID: "qwen3.6:27b", ContextWindow: 32768, Local: true},
{ID: "mimo-v2.5-pro", ContextWindow: 128000, Local: false},
}
const (
ctxOutputReserve = 2048 // model_registry.OUTPUT_RESERVE["default"]
ctxToolResultReserve = 1024 // model_registry.TOOL_RESULT_RESERVE
ctxNumCtxFloor = 1024
ctxDefaultWindow = 8192
)
// CtxBudget mirrors _ctx_budget — the num_ctx projection of
// model_registry.resolve_context_budget with the default task kind:
// max(1024, window − output_reserve − tool_reserve), tool_reserve 0 for
// local models.
func CtxBudget(modelID string) int {
window, local := ctxDefaultWindow, true
for _, p := range ctxBudgetProfiles {
if p.ID == modelID {
window = p.ContextWindow
if window == 0 {
window = ctxDefaultWindow
}
local = p.Local
break
}
}
toolReserve := 0
if !local {
toolReserve = ctxToolResultReserve
}
numCtx := window - ctxOutputReserve - toolReserve
if numCtx < ctxNumCtxFloor {
numCtx = ctxNumCtxFloor
}
return numCtx
}
// nowClock is the injectable clock seam behind Now — the no-clock law: the
// battery and the live-parity sweep pin it (MCONN_CLOCK / frozen_now), the
// unpinned default reads the wall clock like the oracle's
// datetime.now(timezone.utc).
var nowClock = func() string {
if pinned := os.Getenv("MCONN_CLOCK"); pinned != "" {
return pinned
}
return pyIsoformat(time.Now().UTC())
}
// pyIsoformat renders t like Python datetime.isoformat(): +00:00 offset,
// microseconds only when nonzero.
func pyIsoformat(t time.Time) string {
if t.Nanosecond() == 0 {
return t.Format("2006-01-02T15:04:05+00:00")
}
return t.Format("2006-01-02T15:04:05.000000+00:00")
}
// Now mirrors _now — datetime.now(timezone.utc).isoformat().
func Now() string { return nowClock() }
// cloudModel mirrors one _get_cloud_models entry; field order matches the
// oracle's dict insertion order (name, size_gb, family, parameters,
// capabilities, tier, provider).
type cloudModel struct {
Name string `json:"name"`
SizeGB int `json:"size_gb"`
Family string `json:"family"`
Parameters string `json:"parameters"`
Capabilities []string `json:"capabilities"`
Tier string `json:"tier"`
Provider string `json:"provider"`
}
var cloudCapabilities = []string{"complex-reasoning", "coding", "analysis"}
// jsonObject is an insertion-ordered JSON object (Python dict semantics —
// provider and model iteration order is part of the oracle contract).
type jsonObject struct {
keys []string
values map[string]interface{}
}
func (o *jsonObject) get(key string) (interface{}, bool) {
v, ok := o.values[key]
return v, ok
}
// parseOrderedValue — json.Decoder token walk preserving object key order.
func parseOrderedValue(dec *json.Decoder) (interface{}, error) {
tok, err := dec.Token()
if err != nil {
return nil, err
}
return parseOrderedFrom(dec, tok)
}
func parseOrderedFrom(dec *json.Decoder, tok json.Token) (interface{}, error) {
delim, ok := tok.(json.Delim)
if !ok {
return tok, nil // nil / bool / float64 / string
}
switch delim {
case '{':
obj := &jsonObject{values: map[string]interface{}{}}
for dec.More() {
keyTok, err := dec.Token()
if err != nil {
return nil, err
}
key, ok := keyTok.(string)
if !ok {
return nil, fmt.Errorf("object key is not a string")
}
val, err := parseOrderedValue(dec)
if err != nil {
return nil, err
}
if _, dup := obj.values[key]; !dup {
obj.keys = append(obj.keys, key)
}
obj.values[key] = val
}
if _, err := dec.Token(); err != nil { // consume '}'
return nil, err
}
return obj, nil
case '[':
arr := []interface{}{}
for dec.More() {
val, err := parseOrderedValue(dec)
if err != nil {
return nil, err
}
arr = append(arr, val)
}
if _, err := dec.Token(); err != nil { // consume ']'
return nil, err
}
return arr, nil
}
return nil, fmt.Errorf("unexpected delimiter %v", delim)
}
var (
jsoncCommentRe = regexp.MustCompile(`(?m)//.*?$`)
jsoncTrailingCommaRe = regexp.MustCompile(`,\s*([}\]])`)
)
// stripJsonc mirrors the oracle's naive JSONC → JSON prep. The comment strip
// is deliberately string-blind: a "//" inside a quoted URL eats the rest of
// the line and the parse then fails — the oracle behaves identically.
func stripJsonc(raw string) string {
raw = jsoncCommentRe.ReplaceAllString(raw, "")
return jsoncTrailingCommaRe.ReplaceAllString(raw, "${1}")
}
// GetCloudModels mirrors _get_cloud_models — read ~/.config/mimocode/
// mimocode.jsonc, JSONC-strip, parse, and project every non-ollama provider
// model into the fixed cloud shape. Any error mid-loop returns the PARTIAL
// list accumulated so far (the oracle's try/except wraps the whole loop);
// read/parse errors return an empty list.
func GetCloudModels() []cloudModel {
out := []cloudModel{}
home, err := os.UserHomeDir()
if err != nil {
return out
}
raw, err := os.ReadFile(filepath.Join(home, ".config", "mimocode", "mimocode.jsonc"))
if err != nil {
return out
}
v, err := parseOrderedValue(json.NewDecoder(strings.NewReader(stripJsonc(string(raw)))))
if err != nil {
return out
}
cfg, ok := v.(*jsonObject)
if !ok {
return out // config.get on a non-dict raises in the oracle
}
providersV, ok := cfg.get("provider")
if !ok || providersV == nil {
return out
}
providers, ok := providersV.(*jsonObject)
if !ok {
return out // providers.items() raises in the oracle
}
for _, pname := range providers.keys {
if pname == "ollama" {
continue
}
pcfgV, ok := providers.values[pname]
if !ok || pcfgV == nil {
return out // provider_config.get raises in the oracle
}
pcfg, ok := pcfgV.(*jsonObject)
if !ok {
return out
}
modelsV, present := pcfg.get("models")
if !present {
continue // .get("models", {}) → empty dict, nothing to add
}
if modelsV == nil {
return out // None.items() raises in the oracle
}
models, ok := modelsV.(*jsonObject)
if !ok {
return out // models.items() raises in the oracle
}
for _, mid := range models.keys {
out = append(out, cloudModel{
Name: pname + "/" + mid,
SizeGB: 0,
Family: "cloud",
Parameters: "cloud",
Capabilities: cloudCapabilities,
Tier: "cloud",
Provider: pname,
})
}
}
return out
}
// hookResult mirrors handle_hook's return dict (insertion order:
// status, event, handler).
type hookResult struct {
Status string `json:"status"`
Event string `json:"event"`
Handler string `json:"handler"`
}
// HandleHook mirrors handle_hook — the constant ok envelope; data is
// accepted and ignored exactly like the oracle.
func HandleHook(event string, data interface{}) hookResult {
return hookResult{Status: "ok", Event: event, Handler: "model_connect"}
}
// ── replay (frozen truth) + parity (stdin op) entrypoints ───────────────────
func main() {
if len(os.Args) > 1 && os.Args[1] == "replay" {
os.Exit(replayMain())
}
os.Exit(parityMain())
}
type frozenCase struct {
Fn string `json:"fn"`
Args []interface{} `json:"args"`
Kwargs map[string]interface{} `json:"kwargs"`
Expected json.RawMessage `json:"expected"`
}
// canonical — round-trip a value through JSON so both sides compare as
// decoded data (Python None and Go nil decode to the same null).
func canonical(v interface{}) (interface{}, error) {
raw, err := json.Marshal(v)
if err != nil {
return nil, err
}
var out interface{}
if err := json.Unmarshal(raw, &out); err != nil {
return nil, err
}
return out, nil
}
func replayCase(c frozenCase, seedHome func(fixture string) string, missingHome func() string) (interface{}, error) {
switch c.Fn {
case "_clean_model_name":
return cleanModelName(c.Args[0].(string)), nil
case "is_moe_model":
return isMoeModel(c.Args[0].(string)), nil
case "route_model":
var models []string
b, _ := json.Marshal(c.Kwargs["available_models"])
if err := json.Unmarshal(b, &models); err != nil {
return nil, err
}
return routeModel(c.Args[0].(string), models), nil
case "_ctx_budget":
return CtxBudget(c.Args[0].(string)), nil
case "_now":
return Now(), nil
case "handle_hook":
return HandleHook(c.Args[0].(string), c.Args[1]), nil
case "_get_cloud_models_seeded":
return GetCloudModelsIn(seedHome(c.Args[0].(string))), nil
case "_get_cloud_models_missing":
return GetCloudModelsIn(missingHome()), nil
}
return nil, fmt.Errorf("unknown frozen fn %q", c.Fn)
}
// GetCloudModelsIn — GetCloudModels against an explicit HOME (the battery's
// seeded/missing rows rebuild the oracle's pinned-HOME rig).
func GetCloudModelsIn(home string) []cloudModel {
t := os.Getenv("HOME")
os.Setenv("HOME", home)
defer os.Setenv("HOME", t)
return GetCloudModels()
}
func replayMain() int {
path := "truth.json"
if len(os.Args) > 2 {
path = os.Args[2]
}
raw, err := os.ReadFile(path)
fatal := func(e error) { fmt.Fprintln(os.Stderr, "FATAL:", e); os.Exit(1) }
if err != nil {
fatal(err)
fmt.Fprintln(os.Stderr, "FATAL:", err)
return 1
}
var rows []struct {
Op string `json:"op"`
Input json.RawMessage `json:"input"`
Want json.RawMessage `json:"want"`
var env struct {
FrozenNow string `json:"frozen_now"`
Cases []frozenCase `json:"cases"`
}
if err := json.Unmarshal(raw, &rows); err != nil {
fatal(err)
if err := json.Unmarshal(raw, &env); err != nil {
fmt.Fprintln(os.Stderr, "FATAL:", err)
return 1
}
if len(env.Cases) == 0 {
fmt.Fprintln(os.Stderr, "FATAL: no truth cases")
return 1
}
savedClock := nowClock
os.Setenv("MCONN_CLOCK", env.FrozenNow)
nowClock = func() string { return env.FrozenNow }
defer func() {
nowClock = savedClock
os.Unsetenv("MCONN_CLOCK")
}()
seedHome := func(fixture string) string {
home, _ := os.MkdirTemp("", "mconn-replay-")
cfgDir := filepath.Join(home, ".config", "mimocode")
os.MkdirAll(cfgDir, 0o755)
os.WriteFile(filepath.Join(cfgDir, "mimocode.jsonc"), []byte(fixture), 0o644)
return home
}
missingHome := func() string {
home, _ := os.MkdirTemp("", "mconn-replay-")
return home
}
fails := 0
for _, r := range rows {
switch r.Op {
case "clean":
var in, want string
json.Unmarshal(r.Input, &in)
json.Unmarshal(r.Want, &want)
if got := cleanModelName(in); got != want {
fails++
fmt.Printf("FAIL clean %q: got %q want %q\n", in, got, want)
}
case "moe":
var in string
var want bool
json.Unmarshal(r.Input, &in)
json.Unmarshal(r.Want, &want)
if got := isMoeModel(in); got != want {
fails++
fmt.Printf("FAIL moe %q: got %v want %v\n", in, got, want)
}
case "route":
var in struct {
Task string `json:"task"`
Models []string `json:"models"`
}
json.Unmarshal(r.Input, &in)
wantRaw, _ := r.Want.MarshalJSON()
var wantMap map[string]interface{}
json.Unmarshal(wantRaw, &wantMap)
got := routeModel(in.Task, in.Models)
gotJSON, _ := json.Marshal(got)
var gotMap map[string]interface{}
json.Unmarshal(gotJSON, &gotMap)
if fmt.Sprint(gotMap["model"]) != fmt.Sprint(wantMap["model"]) ||
fmt.Sprint(gotMap["tier"]) != fmt.Sprint(wantMap["tier"]) ||
gotMap["reason"] != wantMap["reason"] {
fails++
fmt.Printf("FAIL route %s: got %s want %s\n", in.Task, gotJSON, wantRaw)
}
default:
for _, c := range env.Cases {
actual, err := replayCase(c, seedHome, missingHome)
if err != nil {
fails++
fmt.Printf("FAIL unknown op %s\n", r.Op)
fmt.Printf("FAIL %s: %v\n", c.Fn, err)
continue
}
got, err := canonical(actual)
if err != nil {
fails++
fmt.Printf("FAIL %s: canonical actual: %v\n", c.Fn, err)
continue
}
var want interface{}
if err := json.Unmarshal(c.Expected, &want); err != nil {
fails++
fmt.Printf("FAIL %s: decode expected: %v\n", c.Fn, err)
continue
}
gj, _ := json.Marshal(got)
wj, _ := json.Marshal(want)
if string(gj) != string(wj) {
fails++
fmt.Printf("FAIL %s: got %s want %s\n", c.Fn, gj, wj)
}
}
fmt.Printf("model-connect parity: %d/%d passed\n", len(rows)-fails, len(rows))
fmt.Printf("model-connect parity: %d/%d passed\n", len(env.Cases)-fails, len(env.Cases))
if fails > 0 {
os.Exit(1)
return 1
}
return 0
}
func parityMain() int {
var req struct {
Op string `json:"op"`
Name string `json:"name"`
Task string `json:"task"`
Model string `json:"model"`
Event string `json:"event"`
Data json.RawMessage `json:"data"`
Models []string `json:"models"`
}
body, err := io.ReadAll(os.Stdin)
if err != nil {
fmt.Fprintln(os.Stderr, "FATAL:", err)
return 1
}
if err := json.Unmarshal(body, &req); err != nil {
fmt.Fprintln(os.Stderr, "FATAL:", err)
return 1
}
var result interface{}
switch req.Op {
case "clean":
result = map[string]interface{}{"result": cleanModelName(req.Name)}
case "moe":
result = map[string]interface{}{"result": isMoeModel(req.Name)}
case "route":
result = routeModel(req.Task, req.Models)
case "ctx_budget":
result = map[string]interface{}{"result": CtxBudget(req.Model)}
case "now":
result = map[string]interface{}{"result": Now()}
case "cloud_models":
result = map[string]interface{}{"result": GetCloudModels()}
case "hook":
result = map[string]interface{}{"result": HandleHook(req.Event, req.Data)}
default:
fmt.Fprintln(os.Stderr, "FATAL: unknown op", req.Op)
return 1
}
out, _ := json.Marshal(result)
fmt.Println(string(out))
return 0
}
+481 -194
View File
@@ -1,194 +1,481 @@
[
{
"op": "clean",
"input": "hf.co/DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF:IQ2_M",
"want": "Qwen3.6 27B"
},
{
"op": "clean",
"input": "hf.co/org/Llama-3-8B-Instruct:latest",
"want": "Llama 3 8B"
},
{
"op": "clean",
"input": "hf.co/short/gemma-2-9b",
"want": "gemma 2 9B"
},
{
"op": "clean",
"input": "hf.co/user/nano-model",
"want": "nano"
},
{
"op": "clean",
"input": "hf.co/user/ab",
"want": "ab"
},
{
"op": "clean",
"input": "qwen2.5:3b",
"want": "qwen2.5:3b"
},
{
"op": "clean",
"input": "mimo-v2.5-pro",
"want": "mimo-v2.5-pro"
},
{
"op": "clean",
"input": "llava:7b",
"want": "llava:7b"
},
{
"op": "moe",
"input": "mixtral:8x7b",
"want": true
},
{
"op": "moe",
"input": "qwen2.5:3b",
"want": false
},
{
"op": "moe",
"input": "deepseek-moe-16b",
"want": true
},
{
"op": "moe",
"input": "My-Mixture-Model",
"want": true
},
{
"op": "moe",
"input": "expert-v1",
"want": true
},
{
"op": "moe",
"input": "4x22b-tiny",
"want": true
},
{
"op": "route",
"input": {
"task": "coding",
"models": [
"qwen2.5:3b",
"qwen2.5:7b",
"qwen3:14b",
"qwen2.5-coder:14b",
"hf.co/DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF:IQ2_M",
"mixtral:8x7b-instruct-v0.1-q4_0",
"qwen2.5vl:3b",
"llava:7b",
"qwen3-embedding:0.6b",
"nomic-embed-text:latest",
"mxbai-embed-large:latest"
]
},
"want": {
"model": "qwen2.5-coder:14b",
"tier": "local-coder",
"reason": "exact-match"
}
},
{
"op": "route",
"input": {
"task": "research",
"models": [
"qwen2.5:3b",
"qwen2.5:7b",
"qwen3:14b",
"qwen2.5-coder:14b",
"hf.co/DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF:IQ2_M",
"mixtral:8x7b-instruct-v0.1-q4_0",
"qwen2.5vl:3b",
"llava:7b",
"qwen3-embedding:0.6b",
"nomic-embed-text:latest",
"mxbai-embed-large:latest"
]
},
"want": {
"model": "qwen3:14b",
"tier": "local-large",
"reason": "exact-match"
}
},
{
"op": "route",
"input": {
"task": "conversation",
"models": [
"qwen2.5:3b",
"qwen2.5:7b",
"qwen3:14b",
"qwen2.5-coder:14b",
"hf.co/DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF:IQ2_M",
"mixtral:8x7b-instruct-v0.1-q4_0",
"qwen2.5vl:3b",
"llava:7b",
"qwen3-embedding:0.6b",
"nomic-embed-text:latest",
"mxbai-embed-large:latest"
]
},
"want": {
"model": "qwen2.5:3b",
"tier": "local-light",
"reason": "exact-match"
}
},
{
"op": "route",
"input": {
"task": "unknown-type",
"models": [
"qwen2.5:3b",
"qwen2.5:7b",
"qwen3:14b",
"qwen2.5-coder:14b",
"hf.co/DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF:IQ2_M",
"mixtral:8x7b-instruct-v0.1-q4_0",
"qwen2.5vl:3b",
"llava:7b",
"qwen3-embedding:0.6b",
"nomic-embed-text:latest",
"mxbai-embed-large:latest"
]
},
"want": {
"model": "qwen2.5:3b",
"tier": "local-light",
"reason": "exact-match"
}
},
{
"op": "route",
"input": {
"task": "coding",
"models": [
"mystery-model:x"
]
},
"want": {
"model": "mystery-model:x",
"tier": "fallback",
"reason": "no-exact-match"
}
},
{
"op": "route",
"input": {
"task": "coding",
"models": []
},
"want": {
"model": null,
"tier": null,
"reason": "no-models-available"
}
}
]
{
"frozen_now": "2026-10-14T08:00:00+00:00",
"cases": [
{
"fn": "_clean_model_name",
"args": [
"hf.co/DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF:IQ2_M"
],
"kwargs": {},
"expected": "Qwen3.6 27B"
},
{
"fn": "_clean_model_name",
"args": [
"hf.co/org/Llama-3-8B-Instruct:latest"
],
"kwargs": {},
"expected": "Llama 3 8B"
},
{
"fn": "_clean_model_name",
"args": [
"hf.co/short/gemma-2-9b"
],
"kwargs": {},
"expected": "gemma 2 9B"
},
{
"fn": "_clean_model_name",
"args": [
"hf.co/user/nano-model"
],
"kwargs": {},
"expected": "nano"
},
{
"fn": "_clean_model_name",
"args": [
"hf.co/user/ab"
],
"kwargs": {},
"expected": "ab"
},
{
"fn": "_clean_model_name",
"args": [
"qwen2.5:3b"
],
"kwargs": {},
"expected": "qwen2.5:3b"
},
{
"fn": "_clean_model_name",
"args": [
"mimo-v2.5-pro"
],
"kwargs": {},
"expected": "mimo-v2.5-pro"
},
{
"fn": "_clean_model_name",
"args": [
"llava:7b"
],
"kwargs": {},
"expected": "llava:7b"
},
{
"fn": "is_moe_model",
"args": [
"mixtral:8x7b"
],
"kwargs": {},
"expected": true
},
{
"fn": "is_moe_model",
"args": [
"qwen2.5:3b"
],
"kwargs": {},
"expected": false
},
{
"fn": "is_moe_model",
"args": [
"deepseek-moe-16b"
],
"kwargs": {},
"expected": true
},
{
"fn": "is_moe_model",
"args": [
"My-Mixture-Model"
],
"kwargs": {},
"expected": true
},
{
"fn": "is_moe_model",
"args": [
"expert-v1"
],
"kwargs": {},
"expected": true
},
{
"fn": "is_moe_model",
"args": [
"4x22b-tiny"
],
"kwargs": {},
"expected": true
},
{
"fn": "is_moe_model",
"args": [
"plain-conversational"
],
"kwargs": {},
"expected": false
},
{
"fn": "is_moe_model",
"args": [
"GPT-XL"
],
"kwargs": {},
"expected": false
},
{
"fn": "route_model",
"args": [
"coding"
],
"kwargs": {
"available_models": [
"qwen2.5:3b",
"qwen2.5:7b",
"qwen3:14b",
"qwen2.5-coder:14b",
"hf.co/DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF:IQ2_M",
"mixtral:8x7b-instruct-v0.1-q4_0",
"qwen2.5vl:3b",
"llava:7b",
"qwen3-embedding:0.6b",
"nomic-embed-text:latest",
"mxbai-embed-large:latest"
]
},
"expected": {
"model": "qwen2.5-coder:14b",
"tier": "local-coder",
"reason": "exact-match"
}
},
{
"fn": "route_model",
"args": [
"research"
],
"kwargs": {
"available_models": [
"qwen2.5:3b",
"qwen2.5:7b",
"qwen3:14b",
"qwen2.5-coder:14b",
"hf.co/DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF:IQ2_M",
"mixtral:8x7b-instruct-v0.1-q4_0",
"qwen2.5vl:3b",
"llava:7b",
"qwen3-embedding:0.6b",
"nomic-embed-text:latest",
"mxbai-embed-large:latest"
]
},
"expected": {
"model": "qwen3:14b",
"tier": "local-large",
"reason": "exact-match"
}
},
{
"fn": "route_model",
"args": [
"conversation"
],
"kwargs": {
"available_models": [
"qwen2.5:3b",
"qwen2.5:7b",
"qwen3:14b",
"qwen2.5-coder:14b",
"hf.co/DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF:IQ2_M",
"mixtral:8x7b-instruct-v0.1-q4_0",
"qwen2.5vl:3b",
"llava:7b",
"qwen3-embedding:0.6b",
"nomic-embed-text:latest",
"mxbai-embed-large:latest"
]
},
"expected": {
"model": "qwen2.5:3b",
"tier": "local-light",
"reason": "exact-match"
}
},
{
"fn": "route_model",
"args": [
"unknown-type"
],
"kwargs": {
"available_models": [
"qwen2.5:3b",
"qwen2.5:7b",
"qwen3:14b",
"qwen2.5-coder:14b",
"hf.co/DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF:IQ2_M",
"mixtral:8x7b-instruct-v0.1-q4_0",
"qwen2.5vl:3b",
"llava:7b",
"qwen3-embedding:0.6b",
"nomic-embed-text:latest",
"mxbai-embed-large:latest"
]
},
"expected": {
"model": "qwen2.5:3b",
"tier": "local-light",
"reason": "exact-match"
}
},
{
"fn": "route_model",
"args": [
"coding"
],
"kwargs": {
"available_models": [
"mystery-model:x"
]
},
"expected": {
"model": "mystery-model:x",
"tier": "fallback",
"reason": "no-exact-match"
}
},
{
"fn": "route_model",
"args": [
"coding"
],
"kwargs": {
"available_models": []
},
"expected": {
"model": null,
"tier": null,
"reason": "no-models-available"
}
},
{
"fn": "_ctx_budget",
"args": [
"deterministic"
],
"kwargs": {},
"expected": 6144
},
{
"fn": "_ctx_budget",
"args": [
"qwen2.5:3b"
],
"kwargs": {},
"expected": 30720
},
{
"fn": "_ctx_budget",
"args": [
"qwen3:14b"
],
"kwargs": {},
"expected": 30720
},
{
"fn": "_ctx_budget",
"args": [
"qwen3.6:27b"
],
"kwargs": {},
"expected": 30720
},
{
"fn": "_ctx_budget",
"args": [
"mimo-v2.5-pro"
],
"kwargs": {},
"expected": 124928
},
{
"fn": "_ctx_budget",
"args": [
"totally-unknown-model"
],
"kwargs": {},
"expected": 6144
},
{
"fn": "_ctx_budget",
"args": [
""
],
"kwargs": {},
"expected": 6144
},
{
"fn": "_now",
"args": [],
"kwargs": {},
"expected": "2026-10-14T08:00:00+00:00"
},
{
"fn": "handle_hook",
"args": [
"pre_tool_use",
{
"tool": "bash",
"dry": false
}
],
"kwargs": {},
"expected": {
"status": "ok",
"event": "pre_tool_use",
"handler": "model_connect"
}
},
{
"fn": "handle_hook",
"args": [
"",
null
],
"kwargs": {},
"expected": {
"status": "ok",
"event": "",
"handler": "model_connect"
}
},
{
"fn": "handle_hook",
"args": [
"post_response",
{
"ok": true
}
],
"kwargs": {},
"expected": {
"status": "ok",
"event": "post_response",
"handler": "model_connect"
}
},
{
"fn": "_get_cloud_models_seeded",
"args": [
"{\n // MiMoCode config — JSONC: comments and trailing commas allowed\n \"provider\": {\n \"ollama\": { // skipped: local models handled by list_models\n \"models\": {\n \"qwen3:14b\": {},\n },\n },\n \"openai\": {\n \"models\": {\n \"gpt-5.2\": {\"contextLength\": 400000},\n \"o4-mini\": \"not-even-a-dict\", // model_config is unused by the oracle\n },\n },\n \"kimi\": {\n \"models\": {\n \"kimi-for-coding\": {},\n },\n },\n },\n}\n"
],
"kwargs": {},
"expected": [
{
"name": "openai/gpt-5.2",
"size_gb": 0,
"family": "cloud",
"parameters": "cloud",
"capabilities": [
"complex-reasoning",
"coding",
"analysis"
],
"tier": "cloud",
"provider": "openai"
},
{
"name": "openai/o4-mini",
"size_gb": 0,
"family": "cloud",
"parameters": "cloud",
"capabilities": [
"complex-reasoning",
"coding",
"analysis"
],
"tier": "cloud",
"provider": "openai"
},
{
"name": "kimi/kimi-for-coding",
"size_gb": 0,
"family": "cloud",
"parameters": "cloud",
"capabilities": [
"complex-reasoning",
"coding",
"analysis"
],
"tier": "cloud",
"provider": "kimi"
}
]
},
{
"fn": "_get_cloud_models_seeded",
"args": [
"{\n \"provider\": {\n \"ollama\": {\n \"models\": {\"qwen2.5:3b\": {}},\n },\n },\n}\n"
],
"kwargs": {},
"expected": []
},
{
"fn": "_get_cloud_models_seeded",
"args": [
"{\n \"provider\": {\n \"anthropic\": {\n \"apiKey\": \"sk-test\",\n },\n },\n}\n"
],
"kwargs": {},
"expected": []
},
{
"fn": "_get_cloud_models_seeded",
"args": [
"{\n // no providers configured\n \"theme\": \"dark\",\n}\n"
],
"kwargs": {},
"expected": []
},
{
"fn": "_get_cloud_models_seeded",
"args": [
"{\n \"provider\": {\n \"a\": {\"models\": {\"m1\": {}}},\n \"b\": {\"models\": \"not-a-dict\"},\n },\n}\n"
],
"kwargs": {},
"expected": [
{
"name": "a/m1",
"size_gb": 0,
"family": "cloud",
"parameters": "cloud",
"capabilities": [
"complex-reasoning",
"coding",
"analysis"
],
"tier": "cloud",
"provider": "a"
}
]
},
{
"fn": "_get_cloud_models_seeded",
"args": [
"{\n \"provider\": {\n \"gateway\": {\n \"baseUrl\": \"http://api.example.com/v1\",\n \"models\": {\"gw-model\": {}},\n },\n },\n}\n"
],
"kwargs": {},
"expected": []
},
{
"fn": "_get_cloud_models_missing",
"args": [],
"kwargs": {},
"expected": []
}
]
}
+115
View File
@@ -0,0 +1,115 @@
package main
// truth_test.go — the parity gate. Replays every case from truth.json (the
// shared freeze_truth.py envelope: pinned clock + pinned HOME + Python
// oracle cases WITH inputs, generated by scripts/gen_model_connect_truth.py
// against the live oracle under scripts/mconn_freeze_seams) VERBATIM through
// the ported Go functions. A divergence fails the port, not the oracle.
import (
_ "embed"
"encoding/json"
"os"
"path/filepath"
"strings"
"testing"
)
//go:embed truth.json
var frozenTruthJSON string
// TestFrozenTruthReplay — the frozen oracle truth, verbatim, in the recorded
// call order, against the ported functions.
func TestFrozenTruthReplay(t *testing.T) {
var env struct {
FrozenNow string `json:"frozen_now"`
Cases []frozenCase `json:"cases"`
}
if err := json.Unmarshal([]byte(frozenTruthJSON), &env); err != nil {
t.Fatalf("decode truth envelope: %v", err)
}
if len(env.Cases) == 0 {
t.Fatal("no truth cases — the parity gate is empty")
}
savedClock := nowClock
t.Setenv("MCONN_CLOCK", env.FrozenNow)
nowClock = func() string { return env.FrozenNow }
t.Cleanup(func() { nowClock = savedClock })
seedHome := func(fixture string) string {
home := t.TempDir()
cfgDir := filepath.Join(home, ".config", "mimocode")
if err := os.MkdirAll(cfgDir, 0o755); err != nil {
t.Fatalf("seed home: %v", err)
}
if err := os.WriteFile(filepath.Join(cfgDir, "mimocode.jsonc"), []byte(fixture), 0o644); err != nil {
t.Fatalf("seed config: %v", err)
}
return home
}
missingHome := func() string { return t.TempDir() }
for i, c := range env.Cases {
actual, err := replayCase(c, seedHome, missingHome)
if err != nil {
t.Fatalf("case %d fn=%s: %v", i, c.Fn, err)
}
got, err := canonical(actual)
if err != nil {
t.Fatalf("case %d fn=%s: canonical actual: %v", i, c.Fn, err)
}
var want interface{}
if err := json.Unmarshal(c.Expected, &want); err != nil {
t.Fatalf("case %d fn=%s: decode expected: %v", i, c.Fn, err)
}
gj, _ := json.Marshal(got)
wj, _ := json.Marshal(want)
if string(gj) != string(wj) {
t.Errorf("case %d fn=%s: got %s want %s", i, c.Fn, gj, wj)
}
}
}
// TestStripJsoncPreservesOracleQuirks — the naive JSONC strip: string-blind
// comment removal (URLs break the parse downstream) and trailing commas.
func TestStripJsoncPreservesOracleQuirks(t *testing.T) {
stripped := stripJsonc("{\n // comment\n \"a\": 1, // inline\n \"b\": [1, 2,],\n}\n")
want := "{\n \n \"a\": 1, \n \"b\": [1, 2]}\n"
if stripped != want {
t.Errorf("stripJsonc:\n got %q\nwant %q", stripped, want)
}
if _, err := parseOrderedValue(jsonDecoder("{ \"u\": \"http://x\" }")); err == nil {
// sanity: plain JSON parses
} else {
t.Fatalf("plain JSON must parse: %v", err)
}
// the oracle quirk: a URL inside JSONC loses its closing quote
broken := stripJsonc("{ \"u\": \"http://x\" }")
if _, err := parseOrderedValue(jsonDecoder(broken)); err == nil {
t.Errorf("URL-under-JSONC must fail to parse like the oracle, got %q", broken)
}
}
// TestGetCloudModelsPartialOnLateShapeError — a mid-provider shape error
// returns the PARTIAL accumulated list (the oracle's try/except wraps the
// whole loop), not an empty one.
func TestGetCloudModelsPartialOnLateShapeError(t *testing.T) {
home := t.TempDir()
cfgDir := filepath.Join(home, ".config", "mimocode")
if err := os.MkdirAll(cfgDir, 0o755); err != nil {
t.Fatal(err)
}
fixture := "{\n \"provider\": {\n \"a\": {\"models\": {\"m1\": {}}},\n \"b\": {\"models\": \"not-a-dict\"}\n }\n}\n"
if err := os.WriteFile(filepath.Join(cfgDir, "mimocode.jsonc"), []byte(fixture), 0o644); err != nil {
t.Fatal(err)
}
got := GetCloudModelsIn(home)
if len(got) != 1 || got[0].Name != "a/m1" {
t.Errorf("partial cloud list: got %+v want [a/m1]", got)
}
}
func jsonDecoder(s string) *json.Decoder {
return json.NewDecoder(strings.NewReader(s))
}
+1
View File
@@ -0,0 +1 @@
exit:0
+1
View File
@@ -0,0 +1 @@
exit:0
+1
View File
@@ -0,0 +1 @@
exit:0
+1
View File
@@ -0,0 +1 @@
exit:0
+1
View File
@@ -0,0 +1 @@
exit:0
+1
View File
@@ -0,0 +1 @@
exit:0
+1
View File
@@ -0,0 +1 @@
exit:0
+1
View File
@@ -0,0 +1 @@
exit:0
+1
View File
@@ -0,0 +1 @@
exit:0
+1
View File
@@ -0,0 +1 @@
exit:0
+62
View File
@@ -0,0 +1,62 @@
# model_connect dichotomy gap-close — verification summary
- **Module**: model_connect (GO port, src/go/model-connect)
- **Worktree**: /mnt/workdrive/rune-wave-cl (gitea/master base 64cc5125)
- **Parity SHA**: see SHAs below (feat commit; filled at push time)
## Coverage
Dichotomy audit: model_connect **7/11 PARTIAL → 11/11 FULL** (GO).
Ported name-preserving: `CtxBudget` (_ctx_budget), `Now` (_now),
`GetCloudModels` (_get_cloud_models), `HandleHook` (handle_hook).
Oracle quirks preserved (all frozen on the oracle shape):
- JSONC `//` strip is string-blind — a URL inside the config loses its
closing quote and the parse fails → `[]` (frozen row `url_corruption`)
- mid-provider shape error returns the PARTIAL accumulated list (the
oracle's try/except wraps the whole loop; frozen row `broken_late` → `[a/m1]`)
- provider/model iteration order is dict insertion order — Go parses with
a json.Decoder token walk preserving key order
- `null` models dict → `None.items()` raises → partial/empty return
- `_ctx_budget` constants from the LIVE model_registry oracle:
OUTPUT_RESERVE["default"]=2048, TOOL_RESULT_RESERVE=1024 (non-local only),
num_ctx floor 1024, 0-window → 8192 (`or` floor)
- `_now` behind the injectable clock seam (no-clock law); battery/parity
pin it (MCONN_CLOCK / frozen_now), unpinned default reads wall clock
## Commands and exit codes
| Check | Command | Exit | Evidence |
|---|---|---|---|
| truth generation | `PYTHONPATH=scripts/mconn_freeze_seams:src python3 scripts/gen_model_connect_truth.py` | 0 | `mconn-truth-gen.log/.exit` — 40 cases, `_now` pinned row asserted |
| frozen truth byte-identical | `PYTHONPATH=scripts/mconn_freeze_seams:src python3 scripts/freeze_truth.py --verify src/go/model-connect/truth.json --module rune.model_connect` | 0 | `mconn-freeze-verify.log/.exit` — OK: 40 cases byte-identical against live oracle |
| gofmt / vet / build | `gofmt -l . && go vet ./... && go build ./...` (src/go/model-connect) | 0 | `mconn-gofmt.log` (empty), `mconn-govet.log`, `mconn-gobuild.log` + `.exit` |
| Go battery | `go test -count=1 -v ./...` (src/go/model-connect) | 0 | `mconn-gotest.log/.exit` — 3/3 PASS (TestFrozenTruthReplay verbatim truth.json embed, TestStripJsoncPreservesOracleQuirks, TestGetCloudModelsPartialOnLateShapeError) |
| CI replay binary | `./model-connect replay truth.json` | 0 | `mconn-replay.log/.exit` — 40/40 passed |
| live parity | `python3 scripts/check_model_connect_parity.py` | 0 | `mconn-parity.log/.exit` — parity: 44/44 match |
| Python smoke (live) | `python3 /tmp/mconn-smoke.py` | 0 | `mconn-pysmoke-live.log/.exit` — all 7 ported fns healthy |
| Python smoke (python pin) | `RUNE_MODEL_CONNECT_CORE=python python3 /tmp/mconn-smoke.py` | 0 | `mconn-pysmoke-python.log/.exit` — healthy (module has no loader seam; pin vacuous, run per protocol) |
| audit | `dichotomy_audit.py` row capture | 0 | `mconn-audit.log/.exit` — 11/11 matched, status FULL |
Note: model_connect has no dedicated pytest suite and no
RUNE_MODEL_CONNECT_CORE loader (plain plugin module — verified by grep);
"Python suite green in BOTH pins" is satisfied by the oracle smoke above
plus the byte-identity gate, which replays the live oracle itself.
## Truth rig
- `scripts/mconn_freeze_seams/sitecustomize.py` — pin_home (hermetic HOME)
+ binds `_get_cloud_models_seeded` / `_get_cloud_models_missing` seams
onto the oracle module (fixture text rides in the truth rows; the Go
battery rebuilds the same rig from the case args via t.TempDir).
- Envelope: frozen_now `2026-10-14T08:00:00+00:00`, 40 cases:
8 clean / 8 moe / 6 route / 7 ctx_budget / 1 now / 3 hook / 6 seeded
cloud configs / 1 missing config.
## Files
- `src/go/model-connect/main.go` — 4 gap fns + envelope replay + stdin op
dispatch for the parity sweep
- `src/go/model-connect/truth_test.go` — battery (verbatim truth embed)
- `src/go/model-connect/truth.json` — 40-case freeze_truth envelope
- `scripts/gen_model_connect_truth.py`, `scripts/check_model_connect_parity.py`,
`scripts/mconn_freeze_seams/sitecustomize.py`
+1
View File
@@ -0,0 +1 @@
exit:0