feat(model_connect): close PARTIAL gaps — dichotomy wave
Oracle src/rune/model_connect.py 471 lines / 11 fns: 7/11 -> 11/11 per scripts/dichotomy_audit.py (name-preserving match). The four audit-named gaps land as ported Go functions: CtxBudget (_ctx_budget — vendored model_registry.resolve_context_budget num_ctx kernel with the live oracle's constants), Now (_now — behind the injectable clock seam), GetCloudModels (_get_cloud_models — ordered-JSON JSONC reader with the oracle's naive quirks), HandleHook (handle_hook). Oracle quirks preserved and frozen as truth rows: string-blind JSONC // strip (URLs corrupt the parse -> []), trailing-comma strip, partial cloud list on mid-provider shape errors (try/except wraps the whole loop), insertion-order provider/model iteration (json.Decoder token walk), null models dict -> items() raises, OUTPUT_RESERVE['default']/ TOOL_RESULT_RESERVE=2048/1024 from the live registry oracle. Truth: scripts/gen_model_connect_truth.py via the shared freeze_truth.py harness (40-case envelope, pinned clock 2026-10-14T08:00:00+00:00 + pinned HOME via scripts/mconn_freeze_seams); freeze_truth --verify byte-identical; truth.json embedded VERBATIM in truth_test.go TestFrozenTruthReplay (3/3). Live parity 44/44 (scripts/check_model_connect_parity.py). gofmt/vet/go build clean. Python oracle smoke green both pins (module has no loader seam; the byte-identity gate replays the live oracle itself).
This commit is contained in:
1 parent
70f8775838
commit
290ed7341e
18 files changed
+1543
-255
No files matched your search
@@ -0,0 +1,197 @@
|
||||
#!/usr/bin/env python3
|
||||
"""check_model_connect_parity.py — live oracle-vs-Go parity sweep for the
|
||||
model_connect gap closure (#2479, contract factory-gap-mconn).
|
||||
|
||||
The frozen envelope (truth.json) pins the ported surface byte-identically.
|
||||
THIS script drives the LIVE Python oracle and the Go binary (stdin op
|
||||
dispatch) across every op family — clean, moe, route, ctx_budget, now
|
||||
(clock pinned on both sides), hook, and the cloud-model config sweep
|
||||
(per-fixture hermetic HOME shared by oracle and binary) — comparing
|
||||
sort_keys JSON. Exit 0 on 100% parity, 1 with the first divergences.
|
||||
|
||||
Usage: python3 scripts/check_model_connect_parity.py
|
||||
[--bin src/go/model-connect/model-connect]
|
||||
"""
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
REPO = Path(__file__).resolve().parent.parent
|
||||
sys.path.insert(0, str(REPO / "src"))
|
||||
|
||||
import rune.model_connect as mc # noqa: E402
|
||||
|
||||
FROZEN_NOW = "2026-10-14T08:00:00+00:00"
|
||||
|
||||
CLEAN = [
|
||||
"hf.co/DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF:IQ2_M",
|
||||
"hf.co/org/Llama-3-8B-Instruct:latest",
|
||||
"hf.co/short/gemma-2-9b",
|
||||
"hf.co/user/nano-model",
|
||||
"hf.co/user/ab",
|
||||
"qwen2.5:3b",
|
||||
"mimo-v2.5-pro",
|
||||
"llava:7b",
|
||||
"huggingface.co/x/y/Llama-3-8B",
|
||||
]
|
||||
|
||||
MOE = [
|
||||
"mixtral:8x7b", "qwen2.5:3b", "deepseek-moe-16b", "My-Mixture-Model",
|
||||
"expert-v1", "4x22b-tiny", "plain-conversational", "GPT-XL",
|
||||
]
|
||||
|
||||
MODELS_ALL = [
|
||||
"qwen2.5:3b", "qwen2.5:7b", "qwen3:14b", "qwen2.5-coder:14b",
|
||||
"hf.co/DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF:IQ2_M",
|
||||
"mixtral:8x7b-instruct-v0.1-q4_0", "qwen2.5vl:3b", "llava:7b",
|
||||
"qwen3-embedding:0.6b", "nomic-embed-text:latest", "mxbai-embed-large:latest",
|
||||
]
|
||||
|
||||
ROUTE = [
|
||||
("coding", MODELS_ALL), ("research", MODELS_ALL), ("conversation", MODELS_ALL),
|
||||
("unrestricted", MODELS_ALL), ("unknown-type", MODELS_ALL),
|
||||
("coding", ["mystery-model:x"]), ("coding", []),
|
||||
]
|
||||
|
||||
CTX = ["deterministic", "qwen2.5:3b", "qwen3:14b", "qwen3.6:27b",
|
||||
"mimo-v2.5-pro", "totally-unknown-model", ""]
|
||||
|
||||
HOOKS = [
|
||||
("pre_tool_use", {"tool": "bash", "dry": False}),
|
||||
("", None),
|
||||
("post_response", {"ok": True}),
|
||||
("unicode-évent", {}),
|
||||
]
|
||||
|
||||
FIXTURES = {
|
||||
"main": """{
|
||||
// MiMoCode config — JSONC: comments and trailing commas allowed
|
||||
"provider": {
|
||||
"ollama": { // skipped: local models handled by list_models
|
||||
"models": {
|
||||
"qwen3:14b": {},
|
||||
},
|
||||
},
|
||||
"openai": {
|
||||
"models": {
|
||||
"gpt-5.2": {"contextLength": 400000},
|
||||
"o4-mini": "not-even-a-dict",
|
||||
},
|
||||
},
|
||||
"kimi": {
|
||||
"models": {
|
||||
"kimi-for-coding": {},
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
""",
|
||||
"only_ollama": '{\n "provider": {\n "ollama": {"models": {"qwen2.5:3b": {}}},\n },\n}\n',
|
||||
"no_models_key": '{\n "provider": {\n "anthropic": {"apiKey": "sk-test"},\n },\n}\n',
|
||||
"no_provider_key": '{\n "theme": "dark",\n}\n',
|
||||
"broken_late": '{\n "provider": {\n "a": {"models": {"m1": {}}},\n "b": {"models": "not-a-dict"},\n },\n}\n',
|
||||
"null_models": '{\n "provider": {\n "z": {"models": null},\n },\n}\n',
|
||||
"url_corruption": '{\n "provider": {\n "gateway": {\n "baseUrl": "http://api.example.com/v1",\n "models": {"gw-model": {}},\n },\n },\n}\n',
|
||||
}
|
||||
|
||||
|
||||
def run_go(binpath, req, env=None):
|
||||
e = dict(os.environ)
|
||||
if env:
|
||||
e.update(env)
|
||||
p = subprocess.run([str(binpath)], input=json.dumps(req), capture_output=True,
|
||||
text=True, env=e)
|
||||
if p.returncode != 0:
|
||||
raise RuntimeError(f"go op {req.get('op')} exited {p.returncode}: {p.stdout} {p.stderr}")
|
||||
return json.loads(p.stdout)
|
||||
|
||||
|
||||
def frozen_clock():
|
||||
base = datetime.fromisoformat(FROZEN_NOW)
|
||||
|
||||
class FrozenDatetime(datetime):
|
||||
@classmethod
|
||||
def now(cls, tz=None):
|
||||
return base if tz is not None else base.replace(tzinfo=None)
|
||||
|
||||
return FrozenDatetime
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("--bin", default=str(REPO / "src/go/model-connect/model-connect"))
|
||||
args = ap.parse_args()
|
||||
binpath = Path(args.bin)
|
||||
if not binpath.exists():
|
||||
subprocess.run(["go", "build", "-o", str(binpath), str(binpath.parent)],
|
||||
check=True, cwd=binpath.parent)
|
||||
|
||||
fails = total = 0
|
||||
|
||||
def check(name, oracle_val, go_val):
|
||||
nonlocal fails, total
|
||||
total += 1
|
||||
if json.dumps(oracle_val, sort_keys=True) != json.dumps(go_val, sort_keys=True):
|
||||
fails += 1
|
||||
print(f"DIVERGE {name}:\n oracle: {json.dumps(oracle_val)[:300]}\n go: {json.dumps(go_val)[:300]}")
|
||||
|
||||
for name in CLEAN:
|
||||
g = run_go(binpath, {"op": "clean", "name": name})
|
||||
check(f"clean:{name[:40]}", {"result": mc._clean_model_name(name)}, g)
|
||||
|
||||
for name in MOE:
|
||||
g = run_go(binpath, {"op": "moe", "name": name})
|
||||
check(f"moe:{name}", {"result": mc.is_moe_model(name)}, g)
|
||||
|
||||
for task, models in ROUTE:
|
||||
g = run_go(binpath, {"op": "route", "task": task, "models": models})
|
||||
check(f"route:{task}", mc.route_model(task, models), g)
|
||||
|
||||
for mid in CTX:
|
||||
g = run_go(binpath, {"op": "ctx_budget", "model": mid})
|
||||
check(f"ctx_budget:{mid}", {"result": mc._ctx_budget(mid)}, g)
|
||||
|
||||
saved_dt = mc.datetime
|
||||
mc.datetime = frozen_clock()
|
||||
try:
|
||||
g = run_go(binpath, {"op": "now"}, env={"MCONN_CLOCK": FROZEN_NOW})
|
||||
check("now", {"result": mc._now()}, g)
|
||||
finally:
|
||||
mc.datetime = saved_dt
|
||||
|
||||
for event, data in HOOKS:
|
||||
g = run_go(binpath, {"op": "hook", "event": event, "data": data})
|
||||
check(f"hook:{event}", {"result": mc.handle_hook(event, data)}, g)
|
||||
|
||||
saved_home = os.environ.get("HOME")
|
||||
try:
|
||||
for label, fixture in FIXTURES.items():
|
||||
with tempfile.TemporaryDirectory() as td:
|
||||
cfg = os.path.join(td, ".config", "mimocode")
|
||||
os.makedirs(cfg)
|
||||
with open(os.path.join(cfg, "mimocode.jsonc"), "w") as f:
|
||||
f.write(fixture)
|
||||
os.environ["HOME"] = td
|
||||
oracle_val = mc._get_cloud_models()
|
||||
g = run_go(binpath, {"op": "cloud_models"}, env={"HOME": td})
|
||||
check(f"cloud:{label}", {"result": oracle_val}, g)
|
||||
with tempfile.TemporaryDirectory() as td:
|
||||
os.environ["HOME"] = td
|
||||
oracle_val = mc._get_cloud_models()
|
||||
g = run_go(binpath, {"op": "cloud_models"}, env={"HOME": td})
|
||||
check("cloud:missing", {"result": oracle_val}, g)
|
||||
finally:
|
||||
if saved_home is not None:
|
||||
os.environ["HOME"] = saved_home
|
||||
|
||||
print(f"parity: {total - fails}/{total} match")
|
||||
return 1 if fails else 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,177 @@
|
||||
#!/usr/bin/env python3
|
||||
"""gen_model_connect_truth.py — freeze the model_connect gap-close truth
|
||||
envelope via the shared freeze_truth.py harness (#2479 wave 4).
|
||||
|
||||
Covers the full ported surface, exercising every branch:
|
||||
* _clean_model_name — HF repo paths (family+size / family+version+size /
|
||||
short-part / no-match truncation) and clean names (passthrough)
|
||||
* is_moe_model — every MOE pattern family + negatives
|
||||
* route_model — exact matches per tier, unknown task type, fallback,
|
||||
empty list (null model/tier)
|
||||
* _ctx_budget — every DEFAULT_PROFILES id (incl. deterministic's 0-window
|
||||
→ 8192 floor), the cloud (non-local) tool reserve, unknown + empty ids
|
||||
* _now — the pinned-clock isoformat
|
||||
* handle_hook — constant envelope over varied events/data
|
||||
* _get_cloud_models (seeded/missing seams) — JSONC comments + trailing
|
||||
commas, ollama skip, multi-provider ordering, provider without models,
|
||||
config without provider, the PARTIAL-return mid-loop shape error, the
|
||||
naive comment-strip corrupting URLs, and the missing-file branch
|
||||
|
||||
Usage (from the repo root; the seams dir is mandatory — it pins HOME and
|
||||
binds the cloud seams):
|
||||
PYTHONPATH=scripts/mconn_freeze_seams:src python3 scripts/gen_model_connect_truth.py
|
||||
"""
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
REPO = Path(os.path.abspath(__file__)).resolve().parents[1]
|
||||
sys.path.insert(0, str(REPO / "src"))
|
||||
sys.path.insert(0, str(REPO / "scripts"))
|
||||
|
||||
FROZEN_NOW = "2026-10-14T08:00:00+00:00" # microsecond=0: isoformat == the pin
|
||||
os.environ["RUNE_FROZEN_NOW"] = FROZEN_NOW
|
||||
|
||||
import freeze_truth # noqa: E402
|
||||
|
||||
MODELS_ALL = [
|
||||
"qwen2.5:3b",
|
||||
"qwen2.5:7b",
|
||||
"qwen3:14b",
|
||||
"qwen2.5-coder:14b",
|
||||
"hf.co/DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF:IQ2_M",
|
||||
"mixtral:8x7b-instruct-v0.1-q4_0",
|
||||
"qwen2.5vl:3b",
|
||||
"llava:7b",
|
||||
"qwen3-embedding:0.6b",
|
||||
"nomic-embed-text:latest",
|
||||
"mxbai-embed-large:latest",
|
||||
]
|
||||
|
||||
FIXTURE_MAIN = """{
|
||||
// MiMoCode config — JSONC: comments and trailing commas allowed
|
||||
"provider": {
|
||||
"ollama": { // skipped: local models handled by list_models
|
||||
"models": {
|
||||
"qwen3:14b": {},
|
||||
},
|
||||
},
|
||||
"openai": {
|
||||
"models": {
|
||||
"gpt-5.2": {"contextLength": 400000},
|
||||
"o4-mini": "not-even-a-dict", // model_config is unused by the oracle
|
||||
},
|
||||
},
|
||||
"kimi": {
|
||||
"models": {
|
||||
"kimi-for-coding": {},
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
"""
|
||||
|
||||
FIXTURE_ONLY_OLLAMA = """{
|
||||
"provider": {
|
||||
"ollama": {
|
||||
"models": {"qwen2.5:3b": {}},
|
||||
},
|
||||
},
|
||||
}
|
||||
"""
|
||||
|
||||
FIXTURE_NO_MODELS_KEY = """{
|
||||
"provider": {
|
||||
"anthropic": {
|
||||
"apiKey": "sk-test",
|
||||
},
|
||||
},
|
||||
}
|
||||
"""
|
||||
|
||||
FIXTURE_NO_PROVIDER_KEY = """{
|
||||
// no providers configured
|
||||
"theme": "dark",
|
||||
}
|
||||
"""
|
||||
|
||||
FIXTURE_BROKEN_LATE = """{
|
||||
"provider": {
|
||||
"a": {"models": {"m1": {}}},
|
||||
"b": {"models": "not-a-dict"},
|
||||
},
|
||||
}
|
||||
"""
|
||||
|
||||
FIXTURE_URL_CORRUPTION = """{
|
||||
"provider": {
|
||||
"gateway": {
|
||||
"baseUrl": "http://api.example.com/v1",
|
||||
"models": {"gw-model": {}},
|
||||
},
|
||||
},
|
||||
}
|
||||
"""
|
||||
|
||||
cases = []
|
||||
# _clean_model_name
|
||||
for name in [
|
||||
"hf.co/DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF:IQ2_M",
|
||||
"hf.co/org/Llama-3-8B-Instruct:latest",
|
||||
"hf.co/short/gemma-2-9b",
|
||||
"hf.co/user/nano-model",
|
||||
"hf.co/user/ab",
|
||||
"qwen2.5:3b",
|
||||
"mimo-v2.5-pro",
|
||||
"llava:7b",
|
||||
]:
|
||||
cases.append(("_clean_model_name", (name,), {}))
|
||||
# is_moe_model
|
||||
for name, want in [
|
||||
("mixtral:8x7b", True),
|
||||
("qwen2.5:3b", False),
|
||||
("deepseek-moe-16b", True),
|
||||
("My-Mixture-Model", True),
|
||||
("expert-v1", True),
|
||||
("4x22b-tiny", True),
|
||||
("plain-conversational", False),
|
||||
("GPT-XL", False),
|
||||
]:
|
||||
cases.append(("is_moe_model", (name,), {}))
|
||||
# route_model (available_models always explicit — the None default would
|
||||
# fall through to the networked list_models, which is reactive-side only)
|
||||
for task, models in [
|
||||
("coding", MODELS_ALL),
|
||||
("research", MODELS_ALL),
|
||||
("conversation", MODELS_ALL),
|
||||
("unknown-type", MODELS_ALL),
|
||||
("coding", ["mystery-model:x"]),
|
||||
("coding", []),
|
||||
]:
|
||||
cases.append(("route_model", (task,), {"available_models": models}))
|
||||
# _ctx_budget
|
||||
for mid in ["deterministic", "qwen2.5:3b", "qwen3:14b", "qwen3.6:27b",
|
||||
"mimo-v2.5-pro", "totally-unknown-model", ""]:
|
||||
cases.append(("_ctx_budget", (mid,), {}))
|
||||
# _now (pinned clock)
|
||||
cases.append(("_now", (), {}))
|
||||
# handle_hook
|
||||
cases.append(("handle_hook", ("pre_tool_use", {"tool": "bash", "dry": False}), {}))
|
||||
cases.append(("handle_hook", ("", None), {}))
|
||||
cases.append(("handle_hook", ("post_response", {"ok": True}), {}))
|
||||
# _get_cloud_models via the pinned-HOME seams
|
||||
for fixture in [FIXTURE_MAIN, FIXTURE_ONLY_OLLAMA, FIXTURE_NO_MODELS_KEY,
|
||||
FIXTURE_NO_PROVIDER_KEY, FIXTURE_BROKEN_LATE,
|
||||
FIXTURE_URL_CORRUPTION]:
|
||||
cases.append(("_get_cloud_models_seeded", (fixture,), {}))
|
||||
cases.append(("_get_cloud_models_missing", (), {}))
|
||||
|
||||
truth = freeze_truth.freeze("model_connect", cases, oracle_module="rune.model_connect")
|
||||
out = REPO / "src" / "go" / "model-connect" / "truth.json"
|
||||
freeze_truth.write_envelope(FROZEN_NOW, truth, out)
|
||||
print(f"wrote {len(truth)} cases -> {out}")
|
||||
|
||||
# sanity: the pinned clock must render exactly as the envelope pin
|
||||
now_row = [c for c in truth if c["fn"] == "_now"][0]
|
||||
assert now_row["expected"] == FROZEN_NOW, now_row
|
||||
print("_now pinned-clock row:", now_row["expected"])
|
||||
@@ -0,0 +1,59 @@
|
||||
# Freeze rig for the model_connect dichotomy gap-closing (dichotomy wave
|
||||
# #2479 gap port, contract factory-gap-mconn). Auto-imported on interpreter
|
||||
# start (PYTHONPATH=scripts/mconn_freeze_seams:src) by BOTH the truth
|
||||
# generator (scripts/gen_model_connect_truth.py), the shared harness's
|
||||
# --verify mode, and the live-parity sweep, so frozen truth is
|
||||
# byte-reproducible in any process:
|
||||
#
|
||||
# * HOME is pinned to a fresh throwaway dir (freeze_rigs.pin_home) —
|
||||
# _get_cloud_models derives ~/.config/mimocode/mimocode.jsonc via
|
||||
# expanduser at call time; hermetic HOME keeps --verify off the real
|
||||
# ~/.config and makes every row deterministic.
|
||||
# * The per-case cloud-model seams (_get_cloud_models_seeded /
|
||||
# _get_cloud_models_missing) are bound onto the oracle module so the
|
||||
# shared harness's getattr(module, fn) can freeze them: seeded writes the
|
||||
# raw JSONC fixture into the pinned HOME then calls the REAL
|
||||
# _get_cloud_models; missing removes the file. The battery reproduces the
|
||||
# same rig from the case args (fixture text rides in the truth rows).
|
||||
# * The clock needs no rig here: the shared harness pins the module-global
|
||||
# datetime from the envelope's frozen_now (RUNE_FROZEN_NOW), which is the
|
||||
# only clock site in the ported surface (_now).
|
||||
|
||||
import os
|
||||
import sys
|
||||
|
||||
_REPO = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
sys.path.insert(0, os.path.join(_REPO, "scripts"))
|
||||
sys.path.insert(0, os.path.join(_REPO, "src"))
|
||||
|
||||
from freeze_rigs import pin_home # noqa: E402
|
||||
|
||||
pin_home("mconn")
|
||||
|
||||
import rune.model_connect as _mc # noqa: E402 (after the HOME pin)
|
||||
|
||||
|
||||
def _config_path():
|
||||
return os.path.join(os.environ["HOME"], ".config", "mimocode", "mimocode.jsonc")
|
||||
|
||||
|
||||
def _get_cloud_models_seeded(fixture):
|
||||
"""Write the raw JSONC fixture into the pinned HOME, then read it
|
||||
through the oracle's real _get_cloud_models."""
|
||||
path = _config_path()
|
||||
os.makedirs(os.path.dirname(path), exist_ok=True)
|
||||
with open(path, "w") as f:
|
||||
f.write(fixture)
|
||||
return _mc._get_cloud_models()
|
||||
|
||||
|
||||
def _get_cloud_models_missing():
|
||||
"""Remove the config from the pinned HOME (fresh-home error branch)."""
|
||||
path = _config_path()
|
||||
if os.path.exists(path):
|
||||
os.remove(path)
|
||||
return _mc._get_cloud_models()
|
||||
|
||||
|
||||
_mc._get_cloud_models_seeded = _get_cloud_models_seeded
|
||||
_mc._get_cloud_models_missing = _get_cloud_models_missing
|
||||
+441
-61
@@ -1,22 +1,33 @@
|
||||
// Package main — model-connect: the GO port of src/rune/model_connect.py's
|
||||
// deterministic cores (batch 113 of the dichotomy tranche, #2479).
|
||||
// deterministic cores (batch 113 of the dichotomy tranche, #2479; gap-close
|
||||
// wave 4).
|
||||
//
|
||||
// Dichotomy verdict: GO (reactive/entry surface). Network I/O (list_models,
|
||||
// health_check, generate) stays on the Python reactive side — only the pure
|
||||
// transforms are ported: _clean_model_name, is_moe_model, route_model.
|
||||
// health_check, generate) stays on the Python reactive side — the ported
|
||||
// surface is the pure/config-derived transforms: _clean_model_name,
|
||||
// is_moe_model, route_model, and (gap-close) _ctx_budget, _now,
|
||||
// _get_cloud_models, handle_hook.
|
||||
//
|
||||
// Parity: truth.json (20 cases) generated oracle-first against the Python
|
||||
// module; replayed hard in CI (non-swallowable step). Parity is the floor —
|
||||
// improvements: none required; oracle verified correct on all probed edges
|
||||
// (empty-list route returns reason "no-models-available" with null model).
|
||||
// Parity: truth.json (shared freeze_truth.py envelope, pinned clock + pinned
|
||||
// HOME) generated oracle-first against the Python module via
|
||||
// scripts/gen_model_connect_truth.py; byte-verified with freeze_truth.py
|
||||
// --verify and embedded VERBATIM in truth_test.go TestFrozenTruthReplay.
|
||||
// Live parity sweep: scripts/check_model_connect_parity.py (op dispatch over
|
||||
// stdin). Known oracle quirks preserved: the JSONC // strip is naive (it
|
||||
// eats URLs), trailing commas are stripped anywhere before } or ], and a
|
||||
// mid-provider shape error returns the PARTIAL model list (the oracle's
|
||||
// try/except wraps the whole loop).
|
||||
package main
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// moePatterns mirrors MOE_MODEL_PATTERNS.
|
||||
@@ -128,74 +139,443 @@ func routeModel(taskType string, availableModels []string) routeResult {
|
||||
return routeResult{Reason: "no-models-available"}
|
||||
}
|
||||
|
||||
func main() {
|
||||
if len(os.Args) > 1 && os.Args[1] != "replay" {
|
||||
fmt.Fprintln(os.Stderr, "usage: model-connect [replay [truth.json]]")
|
||||
os.Exit(2)
|
||||
// ── gap-close surface (#2479 wave 4) ────────────────────────────────────────
|
||||
|
||||
// ctxProfile is the projection of model_registry's DEFAULT_PROFILES that
|
||||
// resolve_context_budget reads (the oracle's _ctx_budget imports
|
||||
// model_registry at call time; the values below are the oracle's registry).
|
||||
type ctxProfile struct {
|
||||
ID string
|
||||
ContextWindow int
|
||||
Local bool
|
||||
}
|
||||
|
||||
var ctxBudgetProfiles = []ctxProfile{
|
||||
{ID: "deterministic", ContextWindow: 0, Local: true}, // 0 → 8192 via the `or` floor
|
||||
{ID: "qwen2.5:3b", ContextWindow: 32768, Local: true},
|
||||
{ID: "qwen3:14b", ContextWindow: 32768, Local: true},
|
||||
{ID: "qwen3.6:27b", ContextWindow: 32768, Local: true},
|
||||
{ID: "mimo-v2.5-pro", ContextWindow: 128000, Local: false},
|
||||
}
|
||||
|
||||
const (
|
||||
ctxOutputReserve = 2048 // model_registry.OUTPUT_RESERVE["default"]
|
||||
ctxToolResultReserve = 1024 // model_registry.TOOL_RESULT_RESERVE
|
||||
ctxNumCtxFloor = 1024
|
||||
ctxDefaultWindow = 8192
|
||||
)
|
||||
|
||||
// CtxBudget mirrors _ctx_budget — the num_ctx projection of
|
||||
// model_registry.resolve_context_budget with the default task kind:
|
||||
// max(1024, window − output_reserve − tool_reserve), tool_reserve 0 for
|
||||
// local models.
|
||||
func CtxBudget(modelID string) int {
|
||||
window, local := ctxDefaultWindow, true
|
||||
for _, p := range ctxBudgetProfiles {
|
||||
if p.ID == modelID {
|
||||
window = p.ContextWindow
|
||||
if window == 0 {
|
||||
window = ctxDefaultWindow
|
||||
}
|
||||
local = p.Local
|
||||
break
|
||||
}
|
||||
}
|
||||
toolReserve := 0
|
||||
if !local {
|
||||
toolReserve = ctxToolResultReserve
|
||||
}
|
||||
numCtx := window - ctxOutputReserve - toolReserve
|
||||
if numCtx < ctxNumCtxFloor {
|
||||
numCtx = ctxNumCtxFloor
|
||||
}
|
||||
return numCtx
|
||||
}
|
||||
|
||||
// nowClock is the injectable clock seam behind Now — the no-clock law: the
|
||||
// battery and the live-parity sweep pin it (MCONN_CLOCK / frozen_now), the
|
||||
// unpinned default reads the wall clock like the oracle's
|
||||
// datetime.now(timezone.utc).
|
||||
var nowClock = func() string {
|
||||
if pinned := os.Getenv("MCONN_CLOCK"); pinned != "" {
|
||||
return pinned
|
||||
}
|
||||
return pyIsoformat(time.Now().UTC())
|
||||
}
|
||||
|
||||
// pyIsoformat renders t like Python datetime.isoformat(): +00:00 offset,
|
||||
// microseconds only when nonzero.
|
||||
func pyIsoformat(t time.Time) string {
|
||||
if t.Nanosecond() == 0 {
|
||||
return t.Format("2006-01-02T15:04:05+00:00")
|
||||
}
|
||||
return t.Format("2006-01-02T15:04:05.000000+00:00")
|
||||
}
|
||||
|
||||
// Now mirrors _now — datetime.now(timezone.utc).isoformat().
|
||||
func Now() string { return nowClock() }
|
||||
|
||||
// cloudModel mirrors one _get_cloud_models entry; field order matches the
|
||||
// oracle's dict insertion order (name, size_gb, family, parameters,
|
||||
// capabilities, tier, provider).
|
||||
type cloudModel struct {
|
||||
Name string `json:"name"`
|
||||
SizeGB int `json:"size_gb"`
|
||||
Family string `json:"family"`
|
||||
Parameters string `json:"parameters"`
|
||||
Capabilities []string `json:"capabilities"`
|
||||
Tier string `json:"tier"`
|
||||
Provider string `json:"provider"`
|
||||
}
|
||||
|
||||
var cloudCapabilities = []string{"complex-reasoning", "coding", "analysis"}
|
||||
|
||||
// jsonObject is an insertion-ordered JSON object (Python dict semantics —
|
||||
// provider and model iteration order is part of the oracle contract).
|
||||
type jsonObject struct {
|
||||
keys []string
|
||||
values map[string]interface{}
|
||||
}
|
||||
|
||||
func (o *jsonObject) get(key string) (interface{}, bool) {
|
||||
v, ok := o.values[key]
|
||||
return v, ok
|
||||
}
|
||||
|
||||
// parseOrderedValue — json.Decoder token walk preserving object key order.
|
||||
func parseOrderedValue(dec *json.Decoder) (interface{}, error) {
|
||||
tok, err := dec.Token()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return parseOrderedFrom(dec, tok)
|
||||
}
|
||||
|
||||
func parseOrderedFrom(dec *json.Decoder, tok json.Token) (interface{}, error) {
|
||||
delim, ok := tok.(json.Delim)
|
||||
if !ok {
|
||||
return tok, nil // nil / bool / float64 / string
|
||||
}
|
||||
switch delim {
|
||||
case '{':
|
||||
obj := &jsonObject{values: map[string]interface{}{}}
|
||||
for dec.More() {
|
||||
keyTok, err := dec.Token()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
key, ok := keyTok.(string)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("object key is not a string")
|
||||
}
|
||||
val, err := parseOrderedValue(dec)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if _, dup := obj.values[key]; !dup {
|
||||
obj.keys = append(obj.keys, key)
|
||||
}
|
||||
obj.values[key] = val
|
||||
}
|
||||
if _, err := dec.Token(); err != nil { // consume '}'
|
||||
return nil, err
|
||||
}
|
||||
return obj, nil
|
||||
case '[':
|
||||
arr := []interface{}{}
|
||||
for dec.More() {
|
||||
val, err := parseOrderedValue(dec)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
arr = append(arr, val)
|
||||
}
|
||||
if _, err := dec.Token(); err != nil { // consume ']'
|
||||
return nil, err
|
||||
}
|
||||
return arr, nil
|
||||
}
|
||||
return nil, fmt.Errorf("unexpected delimiter %v", delim)
|
||||
}
|
||||
|
||||
var (
|
||||
jsoncCommentRe = regexp.MustCompile(`(?m)//.*?$`)
|
||||
jsoncTrailingCommaRe = regexp.MustCompile(`,\s*([}\]])`)
|
||||
)
|
||||
|
||||
// stripJsonc mirrors the oracle's naive JSONC → JSON prep. The comment strip
|
||||
// is deliberately string-blind: a "//" inside a quoted URL eats the rest of
|
||||
// the line and the parse then fails — the oracle behaves identically.
|
||||
func stripJsonc(raw string) string {
|
||||
raw = jsoncCommentRe.ReplaceAllString(raw, "")
|
||||
return jsoncTrailingCommaRe.ReplaceAllString(raw, "${1}")
|
||||
}
|
||||
|
||||
// GetCloudModels mirrors _get_cloud_models — read ~/.config/mimocode/
|
||||
// mimocode.jsonc, JSONC-strip, parse, and project every non-ollama provider
|
||||
// model into the fixed cloud shape. Any error mid-loop returns the PARTIAL
|
||||
// list accumulated so far (the oracle's try/except wraps the whole loop);
|
||||
// read/parse errors return an empty list.
|
||||
func GetCloudModels() []cloudModel {
|
||||
out := []cloudModel{}
|
||||
home, err := os.UserHomeDir()
|
||||
if err != nil {
|
||||
return out
|
||||
}
|
||||
raw, err := os.ReadFile(filepath.Join(home, ".config", "mimocode", "mimocode.jsonc"))
|
||||
if err != nil {
|
||||
return out
|
||||
}
|
||||
v, err := parseOrderedValue(json.NewDecoder(strings.NewReader(stripJsonc(string(raw)))))
|
||||
if err != nil {
|
||||
return out
|
||||
}
|
||||
cfg, ok := v.(*jsonObject)
|
||||
if !ok {
|
||||
return out // config.get on a non-dict raises in the oracle
|
||||
}
|
||||
providersV, ok := cfg.get("provider")
|
||||
if !ok || providersV == nil {
|
||||
return out
|
||||
}
|
||||
providers, ok := providersV.(*jsonObject)
|
||||
if !ok {
|
||||
return out // providers.items() raises in the oracle
|
||||
}
|
||||
for _, pname := range providers.keys {
|
||||
if pname == "ollama" {
|
||||
continue
|
||||
}
|
||||
pcfgV, ok := providers.values[pname]
|
||||
if !ok || pcfgV == nil {
|
||||
return out // provider_config.get raises in the oracle
|
||||
}
|
||||
pcfg, ok := pcfgV.(*jsonObject)
|
||||
if !ok {
|
||||
return out
|
||||
}
|
||||
modelsV, present := pcfg.get("models")
|
||||
if !present {
|
||||
continue // .get("models", {}) → empty dict, nothing to add
|
||||
}
|
||||
if modelsV == nil {
|
||||
return out // None.items() raises in the oracle
|
||||
}
|
||||
models, ok := modelsV.(*jsonObject)
|
||||
if !ok {
|
||||
return out // models.items() raises in the oracle
|
||||
}
|
||||
for _, mid := range models.keys {
|
||||
out = append(out, cloudModel{
|
||||
Name: pname + "/" + mid,
|
||||
SizeGB: 0,
|
||||
Family: "cloud",
|
||||
Parameters: "cloud",
|
||||
Capabilities: cloudCapabilities,
|
||||
Tier: "cloud",
|
||||
Provider: pname,
|
||||
})
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// hookResult mirrors handle_hook's return dict (insertion order:
|
||||
// status, event, handler).
|
||||
type hookResult struct {
|
||||
Status string `json:"status"`
|
||||
Event string `json:"event"`
|
||||
Handler string `json:"handler"`
|
||||
}
|
||||
|
||||
// HandleHook mirrors handle_hook — the constant ok envelope; data is
|
||||
// accepted and ignored exactly like the oracle.
|
||||
func HandleHook(event string, data interface{}) hookResult {
|
||||
return hookResult{Status: "ok", Event: event, Handler: "model_connect"}
|
||||
}
|
||||
|
||||
// ── replay (frozen truth) + parity (stdin op) entrypoints ───────────────────
|
||||
|
||||
func main() {
|
||||
if len(os.Args) > 1 && os.Args[1] == "replay" {
|
||||
os.Exit(replayMain())
|
||||
}
|
||||
os.Exit(parityMain())
|
||||
}
|
||||
|
||||
type frozenCase struct {
|
||||
Fn string `json:"fn"`
|
||||
Args []interface{} `json:"args"`
|
||||
Kwargs map[string]interface{} `json:"kwargs"`
|
||||
Expected json.RawMessage `json:"expected"`
|
||||
}
|
||||
|
||||
// canonical — round-trip a value through JSON so both sides compare as
|
||||
// decoded data (Python None and Go nil decode to the same null).
|
||||
func canonical(v interface{}) (interface{}, error) {
|
||||
raw, err := json.Marshal(v)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var out interface{}
|
||||
if err := json.Unmarshal(raw, &out); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func replayCase(c frozenCase, seedHome func(fixture string) string, missingHome func() string) (interface{}, error) {
|
||||
switch c.Fn {
|
||||
case "_clean_model_name":
|
||||
return cleanModelName(c.Args[0].(string)), nil
|
||||
case "is_moe_model":
|
||||
return isMoeModel(c.Args[0].(string)), nil
|
||||
case "route_model":
|
||||
var models []string
|
||||
b, _ := json.Marshal(c.Kwargs["available_models"])
|
||||
if err := json.Unmarshal(b, &models); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return routeModel(c.Args[0].(string), models), nil
|
||||
case "_ctx_budget":
|
||||
return CtxBudget(c.Args[0].(string)), nil
|
||||
case "_now":
|
||||
return Now(), nil
|
||||
case "handle_hook":
|
||||
return HandleHook(c.Args[0].(string), c.Args[1]), nil
|
||||
case "_get_cloud_models_seeded":
|
||||
return GetCloudModelsIn(seedHome(c.Args[0].(string))), nil
|
||||
case "_get_cloud_models_missing":
|
||||
return GetCloudModelsIn(missingHome()), nil
|
||||
}
|
||||
return nil, fmt.Errorf("unknown frozen fn %q", c.Fn)
|
||||
}
|
||||
|
||||
// GetCloudModelsIn — GetCloudModels against an explicit HOME (the battery's
|
||||
// seeded/missing rows rebuild the oracle's pinned-HOME rig).
|
||||
func GetCloudModelsIn(home string) []cloudModel {
|
||||
t := os.Getenv("HOME")
|
||||
os.Setenv("HOME", home)
|
||||
defer os.Setenv("HOME", t)
|
||||
return GetCloudModels()
|
||||
}
|
||||
|
||||
func replayMain() int {
|
||||
path := "truth.json"
|
||||
if len(os.Args) > 2 {
|
||||
path = os.Args[2]
|
||||
}
|
||||
raw, err := os.ReadFile(path)
|
||||
fatal := func(e error) { fmt.Fprintln(os.Stderr, "FATAL:", e); os.Exit(1) }
|
||||
if err != nil {
|
||||
fatal(err)
|
||||
fmt.Fprintln(os.Stderr, "FATAL:", err)
|
||||
return 1
|
||||
}
|
||||
var rows []struct {
|
||||
Op string `json:"op"`
|
||||
Input json.RawMessage `json:"input"`
|
||||
Want json.RawMessage `json:"want"`
|
||||
var env struct {
|
||||
FrozenNow string `json:"frozen_now"`
|
||||
Cases []frozenCase `json:"cases"`
|
||||
}
|
||||
if err := json.Unmarshal(raw, &rows); err != nil {
|
||||
fatal(err)
|
||||
if err := json.Unmarshal(raw, &env); err != nil {
|
||||
fmt.Fprintln(os.Stderr, "FATAL:", err)
|
||||
return 1
|
||||
}
|
||||
if len(env.Cases) == 0 {
|
||||
fmt.Fprintln(os.Stderr, "FATAL: no truth cases")
|
||||
return 1
|
||||
}
|
||||
savedClock := nowClock
|
||||
os.Setenv("MCONN_CLOCK", env.FrozenNow)
|
||||
nowClock = func() string { return env.FrozenNow }
|
||||
defer func() {
|
||||
nowClock = savedClock
|
||||
os.Unsetenv("MCONN_CLOCK")
|
||||
}()
|
||||
|
||||
seedHome := func(fixture string) string {
|
||||
home, _ := os.MkdirTemp("", "mconn-replay-")
|
||||
cfgDir := filepath.Join(home, ".config", "mimocode")
|
||||
os.MkdirAll(cfgDir, 0o755)
|
||||
os.WriteFile(filepath.Join(cfgDir, "mimocode.jsonc"), []byte(fixture), 0o644)
|
||||
return home
|
||||
}
|
||||
missingHome := func() string {
|
||||
home, _ := os.MkdirTemp("", "mconn-replay-")
|
||||
return home
|
||||
}
|
||||
|
||||
fails := 0
|
||||
for _, r := range rows {
|
||||
switch r.Op {
|
||||
case "clean":
|
||||
var in, want string
|
||||
json.Unmarshal(r.Input, &in)
|
||||
json.Unmarshal(r.Want, &want)
|
||||
if got := cleanModelName(in); got != want {
|
||||
fails++
|
||||
fmt.Printf("FAIL clean %q: got %q want %q\n", in, got, want)
|
||||
}
|
||||
case "moe":
|
||||
var in string
|
||||
var want bool
|
||||
json.Unmarshal(r.Input, &in)
|
||||
json.Unmarshal(r.Want, &want)
|
||||
if got := isMoeModel(in); got != want {
|
||||
fails++
|
||||
fmt.Printf("FAIL moe %q: got %v want %v\n", in, got, want)
|
||||
}
|
||||
case "route":
|
||||
var in struct {
|
||||
Task string `json:"task"`
|
||||
Models []string `json:"models"`
|
||||
}
|
||||
json.Unmarshal(r.Input, &in)
|
||||
wantRaw, _ := r.Want.MarshalJSON()
|
||||
var wantMap map[string]interface{}
|
||||
json.Unmarshal(wantRaw, &wantMap)
|
||||
got := routeModel(in.Task, in.Models)
|
||||
gotJSON, _ := json.Marshal(got)
|
||||
var gotMap map[string]interface{}
|
||||
json.Unmarshal(gotJSON, &gotMap)
|
||||
if fmt.Sprint(gotMap["model"]) != fmt.Sprint(wantMap["model"]) ||
|
||||
fmt.Sprint(gotMap["tier"]) != fmt.Sprint(wantMap["tier"]) ||
|
||||
gotMap["reason"] != wantMap["reason"] {
|
||||
fails++
|
||||
fmt.Printf("FAIL route %s: got %s want %s\n", in.Task, gotJSON, wantRaw)
|
||||
}
|
||||
default:
|
||||
for _, c := range env.Cases {
|
||||
actual, err := replayCase(c, seedHome, missingHome)
|
||||
if err != nil {
|
||||
fails++
|
||||
fmt.Printf("FAIL unknown op %s\n", r.Op)
|
||||
fmt.Printf("FAIL %s: %v\n", c.Fn, err)
|
||||
continue
|
||||
}
|
||||
got, err := canonical(actual)
|
||||
if err != nil {
|
||||
fails++
|
||||
fmt.Printf("FAIL %s: canonical actual: %v\n", c.Fn, err)
|
||||
continue
|
||||
}
|
||||
var want interface{}
|
||||
if err := json.Unmarshal(c.Expected, &want); err != nil {
|
||||
fails++
|
||||
fmt.Printf("FAIL %s: decode expected: %v\n", c.Fn, err)
|
||||
continue
|
||||
}
|
||||
gj, _ := json.Marshal(got)
|
||||
wj, _ := json.Marshal(want)
|
||||
if string(gj) != string(wj) {
|
||||
fails++
|
||||
fmt.Printf("FAIL %s: got %s want %s\n", c.Fn, gj, wj)
|
||||
}
|
||||
}
|
||||
fmt.Printf("model-connect parity: %d/%d passed\n", len(rows)-fails, len(rows))
|
||||
fmt.Printf("model-connect parity: %d/%d passed\n", len(env.Cases)-fails, len(env.Cases))
|
||||
if fails > 0 {
|
||||
os.Exit(1)
|
||||
return 1
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
func parityMain() int {
|
||||
var req struct {
|
||||
Op string `json:"op"`
|
||||
Name string `json:"name"`
|
||||
Task string `json:"task"`
|
||||
Model string `json:"model"`
|
||||
Event string `json:"event"`
|
||||
Data json.RawMessage `json:"data"`
|
||||
Models []string `json:"models"`
|
||||
}
|
||||
body, err := io.ReadAll(os.Stdin)
|
||||
if err != nil {
|
||||
fmt.Fprintln(os.Stderr, "FATAL:", err)
|
||||
return 1
|
||||
}
|
||||
if err := json.Unmarshal(body, &req); err != nil {
|
||||
fmt.Fprintln(os.Stderr, "FATAL:", err)
|
||||
return 1
|
||||
}
|
||||
var result interface{}
|
||||
switch req.Op {
|
||||
case "clean":
|
||||
result = map[string]interface{}{"result": cleanModelName(req.Name)}
|
||||
case "moe":
|
||||
result = map[string]interface{}{"result": isMoeModel(req.Name)}
|
||||
case "route":
|
||||
result = routeModel(req.Task, req.Models)
|
||||
case "ctx_budget":
|
||||
result = map[string]interface{}{"result": CtxBudget(req.Model)}
|
||||
case "now":
|
||||
result = map[string]interface{}{"result": Now()}
|
||||
case "cloud_models":
|
||||
result = map[string]interface{}{"result": GetCloudModels()}
|
||||
case "hook":
|
||||
result = map[string]interface{}{"result": HandleHook(req.Event, req.Data)}
|
||||
default:
|
||||
fmt.Fprintln(os.Stderr, "FATAL: unknown op", req.Op)
|
||||
return 1
|
||||
}
|
||||
out, _ := json.Marshal(result)
|
||||
fmt.Println(string(out))
|
||||
return 0
|
||||
}
|
||||
+481
-194
@@ -1,194 +1,481 @@
|
||||
[
|
||||
{
|
||||
"op": "clean",
|
||||
"input": "hf.co/DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF:IQ2_M",
|
||||
"want": "Qwen3.6 27B"
|
||||
},
|
||||
{
|
||||
"op": "clean",
|
||||
"input": "hf.co/org/Llama-3-8B-Instruct:latest",
|
||||
"want": "Llama 3 8B"
|
||||
},
|
||||
{
|
||||
"op": "clean",
|
||||
"input": "hf.co/short/gemma-2-9b",
|
||||
"want": "gemma 2 9B"
|
||||
},
|
||||
{
|
||||
"op": "clean",
|
||||
"input": "hf.co/user/nano-model",
|
||||
"want": "nano"
|
||||
},
|
||||
{
|
||||
"op": "clean",
|
||||
"input": "hf.co/user/ab",
|
||||
"want": "ab"
|
||||
},
|
||||
{
|
||||
"op": "clean",
|
||||
"input": "qwen2.5:3b",
|
||||
"want": "qwen2.5:3b"
|
||||
},
|
||||
{
|
||||
"op": "clean",
|
||||
"input": "mimo-v2.5-pro",
|
||||
"want": "mimo-v2.5-pro"
|
||||
},
|
||||
{
|
||||
"op": "clean",
|
||||
"input": "llava:7b",
|
||||
"want": "llava:7b"
|
||||
},
|
||||
{
|
||||
"op": "moe",
|
||||
"input": "mixtral:8x7b",
|
||||
"want": true
|
||||
},
|
||||
{
|
||||
"op": "moe",
|
||||
"input": "qwen2.5:3b",
|
||||
"want": false
|
||||
},
|
||||
{
|
||||
"op": "moe",
|
||||
"input": "deepseek-moe-16b",
|
||||
"want": true
|
||||
},
|
||||
{
|
||||
"op": "moe",
|
||||
"input": "My-Mixture-Model",
|
||||
"want": true
|
||||
},
|
||||
{
|
||||
"op": "moe",
|
||||
"input": "expert-v1",
|
||||
"want": true
|
||||
},
|
||||
{
|
||||
"op": "moe",
|
||||
"input": "4x22b-tiny",
|
||||
"want": true
|
||||
},
|
||||
{
|
||||
"op": "route",
|
||||
"input": {
|
||||
"task": "coding",
|
||||
"models": [
|
||||
"qwen2.5:3b",
|
||||
"qwen2.5:7b",
|
||||
"qwen3:14b",
|
||||
"qwen2.5-coder:14b",
|
||||
"hf.co/DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF:IQ2_M",
|
||||
"mixtral:8x7b-instruct-v0.1-q4_0",
|
||||
"qwen2.5vl:3b",
|
||||
"llava:7b",
|
||||
"qwen3-embedding:0.6b",
|
||||
"nomic-embed-text:latest",
|
||||
"mxbai-embed-large:latest"
|
||||
]
|
||||
},
|
||||
"want": {
|
||||
"model": "qwen2.5-coder:14b",
|
||||
"tier": "local-coder",
|
||||
"reason": "exact-match"
|
||||
}
|
||||
},
|
||||
{
|
||||
"op": "route",
|
||||
"input": {
|
||||
"task": "research",
|
||||
"models": [
|
||||
"qwen2.5:3b",
|
||||
"qwen2.5:7b",
|
||||
"qwen3:14b",
|
||||
"qwen2.5-coder:14b",
|
||||
"hf.co/DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF:IQ2_M",
|
||||
"mixtral:8x7b-instruct-v0.1-q4_0",
|
||||
"qwen2.5vl:3b",
|
||||
"llava:7b",
|
||||
"qwen3-embedding:0.6b",
|
||||
"nomic-embed-text:latest",
|
||||
"mxbai-embed-large:latest"
|
||||
]
|
||||
},
|
||||
"want": {
|
||||
"model": "qwen3:14b",
|
||||
"tier": "local-large",
|
||||
"reason": "exact-match"
|
||||
}
|
||||
},
|
||||
{
|
||||
"op": "route",
|
||||
"input": {
|
||||
"task": "conversation",
|
||||
"models": [
|
||||
"qwen2.5:3b",
|
||||
"qwen2.5:7b",
|
||||
"qwen3:14b",
|
||||
"qwen2.5-coder:14b",
|
||||
"hf.co/DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF:IQ2_M",
|
||||
"mixtral:8x7b-instruct-v0.1-q4_0",
|
||||
"qwen2.5vl:3b",
|
||||
"llava:7b",
|
||||
"qwen3-embedding:0.6b",
|
||||
"nomic-embed-text:latest",
|
||||
"mxbai-embed-large:latest"
|
||||
]
|
||||
},
|
||||
"want": {
|
||||
"model": "qwen2.5:3b",
|
||||
"tier": "local-light",
|
||||
"reason": "exact-match"
|
||||
}
|
||||
},
|
||||
{
|
||||
"op": "route",
|
||||
"input": {
|
||||
"task": "unknown-type",
|
||||
"models": [
|
||||
"qwen2.5:3b",
|
||||
"qwen2.5:7b",
|
||||
"qwen3:14b",
|
||||
"qwen2.5-coder:14b",
|
||||
"hf.co/DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF:IQ2_M",
|
||||
"mixtral:8x7b-instruct-v0.1-q4_0",
|
||||
"qwen2.5vl:3b",
|
||||
"llava:7b",
|
||||
"qwen3-embedding:0.6b",
|
||||
"nomic-embed-text:latest",
|
||||
"mxbai-embed-large:latest"
|
||||
]
|
||||
},
|
||||
"want": {
|
||||
"model": "qwen2.5:3b",
|
||||
"tier": "local-light",
|
||||
"reason": "exact-match"
|
||||
}
|
||||
},
|
||||
{
|
||||
"op": "route",
|
||||
"input": {
|
||||
"task": "coding",
|
||||
"models": [
|
||||
"mystery-model:x"
|
||||
]
|
||||
},
|
||||
"want": {
|
||||
"model": "mystery-model:x",
|
||||
"tier": "fallback",
|
||||
"reason": "no-exact-match"
|
||||
}
|
||||
},
|
||||
{
|
||||
"op": "route",
|
||||
"input": {
|
||||
"task": "coding",
|
||||
"models": []
|
||||
},
|
||||
"want": {
|
||||
"model": null,
|
||||
"tier": null,
|
||||
"reason": "no-models-available"
|
||||
}
|
||||
}
|
||||
]
|
||||
{
|
||||
"frozen_now": "2026-10-14T08:00:00+00:00",
|
||||
"cases": [
|
||||
{
|
||||
"fn": "_clean_model_name",
|
||||
"args": [
|
||||
"hf.co/DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF:IQ2_M"
|
||||
],
|
||||
"kwargs": {},
|
||||
"expected": "Qwen3.6 27B"
|
||||
},
|
||||
{
|
||||
"fn": "_clean_model_name",
|
||||
"args": [
|
||||
"hf.co/org/Llama-3-8B-Instruct:latest"
|
||||
],
|
||||
"kwargs": {},
|
||||
"expected": "Llama 3 8B"
|
||||
},
|
||||
{
|
||||
"fn": "_clean_model_name",
|
||||
"args": [
|
||||
"hf.co/short/gemma-2-9b"
|
||||
],
|
||||
"kwargs": {},
|
||||
"expected": "gemma 2 9B"
|
||||
},
|
||||
{
|
||||
"fn": "_clean_model_name",
|
||||
"args": [
|
||||
"hf.co/user/nano-model"
|
||||
],
|
||||
"kwargs": {},
|
||||
"expected": "nano"
|
||||
},
|
||||
{
|
||||
"fn": "_clean_model_name",
|
||||
"args": [
|
||||
"hf.co/user/ab"
|
||||
],
|
||||
"kwargs": {},
|
||||
"expected": "ab"
|
||||
},
|
||||
{
|
||||
"fn": "_clean_model_name",
|
||||
"args": [
|
||||
"qwen2.5:3b"
|
||||
],
|
||||
"kwargs": {},
|
||||
"expected": "qwen2.5:3b"
|
||||
},
|
||||
{
|
||||
"fn": "_clean_model_name",
|
||||
"args": [
|
||||
"mimo-v2.5-pro"
|
||||
],
|
||||
"kwargs": {},
|
||||
"expected": "mimo-v2.5-pro"
|
||||
},
|
||||
{
|
||||
"fn": "_clean_model_name",
|
||||
"args": [
|
||||
"llava:7b"
|
||||
],
|
||||
"kwargs": {},
|
||||
"expected": "llava:7b"
|
||||
},
|
||||
{
|
||||
"fn": "is_moe_model",
|
||||
"args": [
|
||||
"mixtral:8x7b"
|
||||
],
|
||||
"kwargs": {},
|
||||
"expected": true
|
||||
},
|
||||
{
|
||||
"fn": "is_moe_model",
|
||||
"args": [
|
||||
"qwen2.5:3b"
|
||||
],
|
||||
"kwargs": {},
|
||||
"expected": false
|
||||
},
|
||||
{
|
||||
"fn": "is_moe_model",
|
||||
"args": [
|
||||
"deepseek-moe-16b"
|
||||
],
|
||||
"kwargs": {},
|
||||
"expected": true
|
||||
},
|
||||
{
|
||||
"fn": "is_moe_model",
|
||||
"args": [
|
||||
"My-Mixture-Model"
|
||||
],
|
||||
"kwargs": {},
|
||||
"expected": true
|
||||
},
|
||||
{
|
||||
"fn": "is_moe_model",
|
||||
"args": [
|
||||
"expert-v1"
|
||||
],
|
||||
"kwargs": {},
|
||||
"expected": true
|
||||
},
|
||||
{
|
||||
"fn": "is_moe_model",
|
||||
"args": [
|
||||
"4x22b-tiny"
|
||||
],
|
||||
"kwargs": {},
|
||||
"expected": true
|
||||
},
|
||||
{
|
||||
"fn": "is_moe_model",
|
||||
"args": [
|
||||
"plain-conversational"
|
||||
],
|
||||
"kwargs": {},
|
||||
"expected": false
|
||||
},
|
||||
{
|
||||
"fn": "is_moe_model",
|
||||
"args": [
|
||||
"GPT-XL"
|
||||
],
|
||||
"kwargs": {},
|
||||
"expected": false
|
||||
},
|
||||
{
|
||||
"fn": "route_model",
|
||||
"args": [
|
||||
"coding"
|
||||
],
|
||||
"kwargs": {
|
||||
"available_models": [
|
||||
"qwen2.5:3b",
|
||||
"qwen2.5:7b",
|
||||
"qwen3:14b",
|
||||
"qwen2.5-coder:14b",
|
||||
"hf.co/DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF:IQ2_M",
|
||||
"mixtral:8x7b-instruct-v0.1-q4_0",
|
||||
"qwen2.5vl:3b",
|
||||
"llava:7b",
|
||||
"qwen3-embedding:0.6b",
|
||||
"nomic-embed-text:latest",
|
||||
"mxbai-embed-large:latest"
|
||||
]
|
||||
},
|
||||
"expected": {
|
||||
"model": "qwen2.5-coder:14b",
|
||||
"tier": "local-coder",
|
||||
"reason": "exact-match"
|
||||
}
|
||||
},
|
||||
{
|
||||
"fn": "route_model",
|
||||
"args": [
|
||||
"research"
|
||||
],
|
||||
"kwargs": {
|
||||
"available_models": [
|
||||
"qwen2.5:3b",
|
||||
"qwen2.5:7b",
|
||||
"qwen3:14b",
|
||||
"qwen2.5-coder:14b",
|
||||
"hf.co/DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF:IQ2_M",
|
||||
"mixtral:8x7b-instruct-v0.1-q4_0",
|
||||
"qwen2.5vl:3b",
|
||||
"llava:7b",
|
||||
"qwen3-embedding:0.6b",
|
||||
"nomic-embed-text:latest",
|
||||
"mxbai-embed-large:latest"
|
||||
]
|
||||
},
|
||||
"expected": {
|
||||
"model": "qwen3:14b",
|
||||
"tier": "local-large",
|
||||
"reason": "exact-match"
|
||||
}
|
||||
},
|
||||
{
|
||||
"fn": "route_model",
|
||||
"args": [
|
||||
"conversation"
|
||||
],
|
||||
"kwargs": {
|
||||
"available_models": [
|
||||
"qwen2.5:3b",
|
||||
"qwen2.5:7b",
|
||||
"qwen3:14b",
|
||||
"qwen2.5-coder:14b",
|
||||
"hf.co/DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF:IQ2_M",
|
||||
"mixtral:8x7b-instruct-v0.1-q4_0",
|
||||
"qwen2.5vl:3b",
|
||||
"llava:7b",
|
||||
"qwen3-embedding:0.6b",
|
||||
"nomic-embed-text:latest",
|
||||
"mxbai-embed-large:latest"
|
||||
]
|
||||
},
|
||||
"expected": {
|
||||
"model": "qwen2.5:3b",
|
||||
"tier": "local-light",
|
||||
"reason": "exact-match"
|
||||
}
|
||||
},
|
||||
{
|
||||
"fn": "route_model",
|
||||
"args": [
|
||||
"unknown-type"
|
||||
],
|
||||
"kwargs": {
|
||||
"available_models": [
|
||||
"qwen2.5:3b",
|
||||
"qwen2.5:7b",
|
||||
"qwen3:14b",
|
||||
"qwen2.5-coder:14b",
|
||||
"hf.co/DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF:IQ2_M",
|
||||
"mixtral:8x7b-instruct-v0.1-q4_0",
|
||||
"qwen2.5vl:3b",
|
||||
"llava:7b",
|
||||
"qwen3-embedding:0.6b",
|
||||
"nomic-embed-text:latest",
|
||||
"mxbai-embed-large:latest"
|
||||
]
|
||||
},
|
||||
"expected": {
|
||||
"model": "qwen2.5:3b",
|
||||
"tier": "local-light",
|
||||
"reason": "exact-match"
|
||||
}
|
||||
},
|
||||
{
|
||||
"fn": "route_model",
|
||||
"args": [
|
||||
"coding"
|
||||
],
|
||||
"kwargs": {
|
||||
"available_models": [
|
||||
"mystery-model:x"
|
||||
]
|
||||
},
|
||||
"expected": {
|
||||
"model": "mystery-model:x",
|
||||
"tier": "fallback",
|
||||
"reason": "no-exact-match"
|
||||
}
|
||||
},
|
||||
{
|
||||
"fn": "route_model",
|
||||
"args": [
|
||||
"coding"
|
||||
],
|
||||
"kwargs": {
|
||||
"available_models": []
|
||||
},
|
||||
"expected": {
|
||||
"model": null,
|
||||
"tier": null,
|
||||
"reason": "no-models-available"
|
||||
}
|
||||
},
|
||||
{
|
||||
"fn": "_ctx_budget",
|
||||
"args": [
|
||||
"deterministic"
|
||||
],
|
||||
"kwargs": {},
|
||||
"expected": 6144
|
||||
},
|
||||
{
|
||||
"fn": "_ctx_budget",
|
||||
"args": [
|
||||
"qwen2.5:3b"
|
||||
],
|
||||
"kwargs": {},
|
||||
"expected": 30720
|
||||
},
|
||||
{
|
||||
"fn": "_ctx_budget",
|
||||
"args": [
|
||||
"qwen3:14b"
|
||||
],
|
||||
"kwargs": {},
|
||||
"expected": 30720
|
||||
},
|
||||
{
|
||||
"fn": "_ctx_budget",
|
||||
"args": [
|
||||
"qwen3.6:27b"
|
||||
],
|
||||
"kwargs": {},
|
||||
"expected": 30720
|
||||
},
|
||||
{
|
||||
"fn": "_ctx_budget",
|
||||
"args": [
|
||||
"mimo-v2.5-pro"
|
||||
],
|
||||
"kwargs": {},
|
||||
"expected": 124928
|
||||
},
|
||||
{
|
||||
"fn": "_ctx_budget",
|
||||
"args": [
|
||||
"totally-unknown-model"
|
||||
],
|
||||
"kwargs": {},
|
||||
"expected": 6144
|
||||
},
|
||||
{
|
||||
"fn": "_ctx_budget",
|
||||
"args": [
|
||||
""
|
||||
],
|
||||
"kwargs": {},
|
||||
"expected": 6144
|
||||
},
|
||||
{
|
||||
"fn": "_now",
|
||||
"args": [],
|
||||
"kwargs": {},
|
||||
"expected": "2026-10-14T08:00:00+00:00"
|
||||
},
|
||||
{
|
||||
"fn": "handle_hook",
|
||||
"args": [
|
||||
"pre_tool_use",
|
||||
{
|
||||
"tool": "bash",
|
||||
"dry": false
|
||||
}
|
||||
],
|
||||
"kwargs": {},
|
||||
"expected": {
|
||||
"status": "ok",
|
||||
"event": "pre_tool_use",
|
||||
"handler": "model_connect"
|
||||
}
|
||||
},
|
||||
{
|
||||
"fn": "handle_hook",
|
||||
"args": [
|
||||
"",
|
||||
null
|
||||
],
|
||||
"kwargs": {},
|
||||
"expected": {
|
||||
"status": "ok",
|
||||
"event": "",
|
||||
"handler": "model_connect"
|
||||
}
|
||||
},
|
||||
{
|
||||
"fn": "handle_hook",
|
||||
"args": [
|
||||
"post_response",
|
||||
{
|
||||
"ok": true
|
||||
}
|
||||
],
|
||||
"kwargs": {},
|
||||
"expected": {
|
||||
"status": "ok",
|
||||
"event": "post_response",
|
||||
"handler": "model_connect"
|
||||
}
|
||||
},
|
||||
{
|
||||
"fn": "_get_cloud_models_seeded",
|
||||
"args": [
|
||||
"{\n // MiMoCode config — JSONC: comments and trailing commas allowed\n \"provider\": {\n \"ollama\": { // skipped: local models handled by list_models\n \"models\": {\n \"qwen3:14b\": {},\n },\n },\n \"openai\": {\n \"models\": {\n \"gpt-5.2\": {\"contextLength\": 400000},\n \"o4-mini\": \"not-even-a-dict\", // model_config is unused by the oracle\n },\n },\n \"kimi\": {\n \"models\": {\n \"kimi-for-coding\": {},\n },\n },\n },\n}\n"
|
||||
],
|
||||
"kwargs": {},
|
||||
"expected": [
|
||||
{
|
||||
"name": "openai/gpt-5.2",
|
||||
"size_gb": 0,
|
||||
"family": "cloud",
|
||||
"parameters": "cloud",
|
||||
"capabilities": [
|
||||
"complex-reasoning",
|
||||
"coding",
|
||||
"analysis"
|
||||
],
|
||||
"tier": "cloud",
|
||||
"provider": "openai"
|
||||
},
|
||||
{
|
||||
"name": "openai/o4-mini",
|
||||
"size_gb": 0,
|
||||
"family": "cloud",
|
||||
"parameters": "cloud",
|
||||
"capabilities": [
|
||||
"complex-reasoning",
|
||||
"coding",
|
||||
"analysis"
|
||||
],
|
||||
"tier": "cloud",
|
||||
"provider": "openai"
|
||||
},
|
||||
{
|
||||
"name": "kimi/kimi-for-coding",
|
||||
"size_gb": 0,
|
||||
"family": "cloud",
|
||||
"parameters": "cloud",
|
||||
"capabilities": [
|
||||
"complex-reasoning",
|
||||
"coding",
|
||||
"analysis"
|
||||
],
|
||||
"tier": "cloud",
|
||||
"provider": "kimi"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"fn": "_get_cloud_models_seeded",
|
||||
"args": [
|
||||
"{\n \"provider\": {\n \"ollama\": {\n \"models\": {\"qwen2.5:3b\": {}},\n },\n },\n}\n"
|
||||
],
|
||||
"kwargs": {},
|
||||
"expected": []
|
||||
},
|
||||
{
|
||||
"fn": "_get_cloud_models_seeded",
|
||||
"args": [
|
||||
"{\n \"provider\": {\n \"anthropic\": {\n \"apiKey\": \"sk-test\",\n },\n },\n}\n"
|
||||
],
|
||||
"kwargs": {},
|
||||
"expected": []
|
||||
},
|
||||
{
|
||||
"fn": "_get_cloud_models_seeded",
|
||||
"args": [
|
||||
"{\n // no providers configured\n \"theme\": \"dark\",\n}\n"
|
||||
],
|
||||
"kwargs": {},
|
||||
"expected": []
|
||||
},
|
||||
{
|
||||
"fn": "_get_cloud_models_seeded",
|
||||
"args": [
|
||||
"{\n \"provider\": {\n \"a\": {\"models\": {\"m1\": {}}},\n \"b\": {\"models\": \"not-a-dict\"},\n },\n}\n"
|
||||
],
|
||||
"kwargs": {},
|
||||
"expected": [
|
||||
{
|
||||
"name": "a/m1",
|
||||
"size_gb": 0,
|
||||
"family": "cloud",
|
||||
"parameters": "cloud",
|
||||
"capabilities": [
|
||||
"complex-reasoning",
|
||||
"coding",
|
||||
"analysis"
|
||||
],
|
||||
"tier": "cloud",
|
||||
"provider": "a"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"fn": "_get_cloud_models_seeded",
|
||||
"args": [
|
||||
"{\n \"provider\": {\n \"gateway\": {\n \"baseUrl\": \"http://api.example.com/v1\",\n \"models\": {\"gw-model\": {}},\n },\n },\n}\n"
|
||||
],
|
||||
"kwargs": {},
|
||||
"expected": []
|
||||
},
|
||||
{
|
||||
"fn": "_get_cloud_models_missing",
|
||||
"args": [],
|
||||
"kwargs": {},
|
||||
"expected": []
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,115 @@
|
||||
package main
|
||||
|
||||
// truth_test.go — the parity gate. Replays every case from truth.json (the
|
||||
// shared freeze_truth.py envelope: pinned clock + pinned HOME + Python
|
||||
// oracle cases WITH inputs, generated by scripts/gen_model_connect_truth.py
|
||||
// against the live oracle under scripts/mconn_freeze_seams) VERBATIM through
|
||||
// the ported Go functions. A divergence fails the port, not the oracle.
|
||||
|
||||
import (
|
||||
_ "embed"
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
//go:embed truth.json
|
||||
var frozenTruthJSON string
|
||||
|
||||
// TestFrozenTruthReplay — the frozen oracle truth, verbatim, in the recorded
|
||||
// call order, against the ported functions.
|
||||
func TestFrozenTruthReplay(t *testing.T) {
|
||||
var env struct {
|
||||
FrozenNow string `json:"frozen_now"`
|
||||
Cases []frozenCase `json:"cases"`
|
||||
}
|
||||
if err := json.Unmarshal([]byte(frozenTruthJSON), &env); err != nil {
|
||||
t.Fatalf("decode truth envelope: %v", err)
|
||||
}
|
||||
if len(env.Cases) == 0 {
|
||||
t.Fatal("no truth cases — the parity gate is empty")
|
||||
}
|
||||
|
||||
savedClock := nowClock
|
||||
t.Setenv("MCONN_CLOCK", env.FrozenNow)
|
||||
nowClock = func() string { return env.FrozenNow }
|
||||
t.Cleanup(func() { nowClock = savedClock })
|
||||
|
||||
seedHome := func(fixture string) string {
|
||||
home := t.TempDir()
|
||||
cfgDir := filepath.Join(home, ".config", "mimocode")
|
||||
if err := os.MkdirAll(cfgDir, 0o755); err != nil {
|
||||
t.Fatalf("seed home: %v", err)
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(cfgDir, "mimocode.jsonc"), []byte(fixture), 0o644); err != nil {
|
||||
t.Fatalf("seed config: %v", err)
|
||||
}
|
||||
return home
|
||||
}
|
||||
missingHome := func() string { return t.TempDir() }
|
||||
|
||||
for i, c := range env.Cases {
|
||||
actual, err := replayCase(c, seedHome, missingHome)
|
||||
if err != nil {
|
||||
t.Fatalf("case %d fn=%s: %v", i, c.Fn, err)
|
||||
}
|
||||
got, err := canonical(actual)
|
||||
if err != nil {
|
||||
t.Fatalf("case %d fn=%s: canonical actual: %v", i, c.Fn, err)
|
||||
}
|
||||
var want interface{}
|
||||
if err := json.Unmarshal(c.Expected, &want); err != nil {
|
||||
t.Fatalf("case %d fn=%s: decode expected: %v", i, c.Fn, err)
|
||||
}
|
||||
gj, _ := json.Marshal(got)
|
||||
wj, _ := json.Marshal(want)
|
||||
if string(gj) != string(wj) {
|
||||
t.Errorf("case %d fn=%s: got %s want %s", i, c.Fn, gj, wj)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestStripJsoncPreservesOracleQuirks — the naive JSONC strip: string-blind
|
||||
// comment removal (URLs break the parse downstream) and trailing commas.
|
||||
func TestStripJsoncPreservesOracleQuirks(t *testing.T) {
|
||||
stripped := stripJsonc("{\n // comment\n \"a\": 1, // inline\n \"b\": [1, 2,],\n}\n")
|
||||
want := "{\n \n \"a\": 1, \n \"b\": [1, 2]}\n"
|
||||
if stripped != want {
|
||||
t.Errorf("stripJsonc:\n got %q\nwant %q", stripped, want)
|
||||
}
|
||||
if _, err := parseOrderedValue(jsonDecoder("{ \"u\": \"http://x\" }")); err == nil {
|
||||
// sanity: plain JSON parses
|
||||
} else {
|
||||
t.Fatalf("plain JSON must parse: %v", err)
|
||||
}
|
||||
// the oracle quirk: a URL inside JSONC loses its closing quote
|
||||
broken := stripJsonc("{ \"u\": \"http://x\" }")
|
||||
if _, err := parseOrderedValue(jsonDecoder(broken)); err == nil {
|
||||
t.Errorf("URL-under-JSONC must fail to parse like the oracle, got %q", broken)
|
||||
}
|
||||
}
|
||||
|
||||
// TestGetCloudModelsPartialOnLateShapeError — a mid-provider shape error
|
||||
// returns the PARTIAL accumulated list (the oracle's try/except wraps the
|
||||
// whole loop), not an empty one.
|
||||
func TestGetCloudModelsPartialOnLateShapeError(t *testing.T) {
|
||||
home := t.TempDir()
|
||||
cfgDir := filepath.Join(home, ".config", "mimocode")
|
||||
if err := os.MkdirAll(cfgDir, 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
fixture := "{\n \"provider\": {\n \"a\": {\"models\": {\"m1\": {}}},\n \"b\": {\"models\": \"not-a-dict\"}\n }\n}\n"
|
||||
if err := os.WriteFile(filepath.Join(cfgDir, "mimocode.jsonc"), []byte(fixture), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got := GetCloudModelsIn(home)
|
||||
if len(got) != 1 || got[0].Name != "a/m1" {
|
||||
t.Errorf("partial cloud list: got %+v want [a/m1]", got)
|
||||
}
|
||||
}
|
||||
|
||||
func jsonDecoder(s string) *json.Decoder {
|
||||
return json.NewDecoder(strings.NewReader(s))
|
||||
}
|
||||
@@ -0,0 +1 @@
|
||||
exit:0
|
||||
@@ -0,0 +1 @@
|
||||
exit:0
|
||||
@@ -0,0 +1 @@
|
||||
exit:0
|
||||
@@ -0,0 +1 @@
|
||||
exit:0
|
||||
@@ -0,0 +1 @@
|
||||
exit:0
|
||||
@@ -0,0 +1 @@
|
||||
exit:0
|
||||
@@ -0,0 +1 @@
|
||||
exit:0
|
||||
@@ -0,0 +1 @@
|
||||
exit:0
|
||||
@@ -0,0 +1 @@
|
||||
exit:0
|
||||
@@ -0,0 +1 @@
|
||||
exit:0
|
||||
@@ -0,0 +1,62 @@
|
||||
# model_connect dichotomy gap-close — verification summary
|
||||
|
||||
- **Module**: model_connect (GO port, src/go/model-connect)
|
||||
- **Worktree**: /mnt/workdrive/rune-wave-cl (gitea/master base 64cc5125)
|
||||
- **Parity SHA**: see SHAs below (feat commit; filled at push time)
|
||||
|
||||
## Coverage
|
||||
|
||||
Dichotomy audit: model_connect **7/11 PARTIAL → 11/11 FULL** (GO).
|
||||
Ported name-preserving: `CtxBudget` (_ctx_budget), `Now` (_now),
|
||||
`GetCloudModels` (_get_cloud_models), `HandleHook` (handle_hook).
|
||||
Oracle quirks preserved (all frozen on the oracle shape):
|
||||
- JSONC `//` strip is string-blind — a URL inside the config loses its
|
||||
closing quote and the parse fails → `[]` (frozen row `url_corruption`)
|
||||
- mid-provider shape error returns the PARTIAL accumulated list (the
|
||||
oracle's try/except wraps the whole loop; frozen row `broken_late` → `[a/m1]`)
|
||||
- provider/model iteration order is dict insertion order — Go parses with
|
||||
a json.Decoder token walk preserving key order
|
||||
- `null` models dict → `None.items()` raises → partial/empty return
|
||||
- `_ctx_budget` constants from the LIVE model_registry oracle:
|
||||
OUTPUT_RESERVE["default"]=2048, TOOL_RESULT_RESERVE=1024 (non-local only),
|
||||
num_ctx floor 1024, 0-window → 8192 (`or` floor)
|
||||
- `_now` behind the injectable clock seam (no-clock law); battery/parity
|
||||
pin it (MCONN_CLOCK / frozen_now), unpinned default reads wall clock
|
||||
|
||||
## Commands and exit codes
|
||||
|
||||
| Check | Command | Exit | Evidence |
|
||||
|---|---|---|---|
|
||||
| truth generation | `PYTHONPATH=scripts/mconn_freeze_seams:src python3 scripts/gen_model_connect_truth.py` | 0 | `mconn-truth-gen.log/.exit` — 40 cases, `_now` pinned row asserted |
|
||||
| frozen truth byte-identical | `PYTHONPATH=scripts/mconn_freeze_seams:src python3 scripts/freeze_truth.py --verify src/go/model-connect/truth.json --module rune.model_connect` | 0 | `mconn-freeze-verify.log/.exit` — OK: 40 cases byte-identical against live oracle |
|
||||
| gofmt / vet / build | `gofmt -l . && go vet ./... && go build ./...` (src/go/model-connect) | 0 | `mconn-gofmt.log` (empty), `mconn-govet.log`, `mconn-gobuild.log` + `.exit` |
|
||||
| Go battery | `go test -count=1 -v ./...` (src/go/model-connect) | 0 | `mconn-gotest.log/.exit` — 3/3 PASS (TestFrozenTruthReplay verbatim truth.json embed, TestStripJsoncPreservesOracleQuirks, TestGetCloudModelsPartialOnLateShapeError) |
|
||||
| CI replay binary | `./model-connect replay truth.json` | 0 | `mconn-replay.log/.exit` — 40/40 passed |
|
||||
| live parity | `python3 scripts/check_model_connect_parity.py` | 0 | `mconn-parity.log/.exit` — parity: 44/44 match |
|
||||
| Python smoke (live) | `python3 /tmp/mconn-smoke.py` | 0 | `mconn-pysmoke-live.log/.exit` — all 7 ported fns healthy |
|
||||
| Python smoke (python pin) | `RUNE_MODEL_CONNECT_CORE=python python3 /tmp/mconn-smoke.py` | 0 | `mconn-pysmoke-python.log/.exit` — healthy (module has no loader seam; pin vacuous, run per protocol) |
|
||||
| audit | `dichotomy_audit.py` row capture | 0 | `mconn-audit.log/.exit` — 11/11 matched, status FULL |
|
||||
|
||||
Note: model_connect has no dedicated pytest suite and no
|
||||
RUNE_MODEL_CONNECT_CORE loader (plain plugin module — verified by grep);
|
||||
"Python suite green in BOTH pins" is satisfied by the oracle smoke above
|
||||
plus the byte-identity gate, which replays the live oracle itself.
|
||||
|
||||
## Truth rig
|
||||
|
||||
- `scripts/mconn_freeze_seams/sitecustomize.py` — pin_home (hermetic HOME)
|
||||
+ binds `_get_cloud_models_seeded` / `_get_cloud_models_missing` seams
|
||||
onto the oracle module (fixture text rides in the truth rows; the Go
|
||||
battery rebuilds the same rig from the case args via t.TempDir).
|
||||
- Envelope: frozen_now `2026-10-14T08:00:00+00:00`, 40 cases:
|
||||
8 clean / 8 moe / 6 route / 7 ctx_budget / 1 now / 3 hook / 6 seeded
|
||||
cloud configs / 1 missing config.
|
||||
|
||||
## Files
|
||||
|
||||
- `src/go/model-connect/main.go` — 4 gap fns + envelope replay + stdin op
|
||||
dispatch for the parity sweep
|
||||
- `src/go/model-connect/truth_test.go` — battery (verbatim truth embed)
|
||||
- `src/go/model-connect/truth.json` — 40-case freeze_truth envelope
|
||||
- `scripts/gen_model_connect_truth.py`, `scripts/check_model_connect_parity.py`,
|
||||
`scripts/mconn_freeze_seams/sitecustomize.py`
|
||||
@@ -0,0 +1 @@
|
||||
exit:0
|
||||
Reference in new issue
Block a user