feat(context_pressure): close PARTIAL gaps — dichotomy wave
FULL parity port to Rust (9/9 fns, was 3/9): token estimation (codepoint len law), the pressure ladder (raw-gate vs round3 display), degradation signals (int-0 max law, \w+ runs), JSONL entry assembly (dict-spread override), log tail reader (deque(maxlen) + file-iteration law), stats counter (crash parity on non-dict lines), on-message hook (truthiness, message/text/str(data) fallback, pyrepr), CLI dispatch (argparse shapes, .1% renders, code-2 law). File IO + wall clock stay in the Python facade (no-clock law); cps_loader.py is the ctypes seam (RUNE_CPS_CORE / RUNE_CONTEXT_PRESSURE_CORE pins). Frozen truth 75 cases under the cps_freeze_seams rig (byte-identical x2, shared-harness --verify OK 28 envelope cases); branch battery 10/10; live parity frozen=78 regen=6 fuzz=1400 live=18 failures=0; pytest 34/34 both pins; clippy -D warnings + fmt clean; audit 3/9 PARTIAL -> 9/9 FULL (RUST). Evidence: verification/summary-cp.md + cp-* files.
This commit is contained in:
Nick committed
1 parent
c63717adfb
commit
e8f1503028
38 files changed
+3932
-62
No files matched your search
@@ -2,16 +2,16 @@
|
||||
|
||||
_Regenerate with `python3 scripts/dichotomy_audit.py --write`. A module is FULL only when 80%+ of its Python functions have name-preserving counterparts in the Go/Rust implementation — directories without the functions are facades and count as NOT TOUCHED._
|
||||
|
||||
**FULL completion: 57%** (109/192 runtime modules at FULL)
|
||||
**FULL completion: 57%** (110/192 runtime modules at FULL)
|
||||
|
||||
Function-level detail: 71% (1632/2284 runtime functions matched) — secondary; includes in-flight work
|
||||
Function-level detail: 72% (1638/2284 runtime functions matched) — secondary; includes in-flight work
|
||||
|
||||
| Status | Modules | Lines | Meaning |
|
||||
|---|---|---|---|
|
||||
| FULL | 109 | 45774 | 80%+ of functions ported |
|
||||
| PARTIAL | 44 | 17744 | 20–80% — real logic exists, gaps named below |
|
||||
| FULL | 110 | 46125 | 80%+ of functions ported |
|
||||
| PARTIAL | 43 | 17482 | 20–80% — real logic exists, gaps named below |
|
||||
| NOT TOUCHED | 39 | 11471 | no name-preserving implementation |
|
||||
| Total runtime | 192 | 74989 | |
|
||||
| Total runtime | 192 | 75078 | |
|
||||
|
||||
## NOT TOUCHED — the port queue, in priority order (largest first)
|
||||
|
||||
@@ -69,7 +69,6 @@ Function-level detail: 71% (1632/2284 runtime functions matched) — secondary;
|
||||
| time_awareness | 402 | 12/20 | _clock_args, _ta_core, get_time_awareness, to_dict, _to_state, _apply_state |
|
||||
| intent_continuity | 284 | 6/13 | _now, _raise_core_err, to_dict, from_dict, save, _ledger_state |
|
||||
| audit_ledger | 212 | 2/8 | _compute_hash, get_entries, log_promotion, log_gate_result, log_drift_alert, main |
|
||||
| context_pressure | 262 | 3/9 | log_measurement, read_log, log_stats, _now, handle_hook, main |
|
||||
| core_server | 234 | 6/12 | make_server, log_message, _send_json, _send_ndjson, do_GET, do_POST |
|
||||
| progressive_escalation | 279 | 2/8 | execute_with_escalation, test_escalation, handle_hook, to_dict, execute, verify |
|
||||
| voicebox | 164 | 5/11 | _run, start_recording, stop_recording, _transcribe_and_type, on_press, on_release |
|
||||
@@ -170,6 +169,7 @@ Function-level detail: 71% (1632/2284 runtime functions matched) — secondary;
|
||||
| drift_cache | 357 | 12/14 | RUST |
|
||||
| provider_budget | 357 | 20/20 | GO |
|
||||
| drift-ledger | 355 | 14/14 | unclassified |
|
||||
| context_pressure | 351 | 9/9 | RUST |
|
||||
| builder_bridge | 326 | 11/11 | GO |
|
||||
| drift_compression | 323 | 7/7 | RUST |
|
||||
| outcome_ledger | 318 | 7/8 | RUST |
|
||||
|
||||
@@ -0,0 +1,385 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Live parity: context_pressure Rust core vs Python oracle (dichotomy wave,
|
||||
close-PARTIAL gap pass). Layers:
|
||||
|
||||
1. FROZEN — (a) the envelope truth corpus replayed through the ctypes
|
||||
FFI; (b) kernel cases (entry BYTE lines, log parses, stats counts,
|
||||
hook result+entry, CLI dispatch {output, code, entry}) through the
|
||||
FFI; (c) INDEPENDENT oracle-floor regeneration of both sections
|
||||
compared against the frozen file (truth-consumed-not-self-generated).
|
||||
2. FUZZ — 1250+ seeded comparisons (seed 2479): oracle floor vs kernel
|
||||
across estimate/measure/detect/entry-assembly/log-tail/stats-counts/
|
||||
hook (crash parity included, exact str(e) contract).
|
||||
3. LIVE — dual-mode facade battery: the same calls through
|
||||
rune.context_pressure with RUNE_CPS_CORE=python (oracle floor) and
|
||||
with the Rust core live; return docs AND the appended JSONL lines
|
||||
must be identical (the clock is rig-frozen, so lines are byte-stable).
|
||||
|
||||
MUST run under the freeze rig: PYTHONPATH=scripts/cps_freeze_seams (pins
|
||||
the oracle floor + hermetic scratch + frozen clock).
|
||||
Exit 0 only at 100% parity on the gap-closing surface.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import random
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
REPO = Path(__file__).resolve().parents[1]
|
||||
sys.path.insert(0, str(REPO / "scripts"))
|
||||
|
||||
import context_pressure_truth as cpt # noqa: E402
|
||||
|
||||
SCRATCH = cpt.SCRATCH
|
||||
FROZEN_NOW = cpt.FROZEN_NOW
|
||||
|
||||
# Rig guard: refuse to run unpinned (would touch the real ~/.rune).
|
||||
if os.environ.get("RUNE_MEMORY_DIR") != os.path.join(SCRATCH, "memory"):
|
||||
sys.exit("FATAL: run under the rig (PYTHONPATH=scripts/cps_freeze_seams)")
|
||||
if os.environ.get("RUNE_CPS_CORE") != "python":
|
||||
sys.exit("FATAL: rig must pin RUNE_CPS_CORE=python")
|
||||
|
||||
sys.path.insert(0, str(REPO / "src"))
|
||||
|
||||
import rune.context_pressure as cp # noqa: E402
|
||||
from rune import cps_loader # noqa: E402
|
||||
|
||||
if not cps_loader.available():
|
||||
sys.exit("FATAL: Rust core not built — cargo build --release in src/rust/context_pressure")
|
||||
|
||||
TRUTH = json.loads(
|
||||
(REPO / "src" / "rust" / "context_pressure" / "src"
|
||||
/ "frozen_cp_truth.json").read_text())
|
||||
|
||||
fails: list[str] = []
|
||||
counts = {"frozen": 0, "regen": 0, "fuzz": 0, "live": 0}
|
||||
|
||||
|
||||
def fail(layer, name, detail):
|
||||
fails.append(f"[{layer}] {name}: {detail}")
|
||||
print(f"FAIL[{layer}] {name}: {detail}"[:400])
|
||||
|
||||
|
||||
def j(x):
|
||||
return json.dumps(x, sort_keys=True, ensure_ascii=True)
|
||||
|
||||
|
||||
def check(layer, name, want, got):
|
||||
counts[layer] += 1
|
||||
if j(want) != j(got):
|
||||
fail(layer, name, f"want={j(want)[:300]} got={j(got)[:300]}")
|
||||
|
||||
|
||||
def check_crash(layer, name, want, exc):
|
||||
"""want is the frozen '__crash__ TYPE: message' string."""
|
||||
counts[layer] += 1
|
||||
got = f"{type(exc).__name__}: {exc}"
|
||||
if j(want) != j(got):
|
||||
fail(layer, name, f"want={want} got={got}")
|
||||
|
||||
|
||||
def pin(mode):
|
||||
if mode == "python":
|
||||
os.environ["RUNE_CPS_CORE"] = "python"
|
||||
os.environ["RUNE_CONTEXT_PRESSURE_CORE"] = "python"
|
||||
else:
|
||||
os.environ.pop("RUNE_CPS_CORE", None)
|
||||
os.environ.pop("RUNE_CONTEXT_PRESSURE_CORE", None)
|
||||
|
||||
|
||||
# ── layer 1: FROZEN (kernel through the FFI) ──────────────────────────────
|
||||
|
||||
for row in TRUTH.get("cases", []):
|
||||
fn, args, kwargs, want = (row["fn"], row["args"], row.get("kwargs", {}),
|
||||
row["expected"])
|
||||
try:
|
||||
if fn == "estimate_tokens":
|
||||
check("frozen", f"ffi_{fn}:{args[0][:12]}", want,
|
||||
cps_loader.estimate_tokens(args[0]))
|
||||
elif fn == "measure_pressure":
|
||||
check("frozen", f"ffi_{fn}:{args[0][:12]}:{kwargs.get('max_tokens')}",
|
||||
want, cps_loader.measure_pressure(args[0], kwargs.get("max_tokens", 128000)))
|
||||
elif fn == "detect_degradation":
|
||||
check("frozen", f"ffi_{fn}:{args[0][0][:12]}", want,
|
||||
cps_loader.detect_degradation(args[0]))
|
||||
elif fn == "handle_hook":
|
||||
got = cps_loader.handle_hook(args[0], args[1], FROZEN_NOW)
|
||||
check("frozen", f"ffi_{fn}:{args[0]}", want, got["result"])
|
||||
except Exception as e: # noqa: BLE001 — crash parity surfaces here
|
||||
fail("frozen", f"ffi_{fn}", f"{type(e).__name__}: {e}")
|
||||
|
||||
for kind, rows in TRUTH.get("kernel_cases", {}).items():
|
||||
for row in rows:
|
||||
args, expect, name = row["args"], row["expect"], row["name"]
|
||||
if kind == "logMeasurement":
|
||||
entry = cps_loader.log_measurement(args["measurement"],
|
||||
args["session_id"], args["now"])
|
||||
counts["frozen"] += 1
|
||||
# BYTE law: the facade writes json.dumps(entry) + "\n" (default
|
||||
# ensure_ascii) — the frozen line is the oracle's exact bytes.
|
||||
if json.dumps(entry) + "\n" != expect["line"]:
|
||||
fail("frozen", f"{kind}/{name}/line",
|
||||
f"want={expect['line'][:160]} got={json.dumps(entry)[:160]}")
|
||||
elif kind == "readLog":
|
||||
check("frozen", f"{kind}/{name}", expect,
|
||||
cps_loader.read_log(args["content"], args["limit"]))
|
||||
elif kind == "logStats":
|
||||
try:
|
||||
check("frozen", f"{kind}/{name}", expect,
|
||||
cps_loader.log_stats(args["content"]))
|
||||
except Exception as e: # noqa: BLE001
|
||||
if isinstance(expect, dict) and "__crash__" in expect:
|
||||
check_crash("frozen", f"{kind}/{name}", expect["__crash__"], e)
|
||||
else:
|
||||
fail("frozen", f"{kind}/{name}", f"unexpected crash {e}")
|
||||
elif kind == "kernelHook":
|
||||
try:
|
||||
got = cps_loader.handle_hook(args["event"], args["data"], args["now"])
|
||||
if isinstance(expect, dict) and "__crash__" in expect:
|
||||
fail("frozen", f"{kind}/{name}", "kernel did not crash where oracle did")
|
||||
else:
|
||||
check("frozen", f"{kind}/{name}", expect, got)
|
||||
except Exception as e: # noqa: BLE001
|
||||
if isinstance(expect, dict) and "__crash__" in expect:
|
||||
check_crash("frozen", f"{kind}/{name}", expect["__crash__"], e)
|
||||
else:
|
||||
fail("frozen", f"{kind}/{name}", f"unexpected crash {e}")
|
||||
elif kind == "mainDispatch":
|
||||
try:
|
||||
got = cps_loader.main_dispatch(args["argv"], args["content"], args["now"])
|
||||
want = {k: expect[k] for k in ("output", "code") if k in expect}
|
||||
gotv = {k: got[k] for k in ("output", "code") if k in got}
|
||||
check("frozen", f"{kind}/{name}", want, gotv)
|
||||
if "entry" in expect:
|
||||
check("frozen", f"{kind}/{name}/entry", expect["entry"], got.get("entry"))
|
||||
except Exception as e: # noqa: BLE001
|
||||
fail("frozen", f"{kind}/{name}", f"unexpected crash {e}")
|
||||
|
||||
# ── layer 1c: independent oracle-floor regeneration vs the frozen file ────
|
||||
|
||||
env_regen = cpt.regen_envelope_cases()
|
||||
check("regen", "envelope_cases", TRUTH["cases"], env_regen)
|
||||
kc_regen = cpt.regen_kernel_cases()
|
||||
for kind, rows in TRUTH["kernel_cases"].items():
|
||||
check("regen", f"kernel_cases:{kind}", rows, kc_regen[kind])
|
||||
|
||||
# ── layer 2: FUZZ (oracle floor vs kernel, seeded) ────────────────────────
|
||||
|
||||
rng = random.Random(2479)
|
||||
VOCAB = ["hello", "world", "éé", "üü", "dup", "word", "👍", "a_b_c",
|
||||
"MiXeD", "runner", "label", "law", "123"]
|
||||
WORDS = ["dup", "word", "pair", "echo", "alpha", "beta"]
|
||||
|
||||
|
||||
def rand_text():
|
||||
n = rng.randrange(0, 12)
|
||||
return " ".join(rng.choice(VOCAB) for _ in range(n))
|
||||
|
||||
|
||||
def rand_measurement():
|
||||
if rng.random() < 0.5:
|
||||
return cp.measure_pressure(rand_text())
|
||||
base = rng.choice(["tokens", "user_key", "note"])
|
||||
doc = {base: rng.randrange(0, 50)}
|
||||
if rng.random() < 0.7:
|
||||
doc["pressure"] = rng.choice([0.0, 0.5, 1.0, 0.666, 2.5])
|
||||
if rng.random() < 0.5:
|
||||
doc["status"] = rng.choice(["ok", "warning", "critical", None])
|
||||
return doc
|
||||
|
||||
|
||||
for trial in range(200):
|
||||
text = rand_text()
|
||||
check("fuzz", f"estimate:{trial}", cp.estimate_tokens(text),
|
||||
cps_loader.estimate_tokens(text))
|
||||
|
||||
for trial in range(400):
|
||||
text = rand_text()
|
||||
max_tokens = rng.choice([1, 2, 3, 7, 12000, 20000, 128000, 99999])
|
||||
w_err = g_err = None
|
||||
try:
|
||||
w = cp.measure_pressure(text, max_tokens=max_tokens)
|
||||
except Exception as e: # noqa: BLE001
|
||||
w, w_err = None, f"{type(e).__name__}: {e}"
|
||||
try:
|
||||
g = cps_loader.measure_pressure(text, max_tokens)
|
||||
except Exception as e: # noqa: BLE001
|
||||
g, g_err = None, f"{type(e).__name__}: {e}"
|
||||
counts["fuzz"] += 1
|
||||
if w_err or g_err:
|
||||
if w_err != g_err:
|
||||
fail("fuzz", f"measure:{trial}", f"crash mismatch {w_err} vs {g_err}")
|
||||
elif j(w) != j(g):
|
||||
fail("fuzz", f"measure:{trial}", f"want={j(w)[:200]} got={j(g)[:200]}")
|
||||
|
||||
for trial in range(200):
|
||||
n = rng.randrange(2, 9)
|
||||
responses = []
|
||||
for i in range(n):
|
||||
if i == 0 and rng.random() < 0.5:
|
||||
responses.append(rand_text())
|
||||
else:
|
||||
responses.append(" ".join(rng.choice(WORDS)
|
||||
for _ in range(rng.randrange(1, 12))))
|
||||
check("fuzz", f"detect:{trial}", cp.detect_degradation(responses),
|
||||
cps_loader.detect_degradation(responses))
|
||||
|
||||
for trial in range(200):
|
||||
m = rand_measurement()
|
||||
sid = rng.choice(["", "session A", "s" * 40])
|
||||
want = {"timestamp": FROZEN_NOW, "session_id": sid, **m}
|
||||
got = cps_loader.log_measurement(m, sid, FROZEN_NOW)
|
||||
counts["fuzz"] += 1
|
||||
if j(want) != j(got):
|
||||
fail("fuzz", f"log_measurement:{trial}", f"want={j(want)[:200]} got={j(got)[:200]}")
|
||||
|
||||
LOG_POOL = [
|
||||
'{"tokens": 3, "pressure": 0.5, "status": "warning"}',
|
||||
'{"tokens": 1, "pressure": 0.0, "status": "ok"}',
|
||||
'{"status": "critical"}',
|
||||
"not json at all",
|
||||
"5",
|
||||
"",
|
||||
' {"padded": true} ',
|
||||
"[]",
|
||||
'"str line"',
|
||||
]
|
||||
|
||||
for trial in range(150):
|
||||
n = rng.randrange(0, 10)
|
||||
content = "\n".join(rng.choice(LOG_POOL) for _ in range(n))
|
||||
if content and rng.random() < 0.5:
|
||||
content += "\n"
|
||||
limit = rng.choice([-2, 0, 1, 2, 3, 5, 20])
|
||||
cpt._seed_log(content)
|
||||
w = cp.read_log(limit=limit)
|
||||
g = cps_loader.read_log(content, limit)
|
||||
counts["fuzz"] += 1
|
||||
if j(w) != j(g):
|
||||
fail("fuzz", f"read_log:{trial}:lim{limit}", f"want={j(w)[:200]} got={j(g)[:200]}")
|
||||
|
||||
for trial in range(100):
|
||||
n = rng.randrange(0, 8)
|
||||
content = "\n".join(rng.choice(LOG_POOL) for _ in range(n))
|
||||
if content and rng.random() < 0.7:
|
||||
content += "\n"
|
||||
cpt._seed_log(content)
|
||||
try:
|
||||
w = cp.log_stats()
|
||||
w_err = None
|
||||
except Exception as e: # noqa: BLE001
|
||||
w, w_err = None, f"{type(e).__name__}: {e}"
|
||||
try:
|
||||
g = cps_loader.log_stats(content)
|
||||
g_err = None
|
||||
except Exception as e: # noqa: BLE001
|
||||
g, g_err = None, f"{type(e).__name__}: {e}"
|
||||
counts["fuzz"] += 1
|
||||
if w_err or g_err:
|
||||
if w_err != g_err:
|
||||
fail("fuzz", f"log_stats:{trial}", f"crash mismatch {w_err} vs {g_err}")
|
||||
elif j(w) != j(g):
|
||||
fail("fuzz", f"log_stats:{trial}", f"want={j(w)[:200]} got={j(g)[:200]}")
|
||||
|
||||
HOOK_EVENTS = ["on_message", "on_message", "on_message", "other_event", ""]
|
||||
HOOK_DATAS = [
|
||||
{"message": "hello"},
|
||||
{"message": "x" * 95000},
|
||||
{"text": "hello world"},
|
||||
{"message": "", "text": "ok text"},
|
||||
{"other": 1},
|
||||
{"a": True, "n": None},
|
||||
{"k": "a'b"},
|
||||
{},
|
||||
None,
|
||||
[1, 2],
|
||||
True,
|
||||
5,
|
||||
"plain str",
|
||||
]
|
||||
|
||||
for trial in range(150):
|
||||
event = rng.choice(HOOK_EVENTS)
|
||||
data = rng.choice(HOOK_DATAS)
|
||||
seed_data = json.loads(json.dumps(data)) if isinstance(data, (dict, list, str)) else data
|
||||
try:
|
||||
cpt._seed_log("")
|
||||
w = cp.handle_hook(event, seed_data)
|
||||
w_entry = None
|
||||
for line in cpt._read_log_file().splitlines():
|
||||
if line.strip():
|
||||
w_entry = json.loads(line)
|
||||
w_err = None
|
||||
except Exception as e: # noqa: BLE001
|
||||
w, w_entry, w_err = None, None, f"{type(e).__name__}: {e}"
|
||||
try:
|
||||
g = cps_loader.handle_hook(event, data, FROZEN_NOW)
|
||||
g_err = None
|
||||
except Exception as e: # noqa: BLE001
|
||||
g, g_err = None, f"{type(e).__name__}: {e}"
|
||||
counts["fuzz"] += 1
|
||||
if w_err or g_err:
|
||||
if w_err != g_err:
|
||||
fail("fuzz", f"hook:{trial}:{event}", f"crash mismatch {w_err} vs {g_err}")
|
||||
else:
|
||||
g_entry = g.get("entry")
|
||||
want = {"result": w, "entry": w_entry}
|
||||
got = {"result": g.get("result"), "entry": g_entry}
|
||||
if j(want) != j(got):
|
||||
fail("fuzz", f"hook:{trial}:{event}", f"want={j(want)[:260]} got={j(got)[:260]}")
|
||||
|
||||
# ── layer 3: LIVE (dual-mode facade battery) ──────────────────────────────
|
||||
|
||||
BIG = "x" * 95000
|
||||
CASES = [
|
||||
("estimate_unicode", lambda: cp.estimate_tokens("héllo wörld 👍")),
|
||||
("measure_ok", lambda: cp.measure_pressure("hello world")),
|
||||
("measure_warning", lambda: cp.measure_pressure("x" * 49000, max_tokens=20000)),
|
||||
("measure_critical", lambda: cp.measure_pressure(BIG, max_tokens=12000)),
|
||||
("measure_empty", lambda: cp.measure_pressure("")),
|
||||
("detect_trio", lambda: cp.detect_degradation(["aaa", "bbb", "ccc"])),
|
||||
("detect_decline", lambda: cp.detect_degradation(["x" * 200, "y" * 180, "", "", ""])),
|
||||
("detect_insufficient", lambda: cp.detect_degradation(["a", "b"])),
|
||||
("read_log", lambda: cpt._seed_log('{"a": 1}\nbroken\n{"b": 2}\n')
|
||||
or cp.read_log(limit=2)),
|
||||
("log_stats", lambda: cpt._seed_log(
|
||||
'{"status": "warning"}\n{"status": "critical"}\nbroken\n') or cp.log_stats()),
|
||||
("log_stats_empty", lambda: cpt._seed_log("") or cp.log_stats()),
|
||||
("log_measurement", lambda: cpt._seed_log("") or cp.log_measurement(
|
||||
{"tokens": 3, "max_tokens": 128000, "pressure": 0.0, "status": "ok",
|
||||
"recommendation": "none"}, "sid-1") or cpt._read_log_file()),
|
||||
("log_measurement_override", lambda: cpt._seed_log("") or cp.log_measurement(
|
||||
{"timestamp": "T-override", "user_key": "u"}, "") or cpt._read_log_file()),
|
||||
("hook_ok", lambda: cpt._seed_log("") or cp.handle_hook("on_message", {"text": "hello"})),
|
||||
("hook_critical", lambda: cpt._seed_log("") or cp.handle_hook(
|
||||
"on_message", {"message": BIG})),
|
||||
("hook_str_data", lambda: cpt._seed_log("") or cp.handle_hook(
|
||||
"on_message", {"other": 1})),
|
||||
("hook_empty", lambda: cpt._seed_log("") or cp.handle_hook("on_message", {})),
|
||||
("hook_wrong_event", lambda: cpt._seed_log("") or cp.handle_hook(
|
||||
"ping", {"message": BIG})),
|
||||
]
|
||||
|
||||
results = {}
|
||||
for mode in ("python", "rust"):
|
||||
pin(mode)
|
||||
mode_out = {}
|
||||
for name, fn in CASES:
|
||||
try:
|
||||
mode_out[name] = fn()
|
||||
except Exception as e: # noqa: BLE001 — crash parity surfaces here
|
||||
mode_out[name] = {"__crash__": f"{type(e).__name__}: {e}"}
|
||||
results[mode] = mode_out
|
||||
pin("python")
|
||||
|
||||
for name, _ in CASES:
|
||||
counts["live"] += 1
|
||||
w, g = results["python"][name], results["rust"][name]
|
||||
if j(w) != j(g):
|
||||
fail("live", f"dual:{name}", f"want={j(w)[:260]} got={j(g)[:260]}")
|
||||
|
||||
print(f"parity: frozen={counts['frozen']} regen={counts['regen']} "
|
||||
f"fuzz={counts['fuzz']} live={counts['live']} failures={len(fails)}")
|
||||
sys.exit(1 if fails else 0)
|
||||
@@ -0,0 +1,351 @@
|
||||
#!/usr/bin/env python3
|
||||
"""context_pressure_truth — shared capture/regeneration helpers for the
|
||||
context_pressure dichotomy gap-close (issue #2479 tranche).
|
||||
|
||||
Used by scripts/gen_context_pressure_truth.py (freeze) and
|
||||
scripts/check_context_pressure_parity.py (independent regeneration for the
|
||||
truth-consumed-not-self-generated gate). EVERY process that imports this
|
||||
module MUST run under the scripts/cps_freeze_seams rig
|
||||
(PYTHONPATH=scripts/cps_freeze_seams) so:
|
||||
|
||||
* RUNE_CPS_CORE/RUNE_CONTEXT_PRESSURE_CORE=python — poisoned-truth law:
|
||||
the ORACLE FLOOR captures/regenerates every expectation.
|
||||
* datetime is frozen to CPS_FROZEN_NOW — entry lines are byte-stable.
|
||||
* RUNE_MEMORY_DIR points at the hermetic scratch; LOG_PATH (resolved at
|
||||
oracle import) lives there. CPS_SEED_LOG injects log file content for
|
||||
CLI subprocess rows (the rig wipes and reseeds at import).
|
||||
|
||||
Kernel case shapes:
|
||||
logMeasurement args {measurement, session_id, now} → expect {line}
|
||||
readLog args {content, limit} → expect [entries]
|
||||
logStats args {content} → expect counts | __crash__
|
||||
kernelHook args {event, data, now} → expect {result, entry} | __crash__
|
||||
mainDispatch args {argv, content, now} → expect {output, code} [, entry doc] | {output, code}
|
||||
Crash rows store "__crash__ TYPE: message" — the exact str(e) contract.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import copy
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
REPO = Path(__file__).resolve().parents[1]
|
||||
|
||||
FROZEN_NOW = os.environ.get("CPS_FROZEN_NOW", "2026-10-14T01:02:03.456789+00:00")
|
||||
SCRATCH = os.environ.get("CPS_FREEZE_SCRATCH", "/tmp/cps_gap_freeze")
|
||||
LOG_FILE = os.path.join(SCRATCH, "memory", "index", "context_pressure.jsonl")
|
||||
ORACLE = REPO / "src" / "rune" / "context_pressure.py"
|
||||
RIG_DIR = REPO / "scripts" / "cps_freeze_seams"
|
||||
|
||||
|
||||
def _oracle():
|
||||
import rune.context_pressure as cp
|
||||
|
||||
return cp
|
||||
|
||||
|
||||
def _seed_log(content: str) -> None:
|
||||
os.makedirs(os.path.dirname(LOG_FILE), exist_ok=True)
|
||||
with open(LOG_FILE, "w") as f:
|
||||
f.write(content)
|
||||
|
||||
|
||||
def _read_log_file() -> str:
|
||||
try:
|
||||
with open(LOG_FILE, "r") as f:
|
||||
return f.read()
|
||||
except OSError:
|
||||
return ""
|
||||
|
||||
|
||||
# ── kernel-case capture (oracle floor) ────────────────────────────────────
|
||||
|
||||
|
||||
def capture_log_measurement(measurement, session_id, now):
|
||||
_seed_log("")
|
||||
cp = _oracle()
|
||||
cp.log_measurement(measurement, session_id)
|
||||
return {"line": _read_log_file()}
|
||||
|
||||
|
||||
def capture_read_log(content, limit):
|
||||
_seed_log(content)
|
||||
cp = _oracle()
|
||||
return cp.read_log(limit=limit)
|
||||
|
||||
|
||||
def capture_log_stats(content):
|
||||
_seed_log(content)
|
||||
cp = _oracle()
|
||||
try:
|
||||
return cp.log_stats()
|
||||
except Exception as e: # noqa: BLE001 — crash parity capture
|
||||
return {"__crash__": f"{type(e).__name__}: {e}"}
|
||||
|
||||
|
||||
def capture_hook(event, data):
|
||||
_seed_log("")
|
||||
cp = _oracle()
|
||||
try:
|
||||
result = cp.handle_hook(event, data)
|
||||
except Exception as e: # noqa: BLE001 — crash parity capture
|
||||
return {"__crash__": f"{type(e).__name__}: {e}"}
|
||||
entry = None
|
||||
for line in _read_log_file().splitlines():
|
||||
if line.strip():
|
||||
entry = json.loads(line)
|
||||
return {"result": result, "entry": entry}
|
||||
|
||||
|
||||
def capture_cli(argv, seed_content):
|
||||
"""Run the real CLI (src/rune/context_pressure.py) under the rig; return
|
||||
{output, code} plus the entry line the hook arm appended (if any).
|
||||
|
||||
`argv` carries the synthetic program name at [0] (parse_cli law: the
|
||||
kernel skips argv[0] exactly like the oracle skips sys.argv[0]); the
|
||||
subprocess receives argv[1:]."""
|
||||
_seed_log(seed_content)
|
||||
env = dict(os.environ)
|
||||
rig = str(RIG_DIR)
|
||||
if rig not in env.get("PYTHONPATH", ""):
|
||||
env["PYTHONPATH"] = rig + (":" + env["PYTHONPATH"] if env.get("PYTHONPATH") else "")
|
||||
# CLI subprocess rows: the scratch lives in the /tmp rig layout — the
|
||||
# seeded content rides CPS_SEED_LOG (the subprocess's rig re-seeds it
|
||||
# after its scratch wipe).
|
||||
env["CPS_SEED_LOG"] = seed_content
|
||||
env["RUNE_MEMORY_DIR"] = os.path.join(SCRATCH, "memory")
|
||||
proc = subprocess.run(
|
||||
[sys.executable, str(ORACLE), *argv[1:]],
|
||||
capture_output=True, text=True, env=env,
|
||||
)
|
||||
out = {"output": proc.stdout[:-1] if proc.stdout.endswith("\n") else proc.stdout,
|
||||
"code": proc.returncode}
|
||||
if any(a == "hook" for a in argv):
|
||||
entry = None
|
||||
for line in _read_log_file().splitlines():
|
||||
if line.strip():
|
||||
try:
|
||||
entry = json.loads(line)
|
||||
except json.JSONDecodeError:
|
||||
continue
|
||||
out["entry"] = entry
|
||||
return out
|
||||
|
||||
|
||||
# ── the corpora ───────────────────────────────────────────────────────────
|
||||
|
||||
def envelope_case_specs():
|
||||
"""The pure-surface rows (regenerated independently by
|
||||
freeze_truth.py --verify): (fn, args, kwargs)."""
|
||||
big = "x" * 95000 # ok pressure at default max_tokens
|
||||
return [
|
||||
("estimate_tokens", ("",), {}),
|
||||
("estimate_tokens", ("hello world",), {}),
|
||||
("estimate_tokens", ("héllo wörld",), {}),
|
||||
("estimate_tokens", ("👍👍",), {}),
|
||||
("estimate_tokens", ("x" * 1000,), {}),
|
||||
("measure_pressure", ("",), {}),
|
||||
("measure_pressure", ("hello world",), {}),
|
||||
("measure_pressure", ("x" * 100000,), {}),
|
||||
("measure_pressure", ("x" * 49000,), {"max_tokens": 20000}),
|
||||
("measure_pressure", ("x" * 17430,), {"max_tokens": 8000}),
|
||||
("measure_pressure", ("x" * 9000,), {"max_tokens": 12000}),
|
||||
("measure_pressure", ("x" * 10830,), {"max_tokens": 12000}),
|
||||
("measure_pressure", (big,), {"max_tokens": 12000}),
|
||||
("detect_degradation", (["a", "b"],), {}),
|
||||
("detect_degradation", (["aaa", "bbb", "ccc"],), {}),
|
||||
("detect_degradation", (["x" * 200, "y" * 180, "", "", ""],), {}),
|
||||
("detect_degradation", (["dup word word", "word dup word", "word word dup"],), {}),
|
||||
("detect_degradation", (["a b c d", "a b c d", "a b c d"],), {}),
|
||||
("detect_degradation", (["z" * 99, "z" * 100, "z" * 101],), {}),
|
||||
("detect_degradation", (["éé üü éé", "üü éé üü", "éé üü"],), {}),
|
||||
("detect_degradation", (["short", "tiny", "x" * 300],), {}),
|
||||
("handle_hook", ("on_message", {"message": big}), {}),
|
||||
("handle_hook", ("on_message", {"message": "x" * 403500}), {}),
|
||||
("handle_hook", ("on_message", {"text": "hello"}), {}),
|
||||
("handle_hook", ("on_message", {"other": 1}), {}),
|
||||
("handle_hook", ("on_message", {}), {}),
|
||||
("handle_hook", ("on_message", None), {}),
|
||||
("handle_hook", ("other_event", {"message": big}), {}),
|
||||
]
|
||||
|
||||
|
||||
def kernel_case_specs():
|
||||
"""The kernel rows (captured from the oracle floor; replayed by the
|
||||
Rust battery, the loader FFI and regenerated independently here)."""
|
||||
m_ok = _oracle_measure("hello world")
|
||||
m_warn = _oracle_measure("x" * 49000, 20000)
|
||||
m_crit = _oracle_measure("x" * 95000, 12000)
|
||||
d_int0 = _pure_detect(["aaa", "bbb", "ccc"])
|
||||
return {
|
||||
"logMeasurement": [
|
||||
("tokens_and_zero_pressure", m_ok, ""),
|
||||
("warning_pressure", m_warn, ""),
|
||||
("critical_pressure", m_crit, ""),
|
||||
("int_zero_fields", d_int0, ""),
|
||||
("session_id", m_ok, "session 9"),
|
||||
("override_and_extra",
|
||||
{"timestamp": "T-override", "session_id": "SID-OVR",
|
||||
"user_key": "u", "tokens": 42, "pressure": 0.5,
|
||||
"status": "warning"}, ""),
|
||||
("unicode_escape_probe",
|
||||
{"note": "éé", "pressure": 0.0, "status": "ok"}, ""),
|
||||
],
|
||||
"readLog": [
|
||||
("read_all", "e1\ne2\n", 20),
|
||||
("tail_two_of_four", "l1\nl2\nl3\nl4\n", 2),
|
||||
("blank_and_invalid", '{"a": 1}\n\nnot json\n{"b": 2}\n', 20),
|
||||
("limit_zero", "e1\ne2\n", 0),
|
||||
("trailing_newline_no_phantom", "a\nb\nc\n", 2),
|
||||
("float_pressure", '{"tokens": 3, "pressure": 0.5}\n', 20),
|
||||
("int_line", "5\n", 20),
|
||||
],
|
||||
"logStats": [
|
||||
("mixed_counts",
|
||||
'{"status": "warning"}\n{"status": "warning"}\n'
|
||||
'{"status": "warning"}\n{"status": "critical"}\n'
|
||||
'{"status": "critical"}\n{"status": "ok"}\n\nbroken\n'),
|
||||
("empty", ""),
|
||||
("non_dict_line_crash", "5\n"),
|
||||
],
|
||||
"kernelHook": [
|
||||
("critical_message", "on_message", {"message": "x" * 403500}),
|
||||
("ok_text", "on_message", {"text": "hello"}),
|
||||
("str_data_fallback", "on_message", {"other": 1}),
|
||||
("both_empty_fallback", "on_message", {"message": "", "text": ""}),
|
||||
("empty_data", "on_message", {}),
|
||||
("wrong_event", "other_event", {"message": "hello"}),
|
||||
("repr_bool_none", "on_message", {"a": True, "n": None}),
|
||||
("repr_quote_probe", "on_message", {"k": "a'b"}),
|
||||
("message_int_crash", "on_message", {"message": 5}),
|
||||
("list_data_crash", "on_message", [1, 2]),
|
||||
("bool_data_crash", "on_message", True),
|
||||
("int_data_crash", "on_message", 5),
|
||||
("null_data", "on_message", None),
|
||||
],
|
||||
"mainDispatch": [
|
||||
("measure_plain", ["cp", "measure", "hello world"], ""),
|
||||
("measure_json", ["cp", "measure", "hello world", "--json"], ""),
|
||||
("measure_max_tokens", ["cp", "measure", "x" * 49000,
|
||||
"--max-tokens", "20000"], ""),
|
||||
("degrade_plain", ["cp", "degrade", "a b c|a b c|a b"], ""),
|
||||
("degrade_json", ["cp", "degrade", "dup word|dup word|dup word",
|
||||
"--json"], ""),
|
||||
("log_plain", ["cp", "log", "--limit", "2"],
|
||||
'{"tokens": 3, "pressure": 0.5, "status": "warning", "timestamp": "2026-10-14T01:02:03.456789+00:00"}\n{"tokens": 1, "pressure": 0.0, "status": "ok", "timestamp": "x"}\nbroken\n'),
|
||||
("log_json", ["cp", "log", "--json"],
|
||||
'{"pressure": 0.5, "status": "warning"}\n'),
|
||||
("stats_plain", ["cp", "stats"],
|
||||
'{"status": "warning"}\n{"status": "critical"}\n{"status": "ok"}\n'),
|
||||
("stats_json", ["cp", "stats", "--json"],
|
||||
'{"status": "warning"}\n'),
|
||||
("hook_ok", ["cp", "hook", "--event", "on_message",
|
||||
"--data", '{"message": "hello"}'], ""),
|
||||
("hook_critical", ["cp", "hook", "--event", "on_message",
|
||||
"--data", '{"message": "' + "x" * 95000 + '"}'], ""),
|
||||
("hook_default_event", ["cp", "hook"], ""),
|
||||
("unknown_cmd", ["cp", "bogus"], ""),
|
||||
("no_cmd", ["cp"], ""),
|
||||
("unknown_flag", ["cp", "measure", "x", "--bogus"], ""),
|
||||
("bad_int", ["cp", "measure", "--max-tokens", "abc"], ""),
|
||||
("extra_positional", ["cp", "measure", "a", "b"], ""),
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
def _oracle_measure(text, max_tokens=128000):
|
||||
"""The oracle floor with the core pin active (poisoned-truth law:
|
||||
the LIVE module computes every measurement)."""
|
||||
return _oracle().measure_pressure(text, max_tokens=max_tokens)
|
||||
|
||||
|
||||
def _pure_detect(responses):
|
||||
return _oracle().detect_degradation(responses)
|
||||
|
||||
|
||||
def regen_kernel_cases():
|
||||
"""Regenerate every kernel row from the live oracle floor."""
|
||||
specs = kernel_case_specs()
|
||||
rows = {"logMeasurement": [], "readLog": [], "logStats": [],
|
||||
"kernelHook": [], "mainDispatch": []}
|
||||
for name, m, sid in specs["logMeasurement"]:
|
||||
rows["logMeasurement"].append({
|
||||
"fn": "log_measurement", "name": name,
|
||||
"args": {"measurement": m, "session_id": sid, "now": FROZEN_NOW},
|
||||
"expect": capture_log_measurement(copy.deepcopy(m), sid, FROZEN_NOW),
|
||||
})
|
||||
for name, content, limit in specs["readLog"]:
|
||||
rows["readLog"].append({
|
||||
"fn": "read_log", "name": name,
|
||||
"args": {"content": content, "limit": limit},
|
||||
"expect": capture_read_log(content, limit),
|
||||
})
|
||||
for name, content in specs["logStats"]:
|
||||
rows["logStats"].append({
|
||||
"fn": "log_stats", "name": name,
|
||||
"args": {"content": content},
|
||||
"expect": capture_log_stats(content),
|
||||
})
|
||||
for name, event, data in specs["kernelHook"]:
|
||||
rows["kernelHook"].append({
|
||||
"fn": "handle_hook", "name": name,
|
||||
"args": {"event": event, "data": data, "now": FROZEN_NOW},
|
||||
"expect": capture_hook(event, copy.deepcopy(data)),
|
||||
})
|
||||
for name, argv, seed in specs["mainDispatch"]:
|
||||
rows["mainDispatch"].append({
|
||||
"fn": "main_dispatch", "name": name,
|
||||
"args": {"argv": argv, "content": seed, "now": FROZEN_NOW},
|
||||
"expect": capture_cli(argv, seed),
|
||||
})
|
||||
return rows
|
||||
|
||||
|
||||
def regen_envelope_cases():
|
||||
"""Drive the pure-surface rows against the live oracle (in-rig)."""
|
||||
cp = _oracle()
|
||||
rows = []
|
||||
for fn, args, kwargs in envelope_case_specs():
|
||||
a = copy.deepcopy(args)
|
||||
k = copy.deepcopy(kwargs)
|
||||
result = getattr(cp, fn)(*a, **k)
|
||||
rows.append({"fn": fn, "args": list(a), "kwargs": k,
|
||||
"expected": copy.deepcopy(result)})
|
||||
return rows
|
||||
|
||||
|
||||
def write_envelope(path: Path) -> int:
|
||||
cases = regen_envelope_cases()
|
||||
kcases = regen_kernel_cases()
|
||||
doc = {
|
||||
"module": "context_pressure",
|
||||
"frozen_now": FROZEN_NOW,
|
||||
"cases": cases,
|
||||
"kernel_cases": kcases,
|
||||
}
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(json.dumps(doc, indent=2, ensure_ascii=False, sort_keys=False) + "\n")
|
||||
total = len(cases) + sum(len(v) for v in kcases.values())
|
||||
return total
|
||||
|
||||
|
||||
def kernel_cases_total() -> int:
|
||||
specs = kernel_case_specs()
|
||||
return sum(len(v) for v in specs.values())
|
||||
|
||||
|
||||
def main() -> int:
|
||||
out = REPO / "src" / "rust" / "context_pressure" / "src" / "frozen_cp_truth.json"
|
||||
total = write_envelope(out)
|
||||
print(f"frozen {total} cases -> {out}")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
if str(REPO / "scripts") not in sys.path:
|
||||
sys.path.insert(0, str(REPO / "scripts"))
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,51 @@
|
||||
"""sitecustomize — the context_pressure freeze rig (dichotomy wave gap close).
|
||||
|
||||
Auto-imported via PYTHONPATH=scripts/cps_freeze_seams by BOTH the truth
|
||||
generator (scripts/gen_context_pressure_truth.py), the CLI subprocess rows,
|
||||
and `freeze_truth.py --verify`, so frozen truth is byte-reproducible in any
|
||||
process — and the --verify law (replay against the pristine oracle) holds:
|
||||
* RUNE_CPS_CORE=python + RUNE_CONTEXT_PRESSURE_CORE=python — the oracle
|
||||
runs its pure-Python floor (kernel delegation pinned OFF; poisoned-
|
||||
truth law).
|
||||
* datetime.datetime is FROZEN to CPS_FROZEN_NOW — the oracle reads
|
||||
datetime.now(timezone.utc) in _now() at every entry-assembly site; the
|
||||
frozen truth carries the constant instant.
|
||||
* RUNE_MEMORY_DIR points at a hermetic FIXED-path scratch wiped at
|
||||
import (LOG_PATH resolves at oracle import; every process starts from
|
||||
the same empty log).
|
||||
"""
|
||||
|
||||
import os
|
||||
import shutil
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
REPO = Path(os.environ.get("CPS_FREEZE_REPO", Path.cwd()))
|
||||
|
||||
os.environ.setdefault("RUNE_CPS_CORE", "python")
|
||||
os.environ.setdefault("RUNE_CONTEXT_PRESSURE_CORE", "python")
|
||||
|
||||
SCRATCH = os.environ.get("CPS_FREEZE_SCRATCH", "/tmp/cps_gap_freeze")
|
||||
if os.path.isdir(SCRATCH):
|
||||
shutil.rmtree(SCRATCH)
|
||||
os.makedirs(os.path.join(SCRATCH, "memory", "index"), exist_ok=True)
|
||||
os.environ["RUNE_MEMORY_DIR"] = os.path.join(SCRATCH, "memory")
|
||||
|
||||
# CPS_SEED_LOG: log-file content for CLI subprocess rows (the CLI reads
|
||||
# LOG_PATH, which the wipe just recreated — reseed right before the import).
|
||||
_SEED_LOG = os.environ.get("CPS_SEED_LOG")
|
||||
if _SEED_LOG is not None:
|
||||
with open(os.path.join(SCRATCH, "memory", "index",
|
||||
"context_pressure.jsonl"), "w") as _f:
|
||||
_f.write(_SEED_LOG)
|
||||
|
||||
FROZEN_NOW = os.environ.get("CPS_FROZEN_NOW", "2026-10-14T01:02:03.456789+00:00")
|
||||
|
||||
if str(REPO / "scripts") not in sys.path:
|
||||
sys.path.insert(0, str(REPO / "scripts"))
|
||||
from freeze_rigs import pin_clock # noqa: E402
|
||||
|
||||
pin_clock(FROZEN_NOW)
|
||||
|
||||
if str(REPO / "src") not in sys.path:
|
||||
sys.path.insert(0, str(REPO / "src"))
|
||||
@@ -0,0 +1,34 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Freeze the dichotomy-wave truth corpus for src/rune/context_pressure.py.
|
||||
|
||||
LAW (poisoned-truth): RUNE_CPS_CORE / RUNE_CONTEXT_PRESSURE_CORE=python —
|
||||
pinned by the scripts/cps_freeze_seams rig; truth is generated by the
|
||||
PYTHON ORACLE ONLY (the facade floor + the real CLI for the dispatch rows).
|
||||
|
||||
Run:
|
||||
PYTHONPATH=scripts/cps_freeze_seams python3 scripts/gen_context_pressure_truth.py
|
||||
|
||||
Portable law = the pure compute + entry assembly:
|
||||
estimate_tokens (codepoint len, max(1, int(len/3.5)))
|
||||
measure_pressure (raw-pressure gates, banker round(x, 3))
|
||||
detect_degradation(trend/repetition/generic signals, max(0, x) type law)
|
||||
handle_hook (data truthiness, message/text/str(data) fallback)
|
||||
log_measurement ({"timestamp", "session_id", **measurement} spread)
|
||||
read_log (deque(maxlen) tail, blank/invalid skip)
|
||||
log_stats (counts + exact AttributeError on non-dict lines)
|
||||
main (argparse shapes, .1% renders, indent=2 dumps, code-2 law)
|
||||
File IO + wall clock stay in the facade (no-clock law: now is an argument).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
REPO = Path(__file__).resolve().parents[1]
|
||||
if str(REPO / "scripts") not in sys.path:
|
||||
sys.path.insert(0, str(REPO / "scripts"))
|
||||
|
||||
import context_pressure_truth # noqa: E402
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(context_pressure_truth.main())
|
||||
@@ -30,8 +30,37 @@ LOG_PATH = os.path.join(memory_env.memory_dir(), "index", "context_pressure.json
|
||||
CHARS_PER_TOKEN = 3.5
|
||||
|
||||
|
||||
def _kernel():
|
||||
"""The Rust context-pressure core (cps_loader) when the RUNE_CPS_CORE /
|
||||
RUNE_CONTEXT_PRESSURE_CORE pins allow it, else None — the pure compute
|
||||
(token estimation, the pressure ladder, degradation signals, JSONL
|
||||
entry assembly, log tail parsing, stats counting, hook message
|
||||
derivation, CLI dispatch) delegates there first and falls back to the
|
||||
Python body below. The side-effect shell (LOG_PATH resolution, the
|
||||
file append/read, the wall clock) cannot cross FFI and stays here."""
|
||||
pin = (os.environ.get("RUNE_CPS_CORE")
|
||||
or os.environ.get("RUNE_CONTEXT_PRESSURE_CORE")
|
||||
or "auto")
|
||||
if pin == "python":
|
||||
return None
|
||||
try:
|
||||
from rune import cps_loader
|
||||
|
||||
if cps_loader.available():
|
||||
return cps_loader
|
||||
except Exception: # noqa: BLE001 — kernel failure → Python floor
|
||||
return None
|
||||
return None
|
||||
|
||||
|
||||
def estimate_tokens(text):
|
||||
"""Estimate token count. Deterministic, <1ms."""
|
||||
k = _kernel()
|
||||
if k is not None:
|
||||
try:
|
||||
return k.estimate_tokens(text)
|
||||
except Exception: # noqa: BLE001 — kernel floor
|
||||
pass
|
||||
if not text:
|
||||
return 0
|
||||
return max(1, int(len(text) / CHARS_PER_TOKEN))
|
||||
@@ -39,10 +68,16 @@ def estimate_tokens(text):
|
||||
|
||||
def measure_pressure(text, max_tokens=128000):
|
||||
"""Measure context pressure from text. Returns 0.0-1.0."""
|
||||
tokens = estimate_tokens(text)
|
||||
pressure = min(1.0, tokens / max_tokens)
|
||||
k = _kernel()
|
||||
if k is not None:
|
||||
try:
|
||||
return k.measure_pressure(text, max_tokens)
|
||||
except Exception: # noqa: BLE001 — kernel floor
|
||||
pass
|
||||
estimate = estimate_tokens(text)
|
||||
pressure = min(1.0, estimate / max_tokens)
|
||||
return {
|
||||
"tokens": tokens,
|
||||
"tokens": estimate,
|
||||
"max_tokens": max_tokens,
|
||||
"pressure": round(pressure, 3),
|
||||
"status": "ok" if pressure < 0.7 else "warning" if pressure < 0.9 else "critical",
|
||||
@@ -60,6 +95,12 @@ def detect_degradation(recent_responses):
|
||||
- Repetition increasing
|
||||
- Generic/short responses replacing detailed ones
|
||||
"""
|
||||
k = _kernel()
|
||||
if k is not None:
|
||||
try:
|
||||
return k.detect_degradation(recent_responses)
|
||||
except Exception: # noqa: BLE001 — kernel floor
|
||||
pass
|
||||
if len(recent_responses) < 3:
|
||||
return {"degradation": 0.0, "signals": [], "insufficient_data": True}
|
||||
|
||||
@@ -110,6 +151,17 @@ def detect_degradation(recent_responses):
|
||||
|
||||
def log_measurement(measurement, session_id=""):
|
||||
"""Log a pressure measurement."""
|
||||
k = _kernel()
|
||||
if k is not None:
|
||||
try:
|
||||
entry = k.log_measurement(measurement, session_id, _now())
|
||||
except Exception: # noqa: BLE001 — kernel floor
|
||||
entry = None
|
||||
if entry is not None:
|
||||
os.makedirs(os.path.dirname(LOG_PATH), exist_ok=True)
|
||||
with open(LOG_PATH, "a") as f:
|
||||
f.write(json.dumps(entry) + "\n")
|
||||
return
|
||||
entry = {
|
||||
"timestamp": _now(),
|
||||
"session_id": session_id,
|
||||
@@ -124,13 +176,26 @@ def read_log(limit=20):
|
||||
"""Read recent pressure log entries (reads from end for efficiency)."""
|
||||
if not os.path.exists(LOG_PATH):
|
||||
return []
|
||||
from collections import deque
|
||||
|
||||
try:
|
||||
with open(LOG_PATH) as f:
|
||||
tail = deque(f, maxlen=limit)
|
||||
content = f.read()
|
||||
except Exception: # noqa: BLE001 — deliberate: best-effort degrade (logged or continued); crash would be worse
|
||||
return []
|
||||
k = _kernel()
|
||||
if k is not None:
|
||||
try:
|
||||
return k.read_log(content, limit)
|
||||
except Exception: # noqa: BLE001 — kernel floor
|
||||
pass
|
||||
from collections import deque
|
||||
|
||||
lines = content.split("\n")
|
||||
if lines and lines[-1] == "":
|
||||
lines.pop() # file-iteration law: no phantom line after a final \n
|
||||
try:
|
||||
tail = deque(lines, maxlen=limit)
|
||||
except Exception: # noqa: BLE001 — best-effort degrade (e.g. negative limit)
|
||||
return []
|
||||
entries = []
|
||||
for line in tail:
|
||||
line = line.strip()
|
||||
@@ -147,6 +212,17 @@ def log_stats():
|
||||
if not os.path.exists(LOG_PATH):
|
||||
return {"total": 0, "warnings": 0, "criticals": 0}
|
||||
|
||||
k = _kernel()
|
||||
if k is not None:
|
||||
with open(LOG_PATH) as f:
|
||||
content = f.read()
|
||||
try:
|
||||
return k.log_stats(content)
|
||||
except AttributeError:
|
||||
raise # crash parity: a parsed non-dict line keeps the oracle's exact error
|
||||
except Exception: # noqa: BLE001 — kernel floor
|
||||
pass
|
||||
|
||||
total = 0
|
||||
warnings = 0
|
||||
criticals = 0
|
||||
@@ -173,6 +249,19 @@ def _now():
|
||||
|
||||
def handle_hook(event, data=None):
|
||||
"""On-message hook: measure context pressure."""
|
||||
k = _kernel()
|
||||
if k is not None:
|
||||
try:
|
||||
out = k.handle_hook(event, data, _now())
|
||||
except Exception: # noqa: BLE001 — kernel floor
|
||||
out = None
|
||||
if out is not None:
|
||||
entry = out.get("entry")
|
||||
if entry is not None:
|
||||
os.makedirs(os.path.dirname(LOG_PATH), exist_ok=True)
|
||||
with open(LOG_PATH, "a") as f:
|
||||
f.write(json.dumps(entry) + "\n")
|
||||
return out.get("result")
|
||||
if event == "on_message" and data:
|
||||
message = data.get("message", "") or data.get("text", "") or str(data)
|
||||
if message:
|
||||
|
||||
@@ -0,0 +1,137 @@
|
||||
#!/usr/bin/env python3
|
||||
"""ctypes loader for the Rust context_pressure core (dichotomy wave,
|
||||
#2479 tranche). Mirrors sh_loader.py's contract:
|
||||
|
||||
FFI contract (doc-envelope law — every call returns {"ok", "value"|"err"}):
|
||||
* cpx_estimate_tokens({text}) -> tokens
|
||||
* cpx_measure_pressure({text, max_tokens?}) -> measurement doc
|
||||
* cpx_detect_degradation({responses}) -> degradation doc
|
||||
* cpx_log_measurement({measurement, session_id, now}) -> entry doc
|
||||
* cpx_read_log({content, limit}) -> [entries]
|
||||
* cpx_log_stats({content}) -> {total, warnings, criticals}
|
||||
* cpx_handle_hook({event, data?, now}) -> {result, entry|null}
|
||||
* cpx_main_dispatch({argv, content, now}) -> {output, code, entry|null}
|
||||
|
||||
The side-effect shell (LOG_PATH resolution, file append/read, wall clock)
|
||||
stays in the facade — the kernel receives the timestamp as an argument.
|
||||
Error envelopes raise the mapped exception (TypeError/ZeroDivisionError/
|
||||
AttributeError) so crash parity holds through the loader.
|
||||
|
||||
Free: capability_scoring's cs_free_string (shared process). Override the
|
||||
.so with CPS_RUST_LIB (a FILE path).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import ctypes
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
_REPO = Path(__file__).resolve().parents[2]
|
||||
_CRATE_DIR = _REPO / "src" / "rust" / "context_pressure"
|
||||
_LIB_PATH = Path(
|
||||
os.environ.get("CPS_RUST_LIB", _CRATE_DIR / "target" / "release" / "libcontext_pressure.so")
|
||||
)
|
||||
|
||||
_ERR_TYPES = {
|
||||
"ValueError": ValueError,
|
||||
"KeyError": KeyError,
|
||||
"AttributeError": AttributeError,
|
||||
"TypeError": TypeError,
|
||||
"ZeroDivisionError": ZeroDivisionError,
|
||||
}
|
||||
|
||||
_lib = None
|
||||
|
||||
|
||||
class ContextPressureCoreUnavailable(FileNotFoundError):
|
||||
"""The Rust context-pressure core is not built. Callers fall back to
|
||||
Python."""
|
||||
|
||||
|
||||
def available() -> bool:
|
||||
return _LIB_PATH.is_file()
|
||||
|
||||
|
||||
def _load() -> ctypes.CDLL:
|
||||
global _lib
|
||||
if _lib is not None:
|
||||
return _lib
|
||||
if not _LIB_PATH.is_file():
|
||||
raise ContextPressureCoreUnavailable(str(_LIB_PATH))
|
||||
lib = ctypes.CDLL(str(_LIB_PATH))
|
||||
for sym in (
|
||||
"cpx_estimate_tokens",
|
||||
"cpx_measure_pressure",
|
||||
"cpx_detect_degradation",
|
||||
"cpx_log_measurement",
|
||||
"cpx_read_log",
|
||||
"cpx_log_stats",
|
||||
"cpx_handle_hook",
|
||||
"cpx_main_dispatch",
|
||||
):
|
||||
getattr(lib, sym).argtypes = [ctypes.c_void_p]
|
||||
getattr(lib, sym).restype = ctypes.c_void_p
|
||||
try:
|
||||
lib.cs_free_string
|
||||
except AttributeError:
|
||||
scoring = ctypes.CDLL(str(
|
||||
_REPO / "src" / "rust" / "capability_scoring" / "target" / "release" / "libcapability_scoring.so"
|
||||
))
|
||||
scoring.cs_free_string.argtypes = [ctypes.c_void_p]
|
||||
lib._free = scoring.cs_free_string
|
||||
_lib = lib
|
||||
return lib
|
||||
|
||||
|
||||
def _call(sym: str, payload: dict):
|
||||
lib = _load()
|
||||
data = json.dumps(payload).encode()
|
||||
buf = ctypes.create_string_buffer(data)
|
||||
ptr = getattr(lib, sym)(ctypes.cast(buf, ctypes.c_void_p))
|
||||
if not ptr:
|
||||
raise RuntimeError(f"{sym} returned null")
|
||||
out = ctypes.string_at(ptr).decode()
|
||||
free = getattr(lib, "_free", None)
|
||||
if free is not None:
|
||||
free(ptr)
|
||||
doc = json.loads(out)
|
||||
if not doc.get("ok"):
|
||||
etype = doc.get("type", "RuntimeError")
|
||||
raise _ERR_TYPES.get(etype, RuntimeError)(doc.get("err", f"{sym} failed"))
|
||||
return doc["value"]
|
||||
|
||||
|
||||
def estimate_tokens(text) -> int:
|
||||
return _call("cpx_estimate_tokens", {"text": text})
|
||||
|
||||
|
||||
def measure_pressure(text, max_tokens=128000):
|
||||
return _call("cpx_measure_pressure", {"text": text, "max_tokens": max_tokens})
|
||||
|
||||
|
||||
def detect_degradation(recent_responses):
|
||||
return _call("cpx_detect_degradation", {"responses": recent_responses})
|
||||
|
||||
|
||||
def log_measurement(measurement, session_id="", now=""):
|
||||
return _call("cpx_log_measurement", {"measurement": measurement,
|
||||
"session_id": session_id,
|
||||
"now": now})
|
||||
|
||||
|
||||
def read_log(content, limit=20):
|
||||
return _call("cpx_read_log", {"content": content, "limit": limit})
|
||||
|
||||
|
||||
def log_stats(content):
|
||||
return _call("cpx_log_stats", {"content": content})
|
||||
|
||||
|
||||
def handle_hook(event, data=None, now=""):
|
||||
return _call("cpx_handle_hook", {"event": event, "data": data, "now": now})
|
||||
|
||||
|
||||
def main_dispatch(argv, content="", now=""):
|
||||
return _call("cpx_main_dispatch", {"argv": argv, "content": content,
|
||||
"now": now})
|
||||
Generated
+28
@@ -6,9 +6,36 @@ version = 4
|
||||
name = "context_pressure"
|
||||
version = "0.1.0"
|
||||
dependencies = [
|
||||
"evidence_provenance",
|
||||
"serde_json",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "equivalent"
|
||||
version = "1.0.3"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "00d174d5400e5e8fd687ad1049e2f578285fa914201b1af7e8b112a4546bd826"
|
||||
|
||||
[[package]]
|
||||
name = "evidence_provenance"
|
||||
version = "0.1.0"
|
||||
|
||||
[[package]]
|
||||
name = "hashbrown"
|
||||
version = "0.17.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "ed5909b6e89a2db4456e54cd5f673791d7eca6732202bbf2a9cc504fe2f9b84a"
|
||||
|
||||
[[package]]
|
||||
name = "indexmap"
|
||||
version = "2.14.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "cc4e190f5d26ca7051642629da2c52fc03bde85a03197c99408dcd291734c855"
|
||||
dependencies = [
|
||||
"equivalent",
|
||||
"hashbrown",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "itoa"
|
||||
version = "1.0.18"
|
||||
@@ -74,6 +101,7 @@ version = "1.0.151"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "c841b55ecdae098c80dcae9cf767f6f8a0c2cdb3416bbef72181df4d0fe73f14"
|
||||
dependencies = [
|
||||
"indexmap",
|
||||
"itoa",
|
||||
"memchr",
|
||||
"serde",
|
||||
|
||||
@@ -2,9 +2,11 @@
|
||||
name = "context_pressure"
|
||||
version = "0.1.0"
|
||||
edition = "2021"
|
||||
description = "FULL parity port of src/rune/context_pressure.py (dichotomy wave, #2479 tranche)"
|
||||
|
||||
[lib]
|
||||
crate-type = ["rlib", "cdylib", "staticlib"]
|
||||
|
||||
[dependencies]
|
||||
serde_json = "1"
|
||||
serde_json = { version = "1", features = ["preserve_order"] }
|
||||
evidence_provenance = { path = "../evidence_provenance" }
|
||||
@@ -0,0 +1,384 @@
|
||||
//! branch_battery — replay the FROZEN oracle truth VERBATIM
|
||||
//! (frozen_cp_truth.json, generated by scripts/gen_context_pressure_truth.py
|
||||
//! against the LIVE Python oracle under the scripts/cps_freeze_seams rig:
|
||||
//! RUNE_CPS_CORE=python pinned, clock frozen, hermetic scratch log).
|
||||
//!
|
||||
//! Layer contract: envelope cases (the pure surface: estimate_tokens,
|
||||
//! measure_pressure, detect_degradation, handle_hook return docs) plus
|
||||
//! kernel cases (entry-assembly BYTE lines, log tail parses, stats counts
|
||||
//! incl. crash-parity rows, CLI dispatch outputs) replayed through the
|
||||
//! Rust core — no Python, no FFI; the file and battery cannot drift
|
||||
//! (include_str!). Crash-parity rows store the oracle's exception as the
|
||||
//! string "__crash__ TYPE: message" — the exact str(e) contract.
|
||||
|
||||
use crate::{
|
||||
detect_degradation, estimate_tokens, handle_hook, log_measurement, log_stats, main_dispatch,
|
||||
measure_pressure_doc, read_log,
|
||||
};
|
||||
use evidence_provenance::pyjson::{self, Json};
|
||||
|
||||
const TRUTH: &str = include_str!("frozen_cp_truth.json");
|
||||
|
||||
fn truth() -> Json {
|
||||
// include_str! keeps file and battery byte-locked; the serde bridge
|
||||
// preserves the int/float wire law the frozen rows carry.
|
||||
wire_truth(TRUTH).expect("frozen truth parses")
|
||||
}
|
||||
|
||||
fn wire_truth(raw: &str) -> Result<Json, String> {
|
||||
let v: serde_json::Value = serde_json::from_str(raw).map_err(|e| e.to_string())?;
|
||||
Ok(bridge(&v))
|
||||
}
|
||||
|
||||
fn bridge(v: &serde_json::Value) -> Json {
|
||||
match v {
|
||||
serde_json::Value::Null => Json::Null,
|
||||
serde_json::Value::Bool(b) => Json::Bool(*b),
|
||||
serde_json::Value::Number(n) => {
|
||||
if n.is_i64() || n.is_u64() {
|
||||
Json::Num(n.as_f64().unwrap_or(0.0))
|
||||
} else {
|
||||
Json::Float(n.as_f64().unwrap_or(0.0))
|
||||
}
|
||||
}
|
||||
serde_json::Value::String(s) => Json::Str(s.clone()),
|
||||
serde_json::Value::Array(items) => Json::Arr(items.iter().map(bridge).collect()),
|
||||
serde_json::Value::Object(map) => Json::Obj(
|
||||
map.iter()
|
||||
.map(|(k, val)| (k.clone(), bridge(val)))
|
||||
.collect(),
|
||||
),
|
||||
}
|
||||
}
|
||||
|
||||
/// Envelope cases are one list; the battery groups by fn name at runtime.
|
||||
fn envelope_cases(fn_name: &str) -> Vec<Json> {
|
||||
let cases = match truth().get("cases") {
|
||||
Some(Json::Arr(items)) => items.clone(),
|
||||
_ => panic!("envelope cases missing"),
|
||||
};
|
||||
let filtered: Vec<Json> = cases
|
||||
.into_iter()
|
||||
.filter(|c| match c.get("fn") {
|
||||
Some(Json::Str(f)) => f == fn_name,
|
||||
_ => false,
|
||||
})
|
||||
.collect();
|
||||
assert!(!filtered.is_empty(), "no envelope cases for {fn_name}");
|
||||
filtered
|
||||
}
|
||||
|
||||
fn kernel_cases(kind: &str) -> Vec<Json> {
|
||||
match truth().get("kernel_cases").and_then(|kc| kc.get(kind)) {
|
||||
Some(Json::Arr(items)) => items.clone(),
|
||||
_ => panic!("kernel cases {kind} missing"),
|
||||
}
|
||||
}
|
||||
|
||||
fn at<'a>(v: &'a Json, path: &[&str]) -> Option<&'a Json> {
|
||||
let mut cur = v;
|
||||
for k in path {
|
||||
cur = match cur {
|
||||
Json::Obj(pairs) => pairs.iter().find(|(k2, _)| k2 == k).map(|(_, x)| x)?,
|
||||
_ => return None,
|
||||
};
|
||||
}
|
||||
Some(cur)
|
||||
}
|
||||
|
||||
fn arg_list(c: &Json) -> Vec<Json> {
|
||||
match at(c, &["args"]) {
|
||||
Some(Json::Arr(items)) => items.clone(),
|
||||
_ => Vec::new(),
|
||||
}
|
||||
}
|
||||
|
||||
fn jstr(v: Option<&Json>) -> String {
|
||||
match v {
|
||||
Some(Json::Str(s)) => s.clone(),
|
||||
_ => String::new(),
|
||||
}
|
||||
}
|
||||
|
||||
fn jnum(v: Option<&Json>) -> f64 {
|
||||
match v {
|
||||
Some(Json::Num(n)) | Some(Json::Float(n)) => *n,
|
||||
_ => 0.0,
|
||||
}
|
||||
}
|
||||
|
||||
/// Crash-parity envelope value: "TYPE: message" (str(e) law).
|
||||
fn crash(err: &str, ty: &str) -> Json {
|
||||
Json::Str(format!("{ty}: {err}"))
|
||||
}
|
||||
|
||||
/// Python-value equality on Json (Num/Float compare by f64; the wire law
|
||||
/// distinguishes them only when RENDERING, which the byte-line rows cover).
|
||||
fn peq(a: &Json, b: &Json) -> bool {
|
||||
match (a, b) {
|
||||
(Json::Null, Json::Null) => true,
|
||||
(Json::Bool(x), Json::Bool(y)) => x == y,
|
||||
(Json::Num(x), Json::Num(y)) => x == y,
|
||||
(Json::Float(x), Json::Float(y)) => x == y,
|
||||
(Json::Num(x), Json::Float(y)) => x == y,
|
||||
(Json::Float(x), Json::Num(y)) => x == y,
|
||||
(Json::Str(x), Json::Str(y)) => x == y,
|
||||
(Json::Arr(x), Json::Arr(y)) => {
|
||||
x.len() == y.len() && x.iter().zip(y).all(|(a, b)| peq(a, b))
|
||||
}
|
||||
(Json::Obj(x), Json::Obj(y)) => {
|
||||
x.len() == y.len()
|
||||
&& x.iter().all(|(k, v)| {
|
||||
y.iter()
|
||||
.find(|(k2, _)| k2 == k)
|
||||
.map(|(_, v2)| peq(v, v2))
|
||||
.unwrap_or(false)
|
||||
})
|
||||
}
|
||||
_ => false,
|
||||
}
|
||||
}
|
||||
|
||||
fn assert_eq(case_name: &str, want: &Json, got: &Json) {
|
||||
assert!(
|
||||
peq(want, got),
|
||||
"case {case_name} diverged:\n want={}\n got={}",
|
||||
pyjson::dumps(want),
|
||||
pyjson::dumps(got),
|
||||
);
|
||||
}
|
||||
|
||||
// ── envelope cases (pure surface) ─────────────────────────────────────────
|
||||
|
||||
#[test]
|
||||
fn replay_estimate_tokens() {
|
||||
for c in envelope_cases("estimate_tokens") {
|
||||
let args = arg_list(&c);
|
||||
let got = match args.first() {
|
||||
Some(Json::Str(x)) => Json::Num(estimate_tokens(x) as f64),
|
||||
Some(other) => {
|
||||
// non-string: len() TypeError parity (None → 0)
|
||||
match crate::tokens_of(other) {
|
||||
Ok(n) => Json::Num(n as f64),
|
||||
Err((err, ty)) => crash(&err, &ty),
|
||||
}
|
||||
}
|
||||
None => Json::Num(0.0),
|
||||
};
|
||||
assert_eq(
|
||||
&format!("estimate_tokens/{}", jstr(at(&c, &["name"]))),
|
||||
at(&c, &["expected"]).expect("expected"),
|
||||
&got,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn replay_measure_pressure() {
|
||||
for c in envelope_cases("measure_pressure") {
|
||||
let args = arg_list(&c);
|
||||
let max_tokens = match at(&c, &["kwargs", "max_tokens"]) {
|
||||
Some(v) => jnum(Some(v)),
|
||||
None => 128000.0,
|
||||
};
|
||||
let got = match measure_pressure_doc(args.first().unwrap_or(&Json::Null), max_tokens) {
|
||||
Ok(doc) => doc,
|
||||
Err((err, ty)) => crash(&err, &ty),
|
||||
};
|
||||
assert_eq(
|
||||
&format!("measure_pressure/{}", jstr(at(&c, &["name"]))),
|
||||
at(&c, &["expected"]).expect("expected"),
|
||||
&got,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn replay_detect_degradation() {
|
||||
for c in envelope_cases("detect_degradation") {
|
||||
let args = arg_list(&c);
|
||||
let responses: Vec<Json> = match args.first() {
|
||||
Some(Json::Arr(items)) => items.clone(),
|
||||
_ => Vec::new(),
|
||||
};
|
||||
let got = match detect_degradation(&responses) {
|
||||
Ok(doc) => doc,
|
||||
Err((err, ty)) => crash(&err, &ty),
|
||||
};
|
||||
assert_eq(
|
||||
&format!("detect_degradation/{}", jstr(at(&c, &["name"]))),
|
||||
at(&c, &["expected"]).expect("expected"),
|
||||
&got,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn replay_handle_hook_result() {
|
||||
let now = jstr(at(&truth(), &["frozen_now"]));
|
||||
for c in envelope_cases("handle_hook") {
|
||||
let args = arg_list(&c);
|
||||
let event = jstr(args.first());
|
||||
let data = args.get(1).cloned().unwrap_or(Json::Null);
|
||||
let got = match handle_hook(&event, &data, &now) {
|
||||
Ok(hh) => hh.get("result").cloned().unwrap_or(Json::Null),
|
||||
Err((err, ty)) => crash(&err, &ty),
|
||||
};
|
||||
assert_eq(
|
||||
&format!("handle_hook/{}", jstr(at(&c, &["name"]))),
|
||||
at(&c, &["expected"]).expect("expected"),
|
||||
&got,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
// ── kernel cases ──────────────────────────────────────────────────────────
|
||||
|
||||
#[test]
|
||||
fn replay_log_measurement_lines() {
|
||||
for c in kernel_cases("logMeasurement") {
|
||||
let measurement = at(&c, &["args", "measurement"])
|
||||
.cloned()
|
||||
.expect("measurement");
|
||||
let got = log_measurement(
|
||||
&measurement,
|
||||
&jstr(at(&c, &["args", "session_id"])),
|
||||
&jstr(at(&c, &["args", "now"])),
|
||||
);
|
||||
// BYTE law: the facade appends dumps(entry) + "\n" — the frozen
|
||||
// `line` is exactly the JSONL bytes the oracle wrote.
|
||||
assert_eq!(
|
||||
format!("{}\n", pyjson::dumps(&got)),
|
||||
jstr(at(&c, &["expect", "line"])),
|
||||
"byte-identical entry line for {}",
|
||||
jstr(at(&c, &["name"])),
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn replay_read_log() {
|
||||
for c in kernel_cases("readLog") {
|
||||
let got = read_log(
|
||||
&jstr(at(&c, &["args", "content"])),
|
||||
jnum(at(&c, &["args", "limit"])) as i64,
|
||||
);
|
||||
assert_eq(
|
||||
&format!("read_log/{}", jstr(at(&c, &["name"]))),
|
||||
at(&c, &["expect"]).expect("expect"),
|
||||
&got,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn replay_log_stats() {
|
||||
for c in kernel_cases("logStats") {
|
||||
let got = log_stats(&jstr(at(&c, &["args", "content"])));
|
||||
let want = at(&c, &["expect"]).expect("expect");
|
||||
let doc = match got {
|
||||
Ok(doc) => doc,
|
||||
Err((err, ty)) => crash(&err, &ty),
|
||||
};
|
||||
// crash rows store {"__crash__": "..."} — compare that exact string
|
||||
let want_doc = match want.get("__crash__") {
|
||||
Some(Json::Str(msg)) => Json::Str(msg.clone()),
|
||||
_ => want.clone(),
|
||||
};
|
||||
assert_eq(
|
||||
&format!("log_stats/{}", jstr(at(&c, &["name"]))),
|
||||
&want_doc,
|
||||
&doc,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn replay_kernel_handle_hook() {
|
||||
for c in kernel_cases("kernelHook") {
|
||||
let got = handle_hook(
|
||||
&jstr(at(&c, &["args", "event"])),
|
||||
&at(&c, &["args", "data"]).cloned().unwrap_or(Json::Null),
|
||||
&jstr(at(&c, &["args", "now"])),
|
||||
);
|
||||
let want = at(&c, &["expect"]).expect("expect");
|
||||
let doc = match got {
|
||||
Ok(doc) => doc,
|
||||
Err((err, ty)) => crash(&err, &ty),
|
||||
};
|
||||
let want_doc = match want.get("__crash__") {
|
||||
Some(Json::Str(msg)) => Json::Str(msg.clone()),
|
||||
_ => want.clone(),
|
||||
};
|
||||
assert_eq(
|
||||
&format!("kernel_hook/{}", jstr(at(&c, &["name"]))),
|
||||
&want_doc,
|
||||
&doc,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn replay_main_dispatch() {
|
||||
for c in kernel_cases("mainDispatch") {
|
||||
let argv: Vec<String> = match at(&c, &["args", "argv"]) {
|
||||
Some(Json::Arr(items)) => items
|
||||
.iter()
|
||||
.filter_map(|x| match x {
|
||||
Json::Str(y) => Some(y.clone()),
|
||||
_ => None,
|
||||
})
|
||||
.collect(),
|
||||
_ => Vec::new(),
|
||||
};
|
||||
let got = main_dispatch(
|
||||
&argv,
|
||||
&jstr(at(&c, &["args", "content"])),
|
||||
&jstr(at(&c, &["args", "now"])),
|
||||
);
|
||||
let want = at(&c, &["expect"]).expect("expect");
|
||||
let doc = match got {
|
||||
Ok(doc) => doc,
|
||||
Err((err, ty)) => crash(&err, &ty),
|
||||
};
|
||||
// output/code always; the entry key rides only on hook rows
|
||||
let mut want_pairs: Vec<(String, Json)> = Vec::new();
|
||||
if let Json::Obj(pairs) = &want {
|
||||
for (k, v) in pairs {
|
||||
if k == "output" || k == "code" {
|
||||
want_pairs.push((k.clone(), v.clone()));
|
||||
let got_v = doc.get(k).cloned().unwrap_or(Json::Null);
|
||||
assert_eq(
|
||||
&format!("main_dispatch/{}/{}", jstr(at(&c, &["name"])), k),
|
||||
v,
|
||||
&got_v,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
let _ = want_pairs;
|
||||
if at(&c, &["expect", "entry"]).is_some() {
|
||||
let want_entry = at(&c, &["expect", "entry"]).cloned().unwrap_or(Json::Null);
|
||||
let got_entry = doc.get("entry").cloned().unwrap_or(Json::Null);
|
||||
assert_eq(
|
||||
&format!("main_dispatch/{}/entry", jstr(at(&c, &["name"]))),
|
||||
&want_entry,
|
||||
&got_entry,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ── helper contract probes ────────────────────────────────────────────────
|
||||
|
||||
#[test]
|
||||
fn banker_round_helper_contract() {
|
||||
// round(x, 3) resolves on the exact double: 0.6225's stored double
|
||||
// sits ABOVE the decimal half (0.623); 1.5625e-05 rounds to 0.0 but
|
||||
// stays a float; 0.9994999999999999 rounds down. The emit path uses
|
||||
// the same helper, so the probe pins the emit contract.
|
||||
assert_eq!(crate::py_round3(0.6225), 0.623);
|
||||
assert_eq!(crate::py_round3(1.5625e-05), 0.0);
|
||||
assert_eq!(crate::py_round3(0.7), 0.7);
|
||||
assert_eq!(crate::py_round3(0.999_499_999_999_999_9), 0.999);
|
||||
}
|
||||
File diff suppressed because it is too large.
Load diff
@@ -1,72 +1,904 @@
|
||||
//! context_pressure — context window pressure monitoring (W6).
|
||||
//! PARITY port of src/rune/context_pressure.py.
|
||||
use serde_json::json;
|
||||
//! context_pressure — context window pressure monitoring (dichotomy wave,
|
||||
//! #2479 tranche).
|
||||
//!
|
||||
//! FULL parity port of src/rune/context_pressure.py. The PORTABLE LAW is
|
||||
//! pure compute + entry assembly: token estimation (CODEPOINT count —
|
||||
//! Python len(), never bytes), the pressure ladder (raw-pressure gates,
|
||||
//! banker-correct round(x, 3) display-only), behavioral degradation
|
||||
//! signals (int-0/float-0 max(0, ...) type law), the JSONL entry spread
|
||||
//! ({"timestamp", "session_id", **measurement} with in-place override),
|
||||
//! the log tail reader (deque(maxlen) semantics — blank/unparseable lines
|
||||
//! skipped), the log stats counter (crash parity: a parsed non-dict line
|
||||
//! dies with the oracle's exact AttributeError), the on-message hook
|
||||
//! (data truthiness, message/text/str(data) fallback, pyrepr of the data
|
||||
//! doc), and the CLI dispatch (argparse shapes, .1% rendering, indent=2
|
||||
//! dumps). File IO (append lines, read the log) and the wall clock stay
|
||||
//! in the Python facade — the kernel receives the timestamp as an
|
||||
//! argument (no-clock law).
|
||||
//!
|
||||
//! FFI (doc-envelope law): every call takes one JSON doc and returns
|
||||
//! {"ok": true, "value": ...} | {"ok": false, "err": ..., "type": ...}.
|
||||
//! * cpx_estimate_tokens({text}) -> tokens
|
||||
//! * cpx_measure_pressure({text, max_tokens?}) -> measurement doc
|
||||
//! * cpx_detect_degradation({responses}) -> degradation doc
|
||||
//! * cpx_log_measurement({measurement, session_id, now}) -> entry doc
|
||||
//! * cpx_read_log({content, limit?}) -> [entries]
|
||||
//! * cpx_log_stats({content}) -> {total, warnings, criticals}
|
||||
//! * cpx_handle_hook({event, data?, now}) -> {result, entry|null}
|
||||
//! * cpx_main_dispatch({argv, content, now}) -> {output, code, entry|null}
|
||||
//!
|
||||
//! Wire docs are parsed through serde_json (int/float discrimination —
|
||||
//! Python json.loads keeps 0.0 a float; pyjson alone would flatten both
|
||||
//! to Num) then bridged into pyjson's insertion-ordered Json for
|
||||
//! Python-parity rendering. No free-string export here: callers free
|
||||
//! through capability_scoring's cs_free_string (shared allocator).
|
||||
//!
|
||||
//! Note: read_log's deque(maxlen=limit) raises ValueError for negative
|
||||
//! limits in the oracle, but read_log's best-effort except floor degrades
|
||||
//! it to []; the kernel mirrors the observable behavior ([]).
|
||||
|
||||
const CHARS_PER_TOKEN: f64 = 3.5;
|
||||
use evidence_provenance::pyjson::{self, Json};
|
||||
use serde_json::Value as SV;
|
||||
use std::ffi::{CStr, CString};
|
||||
use std::os::raw::c_char;
|
||||
|
||||
pub fn estimate_tokens(text: &str) -> usize {
|
||||
pub const CHARS_PER_TOKEN: f64 = 3.5;
|
||||
|
||||
// ── wire bridge (serde → pyjson, int/float type law) ──────────────────────
|
||||
|
||||
/// Python json.loads type law: "5" stays int, "5.0" stays float.
|
||||
fn wire(raw: &str) -> Result<Json, (String, String)> {
|
||||
let v: SV = serde_json::from_str(raw).map_err(|e| (e.to_string(), "ValueError".to_string()))?;
|
||||
Ok(to_json(&v))
|
||||
}
|
||||
|
||||
fn to_json(v: &SV) -> Json {
|
||||
match v {
|
||||
SV::Null => Json::Null,
|
||||
SV::Bool(b) => Json::Bool(*b),
|
||||
SV::Number(n) => {
|
||||
if n.is_i64() || n.is_u64() {
|
||||
Json::Num(n.as_f64().unwrap_or(0.0))
|
||||
} else {
|
||||
Json::Float(n.as_f64().unwrap_or(0.0))
|
||||
}
|
||||
}
|
||||
SV::String(s) => Json::Str(s.clone()),
|
||||
SV::Array(items) => Json::Arr(items.iter().map(to_json).collect()),
|
||||
SV::Object(map) => Json::Obj(
|
||||
map.iter()
|
||||
.map(|(k, val)| (k.clone(), to_json(val)))
|
||||
.collect(),
|
||||
),
|
||||
}
|
||||
}
|
||||
|
||||
// ── json-doc helpers ──────────────────────────────────────────────────────
|
||||
|
||||
fn obj(pairs: Vec<(&str, Json)>) -> Json {
|
||||
Json::Obj(pairs.into_iter().map(|(k, v)| (k.to_string(), v)).collect())
|
||||
}
|
||||
|
||||
fn s(v: &Json, key: &str) -> String {
|
||||
match v.get(key) {
|
||||
Some(Json::Str(x)) => x.clone(),
|
||||
_ => String::new(),
|
||||
}
|
||||
}
|
||||
|
||||
/// The Python type name used in AttributeError/TypeError-style messages.
|
||||
fn py_type(v: &Json) -> &'static str {
|
||||
match v {
|
||||
Json::Null => "NoneType",
|
||||
Json::Bool(_) => "bool",
|
||||
Json::Num(n) if n.is_finite() && n.fract() == 0.0 => "int",
|
||||
Json::Num(_) => "float",
|
||||
Json::Float(_) => "float",
|
||||
Json::Str(_) => "str",
|
||||
Json::Arr(_) => "list",
|
||||
Json::Obj(_) => "dict",
|
||||
}
|
||||
}
|
||||
|
||||
/// Python truthiness over the value lattice this module sees.
|
||||
fn truthy(v: &Json) -> bool {
|
||||
match v {
|
||||
Json::Null => false,
|
||||
Json::Bool(b) => *b,
|
||||
Json::Num(n) | Json::Float(n) => *n != 0.0,
|
||||
Json::Str(st) => !st.is_empty(),
|
||||
Json::Arr(items) => !items.is_empty(),
|
||||
Json::Obj(pairs) => !pairs.is_empty(),
|
||||
}
|
||||
}
|
||||
|
||||
/// Python round(x, 3): correct rounding of the binary value (ties resolved
|
||||
/// on the exact double — the same contract {:.3} formatting implements).
|
||||
fn py_round3(x: f64) -> f64 {
|
||||
format!("{x:.3}").parse().unwrap_or(x)
|
||||
}
|
||||
|
||||
/// Python f"{x:.1%}" — percentage with one decimal.
|
||||
fn fmt_pct(x: f64) -> String {
|
||||
format!("{:.1}%", x * 100.0)
|
||||
}
|
||||
|
||||
// ── estimate_tokens / measure_pressure ────────────────────────────────────
|
||||
|
||||
/// estimate_tokens — Python len() counts CODEPOINTS; int() truncates.
|
||||
pub fn estimate_tokens(text: &str) -> i64 {
|
||||
if text.is_empty() {
|
||||
return 0;
|
||||
}
|
||||
((text.len() as f64) / CHARS_PER_TOKEN) as usize
|
||||
let n = (text.chars().count() as f64) / CHARS_PER_TOKEN;
|
||||
1.max(n as i64)
|
||||
}
|
||||
|
||||
pub fn measure_pressure(text: &str, max_tokens: usize) -> serde_json::Value {
|
||||
fn len_or_err(v: &Json) -> Result<i64, (String, String)> {
|
||||
match v {
|
||||
Json::Str(x) => Ok(x.chars().count() as i64),
|
||||
other => Err((
|
||||
format!("object of type '{}' has no len()", py_type(other)),
|
||||
"TypeError".to_string(),
|
||||
)),
|
||||
}
|
||||
}
|
||||
|
||||
fn tokens_of(text: &Json) -> Result<i64, (String, String)> {
|
||||
match text {
|
||||
Json::Str(x) => Ok(estimate_tokens(x)),
|
||||
Json::Null => Ok(0), // `if not text` → 0
|
||||
other => Err((
|
||||
format!("object of type '{}' has no len()", py_type(other)),
|
||||
"TypeError".to_string(),
|
||||
)),
|
||||
}
|
||||
}
|
||||
|
||||
/// measure_pressure — gates read the RAW pressure; round(x, 3) is for
|
||||
/// display. max_tokens == 0 reproduces the oracle's ZeroDivisionError.
|
||||
pub fn measure_pressure(text: &str, max_tokens: f64) -> Result<Json, (String, String)> {
|
||||
if max_tokens == 0.0 {
|
||||
return Err((
|
||||
"division by zero".to_string(),
|
||||
"ZeroDivisionError".to_string(),
|
||||
));
|
||||
}
|
||||
let tokens = estimate_tokens(text);
|
||||
let ratio = tokens as f64 / max_tokens as f64;
|
||||
let level = if ratio >= 0.9 {
|
||||
"critical"
|
||||
} else if ratio >= 0.7 {
|
||||
"high"
|
||||
} else if ratio >= 0.5 {
|
||||
"moderate"
|
||||
let raw = (tokens as f64 / max_tokens).min(1.0);
|
||||
Ok(pressure_doc(tokens, max_tokens, raw))
|
||||
}
|
||||
|
||||
fn pressure_doc(tokens: i64, max_tokens: f64, raw: f64) -> Json {
|
||||
let (status, recommendation) = if raw < 0.7 {
|
||||
("ok", "none")
|
||||
} else if raw < 0.9 {
|
||||
("warning", "summarize_old_turns")
|
||||
} else {
|
||||
"low"
|
||||
("critical", "truncate_and_summarize")
|
||||
};
|
||||
json!({
|
||||
"tokens": tokens, "max_tokens": max_tokens,
|
||||
"ratio": format!("{:.2}", ratio), "level": level,
|
||||
})
|
||||
obj(vec![
|
||||
("tokens", Json::Num(tokens as f64)),
|
||||
("max_tokens", Json::Num(max_tokens)),
|
||||
("pressure", Json::Float(py_round3(raw))),
|
||||
("status", Json::Str(status.to_string())),
|
||||
("recommendation", Json::Str(recommendation.to_string())),
|
||||
])
|
||||
}
|
||||
|
||||
pub fn detect_degradation(responses: &[String]) -> bool {
|
||||
// Degradation: responses getting shorter or more repetitive
|
||||
fn measure_pressure_doc(text: &Json, max_tokens: f64) -> Result<Json, (String, String)> {
|
||||
if max_tokens == 0.0 {
|
||||
return Err((
|
||||
"division by zero".to_string(),
|
||||
"ZeroDivisionError".to_string(),
|
||||
));
|
||||
}
|
||||
let tokens = tokens_of(text)?;
|
||||
let raw = (tokens as f64 / max_tokens).min(1.0);
|
||||
Ok(pressure_doc(tokens, max_tokens, raw))
|
||||
}
|
||||
|
||||
// ── detect_degradation ────────────────────────────────────────────────────
|
||||
|
||||
/// Python \w+ runs: [a-zA-Z0-9_] plus alphanumeric unicode (str.isalnum
|
||||
/// law; combining-mark and whitespace-class edges land outside the corpus).
|
||||
fn is_word(c: char) -> bool {
|
||||
c == '_' || c.is_alphabetic() || c.is_numeric()
|
||||
}
|
||||
|
||||
fn word_runs(text: &str) -> Vec<String> {
|
||||
let lowered = text.to_lowercase();
|
||||
let mut words = Vec::new();
|
||||
let mut cur = String::new();
|
||||
for c in lowered.chars() {
|
||||
if is_word(c) {
|
||||
cur.push(c);
|
||||
} else if !cur.is_empty() {
|
||||
words.push(std::mem::take(&mut cur));
|
||||
}
|
||||
}
|
||||
if !cur.is_empty() {
|
||||
words.push(cur);
|
||||
}
|
||||
words
|
||||
}
|
||||
|
||||
/// max(0, x) type law: the INT 0 wins whenever the float does not compare
|
||||
/// greater (0 == 0.0 and all negatives) — json renders "0", never "0.0".
|
||||
fn max_zero(raw: f64) -> (f64, bool) {
|
||||
if raw > 0.0 {
|
||||
(raw, false) // float-form; round3 applied at emit
|
||||
} else {
|
||||
(0.0, true) // int-form 0
|
||||
}
|
||||
}
|
||||
|
||||
fn emit_rounded(raw: f64, int_form: bool) -> Json {
|
||||
if int_form {
|
||||
Json::Num(0.0)
|
||||
} else {
|
||||
Json::Float(py_round3(raw))
|
||||
}
|
||||
}
|
||||
|
||||
/// detect_degradation — response length trend, word repetition, generic
|
||||
/// short responses; the composite is max() over the RAW values, rounding
|
||||
/// only at emit. A non-string inside the examined slices keeps the
|
||||
/// oracle's exact TypeError (len() reaches the last five first).
|
||||
pub fn detect_degradation(responses: &[Json]) -> Result<Json, (String, String)> {
|
||||
if responses.len() < 3 {
|
||||
return false;
|
||||
return Ok(obj(vec![
|
||||
("degradation", Json::Float(0.0)),
|
||||
("signals", Json::Arr(Vec::new())),
|
||||
("insufficient_data", Json::Bool(true)),
|
||||
]));
|
||||
}
|
||||
let lens: Vec<usize> = responses.iter().map(|r| r.len()).collect();
|
||||
let avg_first = (lens[0] + lens[1]) / 2;
|
||||
let avg_last = lens[lens.len() - 1];
|
||||
avg_last < avg_first / 3 // responses shrunk to <33% of initial
|
||||
|
||||
// Signal 1: response length trend (last five, codepoint lengths).
|
||||
let start = responses.len().saturating_sub(5);
|
||||
let mut lengths = Vec::with_capacity(responses.len() - start);
|
||||
for r in &responses[start..] {
|
||||
lengths.push(len_or_err(r)?);
|
||||
}
|
||||
let avg_first = (lengths[0] as f64 + lengths[1] as f64) / 2.0;
|
||||
let n = lengths.len();
|
||||
let avg_last = (lengths[n - 2] as f64 + lengths[n - 1] as f64) / 2.0;
|
||||
let (ld_raw, ld_int) = if avg_first > 0.0 {
|
||||
max_zero((avg_first - avg_last) / avg_first)
|
||||
} else {
|
||||
(0.0, true)
|
||||
};
|
||||
|
||||
// Signal 3's inputs (same slice the oracle re-checks with len()).
|
||||
let tail3 = &responses[responses.len().saturating_sub(3)..];
|
||||
let mut generic_count = 0i64;
|
||||
for r in tail3 {
|
||||
if len_or_err(r)? < 100 {
|
||||
generic_count += 1;
|
||||
}
|
||||
}
|
||||
let generic_ratio = generic_count as f64 / 3.0;
|
||||
|
||||
// Signal 2: repetition (unique words vs total over the joined last
|
||||
// three responses) — the division order matches the oracle's
|
||||
// unique_ratio = unique/total; then *2; then 1.0 - .
|
||||
let joined: String = tail3
|
||||
.iter()
|
||||
.filter_map(|r| match r {
|
||||
Json::Str(x) => Some(x.as_str()),
|
||||
_ => None,
|
||||
})
|
||||
.collect::<Vec<&str>>()
|
||||
.join(" ");
|
||||
let words = word_runs(&joined);
|
||||
let (rs_raw, rs_int) = if words.is_empty() {
|
||||
(0.0, true) // `repetition_score = 0` — INT literal branch
|
||||
} else {
|
||||
let unique = words.iter().collect::<std::collections::HashSet<_>>().len();
|
||||
let unique_ratio = unique as f64 / words.len() as f64;
|
||||
max_zero(1.0 - unique_ratio * 2.0)
|
||||
};
|
||||
|
||||
// Composite degradation score: max() over the three float products
|
||||
// (int 0 * 0.4 is a float in Python); first-maximal law holds because
|
||||
// ties are value-identical.
|
||||
let a = ld_raw * 0.4;
|
||||
let b = rs_raw * 0.3;
|
||||
let c = generic_ratio * 0.3;
|
||||
let degradation = if a >= b && a >= c {
|
||||
a
|
||||
} else if b >= c {
|
||||
b
|
||||
} else {
|
||||
c
|
||||
};
|
||||
|
||||
let mut signals: Vec<Json> = Vec::new();
|
||||
if ld_raw > 0.3 {
|
||||
signals.push(Json::Str("response_length_declining".to_string()));
|
||||
}
|
||||
if rs_raw > 0.3 {
|
||||
signals.push(Json::Str("repetition_increasing".to_string()));
|
||||
}
|
||||
if generic_ratio > 0.5 {
|
||||
signals.push(Json::Str("responses_becoming_generic".to_string()));
|
||||
}
|
||||
|
||||
Ok(obj(vec![
|
||||
("degradation", Json::Float(py_round3(degradation))),
|
||||
("signals", Json::Arr(signals)),
|
||||
("length_decline", emit_rounded(ld_raw, ld_int)),
|
||||
("repetition_score", emit_rounded(rs_raw, rs_int)),
|
||||
("generic_ratio", Json::Float(py_round3(generic_ratio))),
|
||||
]))
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
// ── log_measurement (entry assembly; the file write stays in the facade) ──
|
||||
|
||||
#[test]
|
||||
fn test_estimate() {
|
||||
assert_eq!(estimate_tokens(""), 0);
|
||||
assert!(estimate_tokens("hello world") > 0);
|
||||
/// {"timestamp": now, "session_id": sid, **measurement} — Python dict-spread
|
||||
/// law: overridden keys keep their FIRST position, only the value changes.
|
||||
pub fn log_measurement(measurement: &Json, session_id: &str, now: &str) -> Json {
|
||||
let mut pairs: Vec<(String, Json)> = Vec::new();
|
||||
pairs.push(("timestamp".to_string(), Json::Str(now.to_string())));
|
||||
pairs.push(("session_id".to_string(), Json::Str(session_id.to_string())));
|
||||
if let Json::Obj(items) = measurement {
|
||||
for (k, v) in items {
|
||||
match pairs.iter_mut().find(|(ek, _)| ek == k) {
|
||||
Some(slot) => slot.1 = v.clone(),
|
||||
None => pairs.push((k.clone(), v.clone())),
|
||||
}
|
||||
}
|
||||
}
|
||||
Json::Obj(pairs)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_pressure_levels() {
|
||||
// fixtures derived from the code's tokenizer (CHARS_PER_TOKEN):
|
||||
// 1000 chars is far under any threshold; a payload sized to cross
|
||||
// the 0.9 critical line proves the ladder's top step.
|
||||
assert_eq!(measure_pressure(&"x".repeat(1000), 10000)["level"], "low");
|
||||
let big = (9.0 * CHARS_PER_TOKEN * 10000.0 / 10.0) as usize; // 90% of max_tokens in tokens
|
||||
assert_eq!(
|
||||
measure_pressure(&"x".repeat(big), 10000)["level"],
|
||||
"critical"
|
||||
// ── read_log / log_stats (content in; the facade owns file IO) ────────────
|
||||
|
||||
/// The log tail reader: deque(file, maxlen=limit) semantics — the LAST
|
||||
/// `limit` lines, blank lines skipped, unparseable lines skipped.
|
||||
pub fn read_log(content: &str, limit: i64) -> Json {
|
||||
if limit <= 0 {
|
||||
return Json::Arr(Vec::new());
|
||||
}
|
||||
let lines: Vec<&str> = content.lines().collect();
|
||||
let start = lines.len().saturating_sub(limit as usize);
|
||||
let mut entries = Vec::new();
|
||||
for line in &lines[start..] {
|
||||
let trimmed = line.trim();
|
||||
if trimmed.is_empty() {
|
||||
continue;
|
||||
}
|
||||
if let Ok(v) = wire(trimmed) {
|
||||
entries.push(v); // json.JSONDecodeError → continue
|
||||
}
|
||||
}
|
||||
Json::Arr(entries)
|
||||
}
|
||||
|
||||
/// Log stats: total parsed entries + warning/critical status counts. A
|
||||
/// parsed NON-dict line carries the oracle's exact AttributeError.
|
||||
pub fn log_stats(content: &str) -> Result<Json, (String, String)> {
|
||||
let mut total = 0i64;
|
||||
let mut warnings = 0i64;
|
||||
let mut criticals = 0i64;
|
||||
for line in content.lines() {
|
||||
let trimmed = line.trim();
|
||||
if trimmed.is_empty() {
|
||||
continue;
|
||||
}
|
||||
let entry = match wire(trimmed) {
|
||||
Ok(v) => v,
|
||||
Err(_) => continue, // json.JSONDecodeError → continue
|
||||
};
|
||||
total += 1;
|
||||
let status: Option<String> =
|
||||
match &entry {
|
||||
Json::Obj(pairs) => pairs
|
||||
.iter()
|
||||
.find(|(k, _)| k == "status")
|
||||
.and_then(|(_, v)| match v {
|
||||
Json::Str(x) => Some(x.clone()),
|
||||
_ => None,
|
||||
}),
|
||||
other => {
|
||||
return Err((
|
||||
format!("'{}' object has no attribute 'get'", py_type(other)),
|
||||
"AttributeError".to_string(),
|
||||
))
|
||||
}
|
||||
};
|
||||
match status.as_deref() {
|
||||
Some("warning") => warnings += 1,
|
||||
Some("critical") => criticals += 1,
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
Ok(obj(vec![
|
||||
("total", Json::Num(total as f64)),
|
||||
("warnings", Json::Num(warnings as f64)),
|
||||
("criticals", Json::Num(criticals as f64)),
|
||||
]))
|
||||
}
|
||||
|
||||
// ── handle_hook ───────────────────────────────────────────────────────────
|
||||
|
||||
/// Python repr() for the value lattice the hook's str(data) fallback can
|
||||
/// see: single-quote-first strings, True/False/None, int/float, list, dict.
|
||||
fn py_repr(v: &Json) -> String {
|
||||
match v {
|
||||
Json::Null => "None".to_string(),
|
||||
Json::Bool(true) => "True".to_string(),
|
||||
Json::Bool(false) => "False".to_string(),
|
||||
Json::Num(n) if n.is_finite() && n.fract() == 0.0 => format!("{}", *n as i64),
|
||||
Json::Num(n) => format!("{n}"),
|
||||
Json::Float(n) => format!("{n:?}"),
|
||||
Json::Str(x) => repr_str(x),
|
||||
Json::Arr(items) => {
|
||||
let inner: Vec<String> = items.iter().map(py_repr).collect();
|
||||
format!("[{}]", inner.join(", "))
|
||||
}
|
||||
Json::Obj(pairs) => {
|
||||
let inner: Vec<String> = pairs
|
||||
.iter()
|
||||
.map(|(k, val)| format!("{}: {}", repr_str(k), py_repr(val)))
|
||||
.collect();
|
||||
format!("{{{}}}", inner.join(", "))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn repr_str(x: &str) -> String {
|
||||
let use_dq = x.contains('\'') && !x.contains('"');
|
||||
let mut out = String::new();
|
||||
out.push(if use_dq { '"' } else { '\'' });
|
||||
for c in x.chars() {
|
||||
match c {
|
||||
'\\' => out.push_str("\\\\"),
|
||||
'\n' => out.push_str("\\n"),
|
||||
'\r' => out.push_str("\\r"),
|
||||
'\t' => out.push_str("\\t"),
|
||||
'"' if use_dq => out.push_str("\\\""),
|
||||
'\'' if !use_dq => out.push_str("\\'"),
|
||||
c if (c as u32) < 0x20 || c as u32 == 0x7f => {
|
||||
out.push_str(&format!("\\x{:02x}", c as u32));
|
||||
}
|
||||
c => out.push(c),
|
||||
}
|
||||
}
|
||||
out.push(if use_dq { '"' } else { '\'' });
|
||||
out
|
||||
}
|
||||
|
||||
fn hook_result(event: &str) -> Json {
|
||||
obj(vec![
|
||||
("status", Json::Str("ok".to_string())),
|
||||
("event", Json::Str(event.to_string())),
|
||||
("handler", Json::Str("context_pressure".to_string())),
|
||||
])
|
||||
}
|
||||
|
||||
/// handle_hook — on_message + truthy data → measure the derived message,
|
||||
/// log its entry, surface pressure/recommendation when not "ok".
|
||||
pub fn handle_hook(event: &str, data: &Json, now: &str) -> Result<Json, (String, String)> {
|
||||
if event != "on_message" || !truthy(data) {
|
||||
return Ok(obj(vec![
|
||||
("result", hook_result(event)),
|
||||
("entry", Json::Null),
|
||||
]));
|
||||
}
|
||||
// data.get(...) on a truthy non-dict: the oracle's AttributeError.
|
||||
let get = |want: &str| -> Result<Json, (String, String)> {
|
||||
match data {
|
||||
Json::Obj(pairs) => Ok(pairs
|
||||
.iter()
|
||||
.find(|(k, _)| k == want)
|
||||
.map(|(_, v)| v.clone())
|
||||
.unwrap_or(Json::Str(String::new()))),
|
||||
other => Err((
|
||||
format!("'{}' object has no attribute 'get'", py_type(other)),
|
||||
"AttributeError".to_string(),
|
||||
)),
|
||||
}
|
||||
};
|
||||
let message = get("message")?;
|
||||
let text: Json = if truthy(&message) {
|
||||
message.clone()
|
||||
} else {
|
||||
let alt = get("text")?;
|
||||
if truthy(&alt) {
|
||||
alt
|
||||
} else {
|
||||
Json::Str(py_repr(data))
|
||||
}
|
||||
};
|
||||
let measurement = measure_pressure_doc(&text, 128000.0)?;
|
||||
let entry = log_measurement(&measurement, "", now);
|
||||
if s(&measurement, "status") != "ok" {
|
||||
return Ok(obj(vec![
|
||||
(
|
||||
"result",
|
||||
obj(vec![
|
||||
("status", Json::Str("ok".to_string())),
|
||||
("event", Json::Str(event.to_string())),
|
||||
("handler", Json::Str("context_pressure".to_string())),
|
||||
(
|
||||
"pressure",
|
||||
measurement.get("pressure").cloned().unwrap_or(Json::Null),
|
||||
),
|
||||
(
|
||||
"recommendation",
|
||||
measurement
|
||||
.get("recommendation")
|
||||
.cloned()
|
||||
.unwrap_or(Json::Null),
|
||||
),
|
||||
]),
|
||||
),
|
||||
("entry", entry),
|
||||
]));
|
||||
}
|
||||
Ok(obj(vec![("result", hook_result(event)), ("entry", entry)]))
|
||||
}
|
||||
|
||||
// ── main / CLI dispatch (printing + file IO stay with the caller) ─────────
|
||||
|
||||
struct CliArgs {
|
||||
cmd: String,
|
||||
arg: String,
|
||||
max_tokens: f64,
|
||||
event: String,
|
||||
data: String,
|
||||
limit: i64,
|
||||
json: bool,
|
||||
}
|
||||
|
||||
/// argparse law for valid shapes; anything invalid (missing cmd, unknown
|
||||
/// command/flag, bad int, extra positional, flag without value) exits 2
|
||||
/// with usage on STDERR — stderr text is outside this contract.
|
||||
fn cli_err() -> Json {
|
||||
obj(vec![
|
||||
("output", Json::Str(String::new())),
|
||||
("code", Json::Num(2.0)),
|
||||
("entry", Json::Null),
|
||||
])
|
||||
}
|
||||
|
||||
fn parse_cli(argv: &[String]) -> Result<CliArgs, Json> {
|
||||
let mut args = CliArgs {
|
||||
cmd: String::new(),
|
||||
arg: String::new(),
|
||||
max_tokens: 128000.0,
|
||||
event: String::new(),
|
||||
data: String::new(),
|
||||
limit: 20,
|
||||
json: false,
|
||||
};
|
||||
let mut positionals: Vec<String> = Vec::new();
|
||||
let mut i = 1; // argv[0] is the program name
|
||||
while i < argv.len() {
|
||||
let t = argv[i].as_str();
|
||||
match t {
|
||||
"--json" => {
|
||||
args.json = true;
|
||||
i += 1;
|
||||
}
|
||||
"--max-tokens" | "--limit" | "--event" | "--data" => {
|
||||
let Some(val) = argv.get(i + 1) else {
|
||||
return Err(cli_err());
|
||||
};
|
||||
match t {
|
||||
"--max-tokens" => match val.parse::<i64>() {
|
||||
Ok(v) => args.max_tokens = v as f64,
|
||||
Err(_) => return Err(cli_err()),
|
||||
},
|
||||
"--limit" => match val.parse::<i64>() {
|
||||
Ok(v) => args.limit = v,
|
||||
Err(_) => return Err(cli_err()),
|
||||
},
|
||||
"--event" => args.event = val.clone(),
|
||||
_ => args.data = val.clone(),
|
||||
}
|
||||
i += 2;
|
||||
}
|
||||
t if t.starts_with("--") => return Err(cli_err()),
|
||||
_ => {
|
||||
positionals.push(argv[i].clone());
|
||||
i += 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
match positionals.len() {
|
||||
1 => args.cmd = positionals[0].clone(),
|
||||
2 => {
|
||||
args.cmd = positionals[0].clone();
|
||||
args.arg = positionals[1].clone();
|
||||
}
|
||||
_ => return Err(cli_err()), // 0: missing cmd; 3+: extra positional
|
||||
}
|
||||
if !matches!(
|
||||
args.cmd.as_str(),
|
||||
"measure" | "degrade" | "log" | "stats" | "hook"
|
||||
) {
|
||||
return Err(cli_err());
|
||||
}
|
||||
Ok(args)
|
||||
}
|
||||
|
||||
/// context_pressure.main — CLI dispatch. Returns {output, code, entry|null}:
|
||||
/// the caller prints `output` (print adds the newline) and appends `entry`
|
||||
/// as a JSONL line when the hook arm produced one.
|
||||
pub fn main_dispatch(argv: &[String], content: &str, now: &str) -> Result<Json, (String, String)> {
|
||||
let args = match parse_cli(argv) {
|
||||
Ok(a) => a,
|
||||
Err(e) => return Ok(e), // argparse error: code 2, empty stdout
|
||||
};
|
||||
match args.cmd.as_str() {
|
||||
"measure" => {
|
||||
let result = measure_pressure(&args.arg, args.max_tokens)?;
|
||||
Ok(cli_out(&result, args.json, Json::Null))
|
||||
}
|
||||
"degrade" => {
|
||||
let responses: Vec<Json> = args
|
||||
.arg
|
||||
.split('|')
|
||||
.filter(|r| !r.trim().is_empty())
|
||||
.map(|r| Json::Str(r.trim().to_string()))
|
||||
.collect();
|
||||
let result = detect_degradation(&responses)?;
|
||||
Ok(cli_out(&result, args.json, Json::Null))
|
||||
}
|
||||
"log" => Ok(cli_out(
|
||||
&read_log(content, args.limit),
|
||||
args.json,
|
||||
Json::Null,
|
||||
)),
|
||||
"stats" => {
|
||||
let st = log_stats(content)?;
|
||||
Ok(cli_out(&st, args.json, Json::Null))
|
||||
}
|
||||
"hook" => {
|
||||
let event = if args.event.is_empty() {
|
||||
"unknown"
|
||||
} else {
|
||||
args.event.as_str()
|
||||
};
|
||||
let data = if args.data.is_empty() {
|
||||
Json::Obj(Vec::new())
|
||||
} else {
|
||||
wire(&args.data)?
|
||||
};
|
||||
let hh = handle_hook(event, &data, now)?;
|
||||
let out = pyjson::dumps(&hh.get("result").cloned().unwrap_or(Json::Null));
|
||||
Ok(obj(vec![
|
||||
("output", Json::Str(out)),
|
||||
("code", Json::Num(0.0)),
|
||||
("entry", hh.get("entry").cloned().unwrap_or(Json::Null)),
|
||||
]))
|
||||
}
|
||||
_ => Ok(cli_err()),
|
||||
}
|
||||
}
|
||||
|
||||
fn cli_out(value: &Json, as_json: bool, entry: Json) -> Json {
|
||||
let out = if as_json {
|
||||
pyjson::dump_indent2(value)
|
||||
} else {
|
||||
plain_render(value)
|
||||
};
|
||||
obj(vec![
|
||||
("output", Json::Str(out)),
|
||||
("code", Json::Num(0.0)),
|
||||
("entry", entry),
|
||||
])
|
||||
}
|
||||
|
||||
/// The non-json renderings: measure/degrade/log/stats human lines.
|
||||
fn plain_render(value: &Json) -> String {
|
||||
if let Json::Arr(entries) = value {
|
||||
// log arm
|
||||
return entries
|
||||
.iter()
|
||||
.map(|e| {
|
||||
let ts_full = match e.get("timestamp") {
|
||||
Some(Json::Str(x)) => x.clone(),
|
||||
_ => "?".to_string(),
|
||||
};
|
||||
let ts: String = ts_full.chars().take(19).collect();
|
||||
let p = match e.get("pressure") {
|
||||
Some(Json::Num(n)) | Some(Json::Float(n)) => *n,
|
||||
_ => 0.0,
|
||||
};
|
||||
format!(" {} {} ({})", ts, fmt_pct(p), s(e, "status"))
|
||||
})
|
||||
.collect::<Vec<String>>()
|
||||
.join("\n");
|
||||
}
|
||||
if value.get("recommendation").is_some() {
|
||||
// measure arm — the ROUNDED pressure, {:.1%}-rendered
|
||||
let p = match value.get("pressure") {
|
||||
Some(Json::Num(n)) | Some(Json::Float(n)) => *n,
|
||||
_ => 0.0,
|
||||
};
|
||||
return format!(
|
||||
"Pressure: {} ({}) — {}",
|
||||
fmt_pct(p),
|
||||
s(value, "status"),
|
||||
s(value, "recommendation")
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_degradation() {
|
||||
let responses: Vec<String> =
|
||||
vec!["long response".repeat(100), "short".into(), "tiny".into()];
|
||||
assert!(detect_degradation(&responses));
|
||||
if value.get("degradation").is_some() {
|
||||
// degrade arm
|
||||
let d = match value.get("degradation") {
|
||||
Some(Json::Num(n)) | Some(Json::Float(n)) => *n,
|
||||
_ => 0.0,
|
||||
};
|
||||
let signals = match value.get("signals") {
|
||||
Some(Json::Arr(items)) => items
|
||||
.iter()
|
||||
.filter_map(|x| match x {
|
||||
Json::Str(y) => Some(y.clone()),
|
||||
_ => None,
|
||||
})
|
||||
.collect::<Vec<String>>()
|
||||
.join(", "),
|
||||
_ => String::new(),
|
||||
};
|
||||
return format!("Degradation: {} — signals: {}", fmt_pct(d), signals);
|
||||
}
|
||||
if value.get("total").is_some() {
|
||||
// stats arm
|
||||
let cnt = |k: &str| -> i64 {
|
||||
match value.get(k) {
|
||||
Some(Json::Num(n)) | Some(Json::Float(n)) => *n as i64,
|
||||
_ => 0,
|
||||
}
|
||||
};
|
||||
format!(
|
||||
"Total: {}, Warnings: {}, Criticals: {}",
|
||||
cnt("total"),
|
||||
cnt("warnings"),
|
||||
cnt("criticals")
|
||||
)
|
||||
} else {
|
||||
String::new()
|
||||
}
|
||||
}
|
||||
|
||||
// ── FFI (doc-envelope) ────────────────────────────────────────────────────
|
||||
|
||||
fn envelope(result: Result<Json, (String, String)>) -> *mut c_char {
|
||||
let body = match result {
|
||||
Ok(value) => obj(vec![("ok", Json::Bool(true)), ("value", value)]),
|
||||
Err((err, ty)) => obj(vec![
|
||||
("ok", Json::Bool(false)),
|
||||
("err", Json::Str(err)),
|
||||
("type", Json::Str(ty)),
|
||||
]),
|
||||
};
|
||||
let out = pyjson::dumps(&body);
|
||||
match CString::new(out) {
|
||||
Ok(c) => c.into_raw(),
|
||||
Err(_) => CString::new(r#"{"ok": false, "err": "nul in output", "type": "RuntimeError"}"#)
|
||||
.unwrap()
|
||||
.into_raw(),
|
||||
}
|
||||
}
|
||||
|
||||
fn read_arg(ptr: *const c_char) -> Result<String, (String, String)> {
|
||||
if ptr.is_null() {
|
||||
return Ok(String::new());
|
||||
}
|
||||
unsafe { CStr::from_ptr(ptr) }
|
||||
.to_str()
|
||||
.map(|s| s.to_string())
|
||||
.map_err(|e| (e.to_string(), "ValueError".to_string()))
|
||||
}
|
||||
|
||||
macro_rules! ffi {
|
||||
($name:ident, $body:expr) => {
|
||||
#[no_mangle]
|
||||
pub extern "C" fn $name(arg: *const c_char) -> *mut c_char {
|
||||
let out: Result<Json, (String, String)> = read_arg(arg).and_then(|raw| $body(&raw));
|
||||
envelope(out)
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
ffi!(
|
||||
cpx_estimate_tokens,
|
||||
|raw: &str| -> Result<Json, (String, String)> {
|
||||
let doc = wire(raw)?;
|
||||
Ok(Json::Num(
|
||||
tokens_of(&doc.get("text").cloned().unwrap_or(Json::Null))? as f64,
|
||||
))
|
||||
}
|
||||
);
|
||||
|
||||
ffi!(
|
||||
cpx_measure_pressure,
|
||||
|raw: &str| -> Result<Json, (String, String)> {
|
||||
let doc = wire(raw)?;
|
||||
let max_tokens = match doc.get("max_tokens") {
|
||||
Some(Json::Num(n)) | Some(Json::Float(n)) => *n,
|
||||
_ => 128000.0,
|
||||
};
|
||||
measure_pressure_doc(&doc.get("text").cloned().unwrap_or(Json::Null), max_tokens)
|
||||
}
|
||||
);
|
||||
|
||||
ffi!(cpx_detect_degradation, |raw: &str| -> Result<
|
||||
Json,
|
||||
(String, String),
|
||||
> {
|
||||
let doc = wire(raw)?;
|
||||
let responses: Vec<Json> = match doc.get("responses") {
|
||||
Some(Json::Arr(items)) => items.clone(),
|
||||
_ => Vec::new(),
|
||||
};
|
||||
detect_degradation(&responses)
|
||||
});
|
||||
|
||||
ffi!(
|
||||
cpx_log_measurement,
|
||||
|raw: &str| -> Result<Json, (String, String)> {
|
||||
let doc = wire(raw)?;
|
||||
Ok(log_measurement(
|
||||
&doc.get("measurement").cloned().unwrap_or(Json::Null),
|
||||
&s(&doc, "session_id"),
|
||||
&s(&doc, "now"),
|
||||
))
|
||||
}
|
||||
);
|
||||
|
||||
ffi!(
|
||||
cpx_read_log,
|
||||
|raw: &str| -> Result<Json, (String, String)> {
|
||||
let doc = wire(raw)?;
|
||||
let limit = match doc.get("limit") {
|
||||
Some(Json::Num(n)) | Some(Json::Float(n)) => *n as i64,
|
||||
_ => 20,
|
||||
};
|
||||
Ok(read_log(&s(&doc, "content"), limit))
|
||||
}
|
||||
);
|
||||
|
||||
ffi!(
|
||||
cpx_log_stats,
|
||||
|raw: &str| -> Result<Json, (String, String)> {
|
||||
let doc = wire(raw)?;
|
||||
log_stats(&s(&doc, "content"))
|
||||
}
|
||||
);
|
||||
|
||||
ffi!(
|
||||
cpx_handle_hook,
|
||||
|raw: &str| -> Result<Json, (String, String)> {
|
||||
let doc = wire(raw)?;
|
||||
handle_hook(
|
||||
&s(&doc, "event"),
|
||||
&doc.get("data").cloned().unwrap_or(Json::Null),
|
||||
&s(&doc, "now"),
|
||||
)
|
||||
}
|
||||
);
|
||||
|
||||
ffi!(
|
||||
cpx_main_dispatch,
|
||||
|raw: &str| -> Result<Json, (String, String)> {
|
||||
let doc = wire(raw)?;
|
||||
let argv: Vec<String> = match doc.get("argv") {
|
||||
Some(Json::Arr(items)) => items
|
||||
.iter()
|
||||
.filter_map(|x| match x {
|
||||
Json::Str(y) => Some(y.clone()),
|
||||
_ => None,
|
||||
})
|
||||
.collect(),
|
||||
_ => Vec::new(),
|
||||
};
|
||||
main_dispatch(&argv, &s(&doc, "content"), &s(&doc, "now"))
|
||||
}
|
||||
);
|
||||
|
||||
#[cfg(test)]
|
||||
mod branch_battery;
|
||||
@@ -0,0 +1,212 @@
|
||||
"""Tests: context_pressure.py Rust core port (dichotomy wave, #2479).
|
||||
|
||||
Delegation + fallback-pin discipline: token estimation, the pressure
|
||||
ladder, degradation signals, entry assembly, log tail parsing, stats
|
||||
counting, and hook message derivation come from the Rust core when
|
||||
available (rune.cps_loader); RUNE_CPS_CORE=python pins the Python oracle.
|
||||
Every probe runs in BOTH pins — the dual parametrize fixture covers the
|
||||
strangler transition (the pure-Python floor stays load-bearing when the
|
||||
crate is unbuilt).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
sys.path.insert(0, str(ROOT / "src"))
|
||||
|
||||
from rune import context_pressure as cp # noqa: E402
|
||||
from rune import cps_loader # noqa: E402
|
||||
|
||||
|
||||
@pytest.fixture(params=["python", "kernel"])
|
||||
def both_pins(request, monkeypatch):
|
||||
"""'python' pins the oracle; 'kernel' leaves delegation live (skipped
|
||||
when the crate is unbuilt)."""
|
||||
if request.param == "kernel" and not cps_loader.available():
|
||||
pytest.skip("rust context_pressure core not built")
|
||||
if request.param == "python":
|
||||
monkeypatch.setenv("RUNE_CPS_CORE", "python")
|
||||
monkeypatch.setenv("RUNE_CONTEXT_PRESSURE_CORE", "python")
|
||||
else:
|
||||
monkeypatch.delenv("RUNE_CPS_CORE", raising=False)
|
||||
monkeypatch.delenv("RUNE_CONTEXT_PRESSURE_CORE", raising=False)
|
||||
return request.param
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def log_home(tmp_path, monkeypatch):
|
||||
"""Hermetic LOG_PATH for this test (the oracle resolves LOG_PATH at
|
||||
import — re-point the module attr, mirroring bh.DB_PATH discipline)."""
|
||||
index = tmp_path / "memory" / "index"
|
||||
index.mkdir(parents=True)
|
||||
target = index / "context_pressure.jsonl"
|
||||
monkeypatch.setattr(cp, "LOG_PATH", str(target))
|
||||
return target
|
||||
|
||||
|
||||
def test_estimate_tokens(both_pins):
|
||||
assert cp.estimate_tokens("") == 0
|
||||
assert cp.estimate_tokens("hello world") == 3 # int(11/3.5) = 3
|
||||
assert cp.estimate_tokens("héllo wörld") == 3 # codepoints, not bytes
|
||||
assert cp.estimate_tokens("👍👍") == 1 # int(2/3.5) → max(1, 0)
|
||||
assert cp.estimate_tokens("x" * 1000) == 285
|
||||
|
||||
|
||||
def test_measure_pressure_ladder(both_pins):
|
||||
empty = cp.measure_pressure("")
|
||||
assert empty["tokens"] == 0 and empty["max_tokens"] == 128000
|
||||
# float law: the pressure is 0.0 (float), rendering "0.0" not "0"
|
||||
assert empty["pressure"] == 0.0
|
||||
assert json.dumps(empty["pressure"]) == "0.0"
|
||||
assert empty["status"] == "ok" and empty["recommendation"] == "none"
|
||||
warn = cp.measure_pressure("x" * 49000, max_tokens=20000)
|
||||
assert warn["tokens"] == 14000 and warn["pressure"] == 0.7
|
||||
assert warn["status"] == "warning"
|
||||
assert warn["recommendation"] == "summarize_old_turns"
|
||||
crit = cp.measure_pressure("x" * 95000, max_tokens=12000)
|
||||
assert crit["pressure"] == 1.0 and crit["status"] == "critical"
|
||||
assert crit["recommendation"] == "truncate_and_summarize"
|
||||
# doc key order law (insertion order: tokens, max_tokens, pressure, ...)
|
||||
assert list(empty.keys()) == [
|
||||
"tokens", "max_tokens", "pressure", "status", "recommendation"]
|
||||
|
||||
|
||||
def test_measure_pressure_zero_divides_crash(both_pins):
|
||||
with pytest.raises(ZeroDivisionError):
|
||||
cp.measure_pressure("hello", max_tokens=0)
|
||||
|
||||
|
||||
def test_detect_degradation_insufficient(both_pins):
|
||||
out = cp.detect_degradation(["a", "b"])
|
||||
assert out == {"degradation": 0.0, "signals": [],
|
||||
"insufficient_data": True}
|
||||
assert list(out.keys()) == ["degradation", "signals", "insufficient_data"]
|
||||
|
||||
|
||||
def test_detect_degradation_signals(both_pins):
|
||||
decline = cp.detect_degradation(["x" * 200, "y" * 180, "", "", ""])
|
||||
assert decline["degradation"] == 0.4
|
||||
assert decline["signals"] == ["response_length_declining",
|
||||
"responses_becoming_generic"]
|
||||
assert decline["length_decline"] == 1.0
|
||||
assert json.dumps(decline["length_decline"]) == "1.0"
|
||||
assert decline["generic_ratio"] == 1.0
|
||||
# int-0 law: max(0, 0.0) keeps the INT 0 (json "0", not "0.0")
|
||||
zero = cp.detect_degradation(["aaa", "bbb", "ccc"])
|
||||
assert zero["length_decline"] == 0
|
||||
assert json.dumps(zero["length_decline"]) == "0"
|
||||
assert zero["repetition_score"] == 0
|
||||
assert zero["generic_ratio"] == 1.0
|
||||
assert zero["degradation"] == 0.3
|
||||
assert zero["signals"] == ["responses_becoming_generic"]
|
||||
|
||||
|
||||
def test_detect_degradation_repetition(both_pins):
|
||||
rep = cp.detect_degradation(["dup word word", "word dup word",
|
||||
"word word dup"])
|
||||
# 9 words, 2 unique → unique_ratio 2/9 → 1.0 - 0.4444 → 0.556
|
||||
assert rep["repetition_score"] == 0.556
|
||||
assert "repetition_increasing" in rep["signals"]
|
||||
|
||||
|
||||
def test_log_measurement_line_shape(both_pins, log_home):
|
||||
m = {"tokens": 3, "max_tokens": 128000, "pressure": 0.0,
|
||||
"status": "ok", "recommendation": "none"}
|
||||
cp.log_measurement(m, "sid-1")
|
||||
line = (log_home.read_text()).rstrip("\n")
|
||||
entry = json.loads(line)
|
||||
# {"timestamp", "session_id", **measurement} — insertion order law
|
||||
assert list(entry.keys()) == ["timestamp", "session_id", "tokens",
|
||||
"max_tokens", "pressure", "status",
|
||||
"recommendation"]
|
||||
assert entry["session_id"] == "sid-1"
|
||||
assert entry["status"] == "ok"
|
||||
# spread-override keeps first position with the new value
|
||||
cp.log_measurement({"timestamp": "T-override", "user_key": "u"}, "")
|
||||
entry2 = json.loads(log_home.read_text().splitlines()[-1])
|
||||
assert list(entry2.keys()) == ["timestamp", "session_id", "user_key"]
|
||||
assert entry2["timestamp"] == "T-override"
|
||||
|
||||
|
||||
def test_read_log_tail_and_filters(both_pins, log_home):
|
||||
log_home.write_text('{"a": 1}\n\nbroken\n{"b": 2}\n')
|
||||
entries = cp.read_log(limit=20)
|
||||
assert entries == [{"a": 1}, {"b": 2}]
|
||||
tail = cp.read_log(limit=2)
|
||||
assert tail == [{"b": 2}]
|
||||
assert cp.read_log(limit=0) == []
|
||||
|
||||
|
||||
def test_read_log_missing_file(both_pins, log_home):
|
||||
assert cp.read_log() == []
|
||||
|
||||
|
||||
def test_read_log_no_phantom_trailing_line(both_pins, log_home):
|
||||
log_home.write_text("a\nb\nc\n")
|
||||
assert cp.read_log(limit=2) == []
|
||||
|
||||
|
||||
def test_log_stats_counts(both_pins, log_home):
|
||||
log_home.write_text('{"status": "warning"}\n{"status": "critical"}\n'
|
||||
'{"status": "warning"}\n\nbroken\n'
|
||||
'{"status": "ok"}\n')
|
||||
assert cp.log_stats() == {"total": 4, "warnings": 2, "criticals": 1}
|
||||
|
||||
|
||||
def test_log_stats_empty_log(both_pins, log_home):
|
||||
log_home.write_text("")
|
||||
assert cp.log_stats() == {"total": 0, "warnings": 0, "criticals": 0}
|
||||
|
||||
|
||||
def test_log_stats_non_dict_line_crash(both_pins, log_home):
|
||||
log_home.write_text("5\n")
|
||||
with pytest.raises(AttributeError):
|
||||
cp.log_stats()
|
||||
|
||||
|
||||
def test_handle_hook_logs_and_surfaces(both_pins, log_home):
|
||||
out = cp.handle_hook("on_message", {"text": "hello"})
|
||||
assert out == {"status": "ok", "event": "on_message",
|
||||
"handler": "context_pressure"}
|
||||
entries = [json.loads(x) for x in log_home.read_text().splitlines() if x]
|
||||
assert len(entries) == 1 and entries[0]["status"] == "ok"
|
||||
|
||||
|
||||
def test_handle_hook_pressure_rich_result(both_pins, log_home):
|
||||
# 403500 chars → 115285 tokens / 128000 = 0.901 ≥ 0.9 → critical
|
||||
out = cp.handle_hook("on_message", {"message": "x" * 403500})
|
||||
assert out["pressure"] == 0.901
|
||||
assert out["recommendation"] == "truncate_and_summarize"
|
||||
assert out["handler"] == "context_pressure"
|
||||
entries = [json.loads(x) for x in log_home.read_text().splitlines() if x]
|
||||
assert entries[0]["status"] == "critical"
|
||||
|
||||
|
||||
def test_handle_hook_fallbacks(both_pins, log_home):
|
||||
# str(data) fallback measures the dict repr and logs one line
|
||||
out = cp.handle_hook("on_message", {"other": 1})
|
||||
assert out["status"] == "ok"
|
||||
assert log_home.read_text() != "" # the measured repr got logged
|
||||
# empty data / wrong event: no ADDITIONAL log line
|
||||
lines_before = len([x for x in log_home.read_text().splitlines() if x])
|
||||
assert cp.handle_hook("on_message", {}) == {
|
||||
"status": "ok", "event": "on_message", "handler": "context_pressure"}
|
||||
assert cp.handle_hook("ping", {"message": "hello"}) == {
|
||||
"status": "ok", "event": "ping", "handler": "context_pressure"}
|
||||
assert cp.handle_hook("on_message", None) == {
|
||||
"status": "ok", "event": "on_message", "handler": "context_pressure"}
|
||||
lines_after = len([x for x in log_home.read_text().splitlines() if x])
|
||||
assert lines_after == lines_before
|
||||
|
||||
|
||||
def test_plugin_shape(both_pins):
|
||||
assert cp.PLUGIN["name"] == "context_pressure"
|
||||
assert cp.PLUGIN["hooks"] == {"on_message": "handle_hook"}
|
||||
assert cp.PLUGIN["priority"] == 2
|
||||
@@ -0,0 +1 @@
|
||||
FULL: 55% (105/192 modules) | fn-level: 1612/2282 (71%) | PARTIAL 48 | NOT TOUCHED 39
|
||||
@@ -0,0 +1 @@
|
||||
| context_pressure | 262 | 3/9 | log_measurement, read_log, log_stats, _now, handle_hook, main |
|
||||
@@ -0,0 +1,2 @@
|
||||
written: /mnt/workdrive/rune-wave-cy-db/docs/dichotomy-rebase-honest-audit.md
|
||||
FULL: 55% (105/192 modules) | fn-level: 1612/2282 (71%) | PARTIAL 48 | NOT TOUCHED 39
|
||||
@@ -0,0 +1 @@
|
||||
0
|
||||
@@ -0,0 +1 @@
|
||||
0
|
||||
@@ -0,0 +1 @@
|
||||
Finished `release` profile [optimized] target(s) in 0.20s
|
||||
@@ -0,0 +1 @@
|
||||
0
|
||||
@@ -0,0 +1,24 @@
|
||||
Compiling context_pressure v0.1.0 (/mnt/workdrive/rune-wave-cy-db/src/rust/context_pressure)
|
||||
Finished `release` profile [optimized] target(s) in 0.63s
|
||||
Running unittests src/lib.rs (target/release/deps/context_pressure-08cc745c6cecf623)
|
||||
|
||||
running 10 tests
|
||||
test branch_battery::banker_round_helper_contract ... ok
|
||||
test branch_battery::replay_log_stats ... ok
|
||||
test branch_battery::replay_log_measurement_lines ... ok
|
||||
test branch_battery::replay_detect_degradation ... ok
|
||||
test branch_battery::replay_measure_pressure ... ok
|
||||
test branch_battery::replay_read_log ... ok
|
||||
test branch_battery::replay_estimate_tokens ... ok
|
||||
test branch_battery::replay_main_dispatch ... ok
|
||||
test branch_battery::replay_kernel_handle_hook ... ok
|
||||
test branch_battery::replay_handle_hook_result ... ok
|
||||
|
||||
test result: ok. 10 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s
|
||||
|
||||
Doc-tests context_pressure
|
||||
|
||||
running 0 tests
|
||||
|
||||
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
0
|
||||
@@ -0,0 +1,2 @@
|
||||
Checking context_pressure v0.1.0 (/mnt/workdrive/rune-wave-cy-db/src/rust/context_pressure)
|
||||
Finished `release` profile [optimized] target(s) in 0.14s
|
||||
@@ -0,0 +1 @@
|
||||
0
|
||||
Whitespace-only changes.
@@ -0,0 +1 @@
|
||||
0
|
||||
@@ -0,0 +1 @@
|
||||
OK: 28 cases byte-identical against live oracle rune.context_pressure
|
||||
@@ -0,0 +1 @@
|
||||
0
|
||||
@@ -0,0 +1 @@
|
||||
parity: frozen=78 regen=6 fuzz=1400 live=18 failures=0
|
||||
@@ -0,0 +1 @@
|
||||
0
|
||||
@@ -0,0 +1,2 @@
|
||||
.................................. [100%]
|
||||
34 passed in 0.36s
|
||||
@@ -0,0 +1 @@
|
||||
0
|
||||
@@ -0,0 +1 @@
|
||||
All checks passed!
|
||||
@@ -0,0 +1 @@
|
||||
0
|
||||
@@ -0,0 +1 @@
|
||||
frozen 75 cases -> /mnt/workdrive/rune-wave-cy-db/src/rust/context_pressure/src/frozen_cp_truth.json
|
||||
@@ -0,0 +1 @@
|
||||
0
|
||||
@@ -0,0 +1 @@
|
||||
frozen 75 cases -> /mnt/workdrive/rune-wave-cy-db/src/rust/context_pressure/src/frozen_cp_truth.json
|
||||
@@ -0,0 +1,47 @@
|
||||
# context_pressure gap-close — verification summary (worktree rune-wave-cy-db)
|
||||
|
||||
Module: src/rune/context_pressure.py (oracle 262L, 9 fns, was 3/9 PARTIAL → RUST side).
|
||||
Port: src/rust/context_pressure (FULL parity, 8 cpx_* FFI kernels + CLI dispatch;
|
||||
file IO + wall clock stay in the Python facade — no-clock law).
|
||||
|
||||
## Commands and exit codes (evidence files in verification/)
|
||||
| Stage | Command | Exit | Evidence |
|
||||
|---|---|---|---|
|
||||
| build | `cargo build --release` | 0 | cp-cargo-build.{log,exit} |
|
||||
| truth regen | `PYTHONPATH=scripts/cps_freeze_seams python3 scripts/gen_context_pressure_truth.py` | 0 | cp-truth-regen.{log,exit} |
|
||||
| truth byte-identity | double regen, sha256 df8fcb27af9ef8fe3a470cbbf06f48c6c11541f7bbb16411493468b16176db6f stable | 0 | cp-truth-byteidentity.{log,exit} |
|
||||
| freeze-verify | `PYTHONPATH=scripts/cps_freeze_seams:src python3 scripts/freeze_truth.py --verify src/rust/context_pressure/src/frozen_cp_truth.json --module rune.context_pressure` → "OK: 28 cases byte-identical" | 0 | cp-freeze-verify.{log,exit} |
|
||||
| battery | `cargo test --release` → 10 passed | 0 | cp-cargo-test.{log,exit} |
|
||||
| clippy | `cargo clippy --release --all-targets -- -D warnings` (after `cargo fmt`) | 0 | cp-clippy.{log,exit}, cp-fmt.{log,exit} |
|
||||
| live parity | `PYTHONPATH=scripts/cps_freeze_seams python3 scripts/check_context_pressure_parity.py` → frozen=78 regen=6 fuzz=1400 live=18 failures=0 | 0 | cp-parity.{log,exit} |
|
||||
| pytest both pins | `python3 -m pytest tests/test_context_pressure.py -q` → 34 passed (both_pins fixture: python pin + kernel live) | 0 | cp-pytest.{log,exit} |
|
||||
| ruff | `ruff check` over all touched Python | 0 | cp-ruff.{log,exit} |
|
||||
| audit | `python3 scripts/dichotomy_audit.py` + `--write` | 0 | cp-audit-after.txt, cp-audit-write.log |
|
||||
|
||||
## Coverage
|
||||
- BEFORE (HEAD audit doc): context_pressure 262L, 3/9 PARTIAL — missing
|
||||
log_measurement, read_log, log_stats, _now, handle_hook, main (cp-audit-before.txt).
|
||||
- AFTER: context_pressure 351L, **9/9 FULL (RUST)** (cp-audit-after query;
|
||||
docs/dichotomy-rebase-honest-audit.md line "context_pressure | 351 | 9/9 | RUST").
|
||||
|
||||
## Divergences (documented, not bugs)
|
||||
- `_now` stays Python (no-clock law: the wall clock cannot cross FFI; the kernel
|
||||
receives ISO timestamps as arguments — audit counts it matched via the frozen
|
||||
clock contract the whole battery pins).
|
||||
- File IO (LOG_PATH append/read, CLI printing) stays in the Python facade;
|
||||
the kernel owns entry assembly, log parsing, stats counting, hook derivation,
|
||||
CLI dispatch decisions (facade prints/appends what the kernel returns).
|
||||
- read_log negative limit: oracle's deque(maxlen=negative) raises ValueError which
|
||||
the best-effort except floor degrades to []; kernel mirrors the OBSERVED [].
|
||||
- Integral floats crossing FFI render int-shaped in envelope JSON (pyjson Num
|
||||
law); live parity compares structurally, byte shape pinned by the battery.
|
||||
|
||||
## Notes
|
||||
- Frozen truth: 75 cases (28 envelope + 47 kernel rows incl. crash-parity rows
|
||||
with the exact str(e) contract) — regenerated byte-identically twice; the
|
||||
shared-harness --verify re-freeze matches the live oracle.
|
||||
- Fuzz: 1400 seeded comparisons (seed 2479) across estimate/measure/detect/
|
||||
entry-assembly/read_log/log_stats/handle_hook including crash parity.
|
||||
- Rig: scripts/cps_freeze_seams (RUNE_CPS_CORE + RUNE_CONTEXT_PRESSURE_CORE
|
||||
pinned EXPLICITLY — rig-pin-gap law; frozen clock via freeze_rigs.pin_clock;
|
||||
hermetic RUNE_MEMORY_DIR scratch; CPS_SEED_LOG for CLI subprocess rows).
|
||||
Reference in new issue
Block a user