feat(persona_definition): close PARTIAL gaps — dichotomy wave

audit 2/7 PARTIAL -> 7/7 FULL (GO side; _by_topic, compile_full,
emit_constitution, emit_identity, emit_persona ported name-preserving
into src/go/persona-definition; handle_hook + CLI main mirror-carried).
The store (memory_store sqlite) and cache (cache.py sqlite) sit behind
Env seams; compile_full's cache round-trip is byte-faithful: the cached
value renders with Python's ensure_ascii=False separators, the
compressed readback applies cache._compress verbatim (char-based
[:797]+'...' past 800 — rune-safe), and an unparseable/truncated
compressed value falls through to the store exactly like the oracle.
emit_persona ports the (priority or 5, -confidence) stable sort with
Python or-falsiness (priority 0 == absent == 5), the noise filters
(<20 chars, short leading-article lines, firecrawl+credit lessons) and
the 200-CHARACTER truncation. CLI is argparse-faithful (choices,
--event/--data, hook prints json.dumps with insertion-ordered keys and
ensure_ascii=True; invalid choice exits 2, bad --data JSON exits 1 —
crash parity).

Frozen truth: 33-case corpus via the shared freeze_truth.py harness +
wrapper shims in scripts/gen_persona_definition_gap_truth.py (per-row
hermetic mkdtemp store+cache DBs bound through the module globals
memory_store.DB_PATH / cache.DB_PATH BEFORE any oracle call; counters
wrap ms.connect / cache.get_cache / cache.put_cache so every capture
PROVES the path: miss->store->put, >800 truncation re-query, corrupted
preseed fallthrough, WHERE filters, importance DESC ordering).
Byte-identical --verify twice + fresh regen diff-empty. Battery
replays 33/33 VERBATIM (replay-gap subcommand = second modality) +
4 Go unit tests (char-boundary pins). Live parity
scripts/check_persona_definition_parity.py: 3/3 scenarios (14 ops) +
11/11 real-CLI invocations byte-identical (oracle CLI vs GO binary).
gofmt/vet/build clean. Python pin RUNE_PERSONA_DEFINITION_CORE=python
verify green (no delegation loader exists; oracle-pinned surface is
the live module).

Evidence: verification/pd-*.exit + pd-summary.md (logs local, *.log
gitignored).
This commit is contained in:
Nick committed 2026-10-14 21:05:48 +01:00
1 parent a0537fe04a
commit 6fb5c4c8d2
22 files changed
+4195 -5

No files matched your search

+5 -5
View File
@@ -2,14 +2,14 @@
_Regenerate with `python3 scripts/dichotomy_audit.py --write`. A module is FULL only when 80%+ of its Python functions have name-preserving counterparts in the Go/Rust implementation — directories without the functions are facades and count as NOT TOUCHED._
**FULL completion: 65%** (125/192 runtime modules at FULL)
**FULL completion: 66%** (126/192 runtime modules at FULL)
Function-level detail: 75% (1720/2301 runtime functions matched) — secondary; includes in-flight work
Function-level detail: 75% (1725/2301 runtime functions matched) — secondary; includes in-flight work
| Status | Modules | Lines | Meaning |
|---|---|---|---|
| FULL | 125 | 51027 | 80%+ of functions ported |
| PARTIAL | 28 | 13534 | 20–80% — real logic exists, gaps named below |
| FULL | 126 | 51237 | 80%+ of functions ported |
| PARTIAL | 27 | 13324 | 20–80% — real logic exists, gaps named below |
| NOT TOUCHED | 39 | 11471 | no name-preserving implementation |
| Total runtime | 192 | 76032 | |
@@ -68,7 +68,6 @@ Function-level detail: 75% (1720/2301 runtime functions matched) — secondary;
| time_awareness | 402 | 12/20 | _clock_args, _ta_core, get_time_awareness, to_dict, _to_state, _apply_state |
| intent_continuity | 284 | 6/13 | _now, _raise_core_err, to_dict, from_dict, save, _ledger_state |
| voicebox | 164 | 5/11 | _run, start_recording, stop_recording, _transcribe_and_type, on_press, on_release |
| persona_definition | 210 | 2/7 | _by_topic, compile_full, emit_constitution, emit_identity, emit_persona |
| drift_observability | 399 | 5/9 | _do_core, main, _load_alerts, _save_alerts |
| prompt_sync | 487 | 11/15 | _now, _rust_available, get_store, main |
| task_gate | 301 | 5/9 | _tg_core, _now, list_gates, main |
@@ -190,6 +189,7 @@ Function-level detail: 75% (1720/2301 runtime functions matched) — secondary;
| openai_compat_transport | 227 | 9/9 | GO |
| cognitive_budgeter | 221 | 3/3 | RUST |
| evidence_probe | 213 | 9/9 | GO |
| persona_definition | 210 | 7/7 | GO |
| project_registry | 205 | 10/12 | RUST |
| forge_client | 203 | 18/18 | GO |
| behaviour_ledger | 195 | 10/12 | RUST |
+259
View File
@@ -0,0 +1,259 @@
#!/usr/bin/env python3
"""check_persona_definition_parity.py — LIVE parity for the
persona_definition GO port (#2479 dichotomy gap-close): drives the Python
oracle (src/rune/persona_definition.py) and the GO binary through
IDENTICAL scenarios and diffs their outputs.
Two stages:
1. scenario parity — fn-level ops (by_topic, emit_*, handle_hook,
compile_full sequences with path-proving counters) against hermetic
throwaway DBs; the Python side runs the LIVE oracle module through
the generator's wrapper shims (rig, not logic), the Go side through
the binary's `scenario` subcommand. Compared as parsed JSON.
2. CLI parity — both real CLIs as subprocesses over identically-seeded
hermetic stores (env-pinned RUNE_STORE_DB/RUNE_MEMORY_DIR), stdout
compared BYTE-for-byte plus exit codes (argparse exit 2, bad --data
exit 1 crash parity).
Usage: python3 scripts/check_persona_definition_parity.py <go-binary>
(build first: cd src/go/persona-definition && go build -o /tmp/persona-definition .)
"""
import json
import os
import subprocess
import sys
import tempfile
from pathlib import Path
REPO = Path(__file__).resolve().parent.parent
sys.path.insert(0, str(REPO / "scripts"))
sys.path.insert(0, str(REPO / "src" / "rune"))
# Hermetic env BEFORE the first oracle import (cache.py/memory_store.py
# derive DB paths at import time).
_BOOT = tempfile.mkdtemp(prefix="pd-parity-boot-")
os.environ.setdefault("RUNE_MEMORY_DIR", _BOOT)
os.environ.setdefault("RUNE_STORE_DB", os.path.join(_BOOT, "store", "memory.db"))
import gen_persona_definition_gap_truth as G # noqa: E402 (rig + shims)
import memory_store as _MS # noqa: E402
PD = G.PD
def run_python_scenario(ops):
"""Oracle side: run each op through the live module / shims."""
steps = []
for op in ops:
if op["op"] == "fn":
fn = op["fn"]
kwargs = {}
if op.get("limit") is not None:
kwargs["limit"] = op["limit"]
if op.get("calls") is not None:
kwargs["calls"] = op["calls"]
if op.get("preseed_cache") is not None:
kwargs["preseed_cache"] = op["preseed_cache"]
args = []
if fn == "_pd_by_topic":
args = [op["rows"], op["topics"]]
elif fn in ("_pd_emit_constitution", "_pd_emit_identity", "_pd_emit_persona"):
args = [op["by"]]
elif fn == "_pd_handle_hook":
args = [op["event"]]
if op.get("data") is not None:
args.append(op["data"])
elif fn == "_pd_compile_full":
args = [op["seed"]]
wrapper = getattr(G, fn)
result = wrapper(*args, **kwargs)
steps.append({"op": "fn", "fn": fn, "result": result})
elif op["op"] == "cli":
# exact oracle main() print paths via the REAL CLI over a
# hermetic env (byte-identical stdout is the contract)
tmp = tempfile.mkdtemp(prefix="pd-parity-cli-")
_seed_full_store(os.path.join(tmp, "store.db"), op["seed"])
out, _rc = _run_cli("python", os.path.join(tmp, "store.db"),
os.path.join(tmp, "mem-py"), op["args"])
steps.append({"op": "cli", "result": out})
return {"steps": steps}
def _seed_full_store(path, seed):
"""Seed through the oracle's own schema (vec/fts included) so the
real CLI reads exactly what the battery's minimal schema mirrors."""
old = _MS.DB_PATH
_MS.DB_PATH = path
try:
db = _MS.connect()
try:
for s in seed:
db.execute(
"INSERT INTO entries (id, text, importance, created_at, topic, "
"status, always_on, confidence, kind, priority) "
"VALUES (?,?,?,?,?,?,?,?,?,?)",
(s["id"], s["text"], s["importance"], "2026-01-01T00:00:00",
s.get("topic"), s.get("status", "active"),
1 if s.get("always_on", True) else 0,
s.get("confidence"), s.get("kind", "value"), s.get("priority")))
db.commit()
finally:
db.close()
finally:
_MS.DB_PATH = old
def _run_cli(which, store_db, memory_dir, args):
"""One real CLI subprocess over a hermetic env; returns (stdout, rc)."""
env = dict(os.environ)
env["RUNE_STORE_DB"] = store_db
env["RUNE_MEMORY_DIR"] = memory_dir
if which == "python":
cmd = [sys.executable, str(REPO / "src" / "rune" / "persona_definition.py")] + args
else:
cmd = [BINARY] + args
r = subprocess.run(cmd, capture_output=True, text=True, timeout=60, env=env,
cwd=str(REPO))
return r.stdout, r.returncode
def run_go_scenario(sc_path):
r = subprocess.run([BINARY, "scenario", str(sc_path)],
capture_output=True, text=True, timeout=120)
if r.returncode != 0:
raise RuntimeError(f"go scenario failed: {r.stderr}")
return json.loads(r.stdout)
CONSTITUTION_TEXT = "The persona must never fabricate evidence it has not verified"
IDENTITY_TEXT = "I am a careful engineer who verifies before attesting 中文 ünïcode"
PERSONA_A = "Prefer root-cause fixes over bypassing safety checks in every landing"
PERSONA_B = "Write compact diffs that preserve unrelated user changes at all times"
SEED_SMALL = [
{"id": 7, "text": CONSTITUTION_TEXT, "topic": "Constitution",
"importance": 0.9, "confidence": 0.8, "kind": "value"},
{"id": 3, "text": PERSONA_A, "topic": "Persona",
"importance": 0.55, "confidence": 0.7, "kind": "value"},
{"id": 11, "text": IDENTITY_TEXT, "topic": "identity",
"importance": 0.3, "confidence": 0.4, "kind": "value"},
]
SEED_FILTER = [
{"id": 1, "text": "inactive entry must never appear", "topic": "persona",
"importance": 0.99, "status": "inactive"},
{"id": 2, "text": PERSONA_B, "topic": "PeRsOnA", "importance": 0.71,
"confidence": 0.5, "kind": "value"},
{"id": 4, "text": PERSONA_A, "topic": None, "importance": 0.66,
"confidence": 0.65, "kind": "rule"},
]
SCENARIOS = [
# 1: pure fns over a mixed by-matrix
{"ops": [
{"op": "fn", "fn": "_pd_by_topic", "rows": [
{"id": 7, "text": CONSTITUTION_TEXT, "topic": "Constitution",
"importance": 0.9, "confidence": 0.8, "kind": "value"},
{"id": 3, "text": PERSONA_A, "topic": "Persona",
"importance": 0.55, "confidence": 0.7, "kind": "value"},
], "topics": ["persona", "constitution"]},
{"op": "fn", "fn": "_pd_emit_persona", "by": {"persona": [
{"id": 21, "text": PERSONA_A, "confidence": 0.6, "priority": 3},
{"id": 22, "text": PERSONA_B, "confidence": 0.9, "priority": 1},
{"id": 23, "text": "Firecrawl ran out of credit again today", "confidence": 0.9, "priority": 1},
{"id": 24, "text": "short", "confidence": 0.9, "priority": 1},
]}},
{"op": "fn", "fn": "_pd_emit_persona", "by": {"persona": []}},
{"op": "fn", "fn": "_pd_handle_hook", "event": "on_startup",
"data": {"k": 1}},
{"op": "fn", "fn": "_pd_handle_hook", "event": "ü event"},
]},
# 2: compile_full store+cache matrix through the LIVE module
{"ops": [
{"op": "fn", "fn": "_pd_compile_full", "seed": SEED_SMALL},
{"op": "fn", "fn": "_pd_compile_full", "seed": SEED_FILTER},
{"op": "fn", "fn": "_pd_compile_full", "seed": SEED_SMALL,
"preseed_cache": "not json {"},
{"op": "fn", "fn": "_pd_compile_full", "seed": SEED_SMALL, "calls": 1},
]},
# 3: the CLI print paths over identically-seeded hermetic stores
{"ops": [
{"op": "cli", "seed": SEED_SMALL, "args": ["full"]},
{"op": "cli", "seed": SEED_SMALL, "args": ["compact"]},
{"op": "cli", "seed": SEED_SMALL, "args": ["constitution"]},
{"op": "cli", "seed": SEED_SMALL, "args": ["identity"]},
{"op": "cli", "seed": [], "args": ["full"]},
]},
]
CLI_CASES = [
# (args, seed) — both real CLIs, identical hermetic env, stdout byte diff
(["full"], SEED_SMALL),
(["compact"], SEED_SMALL),
(["constitution"], SEED_SMALL),
(["identity"], SEED_SMALL),
(["full"], SEED_FILTER),
(["compact"], []),
(["hook", "--event", "on_startup", "--data", '{"k": 1, "ü": "wörld"}'], []),
(["hook"], []),
(["hook", "--event", "ü ö"], []),
(["bogus-cmd"], []), # argparse exit 2 both sides
(["hook", "--data", "not json"], []), # json.loads crash, exit 1
]
def main():
global BINARY
BINARY = sys.argv[1] if len(sys.argv) > 1 else "/tmp/persona-definition"
ok = 0
# stage 1: scenario parity
for i, sc in enumerate(SCENARIOS, 1):
with tempfile.NamedTemporaryFile("w", suffix=".json", delete=False) as f:
json.dump(sc, f)
sc_path = f.name
try:
py = run_python_scenario(sc["ops"])
go = run_go_scenario(sc_path)
finally:
os.unlink(sc_path)
if py == go:
ok += 1
print(f"scenario {i}: LIVE PARITY OK ({len(sc['ops'])} ops)")
else:
print(f"scenario {i}: LIVE PARITY FAILED")
pj, gj = json.dumps(py, indent=1, sort_keys=True), json.dumps(go, indent=1, sort_keys=True)
for pl, gl in zip(pj.splitlines(), gj.splitlines()):
if pl != gl:
print(f" py: {pl}")
print(f" go: {gl}")
break
return 1
# stage 2: real-CLI byte parity
cli_ok = 0
for args, seed in CLI_CASES:
tmp_py = tempfile.mkdtemp(prefix="pd-cli-py-")
tmp_go = tempfile.mkdtemp(prefix="pd-cli-go-")
_seed_full_store(os.path.join(tmp_py, "store.db"), seed)
_seed_full_store(os.path.join(tmp_go, "store.db"), seed)
py_out, py_rc = _run_cli("python", os.path.join(tmp_py, "store.db"),
os.path.join(tmp_py, "mem"), args)
go_out, go_rc = _run_cli("go", os.path.join(tmp_go, "store.db"),
os.path.join(tmp_go, "mem"), args)
if py_out == go_out and py_rc == go_rc:
cli_ok += 1
else:
print(f"CLI PARITY FAILED: {args}")
print(f" py rc={py_rc} stdout={py_out!r}")
print(f" go rc={go_rc} stdout={go_out!r}")
return 1
print(f"CLI LIVE PARITY: {cli_ok}/{len(CLI_CASES)} invocations byte-identical "
f"(oracle CLI vs GO binary)")
total = len(SCENARIOS)
print(f"LIVE PARITY: {ok}/{total} scenarios + {cli_ok}/{len(CLI_CASES)} CLI "
f"invocations identical (oracle vs GO)")
return 0
if __name__ == "__main__":
sys.exit(main())
+356
View File
@@ -0,0 +1,356 @@
#!/usr/bin/env python3
"""gen_persona_definition_gap_truth.py — persona_definition gap-closing
truth battery (#2479 dichotomy wave, GO side).
Closes the PARTIAL audit gaps: _by_topic, compile_full, emit_constitution,
emit_identity, emit_persona (handle_hook/main already matched).
Uses the SHARED harness (scripts/freeze_truth.py) with the wrapper-shim law
(hook_engine/cron_engine pattern): the oracle's IO is sqlite through
module-global DB_PATHs — memory_store.DB_PATH (store) and cache.DB_PATH
(cache.db, computed at cache import from memory_env.memory_dir()) — both
re-pointed per row at hermetic mkdtemp DBs BEFORE any oracle call (the lazy
`import cache` inside compile_full binds the same sys.modules entry, so the
patch reaches it). Counters wrap ms.connect / cache.get_cache /
cache.put_cache so each capture PROVES which path compile_full took (cache
hit vs store re-query vs corrupted-entry fallthrough).
Capture shape for compile_full rows: {"r1", "r2", "store_connects",
"cache_gets": [hit bools], "cache_puts"} — NO timestamps, NO absolute
paths. Emit/by_topic/handle_hook rows are direct (pure fns).
Verify (this file ships its own --verify — same byte-compare law as the
harness, plus the shim-module registration the stock CLI cannot do):
python3 scripts/gen_persona_definition_gap_truth.py --verify
Usage: python3 scripts/gen_persona_definition_gap_truth.py (from repo root)
"""
import copy
import json
import os
import sys
import tempfile
from pathlib import Path
REPO = Path(__file__).resolve().parent.parent
sys.path.insert(0, str(REPO / "scripts"))
sys.path.insert(0, str(REPO / "src" / "rune"))
from freeze_truth import freeze, write_envelope # noqa: E402
FROZEN_NOW = "2026-10-14T18:00:00"
# Hermetic env BEFORE the first oracle import: cache.py/memory_store.py
# derive DB paths at import time (pin_home/oracle-quirks law), so even the
# import-time defaults never touch the live memory dir.
_BOOT_TMP = tempfile.mkdtemp(prefix="pd-boot-")
os.environ.setdefault("RUNE_MEMORY_DIR", _BOOT_TMP)
os.environ.setdefault("RUNE_STORE_DB", os.path.join(_BOOT_TMP, "store", "memory.db"))
os.environ.setdefault("RUNE_CACHE_DB", os.path.join(_BOOT_TMP, "cache.db"))
import memory_store as _MS # noqa: E402
import cache as _CACHE # noqa: E402
import persona_definition as PD # noqa: E402
_PD = {"ready": False}
def _prepare():
"""Idempotent per-process pinning (nothing to patch today — the oracle
reads no clock and its DB globals are set per row — kept for symmetry
with the cron_engine shim law)."""
if not _PD["ready"]:
_PD["ready"] = True
return PD
def _seed_store(seed):
"""Create a hermetic store DB with `seed` entry rows; returns the path.
Seeds use explicit ids, distinct non-integral importances (ORDER BY
importance DESC determinism; integral floats stay out of corpora per
the float-rendering quirk law) and always carry the columns _core's
SELECT touches.
"""
tmp = tempfile.mkdtemp(prefix="pd-freeze-")
store_path = os.path.join(tmp, "store.db")
cache_path = os.path.join(tmp, "cache.db")
_MS.DB_PATH = store_path
_CACHE.DB_PATH = cache_path
db = _MS.connect()
try:
for s in seed:
db.execute(
"INSERT INTO entries (id, text, importance, created_at, topic, "
"status, always_on, confidence, kind, priority) "
"VALUES (?,?,?,?,?,?,?,?,?,?)",
(s["id"], s["text"], s["importance"], "2026-01-01T00:00:00",
s.get("topic"), s.get("status", "active"),
1 if s.get("always_on", True) else 0,
s.get("confidence"), s.get("kind", "value"), s.get("priority")),
)
db.commit()
finally:
db.close()
return store_path, cache_path
# ── wrappers (the frozen fn names) ────────────────────────────────
def _pd_by_topic(rows, topics):
_prepare()
return PD._by_topic(copy.deepcopy(rows), set(topics))
def _pd_emit_constitution(by):
_prepare()
return PD.emit_constitution(copy.deepcopy(by))
def _pd_emit_identity(by):
_prepare()
return PD.emit_identity(copy.deepcopy(by))
def _pd_emit_persona(by, limit=20):
_prepare()
return PD.emit_persona(copy.deepcopy(by), limit)
def _pd_handle_hook(event, data=None):
_prepare()
return PD.handle_hook(event, copy.deepcopy(data))
def _pd_compile_full(seed, calls=2, preseed_cache=None):
"""compile_full under hermetic DBs with path-proving counters.
preseed_cache: optional bogus VALUE cached under persona:compile_full
before the first call (a real put_cache — get_cache does not filter
scope), freezing the corrupted-entry fallthrough branch.
"""
_prepare()
_seed_store(seed)
calls_state = {"store_connects": 0, "cache_gets": [], "cache_puts": 0}
orig_conn, orig_get, orig_put = _MS.connect, _CACHE.get_cache, _CACHE.put_cache
def c_conn(*a, **k):
calls_state["store_connects"] += 1
return orig_conn(*a, **k)
def c_get(key):
r = orig_get(key)
calls_state["cache_gets"].append(bool(r and r.get("hit")))
return r
def c_put(*a, **k):
calls_state["cache_puts"] += 1
return orig_put(*a, **k)
if preseed_cache is not None:
_CACHE.put_cache("persona:compile_full", preseed_cache)
_MS.connect, _CACHE.get_cache, _CACHE.put_cache = c_conn, c_get, c_put
try:
results = []
for _ in range(calls):
rows, by = PD.compile_full()
results.append({"rows": copy.deepcopy(rows), "by": copy.deepcopy(by)})
finally:
_MS.connect, _CACHE.get_cache, _CACHE.put_cache = orig_conn, orig_get, orig_put
out = {"r%d" % (i + 1): r for i, r in enumerate(results)}
out.update(calls_state)
return out
# ── case matrix ───────────────────────────────────────────────────
def _row(id, text, topic=None, importance=0.5, confidence=None, kind="value", **over):
r = {"id": id, "text": text, "topic": topic, "importance": importance,
"confidence": confidence, "kind": kind}
r.update(over)
return r
def _by(rows):
"""The `by` grouping compile_full produces (topic.lower() or 'general')."""
by = {}
for r in rows:
t = (r.get("topic") or "general").lower()
by.setdefault(t, []).append(r)
return by
CONSTITUTION_TEXT = "The persona must never fabricate evidence it has not verified"
IDENTITY_TEXT = "I am a careful engineer who verifies before attesting 中文 ünïcode"
PERSONA_A = "Prefer root-cause fixes over bypassing safety checks in every landing"
PERSONA_B = "Write compact diffs that preserve unrelated user changes at all times"
def cases():
C = []
# _by_topic — order-preserving filter, case-insensitive, None/missing topic
ROWS = [
_row(7, CONSTITUTION_TEXT, "Constitution", 0.9, 0.8),
_row(3, PERSONA_A, "Persona", 0.55, 0.7),
_row(11, IDENTITY_TEXT, "Identity", 0.75, 0.6),
_row(12, "orphan entry", None, 0.4, 0.5),
]
C.append(("_pd_by_topic", [ROWS, ["persona"]], {}))
C.append(("_pd_by_topic", [ROWS, ["constitution"]], {}))
C.append(("_pd_by_topic", [ROWS, ["constitution", "identity"]], {}))
C.append(("_pd_by_topic", [ROWS, []], {}))
C.append(("_pd_by_topic", [ROWS, [""]], {})) # None topic lowers to ""
C.append(("_pd_by_topic", [[_row(1, "MIXED Case", "PeRsOnA")], ["persona"]], {}))
C.append(("_pd_by_topic", [ROWS, ["nomatch"]], {}))
# emit_constitution / emit_identity — join, missing key, empty list
C.append(("_pd_emit_constitution", [_by(ROWS)], {}))
C.append(("_pd_emit_constitution", [{"identity": [ROWS[2]]}], {}))
C.append(("_pd_emit_constitution", [{"constitution": []}], {}))
C.append(("_pd_emit_identity", [_by(ROWS)], {}))
C.append(("_pd_emit_identity", [{}], {}))
C.append(("_pd_emit_identity",
[{"identity": [_row(2, "éè single"), _row(4, "second é")]}], {}))
# emit_persona — sort, filters, truncation, limit
P_ROWS = [
_row(21, PERSONA_A, "persona", 0.5, 0.6, priority=3),
_row(22, PERSONA_B, "persona", 0.5, 0.9, priority=1),
_row(23, "Low confidence entry that is long enough to survive", "persona", 0.5, 0.1, priority=1),
_row(24, "High confidence entry that is long enough to survive", "persona", 0.5, 0.95, priority=1),
]
C.append(("_pd_emit_persona", [{"persona": P_ROWS}], {}))
C.append(("_pd_emit_persona", [{"persona": P_ROWS}], {"limit": 2}))
C.append(("_pd_emit_persona", [{"persona": P_ROWS}], {"limit": 0}))
# priority absent -> 5; priority 0 -> 5 (Python `or` falsiness)
C.append(("_pd_emit_persona", [{"persona": [
_row(31, PERSONA_A, priority=None),
_row(32, PERSONA_B, priority=0),
_row(33, "Zero priority explicit entry long enough to survive ok", priority=2),
]}], {}))
# noise filters
C.append(("_pd_emit_persona", [{"persona": [
_row(41, "too short"), # len < 20
_row(42, "the leading article short line"), # 'the ' + < 40
_row(43, "the leading article line that is now long enough to keep"),
_row(44, "a short article line"), # 'a ' + < 40
_row(45, "an article line"), # 'an ' + < 40
_row(46, "i find this too short"), # 'i find' + < 40
_row(47, "i believe this line is long enough to keep ok"),
_row(48, PERSONA_A),
]}], {}))
# firecrawl+credit skip (both words required)
C.append(("_pd_emit_persona", [{"persona": [
_row(51, "Firecrawl ran out of credit again today"),
_row(52, "firecrawl lesson without the other word stays"),
_row(53, "credit note without the other word stays ok"),
_row(54, PERSONA_A),
]}], {}))
# >200-char truncation, by CHARACTERS (multi-byte mix proves rune slice)
long_ascii = "A" * 180 + "é中" + "B" * 40 # 222 chars, 224 bytes
C.append(("_pd_emit_persona", [{"persona": [_row(61, long_ascii)]}], {}))
# whitespace strip + stable ties
C.append(("_pd_emit_persona", [{"persona": [
_row(62, " " + PERSONA_A + " ", confidence=0.5),
_row(63, PERSONA_B, confidence=0.5),
]}], {}))
C.append(("_pd_emit_persona", [{"persona": []}], {}))
C.append(("_pd_emit_persona", [{}], {}))
# confidence absent -> 0
C.append(("_pd_emit_persona", [{"persona": [
_row(71, PERSONA_A, confidence=None),
_row(72, PERSONA_B, confidence=0.4),
]}], {}))
# handle_hook
C.append(("_pd_handle_hook", ["on_startup", {"k": 1}], {}))
C.append(("_pd_handle_hook", ["whatever"], {}))
C.append(("_pd_handle_hook", ["ü event", None], {}))
# compile_full — the store+cache IO matrix
SEED_SMALL = [
{"id": 7, "text": CONSTITUTION_TEXT, "topic": "Constitution",
"importance": 0.9, "confidence": 0.8, "kind": "value"},
{"id": 3, "text": PERSONA_A, "topic": "Persona",
"importance": 0.55, "confidence": 0.7, "kind": "value"},
]
# (a) small persona: miss -> store -> put; call2 cache HIT (no re-query)
C.append(("_pd_compile_full", [SEED_SMALL], {}))
# (b) empty store: rows [], by {} still cached and round-trips
C.append(("_pd_compile_full", [[]], {}))
# (c) WHERE filters: inactive + always_on=0 excluded, importance DESC,
# NULL topic -> 'general' bucket, unicode topics lowered
SEED_FILTER = [
{"id": 1, "text": "inactive entry must never appear", "topic": "persona",
"importance": 0.99, "status": "inactive"},
{"id": 2, "text": "not always on must never appear either", "topic": "persona",
"importance": 0.98, "always_on": False},
{"id": 3, "text": PERSONA_B, "topic": "PeRsOnA", "importance": 0.71,
"confidence": 0.5, "kind": "value"},
{"id": 4, "text": PERSONA_A, "topic": None, "importance": 0.66,
"confidence": 0.65, "kind": "rule"},
{"id": 5, "text": IDENTITY_TEXT, "topic": "IDENTITY", "importance": 0.44,
"confidence": 0.3, "kind": "value"},
]
C.append(("_pd_compile_full", [SEED_FILTER], {}))
# (d) large persona (>800-char cache JSON): call2 compressed truncated ->
# parse fail -> store re-query -> re-put
big_text = "Long persona value number %03d that keeps the cache entry over the eight hundred character boundary" + "x" * 40
SEED_BIG = [
{"id": 9 + i, "text": big_text % i, "topic": "persona",
"importance": 0.5 + i * 0.01, "confidence": 0.9, "kind": "value"}
for i in range(8)
]
C.append(("_pd_compile_full", [SEED_BIG], {}))
# (e) corrupted pre-seeded cache entry: call1 hit-but-unparseable ->
# store query -> put (overwrite); call2 hits the good entry
C.append(("_pd_compile_full", [SEED_SMALL], {"preseed_cache": "not json {"}))
# (f) one call only (the pure store path without any cache readback)
C.append(("_pd_compile_full", [SEED_SMALL], {"calls": 1}))
return C
SHIMS = "gen_persona_definition_gap_truth_shims"
def main():
sys.modules[SHIMS] = sys.modules["__main__"]
truth = freeze(SHIMS, cases(), oracle_module=SHIMS)
target = REPO / "src" / "go" / "persona-definition" / "truth_gap.json"
write_envelope(FROZEN_NOW, truth, target)
print(f"wrote {target} ({len(truth)} cases)")
def verify(path):
raw = json.loads(path.read_text())
frozen = raw["cases"] if isinstance(raw, dict) else raw
if isinstance(raw, dict) and raw.get("frozen_now"):
os.environ["RUNE_FROZEN_NOW"] = raw["frozen_now"]
sys.modules[SHIMS] = sys.modules["__main__"]
recased = [{"fn": c["fn"], "args": c.get("args", []), "kwargs": c.get("kwargs", {})}
for c in frozen]
fresh = freeze(SHIMS, recased, oracle_module=SHIMS)
if json.dumps(fresh, sort_keys=True) != json.dumps(frozen, sort_keys=True):
print(f"MISMATCH: regenerated truth differs from {path}")
for f, g in zip(frozen, fresh):
if json.dumps(f, sort_keys=True) != json.dumps(g, sort_keys=True):
print(f" first divergence at fn={f['fn']}")
print(f" frozen: {json.dumps(f['expected'])[:300]}")
print(f" oracle: {json.dumps(g['expected'])[:300]}")
break
return 1
print(f"OK: {len(frozen)} cases byte-identical against live oracle")
return 0
if __name__ == "__main__":
if len(sys.argv) > 1 and sys.argv[1] == "--verify":
sys.exit(verify(REPO / "src" / "go" / "persona-definition" / "truth_gap.json"))
main()
+552
View File
@@ -0,0 +1,552 @@
package main
// gaps.go — persona_definition gap closure (#2479 dichotomy wave, GO
// verdict): the five audit-named missing fns of src/rune/persona_definition.py
// ported name-preserving 1:1 — byTopic (_by_topic), compileFull
// (compile_full), emitConstitution (emit_constitution), emitIdentity
// (emit_identity), emitPersona (emit_persona) — plus handleHook
// (handle_hook), coreRows (the _core loader-probe machinery) and the
// module-global ORDER map. The store (memory_store sqlite) and cache
// (cache.py sqlite) sit behind the Env seams (StoreDB / MemoryDir /
// clockFn / pythonBin) so the frozen-truth battery replays truth_gap.json
// VERBATIM against throwaway DBs. Cache hit/miss stat accumulation lives
// in the cache module's own port — invisible to this module's surface.
import (
"crypto/sha256"
"database/sql"
"encoding/hex"
"encoding/json"
"log"
"os"
"os/exec"
"path/filepath"
"sort"
"strings"
"time"
_ "modernc.org/sqlite"
)
// ORDER — Python module-global ORDER: definition order of the topic
// buckets (constitution first — immutable — then identity, persona...).
var orderMap = map[string]int{
"constitution": 0, "identity": 1, "persona": 2, "operating": 3,
"project": 4, "delivery": 5, "general": 6,
}
const cacheKey = "persona:compile_full"
// Env — the module-global IO surface of persona_definition.py (the store
// and cache DB paths, the clock the cache stamps, and the interpreter the
// _core subprocess fallback shells out to) behind injectable seams.
type Env struct {
StoreDB string
MemoryDir string
clockFn func() time.Time
pythonBin string
storePy string
// battery counters (mirror the generator's ms.connect / cache.get /
// cache.put wrappers — they PROVE which path compileFull took)
StoreConnects int
CacheGets []bool
CachePuts int
}
// NewEnv — IO seams over explicit paths (hermetic replay).
func NewEnv(storeDB, memoryDir string) *Env {
return &Env{
StoreDB: storeDB,
MemoryDir: memoryDir,
clockFn: func() time.Time { return time.Now().UTC() },
pythonBin: "python3",
storePy: "src/rune/memory_store.py",
}
}
// NewEnvFromOS — the memory_env path law: RUNE_STORE_DB overrides,
// else <memory_dir>/store/memory.db; memory dir = RUNE_MEMORY_DIR,
// else <rune_home>/memory; rune home = RUNE_HOME, else /mnt/workdrive/rune.
func NewEnvFromOS() *Env {
runeHome := os.Getenv("RUNE_HOME")
if runeHome == "" {
runeHome = "/mnt/workdrive/rune"
}
memoryDir := os.Getenv("RUNE_MEMORY_DIR")
if memoryDir == "" {
memoryDir = filepath.Join(runeHome, "memory")
}
storeDB := os.Getenv("RUNE_STORE_DB")
if storeDB == "" {
storeDB = filepath.Join(memoryDir, "store", "memory.db")
}
py := os.Getenv("RUNE_PYTHON_BIN")
if py == "" {
py = "python3"
}
storePy := os.Getenv("RUNE_SCRIPTS_DIR")
if storePy == "" {
storePy = filepath.Join(runeHome, "src", "rune")
}
e := NewEnv(storeDB, memoryDir)
e.pythonBin = py
e.storePy = filepath.Join(storePy, "memory_store.py")
return e
}
// now — Python cache._now: datetime.now(timezone.utc).isoformat().
func (e *Env) now() string {
t := e.clockFn().UTC()
if t.Nanosecond()/1000 == 0 {
return t.Format("2006-01-02T15:04:05-07:00")
}
return t.Format("2006-01-02T15:04:05.000000-07:00")
}
// coreRows — Python _core (loader-probe machinery, audit-excluded but the
// load-bearing query behind compile_full): SELECT the always-on active
// core entries ordered by importance DESC, as ordered dicts in the SELECT
// literal's key order; on failure degrade through the subprocess fallback
// (query-core --json) exactly like the oracle, ending at [].
func (e *Env) coreRows() []any {
e.StoreConnects++
db, err := sql.Open("sqlite", e.StoreDB)
if err == nil {
defer db.Close()
rows, qerr := db.Query(
"SELECT id, text, topic, importance, confidence, kind " +
"FROM entries WHERE status='active' AND always_on=1 " +
"ORDER BY importance DESC")
if qerr == nil {
defer rows.Close()
out := []any{}
for rows.Next() {
var id int64
var text sql.NullString
var topic sql.NullString
var importance sql.NullFloat64
var confidence sql.NullFloat64
var kind sql.NullString
if err := rows.Scan(&id, &text, &topic, &importance, &confidence, &kind); err != nil {
break
}
r := NewOrderedMap()
r.Set("id", id)
r.Set("text", nullStr(text))
r.Set("topic", nullStr(topic))
r.Set("importance", nullFloat(importance))
r.Set("confidence", nullFloat(confidence))
r.Set("kind", nullStr(kind))
out = append(out, any(r))
}
return out
}
}
// Fallback: subprocess (import/connect failure class in the oracle).
log.Printf("Direct store query failed (%v), falling back to subprocess", err)
cmd := exec.Command(e.pythonBin, e.storePy, "query-core", "--json")
cmd.Env = os.Environ()
out, perr := cmd.Output()
if perr != nil {
return []any{}
}
if len(out) == 0 {
out = []byte("[]")
}
v, derr := decodeJSON(out)
if derr != nil {
log.Printf("Subprocess fallback also failed to parse output: %s", derr)
return []any{}
}
if arr, ok := v.([]any); ok {
return arr
}
return []any{}
}
func nullStr(v sql.NullString) any {
if v.Valid {
return v.String
}
return nil
}
func nullFloat(v sql.NullFloat64) any {
if v.Valid {
return v.Float64
}
return nil
}
// byTopic — Python _by_topic: rows whose (topic or "") lowercased form is
// in the topics set, order preserved.
func byTopic(rows []any, topics []string) []any {
set := map[string]bool{}
for _, t := range topics {
set[t] = true
}
out := []any{}
for _, r := range rows {
if set[pyLower(topicOf(r))] {
out = append(out, r)
}
}
return out
}
// topicOf — r.get("topic") or "".
func topicOf(r any) string {
om, ok := r.(*OrderedMap)
if !ok {
return ""
}
v, _ := om.Get("topic")
s, _ := v.(string)
return s
}
// compileFull — Python compile_full: cache-backed persona compile. Cache
// read first (hit + parseable compressed value returns early); on miss or
// unparseable value query the store, group by lowered topic (orphan rows
// land in "general"), cache the rendered value under scope "persona", and
// return rows + by.
func (e *Env) compileFull() ([]any, *OrderedMap) {
if compressed, hit := e.getCache(cacheKey); hit {
if v, derr := decodeJSON([]byte(compressed)); derr == nil {
if om, ok := v.(*OrderedMap); ok {
if rowsV, present := om.Get("rows"); present {
if byV, byPresent := om.Get("by"); byPresent {
if rows, rok := rowsV.([]any); rok {
if by, bok := byV.(*OrderedMap); bok {
return append([]any{}, rows...), by
}
}
}
}
}
}
}
rows := e.coreRows()
by := NewOrderedMap()
for _, r := range rows {
t := topicOf(r)
if t == "" {
t = "general"
}
t = pyLower(t)
var bucket []any
if existing, ok := by.Get(t); ok {
bucket, _ = existing.([]any)
}
bucket = append(bucket, r)
by.Set(t, bucket)
}
e.putCache(cacheKey, pyDumpsNoASCII(orderedRowsBy(rows, by)), "persona")
return rows, by
}
// orderedRowsBy — the cached dict {"rows": rows, "by": by} in Python's
// literal insertion order.
func orderedRowsBy(rows []any, by *OrderedMap) *OrderedMap {
doc := NewOrderedMap()
doc.Set("rows", rows)
doc.Set("by", by)
return doc
}
// emitConstitution — Python emit_constitution: one "- {text}" line per
// constitution row, or "(none stored)".
func emitConstitution(by *OrderedMap) string {
return emitBucket(by, "constitution")
}
// emitIdentity — Python emit_identity: same shape over the identity rows.
func emitIdentity(by *OrderedMap) string {
return emitBucket(by, "identity")
}
func emitBucket(by *OrderedMap, key string) string {
lines := []string{}
for _, r := range bucketRows(by, key) {
lines = append(lines, "- "+textOf(r))
}
if len(lines) == 0 {
return "(none stored)"
}
return strings.Join(lines, "\n")
}
// emitPersona — Python emit_persona: stable sort by (priority or 5,
// -confidence or 0), noise filter (fragments <20 chars, short
// leading-article lines <40 chars, firecrawl+credit lessons), "- {text}"
// lines truncated to 200 CHARACTERS, capped at limit, or "(none stored)".
func emitPersona(by *OrderedMap, limit int) string {
rows := append([]any{}, bucketRows(by, "persona")...)
sort.SliceStable(rows, func(i, j int) bool {
pi, ci := sortKeys(rows[i])
pj, cj := sortKeys(rows[j])
if pi != pj {
return pi < pj
}
return ci < cj // -confidence ascending = confidence descending
})
filtered := []string{}
for _, r := range rows {
text := strings.TrimSpace(textOf(r))
if utf8Len(text) < 20 {
continue
}
lower := pyLower(text)
if strings.HasPrefix(lower, "the ") || strings.HasPrefix(lower, "a ") ||
strings.HasPrefix(lower, "an ") || strings.HasPrefix(lower, "i find") ||
strings.HasPrefix(lower, "i believe") {
if utf8Len(text) < 40 {
continue
}
}
if strings.Contains(lower, "firecrawl") && strings.Contains(lower, "credit") {
continue
}
filtered = append(filtered, text)
}
if limit >= 0 && len(filtered) > limit {
filtered = filtered[:limit]
}
lines := []string{}
for _, t := range filtered {
lines = append(lines, "- "+truncateRunes(t, 200))
}
if len(lines) == 0 {
return "(none stored)"
}
return strings.Join(lines, "\n")
}
// sortKeys — the Python tuple key (r.get("priority") or 5,
// -(r.get("confidence") or 0)); `or` maps nil AND 0 to the default.
func sortKeys(r any) (float64, float64) {
om, _ := r.(*OrderedMap)
priority := 5.0
if om != nil {
if v, ok := om.Get("priority"); ok {
if f, ok := numFloat(v); ok && f != 0 {
priority = f
}
}
}
confidence := 0.0
if om != nil {
if v, ok := om.Get("confidence"); ok {
if f, ok := numFloat(v); ok {
confidence = f
}
}
}
return priority, -confidence
}
func numFloat(v any) (float64, bool) {
switch t := v.(type) {
case float64:
return t, true
case int64:
return float64(t), true
case int:
return float64(t), true
case json.Number:
if f, err := t.Float64(); err == nil {
return f, true
}
}
return 0, false
}
func bucketRows(by *OrderedMap, key string) []any {
if by == nil {
return nil
}
if v, ok := by.Get(key); ok {
rows, _ := v.([]any)
return rows
}
return nil
}
func textOf(r any) string {
om, _ := r.(*OrderedMap)
if om == nil {
return ""
}
v, _ := om.Get("text")
s, _ := v.(string)
return s
}
func utf8Len(s string) int {
n := 0
for range s {
n++
}
return n
}
// truncateRunes — Python s[:n]: slice by CHARACTERS, never bytes.
func truncateRunes(s string, n int) string {
if utf8Len(s) <= n {
return s
}
out := []rune(s)
return string(out[:n])
}
func pyLower(s string) string {
return strings.ToLower(s)
}
// handleHook — Python handle_hook: the generic default handler; data is
// accepted and ignored.
func handleHook(event string, data any) *OrderedMap {
r := NewOrderedMap()
r.Set("status", "ok")
r.Set("event", event)
r.Set("handler", "default")
return r
}
// ── cache layer (cache.py get_cache/put_cache, the exact surface
// compile_full touches) ────────────────────────────────────────────
const cacheSchema = `
CREATE TABLE IF NOT EXISTS cache (
key TEXT PRIMARY KEY,
scope TEXT DEFAULT 'general',
content_hash TEXT NOT NULL,
full_value TEXT,
compressed_value TEXT,
source_hash TEXT DEFAULT '',
created_at TEXT NOT NULL,
invalidated_at TEXT,
hit_count INTEGER DEFAULT 0,
last_hit TEXT,
last_miss TEXT,
miss_count INTEGER DEFAULT 0
);
CREATE INDEX IF NOT EXISTS idx_cache_scope ON cache(scope);
CREATE INDEX IF NOT EXISTS idx_cache_valid ON cache(invalidated_at);
`
func (e *Env) cacheDBPath() string {
return filepath.Join(e.MemoryDir, "cache.db")
}
func (e *Env) openCache() (*sql.DB, error) {
p := e.cacheDBPath()
if err := os.MkdirAll(filepath.Dir(p), 0o755); err != nil {
return nil, err
}
db, err := sql.Open("sqlite", p)
if err != nil {
return nil, err
}
if _, err := db.Exec(cacheSchema); err != nil {
db.Close()
return nil, err
}
return db, nil
}
// getCache — Python cache.get_cache: the compressed value of a live
// (non-invalidated) entry plus its hit flag. Hit/miss accounting is the
// cache module's own surface.
func (e *Env) getCache(key string) (string, bool) {
e.CacheGets = append(e.CacheGets, false)
db, err := e.openCache()
if err != nil {
return "", false
}
defer db.Close()
var compressed sql.NullString
err = db.QueryRow(
"SELECT compressed_value FROM cache WHERE key=? AND invalidated_at IS NULL",
key).Scan(&compressed)
if err != nil || !compressed.Valid {
return "", false
}
e.CacheGets[len(e.CacheGets)-1] = true
return compressed.String, true
}
// putCache — Python cache.put_cache: INSERT .. ON CONFLICT(key) DO UPDATE
// with the mechanical compression derived from the value.
func (e *Env) putCache(key, value, scope string) {
e.CachePuts++
db, err := e.openCache()
if err != nil {
return
}
defer db.Close()
sum := sha256.Sum256([]byte(value))
contentHash := hex.EncodeToString(sum[:])[:16]
_, err = db.Exec(`
INSERT INTO cache
(key, scope, content_hash, full_value, compressed_value, source_hash,
created_at, invalidated_at, hit_count, miss_count)
VALUES (?, ?, ?, ?, ?, ?, ?, NULL, 0, 0)
ON CONFLICT(key) DO UPDATE SET
scope = excluded.scope,
content_hash = excluded.content_hash,
full_value = excluded.full_value,
compressed_value = excluded.compressed_value,
source_hash = excluded.source_hash,
created_at = excluded.created_at,
invalidated_at = NULL`,
key, scope, contentHash, value, pyCompress(value), "", e.now())
if err != nil {
log.Printf("cache put failed: %v", err)
}
}
// pyCompress — Python cache._compress: deterministic mechanical
// compression (drop empty/comment/decorative/citation/short lines, join,
// truncate to 797 CHARS + "..." past 800 — char-based, never bytes).
func pyCompress(text string) string {
if text == "" {
return ""
}
lines := strings.Split(strings.TrimSpace(text), "\n")
kept := []string{}
for _, line := range lines {
stripped := strings.TrimSpace(line)
if stripped == "" {
continue
}
if strings.HasPrefix(stripped, "#") || strings.HasPrefix(stripped, "//") {
continue
}
deco := true
for _, c := range stripped {
if !strings.ContainsRune("-=*#", c) {
deco = false
break
}
}
if deco {
continue
}
if strings.HasPrefix(stripped, "(_source_:") || strings.HasPrefix(stripped, "[source:") {
continue
}
if utf8Len(stripped) < 5 {
continue
}
kept = append(kept, stripped)
}
result := strings.Join(kept, "\n")
if utf8Len(result) > 800 {
result = truncateRunes(result, 797) + "..."
}
return result
}
+26
View File
@@ -0,0 +1,26 @@
module rune/persona-definition
go 1.21
require modernc.org/sqlite v1.28.0
require (
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/google/uuid v1.3.0 // indirect
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 // indirect
github.com/mattn/go-isatty v0.0.16 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
golang.org/x/mod v0.3.0 // indirect
golang.org/x/sys v0.9.0 // indirect
golang.org/x/tools v0.0.0-20201124115921-2c860bdd6e78 // indirect
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 // indirect
lukechampine.com/uint128 v1.2.0 // indirect
modernc.org/cc/v3 v3.40.0 // indirect
modernc.org/ccgo/v3 v3.16.13 // indirect
modernc.org/libc v1.29.0 // indirect
modernc.org/mathutil v1.6.0 // indirect
modernc.org/memory v1.7.2 // indirect
modernc.org/opt v0.1.3 // indirect
modernc.org/strutil v1.1.3 // indirect
modernc.org/token v1.0.1 // indirect
)
+73
View File
@@ -0,0 +1,73 @@
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
github.com/google/go-cmp v0.5.9 h1:O2Tfq5qg4qc4AmwVlvv0oLiVAGB7enBSJ2x2DqQFi38=
github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/pprof v0.0.0-20221118152302-e6195bd50e26 h1:Xim43kblpZXfIBQsbuBVKCudVG457BR2GZFIz3uw3hQ=
github.com/google/pprof v0.0.0-20221118152302-e6195bd50e26/go.mod h1:dDKJzRmX4S37WGHujM7tX//fmj1uioxKzKxz3lo4HJo=
github.com/google/uuid v1.3.0 h1:t6JiXgmwXMjEs8VusXIJk2BXHsn+wx8BZdTaoZ5fu7I=
github.com/google/uuid v1.3.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 h1:Z9n2FFNUXsshfwJMBgNA0RU6/i7WVaAegv3PtuIHPMs=
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51/go.mod h1:CzGEWj7cYgsdH8dAjBGEr58BoE7ScuLd+fwFZ44+/x8=
github.com/mattn/go-isatty v0.0.16 h1:bq3VjFmv/sOjHtdEhmkEV4x1AJtvUvOJ2PFAZ5+peKQ=
github.com/mattn/go-isatty v0.0.16/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM=
github.com/mattn/go-sqlite3 v1.14.16 h1:yOQRA0RpS5PFz/oikGwBEqvAWhWg5ufRz4ETLjwpU1Y=
github.com/mattn/go-sqlite3 v1.14.16/go.mod h1:2eHXhiwb8IkHr+BDWZGa96P6+rkvnG63S2DGjv9HUNg=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/mod v0.3.0 h1:RM4zey1++hCTbCVQfnWeKs9/IEsaBLA8vTkd0WVtmH4=
golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.9.0 h1:KS/R3tvhPqvJvwcKfnBHJwwthS11LRhmM5D59eEXa0s=
golang.org/x/sys v0.9.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20201124115921-2c860bdd6e78 h1:M8tBwCtWD/cZV9DZpFYRUgaymAYAr+aIUTWzDaM3uPs=
golang.org/x/tools v0.0.0-20201124115921-2c860bdd6e78/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 h1:go1bK/D/BFZV2I8cIQd1NKEZ+0owSTG1fDTci4IqFcE=
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
lukechampine.com/uint128 v1.2.0 h1:mBi/5l91vocEN8otkC5bDLhi2KdCticRiwbdB0O+rjI=
lukechampine.com/uint128 v1.2.0/go.mod h1:c4eWIwlEGaxC/+H1VguhU4PHXNWDCDMUlWdIWl2j1gk=
modernc.org/cc/v3 v3.40.0 h1:P3g79IUS/93SYhtoeaHW+kRCIrYaxJ27MFPv+7kaTOw=
modernc.org/cc/v3 v3.40.0/go.mod h1:/bTg4dnWkSXowUO6ssQKnOV0yMVxDYNIsIrzqTFDGH0=
modernc.org/ccgo/v3 v3.16.13 h1:Mkgdzl46i5F/CNR/Kj80Ri59hC8TKAhZrYSaqvkwzUw=
modernc.org/ccgo/v3 v3.16.13/go.mod h1:2Quk+5YgpImhPjv2Qsob1DnZ/4som1lJTodubIcoUkY=
modernc.org/ccorpus v1.11.6 h1:J16RXiiqiCgua6+ZvQot4yUuUy8zxgqbqEEUuGPlISk=
modernc.org/ccorpus v1.11.6/go.mod h1:2gEUTrWqdpH2pXsmTM1ZkjeSrUWDpjMu2T6m29L/ErQ=
modernc.org/httpfs v1.0.6 h1:AAgIpFZRXuYnkjftxTAZwMIiwEqAfk8aVB2/oA6nAeM=
modernc.org/httpfs v1.0.6/go.mod h1:7dosgurJGp0sPaRanU53W4xZYKh14wfzX420oZADeHM=
modernc.org/libc v1.29.0 h1:tTFRFq69YKCF2QyGNuRUQxKBm1uZZLubf6Cjh/pVHXs=
modernc.org/libc v1.29.0/go.mod h1:DaG/4Q3LRRdqpiLyP0C2m1B8ZMGkQ+cCgOIjEtQlYhQ=
modernc.org/mathutil v1.6.0 h1:fRe9+AmYlaej+64JsEEhoWuAYBkOtQiMEU7n/XgfYi4=
modernc.org/mathutil v1.6.0/go.mod h1:Ui5Q9q1TR2gFm0AQRqQUaBWFLAhQpCwNcuhBOSedWPo=
modernc.org/memory v1.7.2 h1:Klh90S215mmH8c9gO98QxQFsY+W451E8AnzjoE2ee1E=
modernc.org/memory v1.7.2/go.mod h1:NO4NVCQy0N7ln+T9ngWqOQfi7ley4vpwvARR+Hjw95E=
modernc.org/opt v0.1.3 h1:3XOZf2yznlhC+ibLltsDGzABUGVx8J6pnFMS3E4dcq4=
modernc.org/opt v0.1.3/go.mod h1:WdSiB5evDcignE70guQKxYUl14mgWtbClRi5wmkkTX0=
modernc.org/sqlite v1.28.0 h1:Zx+LyDDmXczNnEQdvPuEfcFVA2ZPyaD7UCZDjef3BHQ=
modernc.org/sqlite v1.28.0/go.mod h1:Qxpazz0zH8Z1xCFyi5GSL3FzbtZ3fvbjmywNogldEW0=
modernc.org/strutil v1.1.3 h1:fNMm+oJklMGYfU9Ylcywl0CO5O6nTfaowNsh2wpPjzY=
modernc.org/strutil v1.1.3/go.mod h1:MEHNA7PdEnEwLvspRMtWTNnp2nnyvMfkimT1NKNAGbw=
modernc.org/tcl v1.15.2 h1:C4ybAYCGJw968e+Me18oW55kD/FexcHbqH2xak1ROSY=
modernc.org/tcl v1.15.2/go.mod h1:3+k/ZaEbKrC8ePv8zJWPtBSW0V7Gg9g8rkmhI1Kfs3c=
modernc.org/token v1.0.1 h1:A3qvTqOwexpfZZeyI0FeGPDlSWX5pjZu9hF4lU+EKWg=
modernc.org/token v1.0.1/go.mod h1:UGzOrNV1mAFSEB63lOFHIpNRUVMvYTc6yu1SMY/XTDM=
modernc.org/z v1.7.3 h1:zDJf6iHjrnB+WRD88stbXokugjyc0/pB91ri1gO6LZY=
modernc.org/z v1.7.3/go.mod h1:Ipv4tsdxZRbQyLq9Q1M6gdbkxYzdlrciF2Hi/lS7nWE=
+131
View File
@@ -0,0 +1,131 @@
package main
// main.go — persona-definition CLI: byte-parity mirror of src/rune/
// persona_definition.py main() (argparse choices full/constitution/
// identity/compact/hook, --event/--data), plus the replay-gap and
// scenario subcommands the battery and live parity drive.
//
// Usage:
//
// persona-definition {full,constitution,identity,compact,hook} [--event E] [--data JSON]
// persona-definition replay-gap <truth_gap.json>
// persona-definition scenario <scenario.json>
import (
"fmt"
"os"
)
func main() {
if len(os.Args) > 2 && os.Args[1] == "replay-gap" {
if err := replayGap(os.Args[2]); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
return
}
if len(os.Args) > 2 && os.Args[1] == "scenario" {
if err := runScenario(os.Args[2]); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
return
}
cliMain(os.Args[1:])
}
var usage = "usage: persona_definition.py [-h] [--event EVENT] [--data DATA] " +
"{full,constitution,identity,compact,hook}"
// cliMain — argparse semantics: one positional cmd from the choices set,
// optional --event/--data flags; bad cmd/flag exits 2 like argparse.
func cliMain(argv []string) {
cmd := ""
event := ""
data := ""
positionalDone := false
for i := 0; i < len(argv); i++ {
a := argv[i]
switch {
case a == "--event" && i+1 < len(argv):
i++
event = argv[i]
case a == "--data" && i+1 < len(argv):
i++
data = argv[i]
case a == "-h" || a == "--help":
fmt.Println(usage)
return
case len(a) > 2 && a[:2] == "--":
fmt.Fprintf(os.Stderr, "unrecognized arguments: %s\n", a)
fmt.Fprintln(os.Stderr, usage)
os.Exit(2)
case !positionalDone:
cmd = a
positionalDone = true
default:
fmt.Fprintf(os.Stderr, "unrecognized arguments: %s\n", a)
fmt.Fprintln(os.Stderr, usage)
os.Exit(2)
}
}
switch cmd {
case "full", "constitution", "identity", "compact", "hook":
default:
fmt.Fprintf(os.Stderr,
"argument cmd: invalid choice: '%s' (choose from 'full', 'constitution', 'identity', 'compact', 'hook')\n", cmd)
fmt.Fprintln(os.Stderr, usage)
os.Exit(2)
}
if cmd == "hook" {
ev := event
if ev == "" {
ev = "unknown"
}
var parsed any = map[string]any{}
if data != "" {
v, err := decodeJSON([]byte(data))
if err != nil {
// crash parity: the oracle's json.loads raises
fmt.Fprintln(os.Stderr, "json.JSONDecodeError:", err)
os.Exit(1)
}
parsed = v
}
fmt.Println(encodePythonJSON(handleHook(ev, parsed), -1))
return
}
e := NewEnvFromOS()
out, err := compileCmdOutput(e, cmd)
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(2)
}
fmt.Print(out)
}
// compileCmdOutput — the exact stdout of the oracle's main() print paths
// (full/constitution/identity/compact) over one compile_full pass.
func compileCmdOutput(e *Env, cmd string) (string, error) {
rows, by := e.compileFull()
switch cmd {
case "constitution":
return emitConstitution(by) + "\n", nil
case "identity":
return emitIdentity(by) + "\n", nil
case "compact":
return "CONSTITUTION (immutable):\n" + emitConstitution(by) +
"\n\nIDENTITY (who I am):\n" + emitIdentity(by) +
"\n\nPERSONA (how I behave):\n" + emitPersona(by, 20) + "\n", nil
case "full":
return "CONSTITUTION (immutable):\n" + emitConstitution(by) + "\n" +
"\nIDENTITY (who I am):\n" + emitIdentity(by) + "\n" +
"\nPERSONA (how I behave):\n" + emitPersona(by, 20) + "\n" +
fmt.Sprintf("\n(total %d core entries; constitution %d, identity %d, persona %d)\n",
len(rows), bucketLen(by, "constitution"), bucketLen(by, "identity"),
bucketLen(by, "persona")), nil
}
return "", fmt.Errorf("unknown cli cmd %q", cmd)
}
+378
View File
@@ -0,0 +1,378 @@
// Python json interop: insertion-ordered maps and json.dumps-compatible
// encoding (indent=2, ensure_ascii=True; compact with Python's default
// (', ', ': ') separators) so CLI output is byte-parity with the oracle.
// Vendored from src/go/cache (GO sibling-isolation law) with a Python-repr
// float renderer (integral floats keep ".0") plus the ensure_ascii=False
// compact variant compile_full uses for its cache value.
package main
import (
"encoding/json"
"fmt"
"math"
"strconv"
"strings"
"unicode/utf16"
)
// KV — one ordered key/value pair.
type KV struct {
Key string
Value any
}
// OrderedMap — a dict with preserved insertion order.
type OrderedMap struct {
Keys []string
Vals map[string]any
}
// NewOrderedMap builds an ordered map from pairs.
func NewOrderedMap(pairs ...KV) *OrderedMap {
om := &OrderedMap{Vals: map[string]any{}}
for _, kv := range pairs {
om.Set(kv.Key, kv.Value)
}
return om
}
// Set inserts or replaces, preserving first-insertion order.
func (o *OrderedMap) Set(key string, value any) {
if _, ok := o.Vals[key]; !ok {
o.Keys = append(o.Keys, key)
}
o.Vals[key] = value
}
// Get retrieves a value.
func (o *OrderedMap) Get(key string) (any, bool) {
v, ok := o.Vals[key]
return v, ok
}
// Entries returns key/value pairs in insertion order.
func (o *OrderedMap) Entries() []KV {
out := make([]KV, 0, len(o.Keys))
for _, k := range o.Keys {
out = append(out, KV{k, o.Vals[k]})
}
return out
}
// Len returns the number of entries.
func (o *OrderedMap) Len() int { return len(o.Keys) }
// decodeJSON decodes one JSON value, preserving object order as
// *OrderedMap and numbers as json.Number (Python int/float distinction).
func decodeJSON(b []byte) (any, error) {
dec := json.NewDecoder(strings.NewReader(string(b)))
dec.UseNumber()
return decodeJSONValue(dec)
}
func decodeJSONValue(dec *json.Decoder) (any, error) {
tok, err := dec.Token()
if err != nil {
return nil, err
}
switch t := tok.(type) {
case json.Delim:
switch t {
case '{':
om := &OrderedMap{Vals: map[string]any{}}
for dec.More() {
keyTok, err := dec.Token()
if err != nil {
return nil, err
}
key, _ := keyTok.(string)
val, err := decodeJSONValue(dec)
if err != nil {
return nil, err
}
om.Set(key, val)
}
if _, err := dec.Token(); err != nil { // consume '}'
return nil, err
}
return om, nil
case '[':
arr := []any{}
for dec.More() {
val, err := decodeJSONValue(dec)
if err != nil {
return nil, err
}
arr = append(arr, val)
}
if _, err := dec.Token(); err != nil { // consume ']'
return nil, err
}
return arr, nil
}
return nil, fmt.Errorf("unexpected delim %v", t)
default:
return tok, nil
}
}
// encodePythonJSON renders v exactly like Python json.dumps(v, indent=N)
// with ensure_ascii=True and (',', ': ') separators. indent < 0 renders
// compact with Python's no-indent default (', ', ': ').
func encodePythonJSON(v any, indent int) string {
var sb strings.Builder
encodeValue(&sb, v, indent)
return sb.String()
}
func encodeValue(sb *strings.Builder, v any, indent int) {
switch t := v.(type) {
case nil:
sb.WriteString("null")
case bool:
if t {
sb.WriteString("true")
} else {
sb.WriteString("false")
}
case string:
sb.WriteString(pythonEscapeString(t))
case int:
sb.WriteString(strconv.Itoa(t))
case int64:
sb.WriteString(strconv.FormatInt(t, 10))
case float64:
sb.WriteString(pythonFloat(t))
case json.Number:
sb.WriteString(t.String())
case []any:
if len(t) == 0 {
sb.WriteString("[]")
return
}
if indent < 0 {
sb.WriteString("[")
for i, item := range t {
if i > 0 {
sb.WriteString(", ")
}
encodeValue(sb, item, indent)
}
sb.WriteString("]")
return
}
sb.WriteString("[\n")
for i, item := range t {
writeIndent(sb, indent+1)
encodeValue(sb, item, indent+1)
if i < len(t)-1 {
sb.WriteString(",")
}
sb.WriteString("\n")
}
writeIndent(sb, indent)
sb.WriteString("]")
case *OrderedMap:
if t.Len() == 0 {
sb.WriteString("{}")
return
}
if indent < 0 {
sb.WriteString("{")
for i, kv := range t.Entries() {
if i > 0 {
sb.WriteString(", ")
}
sb.WriteString(pythonEscapeString(kv.Key))
sb.WriteString(": ")
encodeValue(sb, kv.Value, indent)
}
sb.WriteString("}")
return
}
sb.WriteString("{\n")
entries := t.Entries()
for i, kv := range entries {
writeIndent(sb, indent+1)
sb.WriteString(pythonEscapeString(kv.Key))
sb.WriteString(": ")
encodeValue(sb, kv.Value, indent+1)
if i < len(entries)-1 {
sb.WriteString(",")
}
sb.WriteString("\n")
}
writeIndent(sb, indent)
sb.WriteString("}")
default:
sb.WriteString(pythonEscapeString(fmt.Sprintf("%v", t)))
}
}
func writeIndent(sb *strings.Builder, level int) {
for i := 0; i < level; i++ {
sb.WriteString(" ")
}
}
// pythonFloat renders a float like Python repr (json.dumps float
// rendering): shortest digits, fixed notation for exponents in
// [-4, 16), exponent notation outside, integral values keep ".0".
func pythonFloat(f float64) string {
if math.IsInf(f, 1) {
return "Infinity"
}
if math.IsInf(f, -1) {
return "-Infinity"
}
if math.IsNaN(f) {
return "NaN"
}
es := strconv.FormatFloat(f, 'e', -1, 64)
i := strings.IndexByte(es, 'e')
exp, _ := strconv.Atoi(es[i+1:])
if exp >= 16 || exp < -4 {
mant := es[:i]
sign := "+"
if exp < 0 {
sign = "-"
exp = -exp
}
if exp < 10 {
return mant + "e" + sign + "0" + strconv.Itoa(exp)
}
return mant + "e" + sign + strconv.Itoa(exp)
}
fs := strconv.FormatFloat(f, 'f', -1, 64)
if !strings.Contains(fs, ".") {
fs += ".0"
}
return fs
}
// pythonEscapeString — Python json.dumps string escaping with
// ensure_ascii=True: control chars via \uXXXX (with \b \f \n \r \t
// shortcuts), non-ASCII via \uXXXX / surrogate pairs, lowercase hex.
func pythonEscapeString(s string) string {
var sb strings.Builder
sb.WriteByte('"')
for _, r := range s {
switch r {
case '"':
sb.WriteString(`\"`)
case '\\':
sb.WriteString(`\\`)
case '\n':
sb.WriteString(`\n`)
case '\r':
sb.WriteString(`\r`)
case '\t':
sb.WriteString(`\t`)
case '\b':
sb.WriteString(`\b`)
case '\f':
sb.WriteString(`\f`)
default:
if r < 0x20 || r > 0x7e {
if r > 0xffff {
r1, r2 := utf16.EncodeRune(r)
fmt.Fprintf(&sb, `\u%04x\u%04x`, r1, r2)
} else {
fmt.Fprintf(&sb, `\u%04x`, r)
}
} else {
sb.WriteRune(r)
}
}
}
sb.WriteByte('"')
return sb.String()
}
// pyDumpsNoASCII renders v like Python json.dumps(v, ensure_ascii=False)
// with the default (', ', ': ') separators — the exact string
// compile_full caches via put_cache. Byte-parity matters at the _compress
// 800-char truncation boundary (cache hit vs re-query path).
func pyDumpsNoASCII(v any) string {
var sb strings.Builder
encodeValueNoASCII(&sb, v)
return sb.String()
}
func encodeValueNoASCII(sb *strings.Builder, v any) {
switch t := v.(type) {
case nil:
sb.WriteString("null")
case bool:
if t {
sb.WriteString("true")
} else {
sb.WriteString("false")
}
case string:
sb.WriteString(pythonEscapeStringRaw(t))
case int:
sb.WriteString(strconv.Itoa(t))
case int64:
sb.WriteString(strconv.FormatInt(t, 10))
case float64:
sb.WriteString(pythonFloat(t))
case json.Number:
sb.WriteString(t.String())
case []any:
sb.WriteString("[")
for i, item := range t {
if i > 0 {
sb.WriteString(", ")
}
encodeValueNoASCII(sb, item)
}
sb.WriteString("]")
case *OrderedMap:
sb.WriteString("{")
for i, kv := range t.Entries() {
if i > 0 {
sb.WriteString(", ")
}
sb.WriteString(pythonEscapeStringRaw(kv.Key))
sb.WriteString(": ")
encodeValueNoASCII(sb, kv.Value)
}
sb.WriteString("}")
default:
sb.WriteString(pythonEscapeStringRaw(fmt.Sprintf("%v", t)))
}
}
// pythonEscapeStringRaw — the same escaping as pythonEscapeString but
// ensure_ascii=False: non-ASCII runes pass through as UTF-8.
func pythonEscapeStringRaw(s string) string {
var sb strings.Builder
sb.WriteByte('"')
for _, r := range s {
switch r {
case '"':
sb.WriteString(`\"`)
case '\\':
sb.WriteString(`\\`)
case '\n':
sb.WriteString(`\n`)
case '\r':
sb.WriteString(`\r`)
case '\t':
sb.WriteString(`\t`)
case '\b':
sb.WriteString(`\b`)
case '\f':
sb.WriteString(`\f`)
default:
if r < 0x20 {
fmt.Fprintf(&sb, `\u%04x`, r)
} else {
sb.WriteRune(r)
}
}
}
sb.WriteByte('"')
return sb.String()
}
+500
View File
@@ -0,0 +1,500 @@
package main
// scenario.go — the gap battery executor (replay-gap subcommand +
// TestGapParity share execGapRow) and the scenario driver (live parity:
// same ops against the Python oracle and this port, JSON diffed by
// scripts/check_persona_definition_parity.py).
import (
"database/sql"
"encoding/json"
"fmt"
"os"
"path/filepath"
"reflect"
_ "modernc.org/sqlite"
)
type sqlDB = sql.DB
func sqlOpen(driver, path string) (*sqlDB, error) { return sql.Open(driver, path) }
// execGapRow — one truth-row / scenario-op against hermetic throwaway DBs.
// fn mirrors the generator's wrapper names; capture shapes are identical.
func execGapRow(fn string, args []any, kwargs map[string]any) (any, error) {
switch fn {
case "_pd_by_topic":
if len(args) != 2 {
return nil, fmt.Errorf("want (rows, topics)")
}
rows, ok := args[0].([]any)
if !ok {
return nil, fmt.Errorf("rows not a list")
}
topics := []string{}
if ts, ok := args[1].([]any); ok {
for _, t := range ts {
s, _ := t.(string)
topics = append(topics, s)
}
}
return byTopic(rows, topics), nil
case "_pd_emit_constitution":
by, err := asBy(args)
if err != nil {
return nil, err
}
return emitConstitution(by), nil
case "_pd_emit_identity":
by, err := asBy(args)
if err != nil {
return nil, err
}
return emitIdentity(by), nil
case "_pd_emit_persona":
by, err := asBy(args)
if err != nil {
return nil, err
}
limit := 20
if v, ok := kwargs["limit"]; ok {
limit = int(numArg(v, 20))
}
return emitPersona(by, limit), nil
case "_pd_handle_hook":
event, _ := args[0].(string)
var data any
if len(args) > 1 {
data = args[1]
}
return handleHook(event, data), nil
case "_pd_compile_full":
if len(args) < 1 {
return nil, fmt.Errorf("want (seed[, ...])")
}
seed, ok := args[0].([]any)
if !ok {
return nil, fmt.Errorf("seed not a list")
}
calls := 2
if v, ok := kwargs["calls"]; ok {
calls = int(numArg(v, 2))
}
var preseed any
if v, ok := kwargs["preseed_cache"]; ok {
preseed = v
}
return execCompileFullSeq(seed, calls, preseed)
}
return nil, fmt.Errorf("unknown fn %q", fn)
}
func asBy(args []any) (*OrderedMap, error) {
if len(args) < 1 {
return nil, fmt.Errorf("want (by)")
}
by, ok := args[0].(*OrderedMap)
if !ok {
return nil, fmt.Errorf("by not an object")
}
return by, nil
}
func numArg(v any, def float64) float64 {
if f, ok := numFloat(v); ok {
return f
}
return def
}
// execCompileFullSeq — the generator's _pd_compile_full capture: seed a
// throwaway store (+ optional bogus pre-cached value), run compileFull
// `calls` times, capture results + the path-proving counters.
func execCompileFullSeq(seed []any, calls int, preseed any) (any, error) {
dir, err := os.MkdirTemp("", "pd-gap-*")
if err != nil {
return nil, err
}
defer os.RemoveAll(dir)
storePath := filepath.Join(dir, "store.db")
cacheDir := filepath.Join(dir, "mem")
if err := seedBatteryStore(storePath, seed); err != nil {
return nil, err
}
e := NewEnv(storePath, cacheDir)
if preseed != nil {
if s, ok := preseed.(string); ok {
e.putCache(cacheKey, s, "general")
e.CachePuts = 0 // rig write, not oracle behavior
e.CacheGets = nil
}
}
results := []*OrderedMap{}
for i := 0; i < calls; i++ {
rows, by := e.compileFull()
r := NewOrderedMap()
r.Set("rows", rows)
r.Set("by", by)
results = append(results, r)
}
out := NewOrderedMap()
for i, r := range results {
out.Set(fmt.Sprintf("r%d", i+1), r)
}
gets := []any{}
for _, g := range e.CacheGets {
gets = append(gets, g)
}
out.Set("store_connects", e.StoreConnects)
out.Set("cache_gets", gets)
out.Set("cache_puts", e.CachePuts)
return out, nil
}
// seedBatteryStore — a fresh entries table with the seed rows (explicit
// ids, the columns _core's SELECT and the WHERE clause touch).
func seedBatteryStore(path string, seed []any) error {
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
return err
}
db, err := openBatteryStore(path)
if err != nil {
return err
}
defer db.Close()
for _, s := range seed {
m, ok := s.(*OrderedMap)
if !ok {
return fmt.Errorf("seed row not an object")
}
id := int64(numArg(omValue(m, "id"), 0))
text, _ := omValue(m, "text").(string)
status := "active"
if v, present := m.Get("status"); present && v != nil {
if sv, ok := v.(string); ok {
status = sv
}
}
alwaysOn := int64(1)
if v, ok := m.Get("always_on"); ok {
if b, ok := v.(bool); ok && !b {
alwaysOn = 0
}
}
_, err := db.Exec(
"INSERT INTO entries (id, text, importance, created_at, topic, status, always_on, confidence, kind, priority) "+
"VALUES (?,?,?,?,?,?,?,?,?,?)",
id, text, nullableFloat(omValue(m, "importance")), "2026-01-01T00:00:00",
nullableString(omValue(m, "topic")), status, alwaysOn,
nullableFloat(omValue(m, "confidence")), nullableString(omValue(m, "kind")),
nullableFloat(omValue(m, "priority")))
if err != nil {
return err
}
}
return nil
}
func omValue(om *OrderedMap, key string) any {
v, _ := om.Get(key)
return v
}
// openBatteryStore — a throwaway store with the entries surface the
// oracle's SELECT/WHERE touch (the oracle's connect() creates the full
// vec/fts schema; the Go replay only needs what the query reads).
func openBatteryStore(path string) (*sqlDB, error) {
db, err := sqlOpen("sqlite", path)
if err != nil {
return nil, err
}
if _, err := db.Exec(`
CREATE TABLE IF NOT EXISTS entries (
id INTEGER PRIMARY KEY AUTOINCREMENT,
text TEXT NOT NULL,
importance REAL DEFAULT 0.5,
created_at TEXT NOT NULL,
topic TEXT DEFAULT '',
status TEXT DEFAULT 'active',
confidence REAL,
always_on INTEGER DEFAULT 0,
kind TEXT,
priority REAL
)`); err != nil {
db.Close()
return nil, err
}
return db, nil
}
func nullableString(v any) any {
if s, ok := v.(string); ok {
return s
}
return nil
}
func nullableFloat(v any) any {
if f, ok := numFloat(v); ok {
return f
}
return nil
}
// ── frozen-truth replay ───────────────────────────────────────────
type gapTruthRow struct {
Fn string `json:"fn"`
Args []json.RawMessage `json:"args"`
KwArgs json.RawMessage `json:"kwargs"`
Expected any `json:"expected"`
}
// pyToPlain — OrderedMap/json.Number values → plain Go values.
func pyToPlain(v any) any {
switch t := v.(type) {
case *OrderedMap:
out := map[string]any{}
for _, kv := range t.Entries() {
out[kv.Key] = pyToPlain(kv.Value)
}
return out
case []any:
out := make([]any, len(t))
for i, item := range t {
out[i] = pyToPlain(item)
}
return out
default:
return v
}
}
// normalize — render through the Python JSON encoder and re-parse, so
// ordered maps and number distinctions collapse the same way on both the
// frozen and the got side before DeepEqual.
func normalize(v any) any {
var out any
if err := json.Unmarshal([]byte(encodePythonJSON(pyToPlain(v), -1)), &out); err != nil {
return pyToPlain(v)
}
return out
}
// replayGapBytes replays a truth_gap envelope; returns first divergence.
func replayGapBytes(raw []byte) error {
var env struct {
FrozenNow string `json:"frozen_now"`
Cases []gapTruthRow `json:"cases"`
}
if err := json.Unmarshal(raw, &env); err != nil {
return err
}
fails := 0
for i, row := range env.Cases {
args := make([]any, len(row.Args))
for j, rawArg := range row.Args {
v, err := decodeJSON(rawArg)
if err != nil {
return fmt.Errorf("case %d: bad arg %d: %v", i, j, err)
}
args[j] = v
}
kwargs := map[string]any{}
if len(row.KwArgs) > 0 {
if v, err := decodeJSON(row.KwArgs); err == nil {
if om, ok := v.(*OrderedMap); ok {
for _, kv := range om.Entries() {
kwargs[kv.Key] = kv.Value
}
}
}
}
got, err := execGapRow(row.Fn, args, kwargs)
if err != nil {
fails++
fmt.Printf("case %d %s: ERROR %v\n", i, row.Fn, err)
continue
}
gotN := normalize(got)
wantN := normalize(row.Expected)
if !reflect.DeepEqual(gotN, wantN) {
fails++
gb, _ := json.MarshalIndent(gotN, "", " ")
wb, _ := json.MarshalIndent(wantN, "", " ")
fmt.Printf("case %d %s DIVERGE\ngot: %s\nwant: %s\n", i, row.Fn, gb, wb)
}
}
if fails > 0 {
return fmt.Errorf("PARITY FAILED: %d divergence(s)", fails)
}
fmt.Printf("PARITY OK: %d gap cases exact-equal vs the Python oracle\n", len(env.Cases))
return nil
}
func replayGap(path string) error {
raw, err := os.ReadFile(path)
if err != nil {
return err
}
return replayGapBytes(raw)
}
// ── scenario driver (live parity) ─────────────────────────────────
type scenarioOp struct {
Op string `json:"op"`
Fn string `json:"fn"`
Args []any `json:"args"`
Rows json.RawMessage `json:"rows"`
By json.RawMessage `json:"by"`
Topics []string `json:"topics"`
Seed json.RawMessage `json:"seed"`
Limit *int `json:"limit"`
Calls *int `json:"calls"`
Preseed *string `json:"preseed_cache"`
StoreDB string `json:"store_db"`
MemoryDir string `json:"memory_dir"`
Event string `json:"event"`
Data json.RawMessage `json:"data"`
}
// runScenario — ops mirror execGapRow; plus a cli op that runs the four
// print paths against the scenario's own hermetic env (same capture as
// the oracle-side driver in check_persona_definition_parity.py).
func runScenario(path string) error {
raw, err := os.ReadFile(path)
if err != nil {
return err
}
var sc struct {
Ops []scenarioOp `json:"ops"`
}
if err := json.Unmarshal(raw, &sc); err != nil {
return err
}
steps := []any{}
for _, op := range sc.Ops {
var res any
var rerr error
switch op.Op {
case "fn":
args, kwargs, aerr := scenarioArgs(op)
if aerr != nil {
return aerr
}
res, rerr = execGapRow(op.Fn, args, kwargs)
case "cli":
if len(op.Seed) > 0 {
dir, err := os.MkdirTemp("", "pd-cli-*")
if err != nil {
return err
}
defer os.RemoveAll(dir)
seed, err := decodeJSON(op.Seed)
if err != nil {
return err
}
seedRows, _ := seed.([]any)
if err := seedBatteryStore(filepath.Join(dir, "store.db"), seedRows); err != nil {
return err
}
res, rerr = runCLICapture(filepath.Join(dir, "store.db"),
filepath.Join(dir, "mem"), op.Args)
} else {
res, rerr = runCLICapture(op.StoreDB, op.MemoryDir, op.Args)
}
default:
rerr = fmt.Errorf("unknown op %q", op.Op)
}
step := NewOrderedMap()
step.Set("op", op.Op)
if op.Fn != "" {
step.Set("fn", op.Fn)
}
if rerr != nil {
step.Set("error", rerr.Error())
} else {
step.Set("result", res)
}
steps = append(steps, step)
}
out := NewOrderedMap()
out.Set("steps", steps)
fmt.Println(encodePythonJSON(out, -1))
return nil
}
// scenarioArgs — rebuild the wrapper args from explicit scenario fields
// (JSON values decode ORDER-PRESERVING via decodeJSON).
func scenarioArgs(op scenarioOp) ([]any, map[string]any, error) {
kwargs := map[string]any{}
if op.Limit != nil {
kwargs["limit"] = float64(*op.Limit)
}
if op.Calls != nil {
kwargs["calls"] = float64(*op.Calls)
}
if op.Preseed != nil {
kwargs["preseed_cache"] = *op.Preseed
}
var args []any
switch op.Fn {
case "_pd_by_topic":
rows, err := decodeJSON(op.Rows)
if err != nil {
return nil, nil, err
}
topics := make([]any, len(op.Topics))
for j, t := range op.Topics {
topics[j] = t
}
args = []any{rows, topics}
case "_pd_emit_constitution", "_pd_emit_identity", "_pd_emit_persona":
by, err := decodeJSON(op.By)
if err != nil {
return nil, nil, err
}
args = []any{by}
case "_pd_handle_hook":
args = []any{op.Event}
if op.Data != nil {
decoded, err := decodeJSON(op.Data)
if err != nil {
return nil, nil, err
}
args = append(args, decoded)
}
case "_pd_compile_full":
seed, err := decodeJSON(op.Seed)
if err != nil {
return nil, nil, err
}
args = []any{seed}
default:
return nil, nil, fmt.Errorf("unknown fn %q", op.Fn)
}
return args, kwargs, nil
}
// runCLICapture — the four compile-driven print paths against explicit
// hermetic paths (mirrors the oracle CLI stdout byte-for-byte).
func runCLICapture(storeDB, memoryDir string, args []any) (any, error) {
cmd := "full"
if len(args) > 0 {
if s, ok := args[0].(string); ok {
cmd = s
}
}
return compileCmdOutput(NewEnv(storeDB, memoryDir), cmd)
}
func bucketLen(by *OrderedMap, key string) int {
return len(bucketRows(by, key))
}
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,59 @@
package main
// truth_gap_test.go — gap-close battery: replays truth_gap.json (frozen
// against the live Python oracle through scripts/gen_persona_definition_
// gap_truth.py wrappers, byte-identical via its --verify) and pins the
// char-based semantics the frozen rows ride on (_compress truncation and
// emit_persona's r[:200] slice by CHARACTERS, never bytes).
import (
"os"
"strings"
"testing"
)
func TestGapParity(t *testing.T) {
raw, err := os.ReadFile("truth_gap.json")
if err != nil {
t.Fatalf("truth missing: %v", err)
}
if err := replayGapBytes(raw); err != nil {
t.Fatalf("gap parity: %v", err)
}
}
// TestCompressCharBoundary — the 800-char truncation slices CHARACTERS
// (801 chars / 804 bytes: a byte-slicer would cut the multibyte tail).
func TestCompressCharBoundary(t *testing.T) {
s := strings.Repeat("x", 799) + "é中" // 801 chars, 804 bytes
got := pyCompress(s)
want := strings.Repeat("x", 797) + "..."
if got != want {
t.Errorf("compress boundary: got %q…", got[:min(24, len(got))])
}
}
// TestEmitPersonaCharTruncation — r[:200] is a rune slice.
func TestEmitPersonaCharTruncation(t *testing.T) {
by := NewOrderedMap()
row := NewOrderedMap()
row.Set("text", strings.Repeat("é", 210))
row.Set("confidence", 0.5)
by.Set("persona", []any{row})
got := emitPersona(by, 20)
want := "- " + strings.Repeat("é", 200)
if got != want {
t.Errorf("rune truncation: got %d chars want %d", utf8Len(strings.TrimPrefix(got, "- ")), 200)
}
}
// TestCacheValueRenderer — the cached JSON renders with Python's
// ensure_ascii=False separators so the truncation boundary matches.
func TestCacheValueRenderer(t *testing.T) {
om := NewOrderedMap()
om.Set("rows", []any{})
om.Set("by", NewOrderedMap())
if got := pyDumpsNoASCII(om); got != `{"rows": [], "by": {}}` {
t.Errorf("pyDumpsNoASCII: got %q", got)
}
}
+1
View File
@@ -0,0 +1 @@
0
+1
View File
@@ -0,0 +1 @@
0
+1
View File
@@ -0,0 +1 @@
0
+1
View File
@@ -0,0 +1 @@
0
+1
View File
@@ -0,0 +1 @@
0
+1
View File
@@ -0,0 +1 @@
0
+1
View File
@@ -0,0 +1 @@
0
+1
View File
@@ -0,0 +1 @@
0
+1
View File
@@ -0,0 +1 @@
0
+1
View File
@@ -0,0 +1 @@
0
+53
View File
@@ -0,0 +1,53 @@
# persona_definition GO gap-close — verification summary (#2479 dichotomy wave)
Module: src/rune/persona_definition.py (210L, 7 audit fns) → src/go/persona-definition
Worktree: rune-wave-dj (fresh gitea/master checkout at 0eb38f28)
## Coverage
- Before: 2/7 PARTIAL (missing: _by_topic, compile_full, emit_constitution,
emit_identity, emit_persona)
- After: **7/7 FULL** (dichotomy_audit --write regenerated; module-level
125/192 FULL, fn-level 1721/2299)
## Commands and evidence (all exit=0 unless noted; logs local, *.log gitignored)
| Stage | Command | Evidence |
|---|---|---|
| freeze verify | `python3 scripts/gen_persona_definition_gap_truth.py --verify` | pd-freeze-verify.{log,exit} |
| fresh regen | regen + git diff empty (byte-identical) + re-verify | pd-freeze-regen.{log,exit}, pd-freeze-verify2.{log,exit} |
| Go battery | `go test ./...` (4 tests, 33 truth cases) | pd-gotest.{log,exit} |
| replay (2nd modality) | `/tmp/persona-definition replay-gap truth_gap.json` | pd-replay-gap.{log,exit} |
| gofmt | `gofmt -l .` (empty) | pd-gofmt.{log,exit} |
| go vet | `go vet ./...` | pd-govet.{log,exit} |
| go build | `go build ./...` | pd-gobuild.{log,exit} |
| live parity | `python3 scripts/check_persona_definition_parity.py` — 3/3 scenarios (14 ops) + 11/11 real-CLI invocations byte-identical (incl. argparse exit-2 and bad --data exit-1 crash parity) | pd-parity.{log,exit} |
| Python pin | `RUNE_PERSONA_DEFINITION_CORE=python ... --verify` (module has no delegation loader; the pin is inert — oracle-pinned surface = the live oracle module itself, byte-identical) | pd-py-pin-python.{log,exit} |
## Truth-freeze technique
Wrapper shims in scripts/gen_persona_definition_gap_truth.py (cron_engine
pattern) via the shared harness: per-row hermetic mkdtemp store+cache DBs
(module-global memory_store.DB_PATH / cache.DB_PATH re-pointed BEFORE any
oracle call — the lazy `import cache` inside compile_full binds the same
sys.modules entry); counters wrap ms.connect / cache.get_cache /
cache.put_cache so each capture PROVES the path (cache hit / store
re-query / corrupted-entry fallthrough). 33 cases: _by_topic (7),
emit_constitution (4), emit_identity (3), emit_persona (12),
handle_hook (3), compile_full (6: small-hit, empty, WHERE-filters,
>800-truncated-requery, corrupted preseed, single-call). No timestamps,
no absolute paths in truth.
## Documented divergences (not bugs)
1. sqlite3.OperationalError from a broken/unwritable cache.db is UNCAUGHT
by the oracle's compile_full except-list and crashes; the port treats
it as a cache miss and continues. Never exercisable hermetically
(fresh writable dirs); recorded as crash-class divergence.
2. `str.lower()` vs `strings.ToLower` on special-casing runes (e.g. İ
U+0130) — fixtures avoid; standard ASCII/simple-fold cases identical.
3. The _core subprocess fallback (import-failure class, needs a broken
sqlite_vec install) is ported (python3 memory_store.py query-core
--json, [] on parse failure) but not freezable — documented, mirrors
oracle behavior.
## Files
- src/go/persona-definition/{go.mod,go.sum,gaps.go,pyjson.go,scenario.go,main.go,truth_gap.json,truth_gap_test.go}
- scripts/gen_persona_definition_gap_truth.py, scripts/check_persona_definition_parity.py
- docs/dichotomy-rebase-honest-audit.md (regenerated)