phase-g slice 2: supersession ledger + registry remap + shared deletion
- plans/2026-10-03-phaseG-evidence-provenance-supersession.md: the slice-level ledger — port table (6/6, commits + parity + consumer proof), 12-step doctrine discharge, vintage-merge disclosures (symmetry guard, scan_text repair, port-1 sequence break), consumer map, live-data rules honored. - shared/audit_ledger.py DELETED (superseded; zero importers verified by sweep; deployed shim + Rust core proven live before removal — language-governance rule ⑥). - coverage registry: audit_ledger removed from SHARED_D. - regression gate: exit 0 pre- and post-deletion.
This commit is contained in:
Nick committed
1 parent
969e39f6a3
commit
2e68b6b054
3 files changed
+62
-211
No files matched your search
@@ -0,0 +1,62 @@
|
||||
# Phase G slice 2 — evidence-provenance authority supersession ledger
|
||||
|
||||
**Date:** 2026-10-03 · **Branch:** `phase-g-strangler-evidence-provenance` · **Unit issue:** conductor/rune#2449
|
||||
|
||||
## What was superseded
|
||||
|
||||
The pure-Python evidence-provenance implementations (six modules in `src/rune/`, 1796 lines; plus the two OLD deployed vintages and `shared/audit_ledger.py`) are RETIRED as production implementations for their ported surfaces. Production authority now rests with:
|
||||
|
||||
1. **Rust crate `src/rust/evidence_provenance/`** — six cores: `lib.rs` (audit_ledger chain + FFI), `verification.rs` (7 verifiers), `probe.rs` (schema gate + budget), `behaviour.rs` (state machine), `habitmath.rs` (reinforce math), `claims.rs` (claim-audit decision skeleton); cdylib FFI (`ep_*` symbols, 21 total), zero external crates.
|
||||
2. **Python shims** at `src/rune/{audit_ledger,verification_layer,evidence_probe,behaviour_ledger,behavior,claim_audit}.py` — serving the src/rune authority contract through `ep_loader.py` (in-process, no interpreter startup for the ported logic).
|
||||
3. **Deployed shims** at `~/.config/opencode/scripts/{audit_ledger,behavior,claim_audit}.py` — same shape; audit_ledger + claim_audit + behavior carry the LIVE data-path rules (drift-key ledger, live behavior.db, real journal); claim_audit additionally carries the preserved SYMMETRY GUARD (deployed-vintage feature) and the repaired `scan_text()` TUI surface.
|
||||
|
||||
## Port ledger (all six, in contract §4 order)
|
||||
|
||||
| Port | Module | Rust core | Parity | Commit | Consumer proof |
|
||||
|------|--------|-----------|--------|--------|----------------|
|
||||
| 1 | audit_ledger | lib.rs chain ops | shadow 18/18; canaries 28/28+23/23 | `114c394`+`4b0d1d2` | live ledger entries 4242 (post-switch write), verify 23-intact |
|
||||
| 2 | verification_layer | verification.rs | 33/33 (7 verifiers + CLI render) | `9cb93e6` | contracts.py, canary_native, rune_operator:323 dataclass |
|
||||
| 3 | evidence_probe | probe.rs | 24/24 + consumer tests 23/23 | `3450a1c` | rune_operator:581/603, research_stage_executor:312 |
|
||||
| 4 | behaviour_ledger | behaviour.rs | 13/13 full-scenario | `8973caa` | testing_mode:19, iteration:23, rune_tui:3152 |
|
||||
| 5 | behavior | habitmath.rs | 16/16 + full-stack live-db | `2732677` | rune_tui:83; deployed vintage upgraded (live db read-verified) |
|
||||
| 6 | claim_audit | claims.rs | 17/17 + real-journal aggregate (1127 rows) | `969e39f` | rune_tui:62/2468/3752, contracts.py, hook_engine:270 |
|
||||
|
||||
Errors: byte-equal (duplicate/unknown-behaviour/unknown-state; probe schema-gate lists incl. Python repr rendering; CLI argparse shapes + exit codes).
|
||||
|
||||
## Strangler doctrine (master doc lines 685–770) — step discharge (slice level)
|
||||
|
||||
| Step | Requirement | Evidence |
|
||||
|------|-------------|----------|
|
||||
| 1–4 | candidate built, contract frozen, parity proven | commits `114c394` → `969e39f`; frozen contract `plans/2026-10-03-phaseG-evidence-provenance-contract.md` (commit `7c68e41`); per-port parity harnesses ALL GREEN |
|
||||
| 5 | qualification | per-port parity + decision-math batteries; canary pair per port (Python 34/34, Rust 28/28 at slice end); performance follows the slice-1 in-process shape (no interpreter startup; same cdylib) |
|
||||
| 6 | canaries conformance | canary_audit_ledger.py 34/34; canary_evidence_provenance_rust.py 28/28 (incl. port-2/3 sections); embedded tests through shims (verification test, 23 consumer tests) |
|
||||
| 7 | shadow comparison vs live-shaped data | port-1 shadow 18/18 vs `939883d:shared/audit_ledger.py` (banked at `114c394`); ports 2–6 use direct byte-parity batteries vs the live originals (same discipline, stronger: full-document byte-quality) |
|
||||
| 8 | authority switch + observed consumer evidence | shims deployed + live-verified per port: entry 4242 (first post-switch ledger write), live behavior.db read through the port-5 shim, real 1127-row journal through the port-6 aggregation, TUI backend trio imports, TUI scan_text FIRST-EVER live call |
|
||||
| 9 | runtime evidence post-switch | post-switch walls per port: parity round-trips, embedded tests, consumer chains (rune.behaviour_ledger / rune.evidence_probe / rune.claim_audit + deployed trio) — all GREEN |
|
||||
| 10 | old production path unreachable | `import audit_ledger` (deployed consumers) → shim; `rune.behaviour_ledger` / `rune.evidence_probe` / `rune.claim_audit` / `rune.verification_layer` → shims; `shared/audit_ledger.py` had ZERO importers (verified by sweep incl. path-string forms) — DELETED |
|
||||
| 11 | old implementation removed or reference-only | `shared/audit_ledger.py` DELETED (this commit, after the proven wall + zero-importer sweep); six src/rune modules remain as shims (the API contract); original Python lives in git history. Coverage registry remapped: `audit_ledger` removed from SHARED_D; gate exit 0 post-deletion |
|
||||
| 12 | supersession ledger | THIS FILE |
|
||||
|
||||
Deletion discipline (language-governance rule ⑥): `shared/audit_ledger.py` removed only after (a) the deployed shim proved live against the real ledger, (b) the shadow wall was green, (c) a zero-importer sweep across src/ shared/ scripts/ tests/. The remaining five src/rune originals were rewritten AS shims in their own commits (git history holds each original); they are the API contract and stay.
|
||||
|
||||
## Vintage-merge disclosures
|
||||
|
||||
- **SYMMETRY GUARD preserved:** the deployed `claim_audit` vintage carried `research_lens.symmetry_check` integration (2026-08-15) absent from src/rune — preserved in both shims (feature-preserving merge; load-bearing behavior).
|
||||
- **DEAD CONSUMER REPAIRED:** `rune_tui.py:3752` `claim_audit.scan_text()` had NO implementation in either vintage (silent try/except death) — now implemented, first-class, live-verified.
|
||||
- **Sequence break (port 1):** the audit_ledger shim was deployed BEFORE its canary pair existed; both canaries subsequently written and green — disclosed on #2449 comment 33817, not retrofitted.
|
||||
|
||||
## Consumer map at supersession
|
||||
|
||||
- `~/.config/opencode/scripts/rune_tui.py` — claim_audit (report L2468, scan_text L3752), behaviour (L3152), behavior (L83).
|
||||
- `src/rune/rune_operator.py` — evidence_probe (L581/603), VerificationResult dataclass (L323).
|
||||
- `src/rune/contracts.py` — dynamic `__import__` surfaces (verification_layer, claim_audit).
|
||||
- `src/rune/testing_mode.py:19` / `src/rune/iteration.py:23` — behaviour_ledger constants + class.
|
||||
- `scripts/research_stage_executor.py:312` — evidence_probe orchestration.
|
||||
- `~/.config/opencode/scripts/curator.py` / `canary_full_system.py` — audit_ledger (deployed shim).
|
||||
- `src/rune/dependency_tracker.py` — map-strings only (NOT an import).
|
||||
|
||||
## Live-data rules (contract §3) — honored in the shims
|
||||
|
||||
- Live promotion ledger: drift key verified 23-intact through the shim + FFI; explicit `AUDIT_LEDGER_KEY` override honored.
|
||||
- Live behavior.db: read/written through the deployed shim at `~/.config/opencode/memory/behavior.db`; parity scenarios ran on temp dirs only.
|
||||
- Live claim_audit journal (1127 rows): read-only aggregate through the Rust core; writes stay Python.
|
||||
@@ -166,7 +166,6 @@ SCRIPTS_C = {"fake_worker", "tui_startup_canary"}
|
||||
|
||||
# shared/ — harness-facing layer; mirrors of canonical src modules = D
|
||||
SHARED_D = {
|
||||
"audit_ledger": "src/rune/audit_ledger.py",
|
||||
"dependency_tracker": "src/rune/dependency_tracker.py",
|
||||
"drift_cache": "src/rune/drift_cache.py",
|
||||
"drift_compression": "src/rune/drift_compression.py",
|
||||
|
||||
@@ -1,210 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""audit_ledger.py — HMAC-chained tamper-evident promotion ledger.
|
||||
|
||||
Every promotion decision is recorded in a hash-chained ledger.
|
||||
Tampering with any entry breaks the chain.
|
||||
|
||||
Research basis:
|
||||
- PoEM (arXiv:2608.16032): HMAC-chained tamper-evident execution ledger
|
||||
- Autogenesis (arXiv:2604.15034): versioned interfaces, auditable lineage
|
||||
|
||||
Run under ~/.venvs/memory/bin/python3.
|
||||
"""
|
||||
|
||||
import hashlib
|
||||
import hmac
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
from datetime import datetime, timezone
|
||||
from typing import Dict, List, Optional
|
||||
|
||||
LEDGER_FILE = os.path.expanduser("~/.config/opencode/memory/promotion_ledger.jsonl")
|
||||
SECRET_KEY = os.environ.get("AUDIT_LEDGER_KEY", "rune-drift-integrity-2024")
|
||||
|
||||
|
||||
def _compute_hash(prev_hash: str, entry: dict) -> str:
|
||||
"""Compute HMAC chain hash for a ledger entry."""
|
||||
payload = json.dumps(entry, sort_keys=True).encode()
|
||||
msg = (prev_hash + payload.decode()).encode()
|
||||
return hmac.new(SECRET_KEY.encode(), msg, hashlib.sha256).hexdigest()
|
||||
|
||||
|
||||
def append_entry(entry_type: str, entry_id: int, details: dict) -> dict:
|
||||
"""Append an entry to the promotion ledger.
|
||||
|
||||
Returns the ledger entry with chain hash.
|
||||
"""
|
||||
# Read last hash
|
||||
prev_hash = "genesis"
|
||||
if os.path.exists(LEDGER_FILE):
|
||||
with open(LEDGER_FILE, "r") as f:
|
||||
lines = f.readlines()
|
||||
if lines:
|
||||
try:
|
||||
last = json.loads(lines[-1])
|
||||
prev_hash = last.get("hash", "genesis")
|
||||
except json.JSONDecodeError:
|
||||
prev_hash = "genesis"
|
||||
|
||||
entry = {
|
||||
"type": entry_type,
|
||||
"entry_id": entry_id,
|
||||
"details": details,
|
||||
"timestamp": datetime.now(timezone.utc).isoformat(),
|
||||
"prev_hash": prev_hash,
|
||||
}
|
||||
|
||||
# Compute chain hash
|
||||
entry["hash"] = _compute_hash(prev_hash, entry)
|
||||
|
||||
# Append to ledger
|
||||
os.makedirs(os.path.dirname(LEDGER_FILE), exist_ok=True)
|
||||
with open(LEDGER_FILE, "a") as f:
|
||||
f.write(json.dumps(entry) + "\n")
|
||||
|
||||
return entry
|
||||
|
||||
|
||||
def verify_chain() -> dict:
|
||||
"""Verify the integrity of the entire ledger chain.
|
||||
|
||||
Returns dict with verification result.
|
||||
"""
|
||||
if not os.path.exists(LEDGER_FILE):
|
||||
return {"valid": True, "entries": 0, "reason": "empty ledger"}
|
||||
|
||||
with open(LEDGER_FILE, "r") as f:
|
||||
lines = f.readlines()
|
||||
|
||||
prev_hash = "genesis"
|
||||
for i, line in enumerate(lines):
|
||||
try:
|
||||
entry = json.loads(line)
|
||||
except json.JSONDecodeError:
|
||||
return {"valid": False, "entries": i, "reason": f"invalid JSON at line {i+1}"}
|
||||
|
||||
# Verify prev_hash chain
|
||||
if entry.get("prev_hash") != prev_hash:
|
||||
return {
|
||||
"valid": False,
|
||||
"entries": i,
|
||||
"reason": f"chain broken at line {i+1}: expected prev_hash={prev_hash}, got {entry.get('prev_hash')}"
|
||||
}
|
||||
|
||||
# Verify HMAC
|
||||
stored_hash = entry.get("hash")
|
||||
computed = _compute_hash(prev_hash, {k: v for k, v in entry.items() if k != "hash"})
|
||||
if stored_hash != computed:
|
||||
return {
|
||||
"valid": False,
|
||||
"entries": i,
|
||||
"reason": f"HMAC mismatch at line {i+1}: stored={stored_hash[:16]}..., computed={computed[:16]}..."
|
||||
}
|
||||
|
||||
prev_hash = stored_hash
|
||||
|
||||
return {"valid": True, "entries": len(lines), "reason": "chain intact"}
|
||||
|
||||
|
||||
def get_entries(limit: int = 50) -> List[dict]:
|
||||
"""Get recent ledger entries."""
|
||||
if not os.path.exists(LEDGER_FILE):
|
||||
return []
|
||||
|
||||
with open(LEDGER_FILE, "r") as f:
|
||||
lines = f.readlines()
|
||||
|
||||
entries = []
|
||||
for line in lines[-limit:]:
|
||||
try:
|
||||
entries.append(json.loads(line))
|
||||
except json.JSONDecodeError:
|
||||
continue
|
||||
return entries
|
||||
|
||||
|
||||
def log_promotion(entry_id: int, old_status: str, new_status: str, wash_count: int, reason: str) -> dict:
|
||||
"""Log a promotion decision."""
|
||||
return append_entry("promotion", entry_id, {
|
||||
"old_status": old_status,
|
||||
"new_status": new_status,
|
||||
"wash_count": wash_count,
|
||||
"reason": reason,
|
||||
})
|
||||
|
||||
|
||||
def log_gate_result(entry_id: int, verdict: str, critics: dict) -> dict:
|
||||
"""Log a pre-commit gate result."""
|
||||
return append_entry("gate", entry_id, {
|
||||
"verdict": verdict,
|
||||
"critics": critics,
|
||||
})
|
||||
|
||||
|
||||
def log_drift_alert(alert_type: str, message: str, severity: str) -> dict:
|
||||
"""Log a drift alert."""
|
||||
return append_entry("alert", 0, {
|
||||
"alert_type": alert_type,
|
||||
"message": message,
|
||||
"severity": severity,
|
||||
})
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════════════════════════════════════
|
||||
# CLI
|
||||
# ═══════════════════════════════════════════════════════════════════════════════
|
||||
|
||||
def main():
|
||||
import argparse
|
||||
parser = argparse.ArgumentParser(description="Promotion audit ledger")
|
||||
sub = parser.add_subparsers(dest="cmd")
|
||||
|
||||
# verify
|
||||
sub.add_parser("verify", help="Verify ledger chain integrity")
|
||||
|
||||
# list [--limit N]
|
||||
p_list = sub.add_parser("list", help="Show recent ledger entries")
|
||||
p_list.add_argument("--limit", type=int, default=20)
|
||||
|
||||
# log-promotion <entry_id> <old> <new> <wash_count> <reason>
|
||||
p_log = sub.add_parser("log-promotion", help="Log a promotion")
|
||||
p_log.add_argument("entry_id", type=int)
|
||||
p_log.add_argument("old_status")
|
||||
p_log.add_argument("new_status")
|
||||
p_log.add_argument("wash_count", type=int)
|
||||
p_log.add_argument("reason")
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
if not args.cmd:
|
||||
parser.print_help()
|
||||
sys.exit(1)
|
||||
|
||||
if args.cmd == "verify":
|
||||
result = verify_chain()
|
||||
print(f"Chain valid: {result['valid']}")
|
||||
print(f"Entries: {result['entries']}")
|
||||
print(f"Reason: {result['reason']}")
|
||||
|
||||
elif args.cmd == "list":
|
||||
entries = get_entries(args.limit)
|
||||
if entries:
|
||||
print(f"Recent ledger entries ({len(entries)}):")
|
||||
for e in entries:
|
||||
print(f" [{e.get('type')}] entry_id={e.get('entry_id')} @ {e.get('timestamp', '')[:19]}")
|
||||
details = e.get('details', {})
|
||||
if e.get('type') == 'promotion':
|
||||
print(f" {details.get('old_status')} → {details.get('new_status')} (wash={details.get('wash_count')})")
|
||||
elif e.get('type') == 'gate':
|
||||
print(f" verdict={details.get('verdict')}")
|
||||
else:
|
||||
print("No ledger entries.")
|
||||
|
||||
elif args.cmd == "log-promotion":
|
||||
entry = log_promotion(args.entry_id, args.old_status, args.new_status, args.wash_count, args.reason)
|
||||
print(f"Logged: {entry['hash'][:16]}...")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Reference in new issue
Block a user