Nick
3fb28e2461
- testing_mode.py (19L/M/N/O/P): TestingSession (coordination-only, pausable — durable truth stays in the Behaviour Ledger); dynamic queue (fix-for-B-reopens-A via structural closure, 19S-9 proven); Observation→closure gate (closure requires retest AND user confirmation — automated green alone recorded as NOT closed); forge projection through the T18 boundary (failure never rewrites truth, 19S-14 proven); cross-project aggregate. - 19R LIVE_PROOF: /testing command implemented through the full system — self-scaffold lookup → capability-ledger packet (bound exactly ONE file) → qualified INWARD basis → predictive impact → preview → contract 19r-testing-command → event-bridged verification → descriptive impact (prediction matched, delta persisted) → selective invalidation + obligations → ledgers updated → closed. - 19T benchmark (honest): cold agent found the answer in 6 tool calls (4 files) — on this well-organized repo raw speed ≈ equal; the scaffold advantage is correctness/continuity (validation state, obligations, invalidation), recorded without inflation. - 19U: V1 definition gains the T19 layer model + both governing objectives verbatim; implementation audit gains T19 classifications. - type ratchet: forge.py gitea_auth import made resolvable via pyrightconfig extraPaths (the +1 warning FIXED, not re-baselined). Suite 2002 green; ratchets PASS; Henxels rc=0.
6.2 KiB
6.2 KiB
SCAFFOLDING IMPLEMENTATION AUDIT + V1 GAP MATRIX (T18 §18B)
Every claim classified against evidence. Scale: LIVE_PROVEN (production evidence) / TEST_PROVEN (suite green, no production exercise) / IMPLEMENTED_UNPROVEN / PARTIAL / PLANNED / ABSENT / LEGACY / DUPLICATE.
Kernel claims
| Claim | Status | Evidence |
|---|---|---|
| Contract lifecycle + legal transitions | LIVE_PROVEN | 28 contracts driven through real flow; refusal envelopes observed |
| classify_mode (one classifier) | LIVE_PROVEN | gates TUI /build (17G), operator hook, tests 62+ |
| Conversational intent layer | TEST_PROVEN | 22+ held-outs incl. unseen paraphrases; 17R live exercise (one real conversational proof) |
| Programme continuation | LIVE_PROVEN | 17E 4-stage proof on real bridge; restart/dup/failure held-outs |
| Event-driven waits (builder) | LIVE_PROVEN | T15 15Q real model build; T17E async stage |
| Event-driven waits (CI) | LIVE_PROVEN | T16 16D deadline path ×2 + T17 natural wake (mode=event, 112.5s) |
| Event replay (before-registration) | TEST_PROVEN + production-realistic exercise | 17C 4.6s vs real completed CI; natural-cycle confirmation: the 17Z wake arrived mid-wait, replay path itself not yet hit naturally |
| Cross-user bridge delivery | LIVE_PROVEN | 17C repro as runner user (2.05s) + production deploys |
| Provider boundary | TEST_PROVEN (structural guard) + observed loud-failure | 16F/G suites; no silent fallback observed in production runs |
| Builder cannot self-certify | LIVE_PROVEN | T14 closure refusals ×2; phase-gated close |
| Completion truth evidence-driven | LIVE_PROVEN | contract checkpoints; verify_test_truth refusals |
| Interruption/resume | LIVE_PROVEN | T14E SIGKILL; 17E restart held-outs |
| Impact preview (visible, pre-build) | ABSENT → T18 | nothing renders structured impact before build today |
| Structural project model (first-class) | PARTIAL | stores are spine-shaped (entities/rels) but projects model CONTRACTS, not architecture; no domain/subsystem/interface nesting for arbitrary projects |
| Structural queries/impact closure for projects | PARTIAL | spine.impact_frontier exists for Rune; project stores lack query API |
| Research→Build Basis (durable, versioned) | ABSENT → T18 | research output today stays in transient context/artifacts |
| Research-depth decision at construction entry | PARTIAL | ADAPT routes task classes; no explicit construction-stage research gate |
| OUTWARD prior-art-first pass | PARTIAL | stage exists in Research Core; not strengthened to precedent-first ordering |
| Schema versioning + consent-gated migration | PARTIAL | stores carry schema_version=1; no migration machinery/consent flow |
| Environment/runtime model | PARTIAL | deployment_surface exists in Spine; project scaffolds don't model envs |
| Workspace lifecycle (worktree per contract) | PARTIAL | BuildManager worktrees exist (legacy adapter); not wired to contracts |
| Forge abstraction | ABSENT (Gitea-coupled scripts) | deploy/qualify call Gitea API directly |
| Structural drift for project scaffolds | PARTIAL | drift_resume (16P) covers checkpoint-vs-reality; no file/entity-level drift detection |
| Graph projection for visualisation | ABSENT → T18 | Spine has snapshots; no stable read-only graph projection |
| Memory/Substrate | ABSENT (BY DESIGN — out of scope) | rule L |
V1 North Star gap matrix (this tranche's targets)
| North Star capability | Gap | T18 action |
|---|---|---|
| Research-led construction (early research, Build Basis) | ABSENT Build Basis | 18C: build_basis.py + ADAPT depth hook |
| OUTWARD precedent-first | PARTIAL | 18D: strengthen stage contract + real research run |
| First-class structural Scaffold | PARTIAL (contract-shaped) | 18E: structural_model.py over structural_store core |
| Structure→scope→governance | PARTIAL | 18F: queries + impact closure feeding contracts |
| Visible impact preview | ABSENT | 18G: impact_preview projection |
| Structural drift | PARTIAL | 18H: extend drift_resume with entity-level checks |
| Schema versioning + consent | PARTIAL | 18I: migration + consent flow + tests |
| Environment/runtime model | PARTIAL | 18J: env entities + verification distinction |
| Workspace lifecycle | PARTIAL (legacy adapter) | 18K: workspace contract on the BuildManager adapter |
| Forge boundary | ABSENT | 18L: forge.py adapter interface (Gitea impl) |
| Full route convergence | PARTIAL (classify shared; backends differ) | 18M: audit + smallest fix |
| Generalisation (not phrase list) | TEST_PROVEN | 18N: adversarial paraphrase families |
| Self-scaffold | PARTIAL (contracts only) | 18O: structural model of Scaffolding itself |
| Upgrade-via-Scaffold proof | ABSENT | 18P: one real upgrade driven by the scaffold |
| Graph projection | ABSENT | 18Q: versioned read-only projection |
T19 additions (post-T18 classifications)
| Capability | Status | Evidence |
|---|---|---|
| Project/Scaffold Registry | TEST_PROVEN | registry tests; boundaries proven (no truth in index, no Spine leak) |
| /build governance fail-closed | LIVE_PROVEN | 19A: T18 degradation paths removed; failure-path held-outs |
| Capability Ledgers | TEST_PROVEN | lifecycle+history+packet tests |
| Surgical context packets | LIVE_PROVEN | 19R: packet bound exactly 1 file for a real change |
| Behaviour/Test Specs + hardening lifecycle | TEST_PROVEN | ledger suites incl. generation history |
| Selective invalidation | TEST_PROVEN | 19S-2/5/9/10 held-outs |
| Predictive vs descriptive impact | TEST_PROVEN | delta persistence tests |
| Post-change obligations | TEST_PROVEN | machine-derived from structural truth |
| /testing backend + aggregate | TEST_PROVEN + LIVE (19R) | testing_mode suites; command wired in real TUI |
| Observation→closure (user-confirm gate) | TEST_PROVEN | closure requires retest AND user confirm |
| Forge failure never rewrites truth | TEST_PROVEN | 19S-14 held-out |
| Self-hosted T19 change | LIVE_PROVEN | contract 19r-testing-command, 11-step scaffold-driven flow |
Kernel exceptions required (rule B record)
None yet. Every T18 plan so far EXTENDS (new modules consuming structural_store) rather than modifying frozen kernel behavior. Any exception discovered during implementation will be recorded here with evidence that the V1 requirement could not be met otherwise.