conductor/rune
1
2 Fork 1
Code Issues 38 Pull Requests 32 Actions 10 Packages Projects Releases 564 Wiki Activity
Files
2eb3227ad0ec57fb259d12f754a183dfafb94de2
rune/docs/SCAFFOLDING_IMPLEMENTATION_AUDIT.md
T
Nick 3fb28e2461 feat(t19): testing mode + self-hosted /testing command + T19 layer model (19L-19U) 
- testing_mode.py (19L/M/N/O/P): TestingSession (coordination-only,
  pausable — durable truth stays in the Behaviour Ledger); dynamic
  queue (fix-for-B-reopens-A via structural closure, 19S-9 proven);
  Observation→closure gate (closure requires retest AND user
  confirmation — automated green alone recorded as NOT closed);
  forge projection through the T18 boundary (failure never rewrites
  truth, 19S-14 proven); cross-project aggregate.
- 19R LIVE_PROOF: /testing command implemented through the full
  system — self-scaffold lookup → capability-ledger packet (bound
  exactly ONE file) → qualified INWARD basis → predictive impact →
  preview → contract 19r-testing-command → event-bridged verification
  → descriptive impact (prediction matched, delta persisted) →
  selective invalidation + obligations → ledgers updated → closed.
- 19T benchmark (honest): cold agent found the answer in 6 tool calls
  (4 files) — on this well-organized repo raw speed ≈ equal; the
  scaffold advantage is correctness/continuity (validation state,
  obligations, invalidation), recorded without inflation.
- 19U: V1 definition gains the T19 layer model + both governing
  objectives verbatim; implementation audit gains T19 classifications.
- type ratchet: forge.py gitea_auth import made resolvable via
  pyrightconfig extraPaths (the +1 warning FIXED, not re-baselined).

Suite 2002 green; ratchets PASS; Henxels rc=0.
2026-09-16 01:24:29 +01:00

80 lines
6.2 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# SCAFFOLDING IMPLEMENTATION AUDIT + V1 GAP MATRIX (T18 §18B)
_Every claim classified against evidence. Scale: LIVE_PROVEN (production
evidence) / TEST_PROVEN (suite green, no production exercise) /
IMPLEMENTED_UNPROVEN / PARTIAL / PLANNED / ABSENT / LEGACY / DUPLICATE._
## Kernel claims
| Claim | Status | Evidence |
|---|---|---|
| Contract lifecycle + legal transitions | LIVE_PROVEN | 28 contracts driven through real flow; refusal envelopes observed |
| classify_mode (one classifier) | LIVE_PROVEN | gates TUI /build (17G), operator hook, tests 62+ |
| Conversational intent layer | TEST_PROVEN | 22+ held-outs incl. unseen paraphrases; 17R live exercise (one real conversational proof) |
| Programme continuation | LIVE_PROVEN | 17E 4-stage proof on real bridge; restart/dup/failure held-outs |
| Event-driven waits (builder) | LIVE_PROVEN | T15 15Q real model build; T17E async stage |
| Event-driven waits (CI) | LIVE_PROVEN | T16 16D deadline path ×2 + T17 natural wake (mode=event, 112.5s) |
| Event replay (before-registration) | TEST_PROVEN + production-realistic exercise | 17C 4.6s vs real completed CI; natural-cycle confirmation: the 17Z wake arrived mid-wait, replay path itself not yet hit naturally |
| Cross-user bridge delivery | LIVE_PROVEN | 17C repro as runner user (2.05s) + production deploys |
| Provider boundary | TEST_PROVEN (structural guard) + observed loud-failure | 16F/G suites; no silent fallback observed in production runs |
| Builder cannot self-certify | LIVE_PROVEN | T14 closure refusals ×2; phase-gated close |
| Completion truth evidence-driven | LIVE_PROVEN | contract checkpoints; verify_test_truth refusals |
| Interruption/resume | LIVE_PROVEN | T14E SIGKILL; 17E restart held-outs |
| Impact preview (visible, pre-build) | ABSENT → T18 | nothing renders structured impact before build today |
| Structural project model (first-class) | PARTIAL | stores are spine-shaped (entities/rels) but projects model CONTRACTS, not architecture; no domain/subsystem/interface nesting for arbitrary projects |
| Structural queries/impact closure for projects | PARTIAL | spine.impact_frontier exists for Rune; project stores lack query API |
| Research→Build Basis (durable, versioned) | ABSENT → T18 | research output today stays in transient context/artifacts |
| Research-depth decision at construction entry | PARTIAL | ADAPT routes task classes; no explicit construction-stage research gate |
| OUTWARD prior-art-first pass | PARTIAL | stage exists in Research Core; not strengthened to precedent-first ordering |
| Schema versioning + consent-gated migration | PARTIAL | stores carry schema_version=1; no migration machinery/consent flow |
| Environment/runtime model | PARTIAL | deployment_surface exists in Spine; project scaffolds don't model envs |
| Workspace lifecycle (worktree per contract) | PARTIAL | BuildManager worktrees exist (legacy adapter); not wired to contracts |
| Forge abstraction | ABSENT (Gitea-coupled scripts) | deploy/qualify call Gitea API directly |
| Structural drift for project scaffolds | PARTIAL | drift_resume (16P) covers checkpoint-vs-reality; no file/entity-level drift detection |
| Graph projection for visualisation | ABSENT → T18 | Spine has snapshots; no stable read-only graph projection |
| Memory/Substrate | ABSENT (BY DESIGN — out of scope) | rule L |
## V1 North Star gap matrix (this tranche's targets)
| North Star capability | Gap | T18 action |
|---|---|---|
| Research-led construction (early research, Build Basis) | ABSENT Build Basis | 18C: build_basis.py + ADAPT depth hook |
| OUTWARD precedent-first | PARTIAL | 18D: strengthen stage contract + real research run |
| First-class structural Scaffold | PARTIAL (contract-shaped) | 18E: structural_model.py over structural_store core |
| Structure→scope→governance | PARTIAL | 18F: queries + impact closure feeding contracts |
| Visible impact preview | ABSENT | 18G: impact_preview projection |
| Structural drift | PARTIAL | 18H: extend drift_resume with entity-level checks |
| Schema versioning + consent | PARTIAL | 18I: migration + consent flow + tests |
| Environment/runtime model | PARTIAL | 18J: env entities + verification distinction |
| Workspace lifecycle | PARTIAL (legacy adapter) | 18K: workspace contract on the BuildManager adapter |
| Forge boundary | ABSENT | 18L: forge.py adapter interface (Gitea impl) |
| Full route convergence | PARTIAL (classify shared; backends differ) | 18M: audit + smallest fix |
| Generalisation (not phrase list) | TEST_PROVEN | 18N: adversarial paraphrase families |
| Self-scaffold | PARTIAL (contracts only) | 18O: structural model of Scaffolding itself |
| Upgrade-via-Scaffold proof | ABSENT | 18P: one real upgrade driven by the scaffold |
| Graph projection | ABSENT | 18Q: versioned read-only projection |
## T19 additions (post-T18 classifications)
| Capability | Status | Evidence |
|---|---|---|
| Project/Scaffold Registry | TEST_PROVEN | registry tests; boundaries proven (no truth in index, no Spine leak) |
| /build governance fail-closed | LIVE_PROVEN | 19A: T18 degradation paths removed; failure-path held-outs |
| Capability Ledgers | TEST_PROVEN | lifecycle+history+packet tests |
| Surgical context packets | LIVE_PROVEN | 19R: packet bound exactly 1 file for a real change |
| Behaviour/Test Specs + hardening lifecycle | TEST_PROVEN | ledger suites incl. generation history |
| Selective invalidation | TEST_PROVEN | 19S-2/5/9/10 held-outs |
| Predictive vs descriptive impact | TEST_PROVEN | delta persistence tests |
| Post-change obligations | TEST_PROVEN | machine-derived from structural truth |
| /testing backend + aggregate | TEST_PROVEN + LIVE (19R) | testing_mode suites; command wired in real TUI |
| Observation→closure (user-confirm gate) | TEST_PROVEN | closure requires retest AND user confirm |
| Forge failure never rewrites truth | TEST_PROVEN | 19S-14 held-out |
| Self-hosted T19 change | LIVE_PROVEN | contract 19r-testing-command, 11-step scaffold-driven flow |
## Kernel exceptions required (rule B record)
None yet. Every T18 plan so far EXTENDS (new modules consuming
structural_store) rather than modifying frozen kernel behavior. Any
exception discovered during implementation will be recorded here with
evidence that the V1 requirement could not be met otherwise.
Reference in new issue View Git Blame Copy Permalink