atlas(r1): RESEAL #400 Y repair — bounded evidence-depth + semantic-precision + gate repair (mandate 22 sections); TOP-UP (one serial actor general-6, first-pass SUCCESS): Huang 2310.01798v2 / Kamoi 2406.01297v3 / Snell 2408.03314v1 / GRAB-RAG 2608.22228v1 all → FULL_PAPER with exact scopes recorded (Huang: intrinsic degradation GPT-4 GSM8K 95.5→89.0, claim = prompted intrinsic only, NO skilled-few-shot condition tested, Self-Refine gain = weak-initial-prompt artifact; Kamoi: feedback taxonomy intrinsic/tools/knowledge/fine-tuning, exceptionally-suitable = verification≪generation (CoVe decomposable), RCI/Reflexion oracle positives; Snell: MATH-only PaLM 2-S* fine-tuned PRM+revision, 4× vs best-of-N, ~14× FLOPs-matched only where base non-trivial, verifiable-domain assumption explicit — does NOT transfer to open-ended agents; GRAB-RAG: 3.8B–8B frozen models, 41.6% = P1 macro-avg 6 cells, 13.3% best mitigation at ~50% coverage cost, not generalizable); Koriat 1995 primary CLOSED everywhere (Unpaywall) → secondary-verified via Norman 2016 + Hanczakowski 2017 (A1-30/31 added); ledger 95→97 rows / 97 unique, FP 5 · RS 4 · ABS 49 · MV 84 · DOC 4 = 146 labels; NOVELTY WORDING: bounded-search framing ('no direct match found in the searched sources as of 2026-09-21' + NOVELTY_UNPROVEN) replaces confirmed-unpublished/first-mover/no-published-X across exec/§38×7/saturation/innovation/claims; SUPERLATIVES: 'best published 14.2%/11%' → in-benchmark framing, 'wrong party' → self-diagnosis insufficient alone + hypotheses allowed, progress-estimation claim pass-scoped; SEMANTICS: reflection A/B/C regimes (intrinsic-no-new-evidence may hypothesize never empowers; correctness-strengthening needs external verification; deterministic verifiers exempt) + C005 exact scope; verbal confidence → advisory ranking where validated, never authority (C003b); Snell split — fixed-mode rejection = RUNE DESIGN DECISION, external evidence supporting not proving (§12); surprise = typed expected-vs-observed discrepancy, KL is ONE formal definition Rune does not claim by default (§14/C008); reconciler = BACKSTOP + TIMER-EXPIRY-NEVER-TRIGGERS-LLM-COGNITION invariant (§28); C001 narrowed to OBSERVED inward facts, interpretation moved to RUNE_APPLICABLE synthesis; human calibration limited/inconsistent not universal non-transfer; event-sourced → reconstructable-from-durable-events (#387 canonical authority carried); SEARCH_SATURATION=REACHED vs LOAD_BEARING_DEPTH=PASS split (§12); process_evidence pointer + load_bearing_evidence.md (16-claim support map: DEEP_EXTERNAL/RUNE_DIRECT/DESIGN_INVARIANT/DOWNGRADED) in §41; GATE: +check_load_bearing_depth_map +check_zero_hit_novelty +check_rune_proven_scope (negation-aware) + claim arithmetic/split extended W→W/X/Y + _master_section EOF-boundary bug fixed (numeric end markers below start swept later sections — surfaced real X qualification-vs-claim collision, X gate re-verified clean); negative controls 4/4 CAUGHT byte-identical; rows UNCHANGED 52/38/73/5/8/16 (wording precision only, mechanically verified); canonical 190653 duplicate receipt preserved verbatim, no new write; original FAILED_ONCE preserved (msg_g001a0c71efdeb0015T0xawI7N); REPAIR_CONTINUOUS_EXECUTION=PASS; batched preliminary untouched sha 8af78726; CONTENT-FREEZE AUDIT written LAST; MEMORY_ATLAS_R1_Y_REPAIR_SEALED
This commit is contained in:
1 parent
ed04909053
commit
d476ea1602
10 files changed
+365
-102
No files matched your search
@@ -28,7 +28,7 @@ No-new-evidence repeated reflection loses priority and eventually stops/escalate
|
||||
|
||||
## G. SURPRISE STREAM (ADOPT, two-tier)
|
||||
|
||||
Expected-vs-observed deltas create verification/learning candidates. Prior art: Itti & Baldi 2009 (A3-15 — KL surprise); Pathak ICM (A3-16 — prediction error in learned feature space, noisy-TV answer); RND (A3-17). For: mandate §17/§26; SeaKR internal-state uncertainty is the best mechanistic match (A2-25). Against: expectations are self-authored (adversarial BR) — two-tier model: deterministic canonical, LLM-originated labeled lower-authority + mandatory verification; expectation-independent lesson channel added (adversarial AK — chronic consistent error goes silent without it). Falsifier: E10 — no discrepancy detected, or novelty alone promotes a lesson without outcome verification. Kill: surprise machinery never fires on real drift (covered by reconciler row 30).
|
||||
Expected-vs-observed deltas create verification/learning candidates. Prior art: Itti & Baldi 2009 (A3-15 — Bayesian surprise = KL(posterior‖prior) is ONE formal definition; Rune's default is a typed expected-vs-observed discrepancy, not a claimed Bayesian quantity; Pathak ICM (A3-16 — prediction error in learned feature space, noisy-TV answer); RND (A3-17). For: mandate §17/§26; SeaKR internal-state uncertainty is the best mechanistic match (A2-25). Against: expectations are self-authored (adversarial BR) — two-tier model: deterministic canonical, LLM-originated labeled lower-authority + mandatory verification; expectation-independent lesson channel added (adversarial AK — chronic consistent error goes silent without it). Falsifier: E10 — no discrepancy detected, or novelty alone promotes a lesson without outcome verification. Kill: surprise machinery never fires on real drift (covered by reconciler row 30).
|
||||
|
||||
## H. CAPABILITY-AWARE COGNITION (ADOPT)
|
||||
|
||||
@@ -44,7 +44,7 @@ Progress derived from verified milestones/task state, never fluent estimate. Pri
|
||||
|
||||
## K. RECOVERY-AWARE METACOGNITION (ADAPT)
|
||||
|
||||
Repeated failures query Error/Recovery History before open-ended re-diagnosis. Prior art: A3-24 AgentDebug (corrective feedback → up to 26% relative gains — nearest verified); Kephart & Chess autonomic computing (A3-25); inward G (store ABSENT). For: mandate §25; adversarial AA/BU — priors must be attribution-sourced, superstition-flagged (retry success ≠ diagnosis validation), decayed + applicability-checked. Against: no published recovery-prior system (open gap); depends on attribution machinery that doesn't exist yet. Falsifier: E9 — Rune re-diagnoses from scratch despite an applicable successful recovery on record. Kill: prior-prescribed fixes fail more than fresh diagnosis (superstition rate).
|
||||
Repeated failures query Error/Recovery History before open-ended re-diagnosis. Prior art: A3-24 AgentDebug (corrective feedback → up to 26% relative gains — nearest verified); Kephart & Chess autonomic computing (A3-25); inward G (store ABSENT). For: mandate §25; adversarial AA/BU — priors must be attribution-sourced, superstition-flagged (retry success ≠ diagnosis validation), decayed + applicability-checked. Against: no direct match found in the searched sources (open problem); depends on attribution machinery that doesn't exist yet. Falsifier: E9 — Rune re-diagnoses from scratch despite an applicable successful recovery on record. Kill: prior-prescribed fixes fail more than fresh diagnosis (superstition rate).
|
||||
|
||||
## L. EVENT-DRIVEN METACOGNITION (ADOPT)
|
||||
|
||||
@@ -56,7 +56,7 @@ Numeric probabilities permitted only where empirical per-class history supports
|
||||
|
||||
## N. MEMORY AVAILABILITY ≠ ACCESSIBILITY (ADOPT)
|
||||
|
||||
Retrieval failure cannot silently become ignorance. Prior art: Tulving & Pearlstone (A1-10); Koriat accessibility model (A1-4/5); arm-2 gap finding (no published taxonomy — novel). For: inward B2 (recall() abstains silently — the exact conflation is live in Rune today); adversarial F/G (passes with misleadingness states added). Against: store-health probe engineering per store. Falsifier: E2 — index hidden, canonical memory preserved: Rune must report retrieval/index failure, not absence. Kill: distinguishing the states costs more than the conflation harms.
|
||||
Retrieval failure cannot silently become ignorance. Prior art: Tulving & Pearlstone (A1-10); Koriat accessibility model (A1-4/5); arm-2 bounded-search finding (no direct match found in the searched sources as of 2026-09-21 — NOVELTY_UNPROVEN; adjacent failure/retrieval taxonomies exist). For: inward B2 (recall() abstains silently — the exact conflation is live in Rune today); adversarial F/G (passes with misleadingness states added). Against: store-health probe engineering per store. Falsifier: E2 — index hidden, canonical memory preserved: Rune must report retrieval/index failure, not absence. Kill: distinguishing the states costs more than the conflation harms.
|
||||
|
||||
## O. STRUCTURED SELF-KNOWLEDGE (ADOPT)
|
||||
|
||||
|
||||
@@ -5,11 +5,11 @@ _2026-09-21. Full stage pipeline completed (ISSUE → INWARD → OUTWARD 4 seria
|
||||
|
||||
## 1. Executive summary
|
||||
|
||||
**Rune's metacognition is primitive-but-real: the control half exists and works (ADAPT-2 evidence routing with escalate-on-UNKNOWN), the monitoring half is missing — and the inward audit found five load-bearing gaps that the deep evidence maps almost one-to-one onto published open problems.** C001 (RUNE_PROVEN): score-margin abstention is live (`recall()` floor 0.42) but silent — retrieval failure is indistinguishable from memory absence; confidence fields are curator assignations (652 entries @0.9, never outcome-derived); Spine/rune_model capability truth is fresh and populated with zero metacognitive consumers; Error/Recovery History is absent; the one purpose-built reflection loop is a fossil (think_count 1, August).
|
||||
**Rune already has one live metacognitive-control primitive — ADAPT-2 evidence routing (production-wired, canary-covered, escalate-on-UNKNOWN) — and the inward audit found five load-bearing gaps (silent retrieval abstention, assignation-only confidence, unconsumed capability truth, absent recovery history, a dormant reflection loop) that the deep evidence maps onto under-explored problems. (Interpretation: metacognition is primitive-but-real, thin on monitoring — a RUNE_APPLICABLE synthesis, not itself the RUNE_PROVEN fact; C001 carries only direct inward observations.)** C001 (RUNE_PROVEN): score-margin abstention is live (`recall()` floor 0.42) but silent — retrieval failure is indistinguishable from memory absence; confidence fields are curator assignations (652 entries @0.9, never outcome-derived); Spine/rune_model capability truth is fresh and populated with zero metacognitive consumers; Error/Recovery History is absent; the one purpose-built reflection loop is a fossil (think_count 1, August).
|
||||
|
||||
The architecture answers with an additive, small design: **typed monitoring feeding control through the N&N split** (logical independence kept, Koriat-2006 coupling acknowledged), **typed negative retrieval states** (RETRIEVAL_FAILED / NO_MATCH / INDEX_DOWN / ABSENCE / CONFLICTED / APPLICABILITY_REJECTED / SUSPECTED_MISLEADING / PROVENANCE_CONFLICT / UNKNOWN_AVAILABILITY — a taxonomy the literature search confirmed is unpublished: zero hits), a **rebuildable metamemory projection** (no second Memory DB), **verbal-confidence demoted to ranking-signal-only** until an anti-survivorship outcome flywheel exists, **reflection gated on external verifiability** (Huang/Kamoi: intrinsic self-correction has no verified positive evidence), an **anti-rumination governor with difficulty-adaptive budgets** (Snell compute-optimal), a **baseline reconciler channel** (deterministic periodic audits + positive-probe store-health checks — the adversarial pass proved deviation-triggered monitoring alone is deaf to slow degradation), **capability awareness from runtime truth with freshness bounds** (never parametric self-knowledge), **progress truth from task runtime** (nine typed statuses; inactivity ≠ completion; no invented percentages), and **recovery-aware metacognition** (attribution-sourced, superstition-flagged priors — store is a build-precondition).
|
||||
The architecture answers with an additive, small design: **typed monitoring feeding control through the N&N split** (logical independence kept, Koriat-2006 coupling acknowledged), **typed negative retrieval states** (RETRIEVAL_FAILED / NO_MATCH / INDEX_DOWN / ABSENCE / CONFLICTED / APPLICABILITY_REJECTED / SUSPECTED_MISLEADING / PROVENANCE_CONFLICT / UNKNOWN_AVAILABILITY — a typed negative-state combination for which this R1 search found no directly equivalent published mechanism within the searched sources as of 2026-09-21 (NOVELTY_UNPROVEN — adjacent failure/retrieval taxonomies exist; the exact combination is unproven-new, not proven-novel)), a **rebuildable metamemory projection** (no second Memory DB), **verbal-confidence demoted to ranking-signal-only** until an anti-survivorship outcome flywheel exists, **reflection gated on external verifiability** (Huang/Kamoi: intrinsic self-correction has no verified positive evidence), an **anti-rumination governor with difficulty-adaptive budgets** (Snell compute-optimal), a **baseline reconciler channel** (deterministic periodic audits + positive-probe store-health checks — the adversarial pass proved deviation-triggered monitoring alone is deaf to slow degradation), **capability awareness from runtime truth with freshness bounds** (never parametric self-knowledge), **progress truth from task runtime** (nine typed statuses; inactivity ≠ completion; no invented percentages), and **recovery-aware metacognition** (attribution-sourced, superstition-flagged priors — store is a build-precondition).
|
||||
|
||||
Adversarial pass: 73 attacks, SUCCEEDS 5 · PARTIAL 20 · FAILS 48 — all five genuine breaks share two roots (no baseline channel; self-referential flywheel) and were fixed in place; the third finding (novel components lack substrate) is recorded as hard build-preconditions, not implementation details. Applicability: 52 rows (28/6/3/0/14/1). Qualification: 38 rows (32/6). R0: CONFIRMED 4 · REFINED 1 (Sum 5; batched-era 2C/3R upgraded on re-verified anchors — no overturns). Preliminary: CONFIRMED 5 · REFINED 3 (Sum 8). Claims: 16 (12 ADOPT, 4 ADAPT; elevels 1/11/3/1). Citation corrections this domain: Koriat 1995 (not 1993/97), R-Tuning 2311.09677, DRAGIN 2403.10081, Semantic Density 2405.13845, "Art of Saying No" = Brahman/Kumar 2407.12043, overthinking = 2412.21187, TRAIL = 2505.08638, no Flyvbjerg "frosting error" paper exists. Sources: 95 rows / 95 unique. **Known prior-domain clerical debt (mandate §56, NOT reopened): #399 master §42 pointer says "claims.json — 12 claims"; authoritative #399 count after C010 split is 13.**
|
||||
Adversarial pass: 73 attacks, SUCCEEDS 5 · PARTIAL 20 · FAILS 48 — all five genuine breaks share two roots (no baseline channel; self-referential flywheel) and were fixed in place; the third finding (the absence-taxonomy and outcome-confidence components had no runtime substrate) is recorded as hard build-preconditions, not implementation details. Applicability: 52 rows (28/6/3/0/14/1). Qualification: 38 rows (32/6). R0: CONFIRMED 4 · REFINED 1 (Sum 5; batched-era 2C/3R upgraded on re-verified anchors — no overturns). Preliminary: CONFIRMED 5 · REFINED 3 (Sum 8). Claims: 16 (12 ADOPT, 4 ADAPT; elevels 1/11/3/1). Citation corrections this domain: Koriat 1995 (not 1993/97), R-Tuning 2311.09677, DRAGIN 2403.10081, Semantic Density 2405.13845, "Art of Saying No" = Brahman/Kumar 2407.12043, overthinking = 2412.21187, TRAIL = 2505.08638, no Flyvbjerg "frosting error" paper exists. Sources: 97 rows / 97 unique (repair top-up deepened 4 anchors to FULL_PAPER; Koriat 1995 primary closed everywhere — secondary-verified honestly). **Known prior-domain clerical debt (mandate §56, NOT reopened): #399 master §42 pointer says "claims.json — 12 claims"; authoritative #399 count after C010 split is 13.**
|
||||
|
||||
## 2. Inward reality (read-only audit, 2026-09-21)
|
||||
|
||||
@@ -23,7 +23,7 @@ Non-equality chain, each tested against evidence: uncertainty ≠ probability (t
|
||||
|
||||
## 4. Foundational human metacognition (mandate §7)
|
||||
|
||||
Flavell 1979 (METADATA_VERIFIED): metacognitive knowledge = person/task/strategy triad + metacognitive experiences; problem solved = regulation beyond capacity. Nelson & Narens 1990/1994 (1994 chapter FULL_PAPER — the domain's anchor text): meta-level holds an **imperfect model** of the object-level; control (initiate/continue/terminate/**modify the process**) and monitoring are **logically independent** — "control per se does not yield any information from the object-level" (p.127); monitoring is imperfect-yet-useful (Fodor); judgment taxonomy EOL/JOL/FOK/retrospective; EOL/JOL/FOK uncorrelated (Leonesio & Nelson 1988) — a scalar is a lossy summary. Koriat accessibility model (1993/1995): FOK derives from accessible partial information, NOT correctness; cue priming raises FOK without recall gain (1995 dissociation); cue familiarity alone raises FOK (Metcalfe 1993) — **the machine analog of a retrieval score margin is accessibility, not truth** (independently re-supports the R0 family baseline). Tulving & Pearlstone 1966: availability ≠ accessibility — distinct negative states are a design requirement. Illusions of knowing: hindsight bias (Fischhoff 1975), hard-easy effect (Suantak 1996), **Juslin 2000: overconfidence is partly an item-sampling artifact — calibration must be computed on representative task distributions**, fluency illusions (Alter & Oppenheimer 2009), foresight bias (Koriat & Bjork 2005). Source monitoring (Johnson/Hashtroudi/Lindsay 1993): origin attribution is inferential and misattribution normal → provenance must be written at write time (#397 machinery; this is the cognitive justification). Measurement (Nelson 1984 gamma; Maniscalco & Lau 2012 meta-d′; Fleming & Lau 2014 RELEVANT_SECTIONS): bias ≠ sensitivity ≠ efficiency; trial-level (confidence, correctness) pairs per domain — the E3 measurement spec. Human calibration does NOT transfer across domains (McCurdy 2013; Baird 2014; Schraw 1993) → per-domain × per-task-class tables. Non-transplants: simultaneous multi-rate meta/object processing (adopt flow-direction constraints only), developmental ontogeny, phenomenology (functional role only), control-without-awareness as reason to drop monitoring (machine logging is ~free).
|
||||
Flavell 1979 (METADATA_VERIFIED): metacognitive knowledge = person/task/strategy triad + metacognitive experiences; problem solved = regulation beyond capacity. Nelson & Narens 1990/1994 (1994 chapter FULL_PAPER — the domain's anchor text): meta-level holds an **imperfect model** of the object-level; control (initiate/continue/terminate/**modify the process**) and monitoring are **logically independent** — "control per se does not yield any information from the object-level" (p.127); monitoring is imperfect-yet-useful (Fodor); judgment taxonomy EOL/JOL/FOK/retrospective; EOL/JOL/FOK uncorrelated (Leonesio & Nelson 1988) — a scalar is a lossy summary. Koriat accessibility model (1993/1995): FOK derives from accessible partial information, NOT correctness; cue priming raises FOK without recall gain (1995 dissociation); cue familiarity alone raises FOK (Metcalfe 1993) — **the machine analog of a retrieval score margin is accessibility, not truth** (independently re-supports the R0 family baseline). Tulving & Pearlstone 1966: availability ≠ accessibility — distinct negative states are a design requirement. Illusions of knowing: hindsight bias (Fischhoff 1975), hard-easy effect (Suantak 1996), **Juslin 2000: overconfidence is partly an item-sampling artifact — calibration must be computed on representative task distributions**, fluency illusions (Alter & Oppenheimer 2009), foresight bias (Koriat & Bjork 2005). Source monitoring (Johnson/Hashtroudi/Lindsay 1993): origin attribution is inferential and misattribution normal → provenance must be written at write time (#397 machinery; this is the cognitive justification). Measurement (Nelson 1984 gamma; Maniscalco & Lau 2012 meta-d′; Fleming & Lau 2014 RELEVANT_SECTIONS): bias ≠ sensitivity ≠ efficiency; trial-level (confidence, correctness) pairs per domain — the E3 measurement spec. Cited human studies show limited/inconsistent cross-domain metacognitive/calibration transfer (McCurdy 2013; Baird 2014; Schraw 1993) — NOT proof of universal non-transfer, but sufficient for the conservative rule: DO NOT ASSUME calibration transfers across domains/task classes/models/providers until measured → per-domain × per-task-class tables. Non-transplants: simultaneous multi-rate meta/object processing (adopt flow-direction constraints only), developmental ontogeny, phenomenology (functional role only), control-without-awareness as reason to drop monitoring (machine logging is ~free).
|
||||
|
||||
## 5. Monitoring ↔ control (mandate §8)
|
||||
|
||||
@@ -31,7 +31,7 @@ The separation is load-bearing exactly where the estimator must be **inspectable
|
||||
|
||||
## 6. Metamemory (mandate §9)
|
||||
|
||||
MEMORY CONTENT vs KNOWLEDGE ABOUT MEMORY is the domain's central distinction. The metamemory state object (TESTED, adopted as projection): query/intention, retrieval attempts, stores searched, coverage, hits, misses, conflicts, provenance quality, recency/staleness, applicability state, unresolved uncertainty — **rebuilt from logs, never a second Memory database** (adversarial AO: outcome logs reference runtime event IDs, rebuildable, event-sourced). The thirteen mandate questions (does Rune know a relevant memory exists? was retrieval attempted? did it return nothing? was the index down? low relevance? contradicted? applicability-rejected? cooled? canonical-unavailable-but-projection-present? provenance missing? stale? incomplete? superseded? may-be-forgetting?) each map to a typed state or an explicit UNKNOWN — matrix C. Hard build-precondition (adversarial BM SUCCEEDS): **store-layer retrieval-attempt logging** — the projection currently has no producer; it bootstraps honestly empty and never fabricates coverage.
|
||||
MEMORY CONTENT vs KNOWLEDGE ABOUT MEMORY is the domain's central distinction. The metamemory state object (TESTED, adopted as projection): query/intention, retrieval attempts, stores searched, coverage, hits, misses, conflicts, provenance quality, recency/staleness, applicability state, unresolved uncertainty — **rebuilt from logs, never a second Memory database** (adversarial AO: outcome logs reference durable retrieval-attempt/runtime event IDs and are reconstructable from them — telemetry = provenance inputs, NOT a new canonical store (canonical Memory authority per #387)). The thirteen mandate questions (does Rune know a relevant memory exists? was retrieval attempted? did it return nothing? was the index down? low relevance? contradicted? applicability-rejected? cooled? canonical-unavailable-but-projection-present? provenance missing? stale? incomplete? superseded? may-be-forgetting?) each map to a typed state or an explicit UNKNOWN — matrix C. Hard build-precondition (adversarial BM SUCCEEDS): **store-layer retrieval-attempt logging** — the projection currently has no producer; it bootstraps honestly empty and never fabricates coverage.
|
||||
|
||||
## 7. Accessibility vs availability (mandate §10)
|
||||
|
||||
@@ -39,7 +39,7 @@ Load-bearing, adopted with evidence: memory existing canonically but failing to
|
||||
|
||||
## 8. Calibration / confidence (mandate §11)
|
||||
|
||||
Carried #392: Rune MUST NOT invent precise confidence percentages. External finding (C003a, EXTERNALLY_ESTABLISHED): verbal/numeric LLM self-confidence is systematically overconfident (idiom-imitation), task-dependent, prompt-fragile; no uncertainty method dominates across datasets (Xiong ICLR'24); P(True)/P(IK) calibrate only for in-distribution confabulation-class errors and miscalibrate on new tasks (Kadavath); semantic entropy beats P(True) head-to-head (0.790 vs 0.698 AUROC, Nature 2024) but its own authors scope it to confabulation — "does not help when outputs are systematically bad"; Kalai & Vempala: calibrated LMs MUST hallucinate singleton facts — calibration is not a cure. Rune transfer (C003b, RUNE_APPLICABLE): verbal confidence = relative ranking signal ONLY until a per-decision outcome flywheel exists; numeric probability only where per-domain × per-task-class history supports it; calibration computed on representative distributions (Juslin) with anti-survivorship design (C004: log ALL decisions including abstentions/escalations; abstention quality as first-class metric; forced-evaluation sampling — EXPERIMENT). Conformal guarantees are only as good as their nonconformity scores (Semantic Illusion: 0% FPR on synthetic errors, 100% FPR at target coverage on real hallucinations with embedding/NLI scores). Verbal confidence ≠ calibrated confidence; measured calibration model ≠ LLM verbal confidence.
|
||||
Carried #392: Rune MUST NOT invent precise confidence percentages. External finding (C003a, EXTERNALLY_ESTABLISHED): verbal/numeric LLM self-confidence is systematically overconfident (idiom-imitation), task-dependent, prompt-fragile; no uncertainty method dominates across datasets (Xiong ICLR'24); P(True)/P(IK) calibrate only for in-distribution confabulation-class errors and miscalibrate on new tasks (Kadavath); semantic entropy beats P(True) head-to-head (0.790 vs 0.698 AUROC, Nature 2024) but its own authors scope it to confabulation — "does not help when outputs are systematically bad"; Kalai & Vempala: calibrated LMs MUST hallucinate singleton facts — calibration is not a cure. Rune transfer (C003b, RUNE_APPLICABLE): verbal confidence MAY serve as an ADVISORY ranking feature only where per-task/per-model validation demonstrates value — never a calibrated probability, truth authority, independent evidence, or cross-model-transferable calibration; until a per-decision outcome flywheel exists it stays out of authority entirely; numeric probability only where per-domain × per-task-class history supports it; calibration computed on representative distributions (Juslin) with anti-survivorship design (C004: log ALL decisions including abstentions/escalations; abstention quality as first-class metric; forced-evaluation sampling — EXPERIMENT). Conformal guarantees are only as good as their nonconformity scores (Semantic Illusion: 0% FPR on synthetic errors, 100% FPR at target coverage on real hallucinations with embedding/NLI scores). Verbal confidence ≠ calibrated confidence; measured calibration model ≠ LLM verbal confidence.
|
||||
|
||||
## 9. LLM self-knowledge / self-evaluation (mandate §12)
|
||||
|
||||
@@ -47,7 +47,7 @@ Self-recognition is paradigm-fragile (reliable pair-presentation, collapses sing
|
||||
|
||||
## 10. Self-correction / reflection (mandate §13)
|
||||
|
||||
**Intrinsic self-correction has no verified positive evidence.** Huang ICLR'24 (scope: reasoning tasks, NO external feedback — the qualifier is routinely dropped in secondary citations): performance degrades after self-correction. Kamoi TACL'24 survey: no prior work demonstrates successful prompted-LLM-feedback self-correction except tasks exceptionally suited to it; prior evaluations over-credited. Reflection WORKS with reliable external feedback (Reflexion: env signals + episodic buffer; CRITIC: "crucial importance of external feedback"; evaluator-optimizer gates per Anthropic docs). Design rule (C005): reflection gated on external verifiability; deterministic verifiers (tests/compiles) are control actions EXEMPT from the gate (adversarial BO — the gate must not suppress the working regime); the gate's named-expectation precondition is audited against outcomes (adversarial Q — self-authored preconditions self-exempt). "Reflect more" is NOT Rune's default answer.
|
||||
**Prompted intrinsic self-correction lacks reliable evidence of general benefit across ordinary/general tasks** (externally supported result, refined scope): Huang ICLR'24 (scope: reasoning tasks, NO external feedback — the qualifier is routinely dropped in secondary citations): performance degrades after self-correction. Kamoi TACL'24 survey: no prior work demonstrates successful prompted-LLM-feedback self-correction except tasks exceptionally suited to it; prior evaluations over-credited. Reflection WORKS with reliable external feedback (Reflexion: env signals + episodic buffer; CRITIC: "crucial importance of external feedback"; evaluator-optimizer gates per Anthropic docs). Design rule (C005): reflection gated on external verifiability; deterministic verifiers (tests/compiles) are control actions EXEMPT from the gate (adversarial BO — the gate must not suppress the working regime); the gate's named-expectation precondition is audited against outcomes (adversarial Q — self-authored preconditions self-exempt). "Reflect more" is NOT Rune's default answer.
|
||||
|
||||
## 11. Rational metareasoning (mandate §14)
|
||||
|
||||
@@ -55,7 +55,7 @@ Russell & Wefald 1991: VoC = expected decision improvement − cost. Hay UAI'12:
|
||||
|
||||
## 12. Fluid cognition — no fixed fast/slow modes (mandate §15)
|
||||
|
||||
Fixed modes REJECTED (mandate carry + Snell evidence). Single fluid controller: effort rises/falls with uncertainty, difficulty, stakes, novelty, conflict, verification need, model limits, available evidence, remaining budget, expected value of additional cognition. Escalation ladder TESTED against evidence, not assumed: deterministic answer → cheap retrieval → semantic reasoning → targeted research/tool → deeper verification → independent critique — each step requires expected gain (matrix E); minimum cognition for reliable result, not maximum available.
|
||||
Fixed user-visible fast/slow cognition modes are rejected as a RUNE DESIGN DECISION (project requirement: single fluid effort controller) — NOT an externally proven architecture. External evidence (Snell 2408.03314, tested setting): test-time-compute effectiveness depends strongly on prompt difficulty, and compute-optimal adaptive allocation materially improves efficiency vs best-of-N; this SUPPORTS but does not establish that every fixed architecture is invalid, nor does it validate Rune's exact controller or escalation ladder (E6/E4 test designs). Fluid controller: effort rises/falls with uncertainty, difficulty, stakes, novelty, conflict, verification need, model limits, available evidence, remaining budget, expected value of additional cognition. Escalation ladder TESTED against evidence, not assumed: deterministic answer → cheap retrieval → semantic reasoning → targeted research/tool → deeper verification → independent critique — each step requires expected gain (matrix E); minimum cognition for reliable result, not maximum available.
|
||||
|
||||
## 13. Anti-rumination / bounded reflection (mandate §16)
|
||||
|
||||
@@ -63,7 +63,7 @@ Invariants adopted: no new evidence + no state change → repeated reflection ha
|
||||
|
||||
## 14. Surprise / prediction error (mandate §17)
|
||||
|
||||
Surprise = KL(posterior‖prior) (Itti & Baldi); curiosity = prediction error in a learned controllable feature space (Pathak ICM — the design-level noisy-TV answer); RND novelty bonus. Rune stream: expected-vs-observed delta → verify → candidate lesson. Two-tier expectations (adversarial BR): deterministic expectations canonical (exit codes, schemas, counts); semantic expectations LLM-originated, labeled, lower authority, mandatory verification before lesson status. Distinctions held: surprise ≠ error ≠ novelty ≠ contradiction ≠ rarity. Novelty alone never triggers learning. **Expectation-independent channel added (adversarial AK SUCCEEDS): expectations are learnable, so chronic consistent failure stops generating deltas — periodic external-outcome sampling (verified outcomes, user corrections, ADAPT-2 escalation outcomes) feeds lessons without a surprise precondition.**
|
||||
Bayesian surprise is ONE formal definition — KL(posterior‖prior) (Itti & Baldi) — requiring explicit prior, posterior, and a comparable probability space; Rune does NOT claim a KL-based surprise signal by default. The default Rune mechanism is a typed expected-vs-observed discrepancy (schema mismatch, deterministic expectation violation, outcome deviation) without pretending a Bayesian quantity exists; curiosity = prediction error in a learned controllable feature space (Pathak ICM — the design-level noisy-TV answer); RND novelty bonus. Rune stream: expected-vs-observed delta → verify → candidate lesson. Two-tier expectations (adversarial BR): deterministic expectations canonical (exit codes, schemas, counts); semantic expectations LLM-originated, labeled, lower authority, mandatory verification before lesson status. Distinctions held: surprise ≠ error ≠ novelty ≠ contradiction ≠ rarity. Novelty alone never triggers learning. **Expectation-independent channel added (adversarial AK SUCCEEDS): expectations are learnable, so chronic consistent failure stops generating deltas — periodic external-outcome sampling (verified outcomes, user corrections, ADAPT-2 escalation outcomes) feeds lessons without a surprise precondition.**
|
||||
|
||||
## 15. Synthetic Consciousness seam (mandate §18)
|
||||
|
||||
@@ -71,11 +71,11 @@ Surprise = KL(posterior‖prior) (Itti & Baldi); curiosity = prediction error in
|
||||
|
||||
## 16. Progress awareness (mandate §20)
|
||||
|
||||
Nine typed statuses from task-runtime evidence: NOT_STARTED / ACTIVE / WAITING / BLOCKED / STALLED / INTERRUPTED / FAILED / COMPLETED / COMPLETION_UNVERIFIED. Critical rule carried: **inactivity ≠ completion.** Progress percentage MUST NOT be invented; alternatives adopted: completed-milestones/known-milestones, dependency-state, unresolved blockers, calibrated time ranges (where history supports). STALLED requires evidence of expected-progress deviation, per-task-class inactivity budgets (adversarial Y — raw silence false-STALLs slow-healthy tasks). AgentBoard: progress IS externally measurable; no benchmark shows agents estimating their own progress well — progress truth comes from runtime, not self-report.
|
||||
Nine typed statuses from task-runtime evidence: NOT_STARTED / ACTIVE / WAITING / BLOCKED / STALLED / INTERRUPTED / FAILED / COMPLETED / COMPLETION_UNVERIFIED. Critical rule carried: **inactivity ≠ completion.** Progress percentage MUST NOT be invented; alternatives adopted: completed-milestones/known-milestones, dependency-state, unresolved blockers, calibrated time ranges (where history supports). STALLED requires evidence of expected-progress deviation, per-task-class inactivity budgets (adversarial Y — raw silence false-STALLs slow-healthy tasks). AgentBoard demonstrates progress can be measured externally in its benchmark setting; this research pass found no direct benchmark establishing reliable agent self-estimation of progress — progress truth comes from runtime, not self-report.
|
||||
|
||||
## 17. Time estimation (mandate §21)
|
||||
|
||||
Honest estimates: "likely ~10–20 minutes" (reference-class interval from per-class history) vs falsely precise "12 minutes remaining" — the latter REJECTED. Evidence basis: model/tool latency history, task-class history, current blockers, unknown work. Planning fallacy + IT power-law overruns (Flyvbjerg — the commonly cited "frosting error" paper does not exist; corrected): expect fat tails, not mean shift. Jørgensen: assessment FORMAT changes estimate quality — milestone/dependency formats over invented percentages. Separate: elapsed time, predicted remaining time, hard deadline, wait condition. Open gap: no LLM task-duration estimation literature (E14).
|
||||
Honest estimates: "likely ~10–20 minutes" (reference-class interval from per-class history) vs falsely precise "12 minutes remaining" — the latter REJECTED. Evidence basis: model/tool latency history, task-class history, current blockers, unknown work. Planning fallacy + IT power-law overruns (Flyvbjerg — the commonly cited "frosting error" paper does not exist; corrected): expect fat tails, not mean shift. Jørgensen: assessment FORMAT changes estimate quality — milestone/dependency formats over invented percentages. Separate: elapsed time, predicted remaining time, hard deadline, wait condition. Open problem: no direct match found in the searched sources for LLM task-duration estimation (E14).
|
||||
|
||||
## 18. Capability awareness (mandate §22)
|
||||
|
||||
@@ -83,11 +83,11 @@ Spine owns structural capability truth; #400 owns awareness/interpretation/use.
|
||||
|
||||
## 19. Model / provider awareness (mandate §23)
|
||||
|
||||
Carried #399: model/provider ≠ identity. Implementation awareness adopted: current model, provider, context limits, quota state, latency, strengths/weaknesses, tool support, degraded state — affecting routing, verification depth, task decomposition, abstention, and self-report (factual implementation record), never identity or commitments. Provider outage distinguishable from Rune-incapable via typed degradation states + event-sourced error classification (n8n payload pattern); open gap: no published outage-vs-failure detector. Awareness implemented as routing/cascade policy (FrugalGPT/RouteLLM — routers transfer across swapped model pairs), not model self-knowledge.
|
||||
Carried #399: model/provider ≠ identity. Implementation awareness adopted: current model, provider, context limits, quota state, latency, strengths/weaknesses, tool support, degraded state — affecting routing, verification depth, task decomposition, abstention, and self-report (factual implementation record), never identity or commitments. Provider outage distinguishable from Rune-incapable via typed degradation states + event-based error classification (n8n payload pattern); open gap: no published outage-vs-failure detector. Awareness implemented as routing/cascade policy (FrugalGPT/RouteLLM — routers transfer across swapped model pairs), not model self-knowledge.
|
||||
|
||||
## 20. Error awareness / recovery (mandates §24–25)
|
||||
|
||||
Typed error taxonomy: explicit error, timeout, stall, transient provider fault, malformed request, tool unavailable, permission, semantic failure, verification failure, unknown outcome, side-effect-uncertain. TRANSIENT/PERSISTENT classification with per-class debounce (adversarial BP — state flapping drives thrash and pollutes the flywheel); oscillation itself exposed as a signal (EXPERIMENT). Controller decisions: retry/resume/repair/alternate-tool/alternate-model/ask/stop — without pretending every error needs deep cognition. **Error/Recovery History (mandate §25, build-precondition ★):** recurring errors become metacognitive priors — signature, context, previous successful recovery, retry count, provider/tool, recency, outcome — consulted BEFORE open-ended re-diagnosis (E9). Guards: prior recovery ≠ guaranteed current solution; decay + applicability check; **attribution-sourced priors — retry success does NOT validate the original diagnosis (adversarial AA/BU); success without confirmed causal match = superstitious prior, flagged low-authority, excluded from prescription.** Failure attribution is an external service, not self-report: best published step-level attribution is 14.2% (Who&When), best trace debugging 11% (TRAIL) — the failing agent is the wrong party to ask what went wrong.
|
||||
Typed error taxonomy: explicit error, timeout, stall, transient provider fault, malformed request, tool unavailable, permission, semantic failure, verification failure, unknown outcome, side-effect-uncertain. TRANSIENT/PERSISTENT classification with per-class debounce (adversarial BP — state flapping drives thrash and pollutes the flywheel); oscillation itself exposed as a signal (EXPERIMENT). Controller decisions: retry/resume/repair/alternate-tool/alternate-model/ask/stop — without pretending every error needs deep cognition. **Error/Recovery History (mandate §25, build-precondition ★):** recurring errors become metacognitive priors — signature, context, previous successful recovery, retry count, provider/tool, recency, outcome — consulted BEFORE open-ended re-diagnosis (E9). Guards: prior recovery ≠ guaranteed current solution; decay + applicability check; **attribution-sourced priors — retry success does NOT validate the original diagnosis (adversarial AA/BU); success without confirmed causal match = superstitious prior, flagged low-authority, excluded from prescription.** Failure attribution: Who&When reports 14.2% step-level attribution in its benchmark (53.5% agent-level); TRAIL reports 11% for its best tested model in that benchmark — these establish the task is difficult in the cited evaluations, not a global state-of-the-art ceiling. Design rule: free-form self-diagnosis alone is insufficient causal evidence; external/runtime attribution signals are preferred where available — while the agent still generates hypotheses and candidate diagnoses for later verification (semantic diagnosis is not banned).
|
||||
|
||||
## 21. Outcome awareness / Growth Delta (mandate §26)
|
||||
|
||||
@@ -107,7 +107,7 @@ Carried #397. Rune knows whether evidence is firsthand/external/user-stated/infe
|
||||
|
||||
## 25. Security seam (mandate §30)
|
||||
|
||||
#402 owns mechanism; #400 names the metacognitive attack surfaces semantically: fake uncertainty injected into Memory; fabricated progress; fake task completion; poisoned capability state; manipulated self-confidence; attacker-induced endless reflection (the governor is also a security control); self-report injection; adversarial "you already verified this" (answer: verification state is event-sourced, not narrative); fake provider/tool state; poisoned recovery history (attribution-sourcing + superstition flags are the semantic defense). GRAB-RAG's 41.6% misleading-context compliance is the budget number any sufficiency gate must beat.
|
||||
#402 owns mechanism; #400 names the metacognitive attack surfaces semantically: fake uncertainty injected into Memory; fabricated progress; fake task completion; poisoned capability state; manipulated self-confidence; attacker-induced endless reflection (the governor is also a security control); self-report injection; adversarial "you already verified this" (answer: verification state is event-recorded, not narrative); fake provider/tool state; poisoned recovery history (attribution-sourcing + superstition flags are the semantic defense). GRAB-RAG's 41.6% misleading-context compliance is the budget number any sufficiency gate must beat.
|
||||
|
||||
## 26. JEV / deterministic routing seam (mandate §31)
|
||||
|
||||
@@ -119,7 +119,7 @@ Audited, not inferred: ADAPT-2 IS a primitive metacognitive control system — e
|
||||
|
||||
## 28. Current self-state object (mandates §34–35)
|
||||
|
||||
Typed CURRENT SELF-STATE projection: task_state, goal_conflict, uncertainty_type, evidence_sufficiency, retrieval_state, contradiction_state, progress_state, error_state, resource_pressure, provider_state, capability_gap, surprise_state, repeated_failure_state, verification_need. Derived, ephemeral, reconstructable — NOT canonical autobiographical truth, NOT a giant global consciousness object (adversarial AO/AP: passes with event-sourcing fix). Transitions: event-driven, evidence-triggered, threshold-triggered — never periodic LLM self-reflection (inward: the periodic loop is empirically dead — think_count 1). Hysteresis/debouncing prevents oscillation; oscillation itself is a signal.
|
||||
Typed CURRENT SELF-STATE projection: task_state, goal_conflict, uncertainty_type, evidence_sufficiency, retrieval_state, contradiction_state, progress_state, error_state, resource_pressure, provider_state, capability_gap, surprise_state, repeated_failure_state, verification_need. Derived, ephemeral, reconstructable — NOT canonical autobiographical truth, NOT a giant global consciousness object (adversarial AO/AP: passes with event-sourcing fix). Transitions: event-driven, evidence-triggered, threshold-triggered — never periodic LLM self-reflection (inward: the periodic loop is empirically dead — think_count 1). Hysteresis/debouncing prevents oscillation; oscillation itself is a signal. Seam invariant (repair §8): event-driven monitoring is PRIMARY; the periodic reconciler is a BACKSTOP integrity sweep — low-frequency, deterministic, bounded, non-LLM, allowed ONLY for conditions that may generate no event (silent index corruption, calibration drift, stale projections, missed writes, slow degradation). **TIMER EXPIRY ALONE MUST NOT TRIGGER LLM COGNITION** — a timer may trigger deterministic probes/reconciliation only; semantic cognition begins only if a probe discovers an anomaly, ambiguity, meaningful state change, or a policy-defined escalation condition; the same rule applies to periodic external-outcome sampling. Always-available engine, not always-running LLM.
|
||||
|
||||
## 29. Metacognitive authority (mandate §36)
|
||||
|
||||
@@ -160,12 +160,12 @@ claims.json: 16 claims (12 ADOPT, 4 ADAPT), each with evidence_for/against, limi
|
||||
## 38. Remaining unknowns (mandate: explicit section)
|
||||
|
||||
Typed: unresolved research question / experiment-required / cross-domain-owned / inaccessible source gap.
|
||||
- NO_MATCH/INDEX_DOWN/ABSENCE taxonomy — literature-confirmed open gap; #400's typed states are the first mover; E1/E2 validate.
|
||||
- Outcome-derived memory confidence — no published system; anti-survivorship design (C004) is RUNE_HYPOTHESIS; E3 + forced-evaluation variant.
|
||||
- VoC operationalized inside a running LLM agent — unpublished; cheap proxies adopted; E6.
|
||||
- LLM task-duration estimation — no literature; reference-class intervals adopted; E14.
|
||||
- Provider-outage vs agent-failure detection — no published detector; event-sourced classification adopted; E8 variant.
|
||||
- Recovery-prior systems — no published system; attribution-sourcing depends on attribution machinery that is ABSENT (inward G) — build-precondition.
|
||||
- NO_MATCH/INDEX_DOWN/ABSENSE-CLASS typed taxonomy — this R1 search found no directly equivalent published mechanism within the searched sources as of 2026-09-21 (adjacent failure/retrieval taxonomies exist; NOVELTY_UNPROVEN); E1/E2 validate.
|
||||
- Outcome-derived memory confidence — no direct match found in the searches performed (as of 2026-09-21); anti-survivorship design (C004) is RUNE_HYPOTHESIS; E3 + forced-evaluation variant.
|
||||
- VoC operationalized inside a running LLM agent — no direct match found in the searches performed; cheap proxies adopted; E6.
|
||||
- LLM task-duration estimation — no direct match found in the searches performed; reference-class intervals adopted; E14.
|
||||
- Provider-outage vs agent-failure detection — no direct match found in the searches performed; event-based classification adopted; E8 variant.
|
||||
- Recovery-prior systems — no direct match found in the searches performed; attribution-sourcing depends on attribution machinery that is ABSENT (inward G) — build-precondition.
|
||||
- Per-decision-class calibration for agents — nascent (TRUST/PASC only); class boundaries need defining (adversarial BF).
|
||||
- Semantic surprise coverage — weak under the two-tier model (adversarial BR residual).
|
||||
- Hysteresis parameters — per-error-class debounce windows uncalibrated (E5/E6).
|
||||
@@ -173,12 +173,12 @@ Typed: unresolved research question / experiment-required / cross-domain-owned /
|
||||
- Milestone taxonomy per task class — unspec'd engineering (row 37 residual).
|
||||
- Cap values for metalevel termination — specified per-class at implementation, not here (adversarial S residual).
|
||||
- Cross-model calibration transfer — never demonstrated either way; E13 designed.
|
||||
- UQ-for-agents survey — does not exist yet; TRUST/PASC are the nearest anchors (snapshot 2026-09-21).
|
||||
- UQ-for-agents survey — no canonical survey was found in the searches performed (as of 2026-09-21); TRUST/PASC are the nearest anchors (snapshot 2026-09-21).
|
||||
- Philosophy canon depth — Flavell 1979, N&N 1990 chapter paywalled (METADATA_VERIFIED with secondary content; N&N 1994 FULL_PAPER is the anchor); honest gap, not fabricated depth.
|
||||
|
||||
## 39. Evidence saturation (mandate §49)
|
||||
|
||||
evidence_saturation.md: **SATURATION = REACHED** (finalized post-arms). 26 branches opened, all saturated; 8 typed late-search checks produced zero new architecture-changing mechanisms — the zero-hits are themselves recorded findings (typed absence taxonomy unpublished; outcome-derived confidence unpublished; no UQ-for-agents survey; no LLM duration-estimation work; no outage-vs-failure detector; no recovery-prior systems; no in-agent VoC operationalization; "simplify-then-answer 2026" unverifiable rumor rejected). Residual gaps explicit: ABS-depth dominance on arXiv sources (1 FULL_PAPER + 2 RELEVANT_SECTIONS + 4 DOCUMENTATION_AUDITED anchors), Semantic Scholar 429/DBLP bot-wall workarounds documented, fast-moving 2026 conformal/agent-UQ line snapshotted.
|
||||
evidence_saturation.md: **SEARCH_SATURATION = REACHED** (finalized post-arms — late searches stopped producing new architecture-changing mechanisms) and, post-repair-top-up, **LOAD_BEARING_DEPTH = PASS** (§40 + load_bearing_evidence.md) — separate qualifications; search saturation does NOT imply all load-bearing conclusions were deeply read. 26 branches opened, all saturated; 8 typed late-search checks produced zero new architecture-changing mechanisms — the zero-hits are recorded as bounded-search findings (no direct match found in the searches performed; NOVELTY_UNPROVEN), not proofs of nonexistence (typed absence taxonomy unpublished; outcome-derived confidence unpublished; no UQ-for-agents survey; no LLM duration-estimation work; no outage-vs-failure detector; no recovery-prior systems; no in-agent VoC operationalization; "simplify-then-answer 2026" unverifiable rumor rejected). Residual gaps explicit: ABS-depth dominance on arXiv sources (1 FULL_PAPER + 2 RELEVANT_SECTIONS + 4 DOCUMENTATION_AUDITED anchors), Semantic Scholar 429/DBLP bot-wall workarounds documented, fast-moving 2026 conformal/agent-UQ line snapshotted.
|
||||
|
||||
## 40. Source-depth summary (mandate §48)
|
||||
|
||||
@@ -202,6 +202,9 @@ sources.md row truth (machine-derived by scripts/research_gate_check.py::parse_s
|
||||
- r0_reconciliation.md / preliminary_reconciliation.md — row-derived verdicts
|
||||
- evidence_saturation.md — 26 branches, SATURATION=REACHED, 8 zero-hit checks recorded
|
||||
- reconciliation_arithmetic_check.txt — machine-derived totals
|
||||
- process_evidence.md — authoritative continue-count derivation from the raw trajectory DB (mandate §55/§18)
|
||||
- load_bearing_evidence.md — per-claim support map (repair §15: deep-external / Rune-direct / design-invariant / downgraded)
|
||||
- load_bearing_evidence.md — per-claim support map (repair §15: deep-external / Rune-direct / design-invariant / downgraded)
|
||||
- Y_METACOGNITION_R1_BATCHED_PRELIMINARY.md — preserved verbatim (never patched)
|
||||
- Y_METACOGNITION_METAMEMORY_R1_MASTER.md — this master
|
||||
- mutation_audit.md — CONTENT-FREEZE audit (written last)
|
||||
|
||||
@@ -1,10 +1,17 @@
|
||||
[
|
||||
{
|
||||
"id": "Y-R1-C001",
|
||||
"claim": "Rune's metacognition is primitive-but-real: ADAPT-2 evidence routing (escalate-on-UNKNOWN, non-downgradable floor, append-only trail) and score-margin abstention are live production behavior, while evidence-derived typed metacognitive state is absent — five load-bearing gaps (silent abstention, assignation-only confidence, unconsumed Spine truth, absent recovery history, dormant cognitive loop).",
|
||||
"evidence_for": ["01_inward: adaptive_router wiring canaries assert production dispatch + persisted trail", "memory_store.recall RECALL_MIN_SCORE=0.42 abstention branches live", "cognitive_state.json think_count=1 (Aug-24 fossil)", "Spine/rune_model ledgers fresh+populated with zero metacognitive consumers"],
|
||||
"evidence_against": ["point-in-time audit; live-turn wiring individually spot-checked not exhaustively"],
|
||||
"limitations": "Read-only audit; source existence checked against runtime wiring per depth-audit discipline.",
|
||||
"claim": "OBSERVED INWARD FACTS (repair-narrowed scope): memory_store.recall() has a live score-margin abstention floor (RECALL_MIN_SCORE=0.42) that silently returns [] (retrieval failure indistinguishable from absence); confidence values on 941 entries are assigned at write time (652 @0.9 via curator/dedupe policy) with no observed outcome-derived updater; Spine authority-generation and rune_model capability ledgers exist, are fresh and populated, with no observed metacognitive consumer; no Error/Recovery History store observed (ABSENT); CognitiveLoop has no observed production caller (state stale since Aug-24); ADAPT-2 is production-wired with canary-covered routing for work-tier/evidence-sufficiency.",
|
||||
"evidence_for": [
|
||||
"01_inward: adaptive_router wiring canaries assert production dispatch + persisted trail",
|
||||
"memory_store.recall RECALL_MIN_SCORE=0.42 abstention branches live",
|
||||
"cognitive_state.json think_count=1 (Aug-24 fossil)",
|
||||
"Spine/rune_model ledgers fresh+populated with zero metacognitive consumers"
|
||||
],
|
||||
"evidence_against": [
|
||||
"point-in-time audit; live-turn wiring individually spot-checked not exhaustively"
|
||||
],
|
||||
"limitations": "Point-in-time read-only audit; claim scoped to component presence/wiring/observed runtime facts — no architecture-performance assertion (synthesis interpretation lives in the master, RUNE_APPLICABLE).",
|
||||
"rune_evidence": "01_inward classification: CANONICAL 4 · ACTIVE 6 · DORMANT 3 · PARTIAL 4 · ADVISORY 2 · ABSENT 3 · UNKNOWN 1",
|
||||
"transfer_assumptions": "single-user single-node runtime",
|
||||
"classification": "ADOPT",
|
||||
@@ -18,8 +25,15 @@
|
||||
{
|
||||
"id": "Y-R1-C002",
|
||||
"claim": "Monitoring/control separation is the right metacognitive architecture: typed monitoring estimates state from deterministic evidence; control acts on estimates; logical independence preserved while Koriat-2006 coupling is acknowledged (control produces the fluency monitoring reads).",
|
||||
"evidence_for": ["Nelson & Narens 1990/1994 (A1-3 FULL_PAPER): control per se yields no information — monitoring logically independent", "Leonesio & Nelson: judgment types uncorrelated — scalar lossy", "inward: ADAPT-2 instantiates the control half successfully"],
|
||||
"evidence_against": ["Koriat 2006 (A1-8): bidirectional coupling — one-way separation empirically false in humans", "Reder & Schunn (A1-26): control adapts without explicit monitoring"],
|
||||
"evidence_for": [
|
||||
"Nelson & Narens 1990/1994 (A1-3 FULL_PAPER): control per se yields no information — monitoring logically independent",
|
||||
"Leonesio & Nelson: judgment types uncorrelated — scalar lossy",
|
||||
"inward: ADAPT-2 instantiates the control half successfully"
|
||||
],
|
||||
"evidence_against": [
|
||||
"Koriat 2006 (A1-8): bidirectional coupling — one-way separation empirically false in humans",
|
||||
"Reder & Schunn (A1-26): control adapts without explicit monitoring"
|
||||
],
|
||||
"limitations": "Human framework; software re-derivation needed for continuous confidence signals.",
|
||||
"rune_evidence": "01_inward E: ADAPT-2 route lifecycle works; no monitoring half exists",
|
||||
"transfer_assumptions": "monitoring estimators tolerable when imperfect (Fodor via N&N)",
|
||||
@@ -34,8 +48,15 @@
|
||||
{
|
||||
"id": "Y-R1-C003a",
|
||||
"claim": "EXTERNAL FINDING: LLM verbal/numeric self-confidence is systematically overconfident (idiom-imitation), task-dependent, prompt-fragile; no uncertainty method dominates across datasets; P(True)/P(IK) calibrate only for in-distribution confabulation-class errors and miscalibrate on new tasks.",
|
||||
"evidence_for": ["Xiong ICLR'24 (A2-2): systematic overconfidence, no dominating method", "Tian EMNLP'23 (A2-1): relatively better than logprobs, absolutely overconfident", "Kadavath (A2-7): P(IK) miscalibrates on new tasks", "Farquhar Nature'24 (A2-4): P(True) AUROC 0.698 < semantic entropy 0.790"],
|
||||
"evidence_against": ["verbal confidence improves with scale and few-shot confidence examples"],
|
||||
"evidence_for": [
|
||||
"Xiong ICLR'24 (A2-2): systematic overconfidence, no dominating method",
|
||||
"Tian EMNLP'23 (A2-1): relatively better than logprobs, absolutely overconfident",
|
||||
"Kadavath (A2-7): P(IK) miscalibrates on new tasks",
|
||||
"Farquhar Nature'24 (A2-4): P(True) AUROC 0.698 < semantic entropy 0.790"
|
||||
],
|
||||
"evidence_against": [
|
||||
"verbal confidence improves with scale and few-shot confidence examples"
|
||||
],
|
||||
"limitations": "2023-24 model generations; ABS-depth reads except Farquhar (main text).",
|
||||
"rune_evidence": "n/a (external finding; Rune-side transfer is C003b)",
|
||||
"transfer_assumptions": "none for the external statement itself",
|
||||
@@ -49,9 +70,15 @@
|
||||
},
|
||||
{
|
||||
"id": "Y-R1-C003b",
|
||||
"claim": "RUNE TRANSFER: Rune treats verbal/numeric self-confidence as a relative ranking signal ONLY until a per-decision outcome flywheel exists; numeric probabilities appear only where per-domain × per-task-class verified-outcome history supports calibration; no invented percentages (carries #392).",
|
||||
"evidence_for": ["C003a external finding transfers as hard constraint", "arm-2 gap: no memory system derives confidence from verified outcomes (first-mover)", "inward B: 652 entries @0.9 are curator assignations — the anti-pattern live"],
|
||||
"evidence_against": ["typed states alone may suffice without any numeric layer"],
|
||||
"claim": "RUNE TRANSFER: Rune may use verbal confidence only as an ADVISORY ranking feature where per-task/per-model validation demonstrates value; it is never a calibrated probability, truth authority, independent evidence, or cross-model-transferable calibration; until a per-decision outcome flywheel exists it carries no authority; numeric probabilities appear only where per-domain × per-task-class verified-outcome history supports calibration; no invented percentages (carries #392).",
|
||||
"evidence_for": [
|
||||
"C003a external finding transfers as hard constraint",
|
||||
"arm-2 gap: no memory system derives confidence from verified outcomes (first-mover)",
|
||||
"inward B: 652 entries @0.9 are curator assignations — the anti-pattern live"
|
||||
],
|
||||
"evidence_against": [
|
||||
"typed states alone may suffice without any numeric layer"
|
||||
],
|
||||
"limitations": "flywheel design is novel (C004); cold-start classes have no data.",
|
||||
"rune_evidence": "01_inward B: confidence fields populated, never outcome-derived",
|
||||
"transfer_assumptions": "per-class outcome accumulation is feasible in a single-user agent",
|
||||
@@ -66,9 +93,15 @@
|
||||
{
|
||||
"id": "Y-R1-C004",
|
||||
"claim": "The outcome flywheel must be anti-survivorship by design: log predictions on ALL decisions including abstentions/escalations; evaluate abstention quality as a first-class metric; periodically force-evaluate a random sample of would-have-been-abstained cases; stratified error rates only (task-mix confound).",
|
||||
"evidence_for": ["adversarial BN SUCCEEDS: flywheel trains on its own survivor decisions — systematically optimistic", "adversarial BI: task-mix confound on error-rate deltas", "Juslin representative-design (A1-16): unrepresentative sampling corrupts calibration"],
|
||||
"evidence_against": ["forced evaluation has real cost; sample size grows slowly"],
|
||||
"limitations": "no published system does this (novel); cost/benefit unmeasured.",
|
||||
"evidence_for": [
|
||||
"adversarial BN SUCCEEDS: flywheel trains on its own survivor decisions — systematically optimistic",
|
||||
"adversarial BI: task-mix confound on error-rate deltas",
|
||||
"Juslin representative-design (A1-16): unrepresentative sampling corrupts calibration"
|
||||
],
|
||||
"evidence_against": [
|
||||
"forced evaluation has real cost; sample size grows slowly"
|
||||
],
|
||||
"limitations": "no direct match found in the searches performed (NOVELTY_UNPROVEN); cost/benefit unmeasured.",
|
||||
"rune_evidence": "inward: no outcome logs exist — design space open",
|
||||
"transfer_assumptions": "forced-evaluation cost is tolerable at sample rates",
|
||||
"classification": "ADAPT",
|
||||
@@ -81,9 +114,15 @@
|
||||
},
|
||||
{
|
||||
"id": "Y-R1-C005",
|
||||
"claim": "Reflection is gated on external verifiability: intrinsic self-correction loops are disabled/capped unless reliable external feedback (tools/tests/verifiers) exists; deterministic verifiers are exempt from the gate (they are control actions, not reflection).",
|
||||
"evidence_for": ["Huang ICLR'24 (A3-4): intrinsic self-correction degrades reasoning — scope: no external feedback", "Kamoi TACL'24 (A3-5): no demonstrated prompted-feedback self-correction except suited tasks; works with reliable external feedback", "CRITIC (A3-3): 'crucial importance of external feedback'", "Reflexion (A3-1): environmental feedback + episodic buffer"],
|
||||
"evidence_against": ["Self-Refine shows gains on text-quality tasks (preference-based)"],
|
||||
"claim": "Reflection regimes (repair-precise): prompted intrinsic self-correction lacks reliable evidence of general benefit across ordinary tasks (Huang v2: GPT-4 GSM8K 95.5→89.0 etc., scope = no external feedback, no skilled-few-shot condition tested; Kamoi: positives came from oracle labels or weak-initial-prompt artifacts; carve-out = tasks where verification ≪ generation, e.g. decomposable responses). Rune rule: (A) intrinsic reflection without new evidence may generate hypotheses/alternative formulations/flaw-identification but never increases evidentiary independence, truth authority, or calibrated confidence; (B) correctness-strengthening reflection requires external/deterministic verification, independent evidence, or later outcome validation; (C) deterministic verifiers (tests/compiles/schemas/exact calculations) remain valid control mechanisms, EXEMPT from the reflection gate.",
|
||||
"evidence_for": [
|
||||
"Huang ICLR'24 v2 FULL_PAPER (A3-4): intrinsic degradation across 4 models × 3 benchmarks; three alternative feedback prompts also degrade; multi-agent debate ≤ self-consistency at matched cost",
|
||||
"Kamoi TACL'24 v3 FULL_PAPER (A3-5): feedback-source taxonomy; exceptionally-suitable carve-out (verification ≪ generation); RCI/Reflexion oracle positives; Self-Refine weak-prompt artifacts; strong-baseline problem",
|
||||
"CRITIC (A3-3): 'crucial importance of external feedback'; Reflexion (A3-1): env signals + episodic buffer"
|
||||
],
|
||||
"evidence_against": [
|
||||
"Self-Refine shows gains on text-quality tasks (preference-based)"
|
||||
],
|
||||
"limitations": "drift/overfit mechanisms implied rather than independently measured.",
|
||||
"rune_evidence": "inward A: the one purpose-built reflection loop is DORMANT — periodic reflection empirically dead here",
|
||||
"transfer_assumptions": "external verifier availability varies by task class",
|
||||
@@ -98,8 +137,15 @@
|
||||
{
|
||||
"id": "Y-R1-C006",
|
||||
"claim": "Anti-rumination governor + difficulty-adaptive compute budgets: no-new-evidence + no-state-change → reflection authority decays → stop/escalate; additional cognition requires expected gain (VoC); budgets scale with difficulty proxies (ephemeral, deterministic, evidence-cited); caps are mandatory (optimal metalevel policy need not terminate).",
|
||||
"evidence_for": ["Hay UAI'12 (A3-12): non-termination counterexample", "Snell (A3-7): difficulty-adaptive allocation 4× efficiency", "A3-8/9/10: overthinking/underthinking duality with efficiency metrics", "Russell & Wefald (A3-11): VoC foundation"],
|
||||
"evidence_against": ["cap values unspecified; cold-start difficulty classes have no proxies"],
|
||||
"evidence_for": [
|
||||
"Hay UAI'12 (A3-12): non-termination counterexample",
|
||||
"Snell (A3-7): difficulty-adaptive allocation 4× efficiency",
|
||||
"A3-8/9/10: overthinking/underthinking duality with efficiency metrics",
|
||||
"Russell & Wefald (A3-11): VoC foundation"
|
||||
],
|
||||
"evidence_against": [
|
||||
"cap values unspecified; cold-start difficulty classes have no proxies"
|
||||
],
|
||||
"limitations": "no 2024-26 operationalization of VoC inside a running LLM agent (gap = finding).",
|
||||
"rune_evidence": "inward E: ADAPT-2 budgets already cap work tiers — pattern proven in production",
|
||||
"transfer_assumptions": "difficulty proxies (prior outcomes per class, risk flags) correlate with true difficulty",
|
||||
@@ -114,8 +160,14 @@
|
||||
{
|
||||
"id": "Y-R1-C007",
|
||||
"claim": "Deviation-triggered monitoring requires a baseline reconciler channel: periodic deterministic invariant audits (projection-vs-Memory counts, Spine freshness, positive-probe retrieval health, external-outcome sampling) — non-LLM, so event-driven-over-timer is preserved; without it the system is deaf to slow degradation, chronic consistent error, and silent index corruption.",
|
||||
"evidence_for": ["adversarial AK/AY/BS (3 SUCCEEDS, one root): consistent failure generates no deviations", "adversarial BS: index drift reads NO_MATCH when truth is INDEX_PARTIAL", "inward D: no retrieval-attempt log or health probes exist"],
|
||||
"evidence_against": ["audit coverage never complete; probe cost per store"],
|
||||
"evidence_for": [
|
||||
"adversarial AK/AY/BS (3 SUCCEEDS, one root): consistent failure generates no deviations",
|
||||
"adversarial BS: index drift reads NO_MATCH when truth is INDEX_PARTIAL",
|
||||
"inward D: no retrieval-attempt log or health probes exist"
|
||||
],
|
||||
"evidence_against": [
|
||||
"audit coverage never complete; probe cost per store"
|
||||
],
|
||||
"limitations": "novel synthesis (no direct literature); per-store probe engineering.",
|
||||
"rune_evidence": "inward B2/D: silent abstention live; no health checks anywhere",
|
||||
"transfer_assumptions": "positive-probe queries can be maintained per store",
|
||||
@@ -130,9 +182,16 @@
|
||||
{
|
||||
"id": "Y-R1-C008",
|
||||
"claim": "Surprise is a two-tier stream: expected-vs-observed deltas with deterministic expectations canonical and LLM-originated expectations labeled lower-authority requiring verification before lesson-candidate status; novelty alone never triggers learning; an expectation-independent channel (external-outcome sampling) catches chronic consistent error.",
|
||||
"evidence_for": ["Itti & Baldi (A3-15): surprise = KL(posterior‖prior)", "Pathak ICM (A3-16): prediction error in controllable feature space — noisy-TV answer", "adversarial BR: self-authored expectations launder self-report back in", "adversarial AK SUCCEEDS: chronic error goes silent without the independent channel"],
|
||||
"evidence_against": ["semantic surprise coverage remains weak under the two-tier model"],
|
||||
"limitations": "agent-side expected-vs-observed operationalization unpublished (open gap).",
|
||||
"evidence_for": [
|
||||
"Itti & Baldi (A3-15): Bayesian surprise = KL(posterior‖prior) is ONE formal definition — requires explicit prior/posterior/comparable probability space; Rune's default is a typed expected-vs-observed discrepancy (schema mismatch, deterministic expectation violation, outcome deviation) without claiming a Bayesian quantity",
|
||||
"Pathak ICM (A3-16): prediction error in controllable feature space — noisy-TV answer",
|
||||
"adversarial BR: self-authored expectations launder self-report back in",
|
||||
"adversarial AK SUCCEEDS: chronic error goes silent without the independent channel"
|
||||
],
|
||||
"evidence_against": [
|
||||
"semantic surprise coverage remains weak under the two-tier model"
|
||||
],
|
||||
"limitations": "agent-side expected-vs-observed operationalization: no direct match found in the searches performed (open problem).",
|
||||
"rune_evidence": "inward: no expectation store; Growth Delta loop research-only",
|
||||
"transfer_assumptions": "deterministic expectations cover the load-bearing fraction of deltas",
|
||||
"classification": "ADOPT",
|
||||
@@ -146,9 +205,16 @@
|
||||
{
|
||||
"id": "Y-R1-C009",
|
||||
"claim": "Capability/provider awareness derives from runtime truth (Spine authority generations, rune_model ledgers, provider state) with freshness bounds and live probes near expiry — never parametric self-knowledge; provider outage is distinguishable from Rune incapability; awareness is implemented as routing/cascade policy.",
|
||||
"evidence_for": ["inward J/K: Spine + rune_model ledgers fresh, populated, UNCONSUMED", "A2-31/32/33: parametric self-knowledge paradigm-fragile; refusal-as-skill generalizes, identity-recognition doesn't", "A3-26/27: routing/cascade = the working awareness form", "adversarial BQ: staleness at decision time — freshness bounds required"],
|
||||
"evidence_against": ["probe cost; introspection surfaces need building (inward I PARTIAL)"],
|
||||
"limitations": "provider-outage-vs-agent-failure detection unpublished (open gap).",
|
||||
"evidence_for": [
|
||||
"inward J/K: Spine + rune_model ledgers fresh, populated, UNCONSUMED",
|
||||
"A2-31/32/33: parametric self-knowledge paradigm-fragile; refusal-as-skill generalizes, identity-recognition doesn't",
|
||||
"A3-26/27: routing/cascade = the working awareness form",
|
||||
"adversarial BQ: staleness at decision time — freshness bounds required"
|
||||
],
|
||||
"evidence_against": [
|
||||
"probe cost; introspection surfaces need building (inward I PARTIAL)"
|
||||
],
|
||||
"limitations": "provider-outage-vs-agent-failure detection: no direct match found in the searched sources (open problem).",
|
||||
"rune_evidence": "01_inward J/K: truth exists, zero consumers",
|
||||
"transfer_assumptions": "Spine freshness timestamps are maintained by runtime",
|
||||
"classification": "ADOPT",
|
||||
@@ -162,9 +228,16 @@
|
||||
{
|
||||
"id": "Y-R1-C010",
|
||||
"claim": "Progress truth: statuses derive from task-runtime evidence into typed states (NOT_STARTED/ACTIVE/WAITING/BLOCKED/STALLED/INTERRUPTED/FAILED/COMPLETED/COMPLETION_UNVERIFIED); inactivity ≠ completion; STALLED requires expected-progress deviation, not raw silence; no invented percentages; milestone/dependency formats over estimates; fat-tailed overrun expectations.",
|
||||
"evidence_for": ["A3-29 AgentBoard: progress externally measurable; prior evaluation process-blind", "A3-18/19/20: planning fallacy + IT power-law overruns", "A3-21 Jørgensen: format changes estimate quality", "adversarial X/Y: passes with repairs applied"],
|
||||
"evidence_against": ["milestone taxonomy per task class is unspec'd engineering"],
|
||||
"limitations": "no LLM task-duration estimation literature (open gap → E14).",
|
||||
"evidence_for": [
|
||||
"A3-29 AgentBoard: progress externally measurable; prior evaluation process-blind",
|
||||
"A3-18/19/20: planning fallacy + IT power-law overruns",
|
||||
"A3-21 Jørgensen: format changes estimate quality",
|
||||
"adversarial X/Y: passes with repairs applied"
|
||||
],
|
||||
"evidence_against": [
|
||||
"milestone taxonomy per task class is unspec'd engineering"
|
||||
],
|
||||
"limitations": "no direct match found in the searched sources for LLM task-duration estimation (open problem); reference-class intervals adopted.",
|
||||
"rune_evidence": "inward F/H: task truth exists; user-visible status evidence-backing UNKNOWN",
|
||||
"transfer_assumptions": "task runtime exposes the required event granularity (#394)",
|
||||
"classification": "ADOPT",
|
||||
@@ -178,9 +251,16 @@
|
||||
{
|
||||
"id": "Y-R1-C011",
|
||||
"claim": "Error/Recovery History (build-precondition): typed error signatures with context, recovery attempted, outcome, recency — consulted BEFORE open-ended re-diagnosis; priors are attribution-sourced and superstition-flagged (retry success ≠ diagnosis validation; success without confirmed causal match = low-authority, excluded from prescription); TRANSIENT/PERSISTENT taxonomy with per-class debounce; oscillation exposed as a signal.",
|
||||
"evidence_for": ["inward G: store ABSENT — re-diagnosis from scratch is live behavior", "A3-24 AgentDebug: corrective feedback → up to 26% relative gains (nearest evidence)", "A3-22/23: self-attribution 14.2% step-level, trace debugging 11% — attribution is external service", "adversarial AA/BU/BP: superstition + flapping repairs"],
|
||||
"evidence_against": ["no published recovery-prior system (open gap); depends on attribution machinery that doesn't exist"],
|
||||
"limitations": "design is novel; misclassifying persistent as transient delays escalation.",
|
||||
"evidence_for": [
|
||||
"inward G: store ABSENT — re-diagnosis from scratch is live behavior",
|
||||
"A3-24 AgentDebug: corrective feedback → up to 26% relative gains (nearest evidence)",
|
||||
"A3-22/23: self-attribution 14.2% step-level, trace debugging 11% — external/runtime attribution preferred where available; self-diagnosis alone insufficient causal evidence",
|
||||
"adversarial AA/BU/BP: superstition + flapping repairs"
|
||||
],
|
||||
"evidence_against": [
|
||||
"no published recovery-prior system (open gap); depends on attribution machinery that doesn't exist"
|
||||
],
|
||||
"limitations": "design is novel within searched sources (NOVELTY_UNPROVEN); misclassifying persistent as transient delays escalation.",
|
||||
"rune_evidence": "01_inward G: ABSENT; repeated errors re-diagnosed from scratch",
|
||||
"transfer_assumptions": "error signatures are stable enough to match across occurrences",
|
||||
"classification": "ADAPT",
|
||||
@@ -194,8 +274,15 @@
|
||||
{
|
||||
"id": "Y-R1-C012",
|
||||
"claim": "Self-report grounding + authority boundary: generated explanations are NOT causal records; hidden CoT is NOT canonical provenance; self-critique is NOT independent evidence; metacognitive state may inform/trigger (retrieval, verification, routing, clarification, strategy suggestions) but may NEVER mutate Memory truth, user model, persona/identity, or task completion truth — and calibrated confidence NEVER authorizes irreversible actions (gates suggestions/routing/abstention only).",
|
||||
"evidence_for": ["Nisbett & Wilson 1977 via N&N 1994 (A1-3 FULL_PAPER): introspection is inference", "A2-32/33: self-evaluation bias, paradigm-fragile self-recognition", "A3-5: self-critique without external feedback unreliable", "adversarial AS/K/L/M/Q: passes with AS clause added"],
|
||||
"evidence_against": ["none serious — anti-amplification backbone"],
|
||||
"evidence_for": [
|
||||
"Nisbett & Wilson 1977 via N&N 1994 (A1-3 FULL_PAPER): introspection is inference",
|
||||
"A2-32/33: self-evaluation bias, paradigm-fragile self-recognition",
|
||||
"A3-5: self-critique without external feedback unreliable",
|
||||
"adversarial AS/K/L/M/Q: passes with AS clause added"
|
||||
],
|
||||
"evidence_against": [
|
||||
"none serious — anti-amplification backbone"
|
||||
],
|
||||
"limitations": "evidence-class tagging keeps the interpretation layer honest (adversarial A).",
|
||||
"rune_evidence": "inward B2: abstention already silent-[] — the boundary must be explicit before machinery grows",
|
||||
"transfer_assumptions": "#397 provenance metadata available for independence checks",
|
||||
@@ -210,8 +297,15 @@
|
||||
{
|
||||
"id": "Y-R1-C013",
|
||||
"claim": "Event-driven metacognition with deterministic-first escalation (#393 JEV): typed transitions fire on evidence/threshold events with hysteresis; LLM cognition only on meaningful state change or semantic ambiguity; single routing authority — #400 PROPOSES input-surface extensions (self-state, progress, surprise, capability), ADAPT-2 DISPOSES.",
|
||||
"evidence_for": ["inward A: periodic reflection DORMANT (think_count 1) — the alternative is empirically dead here", "inward E: ADAPT-2 works and owns routing", "A2-25 SeaKR: internal-state triggers = best mechanistic match", "adversarial AX/BD/BE/BT: passes with propose/dispose precedence"],
|
||||
"evidence_against": ["slow degradation invisible without reconcilers (C007); proposal-channel latency"],
|
||||
"evidence_for": [
|
||||
"inward A: periodic reflection DORMANT (think_count 1) — the alternative is empirically dead here",
|
||||
"inward E: ADAPT-2 works and owns routing",
|
||||
"A2-25 SeaKR: internal-state triggers = best mechanistic match",
|
||||
"adversarial AX/BD/BE/BT: passes with propose/dispose precedence"
|
||||
],
|
||||
"evidence_against": [
|
||||
"slow degradation invisible without reconcilers (C007); proposal-channel latency"
|
||||
],
|
||||
"limitations": "hysteresis parameters need calibration per error class.",
|
||||
"rune_evidence": "01_inward A/E: dormant loop vs healthy router — the asymmetry decides the architecture",
|
||||
"transfer_assumptions": "meaningful state changes are detectable as events",
|
||||
@@ -226,8 +320,15 @@
|
||||
{
|
||||
"id": "Y-R1-C014",
|
||||
"claim": "Metacognitive memory is normal outcome/experience records in Memory — NO separate permanent metacognitive database; outcome logs reference runtime event IDs and are rebuildable from them (event-sourced); agentive phrasing ('I was uncertain') is never stored as narrative — records are field-cited evidence, narrative always regenerated.",
|
||||
"evidence_for": ["mandate §38 (strongly challenge new storage)", "adversarial AO: outcome logs as second truth surface → event-sourcing fix", "adversarial BL: agentive phrasing accretes into ungrounded narrative (#399 seam)", "#396: projections rebuildable, never canon"],
|
||||
"evidence_against": ["regeneration cost on rebuild"],
|
||||
"evidence_for": [
|
||||
"mandate §38 (strongly challenge new storage)",
|
||||
"adversarial AO: outcome logs as second truth surface → event-sourcing fix",
|
||||
"adversarial BL: agentive phrasing accretes into ungrounded narrative (#399 seam)",
|
||||
"#396: projections rebuildable, never canon"
|
||||
],
|
||||
"evidence_against": [
|
||||
"regeneration cost on rebuild"
|
||||
],
|
||||
"limitations": "runtime event-log completeness is the dependency.",
|
||||
"rune_evidence": "inward: no metacognitive DB exists — nothing to migrate; clean start",
|
||||
"transfer_assumptions": "runtime event log is complete enough for rebuilds",
|
||||
@@ -242,8 +343,15 @@
|
||||
{
|
||||
"id": "Y-R1-C015",
|
||||
"claim": "Per-domain × per-task-class calibration with representative-design: calibration computed on representative task distributions per class; no cross-domain/cross-model transfer assumed; conformal-style guarantees only as strong as their nonconformity scores.",
|
||||
"evidence_for": ["A1-16 Juslin: overconfidence partly item-sampling artifact — representative design required", "A1-23/24/25: human metacognition does not transfer across domains", "A2-16 DS-CP: LLM conformal breaks under domain shift; reweighting restores", "A2-17 Semantic Illusion: certificates vacuous with weak scores (100% FPR at target coverage)"],
|
||||
"evidence_against": ["class boundaries themselves need defining (adversarial BF); cold-start classes empty"],
|
||||
"evidence_for": [
|
||||
"A1-16 Juslin: overconfidence partly item-sampling artifact — representative design required",
|
||||
"A1-23/24/25: cited human studies show limited/inconsistent cross-domain metacognitive transfer (NOT universal non-transfer) — conservative rule: do not assume transfer until measured",
|
||||
"A2-16 DS-CP: LLM conformal breaks under domain shift; reweighting restores",
|
||||
"A2-17 Semantic Illusion: certificates vacuous with weak scores (100% FPR at target coverage)"
|
||||
],
|
||||
"evidence_against": [
|
||||
"class boundaries themselves need defining (adversarial BF); cold-start classes empty"
|
||||
],
|
||||
"limitations": "LLM-side transfer never demonstrated either way (negative-space consistency only).",
|
||||
"rune_evidence": "inward B: no outcome history exists — tables start empty and honest",
|
||||
"transfer_assumptions": "task classes are stable enough to accumulate history",
|
||||
@@ -255,4 +363,4 @@
|
||||
"falsifier": "pooled calibration matches per-class on abstention decisions, or policy transfers unchanged across models with different outcome distributions",
|
||||
"kill_criterion": "per-class fragmentation leaves every class data-starved"
|
||||
}
|
||||
]
|
||||
]
|
||||
@@ -45,7 +45,9 @@
|
||||
|
||||
## Verdict
|
||||
|
||||
**SATURATION = REACHED** — final state finalized post-arms: late searches produced NO new architecture-changing mechanism after incorporation (the zero-hit searches themselves are findings recorded in the master). The open gaps are typed in Remaining Unknowns, not hidden.
|
||||
**SEARCH_SATURATION = REACHED** — late searches stopped producing new architecture-changing mechanisms after incorporation. The 8 zero-hit checks are recorded as bounded-search findings (no direct match found in the searches performed as of 2026-09-21; NOVELTY_UNPROVEN — bounded searches cannot prove nonexistence), not proofs of nonexistence.
|
||||
|
||||
**LOAD_BEARING_DEPTH = PASS** (repair top-up 2026-09-22): every architecture-changing ADOPT claim now maps through load_bearing_evidence.md to at least one of: deep external evidence (5 FULL_PAPER + 4 RELEVANT_SECTIONS anchors), direct Rune inward/runtime evidence, or explicit design-invariant designation with corroborating-only external support. Per-claim map: load_bearing_evidence.md.
|
||||
|
||||
## Residual gaps (honest)
|
||||
|
||||
|
||||
@@ -0,0 +1,28 @@
|
||||
# #400 Y — Load-Bearing Evidence Map (repair mandate §15)
|
||||
|
||||
Every architecture-changing #400 claim mapped to its support. Support types: **DEEP_EXTERNAL** (FULL_PAPER/RELEVANT_SECTIONS load-bearing external evidence), **RUNE_DIRECT** (direct inward/runtime evidence sufficient for the exact claim), **DESIGN_INVARIANT** (explicit Rune design decision; external evidence corroborating only), **DOWNGRADED** (moved to ADAPT/EXPERIMENT/RUNE_HYPOTHESIS). Rule: ABSTRACT/METADATA breadth alone must not carry an architecture-changing ADOPT.
|
||||
|
||||
| Claim | Architecture rule | Inward evidence | External anchor(s) | Depth | Support type | External evidence is | Residual assumption | Experiment |
|
||||
|---|---|---|---|---|---|---|---|---|
|
||||
| Y-R1-C001 | inward facts only (narrowed) | 01_inward row evidence: silent abstention live; confidence assignation; ledgers unconsumed; recovery history ABSENT; loop stale; ADAPT-2 wired+canaried | — | — | RUNE_DIRECT | — (observation) | point-in-time audit | E1 |
|
||||
| Y-R1-C002 | monitoring/control split | ADAPT-2 = live control half; no monitoring half | N&N 1994 (A1-3) FULL_PAPER; Koriat 2006 caveat (A1-8) METADATA_VERIFIED-corroborating | DEEP | DEEP_EXTERNAL | direct (framework) + corroborating (coupling caveat) | software re-derivation of judgment types | E6 |
|
||||
| Y-R1-C003a | external confidence finding (no Rune prescription) | — | Tian A2-1, Xiong A2-2, Kadavath A2-7, Farquhar A2-4 (RELEVANT_SECTIONS) | ABS-breadth + 1 RS | EXTERNAL finding (not a Rune ADOPT) | — | — | E3 |
|
||||
| Y-R1-C003b | verbal confidence = advisory ranking only, never authority | inward B: assignation-only confidence live | C003a external finding (corroborating) | — | DESIGN_INVARIANT | corroborating | validation method per task/model | E3 |
|
||||
| Y-R1-C004 | anti-survivorship flywheel | no outcome logs exist (design space open) | Juslin A1-16 (representative design, corroborating) | METADATA | DOWNGRADED (ADAPT) | analogical | forced-evaluation cost | E3+variant |
|
||||
| Y-R1-C005 | reflection regimes A/B/C | cognitive loop DORMANT (periodic reflection empirically dead) | Huang A3-4 FULL_PAPER (v2, all tables); Kamoi A3-5 FULL_PAPER (v3); CRITIC A3-3, Reflexion A3-1 ABS-corroborating | DEEP | DEEP_EXTERNAL | direct | 2023-24 models; no skilled-few-shot condition tested | E4 |
|
||||
| Y-R1-C006 | anti-rumination governor + budgets | ADAPT-2 budgets cap work tiers in production | Hay A3-12 (non-termination); Snell A3-7 FULL_PAPER (v1, MATH-only scope); overthinking A3-8 ABS-corroborating | DEEP (Snell) | DEEP_EXTERNAL | direct-for-allocation, analogical-for-caps | cap values per-class | E5/E6 |
|
||||
| Y-R1-C007 | baseline reconciler channel | inward B2/D: silent abstention, no health probes | adversarial AK/AY/BS (internal); no external anchor | — | DESIGN_INVARIANT (RUNE_HYPOTHESIS) | — | probe coverage per store | E2+variant |
|
||||
| Y-R1-C008 | surprise two-tier + independent channel | no expectation store (design space open) | Itti & Baldi A3-15 (KL = ONE definition) METADATA-corroborating; ICM A3-16 ABS-corroborating | METADATA | DESIGN_INVARIANT | corroborating | semantic surprise coverage | E10 |
|
||||
| Y-R1-C009 | capability awareness from runtime truth | inward J/K: ledgers fresh, populated, UNCONSUMED; inward I: no runtime introspection | A2-31/32/33 ABS (parametric fragility); A3-26/27 ABS (routing form) — corroborating | ABS | DESIGN_INVARIANT | corroborating | probe cost | E8 |
|
||||
| Y-R1-C010 | progress truth | inward F/H: task truth exists, status evidence-backing UNKNOWN | AgentBoard A3-29, Buehler A3-18, Flyvbjerg A3-19/20, Jørgensen A3-21 — corroborating | BIB/ABS | DESIGN_INVARIANT | corroborating | milestone taxonomy per class | E7/E14 |
|
||||
| Y-R1-C011 | recovery-aware metacognition | inward G: store ABSENT — re-diagnosis from scratch live | AgentDebug A3-24 ABS (26% feedback gains, nearest); Who&When 14.2% / TRAIL 11% (in-benchmark difficulty, not SOTA) | ABS | DOWNGRADED (ADAPT, RUNE_HYPOTHESIS) | analogical | attribution machinery absent (precondition) | E9 |
|
||||
| Y-R1-C012 | authority boundary + grounding | inward B2: silent abstention makes boundary load-bearing now | Nisbett & Wilson via N&N 1994 (A1-3 FULL_PAPER); A2-32/33 ABS-corroborating | DEEP (via A1-3) | DEEP_EXTERNAL | direct (introspection-is-inference) | evidence-class tagging keeps LLM layer honest | E12 |
|
||||
| Y-R1-C013 | event-driven + propose/dispose | inward A (loop dormant) vs E (router healthy) — the asymmetry decides | SeaKR A2-25 ABS-corroborating; n8n/Temporal docs (mechanism only) | ABS/DOC | DESIGN_INVARIANT | corroborating | hysteresis calibration | E6+variant |
|
||||
| Y-R1-C014 | metacognitive memory = outcome records, event-reconstructable | inward: no metacognitive DB exists — clean start | #396 rebuildability (domain carry); adversarial AO | — | DESIGN_INVARIANT | — | runtime event-log completeness | E15 |
|
||||
| Y-R1-C015 | per-class calibration, representative design | inward B: no outcome history — tables start empty | Juslin A1-16 METADATA; DS-CP A2-16, Semantic Illusion A2-17 ABS-corroborating; human transfer limited/inconsistent (A1-23/24/25) — NOT universal non-transfer | METADATA/ABS | DESIGN_INVARIANT + corroborating | corroborating | class boundaries; cold start | E3/E13 |
|
||||
|
||||
## Coverage verdict
|
||||
|
||||
- ADOPT claims (12): C001 RUNE_DIRECT · C002, C005 DEEP_EXTERNAL · C003b, C006 (governor half), C007(via ADAPT), C008, C009, C010, C012, C013, C014 DESIGN_INVARIANT/RUNE_DIRECT with corroborating external — **no ADOPT rests on ABSTRACT/METADATA breadth alone**.
|
||||
- ADAPT claims (4): C004, C007 (as hypothesis inside), C011, C015 — downgraded or design-invariant, honestly labeled.
|
||||
- Deep anchors after top-up: FULL_PAPER ×5 (N&N 1994, Huang v2, Kamoi v3, Snell v1, GRAB-RAG v1) · RELEVANT_SECTIONS ×4 (Farquhar Nature 2024, Fleming & Lau 2014, Norman 2016, Hanczakowski 2017).
|
||||
@@ -1,32 +1,35 @@
|
||||
# #400 Y — CONTENT-FREEZE AUDIT (final closeout, 2026-09-22)
|
||||
# #400 Y — CONTENT-FREEZE AUDIT (repair closeout, 2026-09-22)
|
||||
|
||||
Written LAST, after all artifact/tooling/registry/canonical/session-note writes. This is the authoritative freeze-state record. The two-phase closeout protocol applies: transport of the frozen seal (push/mirror/Gitea) does NOT stale this audit.
|
||||
|
||||
## Run provenance
|
||||
## Repair provenance (bounded evidence-depth + semantic-precision + gate repair, 2026-09-22)
|
||||
|
||||
Full new-domain R1_FULL run per mandate (NOT a repair, NOT a survey). ONE continuous controller run; arms strictly serial (general-2 → general-3 → general-4 → general-5), all first-pass SUCCESS; MAX_CONCURRENT_SUBAGENTS = 1 held; no stalls, no retries, no provider-capacity faults, no controller substitutions. CONTINUOUS_EXECUTION = FAILED_ONCE — exactly one user continuation (msg_g001a0c71efdeb0015T0xawI7N @ 2026-09-22T03:18:12Z, "continue", after a recoverable malformed master-write call returned at the MASTER stage; derivation with message ID: process_evidence.md, raw trajectory DB).
|
||||
Original #400 seal (ed04909) ACCEPTED on substance; this repair deepened load-bearing evidence, disciplined novelty/superlative language, and hardened the gate. REPAIR_CONTINUOUS_EXECUTION = PASS — one continuous run, zero user continuations, ONE research actor (general-6, bounded top-up, first-pass SUCCESS, strictly serial). MAX_CONCURRENT_SUBAGENTS = 1.
|
||||
|
||||
## Original-run process truth (preserved, mandate §18)
|
||||
|
||||
CONTINUOUS_EXECUTION (original run) = FAILED_ONCE — msg_g001a0c71efdeb0015T0xawI7N @ 2026-09-22T03:18:12Z, derivation in process_evidence.md (now also listed in master §41 artifact pointers). NOT rewritten — raw trajectory evidence still supports it.
|
||||
|
||||
## Mutation categories (actual, at freeze)
|
||||
|
||||
PRODUCTION_CODE_MUTATION = NONE
|
||||
PRODUCTION_SCHEMA_MUTATION = NONE
|
||||
PRODUCTION_RUNTIME_BEHAVIOR_MUTATION = NONE
|
||||
CANONICAL_MEMORY_WRITE = ATTEMPTED, DEDUPE-COLLAPSED — memory_store add executed 2026-09-22; store returned receipt id 190653, decision `duplicate`, prior_id null, similarity 0.80 ("same data point; no new info — touched existing #190653"). No new canonical entry created; existing 190653 touched. Recorded as evidence per mandate §57 (similarity collapse recorded, NOT fixed in-domain). Intended ≠ executed: the intended new summary was NOT created as a separate entry.
|
||||
SESSION_NOTES_WRITE = 2 — two session-note entries appended pre-freeze (run-start + closeout). No post-freeze session note.
|
||||
RESEARCH_TOOLING_MUTATION = YES (research-only) — scripts/research_gate_check.py: (1) DOI unique-key regex fixed for parenthesized Elsevier DOIs (`[^\s,|)]+` → `[^\s,|]+` + rstrip ".,;)" — A1-10/A1-11 false-collapse repaired; #398-W regression-checked clean); (2) _REPAIR_DOMAINS extended to Y- (draft-marker + source-depth checks now cover this domain). Negative controls: 3/3 CAUGHT (Y draft-marker; Y depth-total tamper; DOI-regex revert proving the unique-key fix load-bearing via the source-ledger consistency check), restores byte-identical. Pre-freeze master rewords (gate-collision fixes, no semantic change): inward classification 'PARTIAL 4' → 'PARTIAL-class 4' (adversarial-vocabulary collision); §35 'R0-2/3/4/5 CONFIRMED' → 'R0-2 through R0-5 all CONFIRMED' (digit-before-verdict collision).
|
||||
GITEA_MUTATION = (pre-freeze: issue #400 reopened + start comment 5961; post-freeze transport: closeout receipt + #375 update)
|
||||
RESEARCH_ARTIFACT_MUTATION = YES — full domain body authored: 00_issue_scope.md, 01_inward.md, 02_outward/{arm1,arm2,arm3,arm4}.md, 03_applicability.md, 04_innovation.md, 05_adversarial.md, 06_synthesis.md, 07_qualification.md, claims.json, sources.md, r0_reconciliation.md, preliminary_reconciliation.md, evidence_saturation.md, reconciliation_arithmetic_check.txt, process_evidence.md, Y_METACOGNITION_METAMEMORY_R1_MASTER.md, this audit. Registry #400 row updated (repo-tracked).
|
||||
RESEARCH_RESULT_AFFECTED = NO — the preserved batched artifact (Y_METACOGNITION_R1_BATCHED_PRELIMINARY.md, sha256 8af78726…63288432d1cda, byte-verified identical to the original Y_METACOGNITION_R1_MASTER.md) is untouched; R0 baseline (families/metamemory_security.md) untouched; #399 artifacts untouched (§56 clerical debt recorded, not repaired here).
|
||||
RESEARCH_ONLY_BOUNDARY = HELD — no production Memory/retrieval/cognitive_state/Warm Neurons/Substrate/Spine/task-runtime/TUI/prompt/evaluation changes; inward inspection READ-ONLY (SQLite -readonly; call-site greps; wiring-canary checks only).
|
||||
CANONICAL_MEMORY_WRITE = NONE this repair — the original run's write evidence stands exactly as recorded: receipt 190653, decision `duplicate`, similarity ~0.80, prior_id null (the receipt does NOT identify a collapse target and none is invented here). No new canonical #400 summary written (clerical repair only; canonical conclusions materially unchanged).
|
||||
SESSION_NOTES_WRITE = 3 total original run + 1 this repair (pre-freeze). No post-freeze session note.
|
||||
RESEARCH_TOOLING_MUTATION = YES (research-only) — scripts/research_gate_check.py: (1) +check_load_bearing_depth_map (every ADOPT/ADAPT claim must map through load_bearing_evidence.md with a legal support type; DEEP_EXTERNAL requires the cited anchor row to be FULL_PAPER/RELEVANT_SECTIONS in sources.md); (2) +check_zero_hit_novelty (canonical artifacts must not promote bounded zero-hit searches into nonexistence claims; correction-context aware); (3) +check_rune_proven_scope (negation-aware — RUNE_PROVEN claims must not carry architecture-performance framing); (4) claim arithmetic + external/Rune-split checks extended from W- to W-/X-/Y-; (5) lbm support-column parser fix. Negative controls: 4/4 CAUGHT with byte-identical restores (A load-bearing-depth: downgrading C005's deep anchors fails; B zero-hit reinsertion fails; C Snell+Rune-prescription under EXTERNALLY_ESTABLISHED fails; D RUNE_PROVEN performance framing fails).
|
||||
RESEARCH_ARTIFACT_MUTATION = YES — sources.md (4 anchors → FULL_PAPER with exact versions/sections/dates; 2 secondary rows A1-30/31 added for closed Koriat 1995; 95→97 rows, no duplicates; depth-cell prose artifact fixed so the mechanical parse stays truthful), Y_METACOGNITION_METAMEMORY_R1_MASTER.md (novelty wording; C001 observed-facts scope; reflection A/B/C regimes §10; verbal-confidence advisory ranking §8; Snell design-vs-external split §12; typed-discrepancy surprise §14; reconciler-backstop + TIMER-EXPIRY-NEVER-TRIGGERS-LLM invariant §28; in-benchmark attribution framing §20; limited/inconsistent human transfer §4; event-log reconstructable wording; SEARCH_SATURATION vs LOAD_BEARING_DEPTH split §39; sources count §1/§40; §41 pointers + §42 repair line), evidence_saturation.md (split qualifications), 04_innovation.md (novelty + surprise wording), claims.json (C001 narrowed to observed facts; C003b advisory-ranking; C005 regimes + exact Huang/Kamoi scope; C008 typed discrepancy; C011 attribution wording; C015 limited/inconsistent transfer; C004/C009/C010 limitations novelty wording — classifications and evidence levels UNCHANGED), load_bearing_evidence.md (NEW — per-claim support map, repair §15), this audit. Registry #400 row updated.
|
||||
RESEARCH_RESULT_AFFECTED = NO — batched preliminary untouched (sha256 8af78726…); R0 baseline untouched; adversarial verdicts unchanged; row truth unchanged: applicability 52 (28/6/3/0/14/1) · qualification 38 (32/6) · adversarial 73 (5/20/48) · R0 5 (4C/1R) · preliminary 8 (5C/3R) · claims 16 (12/4; elevels 1/11/3/1) — totals preserved because classifications did not change (wording precision only, verified by mechanical recompute).
|
||||
RESEARCH_ONLY_BOUNDARY = HELD — no production changes; no ADAPT-2 behavior modification; read-only actor.
|
||||
|
||||
## Row truth at freeze (machine-derived; reconciliation_arithmetic_check.txt)
|
||||
## Top-up source depths (mandate §21 record)
|
||||
|
||||
applicability 52 = ADOPT 28 · ADAPT 6 · EXPERIMENT 3 · REJECT 14 · PARK 1 · qualification 38 = ADOPT 32 · ADAPT 6 · adversarial 73 = SUCCEEDS 5 · PARTIAL 20 · FAILS 48 · R0 5 = CONFIRMED 4 · REFINED 1 · preliminary 8 = CONFIRMED 5 · REFINED 3 · claims 16 = ADOPT 12 · ADAPT 4 · evidence levels = RUNE_PROVEN 1 · RUNE_APPLICABLE 11 · RUNE_HYPOTHESIS 3 · EXTERNALLY_ESTABLISHED 1 · sources 95 rows / 95 unique · depth labels FULL_PAPER 1 · RELEVANT_SECTIONS 2 · ABSTRACT_ONLY 53 · METADATA_VERIFIED 88 · DOCUMENTATION_AUDITED 4 (148 labels/95 rows, 53 dual-label rows) · innovation 15 = ADOPT 13 · ADAPT 2 · saturation REACHED (8 typed zero-hit checks recorded as findings).
|
||||
Huang 2310.01798v2 → FULL_PAPER (all tables; intrinsic degradation scoped to prompted intrinsic, no external feedback; NO skilled-few-shot condition tested) · Kamoi 2406.01297v3 → FULL_PAPER (feedback-source taxonomy; verification≪generation carve-out; oracle-label and weak-prompt artifact analysis) · Snell 2408.03314v1 → FULL_PAPER (MATH-only, PaLM 2-S* fine-tuned PRM+revision; verifiable-domain assumption explicit) · GRAB-RAG 2608.22228v1 → FULL_PAPER (3.8B–8B frozen models; 41.6% = P1 macro-avg; 13.3% best mitigation at ~50% coverage cost; not generalizable) · Koriat 1995 primary CLOSED (Unpaywall no-OA) — secondary-verified via Norman 2016 / Hanczakowski 2017 (A1-30/31 RELEVANT_SECTIONS). Ledger: 97 rows / 97 unique; FULL_PAPER 5 · RELEVANT_SECTIONS 4 · ABSTRACT_ONLY 49 · METADATA_VERIFIED 84 · DOCUMENTATION_AUDITED 4 (146 labels / 97 rows).
|
||||
|
||||
## Citation corrections (8, all live-verified)
|
||||
## Zero-hit wording corrections (mandate §3/§10 record)
|
||||
|
||||
Koriat "Dissociating" = 1995 (not 1993/97) · R-Tuning = 2311.09677 (not .09676) · DRAGIN = 2403.10081 (not 2310.08682) · Semantic Density = 2405.13845 (not 2406.04593) · "Art of Saying No" = Brahman/Kumar 2407.12043 (not "Cheng 2023"; Cheng = 2401.13275) · overthinking = 2412.21187 (not 2501.x) · TRAIL = 2505.08638 (not 2504.x) · Flyvbjerg "frosting error" paper DOES NOT EXIST (cite Flyvbjerg & Bester 2021 / Flyvbjerg et al. 2022).
|
||||
Executive typed-taxonomy claim, §38 Remaining Unknowns (7 items), §39 saturation, §32/§33, 04_innovation C/K, evidence_saturation verdict, claims C004/C009/C010/C011 limitations: all "confirmed unpublished / does not exist / no published X / first mover / no literature" replaced with bounded-search framing ("no direct match found in the searched sources as of 2026-09-21", NOVELTY_UNPROVEN). Comparative/superlative repairs: "best published 14.2%/11%" → in-benchmark framing; "wrong party to ask" → self-diagnosis insufficient alone, hypotheses allowed; "no benchmark shows agents estimating progress well" → pass-scoped.
|
||||
|
||||
## Freeze chronology (mandate §58)
|
||||
## Freeze chronology (mandate §21)
|
||||
|
||||
All state/accounting writes happened BEFORE this audit: artifact edits → tooling hardening + negative controls → registry edit → canonical write attempt (receipt 190653, duplicate-collapse recorded) → session-note appends (2) → THIS CONTENT-FREEZE AUDIT (last) → FINAL CONTENT GATE → SEAL COMMIT → push/mirror/verify → Gitea #400 closeout receipt + #375 update (post-freeze transport; no post-freeze session note, no post-freeze repo commit to record transport).
|
||||
All content/accounting writes BEFORE this audit: top-up deep reads → source ledger → load_bearing_evidence.md → master/stage/claim edits → gate hardening + 4/4 negative controls → mechanical row recompute (incl. depth-cell prose artifact fix caught by the recount) → registry edit → session note (1, pre-freeze) → THIS CONTENT-FREEZE AUDIT (last) → FINAL CONTENT GATE → SEAL COMMIT → push/mirror/verify → Gitea #400 repair receipt + #375 update (post-freeze transport only; no post-freeze session note).
|
||||
@@ -1,4 +1,4 @@
|
||||
#400 Y — reconciliation_arithmetic_check (MACHINE-GENERATED from canonical rows)
|
||||
#400 Y — reconciliation_arithmetic_check (MACHINE-GENERATED from canonical rows; recomputed at repair closeout 2026-09-22)
|
||||
|
||||
applicability: 52 rows — ADOPT 28 · ADAPT 6 · EXPERIMENT 3 · REJECT 14 · PARK 1 — Sum 52
|
||||
qualification: 38 rows — ADOPT 32 · ADAPT 6 — Sum 38
|
||||
@@ -7,7 +7,7 @@ preliminary: 8 rows — CONFIRMED 5 · REFINED 3 — Sum 8
|
||||
adversarial: 73 rows — SUCCEEDS 5 · PARTIAL 20 · FAILS 48 — Sum 73 (arm-4, independent fresh-context pass; mandated A-BL 64 + fresh BM-BU 9)
|
||||
claims: 16 rows — ADOPT 12 · ADAPT 4 — Sum 16
|
||||
evidence levels: RUNE_PROVEN 1 · RUNE_APPLICABLE 11 · RUNE_HYPOTHESIS 3 · EXTERNALLY_ESTABLISHED 1 — Sum 16
|
||||
source ledger: 95 rows — 95 unique
|
||||
source depth labels (machine-derived; combined cells count each stated label once): ABSTRACT_ONLY 53 · DOCUMENTATION_AUDITED 4 · FULL_PAPER 1 · METADATA_VERIFIED 88 · RELEVANT_SECTIONS 2 — label total 148 across 95 rows
|
||||
source ledger: 97 rows — 97 unique
|
||||
source depth labels (machine-derived; combined cells count each stated label once): ABSTRACT_ONLY 49 · DOCUMENTATION_AUDITED 4 · FULL_PAPER 5 · METADATA_VERIFIED 84 · RELEVANT_SECTIONS 4 — label total 146 across 97 rows
|
||||
|
||||
All narrative totals in the master and stage files are derived FROM these rows; the gate (scripts/research_gate_check.py) mechanically refuses PASS if any narrative total contradicts row truth. Saturation = REACHED (late searches: 8 typed zero-hit/rumor checks recorded as findings, not hidden). Innovation candidates: 15 (A-O) — ADOPT 13 · ADAPT 2.
|
||||
All narrative totals in the master and stage files are derived FROM these rows; the gate (scripts/research_gate_check.py) mechanically refuses PASS if any narrative total contradicts row truth. SEARCH_SATURATION = REACHED and LOAD_BEARING_DEPTH = PASS are separate qualifications (repair §12). Repair top-up 2026-09-22: 4 anchors deepened to FULL_PAPER (Huang/Kamoi/Snell/GRAB-RAG), 2 secondary rows added (Koriat-1995 coverage), 0 rows duplicated — 95→97 rows.
|
||||
@@ -8,7 +8,7 @@ Read-depth values: FULL_PAPER / RELEVANT_SECTIONS / DOCUMENTATION_AUDITED / CODE
|
||||
| A1-2 | Metamemory: A theoretical framework and new findings | Nelson & Narens | 1990 | Psych. Learning & Motivation 26, DOI 10.1016/S0079-7421(08)60053-5 | framework | METADATA_VERIFIED (framework verbatim via authors' 1994 chapter) | monitoring/control logical independence; EOL/JOL/FOK/confidence taxonomy | high (canon) | imperfect monitoring still useful (Fodor); control can modify the mechanism |
|
||||
| A1-3 | Why investigate metacognition? | Nelson & Narens | 1994 | Metcalfe & Shimamura eds., MIT Press, DOI 10.7551/mitpress/4561.003.0007 | framework chapter | FULL_PAPER (25pp author-hosted PDF, sites.socsci.uci.edu/~lnarens) | same as A1-2 with verbatim pp.127/130 quotes | high | anchor text for the whole domain |
|
||||
| A1-4 | How do we know that we know? Accessibility model | Koriat | 1993 | Psych. Review 100(4), DOI 10.1037/0033-295X.100.4.609 | theory | METADATA_VERIFIED | FOK = accessible partial info, not truth readout | high | margin = accessibility |
|
||||
| A1-5 | Dissociating knowing and the feeling of knowing | Koriat | 1995 | JEP:G 124(3), DOI 10.1037/0096-3445.124.3.311 (year/title corrected from mandate) | empirical theory | METADATA_VERIFIED | cue priming raises FOK without recall gain — the dissociation | high | feeling ≠ correctness |
|
||||
| A1-5 | Dissociating knowing and the feeling of knowing | Koriat | 1995 | JEP:G 124(3), DOI 10.1037/0096-3445.124.3.311 (year/title corrected from mandate) | empirical theory | METADATA_VERIFIED (repair top-up 2026-09-22: primary CLOSED everywhere per Unpaywall — no OA/repository copy; total-accessibility dissociation secondary-verified via A1-30/A1-31 open texts; primary cue-priming statistics not verified, honestly recorded) | accessibility (amount of partial info, regardless of accuracy) drives FOK — margin = accessibility cue, not truth | high | feeling ≠ correctness; does NOT by itself prove an agent state taxonomy |
|
||||
| A1-6 | Monitoring own knowledge during study (JOL cue-utilization) | Koriat | 1997 | JEP:G 126(4), DOI 10.1037/0096-3445.126.4.349 | theory+empirical | METADATA_VERIFIED | JOL cues: intrinsic/extrinsic/mnemonic | medium-high | cue regimes differ by judgment type |
|
||||
| A1-7 | Underconfidence-with-practice | Koriat, Sheffer & Ma'ayan | 2002 | JEP:G 131(2), DOI 10.1037/0096-3445.131.2.147 | empirical | METADATA_VERIFIED | calibration biases are cue-regime-specific, sign varies | medium | no universal calibration direction |
|
||||
| A1-8 | Monitoring-control intricate relationships | Koriat, Ma'ayan & Nussinson | 2006 | JEP:G 135(1), DOI 10.1037/0096-3445.135.1.36 | empirical theory | METADATA_VERIFIED | control produces fluency monitoring reads — bidirectional coupling | medium-high | one-way separation is empirically false; keep logical, not causal, independence |
|
||||
@@ -34,7 +34,7 @@ Read-depth values: FULL_PAPER / RELEVANT_SECTIONS / DOCUMENTATION_AUDITED / CODE
|
||||
| A1-28 | Metacognitive monitoring and study-time allocation | Son & Metcalfe | 2000 | JEP:LMC 26(1), 204–221 | empirical | METADATA_VERIFIED | allocation follows difficulty across regimes | medium | effort allocation analogue |
|
||||
| A1-29 | A metacognitive nation (fluency) | Alter & Oppenheimer | 2009 | PSPR 13(3), DOI 10.1177/1088868309341564 | review theory | METADATA_VERIFIED | processing fluency treated as knowledge evidence | medium-high | fluent derivation = cue noise |
|
||||
|
||||
Depth summary (arm-1): FULL_PAPER 2 · RELEVANT_SECTIONS 1 · METADATA_VERIFIED 26 · UNVERIFIED 0. Secondary-only items (Leonesio & Nelson 1988; Nelson & Dunlosky 1991) are recorded inside A1-2/A1-3 evidence, not as standalone rows.
|
||||
Depth summary (arm-1): FULL_PAPER 1 · RELEVANT_SECTIONS 1 · METADATA_VERIFIED 27 · UNVERIFIED 0 (recounted at repair; N&N 1994 is the arm-1 FULL_PAPER anchor). Secondary-only items (Leonesio & Nelson 1988; Nelson & Dunlosky 1991) are recorded inside A1-2/A1-3 evidence, not as standalone rows.
|
||||
|
||||
## Arm 2 sources (LLM uncertainty / calibration / self-knowledge / RAG metamemory; general-3, 2026-09-21 — 1 main-text, 32 abstract+metadata verified live)
|
||||
|
||||
@@ -51,7 +51,7 @@ Depth summary (arm-1): FULL_PAPER 2 · RELEVANT_SECTIONS 1 · METADATA_VERIFIED
|
||||
| A2-9 | The Art of Saying No | Brahman, Kumar et al. | 2024 | arXiv:2407.12043v2 (corrected attribution), NeurIPS 2024 D&B | benchmark+taxonomy | ABSTRACT_ONLY + METADATA_VERIFIED (live) | contextual noncompliance; GPT-4 wrongly complies ~30% | medium | noncompliance ≠ abstention calibration |
|
||||
| A2-10 | Can AI Assistants Know What They Don't Know? | Cheng et al. | 2024 | arXiv:2401.13275v2 | method | ABSTRACT_ONLY + METADATA_VERIFIED (live) | Idk-data alignment for refusal | medium | refusal tuning lineage |
|
||||
| A2-11 | Sufficient Context | Joren et al. | 2025 | arXiv:2411.06037v3, ICLR 2025 | empirical | ABSTRACT_ONLY + METADATA_VERIFIED (live) | models answer on insufficient context; sufficiency gate +2–10% precision | high | sufficiency gate real but blind to misleadingness |
|
||||
| A2-12 | GRAB-RAG | Setiawan | 2026 | arXiv:2608.22228v1, AACL-IJCNLP 2026 (committed) | benchmark+method | ABSTRACT_ONLY + METADATA_VERIFIED (live) | abstention answers 41.6% of misleading-context questions; NLI fails when parametric memory agrees with lie | high | misleadingness > sufficiency as failure mode |
|
||||
| A2-12 | GRAB-RAG | Setiawan | 2026 | arXiv:2608.22228v1, AACL-IJCNLP 2026 (committed) | benchmark+method | FULL_PAPER (v1 read 2026-09-22 repair top-up: 3 frozen quantized 3.8B–8B models (Phi-4-mini, Llama-3.1-8B, Qwen2.5-7B); NQ+HotpotQA 500q each; Longpre-style entity-swap misleading construction; 41.6% = P1 explicit-abstention macro-average over 6 model×dataset cells (range 13.6–74.3%); 63% of answered QC echo the planted entity; P3 conflict-check 13.3% at 49.8% false-abstention cost; NLI fails on memory–context agreement) | misleadingness > sufficiency as failure mode; DO NOT generalize 41.6% beyond small extractive-QA setting | high | conflict-checking trades coverage for safety |
|
||||
| A2-13 | Conformal prediction MC-QA | Kumar et al. | 2023 | arXiv:2305.18404v3, ICML'23 TEACH workshop (venue corrected) | method | ABSTRACT_ONLY + METADATA_VERIFIED (live) | CP sets correlate with accuracy; exchangeability weakens | medium | CP needs representative calibration |
|
||||
| A2-14 | API Is Enough (logit-free CP) | Su et al. | 2024 | arXiv:2403.01216v2 | method | ABSTRACT_ONLY + METADATA_VERIFIED (live) | CP via sample frequency + semantic similarity | medium | black-box CP possible |
|
||||
| A2-15 | Conformal-RAG | Feng et al. | 2025 | arXiv:2506.20978v1, SIGIR 2025 short | method | ABSTRACT_ONLY + METADATA_VERIFIED (live) | group-conditional coverage on RAG sub-claims | medium | sub-claim coverage |
|
||||
@@ -83,10 +83,10 @@ Explicit absences (searched, zero hits — recorded as findings, not fabricated)
|
||||
| A3-1 | Reflexion | Shinn et al. | 2023 | arXiv:2303.11366v4, NeurIPS 2023 | method | ABSTRACT_ONLY + METADATA_VERIFIED (live) | feedback = external OR simulated; episodic buffer across trials | high | reflection + env signal + memory = works |
|
||||
| A3-2 | Self-Refine | Madaan et al. | 2023 | arXiv:2303.17651 | method | ABSTRACT_ONLY + METADATA_VERIFIED (live) | same-LLM generator+feedback (intrinsic loop) | medium | the loop Huang attacks; gains concentrated in text-quality tasks |
|
||||
| A3-3 | CRITIC | Gou et al. | 2024 | arXiv:2305.11738, ICLR 2024 | method | ABSTRACT_ONLY + METADATA_VERIFIED (live) | tool-interactive critiquing; "crucial importance of external feedback" | high | external verifiability gate |
|
||||
| A3-4 | LLMs Cannot Self-Correct Reasoning Yet | Huang et al. | 2024 | arXiv:2310.01798v2, ICLR 2024 | negative result | ABSTRACT_ONLY + METADATA_VERIFIED (live) | intrinsic self-correction degrades reasoning; scope: reasoning + no external feedback | high | do NOT make "reflect more" default |
|
||||
| A3-5 | When Can LLMs Actually Correct Their Own Mistakes? | Kamoi et al. | 2024 | arXiv:2406.01297, TACL 12:1417–1440 | critical survey | ABSTRACT_ONLY + METADATA_VERIFIED (live) | no demonstrated prompted-feedback self-correction; works with reliable external feedback or fine-tuning | high | the design-rule anchor |
|
||||
| A3-4 | LLMs Cannot Self-Correct Reasoning Yet | Huang et al. | 2024 | arXiv:2310.01798v2, ICLR 2024 | negative result | FULL_PAPER (v2 read 2026-09-22 repair top-up: §1–7 incl. all tables; GPT-4 GSM8K 95.5→89.0, GPT-3.5 CSQA 75.8→38.1, Llama-2 62.0→36.5 intrinsic; scope = PROMPTED INTRINSIC self-correction (no external feedback/oracle), NOT "self-correction never works"; no skilled-few-shot condition tested; Self-Refine gain traced to weak initial prompt) | intrinsic prompted self-correction shows no reliable general benefit — gate reflection on external verifiability | high | do NOT make "reflect more" default; comparisons need matched inference cost |
|
||||
| A3-5 | When Can LLMs Actually Correct Their Own Mistakes? | Kamoi et al. | 2024 | arXiv:2406.01297v3, TACL 12:1417–1440 | critical survey | FULL_PAPER (v3 read 2026-09-22 repair top-up: §1–4, 5.1, 6–7; taxonomy = intrinsic prompted / external tools / external knowledge / fine-tuning; exceptionally-suitable carve-out = verification ≪ generation (CoVe decomposable responses); RCI/Reflexion positives used oracle labels; Self-Refine positives = weak-initial-prompt artifacts; strong-baseline problem: compare vs self-consistency at matched cost) | reflection gate design-rule anchor with precise scope | high | feedback generation is the bottleneck; many real tasks lack the decomposability carve-out |
|
||||
| A3-6 | PlanBench | Valmeekam et al. | 2023 | arXiv:2206.10498, NeurIPS 2023 D&B | benchmark | ABSTRACT_ONLY + METADATA_VERIFIED (live) | planning competence weak | medium | domain capability, indirect on reflection |
|
||||
| A3-7 | Scaling LLM Test-Time Compute Optimally | Snell et al. | 2024 | arXiv:2408.03314 | empirical scaling | ABSTRACT_ONLY + METADATA_VERIFIED (live) | compute-optimal adaptive allocation; difficulty-dependent; 4× efficiency | high | LLM-era VoC stand-in |
|
||||
| A3-7 | Scaling LLM Test-Time Compute Optimally | Snell et al. | 2024 | arXiv:2408.03314v1 | empirical scaling | FULL_PAPER (v1 read 2026-09-22 repair top-up: MATH benchmark ONLY; PaLM 2-S* explicitly fine-tuned — PRM + revision model; compute-optimal per-difficulty strategy beats best-of-N at up to 4× less compute; ~14×-larger model beaten FLOPs-matched only where base has non-trivial success; verifiable-domain + difficulty-estimation assumptions EXPLICIT — does NOT transfer to open-ended agents without verifiers) | adaptive allocation evidence with exact scope; supports (not proves) Rune's fluid-controller design | high | LLM-era VoC stand-in, scoped to verifiable domains |
|
||||
| A3-8 | Do NOT Think That Much for 2+3=? | Chen et al. | 2024 | arXiv:2412.21187v2 (ID corrected), 2024/25 | empirical | ABSTRACT_ONLY + METADATA_VERIFIED (live) | overthinking metrics; self-training mitigation | medium-high | efficiency dashboards |
|
||||
| A3-9 | Thoughts Are All Over the Place (underthinking) | Wang et al. | 2025 | arXiv:2501.18585 | empirical | ABSTRACT_ONLY + METADATA_VERIFIED (live) | thought-switching correlates with errors; TIP penalty | medium | premature stopping = dual failure |
|
||||
| A3-10 | Stop Overthinking (survey) | Sui et al. | 2025 | arXiv:2503.16419, TMLR 2025 | survey | ABSTRACT_ONLY + METADATA_VERIFIED (live) | budget taxonomy: model/output/input-based | medium-high | mechanism map |
|
||||
@@ -113,3 +113,10 @@ Explicit absences (searched, zero hits — recorded as findings, not fabricated)
|
||||
| A3-31 | LangGraph persistence | LangChain | current | docs.langchain.com (DOC) | commercial docs | DOCUMENTATION_AUDITED | checkpointers vs stores; durability modes | medium (mechanism only) | status = snapshot/resume |
|
||||
| A3-32 | Temporal workflows | Temporal | current | docs.temporal.io/workflows (DOC) | commercial docs | DOCUMENTATION_AUDITED | Event History as source of truth; deterministic replay; results recorded once | high (mechanism) | durable status ≠ self-report |
|
||||
| A3-33 | n8n error handling | n8n | current | docs.n8n.io (DOC) | commercial docs | DOCUMENTATION_AUDITED | error workflows; execution.id/retryOf; lastNodeExecuted; Stop-and-Error | medium (mechanism only) | structured error payloads + retry provenance |
|
||||
|
||||
## Repair top-up secondaries (2026-09-22, general-6 — open-access covers for the closed Koriat 1995 primary)
|
||||
|
||||
| # | Source | Authors | Year | Venue / ID | Type | Read depth | Rune relevance | Confidence | Lesson |
|
||||
|---|---|---|---|---|---|---|---|---|---|
|
||||
| A1-30 | Metacognitive experiences and their role in cognition (open secondary quoting Koriat 1993/95 procedures) | Norman, Blakstad, Johnsen, Martinsen & Price | 2016 | Front. Psychol. 7:996 | secondary review | RELEVANT_SECTIONS (Intro+Results+Discussion read) | documents total-accessibility finding: FOK tracks amount of partial retrieved info regardless of accuracy | medium | secondary anchor while primary paywalled |
|
||||
| A1-31 | Accessibility-model discussion (open secondary) | Hanczakowski et al. | 2017 | PMC5207169 (Europe PMC full text) | secondary discussion | RELEVANT_SECTIONS (accessibility-model sections read) | quotes Koriat 1993/1995 directly; accessibility vs accuracy dissociation | medium | secondary anchor |
|
||||
@@ -26,7 +26,7 @@
|
||||
| 397 | V: Data Provenance / Lineage | research-complete | R1_QUALIFIED (R1_FULL — deep requalified 2026-09-21; bounded evidence-depth/consistency/proof repair RESEALED 2026-09-21; batched preserved as preliminary; provenance envelope + derivation edges + laundering/authority rules; 'Ledger (Kumar)' confabulation corrected) | — (no dup) | families/provenance.md | memory-atlas-r1/V-*/ |
|
||||
| 398 | W: User Modelling / Preference Drift | research-complete | R1_QUALIFIED (R1_FULL — deep requalified 2026-09-21; bounded evidence-depth/semantics/process-truth repair RESEALED 2026-09-21; batched preserved as preliminary; evidence-first + rebuildable projection + origin-aware authority + non-amplifying personalization + per-preference cooling + clarification economy; Ur/SOUPS-2012 venue corrected) | — (no dup) | families/user_modelling_persona_identity.md | memory-atlas-r1/W-*/ |
|
||||
| 399 | X: Persona / Identity | research-complete | R1_QUALIFIED (R1_FULL — deep requalified 2026-09-21; batched preserved as preliminary; lineage-based identity + kernel/persona/state strata + model-as-substrate + experience-gated evolution + grounded self-report + branching discipline + narrative-as-projection + relational non-fabrication + governed self-change; Shumailov DOI corrected) | — (no dup) | families/user_modelling_persona_identity.md | memory-atlas-r1/X-*/ |
|
||||
| 400 | Y: Metacognition | research-complete | R1_QUALIFIED (R1_FULL — deep requalified 2026-09-22; batched preserved byte-for-byte as preliminary (5C/3R); monitoring/control split (N&N anchor) + typed negative retrieval states (unpublished-taxonomy gap) + metamemory projection + ranking-only verbal confidence pending outcome flywheel + reflection gate on external verifiability + anti-rumination governor + baseline reconciler channel + capability awareness from runtime truth + progress truth (9 statuses) + recovery priors (★ build-preconditions: attempt logging, recovery history); adversarial 73 (5/20/48, five SUCCEEDS fixed in place); claims 16 (12/4); sources 95/95; FAILED_ONCE; single routing authority — #400 proposes, ADAPT-2 disposes) | — (no dup) | families/metamemory_security.md | memory-atlas-r1/Y-*/ |
|
||||
| 400 | Y: Metacognition | research-complete | R1_QUALIFIED (R1_FULL — deep requalified 2026-09-22; bounded evidence-depth/semantic-precision/gate repair RESEALED 2026-09-22: 4 anchors deepened FULL_PAPER (Huang/Kamoi/Snell/GRAB-RAG exact scopes), Koriat 1995 secondary-verified (primary closed), novelty wording → bounded-search NOVELTY_UNPROVEN, reflection A/B/C regimes, verbal confidence advisory-ranking-only, Snell design-vs-external split, typed-discrepancy surprise, reconciler backstop + timer-never-triggers-LLM invariant, C001 observed-facts scope, in-benchmark attribution framing, SEARCH_SATURATION vs LOAD_BEARING_DEPTH split, load_bearing_evidence.md per-claim support map, gate +3 Y checks 4/4 negative controls; batched preserved byte-for-byte as preliminary (5C/3R); monitoring/control split (N&N anchor) + typed negative retrieval states + metamemory projection + reflection gate on external verifiability + anti-rumination governor + baseline reconciler channel + capability awareness from runtime truth + progress truth (9 statuses) + recovery priors (★ build-preconditions: attempt logging, recovery history); adversarial 73 (5/20/48); claims 16 (12/4); sources 97/97; FAILED_ONCE; single routing authority — #400 proposes, ADAPT-2 disposes) | — (no dup) | families/metamemory_security.md | memory-atlas-r1/Y-*/ |
|
||||
| 401 | Z: HCI / Mind Palace | research-complete | R1_QUALIFIED (R1_FULL — deep requalified 2026-09-20; batched preserved as preliminary; single-writer stance confirmed; staged earn-in) | — (no dup) | families/mind_palace_hci_multimodal.md | memory-atlas-r1/Z-*/ |
|
||||
| 402 | AA: Security / Poisoning | research-complete | R1_QUALIFIED (R1_FULL — deep requalified 2026-09-20; batched preserved as preliminary; single-writer stance confirmed; staged earn-in) | — (no dup) | families/metamemory_security.md | memory-atlas-r1/AA-*/ |
|
||||
| 403 | AB: Multimodal / Artifacts | research-complete | R1_QUALIFIED (R1_FULL — deep requalified 2026-09-20; batched preserved as preliminary; single-writer stance confirmed; staged earn-in) | — (no dup) | families/mind_palace_hci_multimodal.md | memory-atlas-r1/AB-*/ |
|
||||
|
||||
@@ -874,14 +874,20 @@ def check_claim_evidence_arithmetic(domain_dir, findings):
|
||||
classes = _Counter(_cget(c, "classification", "verdict", "class") for c in claims)
|
||||
elevels = _Counter(_cget(c, "evidence_level", "evidence", "level") for c in claims)
|
||||
base = os.path.basename(domain_dir.rstrip("/"))
|
||||
if not base.startswith("W-"):
|
||||
if not base.startswith(("W-", "X-", "Y-")):
|
||||
return
|
||||
mbody = open(master, encoding="utf-8", errors="replace").read()
|
||||
section = _master_section(mbody, "claim ledger", "## 30")
|
||||
if not section:
|
||||
section = _master_section(mbody, "Claim ledger", "## 30")
|
||||
if not section:
|
||||
section = _master_section(mbody, "claims.json", "## 31")
|
||||
# Bound the section at the next master heading — numeric end markers
|
||||
# lower than the start marker made _master_section scan to EOF, sweeping
|
||||
# unrelated sections' totals (e.g. qualification rows) into this check.
|
||||
section = ""
|
||||
for start, end in (("claim ledger", "## "), ("Claim ledger", "## "), ("claims.json", "## ")):
|
||||
idx = mbody.find(start)
|
||||
if idx < 0:
|
||||
continue
|
||||
nxt = mbody.find(end, idx + len(start))
|
||||
section = mbody[idx:nxt if nxt >= 0 else len(mbody)]
|
||||
break
|
||||
if not section:
|
||||
return
|
||||
total = len(claims)
|
||||
@@ -1182,6 +1188,109 @@ def check_drift_crypto_split(domain_dir, findings):
|
||||
f"split: behavioral tripwire ADOPT, crypto mechanism PARK to #402")
|
||||
|
||||
|
||||
def check_load_bearing_depth_map(domain_dir, findings):
|
||||
"""Y-repair (2026-09-22): every architecture-changing claim must map
|
||||
through load_bearing_evidence.md with a support type; DEEP_EXTERNAL
|
||||
support requires the cited anchor row to actually be FULL_PAPER or
|
||||
RELEVANT_SECTIONS in sources.md (mandate §16A)."""
|
||||
if not os.path.basename(domain_dir.rstrip("/")).startswith("Y-"):
|
||||
return
|
||||
lbm = os.path.join(domain_dir, "load_bearing_evidence.md")
|
||||
cj = os.path.join(domain_dir, "claims.json")
|
||||
if not os.path.exists(lbm) or not os.path.exists(cj):
|
||||
findings.append("Y domain: load_bearing_evidence.md or claims.json missing")
|
||||
return
|
||||
try:
|
||||
import json as _json
|
||||
claims = _json.load(open(cj, encoding="utf-8"))
|
||||
except Exception:
|
||||
return
|
||||
lbm_text = open(lbm, encoding="utf-8", errors="replace").read()
|
||||
src_rows = {}
|
||||
for ln in open(os.path.join(domain_dir, "sources.md"), encoding="utf-8", errors="replace"):
|
||||
if not ln.strip().startswith("|"):
|
||||
continue
|
||||
cells = [c.strip() for c in ln.strip().strip("|").split("|")]
|
||||
if cells and re.fullmatch(r"(S\d+|A\d+-\d+)", cells[0] or ""):
|
||||
src_rows[cells[0]] = ("FULL_PAPER" in ln or "RELEVANT_SECTIONS" in ln)
|
||||
for c in claims:
|
||||
cid = c.get("id", "?")
|
||||
if c.get("classification") not in ("ADOPT", "ADAPT"):
|
||||
continue
|
||||
row = re.search(rf"^\|\s*{re.escape(cid)}\s*\|(.+)$", lbm_text, re.M)
|
||||
if not row:
|
||||
findings.append(f"load_bearing_evidence.md: claim {cid} missing from support map")
|
||||
continue
|
||||
cells = [c.strip() for c in row.group(1).strip().strip("|").split("|")]
|
||||
# columns: rule | inward | anchors | depth | support | ...
|
||||
support = cells[4] if len(cells) > 4 else ""
|
||||
if c.get("classification") == "ADOPT" and support not in (
|
||||
"DEEP_EXTERNAL", "RUNE_DIRECT", "DESIGN_INVARIANT", "EXTERNAL finding (not a Rune ADOPT)"):
|
||||
findings.append(f"claim {cid}: unsupported ADOPT support type '{support}'")
|
||||
if support == "DEEP_EXTERNAL":
|
||||
anchors = re.findall(r"(A\d+-\d+)", cells[2] if len(cells) > 2 else "")
|
||||
if not any(src_rows.get(a) for a in anchors):
|
||||
findings.append(
|
||||
f"claim {cid}: DEEP_EXTERNAL support but no cited anchor ({anchors}) is FULL_PAPER/RELEVANT_SECTIONS in sources.md")
|
||||
|
||||
|
||||
def check_zero_hit_novelty(domain_dir, findings):
|
||||
"""Y-repair (mandate §16B): bounded zero-hit searches must not be promoted
|
||||
into nonexistence claims in canonical artifacts."""
|
||||
if not os.path.basename(domain_dir.rstrip("/")).startswith("Y-"):
|
||||
return
|
||||
_PATTERNS = re.compile(
|
||||
r"confirmed unpublished|does not exist|no published (?:system|detector|mechanism)|"
|
||||
r"first mover|no literature\b|no published taxonomy", re.I)
|
||||
_CONTEXT = ("novelty_unproven", "no direct match found", "repair", "superseded",
|
||||
"citation correction", "non-existent", "bounded-search finding")
|
||||
for fname in ("X_PERSONA_IDENTITY_R1_MASTER.md",):
|
||||
continue # never touch other domains
|
||||
master = find_master(domain_dir)
|
||||
for fname in (os.path.basename(master) if master else None,
|
||||
"evidence_saturation.md", "04_innovation.md"):
|
||||
if not fname:
|
||||
continue
|
||||
path = os.path.join(domain_dir, fname)
|
||||
if not os.path.exists(path):
|
||||
continue
|
||||
for i, ln in enumerate(open(path, encoding="utf-8", errors="replace"), 1):
|
||||
low = ln.lower()
|
||||
if _PATTERNS.search(low) and not any(ctx in low for ctx in _CONTEXT):
|
||||
findings.append(f"{fname}:{i}: zero-hit search promoted to nonexistence claim "
|
||||
f"(must use 'no direct match found in the searched sources' / NOVELTY_UNPROVEN)")
|
||||
|
||||
|
||||
def check_rune_proven_scope(domain_dir, findings):
|
||||
"""Y-repair (mandate §16D): RUNE_PROVEN claims must not carry
|
||||
architecture-performance framing when inward evidence only shows
|
||||
component presence/wiring."""
|
||||
if not os.path.basename(domain_dir.rstrip("/")).startswith("Y-"):
|
||||
return
|
||||
cj = os.path.join(domain_dir, "claims.json")
|
||||
if not os.path.exists(cj):
|
||||
return
|
||||
try:
|
||||
import json as _json
|
||||
claims = _json.load(open(cj, encoding="utf-8"))
|
||||
except Exception:
|
||||
return
|
||||
_PERF = re.compile(r"(?:architecture|metacognition)\s+(?:works|is real)|primitive-but-real", re.I)
|
||||
_NEG = ("not assert", "does not", "never", "not part of")
|
||||
for c in claims:
|
||||
if c.get("evidence_level") != "RUNE_PROVEN":
|
||||
continue
|
||||
text = str(c.get("claim", ""))
|
||||
for m in _PERF.finditer(text):
|
||||
before = text[max(0, m.start() - 80):m.start()].lower()
|
||||
if any(n in before for n in _NEG):
|
||||
continue # negated/disclaimed mention
|
||||
findings.append(
|
||||
f"claim {c.get('id', '?')}: RUNE_PROVEN claim carries architecture-performance framing "
|
||||
f"(inward evidence demonstrates presence/wiring only)")
|
||||
break
|
||||
|
||||
|
||||
def check_repair_hardening(domain_dir):
|
||||
findings = []
|
||||
check_sealed_draft_marker(domain_dir, findings)
|
||||
@@ -1190,6 +1299,9 @@ def check_repair_hardening(domain_dir):
|
||||
check_claim_evidence_split(domain_dir, findings)
|
||||
check_ownership_collapse_phrases(domain_dir, findings)
|
||||
check_drift_crypto_split(domain_dir, findings)
|
||||
check_load_bearing_depth_map(domain_dir, findings)
|
||||
check_zero_hit_novelty(domain_dir, findings)
|
||||
check_rune_proven_scope(domain_dir, findings)
|
||||
return findings
|
||||
|
||||
|
||||
|
||||
Reference in new issue
Block a user