conductor/rune
1
2 Fork 1
Code Issues 38 Pull Requests 32 Actions 13 Packages Projects Releases 564 Wiki Activity

RP13 (child of epic #257). DEPENDS ON: RP10, RP12. BLOCKS: RP14. ACCEPTANCE: - Recorded comparison of stage times, cognition vs orchestration/wait time, sources read, refinement loops, issue completion time vs the old OUTWARD_DISCOVERY/OUTWARD_READIN… #276

New Issue
Closed
opened 2026-09-08 23:36:19 +01:00 by conductor · 3 comments
conductor commented 2026-09-08 23:36:19 +01:00
Owner
Copy link
Copy source
No description provided.
conductor commented 2026-09-08 23:37:31 +01:00
Author
Owner
Copy link
Copy source

Parent epic: #257 (Research Runtime P0 — canonical pipeline amendment). Canonical handoff 2026-09-08.

Parent epic: #257 (Research Runtime P0 — canonical pipeline amendment). Canonical handoff 2026-09-08.
gitea-actions Bot commented 2026-09-08 23:38:34 +01:00
Copy link
Copy source

⚠️ Priority gate: this issue has no priority. Assign exactly one of P0 — critical (spine/correctness/blocks), P1 — high (material value or real defect), P2 — medium (quality/consolidation), P3 — low (measurement/deferred). An issue without a priority is not filed.

⚠️ **Priority gate:** this issue has no priority. Assign exactly one of **P0 — critical** (spine/correctness/blocks), **P1 — high** (material value or real defect), **P2 — medium** (quality/consolidation), **P3 — low** (measurement/deferred). An issue without a priority is not filed.
conductor commented 2026-09-09 13:53:22 +01:00
Author
Owner
Copy link
Copy source

RP13: throughput/quality comparison (closes #276)

Report: docs/RP13_THROUGHPUT_REPORT.json on branch rp13-metrics.

Measured (two real runs, post-cleanup canonical path):

Metric #161 #163
Stages executed 7 (+3 CI-contract fix loops) 10 (incl. INNOVATION + REFINE round)
MiMo tool calls 101 103
External queries 4 5
Sources discovered / read w/ receipts 16 / 4 45 / 10
Contradictions / limitations captured 4 / 8 4 / 8
Adversarial attacks → survivors → rejected 16 → 6 → 1 13 → 4 → 1
Mitigations / kill-tests carried to synthesis 10 / 7 7 / 6
Refinement loops 0 1 (REFINE_SYNTHESIS, targeted)
Implementation rework 0 (merged first try) n/a (design-stage)
Wall-clock research chain ~75 min (git window 00:16→01:31, incl. live CI-defect fixes) per-stage elapsed recorded (OUTWARD→QUALIFY ≈ 28 min cognition)

Old-process baseline (recorded RSC0 evidence): foreground slept 300s×N while child MiMo was productive; receipts were model-claimed; snippets counted as evidence; no durable state across restarts (issues #245/#254/#255 each restarted from scratch).

Quality outcomes (what the pipeline PREVENTED):

  • #161: MemoryAdapter duplicate-authority rejected before coding; silent non-hybrid benchmark validation prevented (FT1); cache-pollution hazard neutralized (A4)
  • #163: external graph DB adoption blocked (3 CONFLICTS_WITH_RUNE); false bi-temporal premise corrected from code before synthesis; INN-1 falsified and corrected in synthesis v2

Cognition vs orchestration: cognition = recorded stage elapsed (101/103 tool calls; QUALIFY 310s on #161; APPLICABILITY 300s, INNOVATION 422s, SYNTHESIS 677s on #163). Orchestration/wait (runner queue, CI) excluded from pipeline cost.

KEEP / TUNE / REMOVE:

  • KEEP: unified OUTWARD w/ receipts (evidence chain held under adversarial audit), adversarial disposition contract (caught false premises on both runs), narrowed QUALIFY (caught 3 real gaps without reopening research), placement gate
  • TUNE: per-stage elapsed persistence (reconciliation zeroes earlier stages — record at advance() time); OUTWARD file-channel should be the PRIMARY documented channel (stdout parse is the fallback); prose-only stdout still costs one gate-retry
  • REMOVE: nothing further — RP11 retirement complete; no remaining competing authority

Honest data note: elapsed_s zeroed for earlier stages on #161 due to reconciliation overwrites — timing persistence at advance() is the fix (TUNE).

## RP13: throughput/quality comparison (closes #276) Report: `docs/RP13_THROUGHPUT_REPORT.json` on branch `rp13-metrics`. **Measured (two real runs, post-cleanup canonical path):** | Metric | #161 | #163 | |---|---|---| | Stages executed | 7 (+3 CI-contract fix loops) | 10 (incl. INNOVATION + REFINE round) | | MiMo tool calls | 101 | 103 | | External queries | 4 | 5 | | Sources discovered / read w/ receipts | 16 / 4 | 45 / 10 | | Contradictions / limitations captured | 4 / 8 | 4 / 8 | | Adversarial attacks → survivors → rejected | 16 → 6 → 1 | 13 → 4 → 1 | | Mitigations / kill-tests carried to synthesis | 10 / 7 | 7 / 6 | | Refinement loops | 0 | 1 (REFINE_SYNTHESIS, targeted) | | Implementation rework | 0 (merged first try) | n/a (design-stage) | | Wall-clock research chain | ~75 min (git window 00:16→01:31, incl. live CI-defect fixes) | per-stage elapsed recorded (OUTWARD→QUALIFY ≈ 28 min cognition) | **Old-process baseline** (recorded RSC0 evidence): foreground slept 300s×N while child MiMo was productive; receipts were model-claimed; snippets counted as evidence; no durable state across restarts (issues #245/#254/#255 each restarted from scratch). **Quality outcomes (what the pipeline PREVENTED):** - #161: MemoryAdapter duplicate-authority rejected before coding; silent non-hybrid benchmark validation prevented (FT1); cache-pollution hazard neutralized (A4) - #163: external graph DB adoption blocked (3 CONFLICTS_WITH_RUNE); false bi-temporal premise corrected from code before synthesis; INN-1 falsified and corrected in synthesis v2 **Cognition vs orchestration:** cognition = recorded stage elapsed (101/103 tool calls; QUALIFY 310s on #161; APPLICABILITY 300s, INNOVATION 422s, SYNTHESIS 677s on #163). Orchestration/wait (runner queue, CI) excluded from pipeline cost. **KEEP / TUNE / REMOVE:** - KEEP: unified OUTWARD w/ receipts (evidence chain held under adversarial audit), adversarial disposition contract (caught false premises on both runs), narrowed QUALIFY (caught 3 real gaps without reopening research), placement gate - TUNE: per-stage elapsed persistence (reconciliation zeroes earlier stages — record at advance() time); OUTWARD file-channel should be the PRIMARY documented channel (stdout parse is the fallback); prose-only stdout still costs one gate-retry - REMOVE: nothing further — RP11 retirement complete; no remaining competing authority Honest data note: elapsed_s zeroed for earlier stages on #161 due to reconciliation overwrites — timing persistence at advance() is the fix (TUNE).
Sign in to join this conversation.
No Branch/Tag Specified
Labels
Clear labels
P0 — critical
Spine/core correctness or blocks other work — do first
 P1 — high
Material user value or real defect (money, bugs, next-stage dep)
 P2 — medium
Quality/consolidation improvement
 P3 — low
Measurement / benchmark / deferred policy
 blocked
Dependencies open
 bug
Confirmed defect or regression
 custodian
epic
Rune v1 course-correction issue graph
 feature
New capability or improvement
 governance
growth
identity
in-progress
Currently being worked on by Rune
 incomplete
Incomplete work — draft PR with ledger, awaiting resume
 infrastructure
life
memory
memory-atlas
Memory Research Atlas child domain
 needs-testing
Needs manual testing with dumbed-down steps before close
 perception
ready
Order satisfied - Rune may start
 release-drift
research
Research jobs — spikes, watchlists, evidence work
 rune
rune
 scaffolding
scaffolding
 spine,correction
spine,correction
 spine,deferred
spine,deferred
 spine,resolved
spine,resolved
 strangler,burn,endgame,deferred
strangler,burn,endgame,deferred
 strangler,burn,endgame,inventory,epic
strangler,burn,endgame,inventory,epic
 tui
tui
 tui,epic
tui,epic
 tui,openspec
tui,openspec
 v1-course-correction
Rune v1 course-correction issue graph
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
conductor
No Assignees
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: conductor/rune#276