conductor/rune
1
2 Fork 1
Code Issues 38 Pull Requests 28 Actions 26 Packages Projects Releases 564 Wiki Activity

[P3-04] Measure experienced-state transfer across models #47

New Issue
Closed
opened 2026-08-28 20:54:55 +01:00 by conductor · 1 comment
conductor commented 2026-08-28 20:54:55 +01:00
Owner
Copy link
Copy source

Plain summary

Tests the core research hypothesis: take an experienced state and run it under two different models on novel-but-related tasks, versus fresh state on each. Success = the experienced state produces a statistically meaningful advantage under BOTH models.

Example

Model A and Model B both solve novel related tasks better with Rune's accumulated state than without it.

Technical details

Parent epic: #16

Manifest ID: P3-04

Priority: P3 / research milestone

Run:

Model A + fresh state
Model A + experienced state

Model B + fresh state
Model B + same experienced state

Test novel-but-related tasks.

Success criterion: experienced state produces a statistically/repeatedly meaningful advantage under both models.

This is the strongest test of Rune’s core research hypothesis.

Dependencies: P3-01, P2-04, P2-06.

Resolved dependency references

  • Depends on: #44 (P3-01), #38 (P2-04), #40 (P2-06)
  • Blocks: None.

Relevant source files

Not specified in the handoff. P0-01 must identify authoritative paths before implementation; do not guess by adding another path.

Do not do

  • Do not begin this issue before its dependencies are complete.
  • Do not create a parallel authority path; replace, extend, or remain dormant.
  • Do not add unrelated cognitive subsystems or additive feature growth.
  • Do not treat module/schema existence, background activity, or model self-report as acceptance evidence.

Source directive

Preserved from /home/nick/Desktop/rune_v1_course_correction_handoff.md, Section 14. The handoff is authoritative if this issue summary is ambiguous.

Pipeline requirements

  • TDD: frozen/regression tests written first, red → green
  • ruff check and ruff format --check clean
  • Spec-compliance review gate: PASS
  • Code-quality review gate: Approved
  • Full orchestration suite regression green
  • Canary passes (where applicable)
  • Closed with a properly formatted closing comment + commit/PR references
## Plain summary Tests the core research hypothesis: take an experienced state and run it under two different models on novel-but-related tasks, versus fresh state on each. Success = the experienced state produces a statistically meaningful advantage under BOTH models. ## Example Model A and Model B both solve novel related tasks better with Rune's accumulated state than without it. ## Technical details Parent epic: #16 Manifest ID: **P3-04** **Priority:** P3 / research milestone Run: ```text Model A + fresh state Model A + experienced state Model B + fresh state Model B + same experienced state ``` Test novel-but-related tasks. **Success criterion:** experienced state produces a statistically/repeatedly meaningful advantage under both models. This is the strongest test of Rune’s core research hypothesis. **Dependencies:** P3-01, P2-04, P2-06. --- ## Resolved dependency references - Depends on: #44 (P3-01), #38 (P2-04), #40 (P2-06) - Blocks: None. ## Relevant source files Not specified in the handoff. P0-01 must identify authoritative paths before implementation; do not guess by adding another path. ## Do not do - Do not begin this issue before its dependencies are complete. - Do not create a parallel authority path; replace, extend, or remain dormant. - Do not add unrelated cognitive subsystems or additive feature growth. - Do not treat module/schema existence, background activity, or model self-report as acceptance evidence. ## Source directive Preserved from `/home/nick/Desktop/rune_v1_course_correction_handoff.md`, Section 14. The handoff is authoritative if this issue summary is ambiguous. ## Pipeline requirements - [ ] TDD: frozen/regression tests written first, red → green - [ ] `ruff check` and `ruff format --check` clean - [ ] Spec-compliance review gate: PASS - [ ] Code-quality review gate: Approved - [ ] Full orchestration suite regression green - [ ] Canary passes (where applicable) - [ ] Closed with a properly formatted closing comment + commit/PR references
conductor added the P3 — lowv1-course-correction labels 2026-08-28 20:54:55 +01:00
gitea-actions Bot added the featuregrowth labels 2026-08-28 20:56:04 +01:00
gitea-actions Bot added the custodian label 2026-09-03 22:54:51 +01:00
conductor commented 2026-10-08 12:32:28 +01:00
Author
Owner
Copy link
Copy source

CLOSED PROGRAMMATICALLY — commit 0307083 (contract batch 197587/a78c0a/31d506). See scripts/warm_ablation.py, scripts/growth_delta.py, scripts/transfer_experiment.py + tests/test_p3_benchmarks.py (9/9 green).

CLOSED PROGRAMMATICALLY — commit 0307083 (contract batch 197587/a78c0a/31d506). See scripts/warm_ablation.py, scripts/growth_delta.py, scripts/transfer_experiment.py + tests/test_p3_benchmarks.py (9/9 green).
Sign in to join this conversation.
No Branch/Tag Specified
Labels
Clear labels
P0 — critical
Spine/core correctness or blocks other work — do first
 P1 — high
Material user value or real defect (money, bugs, next-stage dep)
 P2 — medium
Quality/consolidation improvement
 P3 — low
Measurement / benchmark / deferred policy
 blocked
Dependencies open
 bug
Confirmed defect or regression
 custodian
epic
Rune v1 course-correction issue graph
 feature
New capability or improvement
 governance
growth
identity
in-progress
Currently being worked on by Rune
 incomplete
Incomplete work — draft PR with ledger, awaiting resume
 infrastructure
life
memory
memory-atlas
Memory Research Atlas child domain
 needs-testing
Needs manual testing with dumbed-down steps before close
 perception
ready
Order satisfied - Rune may start
 release-drift
research
Research jobs — spikes, watchlists, evidence work
 rune
rune
 scaffolding
scaffolding
 spine,correction
spine,correction
 spine,deferred
spine,deferred
 spine,resolved
spine,resolved
 strangler,burn,endgame,deferred
strangler,burn,endgame,deferred
 strangler,burn,endgame,inventory,epic
strangler,burn,endgame,inventory,epic
 tui
tui
 tui,epic
tui,epic
 tui,openspec
tui,openspec
 v1-course-correction
Rune v1 course-correction issue graph
No labels P3 — low custodian feature growth v1-course-correction
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
conductor
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: conductor/rune#47