conductor/rune
1
2 Fork 1
Code Issues 36 Pull Requests 30 Actions 45 Packages Projects Releases 564 Wiki Activity

Dimensional review rubric: structured per-dimension scoring for review gates #129

New Issue
Closed
opened 2026-09-04 19:04:58 +01:00 by conductor · 10 comments
conductor commented 2026-09-04 19:04:58 +01:00
Owner
Copy link
Copy source

Plain summary

Restructure the review prompts from holistic judgment ('is this good?') to structured per-dimension forced-choice scoring — the SPB mitigation that measurably reduced bias in the research while preserving discriminability. Each review explicitly scores: correctness, design, test quality, observability, security — with a verdict per dimension.

Example

Instead of 'the code looks fine, approved' → 'Correctness: PASS (36 call sites verified). Design: PASS with concern (registry singleton). Test quality: CONCERN (no real-path test). Observability: PASS. Security: PASS.' — five explicit verdicts the controller can act on dimensionally.

Technical details

WHAT: update the review dispatch templates to decompose into 5 named dimensions with per-dimension verdicts + evidence. The SPB study (arXiv 2604.22891, Fig 5) showed this reduces self-preference bias substantially (structured > CoT > baseline) while maintaining the ability to identify clearly superior work.
WHY: holistic judgment is where self-preference lives; dimensional decomposition anchors assessments to specific quality dimensions instead of overall impressions (cognitive load theory — structured decisions resist heuristic shortcuts).
FILES: review dispatch templates (M); docs/review-lessons.md cross-reference.
ACCEPTANCE: the next review outputs 5 dimension verdicts; the controller can identify WHICH dimension a finding belongs to.
DEPENDENCIES: #121-adjacent (the review-lessons file informs the dimensions).

Pipeline requirements

  • TDD / verification as appropriate
  • Spec-compliance review gate: PASS
  • Code-quality review gate: Approved
  • Full suite regression green
  • Closed with a properly formatted closing comment + commit references
## Plain summary Restructure the review prompts from holistic judgment ('is this good?') to structured per-dimension forced-choice scoring — the SPB mitigation that measurably reduced bias in the research while preserving discriminability. Each review explicitly scores: correctness, design, test quality, observability, security — with a verdict per dimension. ## Example Instead of 'the code looks fine, approved' → 'Correctness: PASS (36 call sites verified). Design: PASS with concern (registry singleton). Test quality: CONCERN (no real-path test). Observability: PASS. Security: PASS.' — five explicit verdicts the controller can act on dimensionally. ## Technical details **WHAT:** update the review dispatch templates to decompose into 5 named dimensions with per-dimension verdicts + evidence. The SPB study (arXiv 2604.22891, Fig 5) showed this reduces self-preference bias substantially (structured > CoT > baseline) while maintaining the ability to identify clearly superior work. **WHY:** holistic judgment is where self-preference lives; dimensional decomposition anchors assessments to specific quality dimensions instead of overall impressions (cognitive load theory — structured decisions resist heuristic shortcuts). **FILES:** review dispatch templates (M); docs/review-lessons.md cross-reference. **ACCEPTANCE:** the next review outputs 5 dimension verdicts; the controller can identify WHICH dimension a finding belongs to. **DEPENDENCIES:** #121-adjacent (the review-lessons file informs the dimensions). ## Pipeline requirements - [ ] TDD / verification as appropriate - [ ] Spec-compliance review gate: PASS - [ ] Code-quality review gate: Approved - [ ] Full suite regression green - [ ] Closed with a properly formatted closing comment + commit references
conductor added the P2 — medium label 2026-09-04 19:05:00 +01:00
gitea-actions Bot commented 2026-09-04 19:05:21 +01:00
Copy link
Copy source

⚠️ Priority gate: this issue has no priority. Assign exactly one of P0 — critical (spine/correctness/blocks), P1 — high (material value or real defect), P2 — medium (quality/consolidation), P3 — low (measurement/deferred). An issue without a priority is not filed.

⚠️ **Priority gate:** this issue has no priority. Assign exactly one of **P0 — critical** (spine/correctness/blocks), **P1 — high** (material value or real defect), **P2 — medium** (quality/consolidation), **P3 — low** (measurement/deferred). An issue without a priority is not filed.
gitea-actions Bot added the custodian label 2026-09-04 19:05:24 +01:00
conductor added P0 — critical and removed P2 — medium labels 2026-09-07 11:14:10 +01:00
gitea-actions Bot commented 2026-09-07 11:15:57 +01:00
Copy link
Copy source

⚠️ Priority gate: this issue has no priority. Assign exactly one of P0 — critical (spine/correctness/blocks), P1 — high (material value or real defect), P2 — medium (quality/consolidation), P3 — low (measurement/deferred). An issue without a priority is not filed.

⚠️ **Priority gate:** this issue has no priority. Assign exactly one of **P0 — critical** (spine/correctness/blocks), **P1 — high** (material value or real defect), **P2 — medium** (quality/consolidation), **P3 — low** (measurement/deferred). An issue without a priority is not filed.
conductor commented 2026-09-07 12:22:48 +01:00
Author
Owner
Copy link
Copy source

#129 review rubric — OUTWARD evidence trail (fetched 2026-09-07)

Evidence pulled live from primary sources (docs fetched, CLIs run); 4 dead URLs encountered during research were substituted with working primaries. This comment is the verifiable source trail for the plan.

**#129 review rubric — OUTWARD evidence trail (fetched 2026-09-07)** - Google standard.md: "improves the overall code health" as normative gate; "Nit:" prefix convention. https://github.com/google/eng-practices/blob/master/review/reviewer/standard.md - Google looking-for.md dimension headings: Design, Functionality, Complexity, Tests, Naming, Comments, Style, Consistency, Documentation, Every Line, Context — ready-made axes. https://github.com/google/eng-practices/blob/master/review/reviewer/looking-for.md - Conventional Comments spec: `<label> [decorations]: <subject>`, (blocking)/(non-blocking), machine-parseable JSON. https://conventionalcomments.org/ Design consequence: rubric = Google's dimensions as scoring axes + Conventional Comments as output schema; decorations give the gate severity semantics. _Evidence pulled live from primary sources (docs fetched, CLIs run); 4 dead URLs encountered during research were substituted with working primaries. This comment is the verifiable source trail for the plan._
gitea-actions Bot added the ready label 2026-09-08 05:29:26 +01:00
gitea-actions Bot commented 2026-09-08 05:29:27 +01:00
Copy link
Copy source

🎓 NEXT: #129 — Dimensional review rubric: structured per-dimension scoring for review gates — dependencies satisfied (closed: #26); Rune may start.

🎓 **NEXT: #129 — Dimensional review rubric: structured per-dimension scoring for review gates** — dependencies satisfied (closed: #26); Rune may start.
gitea-actions Bot commented 2026-09-08 06:23:12 +01:00
Copy link
Copy source

🎓 NEXT: #129 — Dimensional review rubric: structured per-dimension scoring for review gates — dependencies satisfied (scheduled sweep); Rune may start.

🎓 **NEXT: #129 — Dimensional review rubric: structured per-dimension scoring for review gates** — dependencies satisfied (scheduled sweep); Rune may start.
gitea-actions Bot commented 2026-09-19 22:59:41 +01:00
Copy link
Copy source

🎓 NEXT: #129 — Dimensional review rubric: structured per-dimension scoring for review gates — dependencies satisfied (closed: #358); Rune may start.

🎓 **NEXT: #129 — Dimensional review rubric: structured per-dimension scoring for review gates** — dependencies satisfied (closed: #358); Rune may start.
gitea-actions Bot commented 2026-09-23 03:23:36 +01:00
Copy link
Copy source

🎓 NEXT: #129 — Dimensional review rubric: structured per-dimension scoring for review gates — dependencies satisfied (scheduled sweep); Rune may start.

🎓 **NEXT: #129 — Dimensional review rubric: structured per-dimension scoring for review gates** — dependencies satisfied (scheduled sweep); Rune may start.
gitea-actions Bot commented 2026-09-23 04:23:28 +01:00
Copy link
Copy source

🎓 NEXT: #129 — Dimensional review rubric: structured per-dimension scoring for review gates — dependencies satisfied (scheduled sweep); Rune may start.

🎓 **NEXT: #129 — Dimensional review rubric: structured per-dimension scoring for review gates** — dependencies satisfied (scheduled sweep); Rune may start.
gitea-actions Bot commented 2026-09-23 05:23:27 +01:00
Copy link
Copy source

🎓 NEXT: #129 — Dimensional review rubric: structured per-dimension scoring for review gates — dependencies satisfied (scheduled sweep); Rune may start.

🎓 **NEXT: #129 — Dimensional review rubric: structured per-dimension scoring for review gates** — dependencies satisfied (scheduled sweep); Rune may start.
conductor commented 2026-10-08 11:32:53 +01:00
Author
Owner
Copy link
Copy source

CLOSED PROGRAMMATICALLY — commit f9f83e0 (contract 129-side of BUILDER-2026-10-08T10:27:52Z-a8b4ae).

CLOSED PROGRAMMATICALLY — commit f9f83e0 (contract 129-side of BUILDER-2026-10-08T10:27:52Z-a8b4ae).
Sign in to join this conversation.
No Branch/Tag Specified
Labels
Clear labels
P0 — critical
Spine/core correctness or blocks other work — do first
 P1 — high
Material user value or real defect (money, bugs, next-stage dep)
 P2 — medium
Quality/consolidation improvement
 P3 — low
Measurement / benchmark / deferred policy
 blocked
Dependencies open
 bug
Confirmed defect or regression
 custodian
epic
Rune v1 course-correction issue graph
 feature
New capability or improvement
 governance
growth
identity
in-progress
Currently being worked on by Rune
 incomplete
Incomplete work — draft PR with ledger, awaiting resume
 infrastructure
life
memory
memory-atlas
Memory Research Atlas child domain
 needs-testing
Needs manual testing with dumbed-down steps before close
 perception
ready
Order satisfied - Rune may start
 release-drift
research
Research jobs — spikes, watchlists, evidence work
 rune
rune
 scaffolding
scaffolding
 spine,correction
spine,correction
 spine,deferred
spine,deferred
 spine,resolved
spine,resolved
 strangler,burn,endgame,deferred
strangler,burn,endgame,deferred
 strangler,burn,endgame,inventory,epic
strangler,burn,endgame,inventory,epic
 tui
tui
 tui,epic
tui,epic
 tui,openspec
tui,openspec
 v1-course-correction
Rune v1 course-correction issue graph
No labels P0 — critical custodian ready
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
conductor
No Assignees
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: conductor/rune#129