Compare commits

...
168 Commits
Author SHA1 Message Date
conductor d6acdd5043 Merge pull request 'Plan 7: Old-schema migration (X1-X5) — series complete' (#107) from schema-migration into master 2026-09-04 11:51:03 +01:00
Nick 7a44a4eff0 chore(release): bump version to 0.49.0
Orchestration series Plan 7 (old-schema migration, PR to follow): legacy
event envelope/bus/pins now delegate to the canonical orchestration layer;
migration CLI + canary + docs + CI. Auto-version reads this on merge.
2026-09-04 11:45:46 +01:00
Nick 220a0220a8 fix(migration): flush per record, surface skipped counts, skip status-less pins
Discharges all X5 quality Importants: evidence-orphan window closed
(flush per record), legacy pins without [status] skipped not guessed,
skipped counts surfaced by both migrators + CLI. Red-green proven.
Suite 421->422. Minors pinned to hardening plan.
2026-09-04 11:44:24 +01:00
Nick a39f04115e docs(plan): record X5 execution-time deviations 2026-09-04 11:16:06 +01:00
Nick 8c12552b2e feat(migration): add migration CLI, canary, docs, CI 2026-09-04 11:02:02 +01:00
Nick ea1554d14f docs(plan): discharge X1/X4 pins into X5 spec pre-dispatch 2026-09-04 10:32:47 +01:00
Nick 559fca5464 fix(migration): recall empty context abstains (matches-all edge)
Discharges X4 I4 edge: recall('') matched every open pin; empty context now
returns [] (abstention). Red-green proven. Pins tests 4->5; suite 416->417.
C7 legacy-pins orphan + other Importants pinned to X5 (conversion, ranking,
concurrency, resolve semantics).
2026-09-04 03:30:14 +01:00
Nick efdbdf6440 docs(plan): record X4 canonical PinRecord extension and consumer adaptation 2026-09-04 00:43:35 +01:00
Nick b5b90c7a50 feat(migration): delegate legacy pins to canonical PinRecord storage 2026-09-04 00:29:16 +01:00
Nick 25ed343826 fix(migration): reject non-string source in event_bus emit
Discharges X3 I-1: non-str source silently wrote a dict partition_key
(poison record; TUI emit('on_context', {dict}) was the live writer). emit
now raises ValueError; TUI passes the dict as payload. Red-green proven.
Legacy tests 3->4; suite 411->412. I-2 (O(n) store init per emit) pinned
to X5 (compaction/offset persistence).
2026-09-04 00:22:24 +01:00
Nick 3dbb40e68e docs(plan): record X3 required TUI adaptation and pinned emit bug 2026-09-04 00:16:16 +01:00
Nick 87813737d3 feat(migration): delegate legacy EventBus to DurableEventStore 2026-09-04 00:15:20 +01:00
Nick 88e61a80c0 fix(migration): typed canonical-bridge error and CHANNELS-prior event_type in shim 2026-09-04 00:10:03 +01:00
Nick 76000f8c07 docs(plan): record X2 execution-time fixes and legacy-suite note 2026-09-04 00:04:32 +01:00
Nick a489f53ec2 feat(migration): delegate legacy EventEnvelope to canonical records 2026-09-04 00:03:38 +01:00
Nick 311495a86f fix(orchestration): validate dict path via canonical round-trip in append 2026-09-04 00:00:18 +01:00
Nick 7e9587c2d6 docs(plan): record X1 execution-time defects and amend blocks 2026-09-03 23:55:15 +01:00
Nick 0a097aa5ef feat(orchestration): add old-fixture migration with dialect guard 2026-09-03 23:54:28 +01:00
Nick 94c06e7664 docs(plan): old-schema migration implementation plan
Final plan of the orchestration series: migrate legacy event_envelope /
event_bus / pins to the canonical orchestration schema — old fixture migrates
once (idempotent), new store rejects unmigrated dialects with actionable
errors, legacy writers delegate (no dialect bazaar). Target suite 243.
2026-09-03 23:51:08 +01:00
conductor 97b88d872f Merge pull request 'Plan 6: Surface parity (S1-S5)' (#98) from surface-parity into master 2026-09-03 23:47:33 +01:00
Nick de9b48c793 chore(release): bump version to 0.48.0
Orchestration series Plan 6 (surface parity, PR to follow): 5 new modules +
canary + CI wiring. Auto-version reads this on merge (guard fails loud if
un-bumped).
2026-09-03 23:42:38 +01:00
Nick 06d39a811a fix(orchestration): make restart-orphan discharge test non-vacuous
Discharges C1 from the S5 quality gate (general-134 NOT approved): the
restart test now saves the registry before rebuild and asserts the orphan
detector + first-entry-after-restart key rejection, so it FAILS without
save/load (red-green proven). Pins C2-C6 (auto-persist, typed corrupt-load,
canary reload, docs wording, composition shape) to the real-transport plan.
2026-09-03 23:40:06 +01:00
Nick 4ffe463bcd docs(plan): record S5 execution-time import fixes 2026-09-03 23:33:05 +01:00
Nick 1061e86265 feat(orchestration): compose surface parity with canary and CI coverage 2026-09-03 23:32:30 +01:00
Nick b3669bb48f docs(plan): discharge S4 restart-orphan pin in S5 spec pre-dispatch 2026-09-03 23:28:55 +01:00
Nick b68fdcea76 docs(plan): pin S4 restart-orphan and peer-posture obligations to S5 2026-09-03 23:27:11 +01:00
Nick 802d0fbe85 docs(plan): reconcile SurfaceRegistry.get bidirectional identity (S4 defect) 2026-09-03 23:22:38 +01:00
Nick 9520160a1a feat(orchestration): add surface gateway with peer verification 2026-09-03 23:21:58 +01:00
Nick 210fe4da38 docs(plan): gateway re-reads stored mapping after link (S3 pin) 2026-09-03 23:19:16 +01:00
Nick 84eb94185c fix(orchestration): raise typed CorruptMappingError on malformed store
Discharges S3 quality finding: _load wraps JSONDecodeError / non-list root /
bad item in CorruptMappingError (house convention: G1 CorruptOutboxError,
CorruptEventStoreError). Mapping tests 4->7; suite 216.
2026-09-03 23:18:14 +01:00
Nick 15a9f9bce9 feat(orchestration): add durable bidirectional conversation mapping 2026-09-03 23:13:06 +01:00
Nick d882b073f0 docs(plan): pin S2 structured-absence obligations to Task 5 2026-09-03 23:11:14 +01:00
Nick 9e6235907a docs(plan): record S2 frozen-test literal fix and amend block 2026-09-03 23:07:40 +01:00
Nick 38f44f4e27 feat(orchestration): add capability negotiation with structured absence 2026-09-03 23:06:35 +01:00
Nick ecdb990c2d fix(orchestration): guard surface registry collisions and round-trip capabilities
Discharges all S1 quality-gate findings: external-id collision now raises
(F1), re-register is an explicit capability refresh (F2), registers store a
defensive copy (F3), Capability is canonical in records.py and SurfaceRecord
round-trips Capability objects (F4). Surface tests 5→9; suite 205.
2026-09-03 23:02:46 +01:00
Nick 89ac8ed94a feat(orchestration): add surface records and registry 2026-09-03 22:57:14 +01:00
Nick d7224df678 docs(plan): surface parity plan complete with gate v4 templates 2026-09-03 22:55:13 +01:00
Nick 94ad6f7bd2 docs(gate): add description gate v4 templates — plain summary, example, pipeline requirements
Opening format (feature/bug templates): Plain summary -> Example -> Technical
details (WHAT/WHY/FILES/ACCEPTANCE/DEPENDENCIES) -> Pipeline requirements
checklist. Closing format (CLOSING_COMMENT_TEMPLATE): Plain summary of what was
delivered -> Technical details (commits, spec/quality gates + reviewer ids,
tests + suite before/after, canary, merged PR) -> Pins/follow-ups. PR template
updated to match. Every future issue inherits these at creation.
2026-09-03 22:54:10 +01:00
conductor 2e9dbc0069 Merge pull request 'Plan 5: Gitea delegation and outbox (G1-G6)' (#91) from gitea-delegation into master 2026-09-03 22:47:07 +01:00
Nick b5b2b82677 chore(release): bump version to 0.47.0
Orchestration series Plan 5 (Gitea delegation and outbox, PR to follow):
new orchestration modules + 2 canaries + CI wiring. Auto-version reads this
version on merge (the guard fails loudly if un-bumped).
2026-09-03 22:42:17 +01:00
Nick 80dd313c93 docs(plan): require plain summary + formatted details in closing comments 2026-09-03 22:41:56 +01:00
Nick af062c7561 fix(orchestration): pin copy-independence and raise-recovery in delegation tests 2026-09-03 22:41:56 +01:00
Nick e1bd1b96c4 docs(plan): require plain-language summary in issue description gate 2026-09-03 22:41:56 +01:00
Nick 7ff95b8fa1 docs(plan): establish issue description gate as standing rule 2026-09-03 22:41:56 +01:00
Nick 87b584b5b4 feat(orchestration): compose Gitea delegation with canary and CI coverage 2026-09-03 22:41:56 +01:00
Nick b62583037c docs(plan): correct G6 test count in summary table 2026-09-03 22:41:56 +01:00
Nick 739e8ba684 docs(plan): discharge G3/G2 pins in G6 spec pre-dispatch 2026-09-03 22:41:56 +01:00
Nick 040943cc04 docs(orchestration): document persistent conflict arming and pin replay ordering 2026-09-03 22:41:56 +01:00
Nick c9b1f5fdba feat(orchestration): add Gitea transport protocol and fake 2026-09-03 22:41:56 +01:00
Nick cb7ad5d760 docs(plan): declare G4 transport contract in Task 5 block pre-dispatch 2026-09-03 22:41:56 +01:00
Nick 003c31b072 docs(orchestration): state transport idempotency contract at reconciler call site 2026-09-03 22:41:56 +01:00
Nick 64d23efb43 feat(orchestration): add outbox reconciler with deterministic state mapping 2026-09-03 22:41:56 +01:00
Nick aecfeb7f41 fix(orchestration): include payload hash in mutation ID canon 2026-09-03 22:41:56 +01:00
Nick ff28727269 docs(plan): correct suite counts for G1/G2 discharge tests 2026-09-03 22:41:56 +01:00
Nick f0dbadd9f4 feat(orchestration): add delegation planner for ordered Gitea mutations 2026-09-03 22:41:56 +01:00
Nick 632a5f889d fix(orchestration): allow sending->sending for crash recovery 2026-09-03 22:41:56 +01:00
Nick ccf41757e6 feat(orchestration): add outbox lifecycle state machine 2026-09-03 22:41:56 +01:00
Nick cf33f4cf68 docs(plan): record G1 discharge and corrected suite target 2026-09-03 22:41:56 +01:00
Nick 2cc62b9339 fix(orchestration): validate outbox load types and pin corruption contract 2026-09-03 22:41:56 +01:00
Nick aeb6e5ee8a docs(plan): record G1 outbox corruption-wrap fix 2026-09-03 22:41:56 +01:00
Nick b13cfa5c59 feat(orchestration): add append-only outbox store with stable IDs 2026-09-03 22:41:56 +01:00
Nick 2381e7b649 docs(plan): Gitea delegation and outbox implementation plan
Plan 5 of the orchestration series: append-only outbox (S4-10), lifecyce
state machine (S9), delegation planner (S4-5), reconciler with transport-owned
reconnection version comparison (S4-10, S6), transport protocol + fake, and
composition with canary/docs/CI. Target suite 182.
2026-09-03 22:41:56 +01:00
conductor f574077c1b Merge pull request 'chore(release): bump version to 0.46.0' (#83) from chore/release-0.46.0 into master 2026-09-03 21:36:37 +01:00
Nick b1e079e635 chore(release): sync TUI banner, canary pin, and README to v0.46.0
The reviewed-semantic-version policy surfaces live in five places:
src/rune/__init__.py, rune_tui.py banner + status line, the gitea
canary's exact pin, and README. A bump must carry all of them.
2026-09-03 21:31:14 +01:00
Nick 88421fade8 chore(release): bump version to 0.46.0
Orchestration series (Plans 1-4): foundation, cognitive strata, resource
authority, memory and synthesis — a feature set merged via PRs #55, #66,
#74, #82. Auto-version now guards against unreleased versions so a missed
bump fails loudly instead of 409ing at release creation.
2026-09-03 21:28:07 +01:00
conductor 202477e11c Merge pull request 'Plan 4: memory and synthesis (M1-M6)' (#82) from memory-synthesis into master 2026-09-03 21:25:10 +01:00
Nick b8b93b24d1 fix(orchestration): marks-first ordering and unique-id dedupe in synthesize 2026-09-03 21:19:51 +01:00
Nick 89df26a7b2 feat(orchestration): compose memory and synthesis with canary and CI coverage 2026-09-03 21:13:08 +01:00
Nick 9156a5832f docs(plan): discharge M1/M2/M4 obligations in Task 6 spec pre-dispatch 2026-09-03 21:08:55 +01:00
Nick d7266f8053 docs(plan): pin M5 atomic-store obligations for Task 6 2026-09-03 21:04:27 +01:00
Nick 29a6328664 feat(orchestration): add atomic hot-state writes 2026-09-03 21:01:40 +01:00
Nick f2ccdb03c8 docs(plan): pin M4 budget obligations for Tasks 5-6 2026-09-03 20:59:41 +01:00
Nick 4598c9760b feat(orchestration): add identity pin budgets with visible overflow 2026-09-03 20:57:04 +01:00
Nick 6f4130daa6 docs(plan): pin M3 chain obligations for Tasks 5-6 2026-09-03 20:55:27 +01:00
Nick 96314fc185 docs(plan): record M3 ancestors() guard fix and amend block 2026-09-03 20:53:03 +01:00
Nick c8080b3210 feat(orchestration): add cumulative synthesis chain 2026-09-03 20:52:37 +01:00
Nick 06a24c6661 docs(plan): pin M2 cursor obligations for Tasks 5-6 2026-09-03 20:50:57 +01:00
Nick 3e371259df docs(plan): record M2 cursor commit-path defect and fix 2026-09-03 20:47:03 +01:00
Nick a74132a76c feat(orchestration): add item-id transactional synthesis cursor 2026-09-03 20:46:30 +01:00
Nick 9436f90f8e docs(plan): pin M1 Task 6 obligations for synthesis composition 2026-09-03 20:44:12 +01:00
Nick 56901bf551 feat(orchestration): add episodic memory store 2026-09-03 20:38:10 +01:00
Nick a3cb80751c fix(plan): drop unused field import in M1 block 2026-09-03 20:36:55 +01:00
Nick a609655507 docs(plan): add memory and synthesis implementation plan 2026-09-03 20:35:59 +01:00
conductor 45ac361421 Merge pull request 'feat(orchestration): resource authority — registry, eligibility, router, scheduler, broker' (#74) from resource-authority into master 2026-09-03 20:30:07 +01:00
Nick b201cb8d5d docs(plan): mark R3 deferrals discharged and reconcile inventory interface 2026-09-03 20:25:02 +01:00
Nick 958e7ecf69 fix(orchestration): discharge FixedRouter unavailability and router policy identity 2026-09-03 20:23:31 +01:00
Nick 41626a0246 feat(orchestration): compose resource authority with canary and CI coverage 2026-09-03 20:16:02 +01:00
Nick ccaeb8c65b fix(plan): correct broker provider call and depth example 2026-09-03 20:13:12 +01:00
Nick d430e80bd1 feat(orchestration): add resource broker with subagent inheritance 2026-09-03 20:10:55 +01:00
Nick bb63fe6379 docs(plan): set Task 6 discharge deadline for R3 deferrals 2026-09-03 20:07:53 +01:00
Nick cdc4959085 feat(orchestration): add resource scheduler with bounded fallback 2026-09-03 20:02:21 +01:00
Nick c25ce766f2 docs(plan): pin R3 FixedRouter notes for Task 4 2026-09-03 20:00:49 +01:00
Nick 98dc667774 fix(plan): correct R3 score-tie assertion and record defect 2026-09-03 19:55:57 +01:00
Nick 99b38fe97f feat(orchestration): add advisory resource router 2026-09-03 19:54:47 +01:00
Nick ef05df0fa8 docs(plan): pin R2 requirement-validation notes for Tasks 3-4 2026-09-03 19:52:28 +01:00
Nick 56a537cb98 feat(orchestration): add deterministic eligibility filter 2026-09-03 19:39:34 +01:00
Nick 3e51916d05 docs(plan): pin R1 snapshot lossiness notes 2026-09-03 19:38:16 +01:00
Nick 7fe1303846 feat(orchestration): add resource state registry 2026-09-03 19:34:25 +01:00
Nick 8dc6657ed8 docs(plan): add resource authority implementation plan 2026-09-03 19:32:59 +01:00
conductor ffc21014c6 Merge pull request 'feat(orchestration): cognitive strata — reflex layer and independent cadences' (#66) from cognitive-strata into master 2026-09-03 19:28:45 +01:00
Nick 5af60b1c9b docs(orchestration): correct cognitive canary docstring scope 2026-09-03 19:23:29 +01:00
Nick 9854ed4447 fix(plan): align Task 8 pending_proposals semantics with frozen test 2026-09-03 19:17:05 +01:00
Nick 80f59eeff7 feat(orchestration): compose cognitive strata with canary and CI coverage 2026-09-03 19:17:01 +01:00
Nick 553f8e4e7a fix(plan): drop unused field import from dataclass blocks 2026-09-03 19:03:31 +01:00
Nick 8b3196ab96 feat(orchestration): add cadence health ledger 2026-09-03 19:02:15 +01:00
Nick 3aba512aba fix(plan): type deadline_s as float|None in Task 6 interface 2026-09-03 19:00:42 +01:00
Nick 0bd96a0dd6 feat(orchestration): add deadline policy for forced cadence runs 2026-09-03 18:56:02 +01:00
Nick b1ef707a96 docs(plan): pin cadence-keying convention from C5 review 2026-09-03 18:54:40 +01:00
Nick ef298aa905 fix(plan): correct workers test count and note lint-bytes finding 2026-09-03 18:48:23 +01:00
Nick aede786b65 feat(orchestration): add cognitive worker registry with independent toggles 2026-09-03 18:43:41 +01:00
Nick 54a5c08044 fix(plan): use public worker() accessor in Task 5 run_all 2026-09-03 18:39:47 +01:00
Nick c8c9bc3c83 docs(plan): carry C4 review notes into Task 5/8 wiring 2026-09-03 18:37:34 +01:00
Nick f0eaf98e28 feat(orchestration): add bounded proposal sink with dead letter 2026-09-03 18:32:57 +01:00
Nick e837198267 docs(plan): carry C3 review notes into Task 8 composition 2026-09-03 18:30:20 +01:00
Nick 3ad7550db7 fix(plan): align Task 3 CountReflex with frozen tests (S8 no-change suppression) 2026-09-03 18:25:53 +01:00
Nick b3a54355c5 feat(orchestration): add deterministic reflex engine 2026-09-03 18:23:25 +01:00
Nick 5c20383932 fix(orchestration): never skip a first run on None fingerprint 2026-09-03 18:17:49 +01:00
Nick 4fde382ff5 feat(orchestration): add content fingerprint and no-change gate 2026-09-03 17:52:25 +01:00
Nick ecb50bbaa6 fix(plan): reconcile cursor interface with implemented API 2026-09-03 17:50:11 +01:00
Nick d04e9975ac feat(orchestration): add cadence config and durable cursor 2026-09-03 17:44:51 +01:00
Nick c2fa98f9a6 fix(plan): correct cognitive strata defects after integrity scan 2026-09-03 17:43:16 +01:00
conductor 917cc38eed Merge pull request 'feat(orchestration): foundation event/effect spine for planning and task orchestration' (#55) from orchestration-foundation into master 2026-09-03 17:21:55 +01:00
Nick 619e864cc7 style(orchestration): apply ruff format to orchestration modules 2026-09-03 17:16:31 +01:00
Nick 5a87a45c7c ci(orchestration): cover foundation canary and tests in required checks 2026-09-03 17:13:16 +01:00
Nick aa43455bc6 docs(plan): tick completed task steps in foundation plan 2026-09-03 17:12:09 +01:00
Nick ab188ab54e fix(plan,docs): prefix canary command with PYTHONPATH and guard it in test 2026-09-03 17:12:09 +01:00
Nick c907da7f10 fix(plan,docs): use collection-safe pytest command for orchestration suite 2026-09-03 17:12:09 +01:00
Nick 848927ba96 feat(orchestration): add foundation canary and documentation 2026-09-03 17:12:09 +01:00
Nick 0d429dd3d1 docs(orchestration): document fresh-root requirement for golden fixture 2026-09-03 17:12:09 +01:00
Nick 36b44ce598 feat(orchestration): add golden capability fixture 2026-09-03 17:12:09 +01:00
Nick 3bbf9cf96d fix(plan): remove dead receipt line and correct fixture count 2026-09-03 17:12:09 +01:00
Nick bb1f78be96 docs(plan): pin deferred benchmark obligations from Task 8 review 2026-09-03 17:12:09 +01:00
Nick 3ab5547f75 fix(plan): correct benchmark test data for noise floor and pairing 2026-09-03 17:12:09 +01:00
Nick 5117e2065b feat(orchestration): add benchmark harness with p95 regression rule 2026-09-03 17:12:09 +01:00
Nick 6371db5308 docs(plan): reflect hot bus hardening test 2026-09-03 17:12:09 +01:00
Nick e549efb46b fix(orchestration): lock durability-before-notify and typed read-back failures 2026-09-03 17:12:09 +01:00
Nick ba89436e17 fix(plan): reconcile hot bus interface without payload_digest 2026-09-03 17:12:09 +01:00
Nick c962cbab5c feat(orchestration): add hot bus publishing after durable commit 2026-09-03 17:12:09 +01:00
Nick 684e555b01 fix(plan): correct evidence and projection expected counts 2026-09-03 17:12:09 +01:00
Nick 001fd5c7e3 fix(plan): correct bus and fixture expected test counts 2026-09-03 17:12:09 +01:00
Nick a66262ee1e docs(plan): reflect selector approved-gate test 2026-09-03 17:12:09 +01:00
Nick b6bf3ff3d0 fix(orchestration): honor approved flag in successor selector
select_next previously gated eligibility only on approved_scope
membership, ignoring BatchUnit.approved. A unit is now eligible only
when it is in the approval scope AND approved is True AND all
dependencies are verified.
2026-09-03 17:12:09 +01:00
Nick 89d4533474 docs(plan): correct selector test count 2026-09-03 17:12:09 +01:00
Nick cb47ced62e feat(orchestration): add deterministic successor selector 2026-09-03 17:12:09 +01:00
Nick 7591a4daaf docs(plan): reflect composition root test counts 2026-09-03 17:12:09 +01:00
Nick 4f858a258d fix(orchestration): reject undeclared kinds at register time 2026-09-03 17:12:09 +01:00
Nick 71843b6e8e feat(orchestration): add single-authority composition root 2026-09-03 17:12:09 +01:00
Nick 1df750caef docs(plan): reflect projection hardening tests in expected counts 2026-09-03 17:12:09 +01:00
Nick fb644d3efc fix(orchestration): validate checkpoint hash and offset during rebuild 2026-09-03 17:12:09 +01:00
Nick 5d97399128 fix(plan): correct Task 4 rebuild test to pass full event log 2026-09-03 17:12:09 +01:00
Nick 7a4ca9e233 feat(orchestration): add checkpointed projection engine 2026-09-03 17:12:09 +01:00
Nick feed6e16c2 docs(plan): reflect evidence hardening tests in expected counts 2026-09-03 17:12:09 +01:00
Nick 039668cd55 fix(orchestration): validate evidence digests and verify blob content 2026-09-03 17:12:09 +01:00
Nick 2e3ed5440a feat(orchestration): add content-addressed evidence store 2026-09-03 17:12:09 +01:00
Nick bb3c19cc0f docs(plan): correct event store test count 2026-09-03 17:12:09 +01:00
Nick b2f67cf355 feat(orchestration): add durable batched event store with replay 2026-09-03 17:12:09 +01:00
Nick 4bf2fd4a31 fix(orchestration): preserve tuple fields through JSON round-trip 2026-09-03 17:12:09 +01:00
Nick 3dce56a46d docs(plan): reflect added record coverage in expected counts 2026-09-03 17:12:09 +01:00
Nick 11954962f8 test(orchestration): cover round-trip for proposal, resource, evidence records 2026-09-03 17:12:09 +01:00
Nick 9cc11f2060 fix(plan): correct records schema field and test counts 2026-09-03 17:12:09 +01:00
Nick b76a8f4a4c feat(orchestration): add canonical orchestration records (S9)
Immutable typed records with stable IDs, schema_version, and dict
round-trip. EventEnvelope, EvidenceCapsule, ProposalRecord,
EffectRecord, ReceiptRecord, ApprovalRecord, PinRecord, OutboxRecord,
ResourceSnapshot, ProjectionCheckpoint, plus new_id().
2026-09-03 17:12:09 +01:00
Nick 3b5c5c6c85 docs(orchestration): approve planning design, Souveraine survey, and foundation plan 2026-09-03 17:12:09 +01:00
conductor b78e026480 Merge pull request 'fix(gitea): tolerate concurrent label removal' (#54) from fix/auto-label-race into master 2026-08-31 18:05:53 +01:00
Nick 2844665bc3 fix(gitea): tolerate concurrent label removal
Treat an already-absent managed label as converged state while preserving failures for every non-404 API error.

Add structural and mutation-sensitive canary coverage and advance Rune to v0.45.1.
2026-08-31 18:00:28 +01:00
conductor 023e9575a1 Merge pull request 'feat(gitea): make governance and recovery reproducible' (#52) from feat/gitea-hardening into master 2026-08-31 12:29:59 +01:00
Nick a134d99af6 chore(gitea): reconcile hardening with v0.44.0
Preserve the v1 authority and local-tool test job alongside the offline Gitea verification job.

Keep the protected gate pull-request-only, require both verification paths, pin every checkout action, and advance Rune to v0.45.0.
2026-08-31 12:21:57 +01:00
Nick aebe6daba1 style(gitea): format hardening Python files
Apply the exact Ruff 0.16.4 formatter enforced by the Gitea lint job so newly added Python files pass CI.
2026-08-30 17:18:08 +01:00
Nick a9496d1b06 feat(gitea): make governance and recovery reproducible
Track branch policy, labels, milestones, and wiki content as declarative state with a read-only-by-default provisioner.

Harden off-disk backup publication and add a rootless, networkless restore drill that validates SQLite, application data, and database-declared bare repositories.

Run offline contracts in CI, pin audited dependencies, document operations, and bump Rune to 0.43.0.
2026-08-30 17:10:32 +01:00
152 changed files with 22394 additions and 641 deletions

No files matched your search

+39
View File
@@ -0,0 +1,39 @@
# Issue Closing Comment — Standard Format (DESCRIPTION GATE v4)
Every issue MUST be closed with a comment in this exact shape. The closing
comment is the durable record of what happened: plain terms first, technical
detail second, pipeline results included, so Nick can read one comment and know
what landed and how it was verified.
## Template
```markdown
## Plain summary — what was delivered
<!-- 2-4 sentences, NON-TECHNICAL: what was BUILT and the user-facing outcome.
Not a task list. Include the concrete effect, e.g. "Filing X now queues and
replays exactly once even if the network drops." -->
## Technical details
<!-- One line per fact, bold labels, scannable.
- IMPLEMENTATION: commit refs with one-line descriptions (commits that implement
the feature; commits that discharge review findings).
- SPEC GATE: PASS (reviewer id) — what was verified.
- QUALITY GATE: Approved (reviewer id) — key findings; what was discharged vs pinned.
- TESTS: N test count; suite before → after.
- CANARY: name, PASSED.
- MERGED: PR # → master <sha> (when merged). -->
## Pins / follow-ups
<!-- Anything deferred with a reason + where it will land (e.g. "real-transport plan"). -->
- (none | one line per pin)
```
## Rules
1. **Plain summary comes FIRST** — always.
2. **Every gate result is listed** with the reviewer id and test/suite counts —
the pipeline requirements are part of the record.
3. **Discharges and pins are explicit** — what was fixed before close, what was
deliberately deferred and where.
4. Merged work links the PR and merge commit.
5. Apply to: stage closes, epic closes, and any issue closed after work.
+21 -6
View File
@@ -6,14 +6,28 @@ labels: bug
assignees: conductor
---
## What's broken
<!-- Plain language description of the bug -->
## Plain summary
<!-- 2-4 sentences, NON-TECHNICAL: what's broken and why it matters. -->
## What should happen
<!-- Expected behavior -->
## Example
<!-- A concrete before/after: "Doing X currently produces Y; it should produce Z." -->
## What actually happens
<!-- Actual behavior -->
## Technical details
<!-- WHAT (the bug) / WHERE (files) / EVIDENCE (how it was confirmed). -->
**WHAT (bug):**
**WHERE:**
**EVIDENCE:**
## Pipeline requirements
<!-- Every fix must clear these before the issue closes. -->
- [ ] TDD: regression test written first, red → green
- [ ] `ruff check` and `ruff format --check` clean
- [ ] Spec-compliance review gate: PASS
- [ ] Code-quality review gate: Approved
- [ ] Full orchestration suite regression green (suite total)
- [ ] Canary passes (name)
- [ ] Closed with a properly formatted closing comment + commit references
## Steps to reproduce
<!-- How to trigger the bug -->
@@ -25,6 +39,7 @@ assignees: conductor
- [ ] Growth
- [ ] Perception
- [ ] Infrastructure
- [ ] Orchestration
## Priority
- [ ] High — system broken
+31 -17
View File
@@ -1,22 +1,43 @@
---
name: Feature Request
about: Request a new feature or improvement
about: New feature or improvement — DESCRIPTION GATE v4 format (plain summary + pipeline requirements)
title: "[FEATURE] "
labels: feature
assignees: conductor
---
## What this does
<!-- Plain language description of the feature -->
## Plain summary
<!-- 2-4 sentences, NON-TECHNICAL: what this does and why it matters. Nick should
understand the goal without decoding internals. -->
## Why it matters
<!-- How this improves Rune's growth or Nick's life -->
## Example
<!-- A concrete example of the intended behavior, in plain terms:
"If X happens, the system will Y instead of Z." -->
## How it works
<!-- Technical description of the implementation -->
## Technical details
<!-- WHAT — one-line objective.
WHY — design-spec reference (S#) or reason.
FILES — affected components.
ACCEPTANCE — frozen-test file + test count + canary name.
DEPENDENCIES — parent epic + preceding stage/plan refs. -->
**WHAT:**
**WHY:**
**FILES:**
**ACCEPTANCE:**
**DEPENDENCIES:**
## Pipeline requirements
<!-- Every stage must clear these before the issue closes. -->
- [ ] TDD: frozen tests written first, red → green (N passed)
- [ ] `ruff check` and `ruff format --check` clean
- [ ] Spec-compliance review gate: PASS
- [ ] Code-quality review gate: Approved
- [ ] Full orchestration suite regression green (suite total)
- [ ] Canary passes (name)
- [ ] Closed with a properly formatted closing comment + commit/PR references
## Related umbrella
<!-- Which umbrella system does this belong to? -->
- [ ] Memory
- [ ] Governance
- [ ] Identity
@@ -25,16 +46,9 @@ assignees: conductor
- [ ] Infrastructure
- [ ] Life (Nick's)
- [ ] Custodian
- [ ] Orchestration
## Priority
<!-- How urgent is this? -->
- [ ] High — blocks other work
- [ ] Medium — improves quality of life
- [ ] Low — nice to have
## Acceptance criteria
<!-- How do we know when this is done? -->
- [ ] Implementation complete
- [ ] Canaries pass
- [ ] Documentation generated
- [ ] PR created and reviewed
- [ ] Low — nice to have
+10 -2
View File
@@ -1,5 +1,13 @@
## What this PR does
<!-- Explain the result in plain language. -->
## Plain summary
<!-- 2-4 sentences, NON-TECHNICAL: what this PR does and why it matters, with a
concrete example of the intended behavior. Nick reads this first. -->
## Example
<!-- A concrete example: "With this change, doing X results in Y instead of Z." -->
## Technical details
<!-- WHAT / WHY (design S#) / FILES / ACCEPTANCE (test file + count + canary) /
DEPENDENCIES (epic + issues). -->
## Why it matters
<!-- State whether this advances Rune's growth, Nick's life, or infrastructure. -->
+7
View File
@@ -10,5 +10,12 @@ jobs:
backup:
runs-on: self-hosted
steps:
# actions/checkout v4.2.2, pinned to the audited upstream tag commit.
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
- name: Verify backup and restore tooling offline
run: |
bash -n scripts/gitea-backup.sh scripts/gitea-restore-verify.sh
python3 -m unittest -v tests.test_gitea_backup
python3 src/rune/canary_gitea_automation.py
- name: Create consistent off-disk Gitea backup
run: sudo /usr/local/sbin/rune-gitea-backup
+51 -16
View File
@@ -18,56 +18,91 @@ jobs:
const body = context.payload.issue?.body || context.payload.pull_request?.body || '';
const text = (title + ' ' + body).toLowerCase();
const labels = [];
const desiredLabels = new Set();
const managedLabels = new Set([
'memory', 'governance', 'identity', 'growth',
'perception', 'infrastructure', 'life', 'custodian',
'priority: high', 'bug', 'feature',
]);
// Umbrella labels
if (text.includes('memory') || text.includes('store') || text.includes('recall') || text.includes('consolidat')) {
labels.push('memory');
desiredLabels.add('memory');
}
if (text.includes('governance') || text.includes('drift') || text.includes('curator') || text.includes('audit')) {
labels.push('governance');
desiredLabels.add('governance');
}
if (text.includes('identity') || text.includes('persona') || text.includes('voice') || text.includes('constitution')) {
labels.push('identity');
desiredLabels.add('identity');
}
if (text.includes('growth') || text.includes('delta') || text.includes('learn') || text.includes('improve')) {
labels.push('growth');
desiredLabels.add('growth');
}
if (text.includes('perception') || text.includes('route') || text.includes('capability') || text.includes('witness')) {
labels.push('perception');
desiredLabels.add('perception');
}
if (text.includes('infrastructure') || text.includes('config') || text.includes('hook') || text.includes('event')) {
labels.push('infrastructure');
desiredLabels.add('infrastructure');
}
if (text.includes('life') || text.includes('financ') || text.includes('organiz') || text.includes('revolution')) {
labels.push('life');
desiredLabels.add('life');
}
if (text.includes('custodian') || text.includes('gitea') || text.includes('pipeline')) {
labels.push('custodian');
desiredLabels.add('custodian');
}
// Priority labels
if (text.includes('urgent') || text.includes('critical') || text.includes('broken')) {
labels.push('priority: high');
desiredLabels.add('priority: high');
}
// Type labels
const isBug = title.startsWith('[BUG]') || text.includes('bug') || text.includes('fix') || text.includes('broken');
const isFeature = !isBug && (title.startsWith('[FEATURE]') || text.includes('feature') || text.includes('improve') || text.includes('add'));
if (isBug) {
labels.push('bug');
desiredLabels.add('bug');
}
if (isFeature) {
labels.push('feature');
desiredLabels.add('feature');
}
if (labels.length > 0) {
const issueNumber = context.payload.issue?.number || context.payload.pull_request?.number;
const issueNumber = context.payload.issue?.number || context.payload.pull_request?.number;
const current = await github.rest.issues.listLabelsOnIssue({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: issueNumber,
});
const currentLabels = new Set(current.data.map(label => label.name));
const staleManaged = [...currentLabels].filter(
label => managedLabels.has(label) && !desiredLabels.has(label)
);
const missingDesired = [...desiredLabels].filter(
label => !currentLabels.has(label)
);
for (const name of staleManaged) {
try {
await github.rest.issues.removeLabel({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: issueNumber,
name,
});
} catch (error) {
if (error.status !== 404) {
throw error;
}
console.log(`Managed label already absent: ${name}`);
}
}
if (missingDesired.length > 0) {
await github.rest.issues.addLabels({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: issueNumber,
labels: labels,
labels: missingDesired,
});
console.log(`Added labels: ${labels.join(', ')}`);
}
console.log(`Removed managed labels: ${staleManaged.join(', ') || 'none'}`);
console.log(`Added managed labels: ${missingDesired.join(', ') || 'none'}`);
+25 -1
View File
@@ -13,6 +13,8 @@ jobs:
runs-on: self-hosted
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Read reviewed semantic version
run: |
@@ -20,13 +22,35 @@ jobs:
echo "RELEASE_VERSION=$VERSION" >> "$GITHUB_ENV"
echo "Release version: $VERSION"
- name: Verify version was bumped since last tag
run: |
LATEST=$(git tag --sort=-v:refname | head -1 | sed 's/^v//')
if [ -z "$LATEST" ]; then
echo "No previous tags — releasing first version $RELEASE_VERSION"
exit 0
fi
if [ "$LATEST" = "$RELEASE_VERSION" ]; then
echo "::error::Version $RELEASE_VERSION was already released (tag v$LATEST)."
echo "::error::Semver bump missing from this PR: add a 'chore(release): bump version to X.Y.Z' commit that increments src/rune/__init__.py, then merge again."
exit 1
fi
echo "New version $RELEASE_VERSION (last tag v$LATEST)"
- name: Create release from merged pull request
env:
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
PR_TITLE: ${{ gitea.event.pull_request.title }}
PR_BODY: ${{ gitea.event.pull_request.body }}
RELEASE_API_URL: ${{ gitea.server_url }}/api/v1/repos/${{ gitea.repository }}/releases
LIST_API_URL: ${{ gitea.server_url }}/api/v1/repos/${{ gitea.repository }}/releases?limit=50
run: |
# Skip idempotently if a release for this tag already exists (retry/replay safety).
if curl --fail --silent "$LIST_API_URL" \
-H "Authorization: token $GITEA_TOKEN" \
| python3 -c 'import json,sys,os; exists=[r for r in json.load(sys.stdin) if r.get("tag_name")==f"v{os.environ[\"RELEASE_VERSION\"]}"]; sys.exit(0 if exists else 1)'; then
echo "Release v$RELEASE_VERSION already exists — skipping creation"
exit 0
fi
python3 - <<'PY' > /tmp/rune-release.json
import json, os
version = os.environ["RELEASE_VERSION"]
@@ -46,4 +70,4 @@ jobs:
-H "Content-Type: application/json" \
--data-binary @/tmp/rune-release.json \
"$RELEASE_API_URL"
rm -f /tmp/rune-release.json
rm -f /tmp/rune-release.json
+79 -10
View File
@@ -2,7 +2,6 @@ name: CI Pipeline
on:
push:
branches: [master]
pull_request:
branches: [master]
@@ -10,7 +9,8 @@ jobs:
lint:
runs-on: self-hosted
steps:
- uses: actions/checkout@v4
# actions/checkout v4.2.2, pinned to the audited upstream tag commit.
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
with:
fetch-depth: 0
- name: Create isolated lint environment
@@ -40,7 +40,8 @@ jobs:
unit-tests:
runs-on: self-hosted
steps:
- uses: actions/checkout@v4
# actions/checkout v4.2.2, pinned to the audited upstream tag commit.
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
- name: Create isolated test environment
run: |
python3 -m venv .test-venv
@@ -55,7 +56,53 @@ jobs:
tests/test_rune_operator_tools.py \
tests/test_runtime_architecture_characterization.py \
tests/test_message_widgets.py \
tests/test_v1_authority_map.py
tests/test_v1_authority_map.py \
tests/test_orchestration_records.py \
tests/test_orchestration_event_store.py \
tests/test_orchestration_evidence.py \
tests/test_orchestration_projection.py \
tests/test_orchestration_composition.py \
tests/test_orchestration_selector.py \
tests/test_orchestration_bus.py \
tests/test_orchestration_benchmark.py \
tests/test_orchestration_fixtures.py \
tests/test_orchestration_canary.py \
tests/test_orchestration_cadence.py \
tests/test_orchestration_no_change.py \
tests/test_orchestration_reflex.py \
tests/test_orchestration_proposals.py \
tests/test_orchestration_workers.py \
tests/test_orchestration_deadline.py \
tests/test_orchestration_health.py \
tests/test_orchestration_cognitive.py \
tests/test_orchestration_resources.py \
tests/test_orchestration_eligibility.py \
tests/test_orchestration_router.py \
tests/test_orchestration_scheduler.py \
tests/test_orchestration_broker.py \
tests/test_orchestration_authority.py \
tests/test_orchestration_memory.py \
tests/test_orchestration_cursor.py \
tests/test_orchestration_chain.py \
tests/test_orchestration_budget.py \
tests/test_orchestration_atomic.py \
tests/test_orchestration_memory_synthesis.py \
tests/test_orchestration_outbox.py \
tests/test_orchestration_lifecycle.py \
tests/test_orchestration_delegation.py \
tests/test_orchestration_reconciler.py \
tests/test_orchestration_transport.py \
tests/test_orchestration_gitea_delegation.py \
tests/test_orchestration_surface.py \
tests/test_orchestration_capability.py \
tests/test_orchestration_mapping.py \
tests/test_orchestration_gateway.py \
tests/test_orchestration_surface_gateway.py \
tests/test_orchestration_migrate.py \
tests/test_legacy_event_envelope.py \
tests/test_legacy_event_bus.py \
tests/test_legacy_pins.py \
tests/test_orchestration_migration_canary.py
canary:
runs-on: self-hosted
@@ -63,7 +110,8 @@ jobs:
RUNE_HOME: /tmp/rune-ci-${{ github.run_id }}
RUNE_MEMORY_DIR: /tmp/rune-ci-${{ github.run_id }}/memory
steps:
- uses: actions/checkout@v4
# actions/checkout v4.2.2, pinned to the audited upstream tag commit.
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
- name: Run canary suite
run: |
FAILED=0
@@ -71,7 +119,14 @@ jobs:
for canary in \
src/rune/canary_gitea_pipeline.py \
src/rune/canary_gitea_automation.py \
src/rune/canary_tui_drift.py; do
src/rune/canary_tui_drift.py \
src/rune/canary_orchestration_foundation.py \
src/rune/canary_cognitive_strata.py \
src/rune/canary_resource_authority.py \
src/rune/canary_memory_synthesis.py \
src/rune/canary_gitea_delegation.py \
src/rune/canary_surface_parity.py \
src/rune/canary_schema_migration.py; do
CANARY_COUNT=$((CANARY_COUNT + 1))
echo "Running $canary..."
python3 "$canary" || FAILED=$((FAILED + 1))
@@ -96,7 +151,8 @@ jobs:
best-practices:
runs-on: self-hosted
steps:
- uses: actions/checkout@v4
# actions/checkout v4.2.2, pinned to the audited upstream tag commit.
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
- name: Check best practices
run: |
echo "=== Best Practices Check ==="
@@ -119,7 +175,8 @@ jobs:
documentation:
runs-on: self-hosted
steps:
- uses: actions/checkout@v4
# actions/checkout v4.2.2, pinned to the audited upstream tag commit.
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
- name: Check documentation coverage
run: |
echo "=== Documentation Check ==="
@@ -146,9 +203,21 @@ jobs:
print('WARNING: Less than 50% documentation coverage')
"
required-checks:
gitea-offline:
runs-on: self-hosted
needs: [lint, unit-tests, canary, best-practices, documentation]
steps:
# actions/checkout v4.2.2, pinned to the audited upstream tag commit.
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
- name: Verify Gitea provisioning and backup tooling offline
run: |
python3 -m unittest tests.test_gitea_provision
python3 -m unittest tests.test_gitea_backup
bash -n scripts/gitea-backup.sh scripts/gitea-restore-verify.sh
required-checks:
if: gitea.event_name == 'pull_request'
runs-on: self-hosted
needs: [lint, unit-tests, canary, best-practices, documentation, gitea-offline]
steps:
- name: Publish protected-branch gate
run: echo "All required CI checks passed"
+3 -3
View File
@@ -1,7 +1,7 @@
# Rune TUI — Product Requirements Document (PRD)
**Last Updated**: 2026-08-28
**Version**: v0.44.0
**Last Updated**: 2026-08-30
**Version**: v0.45.1
**Status**: Active Development
---
@@ -35,7 +35,7 @@ Rune is a stateful coding assistant with a Textual-based TUI. It connects to loc
---
## Features (v0.44.0)
## Features (v0.45.1)
### Chat System
- Two channels: Task (focused work) and Chat (free conversation)
+60 -1
View File
@@ -27,7 +27,7 @@ No subsystem may inject model context outside the future authoritative
ContextBroker. The map is an admission and migration plan; it does not claim
those migrations are already implemented.
Rune's ordinary v0.43 turn currently runs through `RuneApp` → `RuneOperator` →
Rune's ordinary v0.49.0 turn currently runs through `RuneApp` → `RuneOperator` →
`LocalToolRuntime`. The repository also contains the following layered Elastic
Cognition prototypes. They are migration candidates, not the authoritative live
path; ownership is recorded in `docs/v1/AUTHORITY_MAP.md`.
@@ -155,6 +155,65 @@ python3 rune/canary_native.py
python3 -m pytest tests/
```
### Gitea repository state
Branch protection, the 11 managed labels, three milestones, and six wiki pages
are declared in the [declarative repository state](config/gitea_repository_state.json).
Wiki bodies are tracked as Markdown under the [wiki documentation](docs/wiki/Home.md).
```bash
# Read-only drift check (also the default when no mode is supplied)
python3 shared/gitea_provision.py --check
# Create or update managed state, then verify convergence
python3 shared/gitea_provision.py --apply
```
The provisioner loads credentials at runtime through `shared/gitea_auth.py`.
Set `GITEA_TOKEN`, or use its mode-0600 token file. Check mode sends GET
requests only. Apply mode never deletes labels, milestones, wiki pages, or
other unmanaged repository objects.
### Gitea backup and restore verification
The daily backup action invokes the root-owned
`/usr/local/sbin/rune-gitea-backup` on the Gitea host. It stops Gitea only for
the official `gitea dump`, restores service and verifies health, validates all
mandatory archive members, and publishes the archive, metadata, and SHA-256
sidecar to the separate Foundry disk. The checksum is linked last as the
publication commit marker, and complete backup sets have bounded retention.
Before creating the backup directory or dumping, the helper requires
`BACKUP_MOUNT` (default `/media/pi/Foundry`) to be the exact active mount,
requires `BACKUP_DIR` to remain inside it lexically and after symlink
resolution, and verifies that its backing device differs from `/`.
Run it only through the installed, administrator-controlled entry point:
```bash
sudo /usr/local/sbin/rune-gitea-backup
```
Restore verification is an explicit, rootless, disposable drill. As a
non-root user, acknowledge the guard and supply an archive with its adjacent
`.sha256` sidecar:
```bash
RUNE_GITEA_RESTORE_VERIFY=1 scripts/gitea-restore-verify.sh \
/media/pi/Foundry/backups/gitea/gitea-dump-<timestamp>.tar.gz
```
The verifier requires the audited digest-pinned Gitea image to exist locally;
it uses `--pull=never`, no container network, no published ports, and a
read-only container root. It extracts only into a fresh temporary directory
and mounts only that disposable restore tree. Archived application data is
restored into `/data/gitea` before the reconstructed SQLite database and
generated isolated configuration overwrite archived versions. Every
database-declared repository must exist at its canonical owner/name path and
pass Git integrity checks; extra repositories such as wikis remain allowed.
Production Gitea configuration, data, repositories, and service paths are
therefore never accepted or mounted as a restore target. See the tracked
[backup and restore runbook](docs/wiki/Backup-and-Restore.md) and
[operations guide](docs/wiki/Operations.md).
## License
Private — Nick's personal AI assistant.
+146
View File
@@ -0,0 +1,146 @@
{
"repository": {
"owner": "conductor",
"name": "rune",
"default_branch": "master"
},
"branch_protection": {
"branch_name": "master",
"enable_push": false,
"enable_push_whitelist": false,
"push_whitelist_usernames": [],
"enable_force_push": false,
"enable_merge_whitelist": false,
"merge_whitelist_usernames": [],
"merge_whitelist_teams": [],
"enable_bypass_allowlist": false,
"bypass_allowlist_usernames": [],
"bypass_allowlist_teams": [],
"required_approvals": 0,
"enable_approvals_whitelist": false,
"approvals_whitelist_username": [],
"approvals_whitelist_teams": [],
"enable_status_check": true,
"status_check_contexts": [
"CI Pipeline / required-checks (pull_request)"
],
"block_on_rejected_reviews": true,
"block_on_official_review_requests": true,
"block_on_outdated_branch": true,
"dismiss_stale_approvals": true,
"ignore_stale_approvals": false,
"require_signed_commits": false,
"protected_file_patterns": "",
"unprotected_file_patterns": "",
"block_admin_merge_override": true
},
"labels": [
{
"name": "bug",
"color": "d73a4a",
"description": "Confirmed defect or regression"
},
{
"name": "custodian",
"color": "0075ca",
"description": ""
},
{
"name": "feature",
"color": "2da44e",
"description": "New capability or improvement"
},
{
"name": "governance",
"color": "0075ca",
"description": ""
},
{
"name": "growth",
"color": "0075ca",
"description": ""
},
{
"name": "identity",
"color": "0075ca",
"description": ""
},
{
"name": "infrastructure",
"color": "0075ca",
"description": ""
},
{
"name": "life",
"color": "0075ca",
"description": ""
},
{
"name": "memory",
"color": "0075ca",
"description": ""
},
{
"name": "perception",
"color": "0075ca",
"description": ""
},
{
"name": "priority: high",
"color": "d1242f",
"description": "Blocks work or breaks a critical path"
}
],
"milestones": [
{
"title": "Gitea pipeline maturity",
"description": "Runner, protected merges, templates, labels, wiki, milestones, verified backups, and reversible Custodian delivery.",
"state": "open",
"due_on": "2026-09-30T00:00:00Z"
},
{
"title": "Sovereign assistant foundation",
"description": "TUI-first assistant capabilities for organisation, finances, and revolutionary tools without vendor dependence.",
"state": "open",
"due_on": "2027-02-24T00:00:00Z"
},
{
"title": "Level 10 autonomy",
"description": "Growth path: reliable memory, governed adaptation, cognition, initiative, and teachable self-improvement with Nick retaining final control.",
"state": "open",
"due_on": "2027-08-24T00:00:00Z"
}
],
"wiki_pages": [
{
"title": "Home",
"slug": "Home",
"source": "docs/wiki/Home.md"
},
{
"title": "Architecture",
"slug": "Architecture",
"source": "docs/wiki/Architecture.md"
},
{
"title": "Build-Pipeline",
"slug": "Build-Pipeline.-",
"source": "docs/wiki/Build-Pipeline.md"
},
{
"title": "Operations",
"slug": "Operations",
"source": "docs/wiki/Operations.md"
},
{
"title": "Growth-Roadmap",
"slug": "Growth-Roadmap.-",
"source": "docs/wiki/Growth-Roadmap.md"
},
{
"title": "Backup-and-Restore",
"slug": "Backup-and-Restore.-",
"source": "docs/wiki/Backup-and-Restore.md"
}
]
}
+27
View File
@@ -0,0 +1,27 @@
# Cognitive Strata
Second orchestration plan (after the foundation): the cognitive consumption
layer over the durable spine, per design section S8.
## Components (src/rune/orchestration/)
- cadence.py — CadenceConfig + durable per-cadence cursor
- no_change.py — content_fingerprint + per-cadence NoChangeGate
- reflex.py — deterministic reflex engine (facts, never judgments)
- proposals.py — bounded ProposalSink with dead letter
- workers.py — CognitiveWorker protocol + WorkerRegistry (independent toggles)
- deadline.py — DeadlinePolicy for forced runs
- health.py — CadenceHealth ledger (per-cadence accounting)
- cognitive.py — build_cognitive composition + DeadlineRunner
## Contracts
- Disabling or failing one cadence never affects another (S8/A1).
- Unchanged evidence is skipped by the no-change gate (>=80% avoidance goal).
- Workers only emit proposals; canonical state is untouched by this plan.
- Deadline policies force a run so quiet periods cannot starve a cadence.
## Run
PYTHONPATH=src:. python3 -m pytest tests/ -k orchestration --ignore=tests/test_message_widgets.py -q
PYTHONPATH=src:. python3 src/rune/canary_cognitive_strata.py
+30
View File
@@ -0,0 +1,30 @@
# Gitea Delegation and Outbox
Fifth orchestration plan (after foundation, cognitive strata, resource
authority, memory and synthesis): the Gitea coordination layer, per design
S4 items 5/10/17 and S9.
## Components (src/rune/orchestration/)
- records.py — OutboxRecord (canonical fields incl. plan/approval versions,
ordering key, returned remote id)
- outbox.py — OutboxStore (append-only, idempotent by mutation ID, atomic)
- lifecycle.py — OutboxLifecycle (queued->sending->accepted + conflict states)
- delegation.py — DelegationPlanner (plan version -> ordered Gitea mutations)
- reconciler.py — Reconciler (version preconditions, deterministic conflicts)
- transport.py — GiteaTransport protocol + FakeGiteaTransport
- gitea_delegation.py — build_gitea_delegation() composition
## Contracts
- Queued mutations are pending intents, never accepted remote state (S4 10).
- Never overwrite newer remote state; conflicts are visible and require Nick
or a policy-named resolver.
- Replays are idempotent by stable mutation ID + ALREADY_APPLIED mapping.
- One outbox store, one lifecycle, one planner; the reconciler is bound per
delegate() call (transport supplied per call, never at build time).
## Run
PYTHONPATH=src:. python3 -m pytest tests/ -k orchestration --ignore=tests/test_message_widgets.py -q
PYTHONPATH=src:. python3 src/rune/canary_gitea_delegation.py
+29
View File
@@ -0,0 +1,29 @@
# Memory and Synthesis
Fourth orchestration plan (after foundation, cognitive strata, resource
authority): the memory and synthesis layer, per design section S8.
## Components (src/rune/orchestration/)
- memory.py — EpisodicRecord + EpisodicStore (stable IDs, synthesis states)
- cursor.py — SynthesisCursor (item-ID transactional journal + receipts)
- chain.py — SynthesisChain (cumulative linked syntheses)
- budget.py — PinBudget + BudgetRegistry (explicit budgets, visible overflow)
- atomic.py — AtomicStore (temp+fsync+rename hot writes)
- memory_synthesis.py — build_memory_synthesis() composition
## Contracts
- Synthesis coverage is per EPISODE ID, idempotent, and advances only on
commit — same-day and capped input can never be omitted or falsely marked
covered (A2).
- Syntheses form a cumulative chain; retrieving the newest never erases
older compressed continuity (C029).
- Pinning is measured; overflow surfaces visible pressure and calls the
approved strategy — never silent truncation.
- Hot cursors/queues are written atomically; a crash never corrupts them.
## Run
PYTHONPATH=src:. python3 -m pytest tests/ -k orchestration --ignore=tests/test_message_widgets.py -q
PYTHONPATH=src:. python3 src/rune/canary_memory_synthesis.py
+25
View File
@@ -0,0 +1,25 @@
# Orchestration Foundation
First plan of the approved planning/task orchestration design
(`specs/2026-09-03-planning-task-orchestration-design.md`). It provides the
semantically complete, unoptimized reference that all later plans build on.
## Components (src/rune/orchestration/)
- records.py — canonical records with schema_version
- event_store.py — durable batched append-only JSONL with offsets and replay
- evidence_store.py — content-addressed immutable blobs (sha256)
- projection.py — checkpointed materialized views with order-insensitive hashes
- composition.py — one instance of each authority component
- selector.py — deterministic successor selection (S10 chaining)
- bus.py — hot bus publishing compact refs after durable commit
- benchmark.py — p95 regression rule with noise bounds
- fixtures.py — golden capability fixtures
## Run
PYTHONPATH=src:. python3 -m pytest tests/ -k orchestration --ignore=tests/test_message_widgets.py -q
PYTHONPATH=src:. python3 src/rune/canary_orchestration_foundation.py
The canary asserts capability preservation across two identical runs and a
single-authority composition root. Any later plan must keep both green.
+27
View File
@@ -0,0 +1,27 @@
# Resource Authority
Third orchestration plan (after foundation + cognitive strata): the resource
authority layer, per design section S8.
## Components (src/rune/orchestration/)
- resources.py — ResourceState + Registry (unknown != zero)
- eligibility.py — deterministic EligibilityFilter (removes, never ranks)
- router.py — advisory router (FixedRouter, CostBiasRouter)
- scheduler.py — ResourceScheduler (single authority: lease + bounded fallback)
- broker.py — ResourceBroker + ElasticBudget (subagent inheritance)
- authority.py — build_resources() composition
## Contracts
- Unknown health is NEVER usable; a request with no eligible candidate is
structured unavailability (None), never a silent wrong pick.
- Eligibility is deterministic; routing is advisory; the scheduler owns the
final lease and records a bounded fallback chain.
- Subagents inherit an ElasticBudget from the parent assignment and cannot
bypass locality/provider/depth/concurrency/cost.
## Run
PYTHONPATH=src:. python3 -m pytest tests/ -k orchestration --ignore=tests/test_message_widgets.py -q
PYTHONPATH=src:. python3 src/rune/canary_resource_authority.py
+96
View File
@@ -0,0 +1,96 @@
# Schema Migration
Canonicalization of Rune's event journal and pins store: old-dialect data is
migrated ONCE into the canonical orchestration schema, and the legacy writers
delegate to the canonical stores (no dialect bazaar).
## The migration contract
- Old fixtures migrate once. `old_fixture_to_new` is idempotent by `event_id`:
a re-run converts only the not-yet-converted entries, so a partial or
interrupted migration resumes cleanly.
- The new store rejects unmigrated dialects. `DurableEventStore.append` raises
`UnmigratedDialectError` for any old-dialect dict (legacy `payload`/`source`/
`channel`/`authority_class`/`task_id` fields, `schema_version < 1`, or a
garbage "canonical"-shaped dict) with an actionable message pointing at
`migrate.old_fixture_to_new()`.
- Legacy writers delegate to canonical records. `event_bus.emit` and
`pins.pin` write canonical `EventEnvelope` / `PinRecord` rows through the
canonical stores — the CLI/`pins.py` surfaces are projections, never a
second dialect.
## The one-dialect rule
There is exactly ONE event dialect the store accepts: the canonical
`EventEnvelope` (event_id, schema_version, event_type, partition_key,
payload_hash, timestamp). Old data must pass through the migration before it
can be appended. The migration CLI is the sanctioned path for converting a
legacy JSONL fixture; the legacy `event_bus.emit` shim is the sanctioned path
for NEW writes (it produces canonical records directly).
## The migration CLI
python -m rune.orchestration.migrate_cli <fixture> [--store DIR]
python -m rune.orchestration.migrate_cli --pins <legacy.db> <out.json>
- Fixture form: converts an old-dialect JSONL fixture to the canonical store
(default `<fixture dir>/events`; `--store DIR` overrides the event store
directory) and prints `MIGRATED N`. A re-run prints `MIGRATED 0` —
migration is idempotent.
- `--pins` form: converts legacy SQLite `wing='pins'` chunks to a canonical
PinRecord JSON array and prints `PINS MIGRATED N`.
- Both exit 0 on success. A malformed / non-dict fixture line fails with a
readable error (exit 1) instead of a traceback.
- The CLI works with only `/usr/bin:/bin` on `PATH` — it is invoked via
`python -m`, never the shell.
## Evidence store (X1 pin discharge)
The canonical envelope carries no payload body (`payload_hash` is one-way).
So `old_fixture_to_new` ALSO writes each migrated raw line to the
EvidenceStore keyed by its `payload_hash` (digest == payload_hash — a 1:1
link). A store-only consumer can recover the original body:
from rune.orchestration.evidence_store import EvidenceStore
body = EvidenceStore(<fixture dir>/evidence).get(rec["payload_hash"])
`EvidenceStore.put` hashes the same canonical bytes (`sha256:` +
sha256(json.dumps(raw, sort_keys=True))) as the event record's `payload_hash`,
so the evidence key and the event record always agree.
## source.channel extraction
Real `event_bus` data nests the channel INSIDE `source`
(`source='{"channel": "chat"}'` or a dict). `old_fixture_to_new` extracts
`source.channel` when present, falling back to the top-level `channel` and
then `conversation` — without this, real data would be silently repartitioned.
## Legacy pins conversion (X4 C7 discharge)
The legacy hybrid-store writer is gone, so existing pins live in a SQLite
`chunks` table (`text`/`wing`/`room` columns) as multi-line text blobs:
[topic] guitar course
[open question] when?
[outcome] launched
[status] open
[context] ctx
`legacy_pins_to_new(legacy_db, out_json)` reads the `wing='pins'` chunks,
parses this `[key] value` format, and writes a JSON array of canonical
`PinRecord` dicts (`kind='pin'`, `item_id=topic`, `owner='pins'`,
`state=open|resolved`, `details` = the question/outcome/context/reason blob,
`sort_keys=True`). Run it once before/after shipping so existing pins are not
orphaned.
## Verify
PYTHONPATH=src:. python3 -m pytest tests/test_orchestration_migration_canary.py -q
PYTHONPATH=src:. python3 -m pytest tests/ -q --ignore=tests/test_message_widgets.py
python3 src/rune/canary_schema_migration.py # SCHEMA MIGRATION CANARY PASSED
The canary walks the whole path in a tempdir: migrate once (1), idempotent
re-run (0), canonical store rejects a fresh old-dialect append with the typed
error, and the legacy `event_bus.emit` shim is proven WIRED through the
canonical store (a `user.message` emitted via the legacy API replays back
from a `DurableEventStore` at the same tempdir).
+33
View File
@@ -0,0 +1,33 @@
# Surface Parity
Sixth orchestration plan (after foundation, cognitive strata, resource
authority, memory and synthesis, Gitea delegation and outbox): the entry-path
parity layer, per design "Entry-path and surface parity".
## Components (src/rune/orchestration/)
- surface.py — Capability + SurfaceRegistry (bidirectional external/internal
ids, stable binding)
- capability.py — CapabilityNegotiator (baseline parity + structured absence)
- mapping.py — MappingStore (durable bidirectional conversation mapping)
- gateway.py — SurfaceGateway (enter: resolve + peer-check + negotiate + map)
- surface_gateway.py — build_surface_parity() composition
## Contracts
- Every production entry path enters through the same gateway and composition
root (S4 doctrine: Rune contains the model; surfaces do not fork the spine).
- Capability-set parity is mandatory; exceptions require structured absence
(owner, reason, expiry, restoration canary). Honest absence is NOT parity.
- External-to-internal mappings are bidirectional and durable: responses can
return to the originating room, device, or peer.
- Transport identity and portable agent identity remain distinct; configured
peer keys are pinned and checked before events enter the spine.
- The SurfaceRegistry is itself persisted (AtomicStore JSON at
<paths_root>/surface_registry/state/registry.json) and reloaded on
composition, so surfaces and pinned peer keys survive restart.
## Run
PYTHONPATH=src:. python3 -m pytest tests/ -k orchestration --ignore=tests/test_message_widgets.py -q
PYTHONPATH=src:. python3 src/rune/canary_surface_parity.py
+17
View File
@@ -0,0 +1,17 @@
# Architecture
## Components
1. **Rune TUI** — primary embodied interface; cognition, autonomy, and dream cycles run here.
2. **MiMoCode** — secondary coding and operations interface.
3. **Shared library** — universal implementations used by both interfaces.
4. **Custodian** — translates plain-language issues into isolated worktree changes and pull requests.
5. **Gitea** — sovereign source of truth for issues, code review, CI, releases, wiki, and milestones.
## State flow
`TUI or MiMoCode → issue → Custodian worktree → pull request → CI gate → review → merge → sync back`
## Boundaries
- Gitea verifies deterministic work; an LLM does not award itself a green badge.
- Master rejects direct pushes and force pushes.
- Stateful canaries use isolated temporary memory roots.
- Credentials belong outside tracked code.
+23
View File
@@ -0,0 +1,23 @@
# Backup and Restore
## Backup policy
A daily Gitea Action invokes the root-owned `/usr/local/sbin/rune-gitea-backup`. The helper stops Gitea for a consistent snapshot, runs the official `gitea dump`, restarts and health-checks Gitea, verifies archive contents, writes a SHA-256 checksum, and retains 30 days.
## Location
`/media/pi/Foundry/backups/gitea/` on the separate Foundry disk. Backups are not stored on the Pi root card.
## Verify
`sha256sum -c gitea-dump-<timestamp>.tar.gz.sha256`
`tar -tzf gitea-dump-<timestamp>.tar.gz`
A valid archive contains repositories, data, `app.ini`, and `gitea-db.sql`.
## Restore outline
1. Stop Gitea.
2. Verify and unpack the chosen archive.
3. Restore configuration, data, repositories, and SQLite database with original ownership.
4. Regenerate repository hooks.
5. Start Gitea and run health checks.
Perform restore drills on a disposable location before trusting the backup as disaster recovery.
+16
View File
@@ -0,0 +1,16 @@
# Build Pipeline
Every substantial build follows this sequence:
1. Research primary documentation and counter-evidence.
2. Pin findings and measurable acceptance criteria.
3. Write canaries before implementation.
4. Check feasibility, cost, risk, and rollback.
5. Build in an isolated branch or worktree.
6. Run lint, canaries, best-practice checks, and documentation checks.
7. Submit a plain-language pull request.
8. Require `CI Pipeline / required-checks (pull_request)`.
9. Review, merge, version, release, and sync back to TUI and MiMoCode.
10. Record failures and lessons for growth.
Two failed repair attempts trigger critique; repeated failure questions the approach rather than piling patches onto it.
+14
View File
@@ -0,0 +1,14 @@
# Growth Roadmap
Gitea milestones make growth measurable rather than mystical.
## Level 10 autonomy
Reliable memory, governed adaptation, cognition, initiative, and teachable self-improvement while Nick retains final control.
## Sovereign assistant foundation
TUI-first organisational, financial, and revolutionary capabilities without dependence on rent-seeking platforms.
## Gitea pipeline maturity
Runner, protected merges, templates, labels, wiki, milestones, verified backups, and reversible Custodian delivery.
Each issue belongs to a milestone. Progress is the ratio of closed work to defined work, not a mood reported by an agent.
+13
View File
@@ -0,0 +1,13 @@
# Rune Documentation
This wiki explains the sovereign assistant system in plain language. Code is evidence; this wiki is the map.
## Start here
- [Architecture](Architecture) — how TUI, MiMoCode, Custodian, and Gitea connect
- [Build Pipeline](Build-Pipeline) — the research-to-merge process
- [Operations](Operations) — runner, protected branch, labels, templates, and maintenance
- [Growth Roadmap](Growth-Roadmap) — milestones toward governed autonomy
- [Backup and Restore](Backup-and-Restore) — where backups live and how to recover
## Decision rights
Custodian may research, build, test, document, and submit. CI may verify. Nick remains the final authority over goals and disputed merges.
+17
View File
@@ -0,0 +1,17 @@
# Operations
## Runner
`rune-pi` runs Gitea Runner 3.3.0 as an unprivileged systemd service. It accepts the repository-scoped `self-hosted` label.
## Master protection
- Direct push: disabled
- Force push: disabled
- Outdated branches: blocked
- Rejected or requested reviews: block merge
- Required context: `CI Pipeline / required-checks (pull_request)`
## Issue taxonomy
Umbrellas: memory, governance, identity, growth, perception, infrastructure, life, custodian. Type and priority labels are added automatically from issue content. Bug and feature labels are mutually exclusive.
## Templates
Feature and bug templates ask what changes, why it matters, umbrella ownership, priority, and acceptance criteria. Pull requests require verification evidence, rollback, documentation, and plain-language review notes.
+97
View File
@@ -0,0 +1,97 @@
# Gitea Hardening Implementation Plan
> [!NOTE]
> This document may not reflect the current implementation.
> See the final report for up-to-date state:
> [Final Report](../reports/gitea-hardening.md)
> **For agentic workers:** REQUIRED SUB-SKILL: Use compose:subagent (recommended) or compose:execute to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Make all eight requested Gitea capabilities reproducible, failure-sensitive, and recoverable rather than dependent on undocumented live state.
**Architecture:** Keep repository policy in a declarative JSON manifest, reconcile it through a standard-library API client, and track wiki Markdown as source. Harden CI and label workflows with contract canaries; harden backup publication and add a networkless disposable restore verifier.
**Tech Stack:** Gitea 1.27 API, Gitea Actions, Python 3 standard library, Bash, SQLite, Git, rootless Podman.
## Global Constraints
- Never store or print API tokens.
- Check mode is read-only; apply mode is idempotent and never deletes unmanaged resources.
- Production Gitea must never be used as a restore target.
- All changes occur in the isolated `feat/gitea-hardening` worktree.
- Tests fail before implementation and pass afterward.
---
### Task 1: CI and label reconciliation
**Covers:** CI on every push; content-based auto-labeling.
**Files:**
- Modify: `.gitea/workflows/ci.yml`
- Modify: `.gitea/workflows/auto-label.yml`
- Test: `src/rune/canary_gitea_pipeline.py`
- Test: `src/rune/canary_gitea_automation.py`
**Interfaces:** Gitea event payloads and `github.rest.issues` label APIs.
- [x] Add failing contracts for unfiltered push events, PR-only merge gate, managed-label boundaries, stale-label removal, and manual-label preservation.
- [x] Run both canaries and observe the expected failures.
- [x] Remove the push branch filter and make `required-checks` PR-only.
- [x] Reconcile desired managed labels against current labels while leaving unmanaged labels untouched.
- [x] Run both canaries and observe 11/11 and 21/21 passing.
### Task 2: Declarative repository provisioning
**Covers:** Branch protection, labels, wiki documentation, milestones.
**Files:**
- Create: `shared/gitea_provision.py`
- Create: `config/gitea_repository_state.json`
- Create: `docs/wiki/{Home,Architecture,Build-Pipeline,Operations,Growth-Roadmap,Backup-and-Restore}.md`
- Create: `tests/test_gitea_provision.py`
- Modify: `README.md`
**Interfaces:** `load_gitea_token()`, Gitea branch-protection/labels/milestones/wiki endpoints, injectable HTTP transport.
- [ ] Write failing unit tests for empty, drifted, converged, and second-apply states; pagination; normalized colors/timestamps; wiki base64; read-only check mode.
- [ ] Run `python3 -m unittest tests.test_gitea_provision` and observe failures caused by missing implementation.
- [ ] Implement manifest validation, API preflight, comparison, check, apply, and post-apply verification without deletion.
- [ ] Fetch the six live wiki pages and save their Markdown as canonical tracked sources.
- [ ] Run unit tests and `gitea_provision.py --check` against live Gitea without exposing credentials.
### Task 3: Backup and restore verification
**Covers:** Automatic off-disk backup and proven recoverability.
**Files:**
- Modify: `scripts/gitea-backup.sh`
- Create: `scripts/gitea-restore-verify.sh`
- Create: `tests/test_gitea_backup.py`
- Modify: `.gitea/workflows/auto-backup.yml`
- Modify: `src/rune/canary_gitea_automation.py`
**Interfaces:** official `gitea dump`, SHA-256 sidecars, SQLite dump, Git repositories, pinned rootless Podman image.
- [ ] Write failing synthetic-archive and command-shim tests for mandatory members, path traversal, checksum, service-state preservation, overlap locking, atomic publication, and cleanup.
- [ ] Run the tests and observe failures caused by missing hardening and verifier.
- [ ] Add `flock`, strict per-member checks, prior-state restoration, atomic publication, metadata, and pairwise retention.
- [ ] Implement a rootless, networkless disposable restore verifier that checks SQLite integrity, repository fsck/master, and an isolated Gitea boot.
- [ ] Run shell syntax, unit tests, canaries, and one real disposable restore drill against the newest backup.
### Task 4: Integration verification
**Covers:** Full eight-part request.
**Files:**
- Modify: `.gitea/workflows/ci.yml`
- Modify: `src/rune/canary_gitea_automation.py`
- Modify: `README.md`
**Interfaces:** CI-safe offline tests plus explicitly invoked live checks.
- [ ] Add offline provisioning and backup tests to CI.
- [ ] Run all focused unit tests and both Gitea canaries.
- [ ] Run live repository-state check and read-only Gitea API verification.
- [ ] Review the complete diff for secrets and unrelated files.
- [ ] Hand off for commit/push only after explicit user authorization.
+84
View File
@@ -0,0 +1,84 @@
# Concurrent Auto-Label Removal Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use compose:subagent (recommended) or compose:execute to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Make Gitea managed-label removal idempotent when concurrent issue or pull-request events remove the same label.
**Architecture:** Keep the existing inline `actions/github-script` workflow and managed-label boundary. Wrap each `removeLabel` call in a narrow `try`/`catch`: HTTP 404 means the desired state already exists and is logged as success; every other error is rethrown.
**Tech Stack:** Gitea Actions, `actions/github-script`, JavaScript, Python contract canary.
## Global Constraints
- Preserve every unmanaged/manual label.
- Ignore only HTTP 404 from stale managed-label removal.
- Rethrow authentication, authorization, transport, and server failures.
- Add no workflow queue, dependency, or unrelated refactor.
- Deliver through issue #53, an isolated branch, protected Gitea PR checks, and normal merge.
---
### Task 1: Idempotent managed-label removal
**Files:**
- Modify: `.gitea/workflows/auto-label.yml:83-91`
- Modify: `src/rune/canary_gitea_automation.py:76-83`
- Modify: `src/rune/__init__.py:6`
- Modify: `src/rune/rune_tui.py:9,1459`
- Modify: `PRD.md:3-4,38`
- Modify: `README.md:30`
**Interfaces:**
- Consumes: `github.rest.issues.removeLabel(...)` errors with numeric `status`.
- Produces: 404-tolerant removal while preserving failure propagation for all other statuses.
- [ ] **Step 1: Add the failing contract**
Add a structural canary assertion requiring the workflow to catch removal errors around `removeLabel`, accept only `error.status === 404`, log the already-absent label, and `throw error` for every other status. Add a mutant check proving inverted 404 handling is rejected. Increase the automation canary total from 37 to 39.
- [ ] **Step 2: Run RED verification**
Run: `python3 src/rune/canary_gitea_automation.py`
Expected: one failure named `concurrent stale-label removal is idempotent`; the other 37 contracts pass.
- [ ] **Step 3: Implement the minimal workflow change**
Replace the direct removal with:
```javascript
try {
await github.rest.issues.removeLabel({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: issueNumber,
name,
});
} catch (error) {
if (error.status !== 404) {
throw error;
}
console.log(`Managed label already absent: ${name}`);
}
```
- [ ] **Step 4: Run GREEN verification**
Run:
```bash
python3 src/rune/canary_gitea_automation.py
python3 src/rune/canary_gitea_pipeline.py
python3 -m unittest tests.test_gitea_backup tests.test_gitea_provision
git diff --check
```
Expected: automation 39/39, pipeline 20/20, all focused tests pass, and no diff errors.
- [ ] **Step 5: Advance the patch release**
Change the automation version contract to `0.45.1` and run it RED against `0.45.0`. Then update `src/rune/__init__.py`, both TUI version displays, PRD, and README to the exact `v0.45.1` identifier. Run `tests/test_build_pipeline_policy.py` to prove all release surfaces agree.
- [ ] **Step 6: Review and deliver**
Run an independent spec/quality review, commit with `fix(gitea): tolerate concurrent label removal`, push `fix/auto-label-race`, open a PR closing #53, wait for protected checks, and merge normally without force-push.
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+988
View File
@@ -0,0 +1,988 @@
# Resource Authority Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use compose:subagent (recommended) or compose:execute to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Build the approved resource authority (design S8) over the orchestration spine: a canonical resource registry of observed facts, a deterministic eligibility filter (unknown is never healthy), an advisory router that ranks eligible candidates, a scheduler that leases the final assignment with a bounded fallback chain, and inherited governance for subagents. Workers resolve resources through one authority path, not model prose.
**Architecture:** New modules in `src/rune/orchestration/`. `ResourceState` records observed capabilities/health/locality/cost/evidence-age (unknown != zero, per the design's "Unknown is not healthy"). `EligibilityFilter` is a pure deterministic gate (removes impossible candidates — never ranks). `AdvisoryRouter` is replaceable but deterministic-under-contract (scores eligible candidates with a documented policy; a `FixedRouter` is provided for tests). `ResourceScheduler` is the single authority: it consults the filter, asks the router, picks one winner, records the assignment + bounded fallback chain as an `ResourceAssignment`, and enforces limits. `ResourceBroker` exposes `acquire(task_signature, requirements)` for workers and propagates the same policies to subagents (depth/concurrency/locality/provider/budget). A `build_resources()` composition wires one instance of each. Deterministic worker stand-ins prove inheritance. Real providers/LLMs are out of scope — contracts and authority are what this plan proves. The composition-root inventory (`assert_single_authority`) must include the new scheduler and registry.
**Tech Stack:** Python 3.14, stdlib only, pytest 9.0.3, ruff (check + format — CI gates BOTH; run `ruff format` on failures first), pyright config present.
## Global Constraints
- Follow the approved design `specs/2026-09-03-planning-task-orchestration-design.md` S8 resource authority: one canonical registry; deterministic eligibility; advisory routing; scheduler records final assignment + fallback chain; subagents inherit the same authority object and cannot bypass locality/provider/budget/depth/concurrency policy; unknown/stale resource state produces structured unavailability or lower confidence, never silent fallback presented as the requested resource.
- Deterministic: identical inputs + same state -> identical outputs; tests assert exact equality. No real model/provider calls, no RNG, no time.sleep.
- Stdlib only. Do NOT modify any existing `src/rune/*.py` (except adding new files) or `shared/*`. Only existing-file edits allowed: `src/rune/orchestration/composition.py` (AUTHORITY_KINDS + registration), `.gitea/workflows/ci.yml` (add tests + canary), `docs/` additions, and the plan file.
- TDD: failing test first, then implementation, passing test, lint, commit. Frozen test blocks are THE SPEC: if a commit-block contradicts a frozen test, the test wins (C3 precedent) and the plan is amended in the same or an immediately-following commit.
- Lint-bytes precedent (C5 note): frozen .py blocks may trip RUF059/F841/W292 or need `ruff format`; implementers apply the sanitizing pass (plan Step 5 pattern) and the review gate accepts cosmetic deltas with assertions intact.
- Commit style: conventional, e.g. `feat(orchestration): add resource eligibility filter`.
- The foundation (63) + cognitive (34) = 97 orchestration tests and both canaries must stay green throughout.
- Every stage files/updates a Gitea issue (standing procedure): stages open + reference their issue; controller closes the issue when the stage is both-gates-green.
## File map
- `src/rune/orchestration/resources.py` — ResourceState, ResourceCapabilities, unknown != zero
- `src/rune/orchestration/eligibility.py` — EligibilityFilter (pure deterministic gate)
- `src/rune/orchestration/router.py` — RoutableResource, AdvisoryRouter protocol, FixedRouter, CostBiasRouter
- `src/rune/orchestration/scheduler.py` — ResourceScheduler (single authority), ResourceAssignment
- `src/rune/orchestration/broker.py` — ResourceBroker.acquire + subagent inherited limits; ElasticBudget
- `src/rune/orchestration/authority.py` — build_resources() composition + register in composition root
- `src/rune/canary_resource_authority.py` — self-bootstrapping canary
- `docs/resource-authority.md` — usage
- Tests: `tests/test_orchestration_resources.py`, `_eligibility.py`, `_router.py`, `_scheduler.py`, `_broker.py`, `_authority.py` (canary + docs wrappers)
- Modify: `src/rune/orchestration/composition.py`, `.gitea/workflows/ci.yml`, `docs/`, `plans/2026-09-03-resource-authority.md`
---
### Task 1: Resource state and capabilities (unknown != zero)
**Covers:** [S8 resource registry: observed capabilities, health, locality, cost, evidence age; unknown never healthy]
**Files:**
- Create: `src/rune/orchestration/resources.py`
- Test: `tests/test_orchestration_resources.py`
**Interfaces:**
- Produces: `@dataclass(frozen=True) ResourceCapabilities(kind: str, locality: str, provider: str = "", max_concurrency: int = 1)`; `@dataclass(frozen=True) ResourceState(resource_id: str, capabilities: ResourceCapabilities, healthy: bool | None, cost_per_call: float = 1.0, evidence_age_s: float | None = None)`; `Registry()` with `register(state)`, `get(resource_id) -> ResourceState | None`, `all() -> dict[str, ResourceState]`, `snapshot() -> dict[str, ResourceSnapshot]` (mapping to records.ResourceSnapshot). `UNKNOWN` sentinel constant for healthy=None. Helper `is_usable(state) -> bool` = healthy is True (None/False unusable).
#### Step 1: Write the failing tests
Create `tests/test_orchestration_resources.py`:
```python
"""Resource state: observed facts; unknown is never healthy."""
from rune.orchestration.resources import (
Registry,
ResourceCapabilities,
ResourceState,
UNKNOWN,
is_usable,
)
def test_unknown_is_never_usable():
state = ResourceState(
"ollama", ResourceCapabilities("provider", "local"), healthy=UNKNOWN
)
assert state.healthy is None
assert is_usable(state) is False
def test_false_is_not_usable_but_true_is():
down = ResourceState("a", ResourceCapabilities("provider", "local"), healthy=False)
up = ResourceState("b", ResourceCapabilities("provider", "local"), healthy=True)
assert is_usable(down) is False
assert is_usable(up) is True
def test_registry_register_get_all():
reg = Registry()
s = ResourceState("r1", ResourceCapabilities("worker", "local"), healthy=True)
reg.register(s)
assert reg.get("r1") == s
assert reg.get("missing") is None
assert list(reg.all()) == ["r1"]
def test_snapshot_maps_to_records_resource_snapshot():
reg = Registry()
reg.register(ResourceState("r1", ResourceCapabilities("worker", "local"), healthy=True))
snap = reg.snapshot()
assert "r1" in snap
assert snap["r1"].resource_id == "r1"
assert snap["r1"].healthy is True
```
#### Step 2: Run tests to verify they fail
Run: `PYTHONPATH=src:. python3 -m pytest tests/test_orchestration_resources.py -q`
Expected: FAIL — ModuleNotFoundError
#### Step 3: Implement resources.py
```python
"""Resource state (S8): observed facts only; unknown is never healthy."""
from __future__ import annotations
from dataclasses import dataclass
from rune.orchestration.records import ResourceSnapshot
UNKNOWN = None # unknown health: never treated as healthy
@dataclass(frozen=True)
class ResourceCapabilities:
kind: str
locality: str
provider: str = ""
max_concurrency: int = 1
@dataclass(frozen=True)
class ResourceState:
resource_id: str
capabilities: ResourceCapabilities
healthy: bool | None
cost_per_call: float = 1.0
evidence_age_s: float | None = None
def is_usable(state: ResourceState) -> bool:
return state.healthy is True
class Registry:
def __init__(self) -> None:
self._states: dict[str, ResourceState] = {}
def register(self, state: ResourceState) -> None:
self._states[state.resource_id] = state
def get(self, resource_id: str) -> ResourceState | None:
return self._states.get(resource_id)
def all(self) -> dict[str, ResourceState]:
return dict(self._states)
def snapshot(self) -> dict[str, ResourceSnapshot]:
return {
rid: ResourceSnapshot(
resource_id=rid,
healthy=s.healthy,
capabilities=(s.capabilities.kind,),
locality=s.capabilities.locality,
evidence_age_s=s.evidence_age_s,
)
for rid, s in self._states.items()
}
```
#### Step 4: Run tests to verify they pass
Run: `PYTHONPATH=src:. python3 -m pytest tests/test_orchestration_resources.py -q`
Expected: 4 passed
#### Step 5: Lint and commit
Run: `ruff check src/rune/orchestration/resources.py tests/test_orchestration_resources.py --output-format=github && ruff format --check src/rune/orchestration/resources.py tests/test_orchestration_resources.py`
Expected: clean
```bash
git add src/rune/orchestration/resources.py tests/test_orchestration_resources.py
git commit -m "feat(orchestration): add resource state registry"
```
---
### Task 2: Deterministic eligibility filter
**Covers:** [S8 deterministic eligibility: remove impossible candidates; unknown/stale -> structured unavailability]
**Files:**
- Create: `src/rune/orchestration/eligibility.py`
- Test: `tests/test_orchestration_eligibility.py`
**Interfaces:**
- Produces: `EligibilityFilter()` with `eligible(states: list[ResourceState], requirement: dict) -> list[ResourceState]` — pure, deterministic; a state is eligible ONLY when `is_usable(state)` is True AND kind matches `requirement["kind"]` AND locality matches `requirement.get("locality")` (when set). `reasons(state, requirement) -> list[str]` for structured unavailability (["unhealthy"], ["unknown_health"], ["kind_mismatch"], ["locality_mismatch"]).
#### Step 1: Write the failing tests
Create `tests/test_orchestration_eligibility.py`:
```python
"""Deterministic eligibility: impossible candidates removed, never ranked."""
from rune.orchestration.eligibility import EligibilityFilter
from rune.orchestration.resources import (
ResourceCapabilities,
ResourceState,
UNKNOWN,
)
def _s(rid: str, kind: str = "worker", locality: str = "local", healthy=True):
return ResourceState(rid, ResourceCapabilities(kind, locality), healthy=healthy)
def test_filters_out_unknown_and_down():
flt = EligibilityFilter()
ok = _s("ok")
down = _s("down", healthy=False)
unk = _s("unk", healthy=UNKNOWN)
result = flt.eligible([ok, down, unk], {"kind": "worker"})
assert [r.resource_id for r in result] == ["ok"]
def test_reasons_are_structured_and_actionable():
flt = EligibilityFilter()
unk = _s("u", healthy=UNKNOWN)
assert flt.reasons(unk, {"kind": "worker"}) == ["unknown_health"]
down = _s("d", healthy=False)
assert flt.reasons(down, {"kind": "worker"}) == ["unhealthy"]
wrong_kind = _s("w", kind="gpu")
assert flt.reasons(wrong_kind, {"kind": "worker"}) == ["kind_mismatch"]
def test_locality_requirement_is_strict_when_set():
flt = EligibilityFilter()
local = _s("l", locality="local")
cloud = _s("c", locality="cloud")
assert [r.resource_id for r in flt.eligible([local, cloud], {"kind": "worker", "locality": "local"})] == ["l"]
assert [r.resource_id for r in flt.eligible([local, cloud], {"kind": "worker"})] == ["l", "c"]
def test_eligibility_is_pure_and_deterministic():
flt = EligibilityFilter()
states = [_s("a"), _s("b")]
req = {"kind": "worker"}
r1 = flt.eligible(states, req)
r2 = flt.eligible(states, req)
assert [r.resource_id for r in r1] == [r.resource_id for r in r2]
# input unmutated
assert [s.resource_id for s in states] == ["a", "b"]
```
#### Step 2: Run tests to verify they fail
Run: `PYTHONPATH=src:. python3 -m pytest tests/test_orchestration_eligibility.py -q`
Expected: FAIL — ModuleNotFoundError
#### Step 3: Implement eligibility.py
```python
"""Deterministic eligibility filter (S8).
Eligibility removes impossible candidates; it never ranks. Unknown health
is NEVER usable (unknown != zero). Reasons are structured so callers can
report unavailability instead of guessing.
"""
from __future__ import annotations
from rune.orchestration.resources import ResourceState, is_usable
class EligibilityFilter:
def eligible(self, states: list[ResourceState], requirement: dict) -> list[ResourceState]:
return [s for s in states if not self.reasons(s, requirement)]
def reasons(self, state: ResourceState, requirement: dict) -> list[str]:
out: list[str] = []
if state.healthy is None:
out.append("unknown_health")
elif not is_usable(state):
out.append("unhealthy")
if state.capabilities.kind != requirement.get("kind"):
out.append("kind_mismatch")
locality = requirement.get("locality")
if locality is not None and state.capabilities.locality != locality:
out.append("locality_mismatch")
return out
```
#### Step 4: Run tests to verify they pass
Run: `PYTHONPATH=src:. python3 -m pytest tests/test_orchestration_eligibility.py -q`
Expected: 4 passed
#### Step 5: Lint and commit
Run: `ruff check src/rune/orchestration/eligibility.py tests/test_orchestration_eligibility.py --output-format=github && ruff format --check src/rune/orchestration/eligibility.py tests/test_orchestration_eligibility.py`
Expected: clean
```bash
git add src/rune/orchestration/eligibility.py tests/test_orchestration_eligibility.py
git commit -m "feat(orchestration): add deterministic eligibility filter"
```
---
### Task 3: Advisory router
**Covers:** [S8 advisory routing: rank eligible resources; never final authority]
**Files:**
- Create: `src/rune/orchestration/router.py`
- Test: `tests/test_orchestration_router.py`
**Interfaces:**
- Produces: `@dataclass(frozen=True) RoutableResource(state: ResourceState, score: float)`; `AdvisoryRouter` Protocol with `rank(candidates: list[ResourceState], requirement: dict) -> list[RoutableResource]` (scores, highest first, deterministic tie-break by resource_id); `FixedRouter(fixed_id: str)`; `CostBiasRouter()` scoring 1.0 / (1.0 + cost_per_call), ties by resource_id ascending.
#### Step 1: Write the failing tests
Create `tests/test_orchestration_router.py`:
```python
"""Advisory router: ranks eligible candidates; never final authority."""
from rune.orchestration.router import CostBiasRouter, FixedRouter
from rune.orchestration.resources import ResourceCapabilities, ResourceState
def _s(rid: str, cost: float = 1.0):
return ResourceState(rid, ResourceCapabilities("worker", "local"), healthy=True, cost_per_call=cost)
def test_fixed_router_picks_fixed_then_id_order():
router = FixedRouter("b")
ranked = router.rank([_s("a"), _s("b"), _s("c")], {"kind": "worker"})
assert [r.state.resource_id for r in ranked] == ["b", "a", "c"]
assert ranked[0].score == 1.0
def test_cost_bias_router_prefers_cheapest_and_ties_by_id():
router = CostBiasRouter()
ranked = router.rank([_s("a", cost=5.0), _s("b", cost=1.0), _s("c", cost=1.0)], {"kind": "worker"})
assert [r.state.resource_id for r in ranked] == ["b", "c", "a"]
assert ranked[0].score == ranked[1].score > ranked[2].score # b and c tie at 0.5 (1/(1+1)); strict > would be unsatisfiable
def test_router_never_mutates_input():
router = CostBiasRouter()
states = [_s("a"), _s("b")]
router.rank(states, {"kind": "worker"})
assert [s.resource_id for s in states] == ["a", "b"]
```
#### Step 2: Run tests to verify they fail
Run: `PYTHONPATH=src:. python3 -m pytest tests/test_orchestration_router.py -q`
Expected: FAIL — ModuleNotFoundError
#### Step 3: Implement router.py
```python
"""Advisory resource router (S8).
Routing is advisory: it ranks ELIGIBLE candidates under a documented
policy. Final assignment and fallback are scheduler decisions, never the
router's. Tie-breaks are deterministic by resource_id.
"""
from __future__ import annotations
from dataclasses import dataclass
from typing import Protocol
from rune.orchestration.resources import ResourceState
@dataclass(frozen=True)
class RoutableResource:
state: ResourceState
score: float
class AdvisoryRouter(Protocol):
def rank(self, candidates: list[ResourceState], requirement: dict) -> list[RoutableResource]: ...
class FixedRouter:
def __init__(self, fixed_id: str):
self._fixed_id = fixed_id
def rank(self, candidates: list[ResourceState], requirement: dict) -> list[RoutableResource]:
out = [
RoutableResource(s, 1.0 if s.resource_id == self._fixed_id else 0.0)
for s in candidates
]
return sorted(out, key=lambda r: (-r.score, r.state.resource_id))
class CostBiasRouter:
def rank(self, candidates: list[ResourceState], requirement: dict) -> list[RoutableResource]:
out = [
RoutableResource(s, 1.0 / (1.0 + s.cost_per_call))
for s in candidates
]
return sorted(out, key=lambda r: (-r.score, r.state.resource_id))
```
#### Step 4: Run tests to verify they pass
Run: `PYTHONPATH=src:. python3 -m pytest tests/test_orchestration_router.py -q`
Expected: 3 passed
#### Step 5: Lint and commit
Run: `ruff check src/rune/orchestration/router.py tests/test_orchestration_router.py --output-format=github && ruff format --check src/rune/orchestration/router.py tests/test_orchestration_router.py`
Expected: clean
```bash
git add src/rune/orchestration/router.py tests/test_orchestration_router.py
git commit -m "feat(orchestration): add advisory resource router"
```
---
### Task 4: Resource scheduler (single authority)
**Covers:** [S8 scheduler records final assignment + bounded fallback chain; limits enforced]
**Files:**
- Create: `src/rune/orchestration/scheduler.py`
- Test: `tests/test_orchestration_scheduler.py`
**Interfaces:**
- Produces: `@dataclass(frozen=True) ResourceAssignment(task_signature: str, resource_id: str, fallback_chain: tuple[str, ...], cap_concurrency: int, policy_ref: str)`; `ResourceScheduler(registry: Registry, filter: EligibilityFilter, router: AdvisoryRouter, limits: dict | None = None)` with `acquire(task_signature: str, requirement: dict) -> ResourceAssignment | None` (None when NO eligible candidate — structured unavailability, never silent fallback-to-wrong), `assignments() -> list[ResourceAssignment]`; `limits` default `{"max_fallback_chain": 2}` — the chain is the rest of the ranked eligible list, bounded by the limit.
#### Step 1: Write the failing tests
Create `tests/test_orchestration_scheduler.py`:
```python
"""Resource scheduler: single authority for final assignment and fallback."""
from rune.orchestration.eligibility import EligibilityFilter
from rune.orchestration.router import CostBiasRouter
from rune.orchestration.resources import Registry, ResourceCapabilities, ResourceState
from rune.orchestration.scheduler import ResourceScheduler
def _reg():
r = Registry()
r.register(ResourceState("a", ResourceCapabilities("worker", "local"), healthy=True, cost_per_call=5.0))
r.register(ResourceState("b", ResourceCapabilities("worker", "local"), healthy=True, cost_per_call=1.0))
r.register(ResourceState("c", ResourceCapabilities("worker", "local"), healthy=True, cost_per_call=1.0))
return r
def test_acquire_picks_router_winner_and_records_fallback_chain():
sch = ResourceScheduler(_reg(), EligibilityFilter(), CostBiasRouter())
a = sch.acquire("t1", {"kind": "worker", "locality": "local"})
assert a.resource_id == "b"
assert a.fallback_chain == ("c", "a")
assert a.policy_ref == "cost_bias"
assert [x.resource_id for x in sch.assignments()] == ["b"]
def test_acquire_returns_none_when_no_eligible_and_unavailable_is_structured():
r = Registry()
r.register(ResourceState("down", ResourceCapabilities("worker", "local"), healthy=False))
sch = ResourceScheduler(r, EligibilityFilter(), CostBiasRouter())
assert sch.acquire("t2", {"kind": "worker"}) is None
def test_acquire_respects_locality_policy():
r = Registry()
r.register(ResourceState("local", ResourceCapabilities("worker", "local"), healthy=True))
r.register(ResourceState("cloud", ResourceCapabilities("worker", "cloud"), healthy=True, cost_per_call=0.5))
sch = ResourceScheduler(r, EligibilityFilter(), CostBiasRouter())
assert sch.acquire("t3", {"kind": "worker", "locality": "local"}).resource_id == "local"
assert sch.acquire("t4", {"kind": "worker", "locality": "cloud"}).resource_id == "cloud"
def test_fallback_chain_is_bounded_by_limits():
r = _reg()
r.register(ResourceState("d", ResourceCapabilities("worker", "local"), healthy=True, cost_per_call=2.0))
sch = ResourceScheduler(r, EligibilityFilter(), CostBiasRouter(), limits={"max_fallback_chain": 1})
a = sch.acquire("t5", {"kind": "worker"})
assert a.fallback_chain == ("c",) # bounded: only 1 fallback
```
#### Step 2: Run tests to verify they fail
Run: `PYTHONPATH=src:. python3 -m pytest tests/test_orchestration_scheduler.py -q`
Expected: FAIL — ModuleNotFoundError
#### Step 3: Implement scheduler.py
```python
"""Resource scheduler (S8): the single authority.
The scheduler owns the final decision: eligibility first (deterministic),
then advisory routing (ranking), then the lease. It records the chosen
assignment with a BOUNDED fallback chain. When nothing is eligible it
returns None — structured unavailability, never a silent wrong pick.
"""
from __future__ import annotations
from dataclasses import dataclass
from rune.orchestration.eligibility import EligibilityFilter
from rune.orchestration.resources import Registry
from rune.orchestration.router import AdvisoryRouter
@dataclass(frozen=True)
class ResourceAssignment:
task_signature: str
resource_id: str
fallback_chain: tuple[str, ...]
cap_concurrency: int
policy_ref: str
class ResourceScheduler:
def __init__(self, registry: Registry, filter: EligibilityFilter, router: AdvisoryRouter, limits: dict | None = None):
self._registry = registry
self._filter = filter
self._router = router
self._limits = limits or {"max_fallback_chain": 2}
self._assignments: list[ResourceAssignment] = []
def acquire(self, task_signature: str, requirement: dict) -> ResourceAssignment | None:
states = list(self._registry.all().values())
eligible = self._filter.eligible(states, requirement)
if not eligible:
return None
ranked = self._router.rank(eligible, requirement)
winner = ranked[0]
chain = tuple(r.state.resource_id for r in ranked[1:])
chain = chain[: self._limits.get("max_fallback_chain", 2)]
assignment = ResourceAssignment(
task_signature=task_signature,
resource_id=winner.state.resource_id,
fallback_chain=chain,
cap_concurrency=winner.state.capabilities.max_concurrency,
policy_ref=self._policy_name(requirement),
)
self._assignments.append(assignment)
return assignment
def assignments(self) -> list[ResourceAssignment]:
return list(self._assignments)
def _policy_name(self, requirement: dict) -> str:
# Router policy is part of the contract; the scheduler records it.
return requirement.get("policy", "cost_bias")
```
#### Step 4: Run tests to verify they pass
Run: `PYTHONPATH=src:. python3 -m pytest tests/test_orchestration_scheduler.py -q`
Expected: 4 passed
#### Step 5: Lint and commit
Run: `ruff check src/rune/orchestration/scheduler.py tests/test_orchestration_scheduler.py --output-format=github && ruff format --check src/rune/orchestration/scheduler.py tests/test_orchestration_scheduler.py`
Expected: clean
```bash
git add src/rune/orchestration/scheduler.py tests/test_orchestration_scheduler.py
git commit -m "feat(orchestration): add resource scheduler with bounded fallback"
```
---
### Task 5: Resource broker with subagent inheritance
**Covers:** [S8 subagents inherit the same authority object and cannot bypass locality/provider/budget/depth/concurrency policy]
**Files:**
- Create: `src/rune/orchestration/broker.py`
- Test: `tests/test_orchestration_broker.py`
**Interfaces:**
- Produces: `@dataclass(frozen=True) ElasticBudget(depth: int, concurrency: int, provider: str | None, locality: str | None, max_cost: float)`; `ResourceBroker(scheduler: ResourceScheduler)` with `acquire(task_signature: str, requirement: dict) -> ResourceAssignment | None` (delegates to scheduler) and `inherit(parent: ResourceAssignment, depth_limit: int = 3) -> ElasticBudget` — a subagent budget derived from the parent assignment: depth = min(depth_limit, 1 + parent depth implied by calls), concurrency = parent.cap_concurrency, provider/locality from the parent resource's capabilities (via a provider map), max_cost = parent cost. `ElasticBudget` carries a `within(budget: ElasticBudget, depth: int, concurrency: int, cost: float) -> bool` check.
#### Step 1: Write the failing tests
Create `tests/test_orchestration_broker.py`:
```python
"""Resource broker: single acquire path; subagents inherit policy."""
from rune.orchestration.broker import ElasticBudget, ResourceBroker
from rune.orchestration.eligibility import EligibilityFilter
from rune.orchestration.router import CostBiasRouter
from rune.orchestration.resources import Registry, ResourceCapabilities, ResourceState
from rune.orchestration.scheduler import ResourceScheduler
def _setup():
r = Registry()
r.register(ResourceState("local", ResourceCapabilities("worker", "local", provider="ollama", max_concurrency=2), healthy=True, cost_per_call=1.0))
r.register(ResourceState("cloud", ResourceCapabilities("worker", "cloud", provider="verboo", max_concurrency=4), healthy=True, cost_per_call=2.0))
sch = ResourceScheduler(r, EligibilityFilter(), CostBiasRouter())
return ResourceBroker(sch), sch
def test_broker_acquire_delegates_to_scheduler():
broker, sch = _setup()
a = broker.acquire("t1", {"kind": "worker", "locality": "local"})
assert a.resource_id == "local"
assert a.cap_concurrency == 2
def test_inherit_derives_bound_budget_from_parent():
broker, _ = _setup()
parent = broker.acquire("t1", {"kind": "worker", "locality": "local"})
budget = broker.inherit(parent)
assert budget.concurrency == 2 # parent's cap
assert budget.provider == "ollama" # parent's provider, not caller-chosen
assert budget.locality == "local"
assert budget.depth <= 3
def test_budget_within_enforces_depth_concurrency_cost():
budget = ElasticBudget(depth=3, concurrency=2, provider="ollama", locality="local", max_cost=10.0)
assert budget.within(depth=1, concurrency=2, cost=5.0) is True
assert budget.within(depth=4, concurrency=2, cost=5.0) is False # too deep
assert budget.within(depth=1, concurrency=3, cost=5.0) is False # too wide
assert budget.within(depth=1, concurrency=2, cost=15.0) is False # too costly
def test_subagent_cannot_bypass_locality_from_parent():
broker, _ = _setup()
parent = broker.acquire("t1", {"kind": "worker", "locality": "local"})
budget = broker.inherit(parent)
# A subagent asking for cloud still inherits the parent's local constraint.
assert budget.locality == "local"
```
#### Step 2: Run tests to verify they fail
Run: `PYTHONPATH=src:. python3 -m pytest tests/test_orchestration_broker.py -q`
Expected: FAIL — ModuleNotFoundError
#### Step 3: Implement broker.py
```python
"""Resource broker (S8): the single acquire path for workers.
Subagents inherit the same authority object: their budget is derived from
the parent assignment, so a subagent cannot bypass the parent's locality,
provider, concurrency, depth, or cost policy.
"""
from __future__ import annotations
from dataclasses import dataclass
from rune.orchestration.scheduler import ResourceAssignment, ResourceScheduler
@dataclass(frozen=True)
class ElasticBudget:
depth: int
concurrency: int
provider: str | None
locality: str | None
max_cost: float
def within(self, depth: int, concurrency: int, cost: float) -> bool:
return depth <= self.depth and concurrency <= self.concurrency and cost <= self.max_cost
class ResourceBroker:
def __init__(self, scheduler: ResourceScheduler):
self._scheduler = scheduler
def acquire(self, task_signature: str, requirement: dict) -> ResourceAssignment | None:
return self._scheduler.acquire(task_signature, requirement)
def inherit(self, parent: ResourceAssignment, depth_limit: int = 3) -> ElasticBudget:
provider = _provider_for(parent.resource_id)
return ElasticBudget(
depth=min(depth_limit, 1 + parent_task_depth(parent.task_signature)),
concurrency=parent.cap_concurrency,
provider=provider,
locality=_locality_for(parent.resource_id),
max_cost=_cost_for(parent.resource_id),
)
# Deterministic stand-ins: provider/locality/cost are looked up from the
# registry in a real build; here they come from a fixed contract map keyed
# by resource_id so the broker stays deterministic and testable.
def _provider_for(resource_id: str) -> str | None:
return _META.get(resource_id, {}).get("provider")
def _locality_for(resource_id: str) -> str | None:
return _META.get(resource_id, {}).get("locality")
def _cost_for(resource_id: str) -> float:
return _META.get(resource_id, {}).get("cost", 1.0)
def parent_task_depth(task_signature: str) -> int:
# "t1" -> 0; deeper signatures encode depth (e.g. "t1.a.b.c" -> 3 dots -> depth min(limit, 4))
return task_signature.count(".")
_META: dict[str, dict] = {
"local": {"provider": "ollama", "locality": "local", "cost": 1.0},
"cloud": {"provider": "verboo", "locality": "cloud", "cost": 2.0},
}
```
#### Step 4: Run tests to verify they pass
Run: `PYTHONPATH=src:. python3 -m pytest tests/test_orchestration_broker.py -q`
Expected: 4 passed
#### Step 5: Lint and commit
Run: `ruff check src/rune/orchestration/broker.py tests/test_orchestration_broker.py --output-format=github && ruff format --check src/rune/orchestration/broker.py tests/test_orchestration_broker.py`
Expected: clean
```bash
git add src/rune/orchestration/broker.py tests/test_orchestration_broker.py
git commit -m "feat(orchestration): add resource broker with subagent inheritance"
```
---
### Task 6: Resource authority composition, canary, docs, CI
**Covers:** [S8 integration; S9 verification; composition root single authority]
**Files:**
- Create: `src/rune/orchestration/authority.py`
- Create: `src/rune/canary_resource_authority.py`
- Create: `docs/resource-authority.md`
- Test: `tests/test_orchestration_authority.py`
- Modify: `src/rune/orchestration/composition.py`, `.gitea/workflows/ci.yml`
**Interfaces:**
- Produces: `build_resources() -> ResourceAuthority` dataclass holding `registry`, `filter`, `router`, `scheduler`, `broker`. `build_root` in composition.py gains `"resource_registry"` and `"resource_scheduler"` in AUTHORITY_KINDS, and a new `compose_resources(root)` helper registers the registry + scheduler into an existing CompositionRoot so `assert_single_authority()` covers them. (The earlier proposed `resource_authority_inventory` helper was dropped — `CompositionRoot.inventory()` covers the "one each" intent; the plan text above is reconciled.)
- Canary prints `RESOURCE AUTHORITY CANARY PASSED` and exits 0; self-bootstrapping sys.path.
#### Step 1: Write the failing tests
Create `tests/test_orchestration_authority.py`:
```python
"""Resource authority composition + canary/docs wrappers."""
import subprocess
import sys
from pathlib import Path
from rune.orchestration.authority import build_resources
from rune.orchestration.composition import CompositionRoot, compose_resources
from rune.orchestration.resources import ResourceCapabilities, ResourceState
ROOT = Path(__file__).resolve().parents[1]
def test_build_resources_wires_one_of_each():
authority = build_resources()
assert authority.registry is not None
assert authority.filter is not None
assert authority.router is not None
assert authority.scheduler is not None
assert authority.broker is not None
def test_compose_resources_registers_single_authority():
root = CompositionRoot()
compose_resources(root)
root.assert_single_authority()
inv = root.inventory()
assert len(inv["resource_registry"]) == 1
assert len(inv["resource_scheduler"]) == 1
def test_end_to_end_unknown_rejected_and_outage_falls_back():
authority = build_resources()
authority.registry.register(ResourceState("r1", ResourceCapabilities("worker", "local"), healthy=None))
authority.registry.register(ResourceState("r2", ResourceCapabilities("worker", "local"), healthy=True))
assignment = authority.broker.acquire("t1", {"kind": "worker", "locality": "local"})
assert assignment.resource_id == "r2" # unknown r1 never considered
def test_canary_script_runs_clean():
result = subprocess.run(
[sys.executable, str(ROOT / "src/rune/canary_resource_authority.py")],
capture_output=True,
text=True,
timeout=60,
)
assert result.returncode == 0, result.stderr
assert "RESOURCE AUTHORITY CANARY PASSED" in result.stdout
def test_docs_exist_and_reference_real_paths():
docs = (ROOT / "docs/resource-authority.md").read_text()
assert "src/rune/orchestration/" in docs
assert "pytest" in docs
```
#### Step 2: Run tests to verify they fail
Run: `PYTHONPATH=src:. python3 -m pytest tests/test_orchestration_authority.py -q`
Expected: FAIL — ModuleNotFoundError
#### Step 3: Implement authority.py, canary, docs, and composition changes
Create `src/rune/orchestration/authority.py`:
```python
"""Resource authority composition (S8).
One canonical registry, one deterministic filter, one advisory router, one
scheduler (the final authority), and one broker (the single acquire path).
"""
from __future__ import annotations
from dataclasses import dataclass
from rune.orchestration.broker import ResourceBroker
from rune.orchestration.eligibility import EligibilityFilter
from rune.orchestration.resources import Registry
from rune.orchestration.router import CostBiasRouter
from rune.orchestration.scheduler import ResourceScheduler
@dataclass
class ResourceAuthority:
registry: Registry
filter: EligibilityFilter
router: CostBiasRouter
scheduler: ResourceScheduler
broker: ResourceBroker
def build_resources() -> ResourceAuthority:
registry = Registry()
filter_ = EligibilityFilter()
router = CostBiasRouter()
scheduler = ResourceScheduler(registry, filter_, router)
broker = ResourceBroker(scheduler)
return ResourceAuthority(registry, filter_, router, scheduler, broker)
```
In `src/rune/orchestration/composition.py`:
1. Extend `AUTHORITY_KINDS` to include `"resource_registry"`, `"resource_scheduler"`.
2. Add:
```python
def compose_resources(root: CompositionRoot) -> None:
from rune.orchestration.authority import build_resources
authority = build_resources()
root.register("resource_registry", authority.registry)
root.register("resource_scheduler", authority.scheduler)
```
(Place the import inside the function to avoid import cycles; `CompositionRoot` and `AUTHORITY_KINDS` stay as-is otherwise.)
Create `src/rune/canary_resource_authority.py`:
```python
#!/usr/bin/env python3
"""Resource authority canary: unknown rejected, outage deterministic, one authority."""
import sys
import tempfile
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
from rune.orchestration.authority import build_resources # noqa: E402
from rune.orchestration.composition import CompositionRoot, compose_resources # noqa: E402
from rune.orchestration.resources import ResourceCapabilities, ResourceState # noqa: E402
def main() -> int:
authority = build_resources()
authority.registry.register(ResourceState("healthy", ResourceCapabilities("worker", "local"), healthy=True))
authority.registry.register(ResourceState("unknown", ResourceCapabilities("worker", "local"), healthy=None))
assignment = authority.broker.acquire("canary", {"kind": "worker", "locality": "local"})
if assignment is None or assignment.resource_id != "healthy":
print("RESOURCE AUTHORITY CANARY FAILED: unknown resource selected", file=sys.stderr)
return 1
root = CompositionRoot()
compose_resources(root)
root.assert_single_authority()
print("RESOURCE AUTHORITY CANARY PASSED", assignment.resource_id)
return 0
if __name__ == "__main__":
sys.exit(main())
```
Create `docs/resource-authority.md`:
```markdown
# Resource Authority
Third orchestration plan (after foundation + cognitive strata): the resource
authority layer, per design section S8.
## Components (src/rune/orchestration/)
- resources.py — ResourceState + Registry (unknown != zero)
- eligibility.py — deterministic EligibilityFilter (removes, never ranks)
- router.py — advisory router (FixedRouter, CostBiasRouter)
- scheduler.py — ResourceScheduler (single authority: lease + bounded fallback)
- broker.py — ResourceBroker + ElasticBudget (subagent inheritance)
- authority.py — build_resources() composition
## Contracts
- Unknown health is NEVER usable; a request with no eligible candidate is
structured unavailability (None), never a silent wrong pick.
- Eligibility is deterministic; routing is advisory; the scheduler owns the
final lease and records a bounded fallback chain.
- Subagents inherit an ElasticBudget from the parent assignment and cannot
bypass locality/provider/depth/concurrency/cost.
## Run
PYTHONPATH=src:. python3 -m pytest tests/ -k orchestration --ignore=tests/test_message_widgets.py -q
PYTHONPATH=src:. python3 src/rune/canary_resource_authority.py
```
#### Step 4: Run tests to verify they pass
Run: `PYTHONPATH=src:. python3 -m pytest tests/test_orchestration_authority.py -q`
Expected: 5 passed
#### Step 5: Update CI and verify everything
Modify `.gitea/workflows/ci.yml`:
1. In `unit-tests`, add `tests/test_orchestration_resources.py`, `tests/test_orchestration_eligibility.py`, `tests/test_orchestration_router.py`, `tests/test_orchestration_scheduler.py`, `tests/test_orchestration_broker.py`, `tests/test_orchestration_authority.py` (backslash-continued after the cognitive entries).
2. In the `canary` job loop, add `src/rune/canary_resource_authority.py`.
Then run:
- `PYTHONPATH=src:. python3 -m pytest tests/ -k orchestration --ignore=tests/test_message_widgets.py -q` — expect 121 passed (97 + 4 resources + 4 eligibility + 3 router + 4 scheduler + 4 broker + 5 authority).
- `PYTHONPATH=src:. python3 src/rune/canary_resource_authority.py` — RESOURCE AUTHORITY CANARY PASSED.
- `ruff check src/rune/orchestration/ tests/test_orchestration_*.py src/rune/canary_resource_authority.py --output-format=github && ruff format --check src/rune/orchestration/ tests/test_orchestration_*.py src/rune/canary_resource_authority.py` — clean (run `ruff format` on failures first).
#### Step 6: Commit
```bash
git add src/rune/orchestration/authority.py src/rune/canary_resource_authority.py docs/resource-authority.md tests/test_orchestration_authority.py src/rune/orchestration/composition.py .gitea/workflows/ci.yml
git commit -m "feat(orchestration): compose resource authority with canary and CI coverage"
```
---
## Self-review notes
Spec coverage: S8 resource authority — registry observed facts (Task 1), deterministic eligibility with structured reasons (Task 2), advisory routing (Task 3), scheduler as single authority with bounded fallback (Task 4), broker with subagent inherited ElasticBudget (Task 5), composition + canary + CI (Task 6). A4 ablation (resource routing under outage/locality/inheritance) is partially exercised by the end-to-end test in Task 6. Deferred to later plans: real provider health probes, cost accounting against live providers, the composition-root get() consumers wiring ResourceBroker into workers, and the run_all cadence-keying follow-up from Plan 2. This plan does not call any real provider; unknown/outage semantics are complete.
Deferred obligations for later tasks (from R1 review, 2026-09-03):
1. Registry.snapshot() is a LOSSY projection: it carries resource_id/healthy/capabilities=(kind,)/locality/evidence_age_s but drops provider, max_concurrency, and cost_per_call. Tasks 3-5 consume ResourceState directly (full facts), so nothing loses data now. If snapshot-level provider/cost is ever needed, records.ResourceSnapshot requires a whole-path schema migration (S9) via plan amendment — do it then, not now.
2. Whole-path schema note: records.ResourceSnapshot (pre-existing, immutable) cannot hold S8's listed "cost, limits, permissions, source evidence, version" without a migration.
Deferred obligations for Tasks 3-4 (from R2 review, 2026-09-03):
1. Eligibility requirement asymmetry: missing "kind" fails CLOSED silently (every state kind_mismatch -> empty); missing "locality" fails OPEN (not enforced). A typo'd key like "localityy" silently drops the constraint. Router (Task 3) and scheduler (Task 4) MUST validate requirements upstream (require kind; reject unknown keys or surface a warning).
2. Author router/scheduler against .get() semantics (kind missing -> fail-closed empty set; locality missing -> not enforced), matching eligibility.py's frozen code (TDD: code block is the spec).
Defect corrected during execution (R3, 2026-09-03): Task 3's frozen test asserted strict `ranked[0].score > ranked[1].score > ranked[2].score`, but under the plan's own CostBiasRouter formula (1.0/(1.0+cost)) with costs (5.0, 1.0, 1.0), b and c TIE at 0.5 — the verbatim assert was unsatisfiable against the verbatim implementation. Per TDD the tests are the spec's intent, so the committed test uses `==` for the tie (preserving the tie that Task 4's fallback_chain == ("c", "a") depends on); plan block amended to match (99b38fe).
Deferred obligations for Task 4 (from R3 review, 2026-09-03):
1. FixedRouter all-0.0 degradation: if the fixed resource is ineligible/missing, all candidates score 0.0 and the scheduler's unconditional `winner = ranked[0]` would silently route to the id-first candidate — a fixed-policy violation presented as normal. Task 4 MUST treat an all-0.0 ranking from a FixedRouter as structured unavailability (return None or a distinct reason), or verify upstream callers only feed eligible-fixed candidates.
2. policy_ref keying: Task 4's `_policy_name` derives from `requirement.get("policy", "cost_bias")`, which is router-instance-agnostic — a FixedRouter call with omitted policy key would mislabel policy_ref. Record the policy from the router instance, not the requirement, or require the policy key at aquire.
3. Minor polish (not blocking): keep the tie-equals comment in the committed test so future readers don't "fix" it back to `>`; consider clamping cost with max(cost, 0.0) if untrusted registry data feeds the router later.
DISCHARGE DEADLINE (R4 review, 2026-09-03): the two R3-pinned deferrals MUST be resolved by Task 6 composition at the latest, not deferred past composition:
1. FixedRouter all-0.0 silent-pick in scheduler.acquire (winner = ranked[0] unconditional) — if the fixed resource is ineligible, the id-first candidate silently wins (S8 violation: "silent fallback presented as the requested resource"). Discharge: treat an all-0.0 ranking from a FixedRouter as structured unavailability (return None) OR make upstream callers only feed eligible-fixed candidates.
2. policy_ref requirement-derived (requirement.get("policy","cost_bias")) is router-instance-agnostic — mislabels FixedRouter acquires. Discharge: record policy from the router instance (give routers a policy identity) when the router gains one in R5/R6.
Also carry: broker.inherit() uses a hardcoded _META map (provider/locality/cost) rather than the registry's ResourceState — determinism-safe stand-in; R5 reviewer should confirm the inherit path composes with the scheduler's recorded assignments.
DISCHARGE COMPLETE (2026-09-03, commit 958e7ec): both R3 deferrals discharged after R6 composition gates, per the R4 deadline:
1. FixedRouter all-0.0 -> classified unavailability: ResourceScheduler.acquire returns None when ranked[0].score == 0.0 (S8: never silently pick a 0.0 candidate). Test: test_fixed_router_miss_is_structured_unavailability.
2. policy_ref from router instance: routers now carry policy_name (FixedRouter="fixed", CostBiasRouter="cost_bias"); _policy_name reads it with requirement fallback. Test: test_policy_ref_comes_from_router_instance.
Also: canary order-masking fixed — unknown registered as "a_unknown" so a regression can no longer hide behind the healthy-first id tie-break. Suite 121 -> 123.
+838
View File
@@ -0,0 +1,838 @@
# Old-Schema Migration Implementation Plan
> **For agentic workers:** REQUIRED reading — the "For agentic workers" preamble
> of the approved design (`specs/2026-09-03-planning-task-orchestration-design.md`),
> and the execution conventions recorded in your session memory (TDD, frozen
> tests are the spec, C5 lint-bytes sanction, Gitea per-stage issues with the
> DESCRIPTION GATE v4, task-board discipline, two-gate review rhythm, discharge
> tests must EXECUTE the discharge — red-green proven, actor dispatch contract).
## Purpose
Plan 7 — the FINAL plan of the orchestration series. Migrates the legacy
old-schema modules (`src/rune/event_envelope.py`, `src/rune/event_bus.py`,
`src/rune/pins.py`) to the orchestration schema, per design:
> "Schema migration is whole-path: writers, readers, projections, APIs, tests,
> docs, and migration tooling move together rather than maintaining a permanent
> old/new dialect bazaar."
> Verification matrix: "Old fixture migrates once; all current readers reject
> unmigrated dialects with actionable errors."
**Contract:**
1. The orchestration package (`src/rune/orchestration/`) is the NEW canonical
schema: `records.py` (typed records incl. EventEnvelope, PinRecord),
`event_store.py` (DurableEventStore), `composition.py` (single authority).
2. The legacy modules are migrated WRITER/READER-BY-WRITER: each legacy module
gains a thin compatibility shim that adapts its old API to the new
orchestration records/store OR is replaced by a call into the orchestration
layer — never maintaining a permanent dialect bazaar.
3. **Old fixture migrates once**: a fixture written in the OLD format loads and
converts to the NEW schema exactly once; re-running is a no-op (idempotent).
4. **Current readers reject unmigrated dialects**: the new orchestration store
raises a clear, actionable error (typed) when handed an old-dialect record —
never silently misparses.
5. This plan does NOT delete legacy behavior — it makes the legacy modules
delegate to the canonical orchestration layer (Functionality is immutable;
efficiency/architecture work improves how existing functionality operates).
## Build procedure (abbreviated for this plan)
1. TDD per task: frozen tests first (the spec), red → green, full-suite
regression, both ruff gates, commit.
2. Two independent review gates per stage (spec, then quality) before the
controller closes the task + its Gitea issue.
3. Whole-plan PR → poll required checks → merge → semver bump (5 surfaces).
4. Every stage files a Gitea issue with the DESCRIPTION GATE v4 (plain summary
first; closing comments likewise).
## Task summary
| Task | Deliverable | Files | Tests |
|---|---|---|---|
| X1 | EventStore dialect guard + old-fixture migration (once, idempotent, typed reject) | event_store.py (M), migrate.py (A), test_orchestration_migrate.py (A) | 4 |
| X2 | EventEnvelope legacy shim → canonical records (writer delegates) | event_envelope.py (M), test_legacy_event_envelope.py (A) | 3 |
| X3 | EventBus legacy shim → DurableEventStore (append/query/stats delegate) | event_bus.py (M), test_legacy_event_bus.py (A) | 3 |
| X4 | pins legacy shim → PinRecord (pin/unpin/recall/list delegate) | pins.py (M), test_legacy_pins.py (A) | 4 |
| X5 | Old-fixture migration tooling + canary + docs + CI | migrate_cli.py (A), canary_schema_migration.py (A), docs/schema-migration.md (A), migrate.py (M), ci.yml (M) | 4 |
Target suite: 227 + 4 + 3 + 3 + 4 + 4 = **245** (count from code blocks; X5 carries 2 pin-discharge tests).
## Task 1: EventStore dialect guard + old-fixture migration
**Covers:** [design: schema migration — old fixture migrates once; all current
readers reject unmigrated dialects with actionable errors]
**Files:**
- Modify: `src/rune/orchestration/event_store.py` (dialect guard)
- Create: `src/rune/orchestration/migrate.py`
- Test: `tests/test_orchestration_migrate.py`
**Interfaces:**
- `event_store.DurableEventStore` gains: `append()` REJECTS an old-dialect
envelope (one whose dict lacks the canonical `event_id` field or has a
`schema_version` < the current SCHEMA_VERSION) with a typed
`UnmigratedDialectError` carrying an ACTIONABLE message (pointing at
`migrate.old_fixture_to_new`).
- `migrate.old_fixture_to_new(path: Path) -> int` — loads a JSONL fixture that
may contain OLD-dialect envelope dicts, converts each to the canonical
`EventEnvelope` via `EventEnvelope.from_dict` (filling missing fields with
the old defaults), appends to a `DurableEventStore(path.parent/'events')` —
**idempotent**: re-running after a partial/full migration converts only the
not-yet-converted entries (by event_id); returns the count of NEW records
appended this run.
#### Step 1: Write the failing tests
Create `tests/test_orchestration_migrate.py`:
```python
"""Old-fixture migration: once, idempotent, rejected-if-unmigrated."""
import json
from pathlib import Path
import pytest
from rune.orchestration.event_store import (
DurableEventStore,
UnmigratedDialectError,
)
from rune.orchestration.migrate import old_fixture_to_new
def _old_envelope(event_id: str = "evt_old_1") -> dict:
# OLD dialect: no schema_version, plain dict, old field names.
return {
"event_id": event_id,
"timestamp": "2026-01-01T00:00:00+00:00",
"source": "user",
"channel": "conversation",
"event_type": "user.message",
"payload": {"text": "hello"},
}
def _write_old_fixture(path: Path, envelopes: list[dict]) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
with open(path, "w") as fh:
for e in envelopes:
fh.write(json.dumps(e) + "\n")
def test_new_store_rejects_old_dialect_with_actionable_error(tmp_path):
store = DurableEventStore(tmp_path / "events")
with pytest.raises(UnmigratedDialectError) as exc:
store.append(_old_envelope())
assert "migrate" in str(exc.value).lower() # actionable: points at migrate
def test_old_fixture_migrates_once(tmp_path):
fixture = tmp_path / "old_fixture.jsonl"
_write_old_fixture(fixture, [_old_envelope("a"), _old_envelope("b")])
n1 = old_fixture_to_new(fixture)
assert n1 == 2
# Second run: idempotent — nothing new to convert.
n2 = old_fixture_to_new(fixture)
assert n2 == 0
store = DurableEventStore(tmp_path / "events")
assert len(store.all()) == 2
def test_partial_migration_continues_from_event_id(tmp_path):
fixture = tmp_path / "old_fixture.jsonl"
_write_old_fixture(fixture, [_old_envelope("a"), _old_envelope("b")])
assert old_fixture_to_new(fixture) == 2
# A NEW old-dialect entry appears after the first migration.
with open(fixture, "a") as fh:
fh.write(json.dumps(_old_envelope("c")) + "\n")
assert old_fixture_to_new(fixture) == 1 # only "c" is new
store = DurableEventStore(tmp_path / "events")
assert len(store.all()) == 3
def test_migrated_events_are_canonical_and_readable(tmp_path):
fixture = tmp_path / "old_fixture.jsonl"
_write_old_fixture(fixture, [_old_envelope("a")])
old_fixture_to_new(fixture)
store = DurableEventStore(tmp_path / "events")
records = [r for r in store.replay() if r["event_id"] == "a"]
assert len(records) == 1
rec = records[0]
assert rec["schema_version"] == 1
assert rec["event_type"] == "user.message"
assert rec["partition_key"] == "conversation" # old channel -> partition_key
assert rec["payload_hash"] # payload content-addressed (hash present)
assert rec["timestamp"]
```
#### Step 2: Run tests to verify they fail
Run: `PYTHONPATH=src:. python3 -m pytest tests/test_orchestration_migrate.py -q`
Expected: FAIL — ModuleNotFoundError (+ UnmigratedDialectError missing)
#### Step 3: Implement
Add to `src/rune/orchestration/event_store.py` (near the top, after the existing
error classes):
```python
class UnmigratedDialectError(RuntimeError):
"""Raised when an old-schema dialect is handed to the new store."""
def _assert_canonical(envelope: dict) -> None:
if not isinstance(envelope, dict) or "event_id" not in envelope:
raise UnmigratedDialectError(
"old-dialect event rejected — run migrate.old_fixture_to_new() "
"on the source fixture first"
)
schema = envelope.get("schema_version")
if schema is not None and schema < 1:
raise UnmigratedDialectError(
"old-dialect event rejected (schema_version < 1) — run "
"migrate.old_fixture_to_new() on the source fixture first"
)
```
And in `DurableEventStore.append`, call `_assert_canonical(event)` FIRST (before
any serialization) so an unmigrated dialect is rejected with the actionable
error. (Append already handles the canonical EventEnvelope/dict path.)
Create `src/rune/orchestration/migrate.py`:
```python
"""Old-schema fixture migration (design: schema migration).
Converts a JSONL fixture written in the OLD event dialect into the canonical
orchestration EventEnvelope schema. Idempotent by event_id: a re-run converts
only the not-yet-converted entries, so 'old fixture migrates once' holds even
after a partial or interrupted migration.
"""
from __future__ import annotations
import json
from pathlib import Path
from rune.orchestration.event_store import DurableEventStore
from rune.orchestration.records import EventEnvelope
def old_fixture_to_new(path: Path) -> int:
fixture = Path(path)
store = DurableEventStore(fixture.parent / "events")
existing = {r["event_id"] for r in store.replay()}
converted = 0
with open(fixture, "r", encoding="utf-8") as fh:
for line in fh:
line = line.strip()
if not line:
continue
raw = json.loads(line)
event_id = raw.get("event_id", raw.get("id", ""))
if not event_id or event_id in existing:
continue
payload = raw.get("payload", {})
envelope = EventEnvelope(
event_id=event_id,
event_type=raw.get("event_type", raw.get("type", "unknown")),
partition_key=raw.get("channel", "conversation"),
timestamp=raw.get("timestamp", ""),
payload_hash=_content_hash(raw),
)
store.append(envelope)
existing.add(event_id)
converted += 1
store.flush()
return converted
def _content_hash(raw: dict) -> str:
# Content-addressed: the old payload/body is hashed into payload_hash
# (the canonical envelope carries no payload body; the old content is
# recoverable via the source fixture). Deterministic across runs.
import hashlib
return "sha256:" + hashlib.sha256(
json.dumps(raw, sort_keys=True).encode("utf-8")
).hexdigest()
```
(Verified against the real API: `EventEnvelope` = event_type/partition_key/
event_id/parent_id/actor_id/device_id/authorization/payload_hash/timestamp —
NO payload body and NO source field; `DurableEventStore` = append(env)/
flush()/read(from_offset)/replay() -> list[dict] — NO all()/get(). The old
payload is content-addressed into payload_hash; the raw old content remains
recoverable via the source fixture. `timestamp` preserved when present.)
#### Step 4: Run tests to verify they pass
Run: `PYTHONPATH=src:. python3 -m pytest tests/test_orchestration_migrate.py -q`
Expected: 4 passed
#### Step 5: Lint and commit
Run: `~/.local/bin/ruff check src/rune/orchestration/event_store.py src/rune/orchestration/migrate.py tests/test_orchestration_migrate.py --output-format=github && ~/.local/bin/ruff format --check src/rune/orchestration/event_store.py src/rune/orchestration/migrate.py tests/test_orchestration_migrate.py`
Expected: clean (sanctioned W292/rewraps allowed)
```bash
git add src/rune/orchestration/event_store.py src/rune/orchestration/migrate.py tests/test_orchestration_migrate.py
git commit -m "feat(orchestration): add old-fixture migration with dialect guard"
```
## Task 2: EventEnvelope legacy shim → canonical records
**Covers:** [design: schema migration is whole-path — writers move to the new
schema, no permanent old/new dialect bazaar]
**Files:**
- Modify: `src/rune/event_envelope.py` (legacy shim)
- Test: `tests/test_legacy_event_envelope.py`
**Interfaces:**
- `EventEnvelope.create(source, channel, payload)` (the legacy classmethod)
now DELEGATES: builds a canonical `rune.orchestration.records.EventEnvelope`
(event_type from channel/source, partition_key from channel) and returns a
thin legacy-shaped wrapper (event_id/timestamp/source/channel/authority_class
accessible) — so existing callers keep working while the mutable record is
canonical. The legacy dataclass keeps its shape (compat) but is constructed
FROM the canonical record.
#### Step 1: Write the failing tests
Create `tests/test_legacy_event_envelope.py`:
```python
"""Legacy EventEnvelope shim delegates to the canonical record."""
from rune.event_envelope import EventEnvelope as LegacyEnvelope
def test_legacy_create_produces_canonical_under_the_hood():
e = LegacyEnvelope.create("user", "conversation", {"text": "hi"})
assert e.event_id
assert e.source == "user"
assert e.channel == "conversation"
def test_legacy_envelope_roundtrips_to_canonical_dict():
e = LegacyEnvelope.create("user", "conversation", {"text": "hi"})
d = e.to_canonical_dict()
assert d["event_type"] # canonical field present
assert d["partition_key"] == "conversation"
assert d["schema_version"] == 1
def test_legacy_defaults_match_old_behavior():
e = LegacyEnvelope.create("tool", "", {"x": 1})
assert e.authority_class == ""
assert e.timestamp
```
#### Step 2: Run tests to verify they fail
Run: `PYTHONPATH=src:. python3 -m pytest tests/test_legacy_event_envelope.py -q`
Expected: FAIL — test_legacy_envelope_roundtrips_to_canonical_dict (no
to_canonical_dict) and canonical delegation absent.
#### Step 3: Implement
Modify `src/rune/event_envelope.py`:
```python
def create(cls, source, channel, payload):
"""Delegate to the canonical orchestration envelope (schema migration).
The legacy EventEnvelope keeps its dataclass shape for callers, but the
canonical record is built first and the legacy wrapper is derived from it
— one dialect, not two.
"""
from rune.orchestration.records import EventEnvelope as Canonical
canonical = Canonical(
event_type=payload.get("type", channel or source),
partition_key=channel,
source=source,
payload=payload,
) if False else Canonical(
event_type=(channel or source or "unknown"),
partition_key=channel,
)
return cls(
event_id=canonical.event_id,
timestamp=canonical.timestamp,
source=source,
channel=channel,
authority_class="",
task_id=None,
payload=payload,
_canonical=canonical,
)
```
(Add `_canonical` as a dataclass field with `repr=False`; `to_canonical_dict()`
returns `self._canonical.to_dict()`; keep the legacy CLI `create`/`test`
behaviors delegating to the same path. NOTE: the canonical envelope has no
`payload`/`source` — if the canonical record keeps no body, the legacy shim is
honest: `payload` lives only on the legacy wrapper; `to_canonical_dict` carries
the canonical event type/partition/id/timestamp and a content `payload_hash`
computed from `payload`.)
#### Step 4: Run tests to verify they pass
Run: `PYTHONPATH=src:. python3 -m pytest tests/test_legacy_event_envelope.py -q`
Expected: 3 passed (count from the code block; adjust the annotation)
#### Step 5: Lint and commit
```bash
git add src/rune/event_envelope.py tests/test_legacy_event_envelope.py
git commit -m "feat(migration): delegate legacy EventEnvelope to canonical records"
```
## Task 3: EventBus legacy shim → DurableEventStore
**Covers:** [design: readers/writers move together; no dialect bazaar]
**Files:**
- Modify: `src/rune/event_bus.py`
- Test: `tests/test_legacy_event_bus.py`
**Interfaces:**
- `emit(event_type, source="", payload=None)` delegates to a canonical
`DurableEventStore` at `~/.rune/events.jsonl` (or the configured path);
`log(limit=None)` reads via `store.replay()`; `stats()` returns
(total, by_type). Existing hooks stay.
#### Step 1: Write the failing tests
Create `tests/test_legacy_event_bus.py`:
```python
"""Legacy EventBus delegates append/query/stats to the canonical store."""
import json
from pathlib import Path
from rune.event_bus import emit, log, stats
def test_emit_writes_canonical_record(tmp_path, monkeypatch):
monkeypatch.setattr("rune.event_bus._store_path", lambda: tmp_path / "events.jsonl")
emit("user.message", source="user", payload={"text": "hi"})
lines = (tmp_path / "events.jsonl").read_text().strip().splitlines()
rec = json.loads(lines[0])
assert rec["event_type"] == "user.message"
assert rec["schema_version"] == 1
def test_log_reads_back_replayed_events(tmp_path, monkeypatch):
monkeypatch.setattr("rune.event_bus._store_path", lambda: tmp_path / "events.jsonl")
emit("tool.result")
emit("user.message")
events = log()
assert [e["event_type"] for e in events] == ["tool.result", "user.message"]
def test_stats_counts_by_type(tmp_path, monkeypatch):
monkeypatch.setattr("rune.event_bus._store_path", lambda: tmp_path / "events.jsonl")
emit("a")
emit("a")
emit("b")
total, by_type = stats()
assert total == 3
assert by_type["a"] == 2 and by_type["b"] == 1
```
#### Step 2: Run tests to verify they fail
Expected: FAIL (no _store_path hook, emit/log/stats not delegating)
#### Step 3: Implement
Modify `src/rune/event_bus.py`: add a `_store_path()` helper (configurable, for
tests); `emit` builds a canonical EventEnvelope + `store.append` + `flush`;
`log` = `store.replay()`; `stats` = count + Counter on `replay()`. Keep the
CLI args intact.
#### Step 4 + 5: verify + commit
```bash
git add src/rune/event_bus.py tests/test_legacy_event_bus.py
git commit -m "feat(migration): delegate legacy EventBus to DurableEventStore"
```
## Task 4: pins legacy shim → PinRecord
**Covers:** [design: old-schema pins migrate; readers reject unmigrated]
**Files:**
- Modify: `src/rune/pins.py`
- Test: `tests/test_legacy_pins.py`
**Interfaces:**
- `pin(topic, open_question, outcome, context)` and `unpin(topic)` /
`recall(context)` / `list()` now delegate to a durable store backed by the
canonical `PinRecord` (records.py) — legacy signatures unchanged; the
outcome field remains REQUIRED (a pin without an outcome is not a pin).
#### Step 1: Write the failing tests
Create `tests/test_legacy_pins.py`:
```python
"""Legacy pins shim delegates to canonical PinRecord storage."""
import json
from pathlib import Path
from rune.pins import _store_path, list_pins, pin, recall, unpin
def test_pin_stores_canonical_pin_record(tmp_path, monkeypatch):
monkeypatch.setattr("rune.pins._store_path", lambda: tmp_path / "pins.json")
pin("guitar course", "when to launch?", "course launched with 10 lessons", "context...")
rec = json.loads((tmp_path / "pins.json").read_text())[0]
assert rec["kind"] == "root_plan" or rec["item_id"] # canonical PinRecord shape
assert rec["schema_version"] == 1
def test_pin_without_outcome_raises(tmp_path, monkeypatch):
monkeypatch.setattr("rune.pins._store_path", lambda: tmp_path / "pins.json")
import pytest
with pytest.raises(ValueError):
pin("topic", "question", "", "ctx") # outcome is REQUIRED
def test_recall_surfaces_open_pins(tmp_path, monkeypatch):
monkeypatch.setattr("rune.pins._store_path", lambda: tmp_path / "pins.json")
pin("a", "q?", "outcome-a", "ctx-a")
pin("b", "q?", "outcome-b", "ctx-b")
assert {p["topic"] for p in recall("a")} == {"a"}
def test_unpin_resolves_and_list_shows_closed(tmp_path, monkeypatch):
monkeypatch.setattr("rune.pins._store_path", lambda: tmp_path / "pins.json")
pin("a", "q?", "outcome-a", "ctx")
unpin("a", reason="done")
items = list_pins()
assert any(p["topic"] == "a" and p["status"] == "resolved" for p in items)
```
#### Step 2: Run tests to verify they fail
Expected: FAIL (no _store_path, pin not delegating, no status)
#### Step 3: Implement
Modify `src/rune/pins.py`: `_store_path()` configurable; `pin()` builds a
canonical `PinRecord(kind='pin', item_id=topic, owner='pins', ...
)` (raise ValueError when outcome empty) and persists via AtomicStore JSON;
`unpin(reason)` updates status; `recall` filters; `list_` returns all.
Legacy CLI args preserved.
#### Step 4 + 5: verify + commit
```bash
git add src/rune/pins.py tests/test_legacy_pins.py
git commit -m "feat(migration): delegate legacy pins to canonical PinRecord storage"
```
## Task 5: Migration tooling, canary, docs, CI
**Covers:** [design verification matrix: old fixture migrates once; whole-path]
**Files:**
- Create: `src/rune/orchestration/migrate_cli.py`
- Create: `src/rune/canary_schema_migration.py`
- Create: `docs/schema-migration.md`
- Test: `tests/test_orchestration_migration_canary.py`
- Modify: `src/rune/orchestration/migrate.py` (evidence-store body preservation + source.channel — X1 pins; legacy-pins conversion — X4 C7)
- Modify: `.gitea/workflows/ci.yml`
**Interfaces:**
- `migrate_cli.py`: `python -m rune.orchestration.migrate_cli <fixture> [--store DIR]`
runs `old_fixture_to_new`, prints `MIGRATED N`, exits 0; idempotent re-run
prints `MIGRATED 0`; a `--pins <legacy.db> <out.json>` mode converts legacy
SQLite wing='pins' chunks to canonical PinRecords (X4 C7 discharge).
- `migrate.py` gains (X1 pins discharge): each migrated raw line is ALSO written
to the EvidenceStore (digest == payload_hash — 1:1 link) so a store-only
consumer can recover the body; `source.channel` is extracted when the old
event nests the channel inside source (real event_bus data); `legacy_pins_to_new(legacy_db, out_json)`.
- Canary: builds a tiny old-dialect fixture in a tempdir, migrates once,
re-runs (0), asserts the store rejects a fresh old-dialect append with the
typed error, prints `SCHEMA MIGRATION CANARY PASSED`.
- Docs: `docs/schema-migration.md` — the migration contract, the CLI, the
one-dialect rule.
- CI: add the 5 new test files (`_migrate`, `_legacy_event_envelope`,
`_legacy_event_bus`, `_legacy_pins`, `_migration_canary`) + canary.
#### Step 1: frozen tests
Create `tests/test_orchestration_migration_canary.py`:
```python
"""Migration tooling + canary wrapper."""
import subprocess
import sys
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
def test_cli_migrates_and_is_idempotent(tmp_path):
import json
fixture = tmp_path / "f.jsonl"
fixture.write_text(json.dumps({"event_id": "a", "event_type": "t", "channel": "c"}) + "\n")
r1 = subprocess.run(
[sys.executable, "-m", "rune.orchestration.migrate_cli", str(fixture)],
capture_output=True, text=True, timeout=60,
env={"PYTHONPATH": "src", "PATH": "/usr/bin:/bin"},
)
assert r1.returncode == 0 and "MIGRATED 1" in r1.stdout
r2 = subprocess.run(
[sys.executable, "-m", "rune.orchestration.migrate_cli", str(fixture)],
capture_output=True, text=True, timeout=60,
env={"PYTHONPATH": "src", "PATH": "/usr/bin:/bin"},
)
assert "MIGRATED 0" in r2.stdout
def test_migrated_body_preserved_in_evidence_store(tmp_path):
# X1 pin discharge: the raw migrated line is stored in the EvidenceStore
# keyed by its payload_hash (1:1), so a store-only consumer can recover
# the body (payload_hash is one-way in the event record alone).
import json
from rune.orchestration.evidence_store import EvidenceStore
from rune.orchestration.migrate import old_fixture_to_new
fixture = tmp_path / "f.jsonl"
raw = {"event_id": "a", "event_type": "t", "channel": "c", "payload": {"x": 1}}
fixture.write_text(json.dumps(raw) + "\n")
old_fixture_to_new(fixture)
store = DurableEventStore(tmp_path / "events")
rec = [r for r in store.replay() if r["event_id"] == "a"][0]
ev = EvidenceStore(tmp_path / "evidence")
body = ev.get(rec["payload_hash"])
assert body is not None
assert json.loads(body)["payload"]["x"] == 1
def test_legacy_pins_converted_to_canonical(tmp_path, monkeypatch):
# X4 C7 discharge: the --pins conversion reads legacy SQLite wing='pins'
# chunks and writes canonical PinRecords, so existing pins are NOT
# orphaned.
import json
import sqlite3
from rune.orchestration.migrate import legacy_pins_to_new
db = sqlite3.connect(tmp_path / "legacy.db")
db.execute("CREATE TABLE chunks (text TEXT, wing TEXT, room TEXT)")
db.execute(
"INSERT INTO chunks (text, wing, room) VALUES (?, 'pins', 'open')",
("[topic] guitar course\n[open question] when?\n[outcome] launched\n[status] open\n[context] ctx",),
)
db.commit()
db.close()
n = legacy_pins_to_new(tmp_path / "legacy.db", tmp_path / "pins.json")
assert n == 1
recs = json.loads((tmp_path / "pins.json").read_text())
assert recs[0]["kind"] == "pin"
assert recs[0]["item_id"] == "guitar course"
assert recs[0]["state"] == "open"
def test_canary_script_runs_clean():
result = subprocess.run(
[sys.executable, str(ROOT / "src/rune/canary_schema_migration.py")],
capture_output=True, text=True, timeout=60,
)
assert result.returncode == 0, result.stderr
assert "SCHEMA MIGRATION CANARY PASSED" in result.stdout
```
#### Step 2..6: implement per the interfaces above; verify (suite 249), lint, CI, commit
```bash
git add src/rune/orchestration/migrate_cli.py src/rune/canary_schema_migration.py docs/schema-migration.md tests/test_orchestration_migration_canary.py src/rune/orchestration/migrate.py .gitea/workflows/ci.yml
git commit -m "feat(migration): add migration CLI, canary, docs, CI"
```
## Self-review note
- The plan-cross-check found Task 1's first draft assumed a `payload`/`source`
body + `all()`/`get()` on the store — the CANONICAL EventEnvelope has no
payload body (content-addressed payload_hash) and the store exposes
replay()/read()/append/flush. Corrected the frozen tests + migrate impl to
the real API BEFORE dispatch.
- Contract: old fixture migrates once (idempotent by event_id); new store
rejects unmigrated dialects with UnmigratedDialectError (actionable);
legacy writers delegate to canonical records (no dialect bazaar).
- Targets: X1 4, X2 3, X3 3, X4 4, X5 2 (count from code blocks); suite
227 → 245.
34. X1 execution-time latent defects (2026-09-03, commit 0a097aa; all semantics-preserving,
plan blocks amended to match committed code):
(1) Frozen tests used store.all() (2 lines) — the canonical store has NO all(); replaced with
len(store.replay()), identical semantics.
(2) Plan's _assert_canonical did NOT reject the frozen old envelope (it HAS event_id, no
schema_version) — added a legacy-field marker check (payload/source/channel present) so
the raw-dict AttributeError case becomes the typed UnmigratedDialectError before
serialization. event_id/schema_version branches kept verbatim.
(3) append now serializes canonical-shaped dicts directly (plan claimed it already handled
the dict path; .to_dict() would otherwise crash).
(4) Removed dead `payload = raw.get("payload", {})` line (F841).
(5) Sanctioned rewraps (W292 + formatter).
RED-GREEN PROVEN: removing the `event_id in existing` idempotency guard makes
test_old_fixture_migrates_once FAIL (assert 2 == 0 — re-converts both) — verified.
Suite 227 -> 231.
35. X1 post-quality discharge (2026-09-03; quality gate general-138 Approved with minors):
- Important 1 DISCHARGED (dict-append validation hole): the dict path now validates via an
EXACT EventEnvelope.from_dict round-trip — a garbage "canonical"-shaped dict (event_id +
schema_version but missing event_type/partition_key) raises UnmigratedDialectError, never
persisted. RED-GREEN PROVEN: reverting the fix makes test_garbage_canonical_shaped_dict_rejected
FAIL. Pinned by 3 new tests (garbage dict rejected + no record persisted; canonical dict
round-trips and appends — false-positive guard; legacy authority_class-only rejected).
- Minor 4 DISCHARGED (marker gap): legacy markers now include authority_class/task_id.
- PINNED to X5 (cross-module MUSTs): (Important 2) body-of-record lives ONLY in the fixture —
payload_hash is one-way; X5 MUST write each raw migrated line into the EvidenceStore
(digest == payload_hash, 1:1 link; evidence_store produces the identical sha256 format) so a
store-only consumer can recover the body; (Important 3) real event_bus lines nest channel
inside source ("channel":"chat") — migrate.py defaults to "conversation" and would
silently repartition real data; X5 MUST extract source.channel when present. Minor 5/6:
crash-mid-flush is safe-by-idempotency (whole-line writes; loud CorruptEventStoreError on
truncation — acceptable); non-dict fixture lines (raw AttributeError) wrapped by X5 CLI.
- Migrate tests 4 -> 7; event_store tests unchanged (6); suite 231 -> 234.
36. X2 execution-time latent defects (2026-09-03, commit a489f53; semantics-preserving):
(1) The plan's Step 3 assumed `create` classmethod EXISTED — it did not; added it (frozen
tests require it). (2) Cleaned the dead `if False else` drafting branch in the plan block
(implemented `Canonical(event_type=(channel or source or "unknown"), partition_key=channel)`
only — the drafted block would have crashed: Canonical has no source/payload). (3)
`_canonical` field set `repr=False` AND `compare=False` (plan said repr only) to preserve
legacy equality. (4) to_canonical_dict() returns canonical.to_dict() + payload_hash
computed from the legacy payload (X1 sha256 convention). (5) CLI/create_envelope() NOT
rerouted (authority/urgency classification must keep working; shim is opt-in via
EventEnvelope.create). (6) to_dict() stays legacy-shaped (callers expect it). Legacy CLI
`test` passes; PRE-EXISTING ruff violations in event_envelope.py left untouched
(minimal-diff discipline — do not reformat legacy files in migration tasks).
NOTE (suite-counting): `-k orchestration` DESELECTS the test_legacy_*.py files (node IDs lack
"orchestration") — deltas must be confirmed via FULL-SUITE runs (406 passed incl. +3).
37. X2 post-quality discharge (2026-09-03; quality gate general-141 Approved with minors):
- Important 1 DISCHARGED (to_canonical_dict trap): envelopes built WITHOUT create()
(legacy direct constructor / create_envelope / CLI path) previously raised a raw
AttributeError on the canonical bridge. Now raises a CLEAR typed RuntimeError
("canonical bridge unavailable — use EventEnvelope.create()"). RED-GREEN PROVEN
(raw AttributeError → test fails). Pinned by test_direct_construction_has_no_canonical_bridge.
- Important 3 DISCHARGED (event_type vocabulary): create() now maps channel through the
legacy CHANNELS prior (tool -> tool_result, conversation -> conversation, etc.) instead of
the raw channel — aligns with legacy authoring intent + canonical dotted vocabulary.
RED-GREEN PROVEN. Pinned by test_create_maps_channel_to_legacy_prior_event_type.
- PINNED to X5 (dual-dialect drift + deprecation): the wrapper is two-dialect AFTER create
(legacy fields mutable, _canonical frozen at create; to_dict stays legacy-shaped); nothing
warns or forces legacy callers to migrate. X5 MUST route create_envelope/CLI through
create() (to_dict stays legacy-shaped) + add a deprecation/warning path for legacy to_dict
callers. Live-payload-hash (hash computed at call time, mutation changes it) + _content_hash
DRY with migrate.py — snapshot/DRY at X5. Tests: legacy tests 3 -> 5; full suite 406 -> 408.
38. X3 execution-time notes (2026-09-03, commit 8781373):
- REQUIRED consumer adaptation (functionality-immutable, NOT scope creep): the frozen test
mandates stats() -> (total, by_type) tuple; the TUI /events consumer (rune_tui.py
_cmd_events) read the old dict shape and would break — adapted to unpack the tuple,
output preserved. The plan's Files list gains src/rune/rune_tui.py (sanctioned).
- LATENT BUG PINNED (NOT fixed, pre-existing): rune_tui.py ~3011 calls
`event_bus.emit("on_context", {"text":..., "channel":...})` — a dict passed as the second
arg (source) — after X3 this becomes a dict partition_key (garbage data, no crash). Fix in
a follow-up (pass the dict as payload: emit("on_context", payload={...}) or adapt).
- Rotation + sidecar stats LOST (accepted tradeoff): legacy 5MB rotation / events_stats.json
update dropped when delegating to DurableEventStore (no rotation). Not reimplemented.
- Real journal data: ~/.rune journal/events.jsonl EXISTS (7.6 KB old-dialect, Aug 28) —
X5's live-data conversion must handle the mixed-dialect file (extract id/type/source.channel).
- stats() tuple change is the frozen-test mandate; the legacy CLI --json output keeps the old
{total,types,file_size} shape (compat).
39. X3 post-quality discharge (2026-09-03; quality gate general-144 Approved with minors):
- I-1 DISCHARGED (non-str source poison): emit() now REJECTS a non-string source with a
clear ValueError ("pass the payload via payload=...") — previously a dict source (the
pinned rune_tui.py emit('on_context', {dict}) misuse) silently became a dict partition_key
in the canonical journal, violating the str contract (round-trip does NOT crash — silent
type-drift poison). The TUI caller is FIXED to pass the dict as payload. RED-GREEN PROVEN
(revert guard -> test fails). Pinned by test_emit_rejects_non_str_source (raises + nothing
written). Legacy tests 3 -> 4; full suite 411 -> 412.
- PINNED to X5 (Important 2): emit() constructs a fresh DurableEventStore PER CALL and the
store __init__ scans the whole journal (O(n) write path, measured 1.2ms@10k / 6.3ms@100k) —
with rotation gone the journal grows unboundedly. X5 MUST add store compaction/rotation OR
persisted offset (and the store fix benefits the TUI /events command immediately).
- Minor carry (M-1 event_type vocabulary guard decision; M-2 unwritable-dir policy — verified
consistently fail-loud incl. legacy; M-3 contracts.py:107 stale 'events_bus.stats returns
dict' — update name at X5; M-4 test gaps -> X5 suite: emit(''), large-journal, vocab, etc.).
40. X4 execution-time notes (2026-09-03, commit b5b90c7; sanctioned deviations):
(1) CANONICAL EXTENSION (the "or extend?" path, sanctioned): PinRecord gains a defaulted
`details: str = ""` field — the pin payload blob (question/outcome/context/reason as
JSON) lives IN the canonical record; the legacy dict (topic/question/outcome/context/
status/reason) is a projection. Defaulted field keeps existing constructions + the
records round-trip (16) green; schema_version stays 1. kind comment gains '| pin';
state comment gains '| open | resolved'.
(2) REQUIRED consumer adaptation (amend-38 precedent): rune_tui.py 1-line — the open-pin
filter changed from `"open" in p.get("text","")` to `p.get("status") == "open"` (the new
projection has no "text" key).
(3) Behavior changes mandated by the frozen tests: pin() now RAISES ValueError on empty
outcome (legacy returned {"pinned": False}); recall()/list_pins() return projections
with topic/status keys (legacy returned text-chunk dicts); pins persist as canonical
PinRecord JSON via AtomicStore (legacy wrote SQLite hybrid-store chunks). The legacy
SQLite hybrid-store writer is REPLACED (migration = new writer, not a dialect bazaar).
Legacy tests 4; full suite 412 -> 416.
41. X4 post-quality discharge (2026-09-03; quality gate general-147 Approved with minors):
- DISCHARGED (I4 edge, cheap in-place): recall("") previously matched EVERY open pin ("" in x
always true) — the matches-all edge. Empty context is now ABSTENTION (no signal, no pins;
list_pins shows all). RED-GREEN PROVEN. Pinned by test_recall_empty_context_abstains.
Legacy pins tests 4 -> 5; full suite 416 -> 417.
- PINNED to X5 (Critical C7 — legacy pins ORPHANED): new pins go to store/pins.json;
nothing reads SQLite wing='pins' anymore; existing users' pins vanish from recall/list
after this ships. X5 MUST add a pins-store conversion (read legacy chunks, parse
[topic]/[question]/[outcome]/[status]/[context]/[reason], write PinRecord array).
- PINNED to X5 (Importants): I4 recall ranking regression (legacy chunk_recall BM25/dense vs
substring — X5 should delegate ranking to the store layer or document); I1 details-blob =
known second-dialect-in-record tradeoff (blob format sort_keys MUST stay frozen); I3
concurrency (whole-array rewrite, last-write-wins, no lock — single-writer/TUI-primary
discipline, document); I5 resolve semantics (unpin resolves only most-recent open;
re-pin duplicates accumulate; re-open impossible — one-way resolve, document + decoupled).
- Minor carry: DRY two decode sites (extract a _set_reason helper at X5); test-1 assertion
tautological (colorable only); list_pins default open_only True->False (TUI consumer
verified safe).
42. X5 pre-dispatch amendments (2026-09-03, controller, G6-capstone rule — accumulated pins land
in the capstone spec BEFORE dispatch):
1. X1 pins DISCHARGED in spec: migrate.py now ALSO writes each migrated raw line to the
EvidenceStore (digest == payload_hash, 1:1 link) so a store-only consumer recovers the
body; source.channel extracted when the old event nests channel inside source (real
event_bus data). Pinned by test_migrated_body_preserved_in_evidence_store.
2. X4 C7 DISCHARGED in spec: migrate_cli gains `--pins <legacy.db> <out.json>`;
migrate.legacy_pins_to_new(legacy_db, out_json) converts legacy SQLite wing='pins'
chunks to canonical PinRecords (parses [topic]/[question]/[outcome]/[status]/[context]).
Pinned by test_legacy_pins_converted_to_canonical.
3. Canary carries the WIRING PROOF (audit fix): the legacy event_bus.emit is exercised at
boot through the canonical store (not just in tests) — the legacy shims are proven wired.
4. Count: X5 tests 2 -> 4; suite target 243 -> 245.
(X3 I-2 compaction/offset + X4 recall-ranking/concurrency/resolve-semantics remain pinned to
the real-transport/hardening follow-up, NOT this capstone — they are store-design changes.)
43. X5 execution-time deviations (2026-09-03, commit 8c12552; semantics-preserving):
(1) The evidence frozen test referenced DurableEventStore unimported (NameError) — added the
import to the plan test block (sanctioned). (2) `[...][0]` -> `next(...)` (RUF015;
identical semantics, single match). (3) Sanctioned ruff rewraps + W292 in the 3 new
files. EvidenceStore digest finding: `put(json.dumps(raw, sort_keys=True).encode())`
returns 'sha256:'+64hex — byte-identical to migrate's `_content_hash` (1:1 confirmed).
X5 tests 4; full suite 417 -> 421.
44. X5 post-quality discharge (2026-09-03; quality gate general-150 Approved with minors;
controller discharged all 3 Importants — they are data-protective for real-data migration):
- I-3 DISCHARGED (evidence-orphan window): old_fixture_to_new now FLUSHES PER RECORD
(evidence first, then append+flush) — a crash never leaves evidence blobs for events that
were never appended. RED-GREEN PROVEN (flush removal -> discharge test fails).
- I-1 DISCHARGED (missing-status guessing): legacy_pins_to_new SKIPS a chunk without an
explicit [status] line (never guesses open — resolved-state protection). RED-GREEN PROVEN.
- I-2 DISCHARGED (silent drops): both migrators now surface skipped counts —
old_fixture_stats(converted, skipped) (old_fixture_to_new keeps the int contract for the
X1 frozen tests; the CLI uses stats) and legacy_pins_to_new -> (converted, skipped); CLI
prints 'MIGRATED N SKIPPED M' / 'PINS MIGRATED N SKIPPED M'. RED-GREEN PROVEN.
Pinned by test_migration_never_silently_drops_data. Full suite 421 -> 422.
- Minor carry (hardening/real-transport plan): CLI exception net narrow (FileNotFoundError/
sqlite3.OperationalError traceback; --pins silently wins over fixture); canary could
read-only assert _store_path() == EVENT_LOG; evidence not re-healed for skipped ids
(doc wording); partition-key vintage mismatch note for pre-X5 partial migrations; test
gaps (non-object JSON source, half-migrated resume >batch_size, evidence-not-rehealed).
File diff suppressed because it is too large. Load diff
+79
View File
@@ -0,0 +1,79 @@
---
feature: gitea-hardening
status: delivered
specs: []
plans:
- plans/2026-08-26-gitea-hardening.md
branch: feat/gitea-hardening
commits: 98b8a70..HEAD
---
# Gitea Hardening — Final Report
## What Was Built
Gitea repository governance and recovery are now reproducible from tracked code rather than undocumented live state. CI runs on every branch push while the protected merge gate remains pull-request-only; managed content labels reconcile without deleting human-owned labels; and branch protection, labels, milestones, and six wiki pages are declared in a validated manifest with a read-only-by-default provisioner.
Backups now fail closed unless the Foundry destination is an exact, separately mounted filesystem. Publication is overlap-safe, collision-safe, checksum-last, signal-safe, and compatible with legacy archive/checksum pairs. The disposable restore verifier validates legacy and current checksums, archive paths and members, SQLite confinement and integrity, database-to-repository completeness, bare Git repositories, archived application data, and a rootless networkless Gitea boot from an immutable local image digest. Production Gitea is never a restore target.
## Architecture
- `config/gitea_repository_state.json` and `docs/wiki/*.md` are the canonical desired state.
- `shared/gitea_provision.py` validates the manifest before credential access, performs GET-only drift checks by default, and applies managed creates/updates without deleting unmanaged objects.
- `.gitea/workflows/ci.yml` runs all-branch CI, offline Gitea tests, shell syntax checks, and the PR-only required gate. Checkout actions in the modified CI workflow use the audited immutable v4.2.2 commit.
- `.gitea/workflows/auto-label.yml` reconciles the explicit managed-label set while preserving labels outside that boundary.
- `scripts/gitea-backup.sh` performs a consistent official dump, verifies mandatory members independently, and atomically publishes archive, metadata, then checksum as the commit marker.
- `scripts/gitea-restore-verify.sh` reconstructs only inside a guarded temporary directory, rejects unsafe SQL/Git/archive inputs, and boots the restored state in rootless Podman with `--network none`, `--read-only`, `--pull=never`, and a digest-pinned image.
### Design Decisions
- Desired repository state is declarative because live policy is evidence, not reproducible configuration.
- Check mode is the default and credential-bearing transport requires HTTPS except literal loopback addresses.
- Managed labels have an explicit ownership boundary because Gitea does not expose label provenance.
- Backup publication uses same-filesystem hard links and inode identity because check-then-rename cannot prevent races or signal-window orphaning.
- Restore SQL uses SQLite semantic authorization rather than textual scanning, and Git runs in an isolated environment with external alternates forbidden.
- A restore is considered recoverable only when database-declared repositories, application data, and an isolated Gitea boot all agree.
## Usage
```bash
# Read-only repository-state drift check
python3 shared/gitea_provision.py --check
# Reconcile managed repository state, then prove convergence
python3 shared/gitea_provision.py --apply
# Production backup; fails unless BACKUP_MOUNT is a distinct exact mount
sudo scripts/gitea-backup.sh
# Disposable restore drill against a local archive and sidecar
RUNE_GITEA_RESTORE_VERIFY=1 \
scripts/gitea-restore-verify.sh /path/to/gitea-dump-YYYYMMDDTHHMMSSZ.tar.gz
```
The restore image must already exist locally at the audited digest. The verifier refuses registry pulls, root execution, non-SQLite backups, unsafe archive members, external Git object stores, incomplete database repositories, and any target outside its guarded temporary workspace.
## Verification
- Combined strict unit suite: 157 tests passed with `ResourceWarning` promoted to errors; the merged v1-focused pytest suite passed 100 tests and 42 subtests.
- CI canaries: pipeline 20/20, Gitea automation 37/37, TUI drift 32/32.
- ShellCheck, Bash syntax, Python compilation, manifest JSON validation, diff whitespace, and secret scans passed.
- Live read-only provisioning check reported that Gitea repository state is converged.
- The newest Pi backup completed a real disposable drill: checksum, safe archive extraction, SQLite integrity, four bare Git repositories reconciled to the database, archived application data, and isolated rootless networkless Gitea boot.
- Independent spec and code-quality reviews ended approved with no open findings.
## Journey Log
> Brief notes on what informed the final design. Not required reading.
- [pivot] Live branch policy, milestones, and wiki pages were converted into tracked desired state after the audit showed operational state was not reproducible.
- [lesson] Redaction must happen at the transport boundary; response bodies, reason phrases, malformed redirects, and partial-token reflections are all untrusted.
- [pivot] Backup publication moved from rename bookkeeping to atomic hard links and inode ownership after race and signal probes exposed partial-publication gaps.
- [lesson] A restore drill must reconcile database rows, Git objects, application data, and an actual isolated boot; validating whichever files happen to exist is not recovery evidence.
- [lesson] Networkless containers still pull images unless local presence and `--pull=never` are both enforced.
## Source Materials
| File | Role | Notes |
|------|------|-------|
| `plans/2026-08-26-gitea-hardening.md` | Implementation plan | Covers CI, provisioning, backup/restore, and integration verification |
+178 -35
View File
@@ -1,52 +1,195 @@
#!/usr/bin/env bash
set -euo pipefail
BACKUP_DIR="/media/pi/Foundry/backups/gitea"
RETENTION_DAYS=30
BACKUP_MOUNT=${BACKUP_MOUNT:-/media/pi/Foundry}
BACKUP_DIR=${BACKUP_DIR:-$BACKUP_MOUNT/backups/gitea}
RETENTION_DAYS=${RETENTION_DAYS:-30}
LOCK_FILE=${LOCK_FILE:-/run/lock/rune-gitea-backup.lock}
WORK_ROOT=${WORK_ROOT:-/var/lib/gitea}
BACKUP_OWNER=${BACKUP_OWNER-pi:pi}
GITEA_RUN_USER=${GITEA_RUN_USER-git}
GITEA_BIN=${GITEA_BIN:-/usr/local/bin/gitea}
GITEA_CONFIG=${GITEA_CONFIG:-/etc/gitea/app.ini}
GITEA_WORK_PATH=${GITEA_WORK_PATH:-/var/lib/gitea}
SYSTEMCTL_BIN=${SYSTEMCTL_BIN:-systemctl}
CURL_BIN=${CURL_BIN:-curl}
SHA256SUM_BIN=${SHA256SUM_BIN:-sha256sum}
REALPATH_BIN=${REALPATH_BIN:-realpath}
MOUNTPOINT_BIN=${MOUNTPOINT_BIN:-mountpoint}
FINDMNT_BIN=${FINDMNT_BIN:-findmnt}
STAMP=$(date -u +%Y%m%dT%H%M%SZ)
ARCHIVE="gitea-dump-${STAMP}.tar.gz"
WORK_DIR=$(mktemp -d /var/lib/gitea/backup.XXXXXX)
GITEA_WAS_ACTIVE=0
cleanup() {
if [ "$GITEA_WAS_ACTIVE" -eq 1 ] && ! systemctl is-active --quiet gitea; then
systemctl start gitea
fi
rm -rf -- "$WORK_DIR"
}
trap cleanup EXIT
die() { printf 'Gitea backup failed: %s\n' "$*" >&2; exit 1; }
install -d -o pi -g pi -m 0750 "$BACKUP_DIR"
chown -R git:git "$WORK_DIR"
# Check both lexical containment and symlink-aware containment before touching
# the destination. The configured mount itself must be mounted on a device
# distinct from / so an absent off-disk volume can never become a local folder.
mount_lex=$("$REALPATH_BIN" -m -s -- "$BACKUP_MOUNT") || die "cannot normalize BACKUP_MOUNT"
backup_lex=$("$REALPATH_BIN" -m -s -- "$BACKUP_DIR") || die "cannot normalize BACKUP_DIR"
mount_real=$("$REALPATH_BIN" -m -- "$BACKUP_MOUNT") || die "cannot resolve BACKUP_MOUNT"
backup_real=$("$REALPATH_BIN" -m -- "$BACKUP_DIR") || die "cannot resolve BACKUP_DIR"
case "$backup_lex" in "$mount_lex"|"$mount_lex"/*) ;; *) die "BACKUP_DIR is outside BACKUP_MOUNT" ;; esac
case "$backup_real" in "$mount_real"|"$mount_real"/*) ;; *) die "BACKUP_DIR resolves outside BACKUP_MOUNT" ;; esac
"$MOUNTPOINT_BIN" -q -- "$BACKUP_MOUNT" || die "BACKUP_MOUNT is not mounted"
mount_device=$("$FINDMNT_BIN" -n -r -o MAJ:MIN --target "$BACKUP_MOUNT") \
|| die "cannot identify BACKUP_MOUNT backing device"
root_device=$("$FINDMNT_BIN" -n -r -o MAJ:MIN --target /) \
|| die "cannot identify root backing device"
[[ "$mount_device" =~ ^[0-9]+:[0-9]+$ && "$root_device" =~ ^[0-9]+:[0-9]+$ ]] \
|| die "ambiguous backing device information"
[[ "$mount_device" != "$root_device" ]] || die "BACKUP_MOUNT is backed by the root device"
if systemctl is-active --quiet gitea; then
GITEA_WAS_ACTIVE=1
systemctl stop gitea
mkdir -p -- "$(dirname -- "$LOCK_FILE")" "$WORK_ROOT"
exec 9>"$LOCK_FILE"
if ! flock -n 9; then
printf 'Another Gitea backup is already running\n' >&2
exit 75
fi
sudo -u git /usr/local/bin/gitea dump \
--config /etc/gitea/app.ini \
--work-path /var/lib/gitea \
--tempdir "$WORK_DIR" \
--type tar.gz \
--file "$WORK_DIR/$ARCHIVE" \
WORK_DIR=$(mktemp -d "$WORK_ROOT/backup.XXXXXX")
GITEA_WAS_ACTIVE=0
SERVICE_RESTORED=0
STAGED_ARCHIVE=""
STAGED_CHECKSUM=""
STAGED_METADATA=""
restore_service() {
if ((GITEA_WAS_ACTIVE)) && ! "$SYSTEMCTL_BIN" is-active --quiet gitea; then
"$SYSTEMCTL_BIN" start gitea
fi
SERVICE_RESTORED=1
}
cleanup() {
local status=$?
if ((GITEA_WAS_ACTIVE)) && ((SERVICE_RESTORED == 0)); then
restore_service || status=$?
fi
rm -rf -- "$WORK_DIR"
# The staged checksum is the rollback marker. It remains linked until all
# three no-clobber links have succeeded, so signal handling never depends on
# a shell assignment racing an atomic ln. Only unlink a final while its inode
# still proves that it is this run's hard link; a raced replacement survives.
if [[ -n "$STAGED_CHECKSUM" && -e "$STAGED_CHECKSUM" ]]; then
if [[ -n "$STAGED_METADATA" && -e "$BACKUP_DIR/$ARCHIVE.metadata" && ! -L "$BACKUP_DIR/$ARCHIVE.metadata" && "$BACKUP_DIR/$ARCHIVE.metadata" -ef "$STAGED_METADATA" ]]; then
rm -f -- "$BACKUP_DIR/$ARCHIVE.metadata"
fi
if [[ -n "$STAGED_ARCHIVE" && -e "$BACKUP_DIR/$ARCHIVE" && ! -L "$BACKUP_DIR/$ARCHIVE" && "$BACKUP_DIR/$ARCHIVE" -ef "$STAGED_ARCHIVE" ]]; then
rm -f -- "$BACKUP_DIR/$ARCHIVE"
fi
if [[ -e "$BACKUP_DIR/$ARCHIVE.sha256" && ! -L "$BACKUP_DIR/$ARCHIVE.sha256" && "$BACKUP_DIR/$ARCHIVE.sha256" -ef "$STAGED_CHECKSUM" ]]; then
rm -f -- "$BACKUP_DIR/$ARCHIVE.sha256"
fi
fi
[[ -z "$STAGED_ARCHIVE" ]] || rm -f -- "$STAGED_ARCHIVE"
[[ -z "$STAGED_CHECKSUM" ]] || rm -f -- "$STAGED_CHECKSUM"
[[ -z "$STAGED_METADATA" ]] || rm -f -- "$STAGED_METADATA"
exit "$status"
}
trap cleanup EXIT
trap 'exit 130' INT
trap 'exit 143' TERM
install -d -m 0750 "$BACKUP_DIR"
if [[ -n "$BACKUP_OWNER" ]]; then
chown "$BACKUP_OWNER" "$BACKUP_DIR"
fi
if [[ -n "$GITEA_RUN_USER" ]]; then
chown -R "$GITEA_RUN_USER:$GITEA_RUN_USER" "$WORK_DIR"
fi
if "$SYSTEMCTL_BIN" is-active --quiet gitea; then
GITEA_WAS_ACTIVE=1
"$SYSTEMCTL_BIN" stop gitea
fi
dump_command=(
"$GITEA_BIN" dump
--config "$GITEA_CONFIG"
--work-path "$GITEA_WORK_PATH"
--tempdir "$WORK_DIR"
--type tar.gz
--file "$WORK_DIR/$ARCHIVE"
--skip-log
)
if [[ -n "$GITEA_RUN_USER" ]]; then
sudo -u "$GITEA_RUN_USER" "${dump_command[@]}"
else
"${dump_command[@]}"
fi
systemctl start gitea
for _ in $(seq 1 30); do
curl -fsS http://127.0.0.1:3000/api/v1/version >/dev/null && break
sleep 1
if ((GITEA_WAS_ACTIVE)); then
restore_service
for _ in $(seq 1 30); do
"$CURL_BIN" -fsS http://127.0.0.1:3000/api/v1/version >/dev/null && break
sleep 1
done
"$CURL_BIN" -fsS http://127.0.0.1:3000/api/v1/version >/dev/null
fi
tar -tzf "$WORK_DIR/$ARCHIVE" >"$WORK_DIR/manifest.txt"
for required in app.ini data repos gitea-db.sql; do
if ! grep -Eq "^(\\./)?${required}(/|$)" "$WORK_DIR/manifest.txt"; then
printf 'Backup archive is missing mandatory member: %s\n' "$required" >&2
exit 1
fi
done
curl -fsS http://127.0.0.1:3000/api/v1/version >/dev/null
tar -tzf "$WORK_DIR/$ARCHIVE" > "$WORK_DIR/manifest.txt"
grep -Eq '(^|/)(app.ini|data|repos|gitea-db.sql)' "$WORK_DIR/manifest.txt"
install -o pi -g pi -m 0640 "$WORK_DIR/$ARCHIVE" "$BACKUP_DIR/$ARCHIVE"
sha256sum "$BACKUP_DIR/$ARCHIVE" > "$BACKUP_DIR/$ARCHIVE.sha256"
chown pi:pi "$BACKUP_DIR/$ARCHIVE.sha256"
# Build every sidecar before publishing. The checksum is published last and is the
# commit marker: consumers must ignore archives without a matching sidecar.
STAGED_ARCHIVE=$(mktemp "$BACKUP_DIR/.${ARCHIVE}.archive.XXXXXX")
STAGED_CHECKSUM=$(mktemp "$BACKUP_DIR/.${ARCHIVE}.sha256.XXXXXX")
STAGED_METADATA=$(mktemp "$BACKUP_DIR/.${ARCHIVE}.metadata.XXXXXX")
install -m 0640 "$WORK_DIR/$ARCHIVE" "$STAGED_ARCHIVE"
digest=$("$SHA256SUM_BIN" "$STAGED_ARCHIVE" | awk '{print $1}')
[[ "$digest" =~ ^[[:xdigit:]]{64}$ ]] || { printf 'Invalid SHA-256 output\n' >&2; exit 1; }
printf '%s %s\n' "$digest" "$ARCHIVE" >"$STAGED_CHECKSUM"
printf 'created_utc=%s\narchive=%s\ngitea_config=%s\nformat=gitea-dump-tar.gz\n' \
"$(date -u +%Y-%m-%dT%H:%M:%SZ)" "$ARCHIVE" "$GITEA_CONFIG" >"$STAGED_METADATA"
chmod 0640 "$STAGED_ARCHIVE" "$STAGED_CHECKSUM" "$STAGED_METADATA"
if [[ -n "$BACKUP_OWNER" ]]; then
chown "$BACKUP_OWNER" "$STAGED_ARCHIVE" "$STAGED_CHECKSUM" "$STAGED_METADATA"
fi
if [[ -e "$BACKUP_DIR/$ARCHIVE" || -e "$BACKUP_DIR/$ARCHIVE.sha256" || -e "$BACKUP_DIR/$ARCHIVE.metadata" ]]; then
printf 'Refusing to overwrite an existing backup publication: %s\n' "$ARCHIVE" >&2
exit 1
fi
ln -T -- "$STAGED_METADATA" "$BACKUP_DIR/$ARCHIVE.metadata"
ln -T -- "$STAGED_ARCHIVE" "$BACKUP_DIR/$ARCHIVE"
ln -T -- "$STAGED_CHECKSUM" "$BACKUP_DIR/$ARCHIVE.sha256"
find "$BACKUP_DIR" -maxdepth 1 -type f \
\( -name 'gitea-dump-*.tar.gz' -o -name 'gitea-dump-*.tar.gz.sha256' \) \
-mtime "+$RETENTION_DAYS" -delete
# Removing the staged checksum disarms rollback only after the checksum-last
# commit has succeeded. The other staged links are then safe to unlink.
rm -f -- "$STAGED_CHECKSUM"
rm -f -- "$STAGED_ARCHIVE" "$STAGED_METADATA"
# Expiration applies to the publication as a unit: if any final artifact has
# aged out, remove the archive, checksum, and metadata together.
while IFS= read -r -d '' expired; do
archive_path=${expired%.sha256}
archive_path=${archive_path%.metadata}
rm -f -- "$archive_path" "$archive_path.sha256" "$archive_path.metadata"
done < <(find "$BACKUP_DIR" -maxdepth 1 -type f \
\( -name 'gitea-dump-*.tar.gz' -o -name 'gitea-dump-*.tar.gz.sha256' -o -name 'gitea-dump-*.tar.gz.metadata' \) \
-mtime "+$RETENTION_DAYS" -print0)
# Preserve complete legacy archive+checksum pairs that predate metadata. New
# publications are three-file sets; any lone pair member or metadata-bearing
# incomplete set is an interrupted/malformed publication and is removed as a
# unit.
while IFS= read -r -d '' artifact; do
archive_path=${artifact%.sha256}
archive_path=${archive_path%.metadata}
if [[ -f "$archive_path.metadata" ]]; then
if [[ ! -f "$archive_path" || ! -f "$archive_path.sha256" ]]; then
rm -f -- "$archive_path" "$archive_path.sha256" "$archive_path.metadata"
fi
elif [[ ! -f "$archive_path" || ! -f "$archive_path.sha256" ]]; then
rm -f -- "$archive_path" "$archive_path.sha256" "$archive_path.metadata"
fi
done < <(find "$BACKUP_DIR" -maxdepth 1 -type f \
\( -name 'gitea-dump-*.tar.gz' -o -name 'gitea-dump-*.tar.gz.sha256' -o -name 'gitea-dump-*.tar.gz.metadata' \) -print0)
printf 'Backup verified: %s\n' "$BACKUP_DIR/$ARCHIVE"
+311
View File
@@ -0,0 +1,311 @@
#!/usr/bin/env bash
set -euo pipefail
# Docker Registry multi-arch manifest digest for tag 1.27.0-rootless.
readonly DEFAULT_IMAGE="docker.io/gitea/gitea@sha256:414ba5b2b1163480e9ed4213a989cd798579cfa88582a2359303273009b2b852"
IMAGE=${GITEA_RESTORE_IMAGE:-$DEFAULT_IMAGE}
PODMAN_BIN=${PODMAN_BIN:-podman}
PYTHON_BIN=${PYTHON_BIN:-python3}
die() { printf 'Restore verification failed: %s\n' "$*" >&2; exit 1; }
[[ ${RUNE_GITEA_RESTORE_VERIFY:-} == 1 ]] || die "set RUNE_GITEA_RESTORE_VERIFY=1 to acknowledge the disposable drill"
((EUID != 0)) || die "must run rootless (never run this verifier as root)"
[[ $# -eq 1 ]] || die "usage: $0 ARCHIVE.tar.gz"
ARCHIVE=$(realpath -- "$1")
[[ -f "$ARCHIVE" ]] || die "archive does not exist: $ARCHIVE"
CHECKSUM="$ARCHIVE.sha256"
[[ -f "$CHECKSUM" ]] || die "checksum sidecar does not exist: $CHECKSUM"
[[ "$IMAGE" == "$DEFAULT_IMAGE" ]] || die "Gitea restore image must match the audited immutable digest"
command -v "$PODMAN_BIN" >/dev/null || die "rootless Podman is required"
GIT_BIN=$(command -v git) || die "git is required"
command -v "$PYTHON_BIN" >/dev/null || die "Python 3 is required"
"$PODMAN_BIN" image exists "$IMAGE" || die "required local image is unavailable; refusing registry contact"
WORK_DIR=$(mktemp -d "${TMPDIR:-/tmp}/gitea-restore-verify.XXXXXX")
CONTAINER_ID=""
cleanup() {
local status=$?
if [[ -n "$CONTAINER_ID" ]]; then
"$PODMAN_BIN" rm -f "$CONTAINER_ID" >/dev/null 2>&1 || true
fi
rm -rf -- "$WORK_DIR"
exit "$status"
}
trap cleanup EXIT INT TERM
# Git validation must not inherit object-store or configuration overrides from
# the caller. An isolated HOME plus an otherwise-empty environment prevents
# fsck from succeeding via host objects or hostile Git config.
mkdir -m 0700 "$WORK_DIR/git-home"
safe_git() {
env -i PATH="$PATH" HOME="$WORK_DIR/git-home" GIT_CONFIG_NOSYSTEM=1 \
"$GIT_BIN" "$@"
}
# Parse the sidecar ourselves: sha256sum -c would accept arbitrary filenames
# from an attacker-controlled sidecar.
"$PYTHON_BIN" - "$ARCHIVE" "$CHECKSUM" <<'PY' || die "checksum validation failed"
from pathlib import Path, PurePosixPath
import hashlib, re, sys
archive, sidecar = map(Path, sys.argv[1:])
contents = sidecar.read_text()
# Accept exactly one record, with either no terminator or one POSIX LF. Do not
# strip whitespace: that would hide blank/extra records or CRLF ambiguity.
line = contents[:-1] if contents.endswith("\n") else contents
if "\n" in line or "\r" in line:
raise SystemExit("invalid checksum sidecar")
match = re.fullmatch(r"([0-9a-fA-F]{64}) (.+)", line)
if not match:
raise SystemExit("invalid checksum sidecar")
# The recorded path is an inert lexical claim. Never open, stat, or resolve it;
# only compare its final POSIX component with the CLI-supplied archive name.
recorded = match.group(2)
if "\\" in recorded:
raise SystemExit("invalid checksum sidecar")
if "/" not in recorded:
valid_name = recorded == archive.name
else:
path = PurePosixPath(recorded)
components = recorded.split("/")
valid_name = (
path.is_absolute()
and not recorded.startswith("//")
and all(component not in {"", ".", ".."} for component in components[1:])
and path.name == archive.name
)
if not valid_name:
raise SystemExit("invalid checksum sidecar")
digest = hashlib.sha256()
with archive.open("rb") as stream:
for chunk in iter(lambda: stream.read(1024 * 1024), b""):
digest.update(chunk)
actual = digest.hexdigest()
if actual.lower() != match.group(1).lower():
raise SystemExit("checksum mismatch")
PY
# Validate every name and link before extracting, then require each top-level
# Gitea dump member independently. No production path is ever accepted here.
"$PYTHON_BIN" - "$ARCHIVE" "$WORK_DIR/extracted" <<'PY'
from pathlib import Path, PurePosixPath
import sys, tarfile
archive, destination = Path(sys.argv[1]), Path(sys.argv[2])
required = {"app.ini", "data", "repos", "gitea-db.sql"}
with tarfile.open(archive, "r:gz") as source:
members = source.getmembers()
present = set()
for member in members:
name = member.name.removeprefix("./")
path = PurePosixPath(name)
if not name or path.is_absolute() or ".." in path.parts:
raise SystemExit(f"unsafe archive path: {member.name}")
if not (member.isfile() or member.isdir()):
raise SystemExit(f"unsafe archive member type: {member.name}")
present.add(path.parts[0])
missing = sorted(required - present)
if missing:
raise SystemExit("missing mandatory member: " + ", ".join(missing))
destination.mkdir(mode=0o700)
source.extractall(destination)
PY
RESTORE_ROOT="$WORK_DIR/extracted"
APP_INI="$RESTORE_ROOT/app.ini"
"$PYTHON_BIN" - "$APP_INI" <<'PY'
from pathlib import Path
import configparser
import sys
def fail(message):
print(f"Restore verification failed: {message}", file=sys.stderr)
raise SystemExit(1)
try:
contents = Path(sys.argv[1]).read_text(encoding="utf-8")
except (OSError, UnicodeError):
fail("cannot read app.ini safely")
if "\x00" in contents:
fail("malformed app.ini")
# Gitea permits global keys such as WORK_PATH before the first named section.
# Give those keys an in-memory section so strict stdlib INI parsing can still
# validate the original bytes. The extracted backup is never rewritten.
config = configparser.RawConfigParser(interpolation=None, strict=True)
try:
config.read_string("[__gitea_top_level__]\n" + contents)
except (configparser.DuplicateSectionError, configparser.DuplicateOptionError) as error:
if error.section.casefold() in {"database", "default"}:
fail("ambiguous database configuration in app.ini")
fail("malformed app.ini")
except configparser.Error:
fail("malformed app.ini")
database_sections = [
section for section in config.sections() if section.casefold() == "database"
]
if len(database_sections) > 1:
fail("ambiguous database configuration in app.ini")
if not database_sections:
fail("missing database section in app.ini")
database_section = database_sections[0]
if "db_type" in config.defaults():
fail("ambiguous database configuration in app.ini")
if not config.has_option(database_section, "db_type"):
fail("missing database DB_TYPE in app.ini")
database_type = config.get(database_section, "db_type", raw=True).strip().casefold()
if database_type not in {"sqlite", "sqlite3"}:
fail("unsupported database DB_TYPE; only sqlite/sqlite3 are supported")
PY
DB="$WORK_DIR/gitea.db"
"$PYTHON_BIN" - "$RESTORE_ROOT/gitea-db.sql" "$DB" <<'PY'
from pathlib import Path
import sqlite3, sys
sql, database = Path(sys.argv[1]), Path(sys.argv[2])
connection = sqlite3.connect(database)
try:
denied_actions = {sqlite3.SQLITE_ATTACH, sqlite3.SQLITE_DETACH}
denied_functions = {"load_extension", "readfile", "writefile"}
denied_pragmas = {"data_store_directory", "temp_store_directory"}
def authorize(action, argument1, argument2, _database_name, _trigger_name):
if action in denied_actions:
return sqlite3.SQLITE_DENY
if action == sqlite3.SQLITE_FUNCTION and (argument2 or "").lower() in denied_functions:
return sqlite3.SQLITE_DENY
if action == sqlite3.SQLITE_PRAGMA and (argument1 or "").lower() in denied_pragmas:
return sqlite3.SQLITE_DENY
return sqlite3.SQLITE_OK
# The dump is untrusted input. Keep SQLite's semantic authorizer active for
# the entire import so file/extension operations cannot escape WORK_DIR.
connection.set_authorizer(authorize)
connection.executescript(sql.read_text())
connection.set_authorizer(None)
result = connection.execute("PRAGMA integrity_check").fetchone()[0]
if result != "ok":
raise SystemExit(f"SQLite integrity_check returned {result!r}")
finally:
connection.close()
PY
# Tie the filesystem inventory back to Gitea's authoritative database rows.
# Exact schema/column names and safe canonical lower-name components are
# required; orphaned owners and missing canonical repositories fail closed.
"$PYTHON_BIN" - "$DB" "$RESTORE_ROOT/repos" <<'PY'
from pathlib import Path
import sqlite3, sys
database, repos = Path(sys.argv[1]), Path(sys.argv[2])
connection = sqlite3.connect(database)
try:
objects = connection.execute(
"SELECT name, type FROM sqlite_master WHERE lower(name) IN ('repository', 'user')"
).fetchall()
if sorted(objects) != [("repository", "table"), ("user", "table")]:
raise SystemExit("absent or ambiguous repository ownership schema")
expected_columns = {
"repository": {"owner_id", "lower_name"},
"user": {"id", "lower_name"},
}
for table, required in expected_columns.items():
columns = {row[1] for row in connection.execute(f'PRAGMA table_info("{table}")')}
if not required <= columns:
raise SystemExit("absent or ambiguous repository ownership schema")
rows = connection.execute(
'SELECT u.lower_name, r.lower_name FROM "repository" AS r '
'LEFT JOIN "user" AS u ON u.id = r.owner_id'
).fetchall()
for owner, name in rows:
components = (owner, name)
if any(
not isinstance(component, str)
or not component
or component in {".", ".."}
or component != component.casefold()
or "/" in component
or "\\" in component
or "\x00" in component
for component in components
):
raise SystemExit("unsafe database repository path component")
canonical = repos / owner / f"{name}.git"
if not canonical.is_dir():
raise SystemExit("database-declared repository is missing")
except sqlite3.Error:
raise SystemExit("absent or ambiguous repository ownership schema") from None
finally:
connection.close()
PY
repo_count=0
while IFS= read -r -d '' repo; do
repo_count=$((repo_count + 1))
for alternates in "$repo/objects/info/alternates" "$repo/objects/info/http-alternates"; do
[[ ! -e "$alternates" ]] || die "repository alternates are forbidden: $alternates"
done
bare_repository=$(safe_git --git-dir="$repo" rev-parse --is-bare-repository 2>/dev/null) \
|| die "repository bare-state validation failed"
[[ "$bare_repository" == true ]] || die "repository is not bare"
safe_git --git-dir="$repo" fsck --full --strict >/dev/null
default_ref=$(safe_git --git-dir="$repo" symbolic-ref HEAD) || die "repository has no symbolic default branch: $repo"
safe_git --git-dir="$repo" show-ref --verify --quiet "$default_ref" || die "repository default branch has no ref: $repo ($default_ref)"
done < <(find "$RESTORE_ROOT/repos" -type d -name '*.git' -print0)
((repo_count > 0)) || die "no Git repositories found in backup"
INSTALL_ROOT="$WORK_DIR/isolated-gitea"
mkdir -p "$INSTALL_ROOT/gitea" "$INSTALL_ROOT/git/repositories"
cp -a -- "$RESTORE_ROOT/data/." "$INSTALL_ROOT/gitea/"
mkdir -p "$INSTALL_ROOT/gitea/conf"
cp "$DB" "$INSTALL_ROOT/gitea/gitea.db"
cp -a "$RESTORE_ROOT/repos/." "$INSTALL_ROOT/git/repositories/"
cat >"$INSTALL_ROOT/gitea/conf/app.ini" <<'EOF'
APP_NAME = Rune restore verification
RUN_MODE = prod
[server]
PROTOCOL = http
HTTP_ADDR = 127.0.0.1
HTTP_PORT = 3000
OFFLINE_MODE = true
[database]
DB_TYPE = sqlite3
PATH = /data/gitea/gitea.db
[repository]
ROOT = /data/git/repositories
[security]
INSTALL_LOCK = true
EOF
# The container has no external network, no published ports, a read-only root,
# and only the disposable restore tree mounted. Probe over its loopback only.
CONTAINER_ID=$("$PODMAN_BIN" run --pull=never -d --rm --network none --read-only \
--userns=keep-id:uid=1000,gid=1000 \
--tmpfs /tmp:rw,noexec,nosuid,nodev --tmpfs /run:rw,nosuid,nodev \
-v "$INSTALL_ROOT:/data:Z" "$IMAGE" \
gitea web --config /data/gitea/conf/app.ini)
[[ -n "$CONTAINER_ID" ]] || die "isolated Gitea container did not start"
booted=0
for _ in $(seq 1 30); do
if "$PODMAN_BIN" exec "$CONTAINER_ID" wget -q -O /dev/null \
http://127.0.0.1:3000/api/v1/version >/dev/null 2>&1; then
booted=1
break
fi
running=$("$PODMAN_BIN" inspect --format '{{.State.Running}}' "$CONTAINER_ID" 2>/dev/null) \
|| die "isolated Gitea container exited before becoming healthy"
[[ "$running" == true ]] \
|| die "isolated Gitea container exited before becoming healthy"
sleep 1
done
((booted)) || die "isolated Gitea did not become healthy"
"$PODMAN_BIN" rm -f "$CONTAINER_ID" >/dev/null
CONTAINER_ID=""
printf 'Disposable restore verified: checksum, archive, SQLite, %d Git repositories, isolated Gitea boot\n' "$repo_count"
+593
View File
@@ -0,0 +1,593 @@
#!/usr/bin/env python3
"""Check or reconcile tracked Gitea repository state.
Check mode is the default and performs only GET requests. Apply mode creates or
updates managed objects, leaves unmanaged objects alone, and verifies the final
state with a fresh read.
"""
from __future__ import annotations
import argparse
import base64
import http.client
import ipaddress
import json
import sys
import urllib.error
import urllib.parse
import urllib.request
from dataclasses import dataclass
from datetime import datetime, timezone
from pathlib import Path
from typing import Any, Callable
DEFAULT_GITEA_URL = "https://gitea.thingscouldgetdicey.online"
DEFAULT_MANIFEST = (
Path(__file__).resolve().parents[1] / "config/gitea_repository_state.json"
)
MANAGED_BRANCH_FIELDS = (
"enable_push",
"enable_push_whitelist",
"push_whitelist_usernames",
"enable_force_push",
"enable_merge_whitelist",
"merge_whitelist_usernames",
"merge_whitelist_teams",
"enable_bypass_allowlist",
"bypass_allowlist_usernames",
"bypass_allowlist_teams",
"required_approvals",
"enable_approvals_whitelist",
"approvals_whitelist_username",
"approvals_whitelist_teams",
"enable_status_check",
"status_check_contexts",
"block_on_rejected_reviews",
"block_on_official_review_requests",
"block_on_outdated_branch",
"dismiss_stale_approvals",
"ignore_stale_approvals",
"require_signed_commits",
"protected_file_patterns",
"unprotected_file_patterns",
"block_admin_merge_override",
)
class ManifestError(ValueError):
"""Raised when repository state cannot be safely interpreted."""
class ProvisioningError(RuntimeError):
"""Raised when apply does not converge."""
@dataclass(frozen=True)
class Drift:
kind: str
name: str
status: str
def __str__(self) -> str:
return f"{self.kind} {self.name}: {self.status}"
def _origin(url: str) -> tuple[str, str | None, int | None]:
parsed = urllib.parse.urlsplit(url)
default_port = {"http": 80, "https": 443}.get(parsed.scheme.lower())
return parsed.scheme.lower(), parsed.hostname, parsed.port or default_port
class _SameOriginRedirectHandler(urllib.request.HTTPRedirectHandler):
def http_error_302(self, req, fp, code, msg, headers):
if "location" in headers:
location = headers["location"]
elif "uri" in headers:
location = headers["uri"]
else:
return super().http_error_302(req, fp, code, msg, headers)
try:
_origin(location)
except ValueError:
fp.close()
raise ProvisioningError("Gitea API refused malformed redirect") from None
return super().http_error_302(req, fp, code, msg, headers)
def redirect_request(self, req, fp, code, msg, headers, newurl):
try:
source_origin = _origin(req.full_url)
redirect_origin = _origin(newurl)
except ValueError:
fp.close()
raise ProvisioningError("Gitea API refused malformed redirect") from None
if source_origin != redirect_origin:
fp.close()
raise ProvisioningError("Gitea API refused cross-origin redirect")
return super().redirect_request(req, fp, code, msg, headers, newurl)
def normalize_color(value: str) -> str:
color = value.strip().removeprefix("#").lower()
if len(color) != 6 or any(
character not in "0123456789abcdef" for character in color
):
raise ManifestError(f"invalid label color: {value!r}")
return color
def normalize_instant(value: str | None) -> str | None:
if value in (None, ""):
return None
try:
instant = datetime.fromisoformat(value.replace("Z", "+00:00"))
except (TypeError, ValueError) as error:
raise ManifestError(f"invalid RFC3339 instant: {value!r}") from error
if instant.tzinfo is None:
raise ManifestError(f"RFC3339 instant must include a timezone: {value!r}")
utc = instant.astimezone(timezone.utc)
if utc.microsecond:
rendered = utc.isoformat(timespec="microseconds").rstrip("0").rstrip(".")
else:
rendered = utc.isoformat(timespec="seconds")
return rendered.replace("+00:00", "Z")
def _require_text(container: dict[str, Any], key: str, context: str) -> str:
value = container.get(key)
if not isinstance(value, str) or not value.strip():
raise ManifestError(f"{context}.{key} must be a non-empty string")
return value
def _unique(items: list[dict[str, Any]], key: str, context: str) -> None:
values = [_require_text(item, key, context) for item in items]
if len(values) != len(set(values)):
raise ManifestError(f"duplicate {context} {key}")
def load_manifest(
path: Path | str, *, repository_root: Path | None = None
) -> dict[str, Any]:
manifest_path = Path(path).resolve()
root = (repository_root or manifest_path.parent.parent).resolve()
try:
manifest = json.loads(manifest_path.read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError) as error:
raise ManifestError(f"cannot read manifest {manifest_path}: {error}") from error
if not isinstance(manifest, dict):
raise ManifestError("manifest root must be an object")
repository = manifest.get("repository")
if not isinstance(repository, dict):
raise ManifestError("repository must be an object")
for key in ("owner", "name", "default_branch"):
_require_text(repository, key, "repository")
protection = manifest.get("branch_protection")
if not isinstance(protection, dict):
raise ManifestError("branch_protection must be an object")
branch_name = _require_text(protection, "branch_name", "branch_protection")
if branch_name != repository["default_branch"]:
raise ManifestError("branch protection must manage the default branch")
for field in MANAGED_BRANCH_FIELDS:
if field not in protection:
raise ManifestError(f"branch_protection.{field} is required")
for collection, identity in (
("labels", "name"),
("milestones", "title"),
("wiki_pages", "slug"),
):
items = manifest.get(collection)
if (
not isinstance(items, list)
or not items
or not all(isinstance(item, dict) for item in items)
):
raise ManifestError(f"{collection} must be a non-empty array of objects")
_unique(items, identity, collection)
expected_counts = {"labels": 11, "milestones": 3, "wiki_pages": 6}
for collection, expected in expected_counts.items():
if len(manifest[collection]) != expected:
raise ManifestError(
f"manifest must contain exactly {expected} {collection}"
)
for label in manifest["labels"]:
label["color"] = normalize_color(_require_text(label, "color", "labels"))
if not isinstance(label.get("description", ""), str):
raise ManifestError("labels.description must be a string")
for milestone in manifest["milestones"]:
if milestone.get("state") not in {"open", "closed"}:
raise ManifestError("milestones.state must be open or closed")
milestone["due_on"] = normalize_instant(milestone.get("due_on"))
if not isinstance(milestone.get("description", ""), str):
raise ManifestError("milestones.description must be a string")
_unique(manifest["wiki_pages"], "title", "wiki_pages")
for page in manifest["wiki_pages"]:
source = Path(_require_text(page, "source", "wiki_pages"))
source_path = (root / source).resolve()
if source.is_absolute() or not source_path.is_relative_to(root):
raise ManifestError(f"wiki source escapes repository: {source}")
if not source_path.is_file():
raise ManifestError(f"wiki source does not exist: {source}")
page["source"] = source.as_posix()
return manifest
class UrllibTransport:
"""Small JSON transport whose credential remains only in request headers."""
def __init__(self, base_url: str, token: str, timeout: float = 15.0):
self.base_url = base_url.rstrip("/")
try:
parsed = urllib.parse.urlsplit(self.base_url)
hostname, _ = parsed.hostname, parsed.port
except ValueError as error:
raise ProvisioningError("Gitea API base URL is invalid") from error
loopback = hostname == "localhost" or hostname == "::1"
if hostname and not loopback:
try:
address = ipaddress.ip_address(hostname)
except ValueError:
pass
else:
loopback = address.version == 4 and address.is_loopback
if parsed.scheme.lower() != "https" and not (
parsed.scheme.lower() == "http" and loopback
):
raise ProvisioningError(
"Gitea API base URL must use HTTPS (HTTP is limited to loopback hosts)"
)
self._token = token
self.timeout = timeout
def request(self, method: str, path: str, *, query=None, payload=None):
url = f"{self.base_url}{path}"
if query:
url = f"{url}?{urllib.parse.urlencode(query)}"
body = None if payload is None else json.dumps(payload).encode("utf-8")
headers = {
"Accept": "application/json",
"Authorization": f"token {self._token}",
}
if body is not None:
headers["Content-Type"] = "application/json"
request = urllib.request.Request(url, data=body, headers=headers, method=method)
try:
opener = urllib.request.build_opener(_SameOriginRedirectHandler())
try:
opened_response = opener.open(request, timeout=self.timeout)
except ValueError:
raise ProvisioningError(
"Gitea API refused malformed redirect"
) from None
with opened_response as response:
raw = response.read()
except urllib.error.HTTPError as error:
error.close()
raise ProvisioningError(
f"Gitea API {method} {path} failed ({error.code})"
) from error
except urllib.error.URLError as error:
raise ProvisioningError(
f"Gitea API {method} {path} failed: {error.reason}"
) from error
except (http.client.HTTPException, TimeoutError, OSError) as error:
raise ProvisioningError(
f"Gitea API {method} {path} failed while reading response"
) from error
if not raw:
return None
try:
return json.loads(raw)
except (json.JSONDecodeError, UnicodeDecodeError) as error:
raise ProvisioningError(
f"Gitea API {method} {path} returned an invalid JSON response"
) from error
class Provisioner:
BRANCH_FIELDS = MANAGED_BRANCH_FIELDS
def __init__(self, manifest, transport, *, repository_root=None, page_size=50):
self.manifest = manifest
self.transport = transport
self.root = Path(
repository_root or Path(__file__).resolve().parents[1]
).resolve()
self.page_size = page_size
repository = manifest["repository"]
owner = urllib.parse.quote(repository["owner"], safe="")
name = urllib.parse.quote(repository["name"], safe="")
self.repo_path = f"/api/v1/repos/{owner}/{name}"
def _request(self, method, suffix, *, query=None, payload=None):
return self.transport.request(
method, f"{self.repo_path}{suffix}", query=query, payload=payload
)
def _list(self, suffix, *, query=None):
results = []
page = 1
while True:
page_query = {**(query or {}), "page": page, "limit": self.page_size}
batch = self._request("GET", suffix, query=page_query)
if not isinstance(batch, list):
raise ProvisioningError(f"Gitea returned a non-list for {suffix}")
results.extend(batch)
if len(batch) < self.page_size:
return results
page += 1
def _read_state(self):
protections = self._request("GET", "/branch_protections")
if not isinstance(protections, list):
raise ProvisioningError("Gitea returned a non-list for /branch_protections")
branch_name = self.manifest["branch_protection"]["branch_name"]
branch = next(
(
item
for item in protections
if item.get("rule_name", item.get("branch_name")) == branch_name
),
None,
)
labels = self._list("/labels")
milestones = self._list("/milestones", query={"state": "all"})
page_summaries = self._list("/wiki/pages")
existing_slugs = {
item.get("slug") or item.get("sub_url") or item.get("title")
for item in page_summaries
}
wiki = {}
for desired in self.manifest["wiki_pages"]:
slug = desired["slug"]
if slug in existing_slugs:
encoded = urllib.parse.quote(slug, safe="")
wiki[slug] = self._request("GET", f"/wiki/page/{encoded}")
return {
"branch": branch,
"labels": labels,
"milestones": milestones,
"wiki": wiki,
}
def _wiki_content(self, page):
return (self.root / page["source"]).read_text(encoding="utf-8")
@staticmethod
def _actual_wiki_content(page):
if page is None:
return None
if "content" in page:
return page["content"]
encoded = page.get("content_base64")
if encoded is None:
return None
try:
return base64.b64decode(encoded, validate=True).decode("utf-8")
except (ValueError, UnicodeDecodeError) as error:
raise ProvisioningError(
"Gitea returned invalid wiki content_base64"
) from error
def _branch_matches(self, actual):
desired = self.manifest["branch_protection"]
if actual is None:
return False
return all(
actual.get(field) == desired.get(field) for field in self.BRANCH_FIELDS
)
@staticmethod
def _label_matches(actual, desired):
desired_color = normalize_color(desired["color"])
try:
actual_color = normalize_color(actual.get("color", ""))
except ManifestError:
return False
return actual_color == desired_color and actual.get(
"description", ""
) == desired.get("description", "")
@staticmethod
def _milestone_matches(actual, desired):
desired_due_on = normalize_instant(desired.get("due_on"))
try:
actual_due_on = normalize_instant(actual.get("due_on"))
except ManifestError:
return False
return (
actual.get("description", "") == desired.get("description", "")
and actual.get("state", "") == desired.get("state", "")
and actual_due_on == desired_due_on
)
def _drift(self, state):
drift = []
branch_name = self.manifest["branch_protection"]["branch_name"]
if not self._branch_matches(state["branch"]):
drift.append(
Drift(
"branch protection",
branch_name,
"missing" if state["branch"] is None else "drifted",
)
)
labels = {item["name"]: item for item in state["labels"]}
for desired in self.manifest["labels"]:
actual = labels.get(desired["name"])
if actual is None or not self._label_matches(actual, desired):
drift.append(
Drift(
"label",
desired["name"],
"missing" if actual is None else "drifted",
)
)
milestones = {item["title"]: item for item in state["milestones"]}
for desired in self.manifest["milestones"]:
actual = milestones.get(desired["title"])
if actual is None or not self._milestone_matches(actual, desired):
drift.append(
Drift(
"milestone",
desired["title"],
"missing" if actual is None else "drifted",
)
)
for desired in self.manifest["wiki_pages"]:
actual = state["wiki"].get(desired["slug"])
content = self._actual_wiki_content(actual)
matches = (
actual is not None
and actual.get("title") == desired["title"]
and content == self._wiki_content(desired)
)
if not matches:
drift.append(
Drift(
"wiki page",
desired["slug"],
"missing" if actual is None else "drifted",
)
)
return drift
def check(self):
return self._drift(self._read_state())
def apply(self):
state = self._read_state()
initial_drift = self._drift(state)
if not initial_drift:
return []
desired_branch = deepcopy_dict(self.manifest["branch_protection"])
branch_name = desired_branch["branch_name"]
if not self._branch_matches(state["branch"]):
if state["branch"] is None:
self._request("POST", "/branch_protections", payload=desired_branch)
else:
desired_branch.pop("branch_name")
encoded = urllib.parse.quote(branch_name, safe="")
self._request(
"PATCH", f"/branch_protections/{encoded}", payload=desired_branch
)
labels = {item["name"]: item for item in state["labels"]}
for desired in self.manifest["labels"]:
payload = {
"name": desired["name"],
"color": normalize_color(desired["color"]),
"description": desired.get("description", ""),
}
actual = labels.get(desired["name"])
if actual is None:
self._request("POST", "/labels", payload=payload)
elif not self._label_matches(actual, desired):
self._request("PATCH", f"/labels/{actual['id']}", payload=payload)
milestones = {item["title"]: item for item in state["milestones"]}
for desired in self.manifest["milestones"]:
payload = {
key: desired.get(key)
for key in ("title", "description", "state", "due_on")
}
actual = milestones.get(desired["title"])
if actual is None:
self._request("POST", "/milestones", payload=payload)
elif not self._milestone_matches(actual, desired):
self._request("PATCH", f"/milestones/{actual['id']}", payload=payload)
for desired in self.manifest["wiki_pages"]:
content = self._wiki_content(desired)
payload = {
"title": desired["title"],
"content_base64": base64.b64encode(content.encode("utf-8")).decode(
"ascii"
),
}
actual = state["wiki"].get(desired["slug"])
actual_content = self._actual_wiki_content(actual)
if actual is None:
self._request("POST", "/wiki/new", payload=payload)
elif actual.get("title") != desired["title"] or actual_content != content:
encoded = urllib.parse.quote(desired["slug"], safe="")
self._request("PATCH", f"/wiki/page/{encoded}", payload=payload)
remaining = self.check()
if remaining:
summary = ", ".join(str(item) for item in remaining)
raise ProvisioningError(f"repository state did not converge: {summary}")
return []
def deepcopy_dict(value):
return {
key: list(item) if isinstance(item, list) else item
for key, item in value.items()
}
def _default_token_loader():
try:
from shared.gitea_auth import load_gitea_token
except ModuleNotFoundError: # Direct execution: python shared/gitea_provision.py
from gitea_auth import load_gitea_token
return load_gitea_token()
def run(
argv=None,
*,
token_loader: Callable[[], str] | None = None,
transport_factory=UrllibTransport,
):
parser = argparse.ArgumentParser(description=__doc__)
mode = parser.add_mutually_exclusive_group()
mode.add_argument(
"--check",
action="store_true",
help="report drift using GET requests only (default)",
)
mode.add_argument(
"--apply", action="store_true", help="create or update managed repository state"
)
parser.add_argument("--manifest", type=Path, default=DEFAULT_MANIFEST)
parser.add_argument("--gitea-url", default=DEFAULT_GITEA_URL)
args = parser.parse_args(argv)
manifest_path = args.manifest.resolve()
repository_root = manifest_path.parent.parent
manifest = load_manifest(manifest_path, repository_root=repository_root)
token = (token_loader or _default_token_loader)()
transport = transport_factory(args.gitea_url, token)
transport.request("GET", "/api/v1/version")
provisioner = Provisioner(manifest, transport, repository_root=repository_root)
if args.apply:
provisioner.apply()
print("Gitea repository state converged.")
return 0
drift = provisioner.check()
if drift:
print(f"Gitea repository state drift: {len(drift)} managed object(s).")
for item in drift:
print(f"- {item}")
return 1
print("Gitea repository state is converged.")
return 0
def main():
try:
return run()
except (ManifestError, ProvisioningError, RuntimeError, PermissionError) as error:
print(f"error: {error}", file=sys.stderr)
return 2
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,115 @@
# Planning and Task Orchestration Benchmark Protocol
**Status:** Proposed protocol; thresholds remain provisional until a measured
baseline is approved. **Implementation permission:** none.
## Environment fingerprint
Every run records commit, dirty-state hash, OS/kernel, CPU model/count, RAM,
GPU/driver, storage device/filesystem, Python/Rust/runtime versions, provider
endpoint/model/digest, power profile, background-load profile, and UTC time.
Results from different fingerprints are not merged silently.
## Fixture manifest
A versioned manifest pins event schemas, agent/config/resource snapshots,
deterministic provider responses, tool outputs, evidence blobs, task graph,
surface capabilities, random seeds, and expected typed invariants. Sensitive
values use deterministic redacted fixtures. The manifest hash appears in every
result and ReceiptRecord.
## Run discipline
- Each scenario runs five paired batches. A batch contains 10 excluded warm-up
operations followed by exactly 100 measured operations; replay/crash batches
contain one warm-up and exactly 10 measured 100,000-envelope runs.
- Interactive/background isolation runs at idle, 50%, and saturated background
quotas with fixed concurrency.
- Model-output equality uses the deterministic provider. Real-model runs measure
resource use and typed invariants, never prose identity.
- Report p50, p95, maximum, median absolute deviation, failures, retries, CPU,
peak RSS, disk bytes, network bytes, model calls, and tokens.
- Compare paired baseline/candidate batches on the same fingerprint and
manifest. The Metric Contracts table is the sole authority for each metric's
per-batch statistic, unit, direction, threshold, and regression rule. Its
named statistic replaces p95 where the row specifies median, maximum, count,
or exact proportion. Raw failures are never discarded; any typed-invariant or
capability loss is an immediate failure. If fewer than five valid paired
batches remain, the run is invalid, not inconclusive-by-convenience.
## Workloads
1. Single direct-user turn with no tools.
2. Tool-heavy turn with 20 evidence blobs and one retryable failure.
3. 100 concurrent partitions with strict per-partition ordering.
4. 100,000-envelope append, projection, checkpoint, restart, and replay.
5. N+1 no-change, changed-evidence, deadline, and failure paths.
6. Reflection and synthesis with capped and uncapped evidence manifests.
7. Resource routing under stable health, version change, outage, and fallback.
8. Mixed direct-user, cancellation, verification, telemetry, and archival load.
9. Local/server/remote/TUI semantic-parity suite.
## Metric contracts
All relative comparisons use five paired batches. For continuous duration and
byte metrics, the comparison value is the median of the five paired differences
for the named statistic and the noise bound is
`max(measurement_resolution * 10, 3 * MAD)`. For integer counts, the bound is
`max(1, 3 * MAD)`. Deterministic invariant proportions use exact fixture
counts and admit no statistical waiver.
| Metric | Workload | Statistic and unit | Direction | Absolute contract | Regression rule |
|---|---|---|---|---|---|
| Reflex latency | 1-3 | per-operation p95 milliseconds | lower | <= 10 ms excluding external I/O | fail if above threshold or paired regression exceeds duration noise and 5% |
| Durable append-to-publish latency | 2-4 | per-envelope p95 milliseconds | lower | <= 25 ms locally | same duration rule |
| Projection read latency | 3-4 | per-read p95 milliseconds | lower | <= 20 ms | same duration rule |
| Interactive start latency | 8 | request-to-lease p95 milliseconds | lower | <= 100 ms at every background load | same duration rule; saturated load cannot be averaged with idle |
| Replay recovery | 4 | p95 seconds across 50 measured runs | lower | < 2 s for 100,000 envelopes | fail if above threshold or paired regression exceeds duration noise and 5% |
| Queue depth | 3,8 | p95 and maximum queued items per priority lane | lower | no reserved lane reaches capacity; configured lane maximum is recorded in baseline | fail on overflow or integer-count regression beyond noise and 5% |
| Model-call avoidance | 5 | exact avoided/eligible no-change fixtures, percent | higher | >= 80% with zero missed changed-evidence fixture | any lower proportion fails; no noise waiver |
| Model calls | 2,5-7 | median calls per completed fixture | lower | all mandatory cognitive outputs still present | integer-count regression fails unless approved trade record applies |
| Input/output tokens | 2,5-7 | median tokens per completed fixture | lower | fixture completion and evidence manifest unchanged | paired regression beyond integer noise and 5% fails |
| Disk I/O | 2-4,7 | median bytes written/read per fixture | lower | every accepted record reaches eventual provenance | paired regression beyond byte noise and 5% fails |
| Network I/O | 3,7,9 | median bytes per completed fixture | lower | typed event/receipt parity unchanged | paired regression beyond byte noise and 5% fails |
| CPU time | 1-8 | median process CPU milliseconds per fixture | lower | none provisional; baseline record sets owner-approved ceiling | paired regression beyond duration noise and 5% fails |
| Peak RSS | 3-5,8 | maximum resident MiB per batch | lower | none provisional; baseline record sets owner-approved ceiling | paired regression beyond byte noise and 5% fails |
| Priority isolation | 8 | p95 start latency and exact urgent-arrival fixtures yielding background capacity | lower latency; 100% required yields | direct-user <= 100 ms; every urgent arrival under saturated background load causes at least one bounded background yield; zero dropped cancellation/halt events | any missed yield or dropped event fails; duration rule for latency |
| Semantic parity | 9 | exact passed capability invariants / total | equal | 100% on every production entry path or valid approved exception | any unexplained mismatch fails; no noise waiver |
| Duplicate suppression | 4-5 | exact duplicate effects committed | lower | zero duplicate canonical effects; duplicate delivery remains replayable evidence | any duplicate effect fails; no noise waiver |
Metrics not declared as optimization targets remain guardrails: they may improve
or stay within noise, but may not regress beyond their rule merely because one
headline latency moved in the flattering direction. Machines are terribly good
at winning the metric they were handed and pawning the furniture to do it.
The implementation plan must name one or more optimization targets before a run.
Eligible targets are reflex latency, append-to-publish latency, projection read
latency, interactive start latency, replay recovery, model calls, tokens, disk
I/O, network I/O, CPU time, or peak RSS. Queue depth, priority isolation,
semantic parity, duplicate suppression, and all typed capabilities are always
guardrails, never targets that may be traded away.
## Baselines and acceptance
The first approved implementation phase writes two distinct records. The
**operational baseline** measures pre-change Rune for comparable costs and
latencies only. The **semantic reference baseline** is recorded after the
smallest unoptimized implementation satisfies every approved schema, authority,
receipt, entry-path, and capability contract. Only the semantic reference owns
golden fixtures for optimization comparisons. Each record contains its type,
manifest, fingerprint, raw-result hashes, derived statistics, noise bounds,
thresholds, and Nick's approval. Threshold changes require a new record linked
to the old one, reasons, raw evidence, capability-parity result, and explicit
approval; CI configuration cannot quietly move the goalposts.
An optimization is accepted only when:
1. every typed invariant and required capability remains present;
2. no production entry path degrades outside an approved exception;
3. every approved absolute threshold is met;
4. at least one declared target metric improves beyond measured noise;
5. no other contracted metric regresses beyond its noise bound unless Nick
approves a named trade record that states the reliability gain, affected
workload, expiry, restoration test, and a hard cap of 5% regression;
6. raw logs and reproduction commands are retained;
7. rollback restores the prior baseline behavior.
@@ -0,0 +1,694 @@
# Planning and Task Orchestration Design
**Status:** Approved by Nick
**Updated:** 2026-09-03
**Authority:** Running design record; revised after each settled decision
**Implementation state:** Not started; implementation plan pending approval
## [S1] Purpose
Design a planning and execution system that preserves intent, challenges weak
assumptions, records evidence, delegates work in accuracy-sized batches, and
keeps every active stage visible until verified. Planning must not drift into
execution: Nick retains the explicit right to say **start**.
## [S2] Selected architecture
Use a layered nervous system, subconscious, and deterministic scheduler:
- deterministic evidence records what is available, authorized, started,
verified, failed, or blocked;
- the subconscious interprets meaningful changes, contradictions, and task fit;
- the scheduler advances approved work within explicit boundaries;
- foundation models are replaceable reasoning resources, not the source of
identity, task truth, or completion truth.
Rejected alternatives:
1. A single orchestration model is simple but can hallucinate availability,
completion, or resource state and wastes inference on mechanical checks.
2. A scheduler alone is reliable but too rigid for changing intent,
contradictory research, and nuanced capability matching.
## [S3] Planning and task experience
**Section status:** Approved by Nick; safeguards and TUI pin visibility recorded.
### Entering planning
- Nick can enter Planning Mode directly.
- Rune may suggest it based on ambiguity, risk, scope, research need, cost,
visual design, or irreversibility.
- Entry always requires consent.
- Planning lives in a named, switchable Chat thread; ordinary Chat remains
available.
### Planning
- Rune interviews intent before proposing implementation.
- The answers create a research brief.
- Research, sources, notes, decisions, and the living report may be recorded
automatically.
- Isolated experiments require a falsifiable hypothesis and predefined
verification.
- Research continues until evidence converges or uncertainty is honestly
bounded.
- Rune proposes planning completion; Nick confirms or reopens it.
### Task-pane report
- Every planning thread owns a living, versioned report.
- It contains the current understanding, rationale, alternatives, unresolved
questions, evidence, stages, and execution strategy.
- Chat carries discussion; the report carries settled decisions.
- The report stays compact, with detail available on demand.
### Pinning
- The root plan remains pinned until the goal is complete.
- The active stage and batch remain pinned until verified.
- Once verified, the completed unit is unpinned and its successor is pinned.
- The latest verified checkpoint remains pinned until replaced or the plan
reaches a terminal state.
- Pinning is a state invariant, not a display convention: at most one stage and
one batch may hold the operational pin for a plan.
- Pin transfer is transactional and evidence-backed. A crash must leave either
the prior pin or its successor valid, never an untracked gap or two active
owners.
- A stale-pin watchdog reports inactivity, lost ownership, or expired evidence
but cannot infer completion or move the pin itself.
- Initial stage/batch activation is a distinct transaction backed by a valid
start ApprovalRecord; it does not require a fictional predecessor receipt.
- Completion, cancellation, or supersession transactionally clears root,
stage, batch, and checkpoint pins after recording the terminal receipt.
- Older detail moves into durable Gitea history; live model context keeps only
compact state and references.
### Pinned-item visibility in the TUI
- The persistent context header shows a compact text indicator such as
**Pinned: 4**. It remains visible across Chat, Planning, and Task views.
- Hovering or focusing the indicator opens a concise tooltip listing the pinned
root plan, active stage, active batch, and latest verified checkpoint.
- Each tooltip row shows item type, short title, state, and age; blocked or
stale items use an accessible text label as well as color.
- The indicator is keyboard-focusable. Enter opens the full pinned-items view,
so essential state is never trapped behind hover behavior.
- The count is derived from canonical pin records rather than maintained as a
separate UI counter.
### Gitea delegation
- Plan confirmation settles a plan version but creates no external task. Only
Nick's explicit **start** approval for that exact version authorizes the
Delegator to create a Ready parent issue, stage issues, and accuracy-sized
batch issues.
- Issues carry dependencies, acceptance criteria, canaries, capability
assignment, resource posture, and evidence requirements.
- Every batch targets one independently verifiable outcome and has a bounded
context budget. File count and line count alone do not define batch size.
- Code batches use isolated branches, pull requests, and protected checks.
- Authority is field-specific rather than duplicated: the Intent and Plan Store
owns intent, design, plan versions, and approvals; Gitea owns accepted remote
issue/PR/milestone coordination state and the merged branch; the Verification
Ledger owns observed execution and completion. The TUI projects all three
with provenance.
- When Gitea is unavailable, Rune writes intended mutations to a local
append-only outbox with stable IDs. They remain pending intents, not accepted
Gitea state. Reconnection applies version preconditions and deterministic
conflict rules; it never silently overwrites newer remote state.
### Starting and revising
- Nothing executes until Nick says **start** from the report or its Planning
thread.
- Goal changes trigger impact review. Valid work survives; affected work
pauses; revisions require approval.
- Internal phases advance only after verification.
- Objective work closes on proof; subjective outcomes wait for Nick's
judgment.
### Advisory viability
- The report ends with a **Viability Card** covering intent clarity, evidence
maturity, technical feasibility, resources, dependencies, verification,
reversibility, and model-task fit.
- It shows a summary score, each dimension, contrary evidence, and fatal
weaknesses; an average cannot conceal a zero.
- Every dimension also exposes confidence and evidence age so a precise-looking
score cannot disguise weak or stale inputs.
- The score updates as evidence and conditions change, then calibrates against
actual outcomes.
- It is always advisory. It informs whether a plan is worth pursuing but never
overrules Nick or becomes a hidden permission gate.
## [S4] Settled decisions
1. Planning is critical inquiry, not task-list formatting.
2. Planning and ordinary Chat remain switchable.
3. Planning may record artifacts automatically but may not begin execution.
4. The root plan, active stage, active batch, and latest checkpoint have
explicit rolling pin semantics.
5. The Delegator converts an approved plan into a Gitea issue hierarchy built
from small, verifiable batches.
6. Gitea carries durable tracking; the TUI presents the live operational view.
7. Viability appears at the end of the report and is always advisory.
8. Pin ownership and transfer are enforced invariants with stale-pin detection.
9. The persistent TUI context header exposes the pin count, a focusable tooltip,
and keyboard access to full pin details.
10. A local append-only outbox preserves Gitea-bound state during outages while
Gitea remains the durable source of truth.
11. Batch size follows one verifiable outcome plus a bounded context budget.
12. One durable event/effect spine, not a model or UI path, carries execution
and completion authority.
13. Reflex, N+1, reflection, and synthesis are independent cognitive timescales
with separate triggers, budgets, cursors, and health.
14. Resource eligibility is deterministic; model routing is advisory; final
assignment and fallback are recorded scheduler decisions.
15. Efficiency comes from references, projections, partitioned parallelism,
adaptive no-change suppression, and immutable snapshots—not functionality
removal.
16. Performance changes must pass semantic-parity and capability-preservation
canaries as well as latency and resource budgets.
17. Canonical authority is field-specific: internal plan records own intent and
approval, Gitea owns accepted coordination objects and merged code, and the
Verification Ledger owns observed completion.
18. Approved execution advances by event-driven batch chaining: a verified
receipt deterministically selects the next unit and hands it a compact
context manifest, so no model must re-process the plan.
19. Chaining never crosses the consent boundary, never skips a receipt, and
resumes from the last verified receipt after any interruption.
## [S5] Open design work
1. Implementation plan generation (in progress after design approval).
## [S6] Decision trace
- Nick required every stage and batch to remain pinned until completion, with
the pin advancing only after verification.
- Nick required a final-plan Delegator that creates Gitea issues in small
batches to protect code accuracy.
- Nick required Gitea to carry as much durable task and goal tracking as
practical.
- Nick required viability to appear at the report's end, inform whether work is
worth pursuing, and remain advisory.
- Nick required this design to be recorded continuously in a running file.
- Rune recommended transactional pin invariants, offline Gitea reconciliation,
and confidence-aware viability and batch sizing; Nick approved all three.
- Nick required a compact TUI header indicator showing the number of pinned
items, with a tooltip identifying them.
- The pin indicator is placed in the persistent context header rather than a
transient status message because pin state must remain continuously visible.
- Repository inspection confirmed that subconscious-related modules exist but
no subconscious pass is wired into the live TUI/operator response path.
- Nick required the subconscious design to absorb the development ideas behind
Souveraine and FIMEG in their strongest primary-source form, then advance
them where evidence supports doing so rather than merely copying them.
- Nick warned that apparently secondary, costly, risky, or conflicting
mechanisms may be critical to capability. Research must therefore separate
faithful reconstruction from later evaluation and may not prune capability
by assumption.
- Nick approved the optimized subconscious/resource architecture and then the
component, data-flow, failure, verification, and provisional performance
design.
- The primary-source survey examined 47 records and mapped 41 claims. It did
not reach claim saturation, so remaining uncertainty is carried as seven
explicit pre-implementation ablations.
- Nick approved the resulting one-spine subconscious/resource architecture and
then asked for greater efficiency and performance without losing intent or
functionality.
- Rune added batched durable append, partitioned ordering, materialized views,
content-addressed evidence, non-erasing coalescing, adaptive cognition,
two-stage routing, immutable snapshots, priority lanes, and measured
capability-preservation contracts; Nick approved those refinements.
- Nick approved the complete design and required explicit batch-chaining and
context-minimization contracts so the LLM never holds or re-processes the
full plan during execution.
- The current phase remains planning and research only; implementation is not
authorized.
## [S7] Subconscious research gate
**Status:** Primary-source survey completed; synthesis incorporated below.
The subconscious architecture cannot be finalized from metaphor, remembered
summaries, or dormant Rune modules. Research must first:
1. identify and verify the authoritative Souveraine and FIMEG sources;
2. reconstruct their development history, goals, mechanisms, and explicit
constraints from primary material;
3. separate implemented behavior from aspiration and terminology;
4. map transferable mechanisms onto Rune's actual live runtime;
5. identify contradictions, missing evidence, and opportunities to advance the
architecture without creating another parallel authority path.
Research must preserve before it judges. It may not discard a mechanism merely
because it conflicts with Rune's current architecture, appears expensive or
risky, or seems unnecessary in isolation. Each mechanism must be reconstructed
in its original system context, including dependencies and interactions that
may be critical to capability. Evaluation comes afterward through an
interaction map, explicit trade-offs, and—where evidence remains insufficient—a
proposed comparative experiment or ablation. Any recommendation to omit or
constrain a source mechanism must state what capability may be lost and the
evidence supporting that trade.
Planned artifacts are a structured research question, auditable source and
claim logs, a synthesis report, and a source-linked revision of this design.
Artifacts:
- `survey/souveraine/question.md`
- `survey/souveraine/sources.tsv`
- `survey/souveraine/claims.tsv`
- `survey/souveraine/report.md`
## [S8] Technical subconscious and resource architecture
**Section status:** Approved by Nick after capability-preserving efficiency and
performance refinements. Research did not reach claim saturation, so
uncertainties remain explicit ablations rather than silently resolved.
### Research conclusion
Souveraine's capability comes from combining durable identity, distinct
cognitive timescales, embodied tools, event-driven surfaces, and one
constitutive substrate. Its principal failure mode is authority divergence:
local, server, remote, subagent, schedule, provider, and UI paths can describe
the same system while executing materially different capabilities. Rune must
preserve the mechanisms while replacing parallel composition roots and
model-authored truth with one evidence-backed execution spine. Full evidence
and contradiction mapping lives in `survey/souveraine/report.md`.
### Canonical event and effect spine
- Every meaningful input becomes an immutable typed envelope with a stable ID,
causal parent, actor/device identity, plan/task/pin references, authorization
context, evidence timestamp, and schema version.
- Envelopes are committed to a durable append-only store before fan-out. The
in-memory event bus is a low-latency projection, never durable completion
truth.
- Consumers own replayable offsets and idempotency keys. Lag, restart, or an
unavailable surface cannot silently discard accepted work.
- Effects become true only after the authority layer commits them and records a
verification receipt. Model prose, displayed status, and emitted events are
not execution evidence.
### Three execution strata
1. **Reflex layer:** deterministic, cheap, and idempotent checks produce facts
about availability, process state, budget, deadlines, stale pins, and failed
verification. Reflexes cannot reinterpret intent or mutate goals.
2. **Subconscious layer:** bounded reasoning passes consume evidence capsules
and emit proposals such as surface, persist, investigate, reroute,
reschedule, or halt. Every proposal carries sources, confidence, expiry,
estimated cost, and possible capability loss. It cannot directly mutate
canonical task, memory, resource, or completion state.
3. **Scheduler/effect layer:** deterministic policy validates authority,
dependency state, budgets, current versions, and idempotency, then commits
accepted effects and their receipts transactionally.
N+1 review, wider reflection, and long-range synthesis are independent
consumers with separate triggers, budgets, cursors, and health. They remain
parts of one identity, but no single enable flag owns the others' lifecycle.
### Subconscious state and surfacing
- Preserve distinct ledger, inbox, inner-voice, and conversation-history roles,
but place them under one versioned schema with stable record IDs and explicit
ownership.
- The subconscious may queue an ordinary observation, an urgent interjection,
or a halt proposal. Urgency changes scheduler treatment, not authority.
- Surfacing records remain pending until acknowledged by a specific primary
turn or explicitly dismissed; reading a queue is not equivalent to delivery.
- Every persistence or surfacing action links back to the evidence capsule and
originating event IDs.
### Resource authority
- One canonical registry records providers, models, tools, workers, surfaces,
and stores with observed health, capabilities, cost, locality, authorization,
concurrency, and evidence age.
- A deterministic eligibility filter removes impossible candidates. An
advisory router ranks eligible resources. The scheduler records the chosen
assignment, rationale, and bounded fallback chain.
- Subagents inherit the same authority object. They cannot bypass provider,
locality, budget, depth, timeout, or concurrency policy.
- Unknown or stale resource state produces structured unavailability or lower
confidence, never silent fallback presented as the requested resource.
### Memory and synthesis
- Episodic records have stable IDs and explicit synthesis states.
- A synthesis transaction records exactly which input IDs were included, the
output ID, previous cursor, and commit receipt before advancing the cursor.
- Syntheses form a cumulative linked chain or retrievable hierarchy. Loading
the newest synthesis cannot erase access to earlier compressed continuity.
- Identity and system pins have explicit budgets and priority classes. Overflow
raises visible context pressure and invokes an approved strategy; recursive
pinning never grows without measurement.
- Git remains provenance and rollback history. Hot mutable cursors, queues, and
ownership transfers use atomic storage rather than treating commits as
locks.
### Entry-path and surface parity
- Local, server, remote, TUI, voice, and network surfaces call the same
conversation constructor and production composition root.
- **Capability-set parity** requires every production entry path to support the
approved baseline for cancellation, images, history, tools, surfacings,
failures, completion receipts, and telemetry.
- **Negotiation correctness** applies only to explicitly approved exceptions.
Structured absence must include the lost capability, owner approval, reason,
expiry, and restoration canary. Honest absence is not parity and cannot make
a degraded path production-equivalent.
- External-to-internal conversation mappings are bidirectional and durable so
responses can return to the originating room, device, or peer.
- Transport identity and portable agent identity remain distinct. Configured
peer keys are pinned and checked before events enter the canonical spine.
### Failure handling
- A failed provider or tool call records a typed failure outcome; stream closure
cannot masquerade as clean completion.
- Configuration changes validate, persist, and atomically swap a complete
runtime snapshot. Failure leaves the previous snapshot authoritative.
- Scheduled and background work receives a durable run ID, completion/failure
receipt, retry policy, and visible owner.
- Stale evidence, replay gaps, cursor conflicts, and split-brain ownership are
first-class blocked states requiring reconciliation rather than inference.
### Required ablations before implementation
1. Independent N+1/reflection/synthesis toggles over 30- and 110-turn fixtures.
2. Item-ID synthesis cursor versus date cursor under same-day and capped input.
3. Local/server/remote semantic parity with a deterministic provider.
4. Resource routing under provider outage, locality restriction, and subagent
inheritance.
5. Event-bus lag and restart with and without durable replay.
6. Ratatui/tuie capability rehearsal before either engine is removed.
7. Git-backed versus transactional hot-state writes under concurrent turns.
Each item is specified as an executable, planning-safe experiment card in
`survey/souveraine/ablations.md`. No ablation mutates Rune's live runtime;
Souveraine tests use its disposable checkout, architecture-only mechanisms use
a disposable harness, and Rune integration tests wait for their implementation
phase.
### Efficiency and performance refinements
These refinements change physical execution, not semantic guarantees. No event,
cognitive timescale, surface capability, evidence link, or verification path is
removed.
1. **Split hot transport from durable history.** Commit envelopes through a
batched write-ahead append, then publish compact references on the in-memory
bus. Consumers fetch payloads only when needed. This retains the complete
record while keeping large tool outputs and media off the latency-sensitive
path.
2. **Partition ordering by authority key.** Preserve strict order per
agent/conversation/task while allowing unrelated partitions to execute in
parallel under bounded concurrency. Global serialization is expensive and
unnecessary; global disorder is worse. The partition key makes the bargain
explicit.
3. **Use incremental materialized views.** Task state, pin state, resource
health, and current context are checkpointed projections with source offsets
and hashes. Normal reads become constant-size lookups; replay remains the
recovery and audit mechanism rather than the price of every screen refresh.
4. **Deduplicate evidence capsules by content hash.** Events and proposals carry
references to immutable evidence blobs instead of repeatedly copying tool
output, transcripts, or source text into queues and prompts. A bounded
manifest supplies only the relevant slices to a model.
5. **Coalesce without erasing.** High-frequency telemetry is durably archived
in cold segments while the hot bus emits typed windows such as latest value,
min/max, count, and anomaly references. Urgent, state-transition, authority,
and completion events are never coalesced.
6. **Use adaptive cognitive triggers with maximum-latency guarantees.** N+1,
reflection, and synthesis may debounce bursts, reuse unchanged evidence, and
skip a model call when a deterministic no-change canary passes. A forced
deadline preserves each timescale's function even during quiet periods.
7. **Route in two stages.** The deterministic eligibility filter runs first and
is cached by task signature plus resource-state version. A small local model
ranks eligible choices; a larger model is used only for low-confidence or
high-impact ambiguity. Cache invalidation follows observed health, policy,
capability, or cost changes—not a timer alone.
8. **Share immutable runtime snapshots.** Configuration, policy, capability,
and resource tables are compiled once into a versioned snapshot. Hot paths
use lock-light reads; validated updates atomically swap the snapshot. This
avoids repeated parsing while preventing live/disk split-brain.
9. **Give priority lanes reserved capacity.** Halt, cancellation, verification
failure, and direct-user turns cannot queue behind archival synthesis or
bulk telemetry. Background work has explicit CPU, inference, I/O, and token
quotas and yields before interactive latency degrades.
10. **Make performance a verified contract.** Canaries measure p50/p95 turn
dispatch latency, queue depth, replay recovery time, duplicate suppression,
model-call avoidance, token/I/O cost per turn, and semantic parity. An
optimization is accepted only when these improve without losing receipts,
source links, surface parity, or cognitive outputs on the reference
fixtures.
### Acceptance boundary
This section defines architecture, not implementation permission. It is
acceptable only if each entry path produces equivalent identity, tool,
history, cancellation, surfacing, failure, and completion semantics; every
canonical mutation has a receipt; each cognitive timescale can fail or be
disabled without suppressing the others; and omissions name the capability
they trade away.
## [S9] Components, data flow, failures, and verification
**Section status:** Approved by Nick; independent consistency defects revised.
### Component boundaries
| Component | Owns | Must not own |
|---|---|---|
| Intent and Plan Store | Versioned goals, decisions, stages, batches, viability, and scoped approvals | Execution or inferred completion |
| Pin Registry | Transactional root/stage/batch/checkpoint ownership and stale evidence | Task truth duplicated from Gitea |
| Durable Event Store | Immutable envelopes, offsets, causal links, payload references | Interpretive conclusions |
| Projection Engine | Incremental task, pin, resource, context, and TUI views | Canonical writes without source events |
| Evidence Store | Content-addressed artifacts, slices, hashes, provenance | Authority decisions |
| Reflex Engine | Deterministic facts and no-change canaries | Goal revision or model judgment |
| Cognitive Workers | N+1, reflection, synthesis proposals and confidence | Direct canonical mutation |
| Resource Registry | Observed capabilities, health, locality, cost, permissions, freshness | Choosing ineligible resources |
| Advisory Router | Ranking eligible resources and explaining trade-offs | Final assignment or silent fallback |
| Authority Scheduler | Policy, dependencies, budgets, leases, idempotency, effects, retries | Inventing evidence or owner consent |
| Delegation Planner | Translating a started plan version into ordered Gitea coordination mutations | Approving the plan or marking execution complete |
| Gitea Outbox/Reconciler | Pending remote mutations, version preconditions, retries, accepted remote IDs, and explicit conflicts | Treating queued intent as accepted remote state |
| Effect Adapters | Tool, provider, Gitea, memory, surface, and device execution | Declaring their own success |
| Verification Ledger | Receipts, canary results, semantic parity, rollback evidence | Treating requested work as completed work |
| Surface Gateway | Bidirectional IDs, capability negotiation, event projection | Entry-path-specific agent construction |
Each component exposes a versioned interface. The production composition root
constructs exactly one instance of every authority-bearing component and passes
references into local, server, remote, TUI, voice, and future device adapters.
### Canonical records
- **IntentRecord:** goal, owner, constraints, version, approval boundary.
- **PlanRecord:** research links, alternatives, stages, dependencies, viability.
- **ApprovalRecord:** approver, plan version, approved item/version set,
dependency closure, scope, granted/revoked timestamps, and superseding record.
- **PinRecord:** plan/item IDs, pin type, owner lease, evidence, state, version.
- **EventEnvelope:** event ID, parent ID, partition key, actor/device IDs,
timestamps, schema, authorization, payload hash.
- **EvidenceCapsule:** immutable content hash, source, slices, freshness,
confidentiality, verification state.
- **ProposalRecord:** cognitive worker, evidence IDs, action, confidence, expiry,
cost, capability-loss note.
- **ResourceSnapshot:** observed capabilities, health, locality, cost, limits,
permissions, source evidence, version.
- **EffectRecord:** accepted proposal/request, policy decision, resource lease,
idempotency key, attempt state.
- **OutboxRecord:** mutation ID, remote target, expected remote version, plan and
ApprovalRecord versions, ordering key, payload hash, attempt state, returned
remote ID/version, conflict state, and resolver identity.
- **ReceiptRecord:** observed result, verifier, canary evidence, completion state,
rollback pointer.
- **ProjectionCheckpoint:** source offset, state hash, projection schema.
All records carry stable IDs and schema versions. Schema migration is whole-path:
writers, readers, projections, APIs, tests, docs, and migration tooling move
together rather than maintaining a permanent old/new dialect bazaar.
### End-to-end planning flow
1. A Planning thread creates or resumes an IntentRecord and pins the root plan.
2. Questions and research write EvidenceCapsules and versioned PlanRecords.
3. Settled decisions update the running report; unresolved claims remain linked
to evidence or ablations.
4. Viability projects from current evidence and resource snapshots. It advises;
it never authorizes.
5. Nick's explicit **start** creates an ApprovalRecord for a precise plan
version and item/version set. Impact analysis computes the dependency closure
of later edits, emits a receipt, and revokes only intersecting approvals;
ambiguous impact revokes the whole approval rather than guessing.
6. The Delegator materializes Gitea parent, stage, and context-bounded batch
issues through an outbox-backed effect. Returned durable IDs are recorded.
7. Initial operational pins activate from the valid ApprovalRecord. Later stage
and batch pins transfer transactionally only after the prior unit has a
verified ReceiptRecord. Terminal plan states clear all operational pins.
### End-to-end execution flow
1. An approved batch emits a WorkRequested event with acceptance canaries.
2. Reflexes assemble current dependencies, repository state, resource health,
authority, and evidence freshness.
3. The eligibility filter rejects impossible resources. The advisory router
ranks the remainder; the scheduler leases one with a bounded fallback chain.
4. An isolated adapter executes under the batch's branch/worktree, capability,
context, token, I/O, and time budgets.
5. Tool and provider outputs become evidence blobs plus compact progress events.
6. Verification runs independently from the implementation claim. Failure
emits typed diagnostics and a retry or blocked decision; success emits a
receipt linked to commands, outputs, commit, PR, and required checks.
7. Gitea and TUI projections update from the same durable events. A successful
display refresh is never substituted for a receipt.
8. Only a verified receipt closes the batch and advances its operational pin.
### Cognitive flow
- Reflexes run on every relevant event and suppress exact no-change duplicates.
- N+1 consumes a bounded capsule after meaningful turns or at its maximum
deadline, emitting omissions, conflicts, commitments, or surfacing proposals.
- Reflection runs at task/session boundaries and its deadline, reviewing linked
evidence and outcomes rather than an arbitrary transcript tail.
- Synthesis runs on evidence pressure or its safety deadline, committing exact
input IDs and a linked output before advancing its cursor.
- Each cadence owns a queue, budget, cursor, health record, and dead-letter
stream. Failure in one cadence cannot prevent another from running.
- Accepted lessons enter governed memory as provisional records; promotion and
identity changes still require their existing evidence gates.
### Failure state machine
Effects move through **requested → eligible → leased → running → observed →
verified**. Exceptional states are **deferred**, **retryable_failure**,
**terminal_failure**, **blocked**, **cancelled**, **superseded**, and
**reconciliation_required**.
- Every transition requires the expected record version and an event ID.
- A lease expiry permits reassignment but never implies the original attempt
stopped; late results are quarantined for reconciliation.
- Retry policy distinguishes transport, capacity, policy, invalid-input,
implementation, and verification failures. Two identical implementation
failures trigger critique rather than a third ceremonial collision with the
same wall.
- Offline Gitea mutations remain in the append-only outbox with stable IDs.
Their lifecycle is **queued → sending → accepted**, with
**retryable_failure**, **conflict**, **cancelled**, and **superseded**
alternatives. Reconnection compares the expected remote version: an
unchanged target applies once by mutation ID; an already-applied payload
records the returned remote version; a newer remote version enters conflict;
deleted or unauthorized targets enter blocked. Only Nick or a policy named in
the ApprovalRecord may resolve content conflicts.
- Cancellation has reserved capacity and reaches adapters directly. Completion
arriving after cancellation is recorded but not silently accepted.
- Projection corruption rebuilds from the last valid checkpoint plus durable
events; the event store remains untouched.
- Evidence or resource health older than policy permits becomes **unknown**,
never false or healthy by default.
### Verification matrix
| Contract | Canary evidence |
|---|---|
| One authority path | Composition-root inventory; fail on duplicate stores, schedulers, or registries |
| Entry-path parity | Capability-set parity is mandatory; approved exceptions must prove negotiation correctness, expiry, and restoration |
| Pin invariants | Crash/property tests prove one owner, no gap after committed transfer, no advance without receipt |
| Event durability | Lag, restart, duplicate delivery, corrupt projection, and replay tests |
| Cognitive independence | Toggle/fail each cadence separately across 30/110-turn fixtures |
| Synthesis continuity | Same-day, capped-input, duplicate, crash-before-commit, and cumulative-retrieval tests |
| Resource governance | Outage, stale health, locality, permission, budget, subagent inheritance, and fallback tests |
| Scheduler correctness | Lease expiry, late result, cancellation race, idempotent retry, and dependency tests |
| Gitea resilience | Offline outbox, conflicting remote update, required-check failure, and reconciliation tests |
| Schema migration | Old fixture migrates once; all current readers reject unmigrated dialects with actionable errors |
| Performance | Versioned benchmark protocol measures p50/p95 latency, queue depth, replay time, I/O, tokens, model calls, and interactive-priority isolation |
| Capability preservation | Deterministic golden fixtures and typed invariants compare cognitive outputs, evidence links, surfaces, and rollback paths before/after optimization |
### Initial performance budgets
Budgets are provisional baselines to measure and revise, not decorative promises:
- reflex processing: p95 under 10 ms excluding external I/O;
- hot event publish after durable append: p95 under 25 ms locally;
- projection read for header/task pane: p95 under 20 ms;
- direct-user work begins despite background load: p95 under 100 ms;
- unchanged-evidence cognitive pass avoids a model call in at least 80% of the
deterministic no-change fixture cases;
- replay from the latest projection checkpoint: under 2 seconds for 100,000
envelopes on Nick's current machine;
- no measurable semantic-parity loss across the reference fixture matrix.
The first implementation phase records the current pre-change system only as an
operational cost/latency baseline; it cannot stand in for capabilities that do
not yet exist. Foundation then builds the smallest semantically complete,
unoptimized reference implementing every approved record, receipt, authority,
and entry-path contract. That reference supplies the golden capability fixtures
and becomes the sole before/after baseline for optimization. Both baselines are
owner-approved BenchmarkBaselineRecords, but only the second can accept or
reject capability preservation. The reproducible workload, environment
fingerprint, sample counts, determinism controls, tolerance rules, and
baseline-change process are defined in
`specs/2026-09-03-planning-task-orchestration-benchmark-protocol.md`. A budget
without a baseline is merely a wish wearing a stopwatch.
## [S10] Batch chaining and context minimization
**Section status:** Approved by Nick; added at his request so approved
execution never requires the LLM to hold or re-process the full plan.
### Successor selection
- Within an approved plan, verified batches chain by event, not by model
attention. A BatchCompletionReceipt triggers a deterministic successor
selector that consults the dependency graph and approval scope, then emits
WorkRequested for exactly the next eligible unit.
- A successor becomes eligible only when every dependency has a verified
receipt, the unit is within the approved item/version set, and resources are
available. The selector records its choice and rationale as an EffectRecord;
it cannot choose a unit whose dependencies are unverified.
- If multiple units become eligible, the scheduler orders them by declared
dependency order and priority policy, never by model preference.
### Per-unit context bundle
- Each worker receives a bounded handoff manifest: unit ID, dependency of
verified receipts, acceptance canaries, resource posture, evidence
references, schema versions, and the exact expected receipt contract.
- The manifest contains references and hashes, not copied transcripts or plan
history. Workers resolve evidence slices on demand through the Evidence
Store's bounded manifest.
- No worker may load the full IntentRecord or PlanRecord; anything beyond the
manifest requires an explicit, logged resolution request with a budget.
### Chain boundaries
- Chaining applies strictly within approved execution. Planning to execution
remains gated by Nick's explicit **start**; mode changes, goal revisions, and
scope changes invalidate eligibility according to the ApprovalRecord rules.
- A chain never skips receipt verification. Failed or unverified units enter
retryable_failure, terminal_failure, or blocked through the failure state
machine. A successor can never be selected on a predecessor's self-report.
- Subjective or owner-judgment outcomes pause the chain and wait for Nick;
objective units continue only while their dependency closure is objective.
### Resume and idempotency
- After interruption, crash, or restart, execution resumes from the last
verified receipt, not from the first unit. The successor selector is
deterministic on the same durable event state, so re-entry produces the same
next unit.
- Unit execution carries idempotency keys; a repeated WorkRequested event after
a crash replays the same EffectRecord instead of spawning a duplicate.
- Cancellation is recorded as a terminal state for the chain position, and any
successor selection after cancellation requires a fresh explicit start
approval for the remaining scope.
### Verification
- Chain canaries prove: exactly one eligible successor per receipt; no
successor on unverified dependencies; bounded manifest size; no full-plan
reads; identical successor choice on replay from the same event state; and
no chain advance after cancellation or outside approval scope.
+1 -1
View File
@@ -3,4 +3,4 @@
This is the main entry point for the Rune program.
"""
__version__ = "0.44.0"
__version__ = "0.49.0"
+46
View File
@@ -0,0 +1,46 @@
#!/usr/bin/env python3
"""Cognitive strata canary: no-change suppression + toggle independence.
Failure isolation is covered by the orchestration test suite
(test_failing_worker_is_isolated_and_health_records_it) rather than by this
canary, which focuses on the two properties that must hold on every boot.
"""
import sys
import tempfile
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
from rune.orchestration.cognitive import build_cognitive
def main() -> int:
with tempfile.TemporaryDirectory() as td:
runtime = build_cognitive(Path(td))
evidence = [{"event_type": "user.message"}]
first = runtime.process({"event_type": "a"}, evidence)
second = runtime.process({"event_type": "b"}, evidence) # unchanged evidence
if first["results"][0]["skipped"] != 0:
print("COGNITIVE CANARY FAILED: first pass skipped", file=sys.stderr)
return 1
if second["results"][0]["skipped"] != 1:
print(
"COGNITIVE CANARY FAILED: unchanged evidence not skipped",
file=sys.stderr,
)
return 1
runtime.registry.disable("reflection")
runtime.registry.disable("synthesis")
third = runtime.process({"event_type": "c"}, [{"event_type": "new"}])
if {r["worker"] for r in third["results"]} != {"n1"}:
print(
"COGNITIVE CANARY FAILED: toggle independence broken", file=sys.stderr
)
return 1
print("COGNITIVE CANARY PASSED", third["facts"])
return 0
if __name__ == "__main__":
sys.exit(main())
+185 -5
View File
@@ -14,6 +14,22 @@ def check(name: str, condition: bool, failures: list[str]) -> None:
failures.append(name)
def has_idempotent_label_removal(workflow: str) -> bool:
"""Require the 404 guard to wrap the stale-label removal itself."""
return bool(
re.search(
r"for \(const name of staleManaged\) \{\s*"
r"try \{\s*await github\.rest\.issues\.removeLabel\(\{.*?"
r"\}\);\s*\} catch \(error\) \{\s*"
r"if \(error\.status !== 404\) \{\s*throw error;\s*\}\s*"
r"console\.log\(`Managed label already absent: \$\{name\}`\);\s*"
r"\}\s*\}",
workflow,
re.DOTALL,
)
)
def main() -> int:
failures: list[str] = []
issue_dir = ROOT / ".gitea" / "ISSUE_TEMPLATE"
@@ -22,7 +38,8 @@ def main() -> int:
backup = (ROOT / ".gitea" / "workflows" / "auto-backup.yml").read_text()
versioning = (ROOT / ".gitea" / "workflows" / "auto-version.yml").read_text()
backup_script = (ROOT / "scripts" / "gitea-backup.sh").read_text()
backup_contract = backup + backup_script
restore_script = (ROOT / "scripts" / "gitea-restore-verify.sh").read_text()
rune_version = (ROOT / "src" / "rune" / "__init__.py").read_text()
gitea_sources = "\n".join(
path.read_text() for path in (ROOT / "shared").glob("gitea*.py")
)
@@ -48,15 +65,83 @@ def main() -> int:
failures,
)
check("bug and feature labels are exclusive", "!isBug" in auto_label, failures)
check("backup uses official gitea dump", "gitea dump" in backup_contract, failures)
managed_labels = (
"memory",
"governance",
"identity",
"growth",
"perception",
"infrastructure",
"life",
"custodian",
"priority: high",
"bug",
"feature",
)
check(
"auto-label declares every managed label",
"managedLabels" in auto_label
and all(f"'{label}'" in auto_label for label in managed_labels),
failures,
)
check(
"auto-label reads current issue labels",
"listLabelsOnIssue" in auto_label,
failures,
)
check(
"stale managed labels are removed",
"managedLabels.has" in auto_label
and "!desiredLabels.has" in auto_label
and "removeLabel" in auto_label,
failures,
)
check(
"concurrent stale-label removal is idempotent",
has_idempotent_label_removal(auto_label),
failures,
)
check(
"race contract rejects inverted 404 handling",
not has_idempotent_label_removal(
auto_label.replace("error.status !== 404", "error.status === 404", 1)
),
failures,
)
check(
"manual labels are excluded from removal",
"managedLabels.has" in auto_label,
failures,
)
check(
"only missing desired labels are added",
"!currentLabels.has" in auto_label and "missingDesired" in auto_label,
failures,
)
check(
"backup uses official gitea dump",
"GITEA_BIN" in backup_script and '"$GITEA_BIN" dump' in backup_script,
failures,
)
check(
"backup stops Gitea for consistency",
"systemctl stop gitea" in backup_script,
'"$SYSTEMCTL_BIN" stop gitea' in backup_script,
failures,
)
check(
"backup is stored off the Pi root disk",
"/media/" in backup_script and "/tmp/rune-backups" not in backup_contract,
"/media/" in backup_script
and "/tmp/rune-backups" not in backup + backup_script,
failures,
)
check(
"backup proves exact mount containment and a distinct backing device",
"BACKUP_MOUNT" in backup_script
and '"$MOUNTPOINT_BIN" -q -- "$BACKUP_MOUNT"' in backup_script
and "MAJ:MIN" in backup_script
and '[[ "$mount_device" != "$root_device" ]]' in backup_script
and "backup_lex" in backup_script
and "backup_real" in backup_script,
failures,
)
check(
@@ -65,6 +150,96 @@ def main() -> int:
failures,
)
check("backup has bounded retention", "RETENTION" in backup_script, failures)
check("backup rejects overlap with flock", "flock -n" in backup_script, failures)
check(
"backup checks every mandatory member independently",
"for required in app.ini data repos gitea-db.sql" in backup_script,
failures,
)
check(
"backup atomically publishes without clobber and links checksum last",
backup_script.rfind('ln -T -- "$STAGED_CHECKSUM"')
> backup_script.rfind('ln -T -- "$STAGED_ARCHIVE"')
and 'mv -f -- "$STAGED_' not in backup_script
and '"$BACKUP_DIR/$ARCHIVE.sha256" -ef "$STAGED_CHECKSUM"' in backup_script
and "Removing the staged checksum disarms rollback" in backup_script,
failures,
)
check(
"backup workflow runs offline safety tests first",
"tests.test_gitea_backup" in backup
and backup.index("tests.test_gitea_backup") < backup.index("rune-gitea-backup"),
failures,
)
check(
"backup workflow pins audited checkout v4.2.2 commit",
"# actions/checkout v4.2.2" in backup
and "actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683" in backup,
failures,
)
check(
"restore verifier is explicitly guarded and rootless",
"RUNE_GITEA_RESTORE_VERIFY" in restore_script
and "EUID != 0" in restore_script,
failures,
)
check(
"restore verifier is networkless and read-only",
"--network none" in restore_script and "--read-only" in restore_script,
failures,
)
check(
"restore verifier safely parses Gitea globals and rejects non-SQLite backups",
"RawConfigParser(interpolation=None, strict=True)" in restore_script
and '"[__gitea_top_level__]\\n" + contents' in restore_script
and "unsupported database DB_TYPE" in restore_script,
failures,
)
check(
"restore SQLite import denies host-file attachment semantically",
"set_authorizer" in restore_script
and "SQLITE_ATTACH" in restore_script
and "SQLITE_DETACH" in restore_script,
failures,
)
check(
"restore verifier checks SQLite and Git integrity",
"PRAGMA integrity_check" in restore_script
and "fsck --full --strict" in restore_script
and "symbolic-ref HEAD" in restore_script,
failures,
)
check(
"restore requires every database-declared canonical repository",
'LEFT JOIN "user" AS u ON u.id = r.owner_id' in restore_script
and 'f"{name}.git"' in restore_script
and "database-declared repository is missing" in restore_script,
failures,
)
check(
"restore copies archived application data before isolated overrides",
'cp -a -- "$RESTORE_ROOT/data/." "$INSTALL_ROOT/gitea/"' in restore_script
and restore_script.index('cp -a -- "$RESTORE_ROOT/data/."')
< restore_script.index('cp "$DB" "$INSTALL_ROOT/gitea/gitea.db"')
< restore_script.index("cat >\"$INSTALL_ROOT/gitea/conf/app.ini\""),
failures,
)
check(
"restore rejects Git alternates and sanitizes Git environment",
"objects/info/alternates" in restore_script
and "objects/info/http-alternates" in restore_script
and "env -i" in restore_script
and "GIT_CONFIG_NOSYSTEM=1" in restore_script,
failures,
)
check(
"restore verifier uses the audited local digest without pulls",
"docker.io/gitea/gitea@sha256:414ba5b2b1163480e9ed4213a989cd798579cfa88582a2359303273009b2b852"
in restore_script
and '"$PODMAN_BIN" image exists "$IMAGE"' in restore_script
and "--pull=never" in restore_script,
failures,
)
check(
"tracked Gitea code contains no API tokens",
not re.search(r"[0-9a-f]{40}", gitea_sources),
@@ -75,6 +250,11 @@ def main() -> int:
"src/rune/__init__.py" in versioning,
failures,
)
check(
"Rune minor version advertises provisioning and restore verification",
'__version__ = "0.49.0"' in rune_version,
failures,
)
check(
"release automation cannot push master", "git push" not in versioning, failures
)
@@ -89,7 +269,7 @@ def main() -> int:
failures,
)
print(f"RESULT: {16 - len(failures)} passed, {len(failures)} failed")
print(f"RESULT: {39 - len(failures)} passed, {len(failures)} failed")
return 1 if failures else 0
+35
View File
@@ -0,0 +1,35 @@
#!/usr/bin/env python3
"""Gitea delegation canary: offline round trip, no network access."""
import sys
import tempfile
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
from rune.orchestration.gitea_delegation import build_gitea_delegation
from rune.orchestration.records import ApprovalRecord
from rune.orchestration.transport import FakeGiteaTransport
def main() -> int:
with tempfile.TemporaryDirectory() as td:
auth = build_gitea_delegation(Path(td))
approval = ApprovalRecord(
approver="nick", plan_version="p", approved_items=("g1", "g2")
)
records = auth.delegate(approval, ["g1", "g2"], FakeGiteaTransport())
if len(records) != 3 or any(r.state != "accepted" for r in records):
print("GITEA DELEGATION CANARY FAILED: round trip", file=sys.stderr)
return 1
if auth.store.pending() != []:
print(
"GITEA DELEGATION CANARY FAILED: pending not drained", file=sys.stderr
)
return 1
print("GITEA DELEGATION CANARY PASSED", len(records))
return 0
if __name__ == "__main__":
sys.exit(main())
+54 -3
View File
@@ -21,10 +21,26 @@ def main() -> int:
text = WORKFLOW.read_text(encoding="utf-8")
lines = text.splitlines()
failures: list[str] = []
push_section = text.split(" push:", 1)[1].split(" pull_request:", 1)[0]
pull_request_section = text.split(" pull_request:", 1)[1].split("\njobs:", 1)[0]
required_gate = text.split(" required-checks:", 1)[1]
offline_job = (
text.split(" gitea-offline:", 1)[1].split(" required-checks:", 1)[0]
if " gitea-offline:" in text
else ""
)
print("=== GITEA PIPELINE CANARY ===")
check("runs on pushes to master", "push:" in text and "master" in text, failures)
check("runs on pull requests to master", "pull_request:" in text, failures)
check(
"runs on pushes to every branch",
"branches:" not in push_section,
failures,
)
check(
"pull requests remain scoped to master",
"branches: [master]" in pull_request_section,
failures,
)
check(
"lint failures are not swallowed",
any("ruff check" in line and "|| true" not in line for line in lines),
@@ -77,13 +93,48 @@ def main() -> int:
failures,
)
check("publishes a required gate", "required-checks:" in text, failures)
check(
"protected gate runs only for pull requests",
"if: gitea.event_name == 'pull_request'" in required_gate,
failures,
)
check(
"required gate depends on unit tests and verification jobs",
"needs: [lint, unit-tests, canary" in text,
failures,
)
check(
"offline Gitea job runs provisioning contracts",
"python3 -m unittest tests.test_gitea_provision" in offline_job,
failures,
)
check(
"offline Gitea job runs backup contracts",
"python3 -m unittest tests.test_gitea_backup" in offline_job,
failures,
)
check(
"offline Gitea job syntax-checks both shell scripts",
"bash -n scripts/gitea-backup.sh scripts/gitea-restore-verify.sh"
in offline_job,
failures,
)
check(
"required gate depends on offline Gitea verification",
"gitea-offline" in required_gate.split("steps:", 1)[0],
failures,
)
check(
"CI checkout actions use the audited v4.2.2 commit",
"actions/checkout@v4" not in text
and text.count(
"actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683"
)
== text.count("uses: actions/checkout@"),
failures,
)
print(f"RESULT: {14 - len(failures)} passed, {len(failures)} failed")
print(f"RESULT: {20 - len(failures)} passed, {len(failures)} failed")
return 1 if failures else 0
+39
View File
@@ -0,0 +1,39 @@
#!/usr/bin/env python3
"""Memory and synthesis canary: item-ID cursor + cumulative chain."""
import sys
import tempfile
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
from rune.orchestration.memory import EpisodicRecord
from rune.orchestration.memory_synthesis import build_memory_synthesis
def main() -> int:
with tempfile.TemporaryDirectory() as td:
auth = build_memory_synthesis(Path(td))
e1 = auth.episodic.append(
EpisodicRecord("", "user.message", ("sha256:a",), "t0")
)
e2 = auth.episodic.append(
EpisodicRecord("", "user.message", ("sha256:b",), "t1")
)
r1 = auth.synthesize([e1.episode_id, e2.episode_id], "sha256:summary1")
r2 = auth.synthesize([e1.episode_id, e2.episode_id], "sha256:summary2")
if r1.covered != 2 or r2.covered != 0:
print("MEMORY SYNTHESIS CANARY FAILED: cursor coverage", file=sys.stderr)
return 1
anc = [n.summary_ref for n in auth.chain.ancestors()]
if anc != ["sha256:summary2", "sha256:summary1"]:
print(
"MEMORY SYNTHESIS CANARY FAILED: chain not cumulative", file=sys.stderr
)
return 1
print("MEMORY SYNTHESIS CANARY PASSED", len(anc))
return 0
if __name__ == "__main__":
sys.exit(main())
@@ -0,0 +1,32 @@
#!/usr/bin/env python3
"""Foundation canary: one spine, one authority, deterministic chain."""
import json
import sys
import tempfile
from pathlib import Path
# Self-bootstrap: CI runs `python3 <canary>` without PYTHONPATH, so put the
# repository src/ on sys.path before importing rune.orchestration.
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
from rune.orchestration.composition import build_root
from rune.orchestration.fixtures import capability_preserved, run_golden_scenario
def main() -> int:
with tempfile.TemporaryDirectory() as td:
root = Path(td)
before = run_golden_scenario(root / "before")
after = run_golden_scenario(root / "after")
if not capability_preserved(before, after):
print("FOUNDATION CANARY FAILED: capability changed", file=sys.stderr)
return 1
comp = build_root({"events": root / "e.jsonl", "evidence": root / "ev"})
comp.assert_single_authority()
print("FOUNDATION CANARY PASSED", json.dumps(before))
return 0
if __name__ == "__main__":
sys.exit(main())
+41
View File
@@ -0,0 +1,41 @@
#!/usr/bin/env python3
"""Resource authority canary: unknown rejected, outage deterministic, one authority."""
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
from rune.orchestration.authority import build_resources
from rune.orchestration.composition import CompositionRoot, compose_resources
from rune.orchestration.resources import ResourceCapabilities, ResourceState
def main() -> int:
authority = build_resources()
authority.registry.register(
ResourceState("healthy", ResourceCapabilities("worker", "local"), healthy=True)
)
authority.registry.register(
ResourceState(
"a_unknown", ResourceCapabilities("worker", "local"), healthy=None
)
)
assignment = authority.broker.acquire(
"canary", {"kind": "worker", "locality": "local"}
)
if assignment is None or assignment.resource_id != "healthy":
print(
"RESOURCE AUTHORITY CANARY FAILED: unknown resource selected",
file=sys.stderr,
)
return 1
root = CompositionRoot()
compose_resources(root)
root.assert_single_authority()
print("RESOURCE AUTHORITY CANARY PASSED", assignment.resource_id)
return 0
if __name__ == "__main__":
sys.exit(main())
+86
View File
@@ -0,0 +1,86 @@
#!/usr/bin/env python3
"""Schema-migration canary: migrate once, idempotent re-run, wiring proof.
Walks the whole migration path in a tempdir:
1. Builds a tiny old-dialect fixture, migrates it once (1) and re-runs (0).
2. Asserts the canonical store rejects a fresh old-dialect append with the
typed UnmigratedDialectError (actionable: points at migrate).
3. WIRING PROOF (audit fix): exercises the LEGACY event_bus.emit through the
canonical store — emit a user.message and replay it back at the same
tempdir. The legacy shims are proven WIRED, not just unit-tested.
Prints 'SCHEMA MIGRATION CANARY PASSED' and exits 0.
"""
import json
import sys
import tempfile
from pathlib import Path
# Self-bootstrap: CI runs `python3 <canary>` without PYTHONPATH, so put the
# repository src/ on sys.path before importing rune modules.
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
from rune.event_bus import emit as legacy_emit
from rune.orchestration.event_store import DurableEventStore, UnmigratedDialectError
from rune.orchestration.migrate import old_fixture_to_new
def main() -> int:
with tempfile.TemporaryDirectory() as td:
root = Path(td)
fixture = root / "old.jsonl"
fixture.write_text(
json.dumps(
{
"event_id": "a",
"event_type": "user.message",
"channel": "conversation",
"payload": {"text": "hello"},
}
)
+ "\n"
)
# 1. Migrate once; the re-run must be idempotent (0).
n1 = old_fixture_to_new(fixture)
if n1 != 1:
print(f"FAILED: expected 1 migrated, got {n1}", file=sys.stderr)
return 1
n2 = old_fixture_to_new(fixture)
if n2 != 0:
print(f"FAILED: expected idempotent re-run 0, got {n2}", file=sys.stderr)
return 1
# 2. The canonical store rejects a fresh old-dialect append with the
# typed error (actionable: points at migrate).
store = DurableEventStore(root / "events")
try:
store.append({"event_id": "x", "event_type": "t", "channel": "c"})
except UnmigratedDialectError:
pass
else:
print("FAILED: old-dialect append was not rejected", file=sys.stderr)
return 1
# 3. WIRING PROOF: the LEGACY event_bus.emit writes THROUGH the
# canonical store — emit a user.message and replay it back at the
# same tempdir (the legacy shims are wired, not just unit-tested).
import rune.event_bus as legacy_bus
legacy_bus._store_path = lambda: root / "events.jsonl"
legacy_emit("user.message", source="user", payload={"text": "wired"})
wired = DurableEventStore(root / "events.jsonl")
recs = [r for r in wired.replay() if r["event_type"] == "user.message"]
if len(recs) != 1:
print(
"FAILED: legacy emit did not reach the canonical store",
file=sys.stderr,
)
return 1
print("SCHEMA MIGRATION CANARY PASSED")
return 0
if __name__ == "__main__":
sys.exit(main())
+33
View File
@@ -0,0 +1,33 @@
#!/usr/bin/env python3
"""Surface parity canary: same root for every entry path, offline."""
import sys
import tempfile
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
from rune.orchestration.surface_gateway import build_surface_parity
def main() -> int:
with tempfile.TemporaryDirectory() as td:
auth = build_surface_parity(Path(td))
local = auth.gateway.enter("local_me", surface_kind="local")
server = auth.gateway.enter("io_server", surface_kind="server")
if not local.peer_verified or not server.peer_verified:
print("SURFACE PARITY CANARY FAILED: peer verification", file=sys.stderr)
return 1
if not local.mapping.internal_conversation_id:
print(
"SURFACE PARITY CANARY FAILED: no internal conversation",
file=sys.stderr,
)
return 1
auth.registry.save(Path(td) / "surface_registry")
print("SURFACE PARITY CANARY PASSED")
return 0
if __name__ == "__main__":
sys.exit(main())
+93 -100
View File
@@ -17,11 +17,11 @@ Usage:
"""
import argparse
import hashlib
import json
import os
import sys
import uuid
from datetime import datetime, timezone
from pathlib import Path
_SD = os.path.dirname(os.path.abspath(__file__))
if _SD not in sys.path:
@@ -39,45 +39,38 @@ EVENT_TYPES = {
"session.started": "New conversation session began",
"session.ended": "Conversation session ended",
"rune.shutdown": "Rune shutting down",
# Conversation
"user.message": "User sent a message",
"response.generated": "Response generated",
"owner.correction": "Owner corrected Rune",
"commitment.created": "A promise/commitment was made",
"commitment.closed": "A commitment was fulfilled or abandoned",
# Memory
"memory.created": "New memory entry added",
"memory.recalled": "Memory retrieved for use",
"memory.conflict": "Contradictory memories detected",
"memory.superseded": "Memory entry superseded",
"memory.abstraction_candidate": "Repeated pattern detected",
# Capability
"capability.requested": "Capability needed",
"capability.selected": "Capability chosen for use",
"capability.failed": "Capability execution failed",
"capability.unavailable": "Needed capability not found",
"plugin.timeout": "Plugin execution timed out",
# Task
"task.created": "New task created",
"task.completed": "Task finished",
"task.failed": "Task failed",
"task.stale": "Task hasn't been touched in a while",
# Growth
"lesson.candidate": "Potential lesson identified",
"lesson.promoted": "Lesson promoted to durable knowledge",
"skill.candidate": "Potential skill identified",
"self_model.evidence": "Self-model evidence collected",
# System
"model.unavailable": "Model provider unavailable",
"storage.degraded": "Storage system degraded",
"context.pressure": "Context window under pressure",
# Hook lifecycle (maps to existing microkernel hooks)
"hook.on_startup": "Startup hook fired",
"hook.on_message": "Message hook fired",
@@ -88,97 +81,80 @@ EVENT_TYPES = {
}
def _store_path() -> Path:
"""Journal path for the canonical store (configurable for tests).
The migration keeps the SAME journal file as the legacy writer
(``~/.rune/journal/events.jsonl``) — only the record dialect changes.
"""
return Path(EVENT_LOG)
def _content_hash(payload: dict) -> str:
"""Content hash following the orchestration sha256: convention (X1)."""
return (
"sha256:"
+ hashlib.sha256(
json.dumps(payload, sort_keys=True).encode("utf-8")
).hexdigest()
)
def emit(event_type, source="", payload=None):
"""Emit an event to the journal. Uses log_utils for rotation and atomic append."""
event = {
"id": str(uuid.uuid4())[:8],
"timestamp": datetime.now(timezone.utc).isoformat(),
"type": event_type,
"source": source,
"payload": payload or {},
}
"""Emit an event to the journal (delegates to the canonical store).
try:
from log_utils import append_jsonl, update_stats
append_jsonl(EVENT_LOG, event, max_size_mb=5, max_rotations=5)
update_stats(STATS_PATH, total=1, **{f"type_{event_type}": 1})
except ImportError:
os.makedirs(JOURNAL_DIR, exist_ok=True)
# Simple rotation: if file exceeds 5MB, rotate
if os.path.exists(EVENT_LOG) and os.path.getsize(EVENT_LOG) > 5 * 1024 * 1024:
for i in range(4, 0, -1):
src = f"{EVENT_LOG}.{i}"
dst = f"{EVENT_LOG}.{i+1}"
if os.path.exists(src):
os.rename(src, dst)
os.rename(EVENT_LOG, f"{EVENT_LOG}.1")
with open(EVENT_LOG, "a") as f:
f.write(json.dumps(event) + "\n")
``source`` must be a plain string — a non-string (e.g. a dict passed by
mistake) would silently land in the canonical ``partition_key`` and
violate its ``str`` contract; fail loud instead of writing a poison
record (X3 quality discharge).
"""
if not isinstance(source, str):
raise ValueError(
f"source must be a string, got {type(source).__name__!r}; "
"pass the payload via payload=..."
)
from rune.orchestration.event_store import DurableEventStore
from rune.orchestration.records import EventEnvelope
return event
env = EventEnvelope(
event_type=event_type,
partition_key=(source or "system"),
payload_hash=_content_hash(payload) if payload else "",
)
# The canonical store buffers to batch_size; flush so the record is
# durable immediately (and the journal is readable right after emit).
store = DurableEventStore(_store_path())
store.append(env)
store.flush()
return env.to_dict()
def log_events(limit=20, event_type=None):
"""Read recent events from the journal. Uses tail reader for efficiency."""
if not os.path.exists(EVENT_LOG):
return []
def log(limit=None, event_type=None):
"""Read recorded events in append order (delegates to store.replay()).
try:
from log_utils import read_tail
events = read_tail(EVENT_LOG, n=limit * 2) # Read extra for filtering
except ImportError:
events = []
with open(EVENT_LOG) as f:
for line in f:
line = line.strip()
if line:
try:
events.append(json.loads(line))
except json.JSONDecodeError:
continue
``limit`` truncates to the most recent N; ``event_type`` filters by the
canonical ``event_type`` field (the legacy CLI's --type/--limit args).
"""
from rune.orchestration.event_store import DurableEventStore
events = DurableEventStore(_store_path()).replay()
if event_type:
events = [e for e in events if e.get("type") == event_type]
return events[-limit:]
events = [e for e in events if e.get("event_type") == event_type]
if limit is not None:
events = events[-limit:]
return events
def stats():
"""Show event journal statistics. Uses sidecar stats file for efficiency."""
try:
from log_utils import load_stats
sidecar = load_stats(STATS_PATH)
if sidecar.get("total", 0) > 0:
file_size = os.path.getsize(EVENT_LOG) if os.path.exists(EVENT_LOG) else 0
return {
"total": sidecar["total"],
"types": {k.replace("type_", ""): v for k, v in sidecar.items()
if k.startswith("type_")},
"file_size": file_size,
}
except ImportError:
pass
"""Return (total, by_type) counted from the canonical store."""
from rune.orchestration.event_store import DurableEventStore
# Fallback: full scan
if not os.path.exists(EVENT_LOG):
return {"total": 0, "types": {}, "file_size": 0}
total = 0
types = {}
with open(EVENT_LOG) as f:
for line in f:
line = line.strip()
if line:
try:
event = json.loads(line)
total += 1
t = event.get("type", "unknown")
types[t] = types.get(t, 0) + 1
except json.JSONDecodeError:
continue
file_size = os.path.getsize(EVENT_LOG)
return {"total": total, "types": types, "file_size": file_size}
events = DurableEventStore(_store_path()).replay()
by_type = {}
for e in events:
t = e.get("event_type", "unknown")
by_type[t] = by_type.get(t, 0) + 1
return len(events), by_type
def handle_hook(event, data=None):
@@ -196,7 +172,12 @@ def handle_hook(event, data=None):
payload = {k: str(v)[:200] for k, v in data.items()}
emitted = emit(event_type, source=source, payload=payload)
return {"status": "ok", "event": event, "handler": "event_bus", "logged": emitted["id"]}
return {
"status": "ok",
"event": event,
"handler": "event_bus",
"logged": emitted["event_id"],
}
def main():
@@ -219,27 +200,39 @@ def main():
return
payload = json.loads(args.payload) if args.payload else None
result = emit(event_type, source=args.source, payload=payload)
print(json.dumps(result, indent=2) if args.json else
f"Emitted {result['type']} ({result['id']})")
print(
json.dumps(result, indent=2)
if args.json
else f"Emitted {result['event_type']} ({result['event_id']})"
)
elif args.cmd == "log":
events = log_events(limit=args.limit, event_type=args.type)
events = log(limit=args.limit, event_type=args.type)
if args.json:
print(json.dumps(events, indent=2))
else:
for e in events:
ts = e.get("timestamp", "?")[:19]
print(f" {ts} [{e.get('type', '?'):25}] {e.get('source', '?')}")
print(
f" {ts} [{e.get('event_type', '?'):25}] {e.get('partition_key', '?')}"
)
elif args.cmd == "stats":
s = stats()
total, types = stats()
file_size = (
os.path.getsize(_store_path()) if os.path.exists(_store_path()) else 0
)
if args.json:
print(json.dumps(s, indent=2))
print(
json.dumps(
{"total": total, "types": types, "file_size": file_size}, indent=2
)
)
else:
print(f"Events: {s['total']}")
print(f"File size: {s['file_size']:,} bytes")
if s.get("types"):
for t, count in sorted(s["types"].items(), key=lambda x: -x[1])[:10]:
print(f"Events: {total}")
print(f"File size: {file_size:,} bytes")
if types:
for t, count in sorted(types.items(), key=lambda x: -x[1])[:10]:
print(f" {t:30} {count}")
elif args.cmd == "hook":
@@ -267,5 +260,5 @@ PLUGIN = {
"dependencies": [],
"provides": ["event_bus"],
"priority": 1,
"auto_activate": True
"auto_activate": True,
}
+158 -23
View File
@@ -9,6 +9,7 @@ Usage:
event_envelope.py test
"""
import hashlib
import json
import re
import sys
@@ -21,8 +22,11 @@ from typing import Optional
@dataclass
class EventEnvelope:
"""Normalized event wrapper."""
event_id: str = field(default_factory=lambda: str(uuid.uuid4()))
timestamp: str = field(default_factory=lambda: datetime.now(timezone.utc).isoformat())
timestamp: str = field(
default_factory=lambda: datetime.now(timezone.utc).isoformat()
)
source: str = "" # user, task, tool, system, background, sensor
channel: str = "" # conversation, work, system, tool, background, sensor
authority_class: str = "" # conversation, status_query, directive, task_amendment, pause, resume, cancel, emergency, system_event
@@ -30,6 +34,54 @@ class EventEnvelope:
payload: dict = field(default_factory=dict)
provenance: dict = field(default_factory=dict)
urgency: str = "normal" # low, normal, high, critical
# The canonical orchestration record this legacy wrapper was built from
# (schema migration shim). Not part of legacy repr/equality.
_canonical: object = field(default=None, repr=False, compare=False)
@classmethod
def create(cls, source, channel, payload):
"""Delegate to the canonical orchestration envelope (schema migration).
The legacy EventEnvelope keeps its dataclass shape for callers, but the
canonical record is built first and the legacy wrapper is derived from it
— one dialect, not two.
"""
from rune.orchestration.records import EventEnvelope as Canonical
canonical = Canonical(
event_type=(
CHANNELS.get(channel, {}).get("prior") or channel or source or "unknown"
),
partition_key=channel,
)
return cls(
event_id=canonical.event_id,
timestamp=canonical.timestamp,
source=source,
channel=channel,
authority_class="",
task_id=None,
payload=payload,
_canonical=canonical,
)
def to_canonical_dict(self):
"""Canonical dict (schema_version 1) with content-addressed payload_hash.
Requires the envelope to have been created via ``EventEnvelope.create``
(the only path that binds a canonical record). Envelopes built through
the legacy direct constructor / ``create_envelope`` do not have the
canonical bridge — fail loudly rather than mislead.
"""
if self._canonical is None:
raise RuntimeError(
"canonical bridge unavailable: this envelope was not created "
"via EventEnvelope.create(); use create(source, channel, "
"payload) for the canonical-backed path"
)
data = self._canonical.to_dict()
data["payload_hash"] = _content_hash(self.payload)
return data
def to_dict(self):
return {
@@ -45,14 +97,48 @@ class EventEnvelope:
}
def _content_hash(payload: dict) -> str:
"""Content hash following the orchestration sha256: convention (X1)."""
return (
"sha256:"
+ hashlib.sha256(
json.dumps(payload, sort_keys=True).encode("utf-8")
).hexdigest()
)
# Channel definitions
CHANNELS = {
"conversation": {"prior": "conversation", "tools_allowed": False, "verification_expected": False},
"work": {"prior": "directive", "tools_allowed": True, "verification_expected": True},
"system": {"prior": "system_event", "tools_allowed": False, "verification_expected": False},
"tool": {"prior": "tool_result", "tools_allowed": False, "verification_expected": False},
"background": {"prior": "background", "tools_allowed": False, "verification_expected": False},
"sensor": {"prior": "sensor", "tools_allowed": False, "verification_expected": False},
"conversation": {
"prior": "conversation",
"tools_allowed": False,
"verification_expected": False,
},
"work": {
"prior": "directive",
"tools_allowed": True,
"verification_expected": True,
},
"system": {
"prior": "system_event",
"tools_allowed": False,
"verification_expected": False,
},
"tool": {
"prior": "tool_result",
"tools_allowed": False,
"verification_expected": False,
},
"background": {
"prior": "background",
"tools_allowed": False,
"verification_expected": False,
},
"sensor": {
"prior": "sensor",
"tools_allowed": False,
"verification_expected": False,
},
}
# Authority class definitions
@@ -75,25 +161,53 @@ def classify_authority(text: str, channel: str) -> str:
# Task amendments — check BEFORE emergency to avoid false positives
# e.g., "Stop doing that" should be task_amendment, not emergency
if any(re.search(r'\b' + re.escape(word) + r'\b', text_lower) for word in ["don't", "dont", "stop doing", "actually", "instead", "rather"]):
if any(
re.search(r"\b" + re.escape(word) + r"\b", text_lower)
for word in ["don't", "dont", "stop doing", "actually", "instead", "rather"]
):
return "task_amendment"
# Emergency signals
if any(re.search(r'\b' + re.escape(word) + r'\b', text_lower) for word in ["stop", "cancel", "abort", "emergency", "halt"]):
if any(
re.search(r"\b" + re.escape(word) + r"\b", text_lower)
for word in ["stop", "cancel", "abort", "emergency", "halt"]
):
return "emergency"
# Pause/resume
if any(re.search(r'\b' + re.escape(word) + r'\b', text_lower) for word in ["pause", "wait", "hold"]):
if any(
re.search(r"\b" + re.escape(word) + r"\b", text_lower)
for word in ["pause", "wait", "hold"]
):
return "pause"
if any(re.search(r'\b' + re.escape(word) + r'\b', text_lower) for word in ["resume", "continue", "go"]):
if any(
re.search(r"\b" + re.escape(word) + r"\b", text_lower)
for word in ["resume", "continue", "go"]
):
return "resume"
# Directives (imperative mood)
if any(re.search(r'\b' + re.escape(word) + r'\b', text_lower) for word in ["fix", "run", "build", "test", "create", "add", "remove", "update", "change"]):
if any(
re.search(r"\b" + re.escape(word) + r"\b", text_lower)
for word in [
"fix",
"run",
"build",
"test",
"create",
"add",
"remove",
"update",
"change",
]
):
return "directive"
# Status queries
if any(re.search(r'\b' + re.escape(word) + r'\b', text_lower) for word in ["how", "what", "status", "progress", "check"]):
if any(
re.search(r"\b" + re.escape(word) + r"\b", text_lower)
for word in ["how", "what", "status", "progress", "check"]
):
return "status_query"
# Default based on channel
@@ -111,19 +225,30 @@ def classify_urgency(text: str, authority_class: str) -> str:
return "critical"
# High urgency
if any(re.search(r'\b' + re.escape(word) + r'\b', text_lower) for word in ["urgent", "asap", "immediately", "now", "critical"]):
if any(
re.search(r"\b" + re.escape(word) + r"\b", text_lower)
for word in ["urgent", "asap", "immediately", "now", "critical"]
):
return "high"
# Low urgency
if any(re.search(r'\b' + re.escape(word) + r'\b', text_lower) for word in ["when you can", "no rush", "eventually", "sometime"]):
if any(
re.search(r"\b" + re.escape(word) + r"\b", text_lower)
for word in ["when you can", "no rush", "eventually", "sometime"]
):
return "low"
return "normal"
def create_envelope(source: str, channel: str, payload: dict,
authority_class: str = None, task_id: str = None,
urgency: str = None) -> EventEnvelope:
def create_envelope(
source: str,
channel: str,
payload: dict,
authority_class: str = None,
task_id: str = None,
urgency: str = None,
) -> EventEnvelope:
"""Create an EventEnvelope from raw input."""
# Auto-classify authority if not provided
@@ -158,19 +283,27 @@ def test_envelope():
# Test directive
envelope = create_envelope("user", "work", {"text": "Fix the failing test"})
print(f"Directive: authority={envelope.authority_class}, urgency={envelope.urgency}")
print(
f"Directive: authority={envelope.authority_class}, urgency={envelope.urgency}"
)
assert envelope.authority_class == "directive"
assert envelope.urgency == "normal"
# Test emergency
envelope = create_envelope("user", "work", {"text": "Stop everything!"})
print(f"Emergency: authority={envelope.authority_class}, urgency={envelope.urgency}")
print(
f"Emergency: authority={envelope.authority_class}, urgency={envelope.urgency}"
)
assert envelope.authority_class == "emergency"
assert envelope.urgency == "critical"
# Test task amendment
envelope = create_envelope("user", "conversation", {"text": "Actually, don't modify the microkernel"})
print(f"Amendment: authority={envelope.authority_class}, urgency={envelope.urgency}")
envelope = create_envelope(
"user", "conversation", {"text": "Actually, don't modify the microkernel"}
)
print(
f"Amendment: authority={envelope.authority_class}, urgency={envelope.urgency}"
)
assert envelope.authority_class == "task_amendment"
print("✅ All tests passed!")
@@ -188,4 +321,6 @@ if __name__ == "__main__":
else:
print("Usage:")
print(" event_envelope.py test — run tests")
print(" event_envelope.py create <source> <channel> <payload_json> — create envelope")
print(
" event_envelope.py create <source> <channel> <payload_json> — create envelope"
)
+1
View File
@@ -0,0 +1 @@
"""Orchestration spine — one event/effect authority (approved design)."""
+50
View File
@@ -0,0 +1,50 @@
"""Atomic hot-state writes (S8).
Hot cursors, queues, and ownership transfers are written via
temp-file + fsync + rename: a crash or writer failure never leaves a
corrupt target, and temp files never leak. Commits, not locks, are the
durability primitive for hot mutable state.
"""
from __future__ import annotations
import os
import tempfile
from collections.abc import Callable
from pathlib import Path
from typing import TextIO
Writer = Callable[[TextIO], None]
def atomic_write(path: Path, writer: Writer) -> None:
path = Path(path)
path.parent.mkdir(parents=True, exist_ok=True)
fd, tmp_name = tempfile.mkstemp(
dir=str(path.parent), prefix=path.name + ".", suffix=".tmp"
)
tmp = Path(tmp_name)
try:
with os.fdopen(fd, "w", encoding="utf-8") as fh:
writer(fh)
fh.flush()
os.fsync(fh.fileno())
os.replace(tmp, path)
except BaseException:
tmp.unlink(missing_ok=True)
raise
class AtomicStore:
def __init__(self, root: Path):
self.root = Path(root)
self.root.mkdir(parents=True, exist_ok=True)
def write(self, name: str, content: str) -> None:
atomic_write(self.root / name, lambda fh: fh.write(content))
def read(self, name: str) -> str | None:
path = self.root / name
if not path.exists():
return None
return path.read_text(encoding="utf-8")
+33
View File
@@ -0,0 +1,33 @@
"""Resource authority composition (S8).
One canonical registry, one deterministic filter, one advisory router, one
scheduler (the final authority), and one broker (the single acquire path).
"""
from __future__ import annotations
from dataclasses import dataclass
from rune.orchestration.broker import ResourceBroker
from rune.orchestration.eligibility import EligibilityFilter
from rune.orchestration.resources import Registry
from rune.orchestration.router import CostBiasRouter
from rune.orchestration.scheduler import ResourceScheduler
@dataclass
class ResourceAuthority:
registry: Registry
filter: EligibilityFilter
router: CostBiasRouter
scheduler: ResourceScheduler
broker: ResourceBroker
def build_resources() -> ResourceAuthority:
registry = Registry()
filter_ = EligibilityFilter()
router = CostBiasRouter()
scheduler = ResourceScheduler(registry, filter_, router)
broker = ResourceBroker(scheduler)
return ResourceAuthority(registry, filter_, router, scheduler, broker)
+73
View File
@@ -0,0 +1,73 @@
"""Benchmark harness and regression rule (S9 / benchmark protocol).
Run discipline per the approved protocol: five paired batches, warm-up
excluded, then 100 measured operations. Noise bound for continuous
duration metrics is max(measurement_resolution * 10, 3 * MAD) of the
paired differences; a regression fails when the median paired difference
is positive, exceeds the noise bound, and exceeds 5% of the baseline
median p95.
"""
from __future__ import annotations
import platform
import statistics
import subprocess
import sys
from collections.abc import Callable
Scenario = Callable[[int], float] # batch index -> seconds per op
def fingerprint() -> dict:
try:
rev = (
subprocess.check_output(
["git", "rev-parse", "HEAD"], stderr=subprocess.DEVNULL
)
.decode()
.strip()
)
except Exception:
rev = "unknown"
return {
"python": sys.version.split()[0],
"platform": platform.platform(),
"git": rev,
}
def run_batches(
scenario: Scenario, batches: int = 5, warmup: int = 10, ops: int = 100
) -> list[list[float]]:
out: list[list[float]] = []
for b in range(batches):
for _ in range(warmup):
scenario(b)
out.append([scenario(b) for _ in range(ops)])
return out
def p95(values: list[float]) -> float:
ordered = sorted(values)
idx = max(0, min(len(ordered) - 1, int(0.95 * len(ordered))))
return ordered[idx]
def evaluate_p95_regression(
baseline: list[float], candidate: list[float], floor_ns: float = 1_000_000.0
) -> bool:
if len(baseline) != len(candidate) or len(baseline) == 0:
raise ValueError("paired batches required")
pairs = [c - b for b, c in zip(baseline, candidate, strict=True)]
median_diff = statistics.median(pairs)
if median_diff <= 0:
return False
mad = statistics.median([abs(d - median_diff) for d in pairs])
noise = max(floor_ns / 1e9, 3 * mad)
if median_diff <= noise:
return False
base_p95 = p95(baseline)
if base_p95 <= 0:
return False
return median_diff / base_p95 > 0.05
+76
View File
@@ -0,0 +1,76 @@
"""Resource broker (S8): the single acquire path for workers.
Subagents inherit the same authority object: their budget is derived from
the parent assignment, so a subagent cannot bypass the parent's locality,
provider, concurrency, depth, or cost policy.
"""
from __future__ import annotations
from dataclasses import dataclass
from rune.orchestration.scheduler import ResourceAssignment, ResourceScheduler
@dataclass(frozen=True)
class ElasticBudget:
depth: int
concurrency: int
provider: str | None
locality: str | None
max_cost: float
def within(self, depth: int, concurrency: int, cost: float) -> bool:
return (
depth <= self.depth
and concurrency <= self.concurrency
and cost <= self.max_cost
)
class ResourceBroker:
def __init__(self, scheduler: ResourceScheduler):
self._scheduler = scheduler
def acquire(
self, task_signature: str, requirement: dict
) -> ResourceAssignment | None:
return self._scheduler.acquire(task_signature, requirement)
def inherit(
self, parent: ResourceAssignment, depth_limit: int = 3
) -> ElasticBudget:
provider = _provider_for(parent.resource_id)
return ElasticBudget(
depth=min(depth_limit, 1 + parent_task_depth(parent.task_signature)),
concurrency=parent.cap_concurrency,
provider=provider,
locality=_locality_for(parent.resource_id),
max_cost=_cost_for(parent.resource_id),
)
# Deterministic stand-ins: provider/locality/cost are looked up from the
# registry in a real build; here they come from a fixed contract map keyed
# by resource_id so the broker stays deterministic and testable.
def _provider_for(resource_id: str) -> str | None:
return _META.get(resource_id, {}).get("provider")
def _locality_for(resource_id: str) -> str | None:
return _META.get(resource_id, {}).get("locality")
def _cost_for(resource_id: str) -> float:
return _META.get(resource_id, {}).get("cost", 1.0)
def parent_task_depth(task_signature: str) -> int:
# "t1" -> 0; deeper signatures encode depth (e.g. "t1.sub3" -> 3)
return task_signature.count(".")
_META: dict[str, dict] = {
"local": {"provider": "ollama", "locality": "local", "cost": 1.0},
"cloud": {"provider": "verboo", "locality": "cloud", "cost": 2.0},
}
+46
View File
@@ -0,0 +1,46 @@
"""Identity pin budgets (S8): explicit, measured, visible overflow.
Pinning never grows without measurement. Each priority class has an
explicit byte budget; reserve() adds to the measured total and returns
False when the class OVERFLOWS — overflow is visible (pressure) and
invokes the approved strategy hook, never silent truncation.
"""
from __future__ import annotations
from collections.abc import Callable
from dataclasses import dataclass
@dataclass(frozen=True)
class PinBudget:
priority: str
max_bytes: int
class BudgetRegistry:
def __init__(self, strategy: Callable[[str, int], None] | None = None):
self._budgets: dict[str, PinBudget] = {}
self._used: dict[str, int] = {}
self._strategy = strategy or (lambda _p, _u: None)
def register(self, budget: PinBudget) -> None:
self._budgets[budget.priority] = budget
self._used.setdefault(budget.priority, 0)
def reserve(self, priority: str, item_id: str, size: int) -> bool:
if size < 0:
raise ValueError(f"size must be >= 0, got {size}")
self._used[priority] += size
if self._used[priority] <= self._budgets[priority].max_bytes:
return True
self._strategy(priority, self._used[priority])
return False
def used(self, priority: str) -> int:
return self._used.get(priority, 0)
def pressure(self) -> list[str]:
return sorted(
p for p, b in self._budgets.items() if self._used.get(p, 0) > b.max_bytes
)
+59
View File
@@ -0,0 +1,59 @@
"""Hot bus: low-latency delivery over the durable spine (S8/S9).
Every meaningful input becomes an immutable envelope, is committed to the
durable append-only store first, and only then is published to in-memory
subscribers as a compact reference. The bus is a projection of the store,
never durable completion truth on its own.
Subscriber contract: subscribers must not raise. A raising subscriber
aborts remaining delivery AFTER the durable commit; the event is never
lost, but the caller sees an exception for an already-committed event.
"""
from __future__ import annotations
from collections.abc import Callable
from rune.orchestration.event_store import DurableEventStore
from rune.orchestration.records import EventEnvelope
Subscriber = Callable[[dict], None]
class HotBus:
def __init__(self, store: DurableEventStore):
self._store = store
self._subscribers: list[Subscriber] = []
def subscribe(self, fn: Subscriber) -> None:
self._subscribers.append(fn)
def publish(self, env: EventEnvelope) -> int:
offset = self._store.append(env)
self._store.flush()
row = next(
(r for o, r in self._store.read(from_offset=offset) if o == offset), None
)
if row is None:
raise DurableEventStore.CorruptEventStoreError(
f"missing event at offset {offset}"
)
for fn in self._subscribers:
fn(dict(row))
return offset
def publish_many(self, envs: list[EventEnvelope]) -> list[int]:
start = self._store.last_offset() + 1
for env in envs:
self._store.append(env)
self._store.flush()
offsets = list(range(start, start + len(envs)))
rows = [r for _, r in self._store.read(from_offset=start)]
if len(rows) < len(envs):
raise DurableEventStore.CorruptEventStoreError(
f"missing event at offset {start + len(rows)}"
)
for row in rows:
for fn in self._subscribers:
fn(dict(row))
return offsets
+45
View File
@@ -0,0 +1,45 @@
"""Cadence configuration and durable cursor (S8).
Each cognitive cadence (N+1, reflection, synthesis) owns its own config
and cursor so that a change to one never leaks into another.
"""
from __future__ import annotations
import json
from dataclasses import dataclass
from pathlib import Path
@dataclass(frozen=True)
class CadenceConfig:
name: str
enabled: bool = True
trigger: str = "event" # event | boundary | deadline
max_proposals: int = 8
deadline_s: float | None = None
no_change_skip: bool = True
class CadenceCursor:
"""Durable per-cadence cursor: source offset + evidence fingerprint."""
def __init__(self, path: Path):
self.path = Path(path)
self._data = {"offset": 0, "fingerprint": None}
if self.path.exists():
raw = json.loads(self.path.read_text(encoding="utf-8"))
self._data.update(raw)
def offset(self) -> int:
return int(self._data["offset"])
def fingerprint(self) -> str | None:
return self._data.get("fingerprint")
def advance_to(self, offset: int, fingerprint: str) -> None:
self._data = {"offset": int(offset), "fingerprint": fingerprint}
self.path.parent.mkdir(parents=True, exist_ok=True)
self.path.write_text(
json.dumps(self._data, sort_keys=True) + "\n", encoding="utf-8"
)
+72
View File
@@ -0,0 +1,72 @@
"""Capability negotiation (design: entry-path and surface parity).
Capability-set parity is MANDATORY for production entry paths. Exceptions
require structured absence: the lost capability, owner approval, reason,
expiry, and a restoration canary. Honest absence is NOT parity and can
never make a degraded path production-equivalent.
"""
from __future__ import annotations
from dataclasses import dataclass
from rune.orchestration.records import SurfaceRecord
BASELINE_CAPABILITIES = frozenset(
{
"cancellation",
"images",
"history",
"tools",
"surfacings",
"failures",
"completion_receipts",
"telemetry",
}
)
@dataclass(frozen=True)
class NegotiatedCapability:
name: str
available: bool
owner: str = ""
reason: str = ""
expires_at: str = ""
restoration_canary: str = ""
def to_dict(self) -> dict:
return {
"name": self.name,
"available": self.available,
"owner": self.owner,
"reason": self.reason,
"expires_at": self.expires_at,
"restoration_canary": self.restoration_canary,
}
@classmethod
def from_dict(cls, data: dict) -> "NegotiatedCapability":
return cls(**data)
class CapabilityNegotiator:
def negotiate(self, surface: SurfaceRecord) -> tuple[NegotiatedCapability, ...]:
available = {c.name for c in surface.capabilities if c.available}
result: list[NegotiatedCapability] = []
for name in sorted(BASELINE_CAPABILITIES):
if name in available:
result.append(NegotiatedCapability(name=name, available=True))
else:
# Honest absence: available=False, only reason, no owner/expiry/canary.
result.append(
NegotiatedCapability(
name=name,
available=False,
reason="not provided",
)
)
return tuple(result)
def is_parity(self, negotiated: tuple[NegotiatedCapability, ...]) -> bool:
return all(n.available for n in negotiated)
+64
View File
@@ -0,0 +1,64 @@
"""Cumulative synthesis chain (C029/S8).
Each synthesis node links to its parent, so retrieving the newest node
never erases access to earlier compressed continuity: ancestors() walks
the parent links in deterministic order.
"""
from __future__ import annotations
from dataclasses import dataclass
from rune.orchestration.records import new_id
@dataclass(frozen=True)
class SynthesisNode:
node_id: str
parent_id: str | None
output_id: str
summary_ref: str
class SynthesisChain:
def __init__(self) -> None:
self._nodes: dict[str, SynthesisNode] = {}
self._order: list[str] = []
self._newest_id: str | None = None
def append(self, output_id: str, summary_ref: str) -> SynthesisNode:
parent_id = self._newest_id
node = SynthesisNode(
node_id=new_id("syn"),
parent_id=parent_id,
output_id=output_id,
summary_ref=summary_ref,
)
self._nodes[node.node_id] = node
self._order.append(node.node_id)
self._newest_id = node.node_id
return node
def newest(self) -> SynthesisNode | None:
if self._newest_id is None:
return None
return self._nodes[self._newest_id]
def get(self, node_id: str) -> SynthesisNode | None:
return self._nodes.get(node_id)
def ancestors(self, node_id: str | None = None) -> list[SynthesisNode]:
current = self._newest_id if node_id is None else node_id
out: list[SynthesisNode] = []
seen: set[str] = set()
while current is not None and current not in seen:
seen.add(current)
node = self._nodes.get(current)
if node is None:
break
out.append(node)
current = node.parent_id
return out
def all(self) -> list[SynthesisNode]:
return [self._nodes[n] for n in self._order]
+142
View File
@@ -0,0 +1,142 @@
"""Cognitive strata composition (S8).
One deterministic reflex layer plus independent, toggleable cadence
workers. Nothing here mutates canonical state: workers only emit
proposals into their own bounded sinks, and health is recorded per
cadence.
"""
from __future__ import annotations
from collections.abc import Callable
from dataclasses import dataclass
from pathlib import Path
from rune.orchestration.deadline import DeadlinePolicy
from rune.orchestration.health import CadenceHealth
from rune.orchestration.no_change import NoChangeGate, content_fingerprint
from rune.orchestration.proposals import ProposalSink
from rune.orchestration.reflex import CountReflex, ReflexEngine
from rune.orchestration.records import ProposalRecord
from rune.orchestration.workers import WorkerResult, WorkerRegistry
CADENCES = ("subconscious", "reflection", "synthesis")
class _CommitmentWorker:
"""Deterministic stand-in for a model-backed subconscious pass."""
name = "n1"
cadence = "subconscious"
def run(self, evidence, sink, gate):
fp = content_fingerprint({"evidence": evidence})
if not gate.should_run(fp):
return WorkerResult(self.name, self.cadence, 0, 1, True)
gate.record(fp)
sink.submit(ProposalRecord(worker=self.name, action="surface", confidence=0.8))
return WorkerResult(self.name, self.cadence, 1, 0, True)
class _ReflectionWorker:
name = "reflection"
cadence = "reflection"
def run(self, evidence, sink, gate):
fp = content_fingerprint({"evidence": evidence})
if not gate.should_run(fp):
return WorkerResult(self.name, self.cadence, 0, 1, True)
gate.record(fp)
sink.submit(ProposalRecord(worker=self.name, action="persist", confidence=0.6))
return WorkerResult(self.name, self.cadence, 1, 0, True)
class _SynthesisWorker:
name = "synthesis"
cadence = "synthesis"
def run(self, evidence, sink, gate):
fp = content_fingerprint({"evidence": evidence})
if not gate.should_run(fp):
return WorkerResult(self.name, self.cadence, 0, 1, True)
gate.record(fp)
sink.submit(
ProposalRecord(worker=self.name, action="synthesize", confidence=0.5)
)
return WorkerResult(self.name, self.cadence, 1, 0, True)
@dataclass
class CognitiveRuntime:
reflex: ReflexEngine
registry: WorkerRegistry
sinks: dict[str, ProposalSink]
gates: dict[str, NoChangeGate]
health: CadenceHealth
def process(self, event: dict, worker_evidence: list[dict]) -> dict:
self.reflex.process(event)
before = {cadence: sink.pending() for cadence, sink in self.sinks.items()}
results: list[WorkerResult] = []
for name in self.registry.enabled():
worker = self.registry.worker(name)
try:
result = worker.run(
worker_evidence,
self.sinks[worker.cadence],
self.gates[worker.cadence],
)
except Exception as exc:
result = WorkerResult(name, worker.cadence, 0, 0, False, str(exc))
results.append(result)
self.health.record(
result.worker,
worker.cadence,
result.ok,
result.proposals_submitted,
result.skipped,
result.error,
)
# Proposals that entered the sink during THIS run and remain pending:
# a skipped pass submits nothing, so an unchanged-evidence run reports 0.
pending = {
cadence: self.sinks[cadence].pending() - before[cadence]
for cadence in self.sinks
}
return {
"facts": self.reflex.facts(),
"results": [vars(r) for r in results],
"pending_proposals": pending,
}
@dataclass
class DeadlineRunner:
deadlines: dict[str, DeadlinePolicy]
last_runs: dict[str, float | None]
on_force: Callable[[str], None]
def tick(self, now: float) -> list[str]:
forced: list[str] = []
for name, policy in self.deadlines.items():
if policy.should_force(self.last_runs.get(name), now=now):
self.last_runs[name] = now
forced.append(name)
self.on_force(name)
return forced
def build_cognitive(base_path: Path) -> CognitiveRuntime:
registry = WorkerRegistry()
registry.register(_CommitmentWorker())
registry.register(_ReflectionWorker())
registry.register(_SynthesisWorker())
sinks = {cadence: ProposalSink() for cadence in CADENCES}
gates = {cadence: NoChangeGate() for cadence in CADENCES}
return CognitiveRuntime(
reflex=ReflexEngine(CountReflex("turns")),
registry=registry,
sinks=sinks,
gates=gates,
health=CadenceHealth(),
)
+96
View File
@@ -0,0 +1,96 @@
"""Single-authority composition root (S9).
The production composition root constructs exactly one instance of every
authority-bearing component. Nothing else may create event stores,
projections, schedulers, or registries. This module exists so the
invariant is testable, not decorative.
Only the declared AUTHORITY_KINDS may be registered. Non-authority
adapters are out of scope for this foundation plan and will be added
through an explicit extension mechanism in a later plan.
"""
from __future__ import annotations
from pathlib import Path
from rune.orchestration.event_store import DurableEventStore
from rune.orchestration.evidence_store import EvidenceStore
AUTHORITY_KINDS = (
"event_store",
"evidence_store",
"pin_registry",
"scheduler",
"resource_registry",
"resource_scheduler",
"outbox_store",
"outbox_reconciler",
"surface_registry",
"surface_gateway",
)
class DuplicateAuthorityError(RuntimeError):
pass
class CompositionRoot:
def __init__(self) -> None:
self._components: dict[str, list[object]] = {}
# A typo in an authority kind must fail fast, never bypass the invariant.
def register(self, kind: str, instance: object) -> None:
if kind not in AUTHORITY_KINDS:
raise ValueError(
f"unknown kind {kind!r}; expected one of {AUTHORITY_KINDS}"
)
self._components.setdefault(kind, []).append(instance)
def get(self, kind: str) -> object:
items = self._components.get(kind, [])
if len(items) != 1:
raise KeyError(f"expected exactly one {kind}, found {len(items)}")
return items[0]
def inventory(self) -> dict[str, list[str]]:
return {k: [type(i).__name__ for i in v] for k, v in self._components.items()}
def assert_single_authority(self) -> None:
for kind, items in self._components.items():
if kind in AUTHORITY_KINDS and len(items) > 1:
raise DuplicateAuthorityError(
f"authority kind {kind!r} has {len(items)} instances"
)
def build_root(paths: dict) -> CompositionRoot:
"""One instance of each authority kind, wired to a shared root dir."""
root = CompositionRoot()
root.register("event_store", DurableEventStore(Path(paths["events"])))
root.register("evidence_store", EvidenceStore(Path(paths["evidence"])))
return root
def compose_resources(root: CompositionRoot) -> None:
from rune.orchestration.authority import build_resources
authority = build_resources()
root.register("resource_registry", authority.registry)
root.register("resource_scheduler", authority.scheduler)
def compose_gitea_delegation(root: CompositionRoot, paths_root: Path) -> None:
from rune.orchestration.gitea_delegation import build_gitea_delegation
auth = build_gitea_delegation(Path(paths_root))
root.register("outbox_store", auth.store)
root.register("outbox_reconciler", auth.reconciler)
def compose_surface_parity(root: CompositionRoot, paths_root: Path) -> None:
from rune.orchestration.surface_gateway import build_surface_parity
authority = build_surface_parity(Path(paths_root))
root.register("surface_registry", authority.registry)
root.register("surface_gateway", authority.gateway)
+77
View File
@@ -0,0 +1,77 @@
"""Item-ID transactional synthesis cursor (A2/S8).
A synthesis transaction records exactly which input item IDs were
included, the output ID, the prior cursor, and a commit receipt, and
advances ONLY on commit. Same-day and capped-input entries cannot be
omitted or falsely marked covered: coverage is per item ID, idempotent,
and replayable from receipts.
"""
from __future__ import annotations
from collections.abc import Iterable
from dataclasses import dataclass
from rune.orchestration.records import new_id
@dataclass(frozen=True)
class SynthesisReceipt:
commit_id: str
input_ids: tuple[str, ...]
output_id: str
prior_cursor: tuple[str, ...] | None
covered: int
class SynthesisCursor:
def __init__(self) -> None:
self._covered: set[str] = set()
self._receipts: list[SynthesisReceipt] = []
# A txn tracks CANDIDATES (the begin input set) separately from
# INCLUDED (what include() added); a commit covers ONLY included.
self._pending: dict[str, dict[str, list[str]]] = {}
def begin(self, input_ids: list[str]) -> str:
txn = new_id("txn")
self._pending[txn] = {"candidates": list(input_ids), "included": []}
return txn
def include(self, txn: str, episode_id: str) -> None:
self._pending[txn]["included"].append(episode_id)
def pending(self, txn: str) -> int:
return len(self._pending.get(txn, {}).get("included", []))
def commit(self, txn: str, output_id: str) -> SynthesisReceipt:
entry = self._pending.pop(txn, {"included": []})
inputs = entry["included"] or entry.get("candidates", [])
new_items = [i for i in inputs if i not in self._covered]
self._covered.update(new_items)
prior = tuple(r.output_id for r in self._receipts) or None
receipt = SynthesisReceipt(
commit_id=new_id("cmt"),
input_ids=tuple(inputs),
output_id=output_id,
prior_cursor=prior,
covered=len(new_items),
)
self._receipts.append(receipt)
return receipt
def seed(self, episode_ids: Iterable[str]) -> None:
"""Bootstrap covered ids without minting receipts (restart recovery).
Receipt-free: rebuilds coverage from durable EpisodicStore states
so re-synthesis after a restart cannot double-cover (A2).
"""
self._covered.update(episode_ids)
def covered(self) -> set[str]:
return set(self._covered)
def receipts(self) -> list[SynthesisReceipt]:
return list(self._receipts)
def prior_cursor(self) -> tuple[str, ...] | None:
return tuple(r.output_id for r in self._receipts) or None
+31
View File
@@ -0,0 +1,31 @@
"""Deadline policy for forced cadence runs (S8).
Even with no new evidence, a cadence with a deadline must eventually run:
a forced deadline preserves each timescale's function during quiet
periods. The clock is injectable so tests are deterministic.
"""
from __future__ import annotations
import time
from collections.abc import Callable
Clock = Callable[[], float]
class DeadlinePolicy:
def __init__(self, deadline_s: float | None, clock: Clock = time.monotonic):
self._deadline_s = deadline_s
self._clock = clock
def should_force(self, last_run_ts: float | None, now: float | None = None) -> bool:
if self._deadline_s is None:
return False
current = now if now is not None else self._clock()
if last_run_ts is None:
return True
return current - last_run_ts >= self._deadline_s
def record_run(self, now: float | None = None) -> float:
current = now if now is not None else self._clock()
return current
+78
View File
@@ -0,0 +1,78 @@
"""Delegation planner (S4 item 5, S6).
Converts a started plan version + approval into an ordered set of Gitea
coordination mutations: one parent issue, then one stage issue per stage.
Mutations are deterministic (same plan + stages -> same mutation IDs and
payload hashes) so replays never duplicate accepted remote state. Queued
mutations are pending intents until the reconciler applies them.
"""
from __future__ import annotations
import hashlib
from rune.orchestration.records import ApprovalRecord, OutboxRecord
def _sha256(value: str) -> str:
return hashlib.sha256(value.encode("utf-8")).hexdigest()
ISSUE_TARGET = "gitea://rune/issues"
class DelegationPlanner:
def __init__(self, plan_version: str) -> None:
self._plan_version = plan_version
def parent_payload(self, plan_version: str) -> dict:
return {
"kind": "parent",
"title": f"[EPIC] {plan_version}",
"body": f"Orchestration stage-plan {plan_version}.",
}
def stage_payload(self, plan_version: str, stage_name: str) -> dict:
return {
"kind": "stage",
"plan_version": plan_version,
"stage_name": stage_name,
"title": f"{stage_name}: {plan_version}",
"body": f"Stage {stage_name} of {plan_version}.",
}
def _mutation(
self, ordering_key: int, remote_target: str, payload: dict
) -> OutboxRecord:
payload_hash = _sha256(str(payload))
canon = "|".join(
[self._plan_version, remote_target, str(ordering_key), payload_hash]
)
mutation_id = _sha256(canon)[:12]
return OutboxRecord(
mutation_id=mutation_id,
remote_target=remote_target,
expected_remote_version=0,
plan_version=self._plan_version,
ordering_key=ordering_key,
payload_hash=payload_hash,
)
def materialize(
self, approval: ApprovalRecord, stage_names: list[str]
) -> list[OutboxRecord]:
muts: list[OutboxRecord] = []
muts.append(
self._mutation(0, ISSUE_TARGET, self.parent_payload(self._plan_version))
)
for n, stage in enumerate(stage_names, start=1):
muts.append(
self._mutation(
n,
ISSUE_TARGET,
self.stage_payload(self._plan_version, stage),
)
)
for m in muts:
m.approval_version = approval.plan_version
return muts
+30
View File
@@ -0,0 +1,30 @@
"""Deterministic eligibility filter (S8).
Eligibility removes impossible candidates; it never ranks. Unknown health
is NEVER usable (unknown != zero). Reasons are structured so callers can
report unavailability instead of guessing.
"""
from __future__ import annotations
from rune.orchestration.resources import ResourceState, is_usable
class EligibilityFilter:
def eligible(
self, states: list[ResourceState], requirement: dict
) -> list[ResourceState]:
return [s for s in states if not self.reasons(s, requirement)]
def reasons(self, state: ResourceState, requirement: dict) -> list[str]:
out: list[str] = []
if state.healthy is None:
out.append("unknown_health")
elif not is_usable(state):
out.append("unhealthy")
if state.capabilities.kind != requirement.get("kind"):
out.append("kind_mismatch")
locality = requirement.get("locality")
if locality is not None and state.capabilities.locality != locality:
out.append("locality_mismatch")
return out
+132
View File
@@ -0,0 +1,132 @@
"""Durable append-only event store (S8/S9).
Envelopes are committed to a durable append-only JSONL store before any
fan-out. Offsets are 1-based line numbers. Writes are batched; flush()
durably commits buffered appends. Corruption is a typed error, never a
silent skip.
"""
from __future__ import annotations
import json
import os
from pathlib import Path
from typing import Iterator
from rune.orchestration.records import EventEnvelope
_ENCODING = "utf-8"
class UnmigratedDialectError(RuntimeError):
"""Raised when an old-schema dialect is handed to the new store."""
def _assert_canonical(envelope: dict) -> None:
"""Reject an old-dialect envelope dict with an actionable error.
The canonical append API takes an EventEnvelope; a dict is acceptable
only when it round-trips as one (has a stable event_id, schema_version
>= 1, and no legacy body fields). Anything else is an unmigrated old
dialect.
"""
if not isinstance(envelope, dict) or "event_id" not in envelope:
raise UnmigratedDialectError(
"old-dialect event rejected — run migrate.old_fixture_to_new() "
"on the source fixture first"
)
schema = envelope.get("schema_version")
if schema is not None and schema < 1:
raise UnmigratedDialectError(
"old-dialect event rejected (schema_version < 1) — run "
"migrate.old_fixture_to_new() on the source fixture first"
)
# Legacy body fields never appear on a canonical envelope dict.
if any(
key in envelope
for key in ("payload", "source", "channel", "authority_class", "task_id")
):
raise UnmigratedDialectError(
"old-dialect event rejected (legacy source/channel/payload "
"fields present) — run migrate.old_fixture_to_new() on the "
"source fixture first"
)
class DurableEventStore:
class CorruptEventStoreError(RuntimeError):
pass
def __init__(self, path: Path, batch_size: int = 8):
self.path = Path(path)
self.path.parent.mkdir(parents=True, exist_ok=True)
self.batch_size = max(1, batch_size)
self._buffer: list[str] = []
self._offset = self._count_existing_lines()
def _count_existing_lines(self) -> int:
if not self.path.exists():
return 0
count = 0
with open(self.path, encoding=_ENCODING) as fh:
for _ in fh:
count += 1
return count
def append(self, env: EventEnvelope | dict) -> int:
if isinstance(env, EventEnvelope):
payload = json.dumps(env.to_dict(), sort_keys=True)
else:
_assert_canonical(env)
# A dict is only acceptable if it round-trips EXACTLY as a
# canonical EventEnvelope — anything else (garbage-shaped
# "canonical" dicts) is an unmigrated dialect, never persisted.
try:
canonical = EventEnvelope.from_dict(env).to_dict()
except (TypeError, ValueError, KeyError) as exc:
raise UnmigratedDialectError(
"not a canonical event envelope — EventEnvelope "
"round-trip failed; run migrate.old_fixture_to_new() "
"on the source fixture first"
) from exc
payload = json.dumps(canonical, sort_keys=True)
self._buffer.append(payload)
if len(self._buffer) >= self.batch_size:
self.flush()
return self._offset + len(self._buffer)
def flush(self) -> None:
if not self._buffer:
return
with open(self.path, "a", encoding=_ENCODING) as fh:
for line in self._buffer:
fh.write(line + "\n")
fh.flush()
os.fsync(fh.fileno())
self._offset += len(self._buffer)
self._buffer.clear()
def last_offset(self) -> int:
return self._offset + len(self._buffer)
def read(self, from_offset: int = 1) -> Iterator[tuple[int, dict]]:
if not self.path.exists():
return
offset = 0
with open(self.path, encoding=_ENCODING) as fh:
for line in fh:
offset += 1
if offset < from_offset:
continue
line = line.strip()
if not line:
continue
try:
yield offset, json.loads(line)
except json.JSONDecodeError as exc:
raise DurableEventStore.CorruptEventStoreError(
f"corrupt event at offset {offset}: {exc}"
) from exc
def replay(self) -> list[dict]:
return [row for _, row in self.read()]
+58
View File
@@ -0,0 +1,58 @@
"""Content-addressed immutable evidence store (S8/S9).
Evidence blobs live outside the hot event path. Events and proposals carry
digests; workers resolve slices on demand. Identical bytes share one blob.
"""
from __future__ import annotations
import hashlib
import re
from pathlib import Path
_PREFIX = "sha256:"
_HEX_RE = re.compile(r"[0-9a-f]{64}")
def _valid_digest(digest: str) -> bool:
return (
digest.startswith(_PREFIX)
and _HEX_RE.fullmatch(digest[len(_PREFIX) :]) is not None
)
class EvidenceStore:
class CorruptBlobError(RuntimeError):
pass
def __init__(self, root: Path):
self.root = Path(root)
self.root.mkdir(parents=True, exist_ok=True)
def _blob_path(self, digest: str) -> Path:
return self.root / (digest[len(_PREFIX) :] + ".bin")
def put(self, data: bytes) -> str:
digest = _PREFIX + hashlib.sha256(data).hexdigest()
path = self._blob_path(digest)
if path.exists():
if path.read_bytes() != data:
raise EvidenceStore.CorruptBlobError(
f"blob exists with different content at digest {digest}"
)
return digest
path.write_bytes(data)
return digest
def contains(self, digest: str) -> bool:
return _valid_digest(digest) and self._blob_path(digest).exists()
def get(self, digest: str) -> bytes:
if not _valid_digest(digest) or not self._blob_path(digest).exists():
raise KeyError(digest)
return self._blob_path(digest).read_bytes()
def size(self, digest: str) -> int:
if not _valid_digest(digest) or not self._blob_path(digest).exists():
raise KeyError(digest)
return self._blob_path(digest).stat().st_size
+70
View File
@@ -0,0 +1,70 @@
"""Golden capability fixtures (S9 / benchmark protocol).
A canned, deterministic end-to-end chain over the durable spine. The
typed invariants are the contract: any optimization that changes them
has broken capability, regardless of speed. No real models or tools are
involved; the fixture is pure orchestration. The root must be fresh:
re-running on the same root appends on top of prior events and changes
the offsets invariant. Use distinct subdirectories for before/after runs.
"""
from __future__ import annotations
from pathlib import Path
from rune.orchestration.bus import HotBus
from rune.orchestration.event_store import DurableEventStore
from rune.orchestration.records import EventEnvelope
from rune.orchestration.selector import BatchUnit, select_next
_GRAPH = {
"b1": BatchUnit("b1", (), True),
"b2": BatchUnit("b2", ("b1",), True),
"b3": BatchUnit("b3", ("b1",), True),
"b4": BatchUnit("b4", ("b2", "b3"), True),
}
_SCOPE = {"b1", "b2", "b3", "b4"}
def run_golden_scenario(root: Path) -> dict:
store = DurableEventStore(root / "events.jsonl", batch_size=64)
bus = HotBus(store)
bus.publish(
EventEnvelope(
event_type="plan.approved",
partition_key="fixture",
actor_id="nick",
authorization="approval:plan:v1",
)
)
order: list[str] = []
verified: set[str] = set()
while True:
nxt = select_next(_GRAPH, verified, _SCOPE)
if nxt is None:
break
bus.publish(EventEnvelope(event_type="work.requested", partition_key="fixture"))
# At this foundation layer the receipt.verified event IS the receipt
# representation; the ReceiptRecord object lands with the
# verification-ledger plan in a later series.
bus.publish(
EventEnvelope(
event_type="receipt.verified",
partition_key="fixture",
payload_hash="sha256:golden",
)
)
order.append(nxt)
verified.add(nxt)
store.flush()
return {
"order": order,
"receipts": ["verified"] * len(order),
"offsets": store.last_offset(),
"selector": "lexicographic",
}
def capability_preserved(before: dict, after: dict) -> bool:
keys = ("order", "receipts", "offsets", "selector")
return all(before[k] == after[k] for k in keys)
+85
View File
@@ -0,0 +1,85 @@
"""Surface gateway (design: entry-path and surface parity).
Every production entry path (local, server, remote, TUI, voice, network)
enters through the SAME gateway: resolve the surface, verify the pinned
peer key when configured, negotiate mandatory capability parity, and bind
the external conversation to a stable internal id. Transport identity and
portable agent identity remain distinct — peer keys gate the spine.
"""
from __future__ import annotations
from dataclasses import dataclass
from rune.orchestration.capability import (
CapabilityNegotiator,
NegotiatedCapability,
)
from rune.orchestration.mapping import ConversationMapping, MappingStore
from rune.orchestration.records import SurfaceRecord, new_id
from rune.orchestration.surface import SurfaceRegistry
class PeerKeyMismatch(RuntimeError):
pass
@dataclass(frozen=True)
class SurfaceEntry:
negotiated: tuple[NegotiatedCapability, ...]
mapping: ConversationMapping
peer_verified: bool
def generate_internal_id() -> str:
return new_id("conv")
class SurfaceGateway:
def __init__(
self,
registry: SurfaceRegistry,
negotiator: CapabilityNegotiator,
mapping_store: MappingStore,
) -> None:
self.registry = registry
self.negotiator = negotiator
self.mapping_store = mapping_store
def enter(
self,
external_id: str,
external_thread: str = "",
surface_kind: str = "local",
peer_key: str = "",
) -> SurfaceEntry:
surface = self.registry.by_external(external_id)
if surface is None:
surface = SurfaceRecord(
surface_id=new_id("sfc"),
external_id=external_id,
kind=surface_kind,
peer_key=peer_key,
)
self.registry.register(surface)
if surface.peer_key and surface.peer_key != peer_key:
raise PeerKeyMismatch(f"surface {external_id}: peer key mismatch")
peer_verified = True
negotiated = self.negotiator.negotiate(surface)
mapping = self.mapping_store.by_external(external_id, external_thread)
if mapping is None:
mapping = ConversationMapping(
external_id=external_id,
external_thread=external_thread,
surface_id=surface.surface_id,
internal_conversation_id=generate_internal_id(),
)
self.mapping_store.link(mapping)
# Re-read the STORED mapping so SurfaceEntry carries the
# persisted created_at (link() fills it on first write).
mapping = self.mapping_store.by_external(external_id, external_thread)
return SurfaceEntry(
negotiated=negotiated,
mapping=mapping,
peer_verified=peer_verified,
)
@@ -0,0 +1,66 @@
"""Gitea delegation composition (S4 item 5/10/17, S9).
One outbox store, one lifecycle, one planner; the reconciler is bound per
delegate() call (the per-call transport decides whether mutations reach
Gitea). Builds stay offline-safe: no network transport at build time.
"""
from __future__ import annotations
from dataclasses import dataclass, replace
from pathlib import Path
from rune.orchestration.delegation import DelegationPlanner
from rune.orchestration.lifecycle import OutboxLifecycle
from rune.orchestration.outbox import OutboxStore
from rune.orchestration.reconciler import Reconciler
from rune.orchestration.records import ApprovalRecord, OutboxRecord
@dataclass
class GiteaDelegationAuthority:
store: OutboxStore
lifecycle: OutboxLifecycle
planner: DelegationPlanner
reconciler: Reconciler
def delegate(
self,
approval: ApprovalRecord,
stages: list[str],
transport,
resolver: str = "",
) -> list[OutboxRecord]:
# VALIDATION FIRST (G3 quality pins): the S6 accuracy-sized-batch
# guarantee is only real when every delegated stage is WITHIN the
# approved set and the approval is not revoked. Fail loud before
# ANY outbox write. Subset delegation is intentional (S4 item 5:
# a batch may cover part of the approved plan).
if approval.revoked:
raise ValueError(f"approval {approval.plan_version} is revoked")
unapproved = [s for s in stages if s not in approval.approved_items]
if unapproved:
raise ValueError(f"stages not in approved_items: {unapproved}")
# The planner is bound to the approval's plan version, not the
# build-time default (the authority builds offline).
planner = DelegationPlanner(approval.plan_version)
mutations = planner.materialize(approval, stages)
for m in mutations:
self.store.append(m)
self.reconciler = Reconciler(
self.store, self.lifecycle, transport, resolver=resolver
)
self.reconciler.reconcile()
# Return COPIES (G2 quality pin): callers must not be able to
# mutate stored state without a mark().
return [replace(self.store.get(m.mutation_id)) for m in mutations]
def build_gitea_delegation(root: Path) -> GiteaDelegationAuthority:
store = OutboxStore(root / "gitea_outbox")
lifecycle = OutboxLifecycle()
planner = DelegationPlanner("")
reconciler = Reconciler(store, lifecycle, None)
return GiteaDelegationAuthority(
store=store, lifecycle=lifecycle, planner=planner, reconciler=reconciler
)
+56
View File
@@ -0,0 +1,56 @@
"""Cadence health ledger (S8).
Each cadence keeps its own run/failure/proposal accounting. One cadence
failing never erases or blocks the health of another.
"""
from __future__ import annotations
from dataclasses import dataclass
@dataclass
class _CadenceStats:
runs: int = 0
failures: int = 0
proposals: int = 0
skipped: int = 0
last_error: str = ""
class CadenceHealth:
def __init__(self) -> None:
self._stats: dict[str, _CadenceStats] = {}
def record(
self,
worker: str,
cadence: str,
ok: bool,
proposals: int = 0,
skipped: int = 0,
error: str = "",
) -> None:
stats = self._stats.setdefault(cadence, _CadenceStats())
stats.runs += 1
stats.proposals += proposals
stats.skipped += skipped
if not ok:
stats.failures += 1
stats.last_error = error
def status(self, cadence: str) -> dict:
stats = self._stats.get(cadence, _CadenceStats())
return {
"runs": stats.runs,
"failures": stats.failures,
"proposals": stats.proposals,
"skipped": stats.skipped,
"last_error": stats.last_error,
}
def all_status(self) -> dict[str, dict]:
return {name: self.status(name) for name in self._stats}
def failed_cadences(self) -> list[str]:
return sorted(n for n, s in self._stats.items() if s.failures > 0)
+49
View File
@@ -0,0 +1,49 @@
"""Outbox lifecycle state machine (S9/S4 item 10).
queued -> sending -> accepted (happy path); sending may also land in
retryable_failure, conflict, superseded, or cancelled; retryable_failure
returns to sending for bounded retries. Terminal states never advance.
Advance returns a NEW record; the store applies it via mark().
"""
from __future__ import annotations
from dataclasses import replace
from rune.orchestration.records import OutboxRecord
_TRANSITIONS: dict[str, frozenset[str]] = {
"queued": frozenset({"sending", "cancelled"}),
"sending": frozenset(
{
"sending", # crash recovery: retry an interrupted apply
"accepted",
"retryable_failure",
"conflict",
"superseded",
"cancelled",
}
),
"retryable_failure": frozenset({"sending", "cancelled", "superseded"}),
"accepted": frozenset(),
"conflict": frozenset(),
"cancelled": frozenset(),
"superseded": frozenset(),
}
class InvalidTransition(RuntimeError):
pass
class OutboxLifecycle:
def can(self, record_state: str, next_state: str) -> bool:
return next_state in _TRANSITIONS.get(record_state, frozenset())
def advance(self, record: OutboxRecord, next_state: str) -> OutboxRecord:
if not self.can(record.state, next_state):
raise InvalidTransition(f"{record.state} -> {next_state}")
attempts = record.attempts
if next_state in ("sending", "retryable_failure"):
attempts += 1
return replace(record, state=next_state, attempts=attempts)
+105
View File
@@ -0,0 +1,105 @@
"""External-to-internal conversation mapping (design: surface parity).
Binds an external room/device/peer to an internal conversation. The
mapping is bidirectional and DURABLE so a response can return to the
originating surface. A conversation can never be silently rebound to a
different internal id.
"""
from __future__ import annotations
import json
from dataclasses import dataclass
from datetime import datetime, timezone
from pathlib import Path
from rune.orchestration.atomic import AtomicStore
def _now() -> str:
return datetime.now(timezone.utc).isoformat()
@dataclass(frozen=True)
class ConversationMapping:
external_id: str
external_thread: str
surface_id: str
internal_conversation_id: str = ""
created_at: str = ""
def to_dict(self) -> dict:
return {
"external_id": self.external_id,
"external_thread": self.external_thread,
"surface_id": self.surface_id,
"internal_conversation_id": self.internal_conversation_id,
"created_at": self.created_at,
}
@classmethod
def from_dict(cls, data: dict) -> "ConversationMapping":
return cls(**data)
class CorruptMappingError(RuntimeError):
"""Raised when the on-disk mappings file cannot be parsed."""
class MappingStore:
def __init__(self, root: Path) -> None:
self._root = Path(root)
self._atomic = AtomicStore(self._root / "state")
self._mappings: list[ConversationMapping] = []
self._load()
def _load(self) -> None:
raw = self._atomic.read("mappings.json")
if raw is None:
return
try:
items = json.loads(raw)
except json.JSONDecodeError as exc:
raise CorruptMappingError(f"mappings corrupt: {exc}") from exc
if not isinstance(items, list):
raise CorruptMappingError("mappings corrupt: expected a list")
for item in items:
try:
mapping = ConversationMapping.from_dict(item)
except (TypeError, ValueError) as exc:
raise CorruptMappingError(f"mappings corrupt item: {exc}") from exc
self._mappings.append(mapping)
def _write(self) -> None:
self._atomic.write(
"mappings.json",
json.dumps([m.to_dict() for m in self._mappings], sort_keys=True),
)
def link(self, mapping: ConversationMapping) -> None:
existing = self.by_external(mapping.external_id, mapping.external_thread)
if existing is not None:
if existing.internal_conversation_id != mapping.internal_conversation_id:
raise ValueError(
f"external {mapping.external_id}/{mapping.external_thread} "
f"already bound to {existing.internal_conversation_id!r}"
)
return # idempotent re-link
if not mapping.created_at:
mapping = ConversationMapping(**{**mapping.to_dict(), "created_at": _now()})
self._mappings.append(mapping)
self._write()
def by_external(
self, external_id: str, external_thread: str
) -> ConversationMapping | None:
for m in self._mappings:
if m.external_id == external_id and m.external_thread == external_thread:
return m
return None
def by_internal(self, internal_id: str) -> list[ConversationMapping]:
return [m for m in self._mappings if m.internal_conversation_id == internal_id]
def all(self) -> list[ConversationMapping]:
return list(self._mappings)
+92
View File
@@ -0,0 +1,92 @@
"""Episodic memory store (S8): stable IDs, explicit synthesis states.
Episodic records are the raw material of synthesis. Each record has a
stable ID and an explicit synthesis_state ("raw" until a synthesis
transaction includes it). Persistence is JSONL with fsync on write.
"""
from __future__ import annotations
import json
import os
from dataclasses import dataclass
from pathlib import Path
from rune.orchestration.records import new_id
@dataclass(frozen=True)
class EpisodicRecord:
episode_id: str
kind: str
evidence_refs: tuple[str, ...]
created_at: str
synthesis_state: str = "raw"
class EpisodicStore:
def __init__(self, path: Path):
self.path = Path(path)
self.path.parent.mkdir(parents=True, exist_ok=True)
self._records: dict[str, EpisodicRecord] = {}
if self.path.exists():
for line in self.path.read_text(encoding="utf-8").splitlines():
if not line.strip():
continue
data = json.loads(line)
self._records[data["episode_id"]] = EpisodicRecord(
episode_id=data["episode_id"],
kind=data["kind"],
evidence_refs=tuple(data["evidence_refs"]),
created_at=data["created_at"],
synthesis_state=data["synthesis_state"],
)
def _write(self) -> None:
with open(self.path, "w", encoding="utf-8") as fh:
for rec in self._records.values():
fh.write(
json.dumps(
dict(
episode_id=rec.episode_id,
kind=rec.kind,
evidence_refs=list(rec.evidence_refs),
created_at=rec.created_at,
synthesis_state=rec.synthesis_state,
),
sort_keys=True,
)
+ "\n"
)
fh.flush()
os.fsync(fh.fileno())
def append(self, record: EpisodicRecord) -> EpisodicRecord:
stored = EpisodicRecord(
episode_id=record.episode_id or new_id("ep"),
kind=record.kind,
evidence_refs=record.evidence_refs,
created_at=record.created_at,
synthesis_state=record.synthesis_state,
)
self._records[stored.episode_id] = stored
self._write()
return stored
def get(self, episode_id: str) -> EpisodicRecord | None:
return self._records.get(episode_id)
def all(self) -> list[EpisodicRecord]:
return list(self._records.values())
def mark_synthesized(self, episode_id: str, state: str) -> None:
rec = self._records[episode_id]
updated = EpisodicRecord(
episode_id=rec.episode_id,
kind=rec.kind,
evidence_refs=rec.evidence_refs,
created_at=rec.created_at,
synthesis_state=state,
)
self._records[episode_id] = updated
self._write()
@@ -0,0 +1,83 @@
"""Memory and synthesis composition (S8).
One episodic store, one item-ID synthesis cursor, one cumulative chain,
one budget registry, and atomic hot-state writes. synthesize() drives an
episode set through the transactional cursor and links the output into
the cumulative chain.
"""
from __future__ import annotations
from collections.abc import Callable
from dataclasses import dataclass
from pathlib import Path
from rune.orchestration.atomic import AtomicStore
from rune.orchestration.budget import BudgetRegistry
from rune.orchestration.chain import SynthesisChain
from rune.orchestration.cursor import SynthesisCursor, SynthesisReceipt
from rune.orchestration.memory import EpisodicStore
from rune.orchestration.records import new_id
@dataclass
class MemorySynthesisAuthority:
episodic: EpisodicStore
cursor: SynthesisCursor
chain: SynthesisChain
budget: BudgetRegistry
atomic: AtomicStore
def synthesize(self, episodes: list[str], summary_ref: str) -> SynthesisReceipt:
"""Drive an episode set through the transactional cursor.
Validates every episode id against the episodic store BEFORE any
state change (M1 P1 discharge): an unknown id raises KeyError with
no cursor txn, no chain node, no mark_synthesized — no partial
commit (M6 quality I1). Input ids are deduplicated
order-preserving so a duplicate id can never double-count coverage
(M6 quality I2). Durable marks happen FIRST — the episodic store is
the only durable source — then the in-memory commit and chain
append, so a mark failure leaves no coverage advance and no chain
node; already-marked episodes are idempotent on retry.
"""
unique: list[str] = list(dict.fromkeys(episodes))
for ep in unique:
if self.episodic.get(ep) is None:
raise KeyError(ep)
for ep in unique:
self.episodic.mark_synthesized(ep, "synthesized")
txn = self.cursor.begin(unique)
for ep in unique:
self.cursor.include(txn, ep)
output_id = new_id("out")
receipt = self.cursor.commit(txn, output_id)
self.chain.append(output_id, summary_ref)
return receipt
def build_memory_synthesis(
root: Path, budget_strategy: Callable[[str, int], None] | None = None
) -> MemorySynthesisAuthority:
"""Build the memory-synthesis authority.
Bootstraps cursor coverage from the episodic store's durable
synthesis states (M1 P1 #2 / M2 Imp #1 discharge): episodes already
marked "synthesized" are re-seeded into the fresh cursor so the A2
no-false-coverage claim survives restarts. budget_strategy is wired
into the budget registry (M4 Imp #1/#2 decision: per-reserve cadence —
the hook fires on every overflowing reserve; side-effect strategies
(eviction) are the sanctioned use).
"""
episodic = EpisodicStore(root / "episodes.jsonl")
cursor = SynthesisCursor()
cursor.seed(
ep.episode_id for ep in episodic.all() if ep.synthesis_state == "synthesized"
)
return MemorySynthesisAuthority(
episodic=episodic,
cursor=cursor,
chain=SynthesisChain(),
budget=BudgetRegistry(strategy=budget_strategy),
atomic=AtomicStore(root / "state"),
)
+207
View File
@@ -0,0 +1,207 @@
"""Old-schema fixture migration (design: schema migration).
Converts a JSONL fixture written in the OLD event dialect into the canonical
orchestration EventEnvelope schema. Idempotent by event_id: a re-run converts
only the not-yet-converted entries, so 'old fixture migrates once' holds even
after a partial or interrupted migration. Each migrated raw line is ALSO
written to the EvidenceStore keyed by its payload_hash (1:1), so a
store-only consumer can recover the body.
"""
from __future__ import annotations
import json
from pathlib import Path
from rune.orchestration.evidence_store import EvidenceStore
from rune.orchestration.event_store import DurableEventStore
from rune.orchestration.records import EventEnvelope, PinRecord
def old_fixture_to_new(path: Path, store_dir: Path | None = None) -> int:
"""Migrate an old-dialect JSONL fixture to canonical records.
Returns the number of NEW canonical records appended this run
(idempotent by event_id — a clean re-run returns 0). Malformed lines
are counted and surfaced by ``old_fixture_stats``; the CLI reports
both. Each migrated raw line is ALSO written to the EvidenceStore
keyed by its payload_hash (1:1), so a store-only consumer can recover
the body. The event store is FLUSHED per record so a crash never
leaves evidence blobs for events that were never appended
(evidence-orphan window closed — X5 quality discharge I-3).
"""
converted, _skipped = old_fixture_stats(path, store_dir)
return converted
def old_fixture_stats(path: Path, store_dir: Path | None = None) -> tuple[int, int]:
"""Migrate and report ``(converted, skipped)`` — X5 discharge I-2.
Skipped = fixture lines dropped THIS RUN (malformed JSON or non-dict;
already-migrated event_ids are silently idempotent, not counted).
The CLI surfaces the skipped count so a migration can never silently
drop data.
"""
fixture = Path(path)
events_dir = Path(store_dir) if store_dir is not None else fixture.parent / "events"
store = DurableEventStore(events_dir)
# X1 pin discharge: the raw migrated body is ALSO written to the
# EvidenceStore keyed by its payload_hash (digest == payload_hash — 1:1
# link), so a store-only consumer can recover the body.
evidence = EvidenceStore(events_dir.parent / "evidence")
existing = {r["event_id"] for r in store.replay()}
converted = 0
skipped = 0
with open(fixture, "r", encoding="utf-8") as fh:
for line in fh:
line = line.strip()
if not line:
continue
try:
raw = json.loads(line)
except json.JSONDecodeError:
skipped += 1
continue
if not isinstance(raw, dict):
skipped += 1
continue
event_id = raw.get("event_id", raw.get("id", ""))
if not event_id or event_id in existing:
continue
payload_hash = _content_hash(raw)
envelope = EventEnvelope(
event_id=event_id,
event_type=raw.get("event_type", raw.get("type", "unknown")),
partition_key=_partition_key(raw),
timestamp=raw.get("timestamp", ""),
payload_hash=payload_hash,
)
# Evidence FIRST, then append + FLUSH per record: a crash after
# the evidence write still leaves the event durable in the same
# instant — no orphan window (X5 discharge I-3).
digest = evidence.put(json.dumps(raw, sort_keys=True).encode("utf-8"))
if digest != payload_hash:
raise RuntimeError(
"evidence digest does not match payload_hash — the "
"evidence key and the event record disagree"
)
store.append(envelope)
store.flush()
existing.add(event_id)
converted += 1
return converted, skipped
def _content_hash(raw: dict) -> str:
# Content-addressed: the old payload/body is hashed into payload_hash
# (the canonical envelope carries no payload body; the old content is
# recoverable via the EvidenceStore keyed by this hash). Deterministic
# across runs.
import hashlib
return (
"sha256:"
+ hashlib.sha256(json.dumps(raw, sort_keys=True).encode("utf-8")).hexdigest()
)
def _partition_key(raw: dict) -> str:
"""Extract the canonical partition_key from an old-dialect event.
Real event_bus data nests the channel INSIDE source
(source='{"channel": "chat"}' or a dict) — without extraction the
migration would silently repartition real data to 'conversation'.
"""
channel = raw.get("channel")
if channel:
return channel
source = raw.get("source")
if isinstance(source, dict):
return source.get("channel") or "conversation"
if isinstance(source, str):
try:
parsed = json.loads(source)
except (json.JSONDecodeError, TypeError):
return "conversation"
if isinstance(parsed, dict):
return parsed.get("channel") or "conversation"
return "conversation"
def legacy_pins_to_new(legacy_db: Path, out_json: Path) -> tuple[int, int]:
"""Convert legacy SQLite wing='pins' chunks to canonical PinRecords.
X4 C7 discharge: the legacy hybrid-store writer is gone, so existing
pins live in a SQLite ``chunks`` table (text/wing/room columns) as
multi-line text blobs. Reads the wing='pins' chunks, parses the
[topic]/[open question]/[outcome]/[status]/[context] format and writes a
JSON array of canonical PinRecord dicts (kind='pin', item_id=topic,
owner='pins', state=open|resolved, details=JSON blob) to ``out_json``.
Data-protective policy (X5 quality discharge I-1/I-2): a chunk WITHOUT
a parseable ``[topic]`` OR without an explicit ``[status]`` line is
SKIPPED, never guessed — a resolved pin missing its status line must
not silently re-surface as open. Returns ``(converted, skipped)`` so
the CLI can surface dropped chunks; a migration never silently loses
data.
"""
import sqlite3
db = sqlite3.connect(str(legacy_db))
try:
rows = db.execute(
"SELECT text FROM chunks WHERE wing = ?", ("pins",)
).fetchall()
finally:
db.close()
records = []
skipped = 0
for (text,) in rows:
fields = _parse_pin_text(text or "")
topic = fields.get("topic", "").strip()
status = fields.get("status", "").strip().lower()
if not topic or status not in ("open", "resolved"):
skipped += 1
continue
records.append(
PinRecord(
kind="pin",
item_id=topic,
owner="pins",
state=status,
details=json.dumps(
{
"question": fields.get("open question", ""),
"outcome": fields.get("outcome", ""),
"context": fields.get("context", ""),
"reason": fields.get("resolved reason", ""),
"source": fields.get("source", "legacy-pins-migration"),
},
sort_keys=True,
),
).to_dict()
)
out = Path(out_json)
out.parent.mkdir(parents=True, exist_ok=True)
out.write_text(json.dumps(records, indent=2))
return len(records), skipped
def _parse_pin_text(text: str) -> dict:
"""Parse a legacy wing='pins' chunk text blob into its fields.
The legacy writer stored each pin as a multi-line ``[key] value`` blob
([topic]/[open question]/[outcome]/[status]/[context]/[source]; unpin
adds [resolved reason]).
"""
fields = {}
for line in text.splitlines():
if not line.startswith("["):
continue
end = line.find("]")
if end < 0:
continue
fields[line[1:end].strip()] = line[end + 1 :].strip()
return fields
+71
View File
@@ -0,0 +1,71 @@
"""Migration CLI (design: schema migration).
Usage:
python -m rune.orchestration.migrate_cli <fixture> [--store DIR]
python -m rune.orchestration.migrate_cli --pins <legacy.db> <out.json>
The fixture form runs old_fixture_to_new (idempotent by event_id) and prints
'MIGRATED N'. The --pins form converts legacy SQLite wing='pins' chunks to a
canonical PinRecord JSON array and prints 'PINS MIGRATED N'. Both exit 0 on
success. The CLI works with only ``/usr/bin:/bin`` on PATH — the subprocess
invokes python via sys.executable, never the shell.
"""
from __future__ import annotations
import argparse
import json
import sys
from pathlib import Path
from rune.orchestration.migrate import legacy_pins_to_new, old_fixture_stats
def main(argv=None) -> int:
ap = argparse.ArgumentParser(
prog="python -m rune.orchestration.migrate_cli",
description=(
"Migrate an old-dialect fixture to the canonical store, or "
"convert legacy SQLite wing='pins' chunks to canonical "
"PinRecords."
),
)
ap.add_argument("fixture", nargs="?", help="old-dialect JSONL fixture to migrate")
ap.add_argument(
"--store",
default=None,
help="event store directory (default: <fixture dir>/events)",
)
ap.add_argument(
"--pins",
nargs=2,
metavar=("LEGACY_DB", "OUT_JSON"),
help="convert legacy SQLite wing='pins' chunks to canonical PinRecords",
)
args = ap.parse_args(argv)
if args.pins:
legacy_db, out_json = args.pins
n, skipped = legacy_pins_to_new(Path(legacy_db), Path(out_json))
print(f"PINS MIGRATED {n} SKIPPED {skipped}")
return 0
if not args.fixture:
ap.error("a fixture path is required (or use --pins <legacy.db> <out.json>)")
try:
n, skipped = old_fixture_stats(
Path(args.fixture), store_dir=Path(args.store) if args.store else None
)
except (json.JSONDecodeError, AttributeError, TypeError, ValueError) as exc:
# X1 Minor 5/6: a malformed / non-dict fixture line surfaces as a
# raw AttributeError — wrap it so the CLI fails with a readable
# message instead of a traceback.
print(f"error: failed to migrate {args.fixture}: {exc}", file=sys.stderr)
return 1
print(f"MIGRATED {n} SKIPPED {skipped}")
return 0
if __name__ == "__main__":
sys.exit(main())
+31
View File
@@ -0,0 +1,31 @@
"""Content fingerprint and deterministic no-change gate (S8).
Unchanged evidence must not trigger a model-bound pass; but a changed
fingerprint must never be skipped. Each cadence owns its own gate so a
skip in one never affects another.
"""
from __future__ import annotations
import hashlib
import json
def content_fingerprint(payload: dict) -> str:
canonical = json.dumps(payload, sort_keys=True, separators=(",", ":"))
return "sha256:" + hashlib.sha256(canonical.encode("utf-8")).hexdigest()
class NoChangeGate:
def __init__(self) -> None:
self._last: str | None = None
def should_run(self, fingerprint: str | None) -> bool:
# None means "no evidence seen yet" — never skip a first run in the
# unsafe direction (CadenceCursor.fingerprint() can return None).
if fingerprint is None:
return True
return fingerprint != self._last
def record(self, fingerprint: str) -> None:
self._last = fingerprint
+105
View File
@@ -0,0 +1,105 @@
"""Append-only outbox with stable IDs and atomic persistence (S4/ S9).
Queued mutations are pending intents, never accepted remote state.
Durability via AtomicStore (temp + fsync + rename). Corruption raises,
it is never silently skipped.
"""
from __future__ import annotations
import json
from pathlib import Path
from rune.orchestration.atomic import AtomicStore
from rune.orchestration.records import OutboxRecord
class CorruptOutboxError(RuntimeError):
"""Raised when the on-disk outbox cannot be parsed."""
_STATES = frozenset(
{
"queued",
"sending",
"retryable_failure",
"accepted",
"conflict",
"cancelled",
"superseded",
}
)
class OutboxStore:
def __init__(self, root: Path) -> None:
self._root = Path(root)
self._atomic = AtomicStore(self._root / "state")
self._records: dict[str, OutboxRecord] = {}
self._load()
def _load(self) -> None:
raw = self._atomic.read("outbox.json")
if raw is None:
return
try:
items = json.loads(raw)
except json.JSONDecodeError as exc:
raise CorruptOutboxError(f"outbox corrupt: {exc}") from exc
if not isinstance(items, list):
raise CorruptOutboxError("outbox corrupt: expected a list")
for item in items:
try:
rec = OutboxRecord.from_dict(item)
except (TypeError, ValueError) as exc:
raise CorruptOutboxError(f"outbox corrupt: {exc}") from exc
if not isinstance(rec.state, str) or rec.state not in _STATES:
raise CorruptOutboxError(
f"outbox corrupt: {rec.mutation_id!r} state {rec.state!r} is not a valid lifecycle state"
)
if not isinstance(rec.ordering_key, int):
raise CorruptOutboxError(
f"outbox corrupt: {rec.mutation_id!r} ordering_key {rec.ordering_key!r} is not an int"
)
if not isinstance(rec.expected_remote_version, int):
raise CorruptOutboxError(
f"outbox corrupt: {rec.mutation_id!r} expected_remote_version "
f"{rec.expected_remote_version!r} is not an int"
)
self._records[rec.mutation_id] = rec
def _write(self) -> None:
payload = json.dumps(
[r.to_dict() for r in self._records.values()],
sort_keys=True,
)
self._atomic.write("outbox.json", payload)
def append(self, record: OutboxRecord) -> None:
if record.mutation_id in self._records:
return # idempotent: an existing ID is a no-op
self._records[record.mutation_id] = record
self._write()
def get(self, mutation_id: str) -> OutboxRecord | None:
return self._records.get(mutation_id)
def pending(self) -> list[OutboxRecord]:
return [
r
for r in sorted(
self._records.values(),
key=lambda r: (r.ordering_key, r.mutation_id),
)
if r.state in {"queued", "sending", "retryable_failure"}
]
def all(self) -> list[OutboxRecord]:
return list(self._records.values())
def mark(self, record: OutboxRecord) -> OutboxRecord:
if record.mutation_id not in self._records:
raise KeyError(record.mutation_id)
self._records[record.mutation_id] = record
self._write()
return self._records[record.mutation_id]
+66
View File
@@ -0,0 +1,66 @@
"""Incremental materialized-view projections (S8/S9).
State is a checkpointed projection over durable events. Normal reads are
constant-size lookups into the current state; replay is the recovery
mechanism, not the price of every screen refresh. State hash is
order-insensitive JSON hashing so any dict ordering produces the same hash.
"""
from __future__ import annotations
import hashlib
import json
from collections.abc import Callable
from rune.orchestration.records import ProjectionCheckpoint
Reducer = Callable[[dict, dict], dict]
def state_hash(state: dict) -> str:
canonical = json.dumps(state, sort_keys=True, separators=(",", ":"))
return "sha256:" + hashlib.sha256(canonical.encode("utf-8")).hexdigest()
class ProjectionEngine:
class ProjectionCorruptError(RuntimeError):
pass
def __init__(self, reducer: Reducer):
self._reducer = reducer
self._state: dict = {}
self._offset = 0
def apply(self, event: dict) -> None:
self._state = self._reducer(self._state, dict(event))
self._offset += 1
def state(self) -> dict:
return dict(self._state)
def checkpoint(self) -> ProjectionCheckpoint:
return ProjectionCheckpoint(
offset=self._offset, state_hash=state_hash(self._state)
)
def rebuild(
self, events: list[dict], checkpoint: ProjectionCheckpoint | None
) -> dict:
if checkpoint is None:
return self._reducer_bulk(events)
if checkpoint.offset > len(events):
raise self.ProjectionCorruptError(
f"checkpoint offset {checkpoint.offset} exceeds event log length {len(events)}"
)
prefix_state = self._reducer_bulk(events[: checkpoint.offset])
if state_hash(prefix_state) != checkpoint.state_hash:
raise self.ProjectionCorruptError(
"checkpoint state hash does not match replayed event prefix"
)
return self._reducer_bulk(events[checkpoint.offset :], prefix_state)
def _reducer_bulk(self, events: list[dict], state: dict | None = None) -> dict:
out = dict(state or {})
for ev in events:
out = self._reducer(out, dict(ev))
return out
+33
View File
@@ -0,0 +1,33 @@
"""Bounded subconscious proposal sink (S8).
Workers emit proposals; they do not mutate canonical state. The sink
bounds the queue and dead-letters overflow so nothing is silently lost.
"""
from __future__ import annotations
from rune.orchestration.records import ProposalRecord
class ProposalSink:
def __init__(self, capacity: int = 8):
self._capacity = max(1, capacity)
self._queue: list[ProposalRecord] = []
self._dead: list[ProposalRecord] = []
def submit(self, proposal: ProposalRecord) -> bool:
if len(self._queue) >= self._capacity:
self._dead.append(proposal)
return False
self._queue.append(proposal)
return True
def drain(self) -> list[ProposalRecord]:
out, self._queue = self._queue, []
return out
def pending(self) -> int:
return len(self._queue)
def dead_letter(self) -> list[ProposalRecord]:
return list(self._dead)
+79
View File
@@ -0,0 +1,79 @@
"""Outbox reconciler (S4 item 10, S6).
Maps transport outcomes to durable outbox state. The transport owns the
reconnection version comparison (a newer remote version is NEVER overwritten
— it is reported as CONFLICT); the reconciler owns durability: accepted,
conflict, cancelled, and retryable_failure all land in the store so a crash
mid-cycle never loses intent.
CONTRACT for transports: a record is marked ``sending`` (durable) BEFORE
``transport.apply`` is called; a crash between the two leaves the record in
``sending`` and a later reconcile() re-advances it (lifecycle self-loop), so
``apply`` may be called TWICE with the same mutation_id. Transports MUST be
idempotent by mutation_id (replaying an already-applied payload reports
ALREADY_APPLIED) and MUST report their own transient failures as RETRYABLE —
raising from apply() is a contract violation that aborts reconcile().
"""
from __future__ import annotations
from dataclasses import replace
from enum import Enum
from rune.orchestration.lifecycle import OutboxLifecycle
from rune.orchestration.outbox import OutboxStore
from rune.orchestration.records import OutboxRecord
class ApplyResult(Enum):
ACCEPTED = "accepted"
ALREADY_APPLIED = "already_applied"
CONFLICT = "conflict"
BLOCKED = "blocked"
RETRYABLE = "retryable"
class Reconciler:
def __init__(
self,
store: OutboxStore,
lifecycle: OutboxLifecycle,
transport,
resolver: str = "",
) -> None:
self._store = store
self._lifecycle = lifecycle
self._transport = transport
self._resolver = resolver
def reconcile(self, limit: int | None = None) -> int:
processed = 0
for record in self._store.pending():
if limit is not None and processed >= limit:
break
self._apply(record)
processed += 1
return processed
def _apply(self, record: OutboxRecord) -> None:
in_flight = self._store.mark(self._lifecycle.advance(record, "sending"))
result = self._transport.apply(in_flight)
if result is ApplyResult.CONFLICT:
self._store.mark(
replace(
self._lifecycle.advance(in_flight, "conflict"),
resolver=self._resolver,
)
)
elif result is ApplyResult.BLOCKED:
self._store.mark(self._lifecycle.advance(in_flight, "cancelled"))
elif result is ApplyResult.RETRYABLE:
self._store.mark(self._lifecycle.advance(in_flight, "retryable_failure"))
else: # ACCEPTED / ALREADY_APPLIED
self._store.mark(
replace(
self._lifecycle.advance(in_flight, "accepted"),
returned_remote_id=self._transport.remote_id(in_flight),
returned_remote_version=self._transport.remote_version(),
)
)
+227
View File
@@ -0,0 +1,227 @@
"""Canonical orchestration records (spec S9).
Every record carries a stable ID and schema_version. Records are the only
shape allowed across component boundaries; dicts are for serialization.
"""
from __future__ import annotations
import uuid
from dataclasses import asdict, dataclass, field
from datetime import datetime, timezone
SCHEMA_VERSION = 1
_LEN = 12
def new_id(prefix: str) -> str:
return f"{prefix}_{uuid.uuid4().hex[:_LEN]}"
def _now() -> str:
return datetime.now(timezone.utc).isoformat()
_TUPLE_FIELDS: dict[type, frozenset[str]] = {}
def _tuple_fields(cls) -> frozenset[str]:
return _TUPLE_FIELDS.get(cls, frozenset())
@dataclass
class OrchestrationRecord:
"""Base: schema versioning and dict round-trip."""
schema_version: int = field(default=SCHEMA_VERSION, init=False)
def to_dict(self) -> dict:
data = asdict(self)
data["schema_version"] = self.schema_version
return data
@classmethod
def from_dict(cls, data: dict):
data = dict(data)
data.pop("schema_version", None)
for _field in _tuple_fields(cls):
if _field in data and data[_field] is not None:
data[_field] = tuple(data[_field])
return cls(**data)
@dataclass
class EventEnvelope(OrchestrationRecord):
"""Immutable typed envelope committed before fan-out (S8 spine)."""
event_type: str
partition_key: str
event_id: str = field(default_factory=lambda: new_id("evt"))
parent_id: str | None = None
actor_id: str = ""
device_id: str = ""
authorization: str = ""
payload_hash: str = ""
timestamp: str = field(default_factory=_now)
@dataclass
class EvidenceCapsule(OrchestrationRecord):
"""Content-addressed immutable evidence reference."""
digest: str
size: int
source: str = ""
@dataclass
class ProposalRecord(OrchestrationRecord):
"""Cognitive-worker proposal; never mutates canonical state directly."""
worker: str
action: str
evidence_refs: tuple[str, ...] = ()
confidence: float = 0.0
expiry: str | None = None
capability_loss_note: str = ""
_TUPLE_FIELDS[ProposalRecord] = frozenset({"evidence_refs"})
@dataclass
class EffectRecord(OrchestrationRecord):
"""Accepted proposal/request with lease and idempotency (S9)."""
unit_id: str
idempotency_key: str
state: str = "requested" # requested->eligible->leased->running->observed->verified
resource_lease: str = ""
attempt: int = 0
@dataclass
class ReceiptRecord(OrchestrationRecord):
"""Observed, independently verified result (S9)."""
unit_id: str
verifier: str
state: str # verified | failed | cancelled | superseded
evidence_refs: tuple[str, ...] = ()
rollback_ref: str = ""
_TUPLE_FIELDS[ReceiptRecord] = frozenset({"evidence_refs"})
@dataclass
class ApprovalRecord(OrchestrationRecord):
"""Explicit Nick start approval for an exact plan version (S9)."""
approver: str
plan_version: str
approved_items: tuple[str, ...]
granted_at: str = field(default_factory=_now)
revoked: bool = False
revoked_at: str | None = None
supersedes: str | None = None
_TUPLE_FIELDS[ApprovalRecord] = frozenset({"approved_items"})
@dataclass
class PinRecord(OrchestrationRecord):
"""Transactional rolling pin ownership (S3)."""
kind: str # root_plan | stage | batch | checkpoint | pin
item_id: str
owner: str
state: str = "active" # active | stale | cleared | open | resolved
evidence_ref: str = ""
details: str = "" # pin payload blob (question/outcome/context/reason) as JSON
@dataclass
class OutboxRecord(OrchestrationRecord):
"""Pending remote mutation with version precondition (S9)."""
mutation_id: str
remote_target: str
expected_remote_version: int
state: str = "queued" # queued->sending->accepted | retryable_failure | conflict | cancelled | superseded
payload_hash: str = ""
returned_remote_version: int | None = None
resolver: str = ""
plan_version: str = ""
approval_version: str = ""
ordering_key: int = 0
returned_remote_id: str = ""
attempts: int = 0
@dataclass(frozen=True)
class Capability:
"""A single surface capability; available=False = structured absence."""
name: str
available: bool = True
def to_dict(self) -> dict:
return {"name": self.name, "available": self.available}
@classmethod
def from_dict(cls, data: dict) -> "Capability":
return cls(name=data["name"], available=data.get("available", True))
@dataclass
class SurfaceRecord(OrchestrationRecord):
"""Bidirectional external/internal surface identity (S9)."""
surface_id: str
external_id: str
kind: str # local | server | remote | tui | voice | network
peer_key: str = ""
capabilities: tuple = () # tuple[Capability, ...]
def to_dict(self) -> dict:
data = asdict(self)
data["schema_version"] = self.schema_version
data["capabilities"] = [
c.to_dict() if isinstance(c, Capability) else c for c in self.capabilities
]
return data
@classmethod
def from_dict(cls, data: dict) -> "SurfaceRecord":
data = dict(data)
data.pop("schema_version", None)
caps = data.get("capabilities") or ()
data["capabilities"] = tuple(
Capability.from_dict(c) if isinstance(c, dict) else c for c in caps
)
return cls(**data)
@dataclass
class ResourceSnapshot(OrchestrationRecord):
"""Observed resource facts; unknown is not healthy (S8)."""
resource_id: str
healthy: bool | None = None # None = unknown
capabilities: tuple[str, ...] = ()
locality: str = ""
evidence_age_s: float | None = None
_TUPLE_FIELDS[ResourceSnapshot] = frozenset({"capabilities"})
@dataclass
class ProjectionCheckpoint(OrchestrationRecord):
"""Projection recovery point (S9)."""
offset: int
state_hash: str
+38
View File
@@ -0,0 +1,38 @@
"""Deterministic reflex layer (S8).
Reflexes are cheap, deterministic, idempotent facts: availability,
counts, deadlines, stale pins, failed checks. They can never reinterpret
intent or mutate goals — only exact, repeatable reductions of events.
"""
from __future__ import annotations
from collections.abc import Callable
Handler = Callable[[dict, dict], dict] # (state, event) -> new state
class ReflexEngine:
def __init__(self, handler: Handler):
self._handler = handler
self._state: dict = {}
def process(self, event: dict) -> dict:
self._state = self._handler(dict(self._state), dict(event))
return dict(self._state)
def facts(self) -> dict:
return dict(self._state)
class CountReflex:
def __init__(self, count_key: str):
self._key = count_key
self._last: str | None = None
def __call__(self, state: dict, event: dict) -> dict:
event_type = event.get("event_type")
if event_type != self._last: # collapse consecutive duplicates
state[self._key] = state.get(self._key, 0) + 1
self._last = event_type
return state
+56
View File
@@ -0,0 +1,56 @@
"""Resource state (S8): observed facts only; unknown is never healthy."""
from __future__ import annotations
from dataclasses import dataclass
from rune.orchestration.records import ResourceSnapshot
UNKNOWN = None # unknown health: never treated as healthy
@dataclass(frozen=True)
class ResourceCapabilities:
kind: str
locality: str
provider: str = ""
max_concurrency: int = 1
@dataclass(frozen=True)
class ResourceState:
resource_id: str
capabilities: ResourceCapabilities
healthy: bool | None
cost_per_call: float = 1.0
evidence_age_s: float | None = None
def is_usable(state: ResourceState) -> bool:
return state.healthy is True
class Registry:
def __init__(self) -> None:
self._states: dict[str, ResourceState] = {}
def register(self, state: ResourceState) -> None:
self._states[state.resource_id] = state
def get(self, resource_id: str) -> ResourceState | None:
return self._states.get(resource_id)
def all(self) -> dict[str, ResourceState]:
return dict(self._states)
def snapshot(self) -> dict[str, ResourceSnapshot]:
return {
rid: ResourceSnapshot(
resource_id=rid,
healthy=s.healthy,
capabilities=(s.capabilities.kind,),
locality=s.capabilities.locality,
evidence_age_s=s.evidence_age_s,
)
for rid, s in self._states.items()
}
+51
View File
@@ -0,0 +1,51 @@
"""Advisory resource router (S8).
Routing is advisory: it ranks ELIGIBLE candidates under a documented
policy. Final assignment and fallback are scheduler decisions, never the
router's. Tie-breaks are deterministic by resource_id.
"""
from __future__ import annotations
from dataclasses import dataclass
from typing import Protocol
from rune.orchestration.resources import ResourceState
@dataclass(frozen=True)
class RoutableResource:
state: ResourceState
score: float
class AdvisoryRouter(Protocol):
def rank(
self, candidates: list[ResourceState], requirement: dict
) -> list[RoutableResource]: ...
class FixedRouter:
policy_name = "fixed"
def __init__(self, fixed_id: str):
self._fixed_id = fixed_id
def rank(
self, candidates: list[ResourceState], requirement: dict
) -> list[RoutableResource]:
out = [
RoutableResource(s, 1.0 if s.resource_id == self._fixed_id else 0.0)
for s in candidates
]
return sorted(out, key=lambda r: (-r.score, r.state.resource_id))
class CostBiasRouter:
policy_name = "cost_bias"
def rank(
self, candidates: list[ResourceState], requirement: dict
) -> list[RoutableResource]:
out = [RoutableResource(s, 1.0 / (1.0 + s.cost_per_call)) for s in candidates]
return sorted(out, key=lambda r: (-r.score, r.state.resource_id))
+78
View File
@@ -0,0 +1,78 @@
"""Resource scheduler (S8): the single authority.
The scheduler owns the final decision: eligibility first (deterministic),
then advisory routing (ranking), then the lease. It records the chosen
assignment with a BOUNDED fallback chain. When nothing is eligible it
returns None — structured unavailability, never a silent wrong pick. A
top-ranked score of 0.0 is also structured unavailability: under both
current routers that only occurs when a FixedRouter's fixed id is absent
from the eligible set (CostBiasRouter always yields score > 0 for finite
cost), so the scheduler never silently picks a 0.0 candidate.
"""
from __future__ import annotations
from dataclasses import dataclass
from rune.orchestration.eligibility import EligibilityFilter
from rune.orchestration.resources import Registry
from rune.orchestration.router import AdvisoryRouter
@dataclass(frozen=True)
class ResourceAssignment:
task_signature: str
resource_id: str
fallback_chain: tuple[str, ...]
cap_concurrency: int
policy_ref: str
class ResourceScheduler:
def __init__(
self,
registry: Registry,
filter: EligibilityFilter,
router: AdvisoryRouter,
limits: dict | None = None,
):
self._registry = registry
self._filter = filter
self._router = router
self._limits = limits or {"max_fallback_chain": 2}
self._assignments: list[ResourceAssignment] = []
def acquire(
self, task_signature: str, requirement: dict
) -> ResourceAssignment | None:
states = list(self._registry.all().values())
eligible = self._filter.eligible(states, requirement)
if not eligible:
return None
ranked = self._router.rank(eligible, requirement)
if ranked[0].score == 0.0:
# Score 0.0 means the fixed resource is missing/ineligible:
# structured unavailability, never a silent wrong pick.
return None
winner = ranked[0]
chain = tuple(r.state.resource_id for r in ranked[1:])
chain = chain[: self._limits.get("max_fallback_chain", 2)]
assignment = ResourceAssignment(
task_signature=task_signature,
resource_id=winner.state.resource_id,
fallback_chain=chain,
cap_concurrency=winner.state.capabilities.max_concurrency,
policy_ref=self._policy_name(requirement),
)
self._assignments.append(assignment)
return assignment
def assignments(self) -> list[ResourceAssignment]:
return list(self._assignments)
def _policy_name(self, requirement: dict) -> str:
# Router policy is part of the contract; the scheduler records it
# from the router instance, falling back to the requirement key.
return getattr(
self._router, "policy_name", requirement.get("policy", "cost_bias")
)
+39
View File
@@ -0,0 +1,39 @@
"""Deterministic successor selection (S10).
A verified receipt must trigger exactly one next eligible unit, chosen by
the dependency graph and approval scope — never by model preference.
A unit is eligible only when it is in the approval scope AND its own
``approved`` flag is True AND all of its dependencies are verified.
This selector is a pure function so re-entry from the same durable event
state reproduces the same next unit.
"""
from __future__ import annotations
from dataclasses import dataclass
@dataclass(frozen=True)
class BatchUnit:
unit_id: str
dependencies: tuple[str, ...] = ()
approved: bool = True
def select_next(
graph: dict[str, BatchUnit],
verified: set[str],
approved_scope: set[str],
) -> str | None:
eligible: list[str] = []
for unit in graph.values():
if unit.unit_id not in approved_scope or not unit.approved:
continue
if unit.unit_id in verified:
continue
if not all(dep in verified for dep in unit.dependencies):
continue
eligible.append(unit.unit_id)
if not eligible:
return None
return min(eligible)
+72
View File
@@ -0,0 +1,72 @@
"""Surface primitives: capabilities and the surface registry (S9).
A surface is an entry path (local, server, remote, TUI, voice, network)
with a stable internal id AND a transport-side external id — the mapping
between them is bidirectional and stable. Capabilities carry an
availability flag so structured absence is representable (a degraded
path declares lost capabilities rather than pretending parity).
"""
from __future__ import annotations
import json
from dataclasses import replace
from pathlib import Path
from rune.orchestration.records import Capability, SurfaceRecord
__all__ = ["Capability", "SurfaceRegistry"]
class SurfaceRegistry:
def __init__(self) -> None:
self._by_id: dict[str, SurfaceRecord] = {}
self._by_external: dict[str, SurfaceRecord] = {}
def register(self, surface: SurfaceRecord) -> None:
existing = self._by_id.get(surface.surface_id)
if existing is not None and existing.external_id != surface.external_id:
raise ValueError(
f"surface {surface.surface_id} already bound to "
f"{existing.external_id!r}, cannot rebind to {surface.external_id!r}"
)
holder = self._by_external.get(surface.external_id)
if holder is not None and holder.surface_id != surface.surface_id:
raise ValueError(
f"external {surface.external_id!r} already bound to "
f"{holder.surface_id}, cannot be claimed by {surface.surface_id}"
)
# Defensive copy: a caller mutating the record they registered must
# never corrupt the registry (records are treated as immutable here).
stored = replace(surface)
self._by_id[surface.surface_id] = stored
self._by_external[surface.external_id] = stored
def get(self, surface_id: str) -> SurfaceRecord | None:
# Bidirectional identity: resolve by the internal id first, then by
# the external id (Task 4 frozen test: after a lazy gateway enter,
# the surface is reachable via the external id it entered with).
return self._by_id.get(surface_id) or self._by_external.get(surface_id)
def by_external(self, external_id: str) -> SurfaceRecord | None:
return self._by_external.get(external_id)
def all(self) -> list[SurfaceRecord]:
return list(self._by_id.values())
def save(self, root: Path) -> None:
from rune.orchestration.atomic import AtomicStore
AtomicStore(Path(root) / "state").write(
"registry.json",
json.dumps([s.to_dict() for s in self._by_id.values()], sort_keys=True),
)
def load(self, root: Path) -> None:
from rune.orchestration.atomic import AtomicStore
raw = AtomicStore(Path(root) / "state").read("registry.json")
if raw is None:
return
for item in json.loads(raw):
self.register(SurfaceRecord.from_dict(item))
+38
View File
@@ -0,0 +1,38 @@
"""Surface parity composition (design: entry-path and surface parity).
One registry, one negotiator, one mapping store, one gateway. Every
production entry path enters through the same gateway (same constructor,
same composition root), so capability-set parity is enforced in one place.
"""
from __future__ import annotations
from dataclasses import dataclass
from pathlib import Path
from rune.orchestration.capability import CapabilityNegotiator
from rune.orchestration.gateway import SurfaceGateway
from rune.orchestration.mapping import MappingStore
from rune.orchestration.surface import SurfaceRegistry
@dataclass
class SurfaceParityAuthority:
registry: SurfaceRegistry
negotiator: CapabilityNegotiator
mapping_store: MappingStore
gateway: SurfaceGateway
def build_surface_parity(root: Path) -> SurfaceParityAuthority:
registry = SurfaceRegistry()
registry.load(Path(root) / "surface_registry") # S4 restart-orphan discharge
negotiator = CapabilityNegotiator()
mapping_store = MappingStore(root / "surface_mappings")
gateway = SurfaceGateway(registry, negotiator, mapping_store)
return SurfaceParityAuthority(
registry=registry,
negotiator=negotiator,
mapping_store=mapping_store,
gateway=gateway,
)
+85
View File
@@ -0,0 +1,85 @@
"""Gitea transport protocol (S4 item 17, S4 item 10).
Transports execute Gitea-bound mutations but never declare their own
success — the reconciler owns acceptance. The version-precondition and
conflict comparison (the reconnection logic) lives HERE: a newer remote
version is reported CONFLICT and never overwritten; a replayed mutation
ID is reported ALREADY_APPLIED (idempotent by mutation ID). The fake
makes this testable and offline-safe.
CONTRACT for real transports (declared per G4 review pin):
- apply() MUST be idempotent by mutation_id (replaying an already-applied
payload reports ALREADY_APPLIED) — the reconciler may call it twice for
a record that crashed mid-apply.
- Transient/transport failures MUST be reported as RETRYABLE inside
apply(); raising from apply() is a contract violation.
- remote_id() MUST be replay-safe: on ALREADY_APPLIED it derives the
remote id from the remote (not a configured constant), so a fresh
transport after a crash can still reconcile.
- Payload resolution (G3 review pin): OutboxRecord carries payload_hash
only; a real transport resolves the canonical title/body through the
DelegationPlanner (or a resolve hook) before applying. The fake applies
by mutation_id/version only — sufficient for this plan's gates.
"""
from __future__ import annotations
from typing import Protocol, runtime_checkable
from rune.orchestration.reconciler import ApplyResult
from rune.orchestration.records import OutboxRecord
@runtime_checkable
class GiteaTransport(Protocol):
def apply(self, record: OutboxRecord) -> ApplyResult: ...
def remote_version(self) -> int: ...
def remote_id(self, record: OutboxRecord) -> str: ...
class FakeGiteaTransport:
def __init__(
self,
remote_version: int = 0,
remote_id: str = "issue_1",
) -> None:
self._remote_version = remote_version
self._remote_id = remote_id
self._conflict_armed = False
self._conflict_version = 0
self.applied: list[OutboxRecord] = []
def conflict_on(self, new_version: int) -> None:
"""Simulate the remote advancing while we were offline.
PERSISTENT: the armed version stays in effect for every subsequent
apply() until the test replaces it (``_conflict_armed`` never
auto-resets) — this models a remote that genuinely moved on. Every
pending create whose expected_remote_version < armed version then
reports CONFLICT. Idempotent replay (same mutation_id already
applied) is checked FIRST and wins over the armed conflict.
``remote_version()`` is NOT the armed version; it tracks only
accepted mutations.
"""
self._conflict_armed = True
self._conflict_version = new_version
def apply(self, record: OutboxRecord) -> ApplyResult:
if any(a.mutation_id == record.mutation_id for a in self.applied):
return ApplyResult.ALREADY_APPLIED # idempotent replay
if (
self._conflict_armed
and self._conflict_version > record.expected_remote_version
):
return ApplyResult.CONFLICT # newer remote: never overwritten
self.applied.append(record)
self._remote_version += 1
return ApplyResult.ACCEPTED
def remote_version(self) -> int:
return self._remote_version
def remote_id(self, record: OutboxRecord) -> str:
return self._remote_id
+78
View File
@@ -0,0 +1,78 @@
"""Cognitive worker registry with independent toggles (S8/A1).
Workers are bounded, proposal-producing consumers of evidence. Each worker
has its own gate and sink; failing or disabling one worker must never
affect another.
"""
from __future__ import annotations
from dataclasses import dataclass
from typing import Protocol
from rune.orchestration.no_change import NoChangeGate
from rune.orchestration.proposals import ProposalSink
@dataclass
class WorkerResult:
worker: str
cadence: str
proposals_submitted: int
skipped: int
ok: bool
error: str = ""
class CognitiveWorker(Protocol):
name: str
cadence: str
def run(
self, evidence: list[dict], sink: ProposalSink, gate: NoChangeGate
) -> WorkerResult: ...
class WorkerRegistry:
def __init__(self) -> None:
self._workers: dict[str, CognitiveWorker] = {}
self._enabled: set[str] = set()
def register(self, worker: CognitiveWorker) -> None:
self._workers[worker.name] = worker
self._enabled.add(worker.name)
def worker(self, name: str) -> CognitiveWorker:
return self._workers[name]
def enable(self, name: str) -> None:
self._enabled.add(name)
def disable(self, name: str) -> None:
self._enabled.discard(name)
def enabled(self) -> list[str]:
return sorted(self._enabled)
def run_all(
self,
evidence: list[dict],
sinks: dict[str, ProposalSink],
gates: dict[str, NoChangeGate],
) -> list[WorkerResult]:
results: list[WorkerResult] = []
for name in self.enabled():
worker = self.worker(name)
try:
results.append(
worker.run(
evidence,
sinks.get(name, ProposalSink()),
gates.get(name, NoChangeGate()),
)
)
except Exception as exc: # isolation: one worker's failure stops no one
results.append(
WorkerResult(name, worker.cadence, 0, 0, False, str(exc))
)
return results
+168 -120
View File
@@ -15,9 +15,13 @@ Design:
a reason). Resolution is journaled.
- recall(context) — surface open pins relevant to the current conversation so
an unfinished thread is never silently forgotten.
- list() — the pinboard: open pins + resolved pins (what came back and closed).
- Stored as wing='pins' chunks so recall works through the existing hybrid
store; resolution is a distinct status.
- list_pins() — the pinboard: open pins + resolved pins (what came back and
closed).
- Stored as canonical PinRecord rows (kind='pin', item_id=topic, owner='pins')
in a JSON array under _store_path(); the pin's detail payload
(question/outcome/context/reason/source) lives in the record's single
`details` blob so the canonical record shape stays the one contract. The
legacy dict returned by pin/recall/list_pins is a projection of the record.
This is the mechanism for 'put a pin in that and we'll discuss it when I'm
more mature' — the thread persists, names its open question, and defines what
@@ -33,123 +37,157 @@ Usage:
import argparse
import json
import os
import sqlite3
import sys
from pathlib import Path
_SD = os.path.dirname(os.path.abspath(__file__))
if _SD not in sys.path: sys.path.insert(0, _SD)
if _SD not in sys.path:
sys.path.insert(0, _SD)
import memory_env
STORE = memory_env.store_db()
WING = "pins"
# Canonical pins store: a JSON array of PinRecord dicts, written
# temp-file + atomic-rename via AtomicStore (never a corrupt target).
PINS_STORE = os.path.join(memory_env.memory_dir(), "store", "pins.json")
def _db_rw():
import memory_store as ms
return ms.connect()
def _store_path():
"""Return the canonical pins store path (configurable for tests)."""
return Path(PINS_STORE)
def _load():
"""Read all canonical pin records from the store (JSON array)."""
from rune.orchestration.atomic import AtomicStore
content = AtomicStore(_store_path().parent).read(_store_path().name)
if content is None or not content.strip():
return []
return json.loads(content)
def _save(records):
"""Atomically persist the full JSON array of canonical pin records."""
from rune.orchestration.atomic import AtomicStore
AtomicStore(_store_path().parent).write(
_store_path().name, json.dumps(records, indent=2)
)
def _detail_blob(question, outcome, context, reason="", source="session"):
"""Encode the pin's detail payload as the canonical record's details blob."""
return json.dumps(
{
"question": question,
"outcome": outcome,
"context": context,
"reason": reason,
"source": source,
},
sort_keys=True,
)
def _project(rec):
"""Project a canonical PinRecord dict onto the legacy pin dict surface."""
details = {}
blob = rec.get("details", "")
if blob:
try:
details = json.loads(blob)
except (json.JSONDecodeError, TypeError):
details = {}
return {
"topic": rec.get("item_id", ""),
"question": details.get("question", ""),
"outcome": details.get("outcome", ""),
"context": details.get("context", ""),
"status": rec.get("state", ""),
"reason": details.get("reason", ""),
}
def pin(topic, question, outcome, context="", source="session"):
"""Put a pin in a topic: a resumable thread with a measurable outcome."""
if not (topic and question and outcome):
return {"pinned": False,
"error": "a pin needs topic, open question, and measurable outcome"}
text = (f"[topic] {topic}\n"
f"[open question] {question}\n"
f"[outcome] {outcome}\n"
f"[status] open\n"
f"[context] {context}\n"
f"[source] {source}")
db = None
try:
import memory_store as ms
db = _db_rw()
ok = ms.add_chunk(db, text, wing=WING, room="open",
source_path=f"pins::{topic}",
doc_id=f"pins::{topic[:40]}",
importance=0.8)
return {"pinned": ok, "wing": WING, "topic": topic[:60],
"outcome": outcome[:60]}
except Exception as e:
return {"pinned": False, "error": str(e)[:120]}
finally:
if db is not None:
db.close()
"""Put a pin in a topic: a resumable thread with a measurable outcome.
Persisted as a canonical PinRecord (kind='pin', item_id=topic,
owner='pins', state='open'); the detail payload is a single JSON blob in
the record's ``details`` field. Returns a legacy-compatible dict.
"""
if not outcome or not str(outcome).strip():
raise ValueError("a pin needs a measurable outcome; empty outcome is not a pin")
if not topic or not question:
raise ValueError("a pin needs topic and open question")
from rune.orchestration.records import PinRecord
rec = PinRecord(
kind="pin",
item_id=topic,
owner=WING,
state="open",
details=_detail_blob(question, outcome, context, source=source),
)
records = _load()
records.append(rec.to_dict())
_save(records)
return {"pinned": True, "wing": WING, "topic": topic, "outcome": outcome}
def unpin(topic, reason=""):
"""Resolve a pin in place. Only pins with a real topic resolve; the
outcome/reason is journaled so resolution is auditable (never a silent
delete)."""
db = None
try:
db = _db_rw()
db.row_factory = sqlite3.Row
# Find the OPEN pin by doc_id prefix (the text-hash chunk_id differs
# per revision, so match on doc_id + status, not chunk_id).
row = db.execute(
"SELECT rowid, chunk_id, text FROM chunks WHERE wing=? "
"AND doc_id=? AND text LIKE '%[status] open%' "
"ORDER BY rowid DESC LIMIT 1",
(WING, f"pins::{topic[:40]}")).fetchone()
if not row:
return {"unpinned": False, "error": f"no open pin: {topic}"}
resolved = row["text"].replace(
"[status] open",
f"[status] resolved\n[resolved reason] {reason}")
db.execute("UPDATE chunks SET text=?, room='resolved' WHERE rowid=?",
(resolved, row["rowid"]))
db.commit()
return {"unpinned": True, "topic": topic[:60], "reason": reason[:60]}
except Exception as e:
return {"unpinned": False, "error": str(e)[:120]}
finally:
if db is not None:
db.close()
"""Resolve a pin in place (status open -> resolved, journaled reason)."""
records = _load()
for i in range(len(records) - 1, -1, -1): # most recent open pin wins
rec = records[i]
if rec.get("item_id") == topic and rec.get("state") == "open":
details = {}
try:
details = json.loads(rec.get("details") or "{}")
except json.JSONDecodeError:
details = {}
details["reason"] = reason
rec["state"] = "resolved"
rec["details"] = json.dumps(details, sort_keys=True)
_save(records)
return {"unpinned": True, "topic": topic, "reason": reason}
return {"unpinned": False, "error": f"no open pin: {topic}"}
def recall(context, limit=4):
"""Surface open pins relevant to the current conversation."""
db = None
try:
import memory_store as ms
db = _db_rw()
rows = ms.chunk_recall(db, context, budget=limit, wing=WING)
out = []
for r in rows:
t = r.get("text", "")
if "[status] open" in t:
out.append({"text": t[:300]})
return out
except Exception as e:
return [{"text": f"(pins recall failed: {str(e)[:80]})"}]
finally:
if db is not None:
db.close()
"""Surface open pins relevant to the current conversation.
def list_pins(open_only=True, limit=30):
"""The pinboard: open pins (default) or all."""
db = None
try:
db = _db_rw()
db.row_factory = sqlite3.Row
where = "wing=? AND text LIKE '%[status] open%'" if open_only else "wing=?"
params = (WING,) if open_only else (WING,)
rows = db.execute(
f"SELECT text, ingested_at FROM chunks WHERE {where} "
"ORDER BY ingested_at DESC LIMIT ?", params + (limit,)).fetchall()
return [{"text": r["text"][:220], "pinned_at": r["ingested_at"]}
for r in rows]
except Exception:
Relevance: the recall term appears in the pin's topic or its context.
An EMPTY context is abstention — no signal, no pins (use list_pins to
see everything). This guards the ``"" in x`` matches-all edge.
"""
term = (context or "").lower()
if not term:
return []
finally:
if db is not None:
db.close()
out = []
for rec in _load():
if rec.get("state") != "open":
continue
p = _project(rec)
if term in p["topic"].lower() or term in p["context"].lower():
out.append(p)
if len(out) >= limit:
break
return out
def list_pins(open_only=False, limit=30):
"""The pinboard: all pins (open + resolved) by default, newest first."""
out = [_project(rec) for rec in _load()]
if open_only:
out = [p for p in out if p["status"] == "open"]
return list(reversed(out))[:limit]
def main():
ap = argparse.ArgumentParser(description="Pinboard — resumable topics with outcomes")
ap = argparse.ArgumentParser(
description="Pinboard — resumable topics with outcomes"
)
ap.add_argument("cmd", choices=["pin", "unpin", "recall", "list"])
ap.add_argument("arg", nargs="*", default=[])
ap.add_argument("--question", default="")
@@ -162,16 +200,31 @@ def main():
if args.cmd == "pin":
topic = " ".join(args.arg)
res = pin(topic, args.question, args.outcome, args.context)
print(json.dumps(res, indent=2) if args.json else
(f"pinned: {res.get('topic')} (outcome: {res.get('outcome')})"
if res.get("pinned") else f"pin failed: {res.get('error')}"))
try:
res = pin(topic, args.question, args.outcome, args.context)
except ValueError as e:
res = {"pinned": False, "error": str(e)}
print(
json.dumps(res, indent=2)
if args.json
else (
f"pinned: {res.get('topic')} (outcome: {res.get('outcome')})"
if res.get("pinned")
else f"pin failed: {res.get('error')}"
)
)
elif args.cmd == "unpin":
res = unpin(" ".join(args.arg), args.reason)
print(json.dumps(res, indent=2) if args.json else
(f"resolved: {res.get('topic')}" if res.get("unpinned")
else f"unpin failed: {res.get('error')}"))
print(
json.dumps(res, indent=2)
if args.json
else (
f"resolved: {res.get('topic')}"
if res.get("unpinned")
else f"unpin failed: {res.get('error')}"
)
)
elif args.cmd == "recall":
ctx = " ".join(args.arg)
@@ -184,7 +237,7 @@ def main():
return
print(f"# Open pins for: {ctx}\n")
for r in rows:
print(f"{r['text']}\n")
print(f"[{r['topic']}] {r['outcome']} ({r['context']})")
elif args.cmd == "list":
rows = list_pins(open_only=args.open_only)
@@ -194,9 +247,7 @@ def main():
label = "open" if args.open_only else "all"
print(f"{len(rows)} {label} pins")
for r in rows:
print(f" [{r['pinned_at']}] {r['text'][:70]}")
print(f" [{r['status']}] {r['topic']} — {r['outcome']}")
# ---------------------------------------------------------------------------
@@ -208,28 +259,25 @@ def handle_hook(event, data=None):
message = data.get("message", "") or data.get("text", "")
if message:
open_pins = list_pins(open_only=True, limit=3)
return {"status": "ok", "event": event, "handler": "pins",
"open_pins": len(open_pins)}
return {
"status": "ok",
"event": event,
"handler": "pins",
"open_pins": len(open_pins),
}
return {"status": "ok", "event": event, "handler": "pins"}
PLUGIN = {
"name": "pins",
"version": "1.0.0",
"description": "pins.py \u2014 the pinboard: resumable topics with a measurable outcome.",
"capabilities": [
"pins",
"memory",
"recall",
"status",
"memory_env"
],
"capabilities": ["pins", "memory", "recall", "status", "memory_env"],
"hooks": {},
"dependencies": [],
"provides": [],
"priority": 100,
"auto_activate": True
"auto_activate": True,
}
if __name__ == "__main__":
+891 -298
View File
File diff suppressed because it is too large. Load diff
+123
View File
@@ -0,0 +1,123 @@
# Planning-safe ablation cards
These experiments resolve architectural uncertainty without mutating Rune's
live runtime. Owner for execution and acceptance is Nick unless explicitly
delegated after **start**. Every run writes raw results and a signed manifest
under the future plan's evidence directory.
## A1 — Cognitive independence
- **Target:** disposable cadence harness modeled on the recorded Souveraine
contracts; later repeated as an implementation acceptance gate for Rune.
- **Hypothesis:** disabling or failing one cadence does not suppress another.
- **Fixture:** deterministic 30- and 110-turn transcripts with N+1, reflection,
and synthesis independently on/off/failing.
- **Measure:** invocation counts, input IDs, outputs, failures, latency, tokens.
- **Pass:** each enabled cadence meets its trigger/deadline in 100% of fixtures;
another cadence's state changes no expected invocation.
- **Capability risk:** missed immediate correction, reflection, or continuity.
- **Decision:** retain separate queues/triggers or revise coupling before build.
## A2 — Synthesis cursor integrity
- **Target:** disposable cursor harness using Souveraine-shaped journal data.
- **Hypothesis:** item-ID transactions cover every input exactly once, unlike a
date-only cursor under same-day and capped batches.
- **Fixture:** duplicate dates, late arrivals, 60k-character overflow, crash
before/after output commit, and replay.
- **Measure:** omitted, duplicated, falsely covered, and recoverable item IDs.
- **Pass:** zero omission/false coverage; replay converges to one committed
output per input set.
- **Capability risk:** irreversible continuity loss.
- **Decision:** item-ID cursor is mandatory unless another scheme meets the same
invariant.
## A3 — Entry-path semantic parity
- **Target:** planning-safe disposable conversation/transport harness using
Rune's documented current entry-path contracts; later repeated against real
adapters as an implementation acceptance gate.
- **Hypothesis:** one seeded request yields equivalent identity, tools, history,
images, cancellation, surfacings, failure, and receipt semantics locally and
through server, remote, and TUI paths.
- **Fixture:** deterministic provider, fixed agent snapshot, identical request,
injected tool failure, cancellation, image, and pending surfacing.
- **Measure:** typed event sequence and canonical-state hashes.
- **Pass:** exact invariant equality; approved negotiation exceptions must match
their owner, expiry, and restoration records.
- **Capability risk:** entry-path-dependent identity or functionality.
- **Decision:** no degraded path becomes production until parity passes.
## A4 — Resource routing and inheritance
- **Target:** disposable registry/router/scheduler harness.
- **Hypothesis:** eligibility removes impossible resources and every child task
inherits locality, provider, budget, depth, timeout, and concurrency policy.
- **Fixture:** provider outage, stale health, forbidden cloud, cost ceiling,
local overload, nested subagents, and fallback exhaustion.
- **Measure:** eligible set, rank, lease, fallback, cost, and policy violations.
- **Pass:** zero ineligible leases or child-policy escapes; identical inputs and
snapshot version yield identical eligible sets. Ranking efficacy is measured
on a fixed task/resource matrix: the advisory router must select an
owner-labelled acceptable top-three resource in at least 90% of cases and
never outrank an ineligible resource.
- **Capability risk:** silent cloud use, runaway workers, or false availability.
- **Decision:** alter filters/leases before connecting real providers.
## A5 — Event lag and replay
- **Target:** disposable event-store/bus/projection harness.
- **Hypothesis:** bounded-bus lag affects latency only; durable offsets restore
every non-coalescible event and the same projection hash; revoked keys cannot
regain authority, an explicitly recovered/rotated key can resume authorized
operation, and no partition permits two simultaneous autonomous hearth
leases.
- **Fixture:** 100k envelopes, forced receiver lag, duplicate delivery, restart,
corrupt projection, interrupted checkpoint, revoked/rotated peer keys,
duplicate agent identity, two claimed hearths, and a network partition that
later heals.
- **Measure:** loss, duplication after idempotency, recovery time, state hash,
rejected signer count, key-recovery result, and concurrent-hearth ownership.
- **Pass:** zero lost authority/completion events, identical final hash, recovery
within the owner-approved benchmark baseline, all unpinned/revoked signers
rejected, the approved replacement key completes a signed round trip, and
never more than one active hearth lease during or after a partition. If
availability must yield to preserve this invariant, the isolated side enters
an explicit read-only/limb state rather than acting autonomously.
- **Capability risk:** silent work loss or fabricated completion state.
- **Decision:** event durability is prerequisite for scheduler authority.
## A6 — UI engine capability rehearsal
- **Target:** existing ratatui and tuie paths in disposable checkouts.
- **Hypothesis:** an engine may be retired only after the candidate supports the
approved capability baseline.
- **Fixture:** chat, resume, tools, images, cancellation, interjection, voice,
schedules, settings, manager, onboarding, pins, failures, accessibility.
- **Measure:** pass/fail plus event and receipt parity per capability.
- **Pass:** 100% required capability parity; exceptions require explicit owner
approval, expiry, and restoration canary.
- **Capability risk:** removing a working surface behind a prettier scaffold.
- **Decision:** keep both engines until parity, then remove confirmed bloat only
with Nick's separate approval.
## A7 — Git provenance versus transactional hot state
- **Target:** disposable concurrent-write harness.
- **Hypothesis:** atomic hot records plus asynchronous Git provenance preserve
history while reducing contention and eliminating lost cursor/queue updates.
- **Fixture:** concurrent turns, queue updates, pin transfer, crash windows,
malformed write, delayed Git commit, rollback, and identical long transcripts
processed with cull, microcompact, sliding-window, sliding-reflect, and summary
policies.
- **Measure:** lost updates, recovery point, write latency, provenance coverage,
retained commitments/source links, token reduction, and retrieval accuracy on
a fixed question set.
- **Pass:** zero lost canonical updates; every committed record eventually maps
to provenance; rollback reconstructs the same accepted state. A compaction
strategy is acceptable only if it retains every golden commitment and source
link while meeting the approved token target; otherwise the result remains a
workload-specific trade-off rather than a default.
- **Capability risk:** either audit loss or fragile hot-path serialization.
- **Decision:** choose the fastest design satisfying both atomicity and eventual
provenance.
+42
View File
@@ -0,0 +1,42 @@
id claim support_sources contradict_sources confidence
C001 The public checkout is not independently buildable because required tuie path dependencies are absent S001,S002 high
C002 Local and server-created conversations do not preserve equivalent identity memory and skill semantics S003 high
C003 Remote conversation resume is an API-design gap rather than only an unwired client call S004 high
C004 Non-loopback CLI remote chat lacks the authenticated backend used by the TUI path S005 high
C005 Memory configuration contains controls that persist but do not govern live repositories S006 high
C006 N+1 currently acts as a load-bearing host for reflection Archivist and compaction warnings S007,S009 high
C007 Documented or configured post-turn cadences diverge from hard-coded execution behavior S008 high
C008 Compaction is advisory and model-invoked rather than a deterministic truncation policy S009 medium-high
C009 Subagent governance controls and per-agent provider isolation are incomplete S010 high
C010 Removing ratatui before parity would remove functional terminal chat capabilities S011 high
C011 Sensorium and Matrix include meaningful structure but are not fully operational output surfaces S012 medium-high
C012 Commit summaries and configuration surfaces cannot be treated as execution evidence S006,S008,S012 high
C013 The architecture pivoted from local-first language to one server-authoritative Rust substrate with local and remote viewports S013,S014 high
C014 The stated lineage combines Coquette and a Unified Consciousness prototype with mechanisms studied from Letta-Code jcode and Claw-Open S013,S015,S016 medium-high
C015 Early plans distinguish intended phases and stubs from implemented mechanisms and must not be read as current behavior S015,S016,S017 high
C016 Public-source documentation is intentionally incomplete and some claimed archives are absent from the public branch S017,S027,S028 high
C017 Tools are deliberately framed as embodied sensors and subagents as bounded forks of the operational substrate S018 medium-high
C018 The nervous system joins bounded event broadcast cron identity credentials and durable event records but durability varies by entry path S019,S036 high
C019 Subconscious state is deliberately distributed across ledger boxes inner voice and persisted conversation rather than held in one blob S019,S029,S030 high
C020 N+25 reflection is a bounded tool-using maintenance pass but its configured controls and write authority diverge from runtime S020,S029,S043 high
C021 N+100 Archivist is implemented but its date cursor and bounded input can mark unseen journal material as covered S021,S029,S031 high
C022 Federation separates machine transport identity from portable agent identity but configured peer trust is not enforced by the bridge S022,S023,S038 high
C023 Hearth and limb roles express a valuable single-home invariant but live split-brain prevention remains incomplete S024,S038 medium-high
C024 Provider abstraction and per-agent routing exist but richer capability routing and subagents do not consistently use them S025,S032,S035 high
C025 The dual-UI migration preserves experimental work but produces capability divergence and continuing dependency residue S026,S042,S046 high
C026 Development history and current public state must be reconstructed together because current docs intentionally omit superseded and private context S027,S028,S046 high
C027 N+1 is incorrectly coupled as the lifecycle host for reflection Archivist and pressure warnings S007,S009,S029,S043 high
C028 Recursive pinning of every system markdown file preserves identity but creates unbounded context-pressure risk S030 medium-high
C029 Injecting only the newest non-cumulative synthesis can remove older compressed continuity from active context S030,S031 high
C030 Archivist cursoring needs item-level transactional progress rather than a maximum date marker S031 high
C031 Subagent background role depth concurrency timeout and provider-governance promises exceed current execution controls S032 high
C032 Schedule creation manual triggers and runtime sensing disagree on paths and semantics S033,S034,S043 high
C033 LocalBackend and standalone server instantiate different autonomous services despite sharing the same core server S034,S040 high
C034 Model physics and task-fit routing exist as a disconnected advisory subsystem rather than the authority for runtime selection S035 high
C035 EventBus provides useful decoupling but bounded non-replay delivery cannot serve as completion or durable task truth S036,S038 high
C036 Matrix has architectural shape but is neither registered nor capable of a complete inbound-to-visible-reply loop S037 high
C037 Federation authenticates self-declared signers but does not pin configured peer keys or replay disconnect gaps S038 high
C038 Remote transport is a lossy projection that can erase lifecycle evidence and make inference failure resemble clean completion S039 high
C039 Mode-specific composition roots are the main cause of local server and remote capability divergence S039,S040 high
C040 Conversational voice is operational in ratatui but demonstrative in tuie and is not a surface-neutral sensorium capability S041,S042 high
C041 Configuration persistence does not prove runtime authority and some updates create live-versus-disk split state S043,S045 high
1 id claim support_sources contradict_sources confidence
2 C001 The public checkout is not independently buildable because required tuie path dependencies are absent S001,S002 high
3 C002 Local and server-created conversations do not preserve equivalent identity memory and skill semantics S003 high
4 C003 Remote conversation resume is an API-design gap rather than only an unwired client call S004 high
5 C004 Non-loopback CLI remote chat lacks the authenticated backend used by the TUI path S005 high
6 C005 Memory configuration contains controls that persist but do not govern live repositories S006 high
7 C006 N+1 currently acts as a load-bearing host for reflection Archivist and compaction warnings S007,S009 high
8 C007 Documented or configured post-turn cadences diverge from hard-coded execution behavior S008 high
9 C008 Compaction is advisory and model-invoked rather than a deterministic truncation policy S009 medium-high
10 C009 Subagent governance controls and per-agent provider isolation are incomplete S010 high
11 C010 Removing ratatui before parity would remove functional terminal chat capabilities S011 high
12 C011 Sensorium and Matrix include meaningful structure but are not fully operational output surfaces S012 medium-high
13 C012 Commit summaries and configuration surfaces cannot be treated as execution evidence S006,S008,S012 high
14 C013 The architecture pivoted from local-first language to one server-authoritative Rust substrate with local and remote viewports S013,S014 high
15 C014 The stated lineage combines Coquette and a Unified Consciousness prototype with mechanisms studied from Letta-Code jcode and Claw-Open S013,S015,S016 medium-high
16 C015 Early plans distinguish intended phases and stubs from implemented mechanisms and must not be read as current behavior S015,S016,S017 high
17 C016 Public-source documentation is intentionally incomplete and some claimed archives are absent from the public branch S017,S027,S028 high
18 C017 Tools are deliberately framed as embodied sensors and subagents as bounded forks of the operational substrate S018 medium-high
19 C018 The nervous system joins bounded event broadcast cron identity credentials and durable event records but durability varies by entry path S019,S036 high
20 C019 Subconscious state is deliberately distributed across ledger boxes inner voice and persisted conversation rather than held in one blob S019,S029,S030 high
21 C020 N+25 reflection is a bounded tool-using maintenance pass but its configured controls and write authority diverge from runtime S020,S029,S043 high
22 C021 N+100 Archivist is implemented but its date cursor and bounded input can mark unseen journal material as covered S021,S029,S031 high
23 C022 Federation separates machine transport identity from portable agent identity but configured peer trust is not enforced by the bridge S022,S023,S038 high
24 C023 Hearth and limb roles express a valuable single-home invariant but live split-brain prevention remains incomplete S024,S038 medium-high
25 C024 Provider abstraction and per-agent routing exist but richer capability routing and subagents do not consistently use them S025,S032,S035 high
26 C025 The dual-UI migration preserves experimental work but produces capability divergence and continuing dependency residue S026,S042,S046 high
27 C026 Development history and current public state must be reconstructed together because current docs intentionally omit superseded and private context S027,S028,S046 high
28 C027 N+1 is incorrectly coupled as the lifecycle host for reflection Archivist and pressure warnings S007,S009,S029,S043 high
29 C028 Recursive pinning of every system markdown file preserves identity but creates unbounded context-pressure risk S030 medium-high
30 C029 Injecting only the newest non-cumulative synthesis can remove older compressed continuity from active context S030,S031 high
31 C030 Archivist cursoring needs item-level transactional progress rather than a maximum date marker S031 high
32 C031 Subagent background role depth concurrency timeout and provider-governance promises exceed current execution controls S032 high
33 C032 Schedule creation manual triggers and runtime sensing disagree on paths and semantics S033,S034,S043 high
34 C033 LocalBackend and standalone server instantiate different autonomous services despite sharing the same core server S034,S040 high
35 C034 Model physics and task-fit routing exist as a disconnected advisory subsystem rather than the authority for runtime selection S035 high
36 C035 EventBus provides useful decoupling but bounded non-replay delivery cannot serve as completion or durable task truth S036,S038 high
37 C036 Matrix has architectural shape but is neither registered nor capable of a complete inbound-to-visible-reply loop S037 high
38 C037 Federation authenticates self-declared signers but does not pin configured peer keys or replay disconnect gaps S038 high
39 C038 Remote transport is a lossy projection that can erase lifecycle evidence and make inference failure resemble clean completion S039 high
40 C039 Mode-specific composition roots are the main cause of local server and remote capability divergence S039,S040 high
41 C040 Conversational voice is operational in ratatui but demonstrative in tuie and is not a surface-neutral sensorium capability S041,S042 high
42 C041 Configuration persistence does not prove runtime authority and some updates create live-versus-disk split state S043,S045 high
+19
View File
@@ -0,0 +1,19 @@
id mechanism implementation_status interactions disposition rune_target capability_loss_if_omitted evidence uncertainty required_ablation
M001 One constitutive substrate with multiple viewports Partly implemented; entry paths diverge Conversation construction, tools, memory, surfaces adapt S8 entry-path parity; S9 composition root Identity and capability vary by interface S003,S014,S039,S040 Production parity unmeasured A3
M002 N+1 immediate subconscious review Implemented but coupled Primary completion, inbox, tools, reflection, synthesis preserve-and-decouple S8 three strata; S9 cognitive flow Immediate omissions and commitments go unnoticed S007,S019,S029 Adaptive trigger selection is deferred tuning after semantic independence A1
M003 N+25 wider reflection Implemented with hard-coded cadence and limited write authority Transcript evidence, memory tools, post-turn lifecycle preserve-and-decouple S8 independent cadences; S9 cognitive flow Cross-turn learning and correction weaken S008,S020,S029,S043 Boundary-versus-deadline selection is deferred tuning; A1 tests independence only A1
M004 N+100 archival synthesis Implemented with cursor risks Journal items, synthesis chain, prompt injection preserve-and-rebuild-cursor S8 memory and synthesis Long-horizon continuity degrades S021,S030,S031 Hierarchy and cadence selection are deferred tuning; A2 tests lossless cursoring only A2
M005 Ledger, inbox, inner voice, and conversation channels Implemented across separate stores N+1, primary prompt, surfacing, durable memory preserve-under-one-schema S8 subconscious state; S9 records Different retrieval and urgency semantics collapse S019,S029,S030 Retention tiers are deferred tuning; A1 and A7 test independence and atomic provenance A1,A7
M006 Git-backed markdown provenance Implemented Identity, memory writes, reflection, synthesis preserve-for-history S8 memory and synthesis Human-readable audit and rollback are lost S006,S030,S031 Hot-write architecture A7
M007 Recursive system-memory pinning Implemented without explicit budget Prompt assembly, identity, context pressure constrain-with-budget S8 memory and synthesis Overconstraint could omit identity context; unconstrained growth exhausts context S030 Priority/overflow tuning is deferred; A3 and A7 guard parity and retention A3,A7
M008 Tools as embodied sensors Implemented conceptually and structurally Capability catalog, prompts, subagents preserve S8 resource authority Agent orientation and boundary legibility weaken S018 Description-quality evaluation is deferred UX research; A4 tests authority only A4
M009 EventBus nervous system Implemented as bounded non-durable broadcast Cron, surfaces, federation, registry, telemetry adapt-with-durable-spine S8 canonical spine; S9 event store Decoupling is lost if removed; events are lost if copied unchanged S019,S036,S038 Throughput and replay cost A5
M010 Schedules and background turns Implemented with path and mode divergence Cron sensor, local backend, turn injector, surfacing preserve-and-unify S9 scheduler and effects Proactive commitments and reminders fail S033,S034,S040,S043 Quota tuning is deferred; A4 and A5 test authority and durability A4,A5
M011 Per-agent provider routing Partly implemented; subagents bypass it Primary, subconscious, reflection, Archivist, subagents preserve-and-enforce S8 resource authority Sovereignty, locality, cost, and model fit erode S025,S032,S035 Ranking quality A4
M012 Task-aware model physics router Implemented but disconnected Catalog, task signature, context pressure adapt-as-advisory S8 two-stage routing Potential efficiency and fit gains are lost S035 Ranking efficacy unmeasured A4
M013 Subagent forks Implemented with unenforced controls Tools, memory root, provider, nesting, background work preserve-with-inherited-authority S8 resource authority; S9 scheduler Parallel investigation and bounded delegated work disappear S010,S018,S032 Optimal depth/concurrency are deferred tuning; A4 tests inherited limits A4
M014 Agent versus device cryptographic identity Implemented Federation transport, reach/consult, consent preserve-and-enforce-peer-pinning S8 surface parity and trust Portable identity or transport accountability is lost S022,S023,S038 Key lifecycle and recovery A5
M015 Hearth and limb single-home model Partly implemented Federation, device registry, split-brain handling preserve-invariant S8 failure handling; S9 authority Competing autonomous homes can act concurrently S024,S038 Consensus under partition A5
M016 Surface-neutral sensorium Architectural shape; Matrix incomplete EventBus, room mapping, remote transport, UI preserve-goal-not-stubs S8 entry-path parity; S9 surface gateway New surfaces fork behavior or cannot return replies S037,S039,S040 Surface baseline A3,A6
M017 Dual ratatui/tuie engines Both present; capability parity absent Chat state, voice, manager controls, dependencies preserve-until-measured S8 parity; S9 verification Premature removal loses voice or working controls S026,S041,S042,S046 Replacement completeness A6
M018 Context compaction strategies Implemented but model-invoked and N+1-warning-bound Prompt pressure, memory tool, evidence retention preserve-with-explicit-policy S8 memory; S9 scheduler Context exhaustion if omitted; evidence loss if applied opaquely S009,S029,S030 Strategy-by-workload quality A2,A7
1 id mechanism implementation_status interactions disposition rune_target capability_loss_if_omitted evidence uncertainty required_ablation
2 M001 One constitutive substrate with multiple viewports Partly implemented; entry paths diverge Conversation construction, tools, memory, surfaces adapt S8 entry-path parity; S9 composition root Identity and capability vary by interface S003,S014,S039,S040 Production parity unmeasured A3
3 M002 N+1 immediate subconscious review Implemented but coupled Primary completion, inbox, tools, reflection, synthesis preserve-and-decouple S8 three strata; S9 cognitive flow Immediate omissions and commitments go unnoticed S007,S019,S029 Adaptive trigger selection is deferred tuning after semantic independence A1
4 M003 N+25 wider reflection Implemented with hard-coded cadence and limited write authority Transcript evidence, memory tools, post-turn lifecycle preserve-and-decouple S8 independent cadences; S9 cognitive flow Cross-turn learning and correction weaken S008,S020,S029,S043 Boundary-versus-deadline selection is deferred tuning; A1 tests independence only A1
5 M004 N+100 archival synthesis Implemented with cursor risks Journal items, synthesis chain, prompt injection preserve-and-rebuild-cursor S8 memory and synthesis Long-horizon continuity degrades S021,S030,S031 Hierarchy and cadence selection are deferred tuning; A2 tests lossless cursoring only A2
6 M005 Ledger, inbox, inner voice, and conversation channels Implemented across separate stores N+1, primary prompt, surfacing, durable memory preserve-under-one-schema S8 subconscious state; S9 records Different retrieval and urgency semantics collapse S019,S029,S030 Retention tiers are deferred tuning; A1 and A7 test independence and atomic provenance A1,A7
7 M006 Git-backed markdown provenance Implemented Identity, memory writes, reflection, synthesis preserve-for-history S8 memory and synthesis Human-readable audit and rollback are lost S006,S030,S031 Hot-write architecture A7
8 M007 Recursive system-memory pinning Implemented without explicit budget Prompt assembly, identity, context pressure constrain-with-budget S8 memory and synthesis Overconstraint could omit identity context; unconstrained growth exhausts context S030 Priority/overflow tuning is deferred; A3 and A7 guard parity and retention A3,A7
9 M008 Tools as embodied sensors Implemented conceptually and structurally Capability catalog, prompts, subagents preserve S8 resource authority Agent orientation and boundary legibility weaken S018 Description-quality evaluation is deferred UX research; A4 tests authority only A4
10 M009 EventBus nervous system Implemented as bounded non-durable broadcast Cron, surfaces, federation, registry, telemetry adapt-with-durable-spine S8 canonical spine; S9 event store Decoupling is lost if removed; events are lost if copied unchanged S019,S036,S038 Throughput and replay cost A5
11 M010 Schedules and background turns Implemented with path and mode divergence Cron sensor, local backend, turn injector, surfacing preserve-and-unify S9 scheduler and effects Proactive commitments and reminders fail S033,S034,S040,S043 Quota tuning is deferred; A4 and A5 test authority and durability A4,A5
12 M011 Per-agent provider routing Partly implemented; subagents bypass it Primary, subconscious, reflection, Archivist, subagents preserve-and-enforce S8 resource authority Sovereignty, locality, cost, and model fit erode S025,S032,S035 Ranking quality A4
13 M012 Task-aware model physics router Implemented but disconnected Catalog, task signature, context pressure adapt-as-advisory S8 two-stage routing Potential efficiency and fit gains are lost S035 Ranking efficacy unmeasured A4
14 M013 Subagent forks Implemented with unenforced controls Tools, memory root, provider, nesting, background work preserve-with-inherited-authority S8 resource authority; S9 scheduler Parallel investigation and bounded delegated work disappear S010,S018,S032 Optimal depth/concurrency are deferred tuning; A4 tests inherited limits A4
15 M014 Agent versus device cryptographic identity Implemented Federation transport, reach/consult, consent preserve-and-enforce-peer-pinning S8 surface parity and trust Portable identity or transport accountability is lost S022,S023,S038 Key lifecycle and recovery A5
16 M015 Hearth and limb single-home model Partly implemented Federation, device registry, split-brain handling preserve-invariant S8 failure handling; S9 authority Competing autonomous homes can act concurrently S024,S038 Consensus under partition A5
17 M016 Surface-neutral sensorium Architectural shape; Matrix incomplete EventBus, room mapping, remote transport, UI preserve-goal-not-stubs S8 entry-path parity; S9 surface gateway New surfaces fork behavior or cannot return replies S037,S039,S040 Surface baseline A3,A6
18 M017 Dual ratatui/tuie engines Both present; capability parity absent Chat state, voice, manager controls, dependencies preserve-until-measured S8 parity; S9 verification Premature removal loses voice or working controls S026,S041,S042,S046 Replacement completeness A6
19 M018 Context compaction strategies Implemented but model-invoked and N+1-warning-bound Prompt pressure, memory tool, evidence retention preserve-with-explicit-policy S8 memory; S9 scheduler Context exhaustion if omitted; evidence loss if applied opaquely S009,S029,S030 Strategy-by-workload quality A2,A7
+60
View File
@@ -0,0 +1,60 @@
# Souveraine/Fimeg architecture survey
## Research question
Which mechanisms, development choices, constraints, and capability-bearing
interactions in Souveraine's public primary-source history should Rune preserve,
adapt, test, or reject when designing a subconscious and resource architecture
that has one authoritative execution path?
## Sub-questions
1. What lineage, goals, and system boundaries does the author state?
2. Which mechanisms are implemented, which are aspirational, and which are
configured but disconnected?
3. How do N+1 subconscious work, reflection, Archivist synthesis, memory,
providers, subagents, sensorium, scheduling, and UI surfaces interact?
4. Which couplings are capability-critical, accidental, or contradicted by live
behavior?
5. What would Rune lose by omitting each mechanism?
6. Which uncertainties require an ablation rather than an architectural guess?
## Ideal answer
An evidence-linked mechanism and interaction map that separates source intent
from observed implementation, exposes contradictions, states capability-loss
risk for every omission, and translates findings into testable design changes
without introducing a second authority path.
## Scope and stopping rule
- Primary material only: public repository files, commit history, task/design
records, forge metadata, and author-authored documentation.
- Public revision baseline: Codeberg branch `public` at
`80bfb3a59935e51b4df21abbf4f9545ee9a0af56`.
- Examine at least 30 distinct source records, including dead ends and
contradictions, or stop after five consecutive examined sources add no
material claim.
- Record whether claim saturation was reached rather than pretending a source
count proves completeness.
- Planning and research only; no Rune runtime implementation.
## Search clusters
- Souveraine subconscious N+1 consciousness engine reflection Archivist
- Souveraine memory git compaction identity substrate
- Souveraine provider per-agent resource routing subagent
- Souveraine EventBus sensorium Matrix federation device registry
- Souveraine scheduler cron reminders background work
- Fimeg Souveraine design task roadmap architecture
- git history N+1 reflection memory provider subagent sensorium
- implemented versus configured unused TODO stub unimplemented
- local remote backend semantic parity conversation prompt skills
- tuie ratatui migration capability parity
## Baseline
The cold baseline used three obvious repository searches: documentation and
manifest inspection, runtime symbol search, and recent commit-history review.
It produced twelve initial source records and twelve claims, but no complete
development-history reconstruction and no source-linked Rune mapping.
Loaded 100 of 152 files, more files were not shown because too many files have changed in this diff. Show more