rune/docs/conversation-state.md

ConversationState and TaskState (#18 / P0-02)

Canonical, durable conversation and task ledgers for Rune. This replaces the "mint a new task on every send" behavior and the display-only ChannelView.messages with two single-authority repositories.

Module

src/rune/orchestration/conversation_state.py

Records (src/rune/orchestration/records.py)

• TaskStateRecord — task_id (new_id("task")), title, status (created→running→awaiting_approval→completed|failed|cancelled|superseded), channel_id, parent_task_id (follow-up lineage), assigned_model, created_at, updated_at, outcome_ref.
• ConversationMessageRecord — msg_id (new_id("msg")), ts, role (user/assistant/system/tool), content, channel_id, task_id, token_estimate.

Both are OrchestrationRecord subclasses: stable ids, schema versioning, dict round-trip.

Repositories

TaskStateRepository (canonical task identity):

• create(title, channel_id, model=None, parent_task_id=None) -> TaskStateRecord
• amend(task_id, title) -> TaskStateRecord — follow-ups keep the SAME task_id; the task continues instead of being replaced.
• transition(task_id, new_status, outcome_ref="") -> TaskStateRecord — legal edges in TASK_TRANSITIONS; illegal moves raise IllegalTaskTransitionError.
• get_active(channel_id) -> TaskStateRecord | None — the open task, if any (cleared automatically on terminal status).
• get(task_id), list_open(channel_id), latest(channel_id).

ConversationStateRepository (durable message history):

• append(msg) — idempotent by msg_id (O(1) in-memory id set); writes one fsync'd JSONL tail line, never a full-checkpoint rewrite.
• window(channel_id, n=8) — last n messages, chronological (hot tier).
• messages_for_task(task_id) — full per-task history, spanning hot and cooled tiers.
• get_message(msg_id) — exact-by-ID read across hot, tail, and cool tiers with the memory-continuity runtime's typed statuses (FOUND_VERIFIED / SOURCE_CORRUPT / NOT_FOUND_CONFIRMED).
• search(text, channel_id=None) — substring search over content spanning hot and cooled messages (tier attribution; cooling never deletes).
• compact() — fold the tail into the hot checkpoint (runs automatically every compact_every appends and on load when a fat legacy checkpoint is found).
• latest_task(channel_id) — most recent task on the channel.

Repositories never assemble prompts — that is #19 (ContextBroker).

Persistence split

• DurableEventStore — every transition and message appends a typed marker envelope (task_state.created|amended|transition|imported, conversation.message) with a sha256 payload_hash of the record body, flushed to disk before the checkpoint advances.
• AtomicStore — hot checkpoints (hot/task_state.json, hot/conversation_messages.json) hold full record bodies for O(1) reads; temp+fsync+rename means a crash never corrupts them. A corrupt checkpoint raises CorruptStateCheckpointError — loud, never silent.
• JSONL tail (hot/conversation_tail.jsonl) — message bodies append INCREMENTALLY: one fsync'd line per message. The conversation checkpoint moves only on compaction (every compact_every=64 appends), not per message — the 6372-message/1.7MB-per-append write amplification the chat-live research flagged is gone.
• Compaction + cooling — compaction folds the tail into the hot checkpoint keeping the last hot_window=512 messages per channel hot; older messages cool through the memory-continuity runtime's cooling contract into durable per-channel archives (hot/conversation_cool/), each record carrying a content digest. Cooling releases the hot window without deleting anything: recovery replays checkpoint + tail with msg_id dedupe, exact-by-ID reads verify cool digests, and search spans both tiers. A corrupt tail line or cool archive raises CorruptStateCheckpointError — loud, never silent.

Composition

AUTHORITY_KINDS gains conversation_state and task_state. Wire with:

from rune.orchestration.composition import build_conversation_state

root = build_conversation_state({
    "events": run_dir / "events.jsonl",
    "evidence": run_dir / "evidence",
    "state": run_dir,
})
tasks = root.get("task_state")
conversation = root.get("conversation_state")

or add onto an existing root with compose_conversation_state(root, state_dir). Run the suite:

PYTHONPATH=src:. python3 -m pytest tests/test_orchestration_conversation_state.py -q
PYTHONPATH=src:. python3 src/rune/canary_conversation_state.py

Operator integration

RuneOperator.send_task delegates identity to TaskStateRepository:

• no active task → create() with parent_task_id = previous task (lineage).
• active task running, or the message matches a confirmation pattern (yes/ok/continue/…) → amend() — same task_id, new title.
• every send appends the user message; every completion appends the assistant message to ConversationStateRepository.
• _send_to_model signature and the response envelope are unchanged.

RuneOperator.__init__ wires both repositories next to operator.json and runs migrate_operator_tasks once (idempotent): the 110 legacy tasks keep their original task ids, completed results are reconstructed as user/assistant messages, and the legacy current_task pointer is honored. conversation_window() / get_conversation_window() read the canonical history.

TUI

ChannelView.messages entries stay a projection: consumers (_cmd_pressure, _cmd_persona_gate, ambient context) keep working off ch.messages[-n:] dicts; the durable history lives in the repository, not the widget list.

Thread safety

The TUI mutates state from workers; both repositories guard all public methods with a re-entrant lock, and append/transition writes are flushed before checkpoints move.