Compare commits

..
8 Commits
Author SHA1 Message Date
conductor affe852ff7 chore(release): auto-bump version to 0.66.63 — earned by # (minor: converged release (push-to-master bump))
Promotion record: (unknown)
2026-09-15 16:27:30 +01:00
Nick e1a71efeb9 feat(ci): event-driven CI qualification wait over the existing W/R bridge (T16 16C/16D/16E)
- deploy.yml sleep-30 polling loop REPLACED: registers a gitea-identity
  PROCESS_EVENT wait, yields in select() on the FIFO, resumes on the
  ci.yml notify-event-bridge WAKE, then re-reads the canonical Gitea
  verdict (deploy_qualify.py). Event payload is never completion truth.
- ci.yml gains notify-event-bridge (always(), best-effort): a red CI
  wakes the waiter as loudly as a green one; bridge unreachable ->
  watchdog recovery covers the lost wake.
- event_wait.py: register_process_wait gains provider/event_type/sha
  identity (16E: ONE bridge primitive; builder + gitea are adapters);
  open_fifo_reader + caller-owned fifo_fd close the 16Y#10 race (event
  before waiter in select()); await_resume_event returns eof instead of
  busy-looping on a caller-owned fd.
- ci_event_wait.py watchdog: failure DETECTOR only — liveness-gated
  re-arm, one-shot authority recovery on lost events, truthful failure
  when no verdict exists; timestamps emitted for the 16D latency proof.
- tests: 8 held-outs on the REAL daemon (shared wr_daemon fixture in
  conftest — no duplicated helpers). Daemon-side: /process-event passes
  through provider/event_type/sha (defaults preserve T15); lease_path
  sanitizes slash-bearing session ids (resume delivery was impossible
  for them — found live, fixed in daemon snapshot 79434e0).

Suite 1861 green; ratchets PASS; Henxels rc=0.
2026-09-15 16:26:54 +01:00
conductor 5bd8cca941 chore(release): auto-bump version to 0.66.62 — earned by # (patch: converged release (push-to-master bump))
Promotion record: (unknown)
2026-09-15 14:15:22 +01:00
Nick fd5f31bba2 fix(scaffolding): §15D tests use ephemeral ports, not fixed 8793
CI failure on the 0.66.61 bump run (job 18584): the latency test got
watchdog/alive:False because its daemon could not bind the FIXED port
8793 — a stale TIME_WAIT socket from an earlier test in the same file
held it (verified locally: 8793 bindable only with SO_REUSEADDR while
no listener exists). The notifier's event POSTs went nowhere, so the
watchdog correctly reported PROCESS_LOST: the harness was at fault,
not the bridge. Daemon fixture now binds port 0 (ephemeral) and reads
back the assigned port. 3x consecutive green locally.
2026-09-15 14:14:44 +01:00
conductor 8a2a4f7b71 chore(release): auto-bump version to 0.66.61 — earned by # (patch: converged release (push-to-master bump))
Promotion record: (unknown)
2026-09-15 13:42:21 +01:00
Nick b2d6802dba fix(scaffolding): §15K test no longer shells out to git worktree on CI
test_15k_build_manager_has_no_parallel_construction_state invoked
BuildManager.start_build for real, which runs 'git worktree add' against
the HARDCODED production path /mnt/workdrive/rune — a real VCS mutation
inside a unit test. Locally it polluted ~/.rune/work/builds; on the CI
runner it fails outright (dubious-ownership: runner checkout is a
different user). The test now redirects BUILDS_DIR/WORKTREE_DIR into tmp
and stubs create_worktree — the property under test (no canonical
construction state from the /build vehicle) is orthogonal to the git op.
This was the CI unit-tests failure on 193d819 (1 failed, 1852 passed).
2026-09-15 13:41:43 +01:00
conductor 3ebbf9a9b1 chore(release): auto-bump version to 0.66.60 — earned by # (patch: converged release (push-to-master bump))
Promotion record: (unknown)
2026-09-15 13:29:02 +01:00
Nick 193d81961d fix(scaffolding): event_wait importlib-clean loading, zero pyright warnings
_load_waitstore now resolves the bridge's own waitstore.py via
importlib.util.spec_from_file_location (no dynamic bare import, no
sys.path mutation) with per-root caching. Verified: 0 errors/0 warnings
in event_wait.py; per-file stash comparison shows my 8 touched files add
zero warnings over baseline (rune_operator 5/5, rune_tui 307/307
identical). The 443→445 CI warning delta is pre-existing environment
skew in untouched files, not T15 regression.
2026-09-15 13:25:39 +01:00
15 changed files with 707 additions and 79 deletions

No files matched your search

BIN
View File
Binary file not shown.
+22
View File
@@ -322,3 +322,25 @@ jobs:
echo "::error::Required CI checks not green — refusing to publish required-checks success"
echo "lint=${{ needs.lint.result }} unit-tests=${{ needs['unit-tests'].result }} canary=${{ needs.canary.result }} best-practices=${{ needs['best-practices'].result }} documentation=${{ needs.documentation.result }} type-check=${{ needs['type-check'].result }} gitea-offline=${{ needs['gitea-offline'].result }}"
exit 1
notify-event-bridge:
# T16 §16C: post the CI completion WAKE to the local event bridge so
# event-driven waiters (deploy.yml → ci_event_wait.py) resume on the
# event instead of sleep-polling. ALWAYS runs (success or failure —
# a red CI must wake the waiter just as loudly as a green one). The
# wake is best-effort and never decides the verdict: the waiter
# re-reads the canonical Gitea state via deploy_qualify.py.
name: CI Pipeline / notify-event-bridge
runs-on: rune-ci-desktop
needs: [required-checks]
if: always()
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
with:
fetch-depth: 1
- name: Wake event bridge (wake only — never completion truth)
env:
CI_SHA: ${{ gitea.sha }}
CI_RESULT: ${{ needs.required-checks.result }}
RUNE_EVENT_BRIDGE_URL: ${{ vars.RUNE_EVENT_BRIDGE_URL || 'http://127.0.0.1:8791' }}
run: python3 scripts/notify_event_bridge.py
+9 -15
View File
@@ -19,7 +19,13 @@ jobs:
with:
fetch-depth: 0
- name: Wait for CI qualification on this exact SHA
- name: Wait for CI qualification on this exact SHA (event-driven, T16 §16C)
# T16 §16C/§16D: the old sleep-30 → query → repeat loop is gone.
# The waiter registers a correlation on the W/R event bridge and
# yields in select() until ci.yml's notify-event-bridge job posts
# the completion WAKE; the verdict is then re-read from canonical
# Gitea (deploy_qualify.py). The 900s deadline is a FAILURE
# DETECTOR (lost-event recovery), never the scheduler.
env:
GITEA_SERVER: ${{ gitea.server_url }}
GITEA_REPO: ${{ gitea.repository }}
@@ -27,20 +33,8 @@ jobs:
run: |
DEPLOY_SHA=$(git rev-parse HEAD)
export DEPLOY_SHA
echo "Qualifying $DEPLOY_SHA"
DEADLINE=$((SECONDS + 900))
while [ $SECONDS -lt $DEADLINE ]; do
STATUS=$(python3 scripts/deploy_qualify.py)
echo "required-checks: $STATUS"
if [ "$STATUS" = "success" ]; then exit 0; fi
if [ "$STATUS" = "failure" ]; then
echo "::error::CI failed on $DEPLOY_SHA — deployment refused"
exit 1
fi
sleep 30
done
echo "::error::CI qualification timeout on $DEPLOY_SHA — deployment refused"
exit 1
echo "Qualifying $DEPLOY_SHA (event-driven)"
python3 scripts/ci_event_wait.py --sha "$DEPLOY_SHA" --deadline 900
- name: Stage, verify, promote (monotonic)
env:
+1 -1
View File
@@ -1,6 +1,6 @@
# Rune
**Current release: v0.66.59**
**Current release: v0.66.63**
**Rune is a persistent personal agent designed to accumulate experience, develop reusable competence, and remain recognisably the same agent even when its underlying foundation model changes.**
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "rune"
version = "0.66.59"
version = "0.66.63"
description = "Rune — a stateful coding assistant with persistent memory, orchestration, and TUI"
requires-python = ">=3.10"
+224
View File
@@ -0,0 +1,224 @@
#!/usr/bin/env python3
"""ci_event_wait.py — T16 §16C/§16D: event-driven CI qualification wait.
Replaces the deploy.yml sleep-30 → query → repeat polling loop with the
EXISTING W/R event bridge (src/rune/event_wait.py — the SAME primitive the
builder adapter uses; §16E: one bridge, adapters differ only in identity
and authority probe).
Flow (mandate §16C):
push / CI workflow starts
↓
register correlation (provider="gitea", event_type="ci_completed",
run_ref="ci-<sha>", sha=<sha>)
↓
WAITING_FOR_EVENT (await_resume_event: select() on FIFO, no floor)
↓
ci.yml final job POSTs /process-event (the WAKE)
↓
daemon matches → resume_cmd fires → FIFO token → controller resumes
↓
canonical Gitea state INDEPENDENTLY re-read (deploy_qualify.py verdict)
↓
verdict
The event is a WAKE, never truth. The final verdict always comes from
deploy_qualify.py (required-checks context + exact-SHA ci.yml run, both
legs). Watchdog deadline is a FAILURE DETECTOR, never the scheduler:
- event arrives → authority re-read; pending → bounded re-arm until the
overall deadline (ack-window semantics, same as the daemon's wake sweep).
- event lost/suppressed → watchdog expires → ONE-SHOT authority query:
terminal verdict → recovered truthfully (no invented wait);
CI still running (non-terminal Gitea runs for the SHA) → re-arm;
nothing alive and no verdict → truthful failure (deployment refused).
Exit codes: 0 = qualified (success), 1 = refused (failure/timeout).
Output: one JSON line with the §16D latency-proof timestamps.
"""
from __future__ import annotations
import json
import os
import subprocess
import sys
import time
from pathlib import Path
_SCRIPT_DIR = Path(__file__).resolve().parent
sys.path.insert(0, str(_SCRIPT_DIR.parent / "src"))
from rune import event_wait as ew # noqa: E402
REPO_ROOT = _SCRIPT_DIR.parent
QUALIFY_SCRIPT = _SCRIPT_DIR / "deploy_qualify.py"
def authority_verdict() -> str:
"""ONE-SHOT canonical Gitea verdict via deploy_qualify.py — the only
completion truth. Never parses the event payload."""
proc = subprocess.run(
[sys.executable, str(QUALIFY_SCRIPT)],
capture_output=True, text=True, timeout=120,
env=dict(os.environ, PYTHONPATH=""),
)
out = (proc.stdout or "").strip().splitlines()
return (out[-1] if out else "pending").strip()
def authority_after_wake(max_tries: int = 3, gap_secs: int = 4) -> str:
"""Post-wake authority read. The daemon's own wake sweep uses a
bounded event-triggered retry (4s ×3) for the job-finished-microseconds-
ago ack window; a verdict read here can hit the same window. This is a
bounded schedule, not a polling loop — 3 reads max, then back to the
event wait / watchdog."""
verdict = "pending"
for attempt in range(max_tries):
verdict = authority_verdict()
if verdict != "pending":
return verdict
if attempt < max_tries - 1:
time.sleep(gap_secs)
return verdict
def ci_alive() -> bool:
"""Liveness probe for THIS adapter: any non-terminal Actions run for
the SHA in canonical Gitea (one-shot on watchdog expiry — not a poll).
Import the probe from deploy_qualify to keep ONE API client."""
sys.path.insert(0, str(_SCRIPT_DIR))
import deploy_qualify as dq # noqa: E402 (spec-loaded scripts pattern)
import gitea_auth
server = os.environ.get("GITEA_SERVER", "").rstrip("/")
repo = os.environ.get("GITEA_REPO", "")
sha = os.environ.get("DEPLOY_SHA", "")
if not (server and repo and sha):
return False
token = gitea_auth.get_token() if gitea_auth.has_token() else ""
if not token:
return False
runs = dq._fetch_runs(server, repo, sha, token)
if not runs:
return False
terminal = {"success", "failure", "cancelled", "skipped"}
return any(str(r.get("status", "")).lower() not in terminal
for r in runs)
def run(*, sha: str, deadline_secs: int,
watchdog_secs: int = 300,
daemon_url: str = ew.DEFAULT_DAEMON_URL) -> int:
op = f"ci-{sha}"
t0 = time.time()
# §16Y #10 race fix: the FIFO reader exists BEFORE registration, so a
# wake arriving in the registration window can always be delivered.
fifo_fd = ew.open_fifo_reader(op)
rec = ew.register_process_wait(
op, project="rune",
objective=f"event-driven CI qualification for {sha}",
watched_for="Gitea CI completion event (wake; verdict re-read "
"from canonical Gitea)",
provider="gitea", event_type="ci_completed", sha=sha,
watchdog_secs=min(watchdog_secs, deadline_secs))
registered_at = rec["created_at"]
deadline = time.monotonic() + deadline_secs
verdict = "pending"
mode = "event"
event_at = ""
resumed_at = ""
rearm_reason = ""
try:
while time.monotonic() < deadline:
remaining = max(1, int(deadline - time.monotonic()))
result = ew.await_resume_event(
op, watchdog_secs=min(remaining, watchdog_secs), pgid=None,
fifo_fd=fifo_fd)
if result["kind"] == "event":
event_at = event_at or time.strftime(
"%Y-%m-%dT%H:%M:%SZ", time.gmtime())
resumed_at = time.strftime("%Y-%m-%dT%H:%M:%SZ",
time.gmtime())
# WAKE received — canonical authority decides, never the
# event payload
verdict = authority_after_wake()
if verdict != "pending":
mode = "event"
break
rearm_reason = "event seen; authority pending"
if not ci_alive():
# CI terminal in Gitea but aggregator lagging: one
# final truthful read, then stop
verdict = authority_verdict()
mode = ("event" if verdict != "pending"
else "authority_unresolved")
break
continue
if result["kind"] == "eof":
# writer gone without a token: the DURABLE record decides
rec = ew.operation_wait(op) or {}
if rec.get("status") in ("COMPLETED", "FAILED",
"READY_TO_RESUME", "RESUMING") or \
(rec.get("status") == "RESUME_UNVERIFIED"
and rec.get("process_event_evidence")):
event_at = event_at or time.strftime(
"%Y-%m-%dT%H:%M:%SZ", time.gmtime())
resumed_at = time.strftime("%Y-%m-%dT%H:%M:%SZ",
time.gmtime())
verdict = authority_after_wake()
if verdict != "pending":
mode = "event_recovered_from_durable_record"
break
rearm_reason = "eof without token; durable record pending"
continue
# watchdog expiry: adapter liveness (Gitea runs) decides
# re-arm vs truthful recovery — failure detector, never
# scheduler
if ci_alive():
rearm_reason = "watchdog expiry; CI runs non-terminal " \
"in Gitea — re-arm"
continue
verdict = authority_verdict()
if verdict == "pending":
mode = "watchdog_lost_event_no_verdict"
else:
mode = "watchdog_authority_recovered"
break
finally:
os.close(fifo_fd)
if verdict == "pending":
if mode == "event":
mode = "deadline_pending"
verdict = authority_verdict() # final one-shot truth
verdict_at = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
out = {
"sha": sha,
"registered_at": registered_at,
"event_received_at": event_at,
"controller_resumed_at": resumed_at,
"verdict_at": verdict_at,
"mode": mode,
"rearm_reason": rearm_reason,
"verdict": verdict,
"wall_secs": round(time.time() - t0, 3),
}
print(json.dumps(out))
return 0 if verdict == "success" else 1
def main() -> int:
sha = os.environ.get("DEPLOY_SHA", "")
args = sys.argv[1:]
if "--sha" in args:
sha = args[args.index("--sha") + 1]
deadline = 900
if "--deadline" in args:
deadline = int(args[args.index("--deadline") + 1])
if not sha:
print(json.dumps({"error": "no sha (DEPLOY_SHA or --sha required)"}))
return 1
return run(sha=sha, deadline_secs=deadline)
if __name__ == "__main__":
sys.exit(main())
+59
View File
@@ -0,0 +1,59 @@
#!/usr/bin/env python3
"""notify_event_bridge.py — T16 §16C: ci.yml's final job posts the CI
completion WAKE to the local W/R event bridge (daemon /process-event).
The event is a WAKE, never completion truth: the waiting controller
(ci_event_wait.py) re-reads the canonical Gitea verdict via
deploy_qualify.py after resuming. Best-effort by design — if the bridge
is unreachable the waiter's watchdog still recovers truthfully (that is
the §16D lost-event contract), so a notify failure must NEVER fail CI.
"""
from __future__ import annotations
import json
import os
import sys
import time
import urllib.request
BRIDGE_URL = os.environ.get("RUNE_EVENT_BRIDGE_URL",
"http://127.0.0.1:8791")
def main() -> int:
sha = os.environ.get("CI_SHA", "")
result = os.environ.get("CI_RESULT", "") # needs.<job>.result
if not sha:
print("notify-event-bridge: no CI_SHA — nothing to wake", flush=True)
return 0
payload = {
"operation_id": f"ci-{sha}",
"provider": "gitea",
"event_type": "ci_completed",
"sha": sha,
# exit_code is informational only; the wake is not the verdict
"exit_code": 0 if result == "success" else 1,
"completed_at": time.time(),
}
for attempt in range(2):
try:
req = urllib.request.Request(
BRIDGE_URL + "/process-event",
data=json.dumps(payload).encode(),
headers={"Content-Type": "application/json"})
with urllib.request.urlopen(req, timeout=5) as r:
resp = json.loads(r.read())
print(f"notify-event-bridge: woke bridge "
f"(matched={resp.get('matched')})", flush=True)
return 0
except Exception as exc: # noqa: BLE001 — wake is best-effort by design; failure is logged and the waiter's watchdog recovery covers a lost wake (§16D contract)
if attempt == 0:
time.sleep(2)
else:
print(f"notify-event-bridge: bridge unreachable ({exc}); "
f"watchdog recovery covers a lost wake", flush=True)
return 0
if __name__ == "__main__":
sys.exit(main())
+1 -1
View File
@@ -6,7 +6,7 @@ This is the main entry point for the Rune program.
import json
from pathlib import Path
__version__ = "0.66.59"
__version__ = "0.66.63"
# One build identity (handoff 2026-09-10 §4): the semantic version above is
# the repo-level truth (auto-bumped at merge); the DEPLOYED revision (exact
+1 -1
View File
@@ -231,7 +231,7 @@ def main() -> int:
)
check(
"Rune minor version advertises provisioning and restore verification",
'__version__ = "0.66.59"' in rune_version,
'__version__ = "0.66.63"' in rune_version,
failures,
)
# #169 policy update: auto-bump on merge is sanctioned (Nick directive:
+113 -49
View File
@@ -47,6 +47,7 @@ waits against Gitea (the event is the authority).
from __future__ import annotations
import importlib.util
import json
import os
import select
@@ -55,6 +56,7 @@ import subprocess
import sys
import time
from pathlib import Path
from typing import Any
DEFAULT_WR_ROOT = Path.home() / ".local" / "share" / "mimo-wait-resume"
FALLBACK_WR_ROOT = Path("/home/nick/.local/share/mimo-wait-resume")
@@ -101,55 +103,82 @@ def wr_root() -> Path:
return FALLBACK_WR_ROOT
def _load_waitstore():
# waitstore.py lives in the DAEMON CODE dir. In production WR_ROOT is
# the daemon dir (code + data together); in isolated test environments
# WR_ROOT is a data-only dir, so the code dir is resolved separately:
# RUNE_WR_CODE env → WR_ROOT → known install locations.
for cand in (os.environ.get("RUNE_WR_CODE"), str(wr_root()),
# waitstore instances cached per data root (isolated tests use per-test
# tmp roots; the daemon uses its install dir). Loading via importlib.util
# keeps the dynamic import pyright-clean — no unresolvable-import warning.
_waitstore_cache: dict[str, Any] = {}
def _load_waitstore() -> Any:
"""Load the bridge's OWN waitstore module (never a copy), bound to
the adapter's data root. waitstore.py lives in the DAEMON CODE dir;
in production WR_ROOT is that dir (code + data together); in isolated
test environments WR_ROOT is a data-only dir, so the code dir is
resolved separately: RUNE_WR_CODE env → WR_ROOT → known installs."""
root = wr_root()
# waitstore binds its DATA dir from WR_ROOT at import time. Pin
# WR_ROOT to the adapter's authority so adapter-side and daemon-side
# state agree — otherwise a mismatch silently splits the wait store
# from the event matcher and continuation dies as a watchdog timeout.
os.environ["WR_ROOT"] = str(root)
cached = _waitstore_cache.get(str(root))
if cached is not None:
return cached
code_dir = None
for cand in (os.environ.get("RUNE_WR_CODE"), str(root),
str(FALLBACK_WR_ROOT)):
if cand and (Path(cand) / "waitstore.py").is_file():
if cand not in sys.path:
sys.path.insert(0, cand)
code_dir = Path(cand)
break
# waitstore binds its DATA dir from WR_ROOT at import time. The
# adapter's authority for the data dir is RUNE_WR_ROOT (wr_root());
# pin WR_ROOT to match so adapter-side and daemon-side state agree —
# otherwise a mismatch silently splits the wait store from the event
# matcher and continuation dies as a watchdog timeout.
os.environ["WR_ROOT"] = str(wr_root())
import waitstore # the bridge's own durable store — NOT a copy
# if the data root CHANGED since the module bound it (isolated tests
# per-test tmp roots; daemon restarts), re-bind by reloading — the
# module-level WAITS/EVENTS paths are derived from WR_ROOT at import
import importlib
root = wr_root()
if str(getattr(waitstore, "WAITS", "").parent) != str(root):
waitstore = importlib.reload(waitstore)
return waitstore
if code_dir is None:
raise RuntimeError(
"waitstore.py not found in known W/R roots "
"(RUNE_WR_CODE, WR_ROOT, ~/.local/share/mimo-wait-resume)")
spec = importlib.util.spec_from_file_location(
"waitstore", str(code_dir / "waitstore.py"))
if spec is None or spec.loader is None:
raise RuntimeError(f"cannot load waitstore from {code_dir}")
mod = importlib.util.module_from_spec(spec)
spec.loader.exec_module(mod)
_waitstore_cache[str(root)] = mod
return mod
def register_process_wait(operation_id: str, *, contract_id: str = "",
project: str = "", objective: str = "",
watched_for: str = "",
provider: str = "builder",
event_type: str = "builder_exit",
sha: str = "",
watchdog_secs: int | None = None) -> dict:
"""Register a durable PROCESS_EVENT wait for one operation. The
per-wait resume_cmd writes a resume token to the operation's FIFO —
the bridge can resume a Python controller, not just MiMo sessions."""
the bridge can resume a Python controller, not just MiMo sessions.
provider/event_type are the adapter's event identity on the shared
bridge (16E: ONE wait primitive; builder / Gitea-CI / future adapters
differ ONLY in these identity fields + their liveness probe — the
registration, FIFO rendezvous, matching, idempotency, restart
recovery, and watchdog semantics are the same code for all).
Defaults preserve the T15 builder semantics."""
ws = _load_waitstore()
fifo = fifo_path(operation_id)
fifo.parent.mkdir(parents=True, exist_ok=True)
if fifo.exists():
fifo.unlink()
os.mkfifo(fifo)
# Do NOT unlink+recreate an existing fifo: a caller that opened its
# read end before registering (open_fifo_reader, §16Y #10 race fix)
# would be orphaned onto the old inode while the daemon's resume
# writer opened the new one. The rendezvous is create-if-missing.
if not fifo.exists():
os.mkfifo(fifo)
writer = ("import sys,json,time,os;"
"fd=os.open(sys.argv[1], os.O_WRONLY | os.O_NONBLOCK);"
"os.write(fd, json.dumps({'wait_id': sys.argv[2],"
" 'resumed_at': time.time()}).encode());os.close(fd)")
rec = ws.new_wait(
session_id=f"controller-{operation_id}",
provider="builder",
event_type="builder_exit",
provider=provider,
event_type=event_type,
sha=sha,
repo=project or "construction",
run_ref=operation_id,
expected="success",
@@ -175,6 +204,23 @@ def fifo_path(operation_id: str) -> Path:
return wr_root() / "resumes" / f"{operation_id}.fifo"
def open_fifo_reader(operation_id: str) -> int:
"""Open the operation's FIFO read end (O_NONBLOCK) BEFORE the wait is
registered. §16Y adversarial #10: an event that arrives after
registration but before the controller enters select() would find no
reader — the resume write fails (ENXIO) and the wait lands
RESUME_UNVERIFIED while the controller sleeps until the watchdog.
Holding the reader open across the registration window closes that
race for every adapter; the fd is passed to await_resume_event and
closed by the caller."""
fifo = fifo_path(operation_id)
fifo.parent.mkdir(parents=True, exist_ok=True)
if fifo.exists():
fifo.unlink()
os.mkfifo(fifo)
return os.open(fifo, os.O_RDONLY | os.O_NONBLOCK)
def run_builder_under_events(builder_cmd: list[str], operation_id: str, *,
daemon_url: str = DEFAULT_DAEMON_URL):
"""Launch the REAL builder under the event notifier (separate process
@@ -246,10 +292,18 @@ def liveness(operation_id: str, pgid: int | None = None) -> bool:
def await_resume_event(operation_id: str, *, watchdog_secs: int,
pgid: int | None = None,
max_rearms: int = 20) -> dict:
max_rearms: int = 20,
fifo_fd: int | None = None) -> dict:
"""YIELD until the bridge resumes this controller. Blocks in select()
on the FIFO — no sleep, no polling.
fifo_fd: a pre-opened read end from open_fifo_reader() (§16Y #10 race
fix — the fd must already exist when the wait becomes registered).
Caller-owned fds are never closed here; on EOF (writer gone, no
token) the call returns {"kind": "eof"} instead of busy-looping on an
always-readable fd — the caller falls back to the durable waitstore
record via resume_or_recover/operation_wait.
Watchdog semantics (§15D): the deadline is a FAILURE DETECTOR, never
the scheduler. On expiry, liveness is inspected ONCE (pgid membership,
then operation-specific cmdline evidence — real builders regroup their
@@ -258,17 +312,21 @@ def await_resume_event(operation_id: str, *, watchdog_secs: int,
build outlived a single 900s deadline, and node regrouping hid the
processes from the pgid scan — neither is a failure of continuation.
"""
fifo = fifo_path(operation_id)
# idempotent re-arm: a prior await consumed+unlinked the fifo; the
# rendezvous point is (re)created so watchdog re-arm and recovery
# paths can wait again on the same operation
fifo.parent.mkdir(parents=True, exist_ok=True)
if not fifo.exists():
os.mkfifo(fifo)
for arm in range(max_rearms + 1):
deadline = time.monotonic() + watchdog_secs
owned_fd = fifo_fd is None
if owned_fd:
fifo = fifo_path(operation_id)
# idempotent re-arm: a prior await consumed+unlinked the fifo; the
# rendezvous point is (re)created so watchdog re-arm and recovery
# paths can wait again on the same operation
fifo.parent.mkdir(parents=True, exist_ok=True)
if not fifo.exists():
os.mkfifo(fifo)
fd = os.open(fifo, os.O_RDONLY | os.O_NONBLOCK)
try:
else:
fd = fifo_fd
try:
for arm in range(max_rearms + 1):
deadline = time.monotonic() + watchdog_secs
while True:
remaining = deadline - time.monotonic()
if remaining <= 0:
@@ -278,7 +336,12 @@ def await_resume_event(operation_id: str, *, watchdog_secs: int,
continue
data = os.read(fd, 4096)
if not data:
continue
if owned_fd:
continue
# caller-owned fd at EOF: writer is gone without
# delivering a token — never a busy loop; the durable
# record decides what actually happened
return {"kind": "eof"}
try:
token = json.loads(data.decode())
except ValueError:
@@ -287,15 +350,16 @@ def await_resume_event(operation_id: str, *, watchdog_secs: int,
rec = ws.load(token.get("wait_id", ""))
return {"kind": "event", "wait": rec,
"resumed_at": token.get("resumed_at")}
finally:
# deadline expired: liveness decides re-arm vs PROCESS_LOST
alive = liveness(operation_id, pgid)
if not alive:
return {"kind": "watchdog", "alive": False,
"rearms": arm}
return {"kind": "watchdog", "alive": liveness(operation_id, pgid),
"rearms": max_rearms}
finally:
if owned_fd:
os.close(fd)
# deadline expired: liveness decides re-arm vs PROCESS_LOST
alive = liveness(operation_id, pgid)
if not alive:
return {"kind": "watchdog", "alive": False,
"rearms": arm}
return {"kind": "watchdog", "alive": liveness(operation_id, pgid),
"rearms": max_rearms}
def operation_wait(operation_id: str) -> dict | None:
+2 -2
View File
@@ -8,7 +8,7 @@ mode system, subject header, shared memory and identity.
This is not an interface. This is how I see, speak, and interact with the world.
Version: v0.66.59
Version: v0.66.63
Features: 41 (24 original + 17 new native commands)
Usage:
@@ -1856,7 +1856,7 @@ class RuneApp(App):
# Show version info
self.chat_channel.add_message(
"system",
f"[dim]Rune v0.66.59 | Textual TUI | {len(loaded)} native modules[/dim]",
f"[dim]Rune v0.66.63 | Textual TUI | {len(loaded)} native modules[/dim]",
)
# Auto-show night mode suggestion if evening/night hours
+27
View File
@@ -12,12 +12,39 @@ override via monkeypatch.setenv, which wins over this fixture.
"""
import os
import sys
from pathlib import Path
import pytest
# W/R bridge code dir (daemon/waitstore live out-of-repo, own snapshot).
# Env-overridable so runner users without $HOME access can point at an
# installed copy — T15 legacy hardcode, resolved T16.
WR_CODE_DIR = Path(os.environ.get(
"RUNE_WR_CODE", "/home/nick/.local/share/mimo-wait-resume"))
if str(WR_CODE_DIR) not in sys.path:
sys.path.insert(0, str(WR_CODE_DIR))
TESTS_DIR = Path(__file__).resolve().parent
REPO_ROOT = TESTS_DIR.parent
if str(REPO_ROOT) not in sys.path:
sys.path.insert(0, str(REPO_ROOT))
@pytest.fixture(autouse=True)
def _isolate_construction_store(tmp_path, monkeypatch):
monkeypatch.setenv("RUNE_CONSTRUCTION_STORE",
str(tmp_path / ".rune" / "scaffold"))
yield
@pytest.fixture()
def wr_daemon(tmp_path, monkeypatch):
"""Real W/R daemon subprocess on an isolated WR_ROOT + ephemeral port.
Shared by test_event_wait.py (T15) and test_ci_event_wait.py (T16)."""
monkeypatch.setenv("RUNE_WR_ROOT", str(tmp_path)) # adapter binds here
monkeypatch.setenv("RUNE_WR_CODE", str(WR_CODE_DIR))
from tests.test_event_wait import Daemon
d = Daemon(tmp_path)
yield d
d.stop()
+211
View File
@@ -0,0 +1,211 @@
"""test_ci_event_wait.py — T16 §16C/§16D/§16E: event-driven CI
qualification wait over the EXISTING W/R bridge.
Same disposable harness as test_event_wait.py (real daemon subprocess on
an isolated WR_ROOT + ephemeral port; Daemon fixture imported — 16W rule:
no duplicated test helpers).
Proofs:
16C event wake — CI-completion event (provider=gitea,
event_type=ci_completed, correlated by sha) resumes
the waiting controller; the EVENT payload is NOT
truth — the verdict is re-read from canonical Gitea
authority (monkeypatched here; production = deploy_
qualify.py).
16C event≠truth — event claims success, authority says failure →
deployment refused (payload never decides).
16D no floor — event → resume latency is event-driven (sub-second),
not a fixed sleep floor.
16D lost event — event suppressed → watchdog expiry → ONE-SHOT
authority query → truthful recovery (or truthful
failure when authority has no verdict). Watchdog is
the failure detector, never the scheduler.
16D re-arm — watchdog expiry while CI runs are non-terminal in
Gitea → re-arm, not failure; a later event resumes.
16E one primitive— builder and gitea waits share the SAME bridge
machinery; identities select the matching events
(a gitea event never resumes a builder wait and
vice versa).
"""
from __future__ import annotations
import importlib.util
import json
import sys
import threading
import time
from pathlib import Path
import pytest
REPO = Path(__file__).resolve().parent.parent
sys.path.insert(0, str(REPO / "src"))
from rune import event_wait as ew # noqa: E402
from tests.test_event_wait import Daemon # noqa: E402 (shared fixture)
def _load_ci_wait_module():
spec = importlib.util.spec_from_file_location(
"ci_event_wait", str(REPO / "scripts" / "ci_event_wait.py"))
mod = importlib.util.module_from_spec(spec)
spec.loader.exec_module(mod)
return mod
CI_SHA = "e" * 40
def _post_when_registered(wr_daemon: Daemon, op: str, payload: dict,
timeout: float = 10.0):
"""Thread body: wait for the wait record to exist, then post the wake
event. The event POST is best-effort (tests assert on outcomes)."""
def go():
deadline = time.time() + timeout
while time.time() < deadline:
if ew.operation_wait(op) is not None:
break
time.sleep(0.05)
wr_daemon.post_event(payload) # loud: a failed wake must fail the test
t = threading.Thread(target=go, daemon=True)
t.start()
return t
def test_16c_event_wake_resumes_and_authority_decides(wr_daemon, monkeypatch):
mod = _load_ci_wait_module()
monkeypatch.setattr(mod, "authority_verdict", lambda: "success")
monkeypatch.setattr(mod, "ci_alive", lambda: False)
payload = {"operation_id": f"ci-{CI_SHA}", "provider": "gitea",
"event_type": "ci_completed", "sha": CI_SHA, "exit_code": 0,
"completed_at": time.time()}
_post_when_registered(wr_daemon, f"ci-{CI_SHA}", payload)
t0 = time.monotonic()
rc = mod.run(sha=CI_SHA, deadline_secs=25, daemon_url=wr_daemon.url())
wall = time.monotonic() - t0
# rc==0 alone is NOT proof: watchdog recovery also returns 0. The wake
# must actually have driven the resume (fast), not the watchdog.
assert rc == 0
assert wall < 15, f"wake-resume took {wall:.1f}s — event never arrived"
rec = ew.operation_wait(f"ci-{CI_SHA}")
assert rec is not None and rec["provider"] == "gitea"
assert rec["wait_kind"] == "PROCESS_EVENT"
def test_16c_event_payload_is_never_truth(wr_daemon, monkeypatch):
mod = _load_ci_wait_module()
# the event says success (exit_code 0); canonical authority says the
# CI is RED — the payload must never decide the verdict
monkeypatch.setattr(mod, "authority_verdict", lambda: "failure")
monkeypatch.setattr(mod, "ci_alive", lambda: False)
payload = {"operation_id": f"ci-{CI_SHA}", "provider": "gitea",
"event_type": "ci_completed", "sha": CI_SHA, "exit_code": 0,
"completed_at": time.time()}
_post_when_registered(wr_daemon, f"ci-{CI_SHA}", payload)
rc = mod.run(sha=CI_SHA, deadline_secs=25, daemon_url=wr_daemon.url())
assert rc == 1 # refused, despite the wake saying 0
def test_16d_event_resume_has_no_sleep_floor(wr_daemon, monkeypatch):
mod = _load_ci_wait_module()
monkeypatch.setattr(mod, "authority_verdict", lambda: "success")
monkeypatch.setattr(mod, "ci_alive", lambda: False)
payload = {"operation_id": f"ci-{CI_SHA}", "provider": "gitea",
"event_type": "ci_completed", "sha": CI_SHA, "exit_code": 0,
"completed_at": time.time()}
_post_when_registered(wr_daemon, f"ci-{CI_SHA}", payload)
t0 = time.monotonic()
rc = mod.run(sha=CI_SHA, deadline_secs=25, daemon_url=wr_daemon.url())
wall = time.monotonic() - t0
# a would-be 30s polling floor is NOT waited: registration + event +
# verdict must be seconds, not the old sleep-30 loop quantum
assert rc == 0
assert wall < 15, f"event-driven wait took {wall:.1f}s — floor crept back"
def test_16d_lost_event_watchdog_recovers_from_authority(
wr_daemon, monkeypatch):
mod = _load_ci_wait_module()
# NO event is ever posted (suppressed/lost); authority IS resolvable
monkeypatch.setattr(mod, "authority_verdict", lambda: "success")
monkeypatch.setattr(mod, "ci_alive", lambda: False)
rc = mod.run(sha=CI_SHA, deadline_secs=4, daemon_url=wr_daemon.url())
assert rc == 0
# truthful recovery, not invented success: the record shows the
# watchdog (failure detector) path, and the verdict came from authority
rec = ew.operation_wait(f"ci-{CI_SHA}")
assert rec is not None
def test_16d_lost_event_with_no_verdict_fails_truthfully(
wr_daemon, monkeypatch):
mod = _load_ci_wait_module()
monkeypatch.setattr(mod, "authority_verdict", lambda: "pending")
monkeypatch.setattr(mod, "ci_alive", lambda: False)
rc = mod.run(sha=CI_SHA, deadline_secs=4, daemon_url=wr_daemon.url())
assert rc == 1 # deployment refused — timeout is failure detection
def test_16d_ci_alive_rearm_then_event(wr_daemon, monkeypatch):
mod = _load_ci_wait_module()
monkeypatch.setattr(mod, "authority_verdict", lambda: "success")
# first watchdog expiry (t=3s): CI runs still non-terminal → re-arm,
# not fail; the event arrives only AFTER that expiry (t≥4s)
alive = {"n": 0}
def flaky_alive():
alive["n"] += 1
return alive["n"] <= 1 # alive on first expiry, gone after event
monkeypatch.setattr(mod, "ci_alive", flaky_alive)
def post_late():
deadline = time.time() + 10
while time.time() < deadline:
if ew.operation_wait(f"ci-{CI_SHA}") is not None:
break
time.sleep(0.05)
time.sleep(4.0) # strictly after the first watchdog expiry (3s)
wr_daemon.post_event({"operation_id": f"ci-{CI_SHA}",
"provider": "gitea",
"event_type": "ci_completed",
"sha": CI_SHA, "exit_code": 0,
"completed_at": time.time()})
threading.Thread(target=post_late, daemon=True).start()
rc = mod.run(sha=CI_SHA, deadline_secs=15, watchdog_secs=3,
daemon_url=wr_daemon.url())
assert rc == 0
assert alive["n"] >= 1 # the re-arm actually happened
def test_16e_one_bridge_two_adapters_identity_selects(wr_daemon):
"""16E: builder and gitea waits share the same primitive; only the
identity fields differ, and identity selects the matching event."""
# builder-style wait
b = ew.register_process_wait("op-builder-1", watchdog_secs=60)
# gitea-style wait
g = ew.register_process_wait(f"ci-{CI_SHA}", provider="gitea",
event_type="ci_completed", sha=CI_SHA,
watchdog_secs=60)
assert b["provider"] == "builder" and g["provider"] == "gitea"
# a gitea CI event must NOT resume the builder wait...
wr_daemon.post_event({"operation_id": f"ci-{CI_SHA}", "provider": "gitea",
"event_type": "ci_completed", "sha": CI_SHA,
"exit_code": 0})
brec = ew.operation_wait("op-builder-1")
grec = ew.operation_wait(f"ci-{CI_SHA}")
assert brec["status"] == "WAITING_EXTERNAL"
assert grec["status"] in ("READY_TO_RESUME", "RESUME_UNVERIFIED",
"COMPLETED")
# ...and a builder event must not resume a (fresh) gitea wait
g2 = ew.register_process_wait(f"ci-{'f' * 40}", provider="gitea",
event_type="ci_completed", sha="f" * 40,
watchdog_secs=60)
wr_daemon.post_event({"operation_id": "op-builder-1", "exit_code": 0})
assert ew.operation_wait(f"ci-{'f' * 40}")["status"] == "WAITING_EXTERNAL"
assert ew.operation_wait("op-builder-1")["status"] in (
"READY_TO_RESUME", "RESUME_UNVERIFIED", "COMPLETED")
+17 -8
View File
@@ -49,7 +49,19 @@ from rune import event_wait as ew # noqa: E402
class Daemon:
"""Real daemon subprocess on an isolated WR_ROOT + ephemeral port."""
def __init__(self, tmp_root: Path, port: int):
def __init__(self, tmp_root: Path, port: int | None = None):
# Ephemeral port (not a fixed one): the CI runner executes the
# FULL suite in one process, and a fixed port collides with
# TIME_WAIT sockets from earlier tests in the same file or with
# parallel jobs on the same runner. 15Z CI failure: the latency
# test's daemon failed to bind 8793 (stale socket), the notifier's
# event POSTs went nowhere, and the watchdog correctly reported
# PROCESS_LOST — the test harness was at fault, not the bridge.
if port is None:
import socket as _socket
with _socket.socket() as _s:
_s.bind(("127.0.0.1", 0))
port = _s.getsockname()[1]
self.root, self.port = tmp_root, port
env = dict(os.environ,
WR_ROOT=str(tmp_root),
@@ -93,14 +105,11 @@ class Daemon:
return json.loads(r.read())
# T16 §16E: the daemon fixture moved to tests/conftest.py (wr_daemon) so
# test_ci_event_wait.py shares ONE harness — no duplicated test helpers.
@pytest.fixture()
def daemon(tmp_path, monkeypatch):
monkeypatch.setenv("RUNE_WR_ROOT", str(tmp_path)) # adapter binds here
monkeypatch.setenv("RUNE_WR_CODE", str(WR))
monkeypatch.setenv("RUNE_WR_ROOT", str(tmp_path))
d = Daemon(tmp_path, port=8793)
yield d
d.stop()
def daemon(wr_daemon):
return wr_daemon
# real fake-builder: sleeps then exits — cheap but genuinely a process
+19 -1
View File
@@ -129,10 +129,28 @@ def test_15k_build_manager_has_no_parallel_construction_state(tmp_path,
monkeypatch):
"""§15K inventory: BuildManager (the /build vehicle) creates its own
phase state but must NOT create project scaffolds/contracts. The
canonical construction record comes only from the send_task hook."""
canonical construction record comes only from the send_task hook.
BuildManager.start_build shells out to `git worktree add` against a
HARDCODED production repo path (/mnt/workdrive/rune) — that is real
VCS mutation and has no place in a unit test (it fails on CI with
dubious-ownership, and would pollute the real repo locally). This
test therefore inspects the manager's state machine WITHOUT invoking
the worktree-creating path: construction-state separation is a
property of where BuildManager writes, not of the git operation."""
from rune import build_mode
from rune.build_mode import BuildManager
monkeypatch.chdir(tmp_path) # NOT bare os.chdir — must not leak
monkeypatch.setattr(build_mode, "BUILDS_DIR",
str(tmp_path / ".rune" / "work" / "builds"))
monkeypatch.setattr(build_mode, "WORKTREE_DIR",
str(tmp_path / ".rune" / "worktrees"))
bm = BuildManager()
# start_build's worktree step is a real VCS mutation — stub it; the
# property under test (no canonical construction state) is orthogonal
monkeypatch.setattr(
BuildManager, "create_worktree",
lambda self: setattr(self.state, "worktree_path", "stubbed") or "stubbed")
build_id = bm.start_build("build a thing")
# BuildManager's own state exists...
assert bm.state.description == "build a thing"