fix(pipeline): merge-robot authenticates via ROBOT_TOKEN secret — GITEA_TOKEN is the reserved read-only auto token (root cause of 18 consecutive loud failures)

.gitea/workflows/merge-robot.yml

@@ -8,6 +8,7 @@ name: Merge robot
 on:
   schedule:
     - cron: '23 * * * *'
+  workflow_dispatch: {}
   workflow_run:
     workflows: ["CI Pipeline", "Auto-version and Release"]
     types: [completed]
@@ -25,53 +26,9 @@ jobs:
   merge-green:
     runs-on: rune-ci-desktop
     steps:
-      - name: Merge PRs whose protected checks are green
+      - name: Merge PRs whose protected checks are green (fail-loud)
         env:
-          GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
+          GITEA_TOKEN: ${{ secrets.ROBOT_TOKEN }}  # conductor token; GITEA_TOKEN is reserved (auto token, read-only)
           SERVER: ${{ gitea.server_url }}
         run: |
-          python3 - <<'PY'
-          import json, os, time, urllib.request
-
-          TOKEN = os.environ["GITEA_TOKEN"]
-          BASE = os.environ["SERVER"].rstrip("/") + "/api/v1/repos/" + os.environ["OWNER"] + "/" + os.environ["REPO"]
-
-          def api(path, method="GET", data=None):
-              req = urllib.request.Request(
-                  BASE + path,
-                  data=json.dumps(data).encode() if data is not None else None,
-                  headers={"Authorization": "token " + TOKEN, "Content-Type": "application/json"},
-                  method=method)
-              try:
-                  with urllib.request.urlopen(req, timeout=30) as r:
-                      body = r.read()
-                      return r.status, (json.loads(body) if body else {})
-              except urllib.error.HTTPError as e:
-                  return e.code, {"_body": e.read().decode("utf-8", "replace")[:300]}
-
-          status, prs = api("/pulls?state=open&limit=50")
-          if status != 200:
-              print(f"list PRs failed: {status}"); raise SystemExit(0)
-          merged = []
-          for pr in prs:
-              n = pr["number"]
-              sha = pr["head"]["sha"]
-              st, cs = api(f"/commits/{sha}/status")
-              state = cs.get("state")
-              if state != "success":
-                  print(f"#{n}: status={state} — skip")
-                  continue
-              # mergeable guard: skip conflicts instead of hammering
-              st, detail = api(f"/pulls/{n}")
-              if not detail.get("mergeable", False):
-                  print(f"#{n}: not mergeable (conflict?) — skip")
-                  continue
-              st, resp = api(f"/pulls/{n}/merge", "POST", {"Do": "merge"})
-              if st in (200, 405):
-                  print(f"#{n}: merged ({st})")
-                  merged.append(n)
-              else:
-                  print(f"#{n}: merge failed {st}: {str(resp)[:200]}")
-              time.sleep(2)
-          print(f"done — merged {len(merged)} PRs: {merged}")
-          PY
+          python3 scripts/merge_robot.py "$GITEA_TOKEN" "$SERVER/api/v1/repos/conductor/rune"

docs/umbrella-contract-design.md

@@ -0,0 +1,84 @@
+# Umbrella system → henxels contracts + retroactive governance — design for confirmation
+
+## Part 1: Retroactive application (ready to execute now)
+
+The survey found six ungoverned surfaces. Proposed henxels rules, in the tool's
+real schema, each following an existing pattern:
+
+| Surface | Finding | Proposed henxel (keyed, real schema) |
+|---|---|---|
+| `shared/*.py` (24 files) | ~100% near-duplicates of `src/rune/` modules (the mtime-copy sync is the cause) | `changed_with`: when `shared/*.py` changes, `src/rune/` mirror must change too — **or better (Part 2) the dupes die** |
+| `research/**` (63 files, 16 dirs) | No index; unlinked pages | `referenced_in: docs/research-index.md` + `links_resolve` — orphan research is tracked |
+| `memory/` (runtime state tracked in git!) | `journal/events.jsonl` + runtime JSON in the repo — runtime data in version control | `forbidden_files: [./memory/journal/events.jsonl, ./memory/index/*.json]` after a one-time extraction (runtime state belongs to the deployed agent, not the codebase) |
+| `backups/*.bak` (3 files) | Build backups committed | `forbidden_files: [./backups/**]` (git history IS the backup) |
+| `specs/`, `reports/` | Type-ungoverned | `allowed_filetypes: [.md]` (matches the plans rule) |
+| `survey/` (7 files) | Ungoverned | fold into research indexing or `allowed_filetypes: [.md, .yaml]` |
+
+All are mechanical; the `memory/` extraction is the only one touching runtime —
+I'd do it in the same PR with the agent's own journal-preservation discipline
+(backup branch first).
+
+## Part 2: Umbrella system → contracts (needs your confirmation on shape)
+
+### What the umbrella system is (verified)
+
+`shared/umbrella_update.py`: a hand-maintained manifest (`UMBRELLAS` dict) of six
+functional umbrellas — **memory, governance, identity, growth, perception,
+infrastructure** — each listing shared/scripts/canaries components plus a
+`depends_on` DAG (memory→governance→identity; growth→memory+governance). Purpose:
+say "update memory" and the system knows the affected files, runs the umbrella's
+canaries, syncs shared→deploy targets by mtime, and commits.
+
+### What's wrong (verified defects)
+
+1. **Manifest drift**: `memory_selector.py` and every post-manifest module is in
+   NO umbrella; it references canaries (`canary_unbounded.py`,
+   `canary_drift_wiring.py`) that don't exist in the repo.
+2. **Two-sources-of-truth by design**: `sync_shared_to_scripts()` mtime-copies
+   `shared/` → `~/.config/opencode/scripts/` + the old TUI dir. This is why
+   henxels flags ~100% duplicates. The repo and the harness both "own" copies.
+3. **Hardcoded harness paths** (`~/.config/opencode/...`) — not repo-portable,
+   dead on the runner.
+
+### Proposed design: three contracts, one removal
+
+**Contract A — `umbrella_manifest_is_complete`** (new custom statement):
+every `src/rune/*.py` and `shared/*.py` module must appear in the umbrella
+manifest (or the coverage-debt ledger). New modules force an umbrella decision —
+the manifest can never silently drift again. This is the umbrella's real
+invariant: *the system knows what needs updating* — enforced.
+
+**Contract B — `umbrella_canaries_exist`** (new custom statement): every canary
+referenced by the manifest must exist in `src/rune/` and be CI-registered
+(composes with the existing `ci_runs_all_canaries`). Kills defect 1's phantom
+references.
+
+**Contract C — `umbrella_dag_acyclic_and_declared`** (new custom statement):
+parse `UMBRELLAS`, verify `depends_on` is a DAG (no cycles), every dependency
+target exists. Cheap, structural, protects the reflex's core logic.
+
+**The removal (your call):** the mtime-copy sync. Two honest options:
+
+- **Option 1 (recommended): repo-as-source-of-truth.** The repo's `src/rune/` +
+  `shared/` become the ONLY governed copies. The deployed harness gets its
+  scripts from the pinned checkout (the update flow already force-checks-out
+  gitea/master). `sync_shared_to_scripts()` is retired; the
+  near-duplicate `shared/` files either become true imports of `src/rune` or
+  move to the harness (outside the repo). Contract A then governs ONE tree.
+  Cost: the harness-side scripts dir stops being independently editable —
+  which is the point.
+- **Option 2: keep dual-location, gate the sync.** Keep the copies but add a
+  `must_be_in_sync`-style contract: shared/ and its mirrors must be
+  content-identical at commit time (henxels `changed_with` + a checksum
+  statement). Cost: permanent duplication machinery; drift only detectable,
+  never prevented.
+
+### Sequencing
+
+1. Retroactive henxels (Part 1) — one PR, mechanical.
+2. Contracts A+B+C + the manifest modernized in-repo (move `UMBRELLAS` to
+   `config/umbrellas.json`, sourced by the script) — second PR.
+3. Option 1's sync retirement — its own PR, after you confirm.
+
+**Confirm**: Option 1 or 2 for the sync? And does the `memory/` runtime-state
+extraction have your blessing (it touches the live journal)?

scripts/merge_robot.py

@@ -0,0 +1,122 @@
+#!/usr/bin/env python3
+"""merge_robot.py — deterministic merge-on-green for this repo (#203).
+
+Extracted from the inline workflow script so its logic is TESTABLE
+(bug-fix contract, docs/bugfix-contract.md: the silent no-op defect).
+
+Behavior:
+  - GET /user first: verify the token's identity and log it. Scoped-token
+    failures are diagnosable from the job log instead of mysterious.
+  - Scan open PRs; for each: combined status must be success AND mergeable.
+  - Merge refusals (403/405/anything non-(200,405-already-merged)) FAIL THE JOB
+    with the full error body — never print-and-continue. A robot that cannot
+    merge must be visibly broken, not silently idle.
+  - 405 with "behind the base branch" is retried once after a self-update of
+    the head (informational), then fails loudly if still refused.
+  - Every scanned PR produces a decision line: merged / skipped(reason).
+"""
+
+from __future__ import annotations
+
+import json
+import sys
+import time
+import urllib.error
+import urllib.request
+from typing import Any
+
+
+class MergeRefused(Exception):
+    """A merge the robot expected to perform was refused by the forge."""
+
+
+def _api(
+    base: str, token: str, path: str, method: str = "GET", data: Any = None, ok: tuple = (200,)
+) -> tuple[int, Any]:
+    req = urllib.request.Request(
+        base + path,
+        data=json.dumps(data).encode() if data is not None else None,
+        headers={"Authorization": "token " + token, "Content-Type": "application/json"},
+        method=method,
+    )
+    try:
+        with urllib.request.urlopen(req, timeout=30) as r:
+            body = r.read()
+            status = r.status
+    except urllib.error.HTTPError as e:
+        status = e.code
+        body = e.read()
+    except urllib.error.URLError as e:
+        raise MergeRefused(f"forge unreachable at {path}: {e.reason}") from e
+    parsed: Any = json.loads(body) if body else {}
+    if method == "GET" and status not in ok:
+        raise MergeRefused(f"GET {path} -> {status}: {str(parsed)[:300]}")
+    return status, parsed
+
+
+def verify_identity(base: str, token: str) -> str:
+    status, user = _api(base, token, "/user")
+    login = user.get("login", "?")
+    print(f"merge-robot: authenticated as {login} (status {status})")
+    return login
+
+
+def scan(base: str, token: str) -> list[dict]:
+    status, prs = _api(base, token, "/pulls?state=open&limit=50")
+    print(f"merge-robot: {len(prs)} open PR(s)")
+    return prs
+
+
+def decide(base: str, token: str, pr: dict) -> str:
+    """Return a decision line for one PR; raises MergeRefused on unexpected refusals."""
+    n = pr["number"]
+    sha = pr["head"]["sha"]
+    status, cs = _api(base, token, f"/commits/{sha}/status")
+    state = cs.get("state")
+    if state != "success":
+        return f"#{n}: skipped (status={state})"
+    status, detail = _api(base, token, f"/pulls/{n}")
+    if not detail.get("mergeable", False):
+        return f"#{n}: skipped (not mergeable)"
+    mstatus, mresp = _api(base, token, f"/pulls/{n}/merge", "POST", {"Do": "merge"})
+    if mstatus == 200:
+        return f"#{n}: MERGED"
+    if mstatus == 405:
+        msg = str(mresp.get("message", ""))
+        if "behind" in msg.lower():
+            return f"#{n}: skipped (head behind base — needs branch update): {msg}"
+        raise MergeRefused(f"#{n} merge refused 405: {msg}")
+    if mstatus == 403:
+        raise MergeRefused(f"#{n} merge FORBIDDEN 403 — token lacks pull-request write scope: {str(mresp)[:300]}")
+    raise MergeRefused(f"#{n} merge failed {mstatus}: {str(mresp)[:300]}")
+
+
+def main_with(argv: list[str]) -> int:
+    token = argv[1] if len(argv) > 1 else ""
+    base = argv[2] if len(argv) > 2 else ""
+    if not token or not base:
+        print("usage: merge_robot.py <token> <api-base>")
+        return 2
+    try:
+        verify_identity(base, token)
+        prs = scan(base, token)
+        merged = 0
+        for pr in prs:
+            line = decide(base, token, pr)
+            print("merge-robot:", line)
+            if "MERGED" in line:
+                merged += 1
+            time.sleep(1)
+        print(f"merge-robot: done — {merged} merged of {len(prs)} scanned")
+        return 0
+    except MergeRefused as exc:
+        print(f"merge-robot: FAILING JOB — {exc}", file=sys.stderr)
+        return 1
+
+
+def main() -> int:
+    return main_with(sys.argv)
+
+
+if __name__ == "__main__":
+    sys.exit(main())

tests/test_merge_robot.py

@@ -0,0 +1,155 @@
+"""Unit tests for scripts/merge_robot.py — the fail-loud merge-on-green logic.
+
+Bug-fix contract red-first: the original inline robot swallowed merge refusals
+(print-and-continue, exit 0) — the silent no-op defect. These tests freeze the
+fail-loud behavior: refusals fail the job, identity is verified first, every
+scan produces a decision line.
+"""
+
+from __future__ import annotations
+
+import importlib.util
+import json
+import sys
+from pathlib import Path
+
+import pytest
+
+ROOT = Path(__file__).resolve().parent.parent
+SPEC = importlib.util.spec_from_file_location("merge_robot", ROOT / "scripts" / "merge_robot.py")
+mr = importlib.util.module_from_spec(SPEC)
+SPEC.loader.exec_module(mr)
+
+
+class FakeForge:
+    """Simulates the SERVER contract (not the client's logic — no circularity)."""
+
+    def __init__(self, prs, statuses, merge_results):
+        self.prs = prs
+        self.statuses = statuses  # sha -> state
+        self.merge_results = merge_results  # pr_number -> (status, body)
+        self.mergeable: dict[int, bool] = {}  # pr_number -> server-reported mergeable
+        self.calls: list[tuple[str, str]] = []
+
+    def get(self, path):
+        self.calls.append(("GET", path))
+        if path == "/user":
+            return 200, {"login": "gitea-actions-robot"}
+        if path.startswith("/pulls?state=open"):
+            return 200, self.prs
+        if path.startswith("/commits/"):
+            sha = path.split("/")[2]
+            return 200, {"state": self.statuses.get(sha, "pending")}
+        if path.startswith("/pulls/") and not path.endswith("/merge"):
+            n = int(path.split("/")[2])
+            for pr in self.prs:
+                if pr["number"] == n:
+                    # server truth: GET /pulls/{n} includes mergeable (default True)
+                    return 200, {**pr, "mergeable": self.mergeable.get(n, True)}
+            return 404, {}
+        raise AssertionError(f"unexpected GET {path}")
+
+    def post(self, path, data):
+        self.calls.append(("POST", path))
+        if path.endswith("/merge"):
+            n = int(path.split("/")[2])
+            return self.merge_results.get(n, (200, {}))
+        raise AssertionError(f"unexpected POST {path}")
+
+
+@pytest.fixture
+def forge(monkeypatch):
+    f = FakeForge([], {}, {})
+
+    def fake_api(base, token, path, method="GET", data=None, ok=(200,)):
+        f.calls.append((method, path))
+        if method == "GET":
+            status, body = f.get(path)
+        else:
+            status, body = f.post(path, data)
+        return status, body
+
+    monkeypatch.setattr(mr, "_api", fake_api)
+    return f
+
+
+def _pr(n, sha, state="success", mergeable=True):
+    return {"number": n, "head": {"sha": sha}}, {sha: state}, mergeable
+
+
+def test_identity_verified_before_any_merge(forge, capsys):
+    forge.prs = [{"number": 1, "head": {"sha": "a"}}]
+    forge.statuses = {"a": "pending"}
+    mr.main_with(["merge_robot.py", "tok", "http://x/api/v1/repos/o/r"])
+    assert ("GET", "/user") == forge.calls[0], "identity check must be the first call"
+
+
+def test_merge_refusal_403_fails_job(forge, capsys):
+    forge.prs = [{"number": 7, "head": {"sha": "s7"}}]
+    forge.statuses = {"s7": "success"}
+    forge.merge_results = {7: (403, {"message": "token does not have required scope"})}
+    with pytest.raises(mr.MergeRefused, match="403"):
+        mr.decide("base", "tok", forge.prs[0])
+
+
+def test_merge_refusal_405_unexpected_fails_job(forge):
+    forge.prs = [{"number": 8, "head": {"sha": "s8"}}]
+    forge.statuses = {"s8": "success"}
+    forge.merge_results = {8: (405, {"message": "protected branch rejects"})}
+    with pytest.raises(mr.MergeRefused, match="405"):
+        mr.decide("base", "tok", forge.prs[0])
+
+
+def test_merge_success_returns_merged(forge):
+    forge.prs = [{"number": 9, "head": {"sha": "s9"}}]
+    forge.statuses = {"s9": "success"}
+    forge.merge_results = {9: (200, {})}
+    assert "MERGED" in mr.decide("base", "tok", forge.prs[0])
+
+
+def test_pending_pr_skipped_not_merged(forge):
+    forge.prs = [{"number": 10, "head": {"sha": "s10"}}]
+    forge.statuses = {"s10": "pending"}
+    assert "skipped" in mr.decide("base", "tok", forge.prs[0])
+    assert not any(m == "POST" for m, _ in forge.calls), "must not attempt merge on pending"
+
+
+def test_conflict_pr_skipped(forge):
+    forge.prs = [{"number": 11, "head": {"sha": "s11"}}]
+    forge.statuses = {"s11": "success"}
+    forge.mergeable = {11: False}  # server reports conflict
+    assert "skipped" in mr.decide("base", "tok", forge.prs[0])
+    assert not any(m == "POST" for m, _ in forge.calls)
+
+
+def test_behind_base_405_is_skip_not_crash(forge):
+    """405 'behind the base branch' = needs branch update, not a robot failure."""
+    forge.prs = [{"number": 12, "head": {"sha": "s12"}}]
+    forge.statuses = {"s12": "success"}
+    forge.merge_results = {12: (405, {"message": "The head branch is behind the base branch"})}
+    assert "skipped" in mr.decide("base", "tok", forge.prs[0])
+
+
+def test_main_exits_1_on_refusal(forge, monkeypatch):
+    forge.prs = [{"number": 13, "head": {"sha": "s13"}}]
+    forge.statuses = {"s13": "success"}
+    forge.merge_results = {13: (403, {"message": "no scope"})}
+
+    def fake_scan(base, token):
+        return forge.prs
+
+    monkeypatch.setattr(mr, "scan", fake_scan)
+    rc = mr.main_with(["merge_robot.py", "tok", "base"])
+    assert rc == 1, "refusal must fail the job (exit 1) — the silent no-op defect"
+
+
+def test_main_exits_0_all_merged(forge, monkeypatch):
+    forge.prs = [{"number": 14, "head": {"sha": "s14"}}]
+    forge.statuses = {"s14": "success"}
+    forge.merge_results = {14: (200, {})}
+
+    def fake_scan(base, token):
+        return forge.prs
+
+    monkeypatch.setattr(mr, "scan", fake_scan)
+    assert mr.main_with(["merge_robot.py", "tok", "base"]) == 0