Files
rune/tests/test_operator_probe_isolation.py
T
Nick fcee8fbec7
deploy / deploy (push) Skipped
CI Pipeline / lint (push) Successful in 35s
CI Pipeline / go-surface (push) Successful in 3s
CI Pipeline / canary (push) Successful in 47s
CI Pipeline / best-practices (push) Successful in 2s
CI Pipeline / documentation (push) Successful in 2s
CI Pipeline / type-check (push) Successful in 27s
CI Pipeline / gitea-offline (push) Successful in 12s
CI Pipeline / lint (pull_request) Successful in 34s
CI Pipeline / go-surface (pull_request) Successful in 3s
CI Pipeline / canary (pull_request) Successful in 49s
CI Pipeline / best-practices (pull_request) Successful in 1s
CI Pipeline / documentation (pull_request) Successful in 2s
CI Pipeline / type-check (pull_request) Successful in 26s
CI Pipeline / gitea-offline (pull_request) Successful in 13s
Spec gate (headless) / spec-gate (pull_request) Successful in 4s
CI Pipeline / unit-tests (push) Successful in 4m56s
CI Pipeline / required-checks (push) Successful in 1s
CI Pipeline / CI Pipeline / notify-event-bridge (push) Successful in 2s
CI Pipeline / unit-tests (pull_request) Successful in 4m53s
CI Pipeline / required-checks (pull_request) Successful in 1s
CI Pipeline / CI Pipeline / notify-event-bridge (pull_request) Successful in 2s
Auto-label / priority-gate (pull_request) Skipped
Auto-label / auto-label (pull_request) Successful in 1s
Auto-version and Release / release (pull_request) Successful in 4s
fix(operator): probe budget exhaustion lands in the failure handler (#2468)
The pyright ratchet caught my first attempt: a try-scoped from-import made ProbeBudgetExceeded possibly-unbound in the except tuple. Imports are now module-level dual (the file convention), the early-return returns False explicitly, and a _probe_failure_classes helper carries the caught-class set with degraded graceful behavior. Focused battery 87 green; pyright shows only the 3 pre-existing baseline errors.
2026-10-06 07:14:24 +01:00

41 lines
1.8 KiB
Python

"""Verification probes have a task budget, not a global chat lifetime budget."""
from pathlib import Path
from types import SimpleNamespace
import hashlib
from rune import evidence_probe, rune_operator
def test_probe_uses_distinct_task_directories(tmp_path, monkeypatch):
monkeypatch.setenv("MIMO_PROBE_ENABLED", "1")
operator = object.__new__(rune_operator.RuneOperator)
operator.storage_path = str(tmp_path / "operator.json")
called = []
def probe(_question, artifact_dir, model):
called.append(Path(artifact_dir))
return SimpleNamespace(status="GROUNDED")
# The operator binds run_probe at module level (_run_probe) — the real
# interception seam is the operator module's binding, not evidence_probe's.
monkeypatch.setattr(rune_operator, "_run_probe", probe)
monkeypatch.setattr(operator, "_route_task_class", lambda *_: None)
for task_id in ("task-a", "task-b"):
task = SimpleNamespace(task_id=task_id, description="verify")
assert operator._probe_and_escalate(task, SimpleNamespace(error="failure"), "chat") is False
assert called == [tmp_path / "probes" / hashlib.sha256(t.encode()).hexdigest()
for t in ("task-a", "task-b")]
def test_exhausted_task_budget_never_escapes_into_chat(tmp_path, monkeypatch):
monkeypatch.setenv("MIMO_PROBE_ENABLED", "1")
operator = object.__new__(rune_operator.RuneOperator)
operator.storage_path = str(tmp_path / "operator.json")
def exhausted(*_args, **_kwargs):
raise evidence_probe.ProbeBudgetExceeded("probe budget exhausted (2)")
monkeypatch.setattr(rune_operator, "_run_probe", exhausted)
task = SimpleNamespace(task_id="task-a", description="verify")
assert operator._probe_and_escalate(task, SimpleNamespace(error="failure"), "chat") is False