CI Pipeline / best-practices (push) Successful in 8s
CI Pipeline / documentation (push) Successful in 4s
CI Pipeline / canary (push) Successful in 32s
CI Pipeline / lint (push) Successful in 42s
CI Pipeline / type-check (push) Successful in 30s
deploy / deploy (push) Skipped
CI Pipeline / gitea-offline (push) Successful in 21s
Auto-label / auto-label (pull_request) Successful in 23s
Auto-label / priority-gate (pull_request) Skipped
CI Pipeline / documentation (pull_request) Successful in 6s
CI Pipeline / canary (pull_request) Successful in 36s
CI Pipeline / type-check (pull_request) Successful in 27s
CI Pipeline / gitea-offline (pull_request) Successful in 22s
Spec gate (headless) / spec-gate (pull_request) Successful in 18s
CI Pipeline / best-practices (pull_request) Successful in 7s
CI Pipeline / lint (pull_request) Successful in 40s
CI Pipeline / unit-tests (push) Successful in 3m22s
CI Pipeline / required-checks (push) Successful in 1s
CI Pipeline / CI Pipeline / notify-event-bridge (push) Successful in 5s
CI Pipeline / unit-tests (pull_request) Successful in 3m22s
CI Pipeline / required-checks (pull_request) Successful in 1s
CI Pipeline / CI Pipeline / notify-event-bridge (pull_request) Successful in 5s
Auto-version and Release / release (pull_request) Successful in 39s
Defect A root cause: Gitea runs-list page flood (scheduled merge-robot runs push relevant SHAs past the limit=50 first page) + collect-only- first-page pagination -> terminal commit read as pending -> parked until deadline. FIX: _fetch_runs paginates up to 4 pages collecting all matches before verdict. Defect B root cause: authority_verdict spawned deploy_qualify reading DEPLOY_SHA from ambient env. FIX: explicit --sha arg (env masked when sha is explicit); _ci_runs_verdict/op_ci_alive take sha as parameter only; no env read remains in the authority path. Defect C root cause: model sha treated as lookup key — a typo derived a ghost operation. FIX: model sha is a HINT; unresolved sha + single pending op -> stored target used with model_sha_mismatch logged; bound op always wins over hints. deploy_qualify gained --sha flag (env kept for workflow backward-compat only). Tests: H-matrix (identity, prefix collision, env staleness, monotonicity, aggregator lag, operation-id-first) + full 2179 suite.
178 lines
7.4 KiB
Python
178 lines
7.4 KiB
Python
#!/usr/bin/env python3
|
|
"""deploy_qualify.py — read the CI qualification status for one commit.
|
|
|
|
Called by .gitea/workflows/deploy.yml (kept out of the YAML so the workflow
|
|
has no embedded multi-line scripts — a YAML block-scalar indentation hazard).
|
|
|
|
Prints one of: success | failure | pending
|
|
Exit code 0 always (the shell decides what to do with the verdict).
|
|
|
|
RELEASE-SAFETY HARDENING (2026-09-13, after red-CI promotions 0.66.28/29):
|
|
the verdict is the AND of two independent legs —
|
|
1. required-checks context (newest observation for this exact SHA);
|
|
2. the latest COMPLETED ci.yml@refs/heads/master run for this exact SHA,
|
|
every job concluded success (branch+tag fan-out resolved by path;
|
|
jobs envelope unwrapped — no aggregate-commit-status fallback, the
|
|
exact bug class verify.py hit in W/R-v0.3).
|
|
Any disagreement, pending, or missing evidence → pending/failure.
|
|
FAIL-CLOSED: not verified → not deployable.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import json
|
|
import os
|
|
import sys
|
|
import urllib.request
|
|
import sys as _sys
|
|
_SCRIPT_DIR = str(__import__('pathlib').Path(__file__).resolve().parent)
|
|
if _SCRIPT_DIR not in _sys.path:
|
|
_sys.path.insert(0, _SCRIPT_DIR)
|
|
import gitea_auth
|
|
|
|
REQUIRED_CONTEXT = "CI Pipeline / required-checks"
|
|
CI_WORKFLOW_PATH = "ci.yml@refs/heads/master"
|
|
|
|
|
|
# ── pure verdict logic (unit-tested; fixtures from the real incidents) ────
|
|
|
|
def required_check_verdict(statuses: list[dict]) -> str:
|
|
"""Newest non-skipped required-checks observation for the SHA.
|
|
Missing → pending. Newest wins (stale success never overrides a newer
|
|
failure, and vice versa)."""
|
|
candidates = [
|
|
s for s in statuses
|
|
if str(s.get("context", "")).startswith(REQUIRED_CONTEXT)
|
|
and s.get("status") != "skipped"
|
|
]
|
|
if not candidates:
|
|
return "pending"
|
|
newest = max(candidates, key=lambda s: str(s.get("created_at", "")))
|
|
return str(newest.get("status", "pending"))
|
|
|
|
|
|
def independent_jobs_verdict(runs: list[dict], jobs_fetch, sha: str = "") -> str:
|
|
"""Latest COMPLETED ci.yml@refs/heads/master run for the exact SHA:
|
|
every job must conclude success. Fan-out-safe (tag runs and sibling
|
|
workflows are excluded by path); the run is resolved by head_sha, never
|
|
by 'latest branch state'. Missing run → pending. Any non-success
|
|
conclusion → failure."""
|
|
mine = [r for r in runs if r.get("path") == CI_WORKFLOW_PATH]
|
|
if sha:
|
|
mine = [r for r in mine if str(r.get("head_sha", "")) == sha]
|
|
completed = [r for r in mine if r.get("status") == "completed"]
|
|
if not completed:
|
|
return "pending"
|
|
latest = max(completed, key=lambda r: str(r.get("created_at", "") or r.get("run_started_at", "")))
|
|
jobs = jobs_fetch(latest.get("id"))
|
|
if jobs is None:
|
|
return "pending" # jobs probe failed → not verified
|
|
if not jobs:
|
|
return "pending" # no jobs visible → cannot verify
|
|
for j in jobs:
|
|
if j.get("conclusion") != "success":
|
|
return "failure"
|
|
return "success"
|
|
|
|
|
|
def combine(a: str, b: str) -> str:
|
|
"""Fail-closed conjunction: any failure → failure; any pending → pending;
|
|
success only when BOTH legs are success."""
|
|
if "failure" in (a, b):
|
|
return "failure"
|
|
if "pending" in (a, b):
|
|
return "pending"
|
|
return "success"
|
|
|
|
|
|
def qualify(statuses: list[dict], runs: list[dict], jobs_fetch, sha: str = "") -> str:
|
|
return combine(required_check_verdict(statuses),
|
|
independent_jobs_verdict(runs, jobs_fetch, sha=sha))
|
|
|
|
|
|
# ── API fetches (thin; injectable in tests) ───────────────────────────────
|
|
|
|
def _get_json(url: str, token: str, timeout: int = 30):
|
|
req = urllib.request.Request(url, headers={"Authorization": f"token {token}"})
|
|
try:
|
|
with urllib.request.urlopen(req, timeout=timeout) as r:
|
|
return json.loads(r.read())
|
|
except (OSError, ValueError):
|
|
return None
|
|
|
|
|
|
def _fetch_status(server: str, repo: str, sha: str, token: str) -> list[dict] | None:
|
|
data = _get_json(f"{server}/api/v1/repos/{repo}/commits/{sha}/status", token)
|
|
return None if data is None else data.get("statuses", [])
|
|
|
|
|
|
def _fetch_runs(server: str, repo: str, sha: str, token: str,
|
|
max_pages: int = 4) -> list[dict] | None:
|
|
"""H-PATCH: paginate the runs list. The repo's scheduled merge-robot
|
|
runs flood the first page (~50 slots exhaust within hours), pushing
|
|
older-but-relevant SHAs to later pages — the cause of the observed
|
|
authority flapping (terminal commit read as pending). Paginate until
|
|
the sha is found or max_pages exhausted."""
|
|
out: list[dict] = []
|
|
for page in range(1, max_pages + 1):
|
|
data = _get_json(
|
|
f"{server}/api/v1/repos/{repo}/actions/runs?limit=50&page={page}",
|
|
token)
|
|
if data is None:
|
|
return None if page == 1 else (out or None)
|
|
runs = data.get("workflow_runs", data if isinstance(data, list) else [])
|
|
out.extend(r for r in runs if str(r.get("head_sha", "")) == sha)
|
|
if len(runs) < 50:
|
|
break # exhausted
|
|
return out
|
|
|
|
|
|
def _fetch_jobs(server: str, repo: str, run_id, token: str) -> list[dict] | None:
|
|
data = _get_json(f"{server}/api/v1/repos/{repo}/actions/runs/{run_id}/jobs", token)
|
|
if data is None:
|
|
return None
|
|
# jobs envelope unwrap (this Gitea wraps in {"jobs": [...]}; some
|
|
# versions return a bare list) — never fall back to aggregate status
|
|
if isinstance(data, dict):
|
|
return data.get("jobs", [])
|
|
return data if isinstance(data, list) else None
|
|
|
|
|
|
# ── CLI ───────────────────────────────────────────────────────────────────
|
|
|
|
def main() -> int:
|
|
# T21.4 fix: fall back to the canonical defaults when the caller did
|
|
# not export GITEA_SERVER/GITEA_REPO (CI always exports them; local
|
|
# and first-class-tool consumers previously got a silent "pending"
|
|
# verdict — a wrong answer, not a fail-closed one).
|
|
# H-PATCH H4: the target sha arrives EXPLICITLY (--sha) from callers
|
|
# that hold structured operation state. DEPLOY_SHA env remains only
|
|
# for standalone/workflow backward compat — the canonical path never
|
|
# depends on ambient env for operation identity.
|
|
ap = __import__("argparse").ArgumentParser()
|
|
ap.add_argument("--sha", dest="sha_arg", default=None)
|
|
args, _unknown = ap.parse_known_args()
|
|
server = (os.environ.get("GITEA_SERVER", "")
|
|
or gitea_auth.DEFAULT_SERVER).rstrip("/")
|
|
repo = os.environ.get("GITEA_REPO", "") or gitea_auth.DEFAULT_REPO
|
|
sha = args.sha_arg or os.environ.get("DEPLOY_SHA", "")
|
|
token = gitea_auth.get_token() if gitea_auth.has_token() else ""
|
|
if not (server and repo and sha and token):
|
|
print("pending") # missing config → cannot verify → not qualified
|
|
return 0
|
|
|
|
statuses = _fetch_status(server, repo, sha, token)
|
|
runs = _fetch_runs(server, repo, sha, token)
|
|
if statuses is None or runs is None:
|
|
# transient API/network failure → keep polling
|
|
print("pending (probe failed)")
|
|
return 0
|
|
jobs_fetch = lambda run_id: _fetch_jobs(server, repo, run_id, token) # noqa: E731
|
|
verdict = qualify(statuses, runs, jobs_fetch, sha=sha)
|
|
print(verdict)
|
|
return 0
|
|
|
|
|
|
if __name__ == "__main__":
|
|
sys.exit(main())
|