Nick
c3c95688f7
Seeds the pipeline with the REAL #192 CONVERGED case-mismatch payload: the safety-net convergence step exported Python-cased True/False from json.load while the steps compared env.CONVERGED == 'false' — both conditionals unreachable (discharged on the release-safety-net branch, public in PR #211's diff). Captured via the pipeline itself (--class env-case-mismatch --source ci, 0 redactions — the payload is already public and credential-free). The generated regression test asserts the boolean-export checker still FLAGS the recorded payload (checker_expectation: flags), so a recurrence of the class fails CI. Manifest gains checker_expectation: capture records whether the class's checker raises on the payload ('flags' — the raise IS the recurrence alarm) or must stay quiet ('passes'); the generated tests enforce the recorded expectation, so a checker that stops catching its recorded defect rots the suite red. Docs: bugfix-contract §10 gains the operational capture step + explicit trivial-bug exemption list; README dev section points at the pipeline; tests/fixtures/production/README.md documents the flow. Henxels: capture_failure.py required in scripts/; production fixtures governed (manifest.json + README.md required); payload.json and manifest.json travel together (changed_with both directions, verified to bite on a payload-only staged change).
109 lines
6.0 KiB
Markdown
109 lines
6.0 KiB
Markdown
# Bug-fix contract
|
|
|
|
_Binding on every agent, session, and pipeline stage in this repository. The
|
|
henxel mirror of this contract lives in `henxels.yaml` ("bugs discharged
|
|
red-first"); the enforcement hooks live in `scripts/check_discharge_contract.py`
|
|
and `.gitea/workflows/pipeline-gate.yml`. Escapes from this contract are made
|
|
by editing those files — visibly — never by silent deviation._
|
|
|
|
## The contract
|
|
|
|
When a **bug is found** — by a gate, a reviewer, CI, a canary, production, or a
|
|
human — the fix MUST follow this cycle before it may merge:
|
|
|
|
1. **Red first.** A failing test that reproduces the bug is written BEFORE the
|
|
fix. The test must fail for the *bug's cause*, not for an unrelated reason.
|
|
A fix without a red test is a symptom patch and is rejected at review.
|
|
2. **Root cause, not symptom.** The fix addresses the cause. If the reporter's
|
|
diagnosis is wrong, the discharge says so and names the real cause.
|
|
3. **Full pipeline re-entry.** The fix is treated as a NEW SUBMISSION: full
|
|
suite, pyright ratchet (not increased), ruff, canaries, henxels, and the
|
|
spec + quality gates for anything beyond a mechanical minor. No
|
|
"hotfix-skips-gates" path exists.
|
|
4. **The regression test is permanent.** Tests written for a discharged bug are
|
|
frozen — they may be refactored, never weakened or deleted. The suite can
|
|
only grow in assertion strength over time.
|
|
5. **Recorded.** The fix's PR (or plan tail) maps every finding → the red test
|
|
→ the commit. A discharge record with holes bounces at review.
|
|
|
|
## Scope and proportionality
|
|
|
|
- **Gate findings (spec/quality):** every BLOCKER/MAJOR gets the full cycle;
|
|
MINORs get at minimum a red test OR an explicit reviewer-approved waiver
|
|
recorded in the plan tail.
|
|
- **Production failures:** the fix follows this contract AND, where the failing
|
|
payload is preservable, #157's fixture pipeline (real failure → sanitised
|
|
fixture → permanent regression test).
|
|
- **CI/workflow bugs:** the red test is the workflow-contract assertion
|
|
(parsed-YAML step/step-logic assertions, as in `test_workflow_script_contract.py`)
|
|
— workflows are code; their bugs get tests too.
|
|
- **Trivial/no-test-class bugs** (typos in comments, log strings): documented
|
|
exemption list in this file. PR-only changes to those strings skip the red test.
|
|
|
|
### Exemption list (trivial/no-test-class)
|
|
|
|
The red test is skipped only when the change is mechanical and carries no
|
|
behavioural surface. Current exemptions:
|
|
|
|
- typos in comments, docstrings, and documentation prose;
|
|
- log-message wording with no parsing consumer (nothing asserts on the text);
|
|
- README/wiki wording.
|
|
|
|
Anything touching executable behaviour — including workflow YAML steps,
|
|
scripts, and configuration consumed at runtime — is NOT exempt.
|
|
|
|
## Why (the failure class this kills)
|
|
|
|
LLM-authored changes share wrong assumptions between implementation, tests, and
|
|
docs. A symptom patch shares the reporter's wrong assumption with the fix. The
|
|
red-first test is the cheapest known breaker of both: it forces the fixer to
|
|
state the bug's cause in executable form before claiming to have removed it.
|
|
|
|
## Enforcement
|
|
|
|
- `scripts/check_discharge_contract.py`: verifies a PR that touches files named
|
|
in a `Discharge:` trailer block carries at least one test-hunk per finding
|
|
id (structural check — PR description convention: `Discharge: <id> -> <test file>`).
|
|
- Gate reviewers are instructed (dispatch templates, issue #151) to reject
|
|
symptom patches: "what test failed red for this bug?" is a mandatory
|
|
reviewer question for every fix PR.
|
|
- `henxels.yaml` carries the tracked one-line mirror so the contract is part
|
|
of the repo's auditable contract surface.
|
|
|
|
## Binding refinements (pipeline doctrine, 2026-09-07)
|
|
|
|
6. **Soak-before-close for intermittent defects.** A flaky, timing, or
|
|
event-delivery bug cannot be proven fixed by one green run. The fix merges
|
|
through the normal pipeline, but the issue CLOSES only on soak evidence: a
|
|
stated criterion (e.g. "N consecutive green runs" or "48h without
|
|
recurrence") plus the observations meeting it. The closing comment carries
|
|
the soak record. Merge ≠ closure for this class.
|
|
7. **Escape accountability.** When a bug reaches production (escaped every
|
|
gate), the discharge must answer: _which gate should have caught this, and
|
|
why didn't it?_ The answer becomes a gate improvement — a new contract
|
|
assertion, a checker rule, or a documented gate limit. Gate hardening is a
|
|
first-class output of bug fixing, not an afterthought.
|
|
8. **Fresh-context root-cause verification.** The red test proves the fixer
|
|
understood _a_ cause. For BLOCKER/MAJOR fixes, a fresh-context reviewer
|
|
additionally asks: does this test fail on the pre-fix code _for the right
|
|
reason_? (Same doctrine as the black-box test author: the person who wrote
|
|
the fix does not certify its diagnosis.)
|
|
9. **Déjà-vu detection.** Before writing the red test, search the suite and
|
|
closed issues for the same failure class. A second occurrence of a class
|
|
means the first fix treated a symptom — the discharge must then cover the
|
|
class, not the instance.
|
|
10. **Fixture promotion.** When the red test needs a complex payload, it goes
|
|
through #157's sanitised-fixture pipeline (real failure → secrets stripped
|
|
→ `tests/fixtures/production/`) instead of a hand-crafted approximation.
|
|
Real failures are the best regression tests. Operationally: after the fix
|
|
merges, run the capture command from
|
|
`tests/fixtures/production/README.md`
|
|
(`python3 scripts/capture_failure.py payload.txt --class <class> --source <issue|ci|manual>`)
|
|
on the ACTUAL payload and commit the fixture + manifest with the discharge
|
|
record. The generated regression tests (`tests/fixtures/production/test_production_fixtures.py`)
|
|
assert the payload's sha256 against the manifest (tamper-evidence) and that
|
|
the class's checker still catches the recorded defect — a recurrence of the
|
|
captured failure fails CI. Exemptions from fixture capture follow the
|
|
trivial-bug rule below: a typo or a log string has no payload worth
|
|
preserving.
|