fix: data-safety audit repairs — serialize the command RMW span; loud atomic durable-state saves #2624

Open

conductor wants to merge 4 commits from fix/data-safety-audits into master

pull from: fix/data-safety-audits

merge into: conductor:master

conductor:master

conductor:fix/shared-ci-unblock

conductor:fix/ci-concurrency-per-ref

conductor:fix/2552-handler-triage

conductor:feat/2618-context

conductor:fix/2604-scaffolding-surgical

conductor:feat/2618-provider-stack

conductor:feat/final-integration

conductor:feat/2618-safety-parity

conductor:feat/2609-integration-w6

conductor:feat/2618-audit-forge

conductor:feat/2618-background-memory

conductor:feat/2618-operator-residual

conductor:feat/2618-warm-recall

conductor:feat/2617-research-r3

conductor:feat/2606-spine-w4

conductor:feat/2617-research-r2

conductor:fix/2607-spine-dichotomy

conductor:feat/2609-go-interception-w3

conductor:feat/2618-memory-store

conductor:feat/2617-research-r1

conductor:feat/2609-go-interception-w2

conductor:fix/integration-audits

conductor:fix/sqlite-floor-audits

conductor:feat/2617-research-r0

conductor:feat/2609-model-replay

conductor:fix/tool-boundary-audits

conductor:feat/2609-go-interception-w1

conductor:fix/2582-release-final

conductor:fix/2613-self-hosting-gate

conductor:removal-prep/dead-superseded-pile

conductor:removal-prep/auto-context

conductor:removal-prep/reconcile-tools

conductor:removal-prep/openai-compat-transport

conductor:removal-prep/message-queue

conductor:removal-prep/forge-client

conductor:removal-prep/cache

conductor:removal-prep/paths

conductor:fix/2532-text-selection

conductor:throwaway-force-merge-test

conductor:fix/2582-namespace-cli-flag

conductor:feat/2582-release-reporting

conductor:fix/2535-pane-scrollback

conductor:dichotomy-gap-f0

conductor:gap-transport

conductor:gap-construction-commands

conductor:wave-ca-ms

conductor:tmp-detach

conductor:gap-ltr

conductor:backup-rune-as

conductor:wave-ax

conductor:feat/wave6-rust-backend

conductor:feat/wave4-consolidation-kernel

conductor:ci/build-cognition-core-unit

conductor:ci/register-w34-canaries

conductor:feat/wire-programme-continuation

conductor:feat/wire-construction-commands-dispatch

conductor:feat/wire-stage-guards-transition

conductor:feat/tui-pane-swap-chat-left

conductor:fix/operator-probe-tools-2468

conductor:r2-wave2-memory-retrieval

conductor:r2-wave2-memory-storage

conductor:phase-g-strangler-evidence-provenance

conductor:phase-g-strangler-retrieval-index

conductor:tranche-r1-wave-0

conductor:m10-derive-refresh

conductor:t22-scaffolding-default-path

conductor:research-depth-gate

conductor:fix-407-autolabel

conductor:fix-tui-banner-surface

conductor:r13-m9-pr-proof

conductor:gitea-internal-endpoint

conductor:auto-label-epic-kinds

conductor:issue-templates-heading

conductor:issue-templates

conductor:event-driven-clean

conductor:deadline-return-fix

conductor:hardening-live-proof

conductor:hardening-authority-identity

conductor:event-driven-final-metrics

conductor:event-driven-gates-evidence

conductor:event-driven-continuation

conductor:async-intent-live-proof

conductor:async-intent-normalization

conductor:memory-t0-external-report

conductor:memory-t0-asis-map

conductor:t21.5-pending-class-fix

conductor:t21.5-gate-evidence

conductor:t21.5-root-cause-continuation

conductor:t21.4-gate-evidence

conductor:t21.4-qualify-defaults-fix

conductor:t21.4-final-v1-proof

conductor:prototype/openai-compat-transport

conductor:wr-experiment

conductor:adapt2-route-authority

conductor:pl0-experience-ledger

conductor:d5-authority-audit

conductor:task-d-stage-contracts

conductor:b3v2-fresh

conductor:d5-tightening

conductor:d4d5-artifact-contract

conductor:correction-1-d2d3

conductor:adapt9-artifact-authority

conductor:prov1-source-ledger

conductor:infra1-accept

conductor:infra1-probe5

conductor:infra1-probe4

conductor:infra1-probe3

conductor:infra1-probe2

conductor:infra1-probe

conductor:obs1-telemetry

conductor:rp13-metrics

conductor:rp12-rerun-163

conductor:rp11-retirement

conductor:issue161-harness-v2

conductor:issue161-harness

conductor:rp2-canonical-stage-model

conductor:sweep-live

conductor:umbrella-rewire

conductor:trigger-bump-2

conductor:release-token-fix

conductor:trigger-bump

conductor:version-reconcile

conductor:umbrella-pipeline

conductor:pinned-live

conductor:research-structuralization

conductor:merge-robot-checkout

conductor:p1-03-memory-selection

conductor:coverage-ratchet

conductor:failure-auto-filing

conductor:release-safety-net

conductor:ci-dependency-lock

conductor:bugfix-contract

conductor:merge-robot-fix

conductor:adversarial-gates

conductor:merge-robot

conductor:runner-desktop-gate

conductor:research-gate-evidence

conductor:backup/pinned-20260908-035000

conductor:release-tui-footer

Conversation 2 Commits 4 Files Changed 22

+2096 -171

conductor commented 2026-10-26 02:01:59 +00:00

Owner

Copy link

Copy source

Data-safety audit repairs: F1 lost concurrent updates, F2 D40 masked durable-state failures

Owner-dispatched repair of the two reproduced data-safety defects (audited on frozen base a52624b3, T17 report; re-verified against current master 79ff61b3 before touching anything; construction contract ADMIT-repair-data-safety-audits, plan plans/2026-10-26-data-safety-audit-repairs.md).

F1 — concurrent task-state updates were being lost. The task-command door read the hot checkpoint once, mutated it in memory, and blind-wrote the whole file back with no synchronization within or across processes — two commands erased each other's effects (reproduced 5/5 through the real serve door, including a core-created task vanishing while the command reported success). Atomic rename prevents torn files, not lost updates. Repair: the whole read-modify-write span now holds one per-path serialization — an in-process mutex plus a cross-process flock on the sidecar <tasks>.lock (the same protocol the factory reconciler and the Rust/Python task_persist ledger already use); the task-state ledger no longer mints a fresh instance per request (the per-instance mutex it documented protected nothing) and holds <dir>/.lock across its mutations; the durable rename now also fsyncs its directory.

F2 — failed durable-state saves reported success (D40). Five state saves ran open+truncate+write with errors thrown away: a failed save printed ✅ and exited 0 with nothing written; a torn batch store read as zero work and the next verb rewrote the queue empty; model_routing renamed without any fsync. Repair: every save now carries the AtomicStore law (temp + fsync(file) + rename + fsync(dir)) and returns its failure loudly — nonzero exits, stderr naming the effect that did NOT persist, honest error envelopes; a torn store fails loud and keeps its bytes instead of being silently zeroed.

Red first, mutation-checked (docs/bugfix-contract.md). Every finding has a failing-before-fixing test that reproduced the exact harm (evidence lines recorded in the plan), and the fix was mutation spot-checked both ways. The regression tests are permanent.

Conscious parity updates (not silently weakened). The Python oracle's suppressed-error shape is retired on the FAILURE paths only, with the justification documented in cli_parity_test.go and the frozen battery; happy-path byte parity is unchanged and re-proved by the live oracle battery.

Verification: full suites + -race green in all seven touched Go modules (task-state, rune-entry, work-manager, model_routing, model-profiles, subsystem-health, sleep-time), including the frozen truth battery and the live Python-oracle parity battery.

Discharge: F1-rune-entry-checkpoint-rmw -> src/go/rune-entry/internal/entry/command_race_test.go
Discharge: F1-ledger-fresh-instance-flock -> src/go/task-state/ledger/ledger_lock_test.go
Discharge: F2-work-manager-masked-saves -> src/go/work-manager/internal/workmanager/persistence_safety_test.go
Discharge: F2-subsystem-health-masked-save -> src/go/subsystem-health/main_test.go
Discharge: F2-sleep-time-masked-save -> src/go/sleep-time/main_test.go
Discharge: F2-model-profiles-masked-save -> src/go/model-profiles/main_test.go
Discharge: F2-model-routing-unsynced-rename -> src/go/model_routing/model_routing.go

## Data-safety audit repairs: F1 lost concurrent updates, F2 D40 masked durable-state failures Owner-dispatched repair of the two reproduced data-safety defects (audited on frozen base a52624b3, T17 report; re-verified against current master 79ff61b3 before touching anything; construction contract `ADMIT-repair-data-safety-audits`, plan `plans/2026-10-26-data-safety-audit-repairs.md`). **F1 — concurrent task-state updates were being lost.** The task-command door read the hot checkpoint once, mutated it in memory, and blind-wrote the whole file back with no synchronization within or across processes — two commands erased each other's effects (reproduced 5/5 through the real serve door, including a core-created task vanishing while the command reported success). Atomic rename prevents torn files, not lost updates. Repair: the whole read-modify-write span now holds one per-path serialization — an in-process mutex plus a cross-process flock on the sidecar `<tasks>.lock` (the same protocol the factory reconciler and the Rust/Python task_persist ledger already use); the task-state ledger no longer mints a fresh instance per request (the per-instance mutex it documented protected nothing) and holds `<dir>/.lock` across its mutations; the durable rename now also fsyncs its directory. **F2 — failed durable-state saves reported success (D40).** Five state saves ran open+truncate+write with errors thrown away: a failed save printed ✅ and exited 0 with nothing written; a torn batch store read as zero work and the next verb rewrote the queue empty; model_routing renamed without any fsync. Repair: every save now carries the AtomicStore law (temp + fsync(file) + rename + fsync(dir)) and returns its failure loudly — nonzero exits, stderr naming the effect that did NOT persist, honest error envelopes; a torn store fails loud and keeps its bytes instead of being silently zeroed. **Red first, mutation-checked (docs/bugfix-contract.md).** Every finding has a failing-before-fixing test that reproduced the exact harm (evidence lines recorded in the plan), and the fix was mutation spot-checked both ways. The regression tests are permanent. **Conscious parity updates (not silently weakened).** The Python oracle's suppressed-error shape is retired on the FAILURE paths only, with the justification documented in `cli_parity_test.go` and the frozen battery; happy-path byte parity is unchanged and re-proved by the live oracle battery. Verification: full suites + `-race` green in all seven touched Go modules (task-state, rune-entry, work-manager, model_routing, model-profiles, subsystem-health, sleep-time), including the frozen truth battery and the live Python-oracle parity battery. Discharge: F1-rune-entry-checkpoint-rmw -> src/go/rune-entry/internal/entry/command_race_test.go Discharge: F1-ledger-fresh-instance-flock -> src/go/task-state/ledger/ledger_lock_test.go Discharge: F2-work-manager-masked-saves -> src/go/work-manager/internal/workmanager/persistence_safety_test.go Discharge: F2-subsystem-health-masked-save -> src/go/subsystem-health/main_test.go Discharge: F2-sleep-time-masked-save -> src/go/sleep-time/main_test.go Discharge: F2-model-profiles-masked-save -> src/go/model-profiles/main_test.go Discharge: F2-model-routing-unsynced-rename -> src/go/model_routing/model_routing.go

conductor added 3 commits 2026-10-26 02:01:59 +00:00

fix(entry,task-state): serialize the command RMW span — concurrent task-state updates no longer lost ... 399ef97eb8

The task-command door read the hot checkpoint once, mutated the parsed
map, and blind-wrote the whole file back: no in-process and no
cross-process serialization, so concurrent commands erasing each other's
effects were reproduced 5/5 through the real serve door (audit F1,
incl. a core-created task vanishing while the command reported success
— atomic rename prevents torn files, not lost updates).

Now the whole read-modify-write span holds one per-path state span
(statelock.go): a per-path mutex plus a cross-process flock on the
sidecar <tasks>.lock — the factory-kernel/task_persist protocol. The
task-state ledger's fresh-per-request Open used to defeat its own
documented per-instance mutex: Open now returns the canonical
per-directory instance and every mutation holds <dir>/.lock (the
task_persist convention); Sync degrades to in-process-only rather than
refusing a serve contact (D38 best-effort law). atomicWriteFile gained
the directory fsync that makes its rename durable.

Red-first (bugfix contract, pre-fix runs): TestConcurrentCompletesAllCompose
(t01..t10 completions erased), TestForeignFlockBlocksCommandSpan (foreign
lock ignored, whole-file write erased the applied completion),
TestOpenReturnsCanonicalPerDirInstance, TestForeignFlockBlocksLedgerMutations
(Initiate charged through a held foreign .lock). Mutation spot-check:
killing the flock alone re-reds the span test; killing the mutex alone
re-reds the compose test (same-fd flock is a no-op) — both halves
load-bearing. Tests are permanent.

Discharge: F1-rune-entry-checkpoint-rmw -> src/go/rune-entry/internal/entry/command_race_test.go
Discharge: F1-ledger-fresh-instance-flock -> src/go/task-state/ledger/ledger_lock_test.go

fix(work-manager,subsystem-health,sleep-time,model-profiles,model_routing): loud atomic durable-state saves — no more open-truncate-write, no more masked failures ... 798cd35cf4

The D40 violation sweep from the data-safety audit: five durable-state
saves ran bare os.WriteFile (open+truncate+write — the torn-file shape)
with errors thrown away. A failed save printed success and exited 0 with
nothing written (✅ Batch B1 marked complete over an unwritable store), a
torn batch store read as zero work and the next mutating verb rewrote it
empty, and model_routing renamed without any fsync (a power loss could
resurrect the old store).

Now: every save carries the AtomicStore law (temp + fsync(file) + rename
+ fsync(dir)) via a per-module writeFileAtomic; work-manager seams return
errors and the CLI verbs exit 1 naming the effect they did NOT persist;
loadBatches fails loud on a torn/unreadable store (the deliberate oracle
shape-reject semantics for well-formed JSON stay pinned — that is the
parity surface, not the defect); loadConfig/loadMeta stay oracle-faithful;
subsystem-health/sleep-time refuse to claim a dispatch/record on a failed
save; model-profiles threads errors from every save path to an honest
error envelope + exit 1; model_routing's FileSystem seam gained
SyncFile/SyncDir and SaveProfiles runs the full law.

RED FIRST (pre-fix runs, exact harms): ✅ success + exit 0 with nothing
written; status over a torn store rendered Batches: 0 and complete rewrote
the queue as []; all three binaries exited 0 claiming failed saves. Post
-fix unit pins for the new error channels (TestSaveSeamsReturnErrors,
TestSavesAtomicallyResidueFree) could not compile pre-fix — the red for
the missing error channel — and are permanent.

CONSCIOUS parity updates (never silently weakened, re-proof recorded):
TestLoadBatchesStrictness keeps the oracle shape law and SPLITS the
malformed row into a loud corruption law; cli_parity_test.go documents the
failure-path divergence from the oracle's suppression (happy-path byte
parity unchanged and still proven by the live battery); frozen dispatch
and parity_live handlers fail the test on save/load errors (no truth bytes
regenerated). Mutation spot-checks: re-swallowing loadBatches corruption
and saveBatches errors each re-red their test.

Discharge: F2-work-manager-masked-saves -> src/go/work-manager/internal/workmanager/persistence_safety_test.go
Discharge: F2-subsystem-health-masked-save -> src/go/subsystem-health/main_test.go
Discharge: F2-sleep-time-masked-save -> src/go/sleep-time/main_test.go
Discharge: F2-model-profiles-masked-save -> src/go/model-profiles/main_test.go
Discharge: F2-model-routing-unsynced-rename -> src/go/model_routing/model_routing.go

docs(plans): data-safety audit discharge record — finding → red test → commit, admission envelope ... fd9a160ff9

The bugfix-contract discharge record for the two reproduced data-safety
defects: red-first evidence (pre-fix failure lines), both mutation
spot-checks per finding, the conscious parity updates with their
justification (oracle suppressed-error shape retired on failure paths
only, happy-path byte parity re-proved), the construction admission
(classify → activate → admit, contract ADMIT-repair-data-safety-audits
naming every touched surface), and the remaining risk (a checkpoint
writer that ignores the <tasks>.lock sidecar can still interleave —
the sidecar is the cross-process contract).

Admission script: scripts/admit_repair_data_safety_audits.py (the
admit_repair_verification_layer pattern); the runtime store state it
mints is left as churn, not committed, per the sweep-race guard.

gitea-actions Bot added the P1 — highbuggovernanceinfrastructurememory labels 2026-10-26 02:02:03 +00:00

gitea-actions Bot commented 2026-10-26 02:02:43 +00:00

Copy link

Copy source

🤖 Headless spec gate (non-blocking, #204 first slice)

Spec-gate agent produced no text output — check runner logs.

Deterministic floors still govern the merge

This comment is advisory. Required checks (lint, tests, canaries, type-check, coverage, dependency lock, impact gate) remain the merge authority per docs/adversarial-review.md proportionality.

🤖 **Headless spec gate (non-blocking, #204 first slice)** Spec-gate agent produced no text output — check runner logs. <details><summary>Deterministic floors still govern the merge</summary> This comment is advisory. Required checks (lint, tests, canaries, type-check, coverage, dependency lock, impact gate) remain the merge authority per docs/adversarial-review.md proportionality.</details>

conductor added 1 commit 2026-10-26 02:34:31 +00:00

docs(plans): record the protected-check observation with log-level attribution ... ed91205ddc

CI run 34866 for the PR head: every failing job attributed from its own
log, then cross-checked against a concurrent PR on a different branch
whose diffs share no files with this change — identical failure
signature, identical pyright totals, byte-identical canary fragment.
All eight failing jobs are inherited master/CI-environment reds
(lint lock drift, runner binary prebuilds, the pipefail cleanup step,
rust fmt backlog, pyright import-resolution drift); none is
branch-local. Per the merge law the PR stays unmerged with the
blockers reported.

gitea-actions Bot commented 2026-10-26 02:35:11 +00:00

Copy link

Copy source

🤖 Headless spec gate (non-blocking, #204 first slice)

Spec-gate agent produced no text output — check runner logs.

Deterministic floors still govern the merge

This comment is advisory. Required checks (lint, tests, canaries, type-check, coverage, dependency lock, impact gate) remain the merge authority per docs/adversarial-review.md proportionality.

🤖 **Headless spec gate (non-blocking, #204 first slice)** Spec-gate agent produced no text output — check runner logs. <details><summary>Deterministic floors still govern the merge</summary> This comment is advisory. Required checks (lint, tests, canaries, type-check, coverage, dependency lock, impact gate) remain the merge authority per docs/adversarial-review.md proportionality.</details>

Some required checks failed

Hide all checks

CI Pipeline / lint (push) Canceled after 0s

Details

CI Pipeline / unit-tests (push) Canceled after 0s

Details

CI Pipeline / go-surface (push) Canceled after 0s

Details

CI Pipeline / rust-surface (push) Canceled after 0s

Details

CI Pipeline / canary (push) Canceled after 0s

Details

CI Pipeline / best-practices (push) Canceled after 0s

Details

CI Pipeline / documentation (push) Canceled after 0s

Details

CI Pipeline / type-check (push) Canceled after 0s

Details

CI Pipeline / gitea-offline (push) Canceled after 0s

Details

CI Pipeline / required-checks (push) Canceled after 0s

Details

CI Pipeline / CI Pipeline / notify-event-bridge (push) Canceled after 0s

Details

deploy / deploy (push) Skipped

Details

Local-Model First-Pass Review (#131) / first-pass-review (pull_request) Successful in 20s

Details

Spec gate (headless) / spec-gate (pull_request) Successful in 16s

Details

CI Pipeline / lint (pull_request) Failing after 40s

Details

CI Pipeline / go-surface (pull_request) Failing after 22s

Details

CI Pipeline / rust-surface (pull_request) Failing after 6s

Details

CI Pipeline / unit-tests (pull_request) Failing after 1m12s

Details

CI Pipeline / best-practices (pull_request) Successful in 2s

Details

CI Pipeline / documentation (pull_request) Successful in 3s

Details

CI Pipeline / type-check (pull_request) Failing after 37s

Details

CI Pipeline / canary (pull_request) Failing after 56s

Details

CI Pipeline / gitea-offline (pull_request) Successful in 17s

Details

CI Pipeline / required-checks (pull_request) Failing after 2s

Required

Details

CI Pipeline / CI Pipeline / notify-event-bridge (pull_request) Failing after 4s

Details

Some required checks were not successful.

This pull request is blocked because it's outdated.

You are not authorized to merge this pull request.

This branch is out-of-date with the base branch

View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u origin fix/data-safety-audits:fix/data-safety-audits

git checkout fix/data-safety-audits

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

Clear labels

P0 — critical

Spine/core correctness or blocks other work — do first

P1 — high

Material user value or real defect (money, bugs, next-stage dep)

P2 — medium

Quality/consolidation improvement

P3 — low

Measurement / benchmark / deferred policy

blocked

Dependencies open

bug

Confirmed defect or regression

custodian

epic

Rune v1 course-correction issue graph

feature

New capability or improvement

governance

growth

identity

in-progress

Currently being worked on by Rune

incomplete

Incomplete work — draft PR with ledger, awaiting resume

infrastructure

life

memory

memory-atlas

Memory Research Atlas child domain

needs-testing

Needs manual testing with dumbed-down steps before close

perception

ready

Order satisfied - Rune may start

release-drift

research

Research jobs — spikes, watchlists, evidence work

rune

rune

scaffolding

scaffolding

spine,correction

spine,correction

spine,deferred

spine,deferred

spine,resolved

spine,resolved

strangler,burn,endgame,deferred

strangler,burn,endgame,deferred

strangler,burn,endgame,inventory,epic

strangler,burn,endgame,inventory,epic

tui

tui

tui,epic

tui,epic

tui,openspec

tui,openspec

v1-course-correction

Rune v1 course-correction issue graph

No labels P1 — high bug governance infrastructure memory

Milestone

No items

No Milestone

Projects

Clear projects

No projects

Assignees

Clear assignees

conductor

No Assignees

2 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: conductor/rune#2624