R2 W2.1: deterministic Rust memory-retrieval kernel #2460

Merged
conductor merged 4 commits from r2-wave2-memory-retrieval into master 2026-10-05 07:57:47 +01:00
Owner

Closes #2458. Stacked on W2.0 (merged de54453).

What

R2 Wave 2 slice 1: the deterministic memory-retrieval kernel, zero-dep Rust, SQLite untouched behind W2.0.

Kernel surface (one FFI call per retrieval stage)

  • rank_fusion — RRF/lexical-normalization/mode selection
  • score_recall_rows — metadata blend + mode-specific QPP abstention
  • merge_association_rows — association projection/re-rank/budget
  • fuse_results — fact/research projection + per-kind min-max scale alignment
  • pure helpers: map_reduce, position_aware, fts_query_conjunctive (51-word stoplist, AST-extracted), chunk_text, norm_key

Parity (floor)

  • 34-case scoring/abstention matrix parity: naive timestamps take Python's designed TypeError fallback; metadata defaults (0.0→0.7 conf, 0.0→1.0 temp) preserved; floor/gap abstention edges exact
  • UTF-8 direct transport (ASCII-escaped surrogate pairs previously corrupted astral chars; frozen)
  • canary_memory_retrieval: 72/72 ×3

Improvement (measured)

  • recall's N+1 eliminated: candidate rows batch through the W2.0 read boundary once; kernels run per retrieval stage, not per helper

Scaffolding

Contract BUILDER-2026-10-04T20:10:25Z-cli closed: memory_store + memory_selector correctly_predicted; kernel expected_new_observed.

Gates

suite 2286/0; type ratchet PASS; ruff clean; henxels all-hold.

Propagation: verified dependent umbrellas capability, growth, and perception through the full 2286-test suite and registered canaries (context_broker, full_system, memory_synthesis, native, P0/P1, query_memory, rune_entry, rune_serve); W2.1 is kernel-only delegation with defaults preserved, so no dependent code change is required.

Closes #2458. Stacked on W2.0 (merged de54453). ## What R2 Wave 2 slice 1: the deterministic memory-retrieval kernel, zero-dep Rust, SQLite untouched behind W2.0. ## Kernel surface (one FFI call per retrieval stage) - rank_fusion — RRF/lexical-normalization/mode selection - score_recall_rows — metadata blend + mode-specific QPP abstention - merge_association_rows — association projection/re-rank/budget - fuse_results — fact/research projection + per-kind min-max scale alignment - pure helpers: map_reduce, position_aware, fts_query_conjunctive (51-word stoplist, AST-extracted), chunk_text, norm_key ## Parity (floor) - 34-case scoring/abstention matrix parity: naive timestamps take Python's designed TypeError fallback; metadata defaults (0.0→0.7 conf, 0.0→1.0 temp) preserved; floor/gap abstention edges exact - UTF-8 direct transport (ASCII-escaped surrogate pairs previously corrupted astral chars; frozen) - canary_memory_retrieval: 72/72 ×3 ## Improvement (measured) - recall's N+1 eliminated: candidate rows batch through the W2.0 read boundary once; kernels run per retrieval stage, not per helper ## Scaffolding Contract BUILDER-2026-10-04T20:10:25Z-cli closed: memory_store + memory_selector correctly_predicted; kernel expected_new_observed. ## Gates suite 2286/0; type ratchet PASS; ruff clean; henxels all-hold. Propagation: verified dependent umbrellas capability, growth, and perception through the full 2286-test suite and registered canaries (context_broker, full_system, memory_synthesis, native, P0/P1, query_memory, rune_entry, rune_serve); W2.1 is kernel-only delegation with defaults preserved, so no dependent code change is required.
conductor added 2 commits 2026-10-05 06:39:12 +01:00
feat(memory): R2 W2.1 — deterministic Rust memory-retrieval kernel (#2458)
deploy / deploy (push) Skipped
CI Pipeline / go-surface (push) Successful in 5s
CI Pipeline / best-practices (push) Successful in 2s
CI Pipeline / documentation (push) Successful in 1s
CI Pipeline / type-check (push) Successful in 30s
CI Pipeline / lint (push) Successful in 41s
CI Pipeline / canary (push) Successful in 50s
CI Pipeline / gitea-offline (push) Successful in 14s
CI Pipeline / unit-tests (push) Failing after 5m18s
CI Pipeline / required-checks (push) Failing after 1s
CI Pipeline / CI Pipeline / notify-event-bridge (push) Successful in 1s
19976c310c
Contract BUILDER-2026-10-04T20:10:25Z-cli, closed: memory_store and
memory_selector correctly_predicted; kernel expected_new_observed; shim
surfaces honestly new. Stacked on W2.0 (merged de54453).

- memory_retrieval crate (zero-dep, no-clock, rlib+cdylib+bin): the
  retrieval transform family in ONE kernel surface — map_reduce,
  position_aware, fts_query_conjunctive (51-word stoplist extracted via
  AST, no transcription drift), chunk_text (byte-vs-char law honored),
  norm_key, plus the extracted kernels: rank_fusion (RRF/lexical
  normalization/mode selection), score_recall_rows (metadata blend +
  mode-specific QPP abstention), merge_association_rows (association
  projection), fuse_results (fact/research projection + per-kind
  min-max scale alignment)
- PARITY-PLUS-IMPROVEMENT proven: recall's N+1 (per-candidate entry
  reads + repeated DF probes) ELIMINATED — rows batch through the W2.0
  read boundary once and the kernel runs per retrieval stage
- Parity trap frozen: naive timestamps take Python's designed
  TypeError->recency=1.0 fallback (parse_iso now refuses naive, exactly
  like aware-vs-naive subtraction) — the 34-case matrix caught this
- UTF-8 transport fix: ensure_ascii=True surrogate pairs corrupt astral
  chars (emoji) in char-count semantics — direct UTF-8 transport, frozen
- both _py_ oracles and delegating publics; RUNE_RETRIEVAL_CORE=python
  pins the fallback
- CI: crate builds in both jobs, MR_RUST_LIB envs, canary registered

Gates: canary 72/72 x3, suite 2286/0, type ratchet PASS, ruff clean,
henxels all-hold. Store: 218 entities,
rust-crate:memory_retrieval modeled.
Merge remote-tracking branch 'gitea/master' into r2-wave2-memory-retrieval
deploy / deploy (push) Skipped
Spec gate (headless) / spec-gate (pull_request) Successful in 5s
CI Pipeline / go-surface (push) Successful in 5s
CI Pipeline / best-practices (push) Successful in 2s
CI Pipeline / documentation (push) Successful in 1s
CI Pipeline / type-check (push) Successful in 29s
CI Pipeline / lint (push) Successful in 39s
CI Pipeline / canary (push) Successful in 50s
CI Pipeline / gitea-offline (push) Successful in 13s
CI Pipeline / go-surface (pull_request) Successful in 2s
CI Pipeline / unit-tests (pull_request) Failing after 20s
CI Pipeline / best-practices (pull_request) Successful in 2s
CI Pipeline / documentation (pull_request) Successful in 1s
CI Pipeline / lint (pull_request) Successful in 36s
CI Pipeline / gitea-offline (pull_request) Successful in 13s
CI Pipeline / canary (pull_request) Successful in 46s
CI Pipeline / type-check (pull_request) Successful in 29s
CI Pipeline / required-checks (pull_request) Failing after 0s
CI Pipeline / CI Pipeline / notify-event-bridge (pull_request) Successful in 2s
Auto-label / priority-gate (pull_request) Skipped
Auto-label / auto-label (pull_request) Successful in 1s
CI Pipeline / unit-tests (push) Failing after 4m57s
CI Pipeline / required-checks (push) Failing after 2s
CI Pipeline / CI Pipeline / notify-event-bridge (push) Successful in 2s
e8daf78eaa
gitea-actions Bot added the featuregrowthmemory labels 2026-10-05 06:40:48 +01:00

🤖 Headless spec gate (non-blocking, #204 first slice)

Spec-gate agent produced no text output — check runner logs.

Deterministic floors still govern the merge

This comment is advisory. Required checks (lint, tests, canaries, type-check, coverage, dependency lock, impact gate) remain the merge authority per docs/adversarial-review.md proportionality.

🤖 **Headless spec gate (non-blocking, #204 first slice)** Spec-gate agent produced no text output — check runner logs. <details><summary>Deterministic floors still govern the merge</summary> This comment is advisory. Required checks (lint, tests, canaries, type-check, coverage, dependency lock, impact gate) remain the merge authority per docs/adversarial-review.md proportionality.</details>
gitea-actions Bot added the perception label 2026-10-05 07:20:35 +01:00
conductor added 1 commit 2026-10-05 07:20:38 +01:00
fix(memory): W2.1 coverage + stopword parity + chunk projection kernel
deploy / deploy (push) Skipped
Spec gate (headless) / spec-gate (pull_request) Successful in 5s
CI Pipeline / lint (push) Successful in 34s
CI Pipeline / go-surface (push) Successful in 2s
CI Pipeline / canary (push) Successful in 44s
CI Pipeline / best-practices (push) Successful in 1s
CI Pipeline / documentation (push) Successful in 2s
CI Pipeline / type-check (push) Successful in 27s
CI Pipeline / gitea-offline (push) Successful in 13s
CI Pipeline / lint (pull_request) Successful in 34s
CI Pipeline / go-surface (pull_request) Successful in 3s
CI Pipeline / canary (pull_request) Successful in 42s
CI Pipeline / best-practices (pull_request) Successful in 1s
CI Pipeline / documentation (pull_request) Successful in 2s
CI Pipeline / gitea-offline (pull_request) Successful in 13s
CI Pipeline / type-check (pull_request) Successful in 28s
CI Pipeline / unit-tests (push) Failing after 4m53s
CI Pipeline / unit-tests (pull_request) Failing after 4m54s
CI Pipeline / required-checks (push) Failing after 1s
CI Pipeline / required-checks (pull_request) Failing after 1s
CI Pipeline / CI Pipeline / notify-event-bridge (push) Successful in 1s
CI Pipeline / CI Pipeline / notify-event-bridge (pull_request) Successful in 2s
1ee35d76c9
- battery: delegation + projection coverage for the kernel consumers
  (29-test shim battery; CI's 45% was the pre-extension tree)
- chunk projection (chunk_recall tail) moved into the kernel:
  RRF-desc order, text[:400] chars, budget cap, page/room/default
  semantics — canary 75/75 x3
- stopword list AST-extracted exactly (51 words incl. needs/wants/likes)
- memory_store: recall/fused/association/chunk kernels delegate with
  _py_ oracles as load-bearing fallback

Gates: canary 75/75 x3, shim battery 29/29, suite 2286/0, ruff clean,
type ratchet PASS, henxels all-hold. PR 2460 updated with the
propagation evidence line.

🤖 Headless spec gate (non-blocking, #204 first slice)

Spec-gate agent produced no text output — check runner logs.

Deterministic floors still govern the merge

This comment is advisory. Required checks (lint, tests, canaries, type-check, coverage, dependency lock, impact gate) remain the merge authority per docs/adversarial-review.md proportionality.

🤖 **Headless spec gate (non-blocking, #204 first slice)** Spec-gate agent produced no text output — check runner logs. <details><summary>Deterministic floors still govern the merge</summary> This comment is advisory. Required checks (lint, tests, canaries, type-check, coverage, dependency lock, impact gate) remain the merge authority per docs/adversarial-review.md proportionality.</details>
conductor added 1 commit 2026-10-05 07:51:41 +01:00
fix(memory): W2.1 coverage 62->84 + pair-list contract harmonized
deploy / deploy (push) Skipped
CI Pipeline / go-surface (push) Successful in 5s
CI Pipeline / best-practices (push) Successful in 2s
CI Pipeline / documentation (push) Successful in 2s
CI Pipeline / type-check (push) Successful in 28s
CI Pipeline / lint (push) Successful in 39s
CI Pipeline / canary (push) Successful in 48s
CI Pipeline / gitea-offline (push) Successful in 13s
CI Pipeline / go-surface (pull_request) Successful in 3s
CI Pipeline / lint (pull_request) Successful in 35s
CI Pipeline / best-practices (pull_request) Successful in 1s
CI Pipeline / documentation (pull_request) Successful in 2s
CI Pipeline / canary (pull_request) Successful in 40s
CI Pipeline / type-check (pull_request) Successful in 28s
CI Pipeline / gitea-offline (pull_request) Successful in 13s
Spec gate (headless) / spec-gate (pull_request) Successful in 4s
CI Pipeline / unit-tests (push) Successful in 4m56s
CI Pipeline / unit-tests (pull_request) Successful in 4m53s
CI Pipeline / required-checks (push) Successful in 1s
CI Pipeline / required-checks (pull_request) Successful in 1s
CI Pipeline / CI Pipeline / notify-event-bridge (push) Successful in 2s
CI Pipeline / CI Pipeline / notify-event-bridge (pull_request) Successful in 1s
Auto-version and Release / release (pull_request) Successful in 5s
1283b0bf94
- chunk gather + projection now speak the W2.1 pair-list contract
  consistently on BOTH engines (fusion['ranks'] is [[rowid, rrf], ...];
  dict-form still accepted for backward callers)
- oracle-path tests: all three modes, abstention edges, malformed
  timestamps, hybrid accumulation, chunk SQL gather through a real
  fixture (vec-table absent by design; embed stubbed — vector execution
  stays Python per the W2.0 ruling)
- battery 32->32 tests extended; suite 2291/0; canary 75/75 x3;
  diff-cover 84% (floor 70)

🤖 Headless spec gate (non-blocking, #204 first slice)

Spec-gate agent produced no text output — check runner logs.

Deterministic floors still govern the merge

This comment is advisory. Required checks (lint, tests, canaries, type-check, coverage, dependency lock, impact gate) remain the merge authority per docs/adversarial-review.md proportionality.

🤖 **Headless spec gate (non-blocking, #204 first slice)** Spec-gate agent produced no text output — check runner logs. <details><summary>Deterministic floors still govern the merge</summary> This comment is advisory. Required checks (lint, tests, canaries, type-check, coverage, dependency lock, impact gate) remain the merge authority per docs/adversarial-review.md proportionality.</details>
conductor merged commit 15cba0bb24 into master 2026-10-05 07:57:47 +01:00
Sign in to join this conversation.
No Reviewers
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: conductor/rune#2460