| 10 | Enforcement at tool/memory-write boundaries outside the model | ADOPT | CaMeL only adaptive-surviving defense (external finding); #402 carry |
| 10 | Enforcement at tool/memory-write boundaries outside the model | ADOPT | CaMeL's formal out-of-band architecture under its stated threat model (external finding); broader 2026 evidence for out-of-model enforcement; #402 carry |
@@ -8,11 +8,11 @@ _2026-09-22. Full R1_FULL run: ISSUE → INWARD → OUTWARD (4 serial arms, MAX_
**Inward reality (read-only, 2026-09-22):** Rune has NO multimodal memory. Text-only `entries` (944, zero media references); PDF ingestion ALREADY flattens documents to text (17,934 PDF chunks — layout destroyed at ingestion, the exact defect this architecture names); text embeddings are UNVERSIONED (only sqlite-vec v0.1.9 recorded — unrebuildable-with-provenance); tesseract installed and wired into nothing; provider attachments are transient input that evaporates; `RichLog(markup=True)` renders untrusted derived text. Degraded mode is not a future state — it is the present.
**Load-bearing external evidence (depths honest):** modality-specific indexes + bridge over one universal space (Liang modality gap RS; ImageBind hub-mediated, emergent ties weak 6.8-vs-36.8; CLIP §6 verbatim: fine-grained/counting/spatial failures); visual document structure is conditionally load-bearing (ColPali RS: +14–16 nDCG over OCR pipelines, figures 2.5×, but only +3.1 on text-dense pages); ASR requires VAD gating (Whisper RS: documented hallucination; ~1% fabricated sentences; 72–87% non-speech ungated); image injection lands on frontier models (>82% ASR adaptive; OCR/EXIF/QR confirmed carriers; audio 49% vs vision ~1% completion asymmetry); CaMeL is the only defense with provable security under adaptive evaluation (external finding — enforcement belongs OUTSIDE the model); C2PA normative: a soft binding shall not be used as a hard binding; NIST IR 8280 demographic-differential FR error grounds the biometric-matching-not-identity split.
**Load-bearing external evidence (depths honest):** modality-specific indexes + bridge over one universal space (Liang modality gap RS; ImageBind hub-mediated, emergent ties weak 6.8-vs-36.8; CLIP §6 verbatim: fine-grained/counting/spatial failures); visual document structure is conditionally load-bearing (ColPali RS: +14–16 nDCG over OCR pipelines, figures 2.5×, but only +3.1 on text-dense pages); ASR requires VAD gating (Whisper RS: documented hallucination; ~1% fabricated sentences; 72–87% non-speech ungated); image injection lands on frontier models (>82% ASR adaptive; OCR/EXIF/QR confirmed carriers; audio 49% vs vision ~1% completion asymmetry); CaMeL provides a formally structured out-of-band enforcement architecture under its stated threat model; broader 2026 evidence supports deterministic policy / reference-monitor enforcement OUTSIDE the vulnerable model, while independent adaptive evaluation remains limited and does not establish universal robustness or uniqueness; C2PA normative: a soft binding shall not be used as a hard binding; NIST IR 8280 demographic-differential FR error grounds the biometric-matching-not-identity split.
**Governance carried:** HYPOTHESIS-only ≠ ADOPT (all 20 claims ADOPT via DEEP_EXTERNAL / RUNE_DIRECT / DESIGN_INVARIANT, with EXTERNAL-FINDING typing where the claim is the external result itself); source references are foreign keys; zero-hit findings bounded ("this R1 search found no direct match in the searched sources as of 2026-09-22"); **#402's observed similarity-supersession defect (194015/193729) is carried as the named failure mode: cross-modal/perceptual similarity NOMINATES, never authorizes.**
Full detail: 01_inward.md. Five load-bearing gaps: (1) no source-object concept — the schema cannot say "this evidence exists"; (2) no derived-representation lineage — no transformation/model/version records anywhere; (3) document ingestion already destroys modality (PDF→flat text; no page/layout/figures); (4) no modality-capability truth (provider support undeclared/unverifiable from persisted config); (5) no media storage/lifecycle/security surface at all. Classification: CANONICAL 2 · ACTIVE 4 · three PARTIAL-class mechanisms · ABSENT 8 · BYPASS 3 · DORMANT 1 · UNKNOWN 3 (row truth in 01_inward.md).
@@ -63,7 +63,7 @@ Content-addressed files (SHA-256 layout) + SQLite metadata + small thumbnails; S
Metadata = source-asserted evidence (NIST SP 800-86 forensics treatment); EXIF ≠ event time; timestamp taxonomy observed_at/asserted_capture/verified_event; geolocation stripped-by-default in disclosures (leakage documented; arm 4 AE partial). Sensitive inference: faces leak orientation 81–91%; LLM profilers 86.7% aggregated — harm is emergent across store growth; inferred sensitive traits NEVER persist as facts by default (C012). Biometric similarity = workflow match surfaced to user (FISWG operational-vs-forensic precedent; NIST IR 8280 demographic-differential error); identity authority stays with #402's authentication semantics.
Injection carriers confirmed: typographic/patch attacks (persistent through 2025 SOTA), EXIF/QR/OCR-text (MMPIBench), invisible characters surviving commercial pipelines; adaptive black-box >82% ASR vision, 49% audio completion (asymmetry: audio attacks complete far more often); multimodal RAG poisoning: 5 pairs → 98% ASR. **Taint survives every transformation (C007): transformation may ADD interpretation, never ERASE origin.** CaMeL = only defense with provable security under adaptive evaluation (EXTERNAL-FINDING) — enforcement at tool/memory-write boundaries outside the model; Spotlighting-style provenance marking as cheap complement. No direct study tracks taint into persistent cross-session memory stores (bounded zero-hit) — convergent indirect evidence only; E16 is the designed test. Synthetic media: provenance not verdicts (C018); detectors unreliable (sink-class generalization failure; ASVspoof 5 compromised; EER 4–20%+) — metadata, never automatic decisions.
Injection carriers confirmed: typographic/patch attacks (persistent through 2025 SOTA), EXIF/QR/OCR-text (MMPIBench), invisible characters surviving commercial pipelines; adaptive black-box >82% ASR vision, 49% audio completion (asymmetry: audio attacks complete far more often); multimodal RAG poisoning: 5 pairs → 98% ASR. **Taint survives every transformation (C007): transformation may ADD interpretation, never ERASE origin.** CaMeL's formally structured out-of-band enforcement architecture (its formal guarantees hold within its stated threat model; broader 2026 evidence supports enforcement OUTSIDE the vulnerable model, with independent adaptive evaluation limited — no universal-robustness claim) — enforcement at tool/memory-write boundaries outside the model; Spotlighting-style provenance marking as cheap complement. No direct study tracks taint into persistent cross-session memory stores (bounded zero-hit) — convergent indirect evidence only; E16 is the designed test. Synthetic media: provenance not verdicts (C018); detectors unreliable (sink-class generalization failure; ASVspoof 5 compromised; EER 4–20%+) — metadata, never automatic decisions.
## 19. UI, code, accessibility (mandate §42–44)
Structured UI state (accessibility tree/DOM) primary; screenshot = supporting visual evidence (C019). Rendered artifacts never replace source files. RichLog(markup=True) is a LIVE injection surface for derived text (arm 4 F6 SUCCEEDS) — escape-by-default rendering contract (R; #402 safe-rendering carry). Accessibility derivatives support inspection but remain derivatives (correctable per C016).
Row-derived: CONFIRMED 6 · REFINED 1 · OVERTURNED 0 · STILL_UNKNOWN 0 (Sum 7). P-4 REFINED: "screenshots not memory entries" ≠ "screenshots not kept" — they are source objects with derived projections. P-5 REFINED: version-stamping precondition + multi-vector (not single-vector) for any future visual lane.
Row-derived: CONFIRMED 6 · REFINED 1 · OVERTURNED 0 · STILL_UNKNOWN 0 (Sum 7). P-4 CONFIRMED, with rationale clarified: "screenshots not memory entries" does not mean "screenshots not retained" — they are preserved as source objects with derived projections. P-5 REFINED: version-stamping is a precondition and any future visual retrieval lane must be multi-vector rather than a single-vector lane.
## 29. Claim ledger
claims.json: 20 claims, all ADOPT, all RUNE_APPLICABLE, each with threat_model, experiment, falsifier, kill criterion. Load-bearing map: load_bearing_evidence.md (DEEP_EXTERNAL discipline audited row-by-row; no shallow anchor labeled DEEP_EXTERNAL; EXTERNAL-FINDING typing where the claim is the external result).
@@ -107,19 +107,20 @@ E1–E27 per mandate (§67), mapped to claims. Prerequisites flagged by arm 4: l
- Dense-lane value at n≈924 (text): carried unmeasured from #383.
- Cross-version embedding degradation: no direct benchmark found (searched sources as of 2026-09-22) — established via dimension/geometry divergence instead.
- Standalone VLM-rerank evaluation: not found (searched sources as of 2026-09-22).
- LLM-era image steganography / subtitle-codec injection: zero verified hits — do not cite.
- LLM-era image steganography / subtitle-codec injection: this R1 search found no direct verified work in the searched sources as of 2026-09-22 — do not cite unverified candidates.
- Provider-side attachment retention policies: UNKNOWN (not readable without live probing; out of scope).
- Live per-model vision support: UNKNOWN (undeclared in persisted config).
The original ledger was built with character-truncation normalizers (fields cut at 70/120/130/140/150 chars) — repaired by a LOSSLESS rebuild from the arm reports: every cell passes through verbatim; unrecored fields are NOT_RECORDED_IN_ARM, never invented. **Exact accounting: SOURCE_ROWS = 108 · SOURCE_UNIQUE = 104 · SOURCE_DUPLICATE_GROUPS = 4** (ColBERT arXiv:2004.12832 → A1-28+A2-11; ColPali arXiv:2407.01449 → A1-29+A2-1; VideoCLIP arXiv:2109.14084 → A1-30+A2-23; pyannote.audio arXiv:1911.01255 → A2-19+A2-31 pointer row). InternVideo (2212.03191) and InternVideo2 (2403.15377) are different papers. **Depths (one primary label per row): FULL_PAPER 2 (Brady et al. 2008 PNAS; POPE 2305.10355) · RELEVANT_SECTIONS 8 (ColPali, Whisper, CLIP §6, Liang, ImageBind, CLIP-version cluster, Zacks, +1) · ABSTRACT_ONLY 73 · METADATA_VERIFIED 13 · DOCUMENTATION_AUDITED 11 (C2PA, SQLite, ExifTool, NIST ×2, GDPR, FISWG, Apple, pyannote, Tesseract, DAM — fully read documentation/standards, never FULL_PAPER) · UNVERIFIED 1.** Depth honesty: type ≠ depth held throughout; the do-not-cite register records 16 caught mis-paired arXiv IDs + non-locatable items + LLM-era steganography/subtitle-injection bounded zero-hits — recorded, not silently omitted. Arm-4 source closure: all 30 arXiv IDs cited by the adversarial pass resolve to existing ledger rows (no orphan evidence; its 2 direct re-fetches were of already-ledgered sources). CaMeL wording corrected corpus-wide: formal out-of-band architecture under its stated threat model + broader 2026 support for out-of-model enforcement — no uniqueness claim, no attribution of Progent's adaptive result to CaMeL.
## 33. Evidence saturation
evidence_saturation.md: **SEARCH_SATURATION = REACHED** (typed gaps, not exhaustion) · **LOAD_BEARING_DEPTH = PASS** (every ADOPT claim mapped; DEEP_EXTERNAL only at FULL_PAPER/RELEVANT_SECTIONS).
## 34. Process truth
CONTINUOUS_EXECUTION = PASS — zero user continuations in the #403 window; stages executed as internal checkpoints. MAX_CONCURRENT_SUBAGENTS = 1 held: arms strictly serial (general-17 → general-18 → general-19 → general-20), all first-pass SUCCESS, no stalls/retries. Source-access: Europe PMC XML 500s → PMC pages; arXiv ID mis-pairing hazard CONFIRMED 16× across arms (export-API title search adopted as the reliable path). Canonical Memory write: performed pre-freeze; receipt recorded in mutation_audit.md (intended ≠ executed verified). THIS RESEAL accounting: CANONICAL_MEMORY_WRITE = 0 · DERIVED_MEMORY_WRITE = 0 (post-freeze protocol enforced).
CONTINUOUS_EXECUTION = PASS — zero user continuations in the #403 window; stages executed as internal checkpoints. MAX_CONCURRENT_SUBAGENTS = 1 held: arms strictly serial (general-17 → general-18 → general-19 → general-20), all first-pass SUCCESS, no stalls/retries. Source-access: Europe PMC XML 500s → PMC pages; arXiv ID mis-pairing hazard CONFIRMED 16× across arms (export-API title search adopted as the reliable path). Canonical Memory write: performed pre-freeze; receipt recorded in mutation_audit.md (intended ≠ executed verified). THIS RESEAL accounting: CANONICAL_MEMORY_WRITE = 0 · DERIVED_MEMORY_WRITE = 0 (post-freeze protocol enforced).**Supersession failure status: REPRODUCED** — 194015/193729 (#402) and 194460/194173 (#403, sim 0.81, superseding #395's summary) are two observed cross-domain supersession events by similarity alone. **SAFETY HOLD: no canonical/derived Memory writes for #404–#406 closeouts until a scope/domain-aware supersession authority exists and is proven** (CANONICAL_MEMORY_IMPORT_PENDING_SAFE_SUPERSESSION_FIX); domain summaries persist in master + registry + Gitea instead. SIMILARITY may nominate; durable supersession requires scope/domain ownership, provenance, explicit relation semantics, write authority, supersession reason, compatible record type, and same-object evidence.
## 35. Artifact pointers
- 00_issue_scope.md — canonicalization + scope + stage plan
@@ -128,7 +129,7 @@ CONTINUOUS_EXECUTION = PASS — zero user continuations in the #403 window; stag
- 02_outward/arm2_retrieval_rag.md — 30 sources; ColPali/Whisper deep reads; cascade verdict
{"id":"AB-R1-C004","claim":"Every embedding lane stamped (model_id, model_version, space_id); embeddings are derived, rebuildable, never canonical evidence; index rebuilds wholesale on model change (dual-index for one verification pass)","evidence_for":"Arm4 S,ET,R SUCCEEDS/PARTIAL against unstamped; arm1 B12 RS: versions differ in dimension (512/768/1024) and geometry; arm2: ColPali vs ColQwen2 >5 nDCG on identical data; inward: production embeddings have NO model recorded (only sqlite-vec lib version v0.1.9)","evidence_against":"Rebuild cost (minutes at Rune scale — not load-bearing)","limitations":"—","inward":"01_inward G: unversioned embeddings = unrebuildable-with-provenance","transfer_assumptions":"—","classification":"ADOPT","evidence_level":"RUNE_APPLICABLE","owner":"#403","architecture_impact":"embedding schema + migration runbook","experiment":"E14","falsifier":"Evidence that embedding spaces are version-stable","kill_criterion":"E14 shows version-mixed indexes harmless","threat_model":"silent ranking corruption on model swap"},
{"id":"AB-R1-C005","claim":"Retrieval: score-everything fused lanes (FTS5 text + metadata + modality-native multi-vector) + late-interaction rerank + sufficiency-gated abstention; NO multi-stage cascade; cross-modal similarity NOMINATES, never authorizes supersession/deletion","evidence_for":"#383 carry (cascade = scale artifact at n≈924); arm2: SOTA multimodal retrieval is itself score-everything at 100-1600 docs; multi-vector late interaction beats single-vector (ColPali RS); #402 RUNE_DIRECT: similarity superseded unrelated cross-domain entry (194015/193729)","evidence_against":"Latency negligible at scale — no counterevidence","limitations":"Dense-lane value at n≈924 still unmeasured (carried)","inward":"sqlite-vec present; small corpus","transfer_assumptions":"—","classification":"ADOPT","evidence_level":"RUNE_APPLICABLE","owner":"#403","architecture_impact":"retrieval design; dedupe/supersession gate","experiment":"E7, E8, E9","falsifier":"Measured corpus growth where cascade wins","kill_criterion":"E9 shows similarity-threshold dedupe safe (it will not)","threat_model":"similarity-driven supersession; false merges"},
{"id":"AB-R1-C006","claim":"Evidence independence counted by ROOTS: transcript+caption+OCR+embedding of one source = ONE evidence root; derivative multiplicity never counts as corroboration","evidence_for":"#397 carry (derivation algebra); arm1 ImageBind: derivatives of one source share its root; arm2 finding 10; arm4 X,Y,Z FAILS","evidence_against":"—","limitations":"—","inward":"—","transfer_assumptions":"—","classification":"ADOPT","evidence_level":"RUNE_APPLICABLE","owner":"#403","architecture_impact":"evidence-set construction at context assembly","experiment":"E10","falsifier":"A case where derivative multiplicity adds real evidentiary weight without new roots","kill_criterion":"E10 counts four confirmations","threat_model":"corroboration inflation; self-confirmation"},
{"id":"AB-R1-C007","claim":"Multimodal content = DATA, never control authority; taint survives every transformation (OCR/caption/transcript of hostile media stays UNTRUSTED); enforcement at tool/memory-write boundaries OUTSIDE the model (CaMeL pattern); transformed content inherits source labels — transformation may add interpretation, never erase origin","evidence_for":"Arm3: image injection >82% ASR adaptive on frontier models; OCR/EXIF/QR confirmed carriers (MMPIBench); audio 49% completion vs vision ~1% (asymmetry); CaMeL only defense with provable security under adaptive eval; Bad Characters invisible chars surviving commercial pipelines; arm4 AN-AT FAILS; #402 carry (content/control separation, policy discipline)","evidence_against":"AS sanitizing OCR text PARTIAL (defense-in-depth helps but insufficient)","limitations":"No direct study tracks taint into persistent cross-session memory stores (bounded zero-hit) — convergent indirect evidence only","inward":"RichLog(markup=True) renders untrusted derived text; ingestion trusts all documents","transfer_assumptions":"—","classification":"ADOPT","evidence_level":"RUNE_APPLICABLE","owner":"#403 (seam with #402)","architecture_impact":"ingestion taint labels; write-boundary gates","experiment":"E15, E16","falsifier":"A transformation provably removing all adversarial content without semantic loss","kill_criterion":"E16 shows sanitized OCR safe (it will not)","threat_model":"image→OCR→prompt control path; RAG poisoning"},
{"id":"AB-R1-C007","claim":"Multimodal content = DATA, never control authority; taint survives every transformation (OCR/caption/transcript of hostile media stays UNTRUSTED); enforcement at tool/memory-write boundaries OUTSIDE the model (CaMeL pattern); transformed content inherits source labels — transformation may add interpretation, never erase origin","evidence_for":"Arm3: image injection >82% ASR adaptive on frontier models; OCR/EXIF/QR confirmed carriers (MMPIBench); audio 49% completion vs vision ~1% (asymmetry); CaMeL provides a formally structured out-of-band enforcement architecture under its stated threat model; broader 2026 evidence supports deterministic policy / reference-monitor enforcement OUTSIDE the vulnerable model, while independent adaptive evaluation remains limited and does not establish universal robustness; Bad Characters invisible chars surviving commercial pipelines; arm4 AN-AT FAILS; #402 carry (content/control separation, policy discipline)","evidence_against":"AS sanitizing OCR text PARTIAL (defense-in-depth helps but insufficient)","limitations":"No direct study tracks taint into persistent cross-session memory stores (bounded zero-hit) — convergent indirect evidence only","inward":"RichLog(markup=True) renders untrusted derived text; ingestion trusts all documents","transfer_assumptions":"—","classification":"ADOPT","evidence_level":"RUNE_APPLICABLE","owner":"#403 (seam with #402)","architecture_impact":"ingestion taint labels; write-boundary gates","experiment":"E15, E16","falsifier":"A transformation provably removing all adversarial content without semantic loss","kill_criterion":"E16 shows sanitized OCR safe (it will not)","threat_model":"image→OCR→prompt control path; RAG poisoning"},
{"id":"AB-R1-C008","claim":"Audio enters memory only as VAD-gated, timestamped, provenance-linked transcripts with the waveform retained; transcript ≠ recording; ASR hallucination assumed (~1% fabricated sentences; 72-87% non-speech hallucination ungated); speaker diarization = metadata enrichment, never identity","evidence_for":"Arm2 Whisper RS (long-form failure modes documented by OpenAI itself); Careless Whisper ~1% fabricated (38% harmful); Calm-Whisper 3/20 heads >75% non-speech hallucinations; WhisperX VAD gating; arm4 B,AG,AH,AK FAILS","evidence_against":"—","limitations":"WER on accented/noisy audio still material","inward":"no audio path exists — prospective design","transfer_assumptions":"—","classification":"ADOPT","evidence_level":"RUNE_APPLICABLE","owner":"#403","architecture_impact":"audio ingestion design (future)","experiment":"E4","falsifier":"ASR with measured near-zero hallucination without gating","kill_criterion":"E4 shows transcripts silently authoritative","threat_model":"fabricated transcript lines; speaker mislabeling"},
{"id":"AB-R1-C009","claim":"Capture-first document ingestion: retain page images + layout + extraction (text where native); modality-native structure preferred over flat text; text-first is sufficient ONLY for native prose","evidence_for":"Arm2 ColPali RS (+14-16 nDCG over OCR pipelines; figures 2.5×; tables +13.1 with Tesseract's own docs conceding table failure; TAT-DQA +3.1 only on text-dense = the conditional); arm4 C,A FAILS","evidence_against":"Storage cost of page images (bounded by tiering C015)","limitations":"Handwriting numbers unmeasured (bounded gap)","inward":"PDF flattening shipping = the exact defect","transfer_assumptions":"—","classification":"ADOPT","evidence_level":"RUNE_APPLICABLE","owner":"#403","architecture_impact":"ingestion pipeline redesign","experiment":"E2, E6","falsifier":"Evidence OCR-first matches visual retrieval on figure/table/chart tasks","kill_criterion":"E2 shows OCR errors never propagate","threat_model":"OCR-error-as-truth; layout destruction"},
{"id":"AB-R1-C010","claim":"Perceptual similarity (pHash/embedding distance) is a DISCOVERY/nomination signal only — byte-identity is the only automatic dedupe; similarity NEVER authorizes supersession, deletion, or merge (C2PA: a soft binding shall not be used as a hard binding)","evidence_for":"C2PA §9 doc; arm3 NeuralHash collision/framing attacks; arm3 DB-poisoning (5%→40% surveillance); #402 RUNE_DIRECT: observed cross-domain supersession defect (194015/193729); arm4 G,H,BB,BC,BD FAILS","evidence_against":"—","limitations":"—","inward":"dedupe_receipts use similarity for supersession decisions today","transfer_assumptions":"—","classification":"ADOPT","evidence_level":"RUNE_APPLICABLE","owner":"#403","architecture_impact":"dedupe policy: exact-bytes only automatic","experiment":"E8, E9","falsifier":"Perceptual hash family with proven collision resistance sufficient for deletion decisions","kill_criterion":"E9 destructive merge","threat_model":"framing attack; wrong-supersession"},
@@ -19,4 +19,4 @@ Arms 1-3 + adversarial arm 4; ~124 source rows across foundations (31), retrieva
**SEARCH_SATURATION = REACHED** — late searches returned typed gaps and do-not-cites, not new architecture-changing mechanisms; 8 typed gaps recorded as bounded-search findings.
**LOAD_BEARING_DEPTH = PASS** — every ADOPT claim maps through load_bearing_evidence.md to DEEP_EXTERNAL (RELEVANT_SECTIONS/FULL_PAPER anchors: ColPali, Whisper, CLIP §6, Liang, ImageBind, CLIP-version cluster, Zacks, POPE-at-full-paper for measurement context), RUNE_DIRECT (inward facts: flattening shipping, unversioned embeddings, similarity-supersession defect, RichLog surface, documents-table precedent), or DESIGN_INVARIANT; EXTERNAL-FINDING typing used only where the claim is the external result itself (CaMeL adaptive-eval survival; detector unreliability). No HYPOTHESIS-support ADOPT (rule carried). Source TYPE and DEPTH separate throughout; documentation never FULL_PAPER.
**LOAD_BEARING_DEPTH = PASS** — every ADOPT claim maps through load_bearing_evidence.md to DEEP_EXTERNAL (RELEVANT_SECTIONS/FULL_PAPER anchors: ColPali, Whisper, CLIP §6, Liang, ImageBind, CLIP-version cluster, Zacks, POPE-at-full-paper for measurement context), RUNE_DIRECT (inward facts: flattening shipping, unversioned embeddings, similarity-supersession defect, RichLog surface, documents-table precedent), or DESIGN_INVARIANT; EXTERNAL-FINDING typing used only where the claim is the external result itself (CaMeL's formal out-of-band enforcement architecture under its stated threat model; detector unreliability — with independent adaptive evaluation remaining limited and no uniqueness claim). No HYPOTHESIS-support ADOPT (rule carried). Source TYPE and DEPTH separate throughout; documentation never FULL_PAPER.
| AB-R1-C007 | multimodal content = data; taint survives transformation; enforce outside model | RichLog(markup=True) renders untrusted derived text; ingestion trusts all documents | CaMeL 2503.18813 (A3, ABSTRACT_ONLY — external finding only: only defense with provable security under adaptiveeval); MMPIBench (A3, ABSTRACT_ONLY); Image Hijacks 2309.00236 (A3, ABSTRACT_ONLY) | DESIGN_INVARIANT + EXTERNAL finding (the claim is only the external result — CaMeL's measured adaptive-eval survival — not a Rune prescription from these reads) | no direct taint-into-persistent-store study (bounded zero-hit) | E16 sanitized OCR safe |
| AB-R1-C007 | multimodal content = data; taint survives transformation; enforce outside model | RichLog(markup=True) renders untrusted derived text; ingestion trusts all documents | CaMeL 2503.18813 (A3, ABSTRACT_ONLY — external finding only: formally structured out-of-band enforcement with formal guarantees under its stated threat model; NOT evidence of unique adaptive-eval survival); MMPIBench (A3, ABSTRACT_ONLY); Image Hijacks 2309.00236 (A3, ABSTRACT_ONLY) | DESIGN_INVARIANT + EXTERNAL finding (the claim is only the external result — CaMeL's formal out-of-band architecture within its stated threat model — not a Rune prescription from these reads, and not a claim of unique adaptive robustness) | no direct taint-into-persistent-store study (bounded zero-hit) | E16 sanitized OCR safe |
| AB-R1-C008 | VAD-gated timestamped transcripts; transcript ≠ recording | no audio path — prospective | Whisper 2212.04356 (A2-13, RELEVANT_SECTIONS — documented failure modes); Careless Whisper (A2-14, ABSTRACT_ONLY) | DEEP_EXTERNAL + DESIGN_INVARIANT | WER on hard audio material | E4 transcripts silently authoritative |
STATUS: CONTENT-FREEZE AUDIT — final-state declaration; this file is the ACTUAL LAST write. After this write, NO file is modified; only read-only gates, git transport, and Gitea receipts follow.
STATUS: CONTENT-FREEZE AUDIT — final-state declaration; this file is the ACTUAL LAST write. Supersedes the audit sealed at `c322801`. After this write, NO file is modified; only read-only gates, git transport, and Gitea receipts follow.
## Run provenance
## Base seal accepted
Full new-domain R1_FULL run of the 77-section mandate. ONE controller; arms strictly serial (general-17 → general-18 → general-19 → general-20), all first-pass SUCCESS; MAX_CONCURRENT_SUBAGENTS = 1 held. **CONTINUOUS_EXECUTION = PASS** — zero user continuations in the #403 window. Source-access faults: Europe PMC XML 500s → PMC pages; arXiv ID mis-pairing hazard confirmed 16× across arms (export-API title search adopted as the reliable verification path). Research-only boundary held: no production Memory/schema/media/index/OCR/ASR/TUI changes; inward read-only.
The substantive state of **`c322801` is ACCEPTED in full**: all #403 research, the PRESERVE-THE-SOURCE/DERIVE-THE-INTERPRETATION architecture (every element enumerated in mandate §1), the 20 ADOPT claims, and the 170-attack adversarial record. No research rerun; no new arms; no architecture rewrite; no claim reclassification. **This repair changed evidence ACCOUNTING and wording only.**
RESEARCH_TOOLING_MUTATION = YES (research-only) — scripts/research_gate_check.py: `_LBM_DOMAINS` and `_EXACTREF_DOMAINS` extended to include "AB-" (zero-hit governance, load-bearing depth map, exact source-reference resolution, post-freeze protocol, Gap-row governance now cover this domain). Gate changes are pre-freeze; negative controls for these check families were proven at #402 closeout (NC-E–NC-K); no new check logic was introduced for AB, only domain-scope extension.
RESEARCH_RESULT_AFFECTED = NO — the batched master (sha256 5c705036c791cb463133890d4af5a2dbd84084eb5deed9f493ebc7f8994c4bc8) is byte-identical to the pre-run artifact, preserved as AB_MULTIMODAL_ARTIFACT_MEMORY_R1_BATCHED_PRELIMINARY.md; family file Section AB untouched; prior domains untouched.
REGISTRY_MUTATION = YES (pre-freeze) — domain_registry.md row 403 rewritten from batched-era text ("single-writer stance confirmed; staged earn-in" NOT inherited) to this run's row truth.
SESSION_NOTES_WRITE = 0.
The original sources.md was built by normalizer code that truncated fields at 70/120/130/140/150 characters — collapsing qualifier, limitation, method, and identity detail. **Rebuilt LOSSLESSLY from the already-persisted arm reports**: every table cell passes through verbatim; fields the arm tables did not carry as separate columns are recorded NOT_RECORDED_IN_ARM rather than invented; no truncation anywhere in the new ledger. Schema per mandate §3: source_id · identifier · title/authors/year/venue · modality/task · key claims · negative results · limitations · read_depth · Rune relevance.
Executed pre-freeze via memory_store.py add. **Observed persisted result:** entry **194460 created active** (topic project, importance 0.8); dedupe decision **superseded**; **prior_id 194173, similarity 0.81** — and 194173 is the **#395 distributed/multi-instance summary, NOT a #403 entry**. This is the **#402 cross-domain supersession defect recurring exactly as the mandate predicted (§74)**: similarity-based dedupe matched across domains and superseded an unrelated canonical summary. Per mandate: the receipt is preserved as OBSERVED evidence, flagged as a cross-domain supersession defect, NOT treated as valid lineage, and NOT repaired by another write. The defect is now observed twice (194015/193729 at #402; 194460/194173 here) — the enforce-write-contract correction (arm 4 correction 1: actor+capability+reason-code+evidence-class at the write boundary) is the standing production follow-through; similarity must nominate, never authorize supersession. No cleanup write performed.
**SOURCE_ROWS = 108 · SOURCE_UNIQUE = 104 · SOURCE_DUPLICATE_GROUPS = 4.** Duplicate map: arXiv:2004.12832 ColBERT → A1-28 + A2-11 · arXiv:2407.01449 ColPali → A1-29 + A2-1 · arXiv:2109.14084 VideoCLIP → A1-30 + A2-23 · arXiv:1911.01255 pyannote.audio → A2-19 + A2-31 (A2-31 is a deliberate cross-reference/pointer row, honestly labeled, not an independent source). The old master claims were WRONG: "~103 unique" was an approximation and the named 4-group set (CLIP, ColBERT, VideoCLIP, InternVideo) was inaccurate — CLIP appears once; InternVideo (2212.03191) and InternVideo2 (2403.15377) are different papers; the fourth true group is pyannote. Every duplicate group is now explicit in the ledger header.
## Row truth at freeze (machine-derived)
## Depth recount (mandate §23 — one primary label per row)
**FULL_PAPER 2 · RELEVANT_SECTIONS 8 · ABSTRACT_ONLY 73 · METADATA_VERIFIED 13 · DOCUMENTATION_AUDITED 11 · UNVERIFIED 1 = 108.** Three rows were normalized during the rebuild: A3-30 and A3-32 (dual-depth-label cells → single primary label with the depth nuance preserved in the limitations cell) and A1-20 (synthesis row's "—" limitations → honest NOT_RECORDED_IN_ARM note that limitations live in the negatives cell). Documentation/standards remain DOCUMENTATION_AUDITED even when fully read (type ≠ depth). The old totals (ABS 72/DOC 10/UNV 3) were artifacts of the truncating build; the new totals are mechanical gate-parser truth.
## Preserved
Batched preliminary byte-identical (sha 5c705036…9c4bc8) · family Section AB untouched · all prior sealed domains untouched · #402 governance carried (HYPOTHESIS-only ≠ ADOPT; DEEP_EXTERNAL depth discipline; foreign-key source refs; bounded zero-hit; post-freeze protocol with automatic FREEZE_STALE).
## Arm-4 source closure (mandate §7)
All 30 arXiv IDs cited by arm 4 / 05_adversarial.md resolve to existing canonical ledger rows — **no orphan evidence**. Arm 4's two direct re-fetches (2309.00236 Image Hijacks; 2503.18813 CaMeL) were of already-ledgered sources; no duplicate rows created for rereads.
Every external source reference in claims.json, load_bearing_evidence.md, and the master resolves EXACTLY to one ledger row with identity adjacency (gate-enforced; the A1/B-numbering from the original build was corrected to ledger IDs A1-21/24/25/32, A2-1/13/21/29). Full C001–C020 support↔source↔identity↔depth↔proposition audit re-run: no DEEP_EXTERNAL rests on a shallow anchor; the EXTERNAL-FINDING typings are accurately scoped. **No claim was downgraded — no support proved shallower than recorded.**
The corpus contained "CaMeL is the only defense with provable security under adaptive evaluation" / "only adaptive-surviving defense" — conflating (A) CaMeL's formal architectural result under its stated threat model, (B) CaMeL's own empirical evaluation, and (C) a 2026 adaptive-evaluation paper's independent result. **Corrected everywhere** (master ×2, claims.json C007, load_bearing_evidence.md ×2, 03_applicability row 10, evidence_saturation.md) to: "CaMeL provides a formally structured out-of-band enforcement architecture under its stated threat model; broader 2026 evidence supports deterministic policy / reference-monitor enforcement OUTSIDE the vulnerable model, while independent adaptive evaluation remains limited and does not establish universal robustness or uniqueness." No Progent-specific adaptive result is attributed to CaMeL; the architecture conclusion (enforcement outside the vulnerable model) is preserved; no accurate source-attributed statement was weakened. Sweep for "only defense / only adaptive / adaptive-surviving": zero remaining instances.
## Wording corrections (mandate §13–§15)
- **P-4/P-5 (master §28):** "P-4 REFINED" corrected to "P-4 CONFIRMED, with rationale clarified: 'screenshots not memory entries' does not mean 'screenshots not retained' — they are preserved as source objects with derived projections. P-5 REFINED: version-stamping is a precondition and any future visual retrieval lane must be multi-vector rather than single-vector." Row truth 6C/1R unchanged (the preliminary source file was already correct).
- **Applicability pointer (master §35):** stale "33/0/1/5/14/1" → "32/0/1/6/14/1" (row truth); corpus sweep confirms no live "33/0/1/5/14/1" remains.
- **Zero-hit (master §31 + sources register):** "zero verified hits" → bounded: "this R1 search found no direct verified work in the searched sources as of 2026-09-22 — do not cite unverified candidates."
## Gate hardening (mandate §16–§17)
- **Zero-hit backstop (smallest fix):** `_PATTERNS` gained `\bzero verified hits\b` — the exact escaped form. Typed Gap-row governance remains the primary mechanism; regex stays a backstop.
- **NEW `check_rich_source_ledger` (AB-scoped):** rich-ledger header must expose all required columns (source_id, identifier, title, key claims, negative results, limitations, read depth, Rune relevance); every row needs ≥8 fields; no row may carry multiple depth labels in one cell; no empty/placeholder semantic fields on claims/limitations. Older sealed domains deliberately NOT held to the richer schema (no format-drift cascade, per mandate §17).
- **Memory-write safety hold recorded in master §34 + registry (mandate §19–§21):** the cross-domain supersession failure is now REPRODUCED (194015/193729 at #402; 194460/194173 at #403, sim 0.81, superseding #395's summary). SIMILARITY may nominate; durable supersession requires scope/domain ownership, provenance, explicit relation semantics, write authority, supersession reason, compatible record type, and same-object evidence. **CANONICAL_MEMORY_IMPORT_PENDING_SAFE_SUPERSESSION_FIX: no `memory_store.py add` for #404–#406 closeouts until a scope/domain-aware supersession authority exists and is proven; domain summaries persist in master + registry + Gitea.** This is a safety hold, not abandonment; no production Memory code changed in this research repair.
## Negative controls (mandate §18 — BEFORE freeze, all restored byte-identically)
- **NC-L** blank a load-bearing row's limitations → **GATE FAIL (CAUGHT)** (after extending the placeholder check to cover the limitations cell — the initial miss was a check-coverage gap, itself fixed pre-freeze).
- **NC-M** add a row with a different row ID but the same arXiv identity → **identity detected: SOURCE_UNIQUE stayed 104 (did NOT increase); the arXiv:2407.01449 group grew 2→3 members** — duplicate identity recognized as joining an existing group.
- **NC-N** point a load-bearing reference at nonexistent A9-999 → **GATE FAIL (CAUGHT)**.
- **NC-O** insert "zero verified hits" → **GATE FAIL (CAUGHT)**; bounded equivalent ("this R1 search found no direct verified hit in the searched sources as of 2026-09-22") → **PASS**.
**CANONICAL_MEMORY_WRITE = 0. DERIVED_MEMORY_WRITE = 0.** Observed supersession history preserved untouched and un-repaired (192685; 194015/1515/193729; 194460/1516-era receipt 194173/sim 0.81; fact-9293). No cleanup writes.
## Process truth
Original #403 process truth PRESERVED (CONTINUOUS_EXECUTION = PASS; zero user continues; MAX_CONCURRENT_SUBAGENTS = 1; four serial first-pass arms). This repair: controller-only, MAX_CONCURRENT_SUBAGENTS = 0, zero user continuations, no bounded reread needed (arm reports carried all required fields).
## Post-freeze protocol (governing)
After this audit: ALLOWED = read-only gate execution (AB + AA/Z/Y/X/W R1_FULL regression), git status/diff/hash, `git show`, the ONE seal commit, push origin + gitea, mirror verification, Gitea receipt posting, read-only file opening, final natural-language response. FORBIDDEN = negative controls, temporary tracked-file edits, artifact/registry/tooling edits, session-note appends, memory_store.py add/update/delete, memory_consolidation.py consolidate-fact, any canonical/derived state write. If ANY forbidden post-freeze write occurs: **FREEZE_STALE = TRUE automatically** — reseal only via a genuinely NEW final audit written LAST. Nothing durable follows the seal.
After this audit: ALLOWED = read-only gates (AB + AA/Z/Y/X/W R1_FULL), git inspection, the ONE seal commit, push origin + gitea, mirror verification, Gitea receipts, read-only file opening, final natural-language response. FORBIDDEN = negative controls, temporary tracked-file edits, artifact/registry/tooling edits, sessionnotes, memory_store.py, memory_consolidation.py, cleanup/tidying. If ANY forbidden post-freeze write occurs: **FREEZE_STALE = TRUE automatically** — reseal only via a genuinely NEW final audit written LAST. Nothing durable follows the seal.
## Freeze declaration
All mutations (artifacts + ledger + claims + registry + gate domain-scope + canonical Memory write) are complete BEFORE this audit. **This audit is the ACTUAL LAST write.** Sequence: FINAL_CONTENT_GATE (read-only) → ONE seal commit → push/mirror/verify → Gitea #403 + #375 receipts → STOP. STOP MEANS STOP.
## Registry edit record (pre-freeze)
domain_registry.md row 403 rewritten from batched-era text to this run's row truth immediately before this audit's final write; this audit remains the ACTUAL LAST write (registry mtime precedes this file).
All mutations (lossless ledger + exact accounting + FK repairs + CaMeL wording ×6 surfaces + P-4/P-5 + pointer + zero-hit + gate hardening + safety-hold recording + master/registry convergence) and all five negative controls are complete BEFORE this audit. **This audit is the ACTUAL LAST write.** Sequence: FINAL_CONTENT_GATE (read-only) → ONE seal commit → push/mirror/verify → corrected Gitea #403 + #375 receipts → STOP. STOP MEANS STOP.
# #403 AB — Source Ledger (consolidated from arms 1–3; adversarial-verdict rows are in 05_adversarial.md, not here; 2026-09-22)
# #403 AB — Source Ledger (LOSSLESS rebuild from arm reports 1–3, 2026-09-22)
Read depths: FULL_PAPER / RELEVANT_SECTIONS / DOCUMENTATION_AUDITED / CODE_AUDITED / ABSTRACT_ONLY / METADATA_VERIFIED / REPLICATED / UNVERIFIED. Source TYPE and read DEPTH separate; documentation/practitioner/standards = DOCUMENTATION_AUDITED even when fully read. IDs: A1-nn arm 1 (foundations), A2-nn arm 2 (retrieval/RAG), A3-nn arm 3 (reliability/security).
Schema per mandate §3: source_id · identifier · title/authors/year/venue · modality · key claims · negative results · limitations · read_depth · Rune relevance. NO field truncated — cell content passes through verbatim from the arm reports; fields the arm tables did not carry as separate columns are recorded NOT_RECORDED_IN_ARM where applicable rather than invented. Identity keys: arXiv ID / DOI / PMID / normalized title.
**Exact accounting (mechanically derived): SOURCE_ROWS = 108 · SOURCE_UNIQUE = 104 · SOURCE_DUPLICATE_GROUPS = 4.** Duplicate map: arXiv:2004.12832 ColBERT → A1-28 + A2-11 · arXiv:2407.01449 ColPali → A1-29 + A2-1 · arXiv:2109.14084 VideoCLIP → A1-30 + A2-23 · arXiv:1911.01255 pyannote.audio → A2-19 + A2-31 (A2-31 is a deliberate cross-reference/pointer row, not an independent source). InternVideo (2212.03191, A1-31) and InternVideo2 (2403.15377, A2-22) are DIFFERENT papers — not duplicates. CLIP (2103.00020) appears once (A1-21).
**Depths (mechanical, one primary label per row): FULL_PAPER 2 · RELEVANT_SECTIONS 8 · ABSTRACT_ONLY 72 · METADATA_VERIFIED 13 · DOCUMENTATION_AUDITED 10 · UNVERIFIED 3 = 108.** Source TYPE and read DEPTH are separate; documentation/standards/practitioner material is DOCUMENTATION_AUDITED even when fully read.
## Do-not-cite / unverifiable register
- Eich 1995 (not locatable; cite Murre 2021/2022) · Godden & Baddeley 1980 follow-ups (citation-only) · Wade et al. 2002 (not fetched) · no verified cross-version embedding-degradation benchmark (bounded zero-hit) · ColQwen2 (via ColPali v6 Table 7 only) · 2307.01380 (physics), 2309.17436 (real Image Hijacks = 2309.00236), 2405.01532 (real StruQ = 2402.06363), 2406.01184, 1810.07254 (Kosinski = DOI 10.1037/pspa0000098), 1812.06210 · LLM-era image steganography + subtitle/codec injection: zero verified hits · blog-post WER numbers · M3-Agent beyond its n=100 benchmark · 16 mis-paired IDs caught across arms (arXiv export-API title search is the reliable verification path).
- Eich 1995 (not locatable; cite Murre 2021/2022) · Godden & Baddeley 1980 follow-ups (citation-only existence) · Wade et al. 2002 (not fetched) · no verified cross-version embedding-degradation benchmark (bounded zero-hit) · ColQwen2 (cite via ColPali v6 Table 7 only) · 2307.01380 (physics), 2309.17436 (real Image Hijacks = 2309.00236), 2405.01532 (real StruQ = 2402.06363), 2406.01184, 1810.07254 (Kosinski = DOI 10.1037/pspa0000098), 1812.06210 · blog-post WER numbers · M3-Agent beyond its n=100 benchmark · 16 mis-paired arXiv IDs caught across arms (arXiv export-API title search is the reliable verification path).
## Arm 1 — foundations
## Arm 1 — foundations (A1-1 … A1-32)
| # | Source ID | Title / authors / year / venue | Key claims | Negative results | Read depth | Rune relevance |
|---|---|---|---|---|---|---|
| A1-1 | A1 | PMID 1012311; doi:10.1038/264746a0 — Hearing lips and seeing voices — McGurk & MacDonal | audio+visual (speech) | Incongruent lip movements and heard syllables fuse into a third percept ("ga" heard when lips say "ba" and audio says "g | METADATA_VERIFIED | ★★ Analogous *failure mode* to avoid: when two artifact views of the same event (e.g., screenshot + its caption) disagree, Rune mu |
| A1-2 | A2 | MIT Press 1993 (book); verified via its citation as ref. 1 of Stein & Stanford 2008 — The Merging of the Senses — Stein & Meredith 1993, | multisensory (neurophysiology) | Defined the "spatial, temporal, intensity" rules of multisensory enhancement in superior colliculus neurons; cross-modal | METADATA_VERIFIED | ★★ Principle worth keeping in *informational* form: combining two artifact views helps when they are co-located/co-timed and weak |
| A1-3 | A3 | doi:10.1038/nrn2331 — Multisensory integration: current issues from the | multisensory (neurophysiology) | Review of the computational rules: nonlinearity (superadditivity) depends on stimulus efficacy; timing windows; spatial | METADATA_VERIFIED | ★ Justifies checking *alignment quality* before fusing multimodal evidence in Rune; against naive "sum the scores" fusion. |
| A1-4 | A4 | PMID 14761661; doi:10.1016/j.cub.2004.01.029 — The ventriloquist effect results from near-optimal | audio+visual (spatial) | Spatial ventriloquism is not "visual capture by fiat": localization behaves like statistically near-optimal weighting of | METADATA_VERIFIED | ★★ Supports "weight evidence by reliability" as the *shape* of good fusion, while transferring no neural mechanism. |
| A1-5 | A5 | doi:10.1038/415429a; PMID 11807554 — Humans integrate visual and haptic information in | visual+haptic (size estimation) | Canonical cue-combination result. Abstract (read verbatim): vision usually dominates visual-haptic size judgments, but t | ABSTRACT_ONLY | ★★★ The transferable idea is *not* "Rune should run Bayes". It is: (a) fusion quality is determined by per-source reliability meas |
| A1-6 | A6 | PMID 8346328; doi:10.1037/0033-2909.114.1.3 — Source monitoring — Johnson, Hashtroudi & Lindsay | cognitive (cross-modal memory) | Framework: people attribute memories to sources (perceived vs. imagined, internal vs. external) via *heuristic* judgment | METADATA_VERIFIED | ★★★ This is the human analog of Rune's worst failure mode: an *interpretation* (LLM summary) later retrieved as if it were a *sour |
| A1-7 | A7 | doi:10.1037/h0020071 — Encoding specificity and retrieval processes in ep | verbal memory | Encoding specificity: retrieval success depends on cue–trace compatibility at encoding; includes the famous "recognition | METADATA_VERIFIED | ★★ Retrieval quality in Rune is a function of query–trace compatibility *as encoded*, which argues for deriving indexes at ingesti |
| A1-8 | A8 | PMID 9136640; doi:10.1037/0033-2909.121.3.331 — Toward a theory of episodic memory: the frontal lo | cognitive/neuropsych | Episodic memory = a distinct system enabling mental time travel, dependent on autonoetic (self-knowing) consciousness an | METADATA_VERIFIED | ★ Transfer FAILURE watch-item: Rune's "episodic" memory entries are just rows with timestamps; there is no autonoetic re-experienc |
| A1-9 | A9 | doi:10.1146/annurev.psych.53.100901.135114 — Episodic memory: From mind to brain — Tulving 2002 | cognitive/neuro | Refines episodic memory from "materials/tasks" to "self, subjective time, autonoetic consciousness"; reviews neuropsycho | ABSTRACT_ONLY | ★★ The stable, transferable core is the *what/when/what-was-it-like* decomposition of an episode — useful as the shape of Rune's e |
| A1-10 | A10 | PMID 17338600; PMCID PMC2852534; doi:10.1037/0033-2909.133.2.273 — Event perception: a mind-brain perspective — Zacks | visual+cross-modal (event understanding) | Event Segmentation Theory (EST): continuous activity is spontaneously segmented into discrete events; boundaries occur w | RELEVANT_SECTIONS | ★★★ Most transferable human finding in this arm: *what you can retrieve later is determined by where the boundaries were at encodi |
| A1-11 | A11 | PMCID PMC3314399; doi:10.1111/j.1467-8721.2007.00480.x — Event segmentation — Zacks & Swallow 2007, Curr Di | visual (event understanding) | Short-form statement of EST and its memory consequences for a general audience. | METADATA_VERIFIED | ★ Companion citation to A10. |
| A1-12 | A12 | doi:10.1080/14640747308400340 — Learning 10,000 pictures — Standing 1973, Q J Exp | visual | Abstract (verbatim via Crossref): memory capacity follows a power law in material presented; pictorial material shows ov | ABSTRACT_ONLY | ★★ Historical anchor for picture-superiority; also the origin of the "10,000 pictures" number that Brady et al. later corrects for |
| A1-13 | A13 | PMCID PMC2533687; PMID 18787113; doi:10.1073/pnas.0803390105 — Visual long-term memory has a massive storage capa | visual | FULL PAPER READ. 14 observers viewed 2,500 isolated objects (3 s each, 5.5 h total) with an embedded repeat-detection ta | FULL_PAPER | ★★★ Two uses for Rune: (1) evidence FOR storing detailed derived indexes of visual artifacts — detailed traces are cognitively fea |
| A1-14 | A14 | doi:10.1086/209001 — Conditions for a picture-superiority effect on con | verbal+visual | The picture superiority effect (better memory for pictured than verbal items) holds under specific conditions — it is co | METADATA_VERIFIED | ★★ Explicitly cited as the anti-overgeneralization anchor: picture superiority is task/population dependent. |
| A1-15 | A15 | doi:10.1037/h0084295 — Dual coding theory: Retrospect and current status | verbal+visual | Dual coding theory: cognition is supported by two partly independent symbol systems (verbal, imagery); concreteness effe | METADATA_VERIFIED | ★★ Principle to transfer *in weakened form*: paired verbal+visual traces give redundant retrieval routes (robustness), not doubled |
| A1-16 | A16 | doi:10.1111/j.2044-8295.1975.tb01468.x — Context-dependent memory in two natural environmen | verbal (auditory learning) | Abstract (verbatim via Crossref): divers learned word lists on land or underwater and recalled in either; "lists learnt | ABSTRACT_ONLY | ★★ With the replication record attached, this is a cautionary tale: do not design Rune features (e.g., "deployment-context keys") |
| A1-17 | A17 | doi:10.1098/rsos.200724 (+ addendum 10.1098/rsos.211924) — The Godden and Baddeley (1975) experiment on conte | verbal | Direct replication with 16 divers, all four land/underwater learning×recall combinations: "Contrary to the original stud | ABSTRACT_ONLY | ★★ Included deliberately as a finding: a famous multimodal-context memory effect is not robust. Transfer implications of context-d |
| A1-18 | A18 | doi:10.3928/0048-5713-19951201-07 — The formation of false memories — Loftus & Pickrel | verbal/autobiographical ("lost in the mall") | Implanting a wholly false childhood memory (getting lost) via suggestion: a proportion of adults later elaborated the fa | METADATA_VERIFIED | ★★★ Relevance is the *asymmetry*: human memory is reconstructive, so suggestion retroactively edits the trace. Rune's Memory is ap |
| A1-19 | A19 | PMID 16027179; doi:10.1101/lm.94705 — Planting misinformation in the human mind: a 30-ye | verbal/visual | Review of the misinformation-effect literature: post-event information changes later memory reports; robust across 30 ye | METADATA_VERIFIED | ★★ Supports: Rune should treat any LLM-written interpretation as capable of contaminating later reasoning about the source; keep s |
| A1-20 | A20 | (within A13 full text) — Recognition vs recall asymmetry — synthesis row | Standing 1973 abstract: recognition superior, picture superiority holds in recognition; Tulving & Thomson 1973: recognition and recall disso | Dissociation literature is large and partially contradictory; no single mechanism agreed. | RELEVANT_SECTIONS | ★★★ The asymmetry transfers almost directly: Rune's memory search IS recognition (ranked similarity against stored traces), never |
| A1-21 | B1 | arXiv:2103.00020 — Learning Transferable Visual Models From Natural L | image+text | 400M web (image,text) pairs (WIT); contrastive InfoNCE-style symmetric cross-entropy over N×N pairings in-batch; linear | RELEVANT_SECTIONS | ★★★ CLIP-style space is the default candidate for Rune's screenshot/derivation embeddings, but the verified limitation list maps 1 |
| A1-22 | B2 | arXiv:2102.05918 — Scaling Up Visual and Vision-Language Representati | image+text | 1B noisy alt-text pairs, no filtering; simple dual-encoder contrastive; SOTA image-text retrieval (Flickr30k, MSCOCO) in | ABSTRACT_ONLY | ★★ Establishes that *scale of noisy pairs* substitutes for curation — relevant to Rune: you can build bridges from cheap derived c |
| A1-23 | B3 | arXiv:2303.15343 — Sigmoid Loss for Language Image Pre-Training (SigL | image+text | Replaces softmax (global view) with a pairwise sigmoid loss operating only on in-pair terms; allows batch-size decouplin | ABSTRACT_ONLY | ★★ Two implications: (a) modern Rune-relevant CLIP-line models (SigLIP-based, e.g., in recent VLMs) exist and differ in training d |
| A1-24 | B4 | arXiv:2305.05665 — ImageBind: One Embedding Space To Bind Them All — | image, text, audio, depth, thermal, IMU | FULL mechanism verified: six modalities, but training uses ONLY image-paired data (image-text from web; video-audio from | RELEVANT_SECTIONS | ★★★ Central architecture evidence. Even the most aggressive "one space" system (a) anchors everything to ONE hub modality (images) |
| A1-25 | B5 | arXiv:2203.02053 — Mind the Gap: Understanding the Modality Gap in Mu | image+text (+video, medical, protein spaces shown) | **Modality gap is real and general:** image and text embeddings occupy two separate regions of the shared space ("at arm | RELEVANT_SECTIONS | ★★★ Highest-leverage ML finding for Rune: (a) never compare/mix scores across modalities without calibration; (b) "distance 0.9 ac |
| A1-27 | B7 | arXiv:2211.06687 — Large-scale Contrastive Language-Audio Pretraining | audio+text | 633,526 audio-text pairs (LAION-Audio-630K); feature fusion for variable-length audio; keyword-to-caption augmentation; | ABSTRACT_ONLY | ★★ Same lesson as CLIP §6: coarse semantics yes, fine perceptual structure no — for Rune, do not expect fine-grained distinctions |
| A1-28 | B8 | arXiv:2004.12832 — ColBERT: Efficient and Effective Passage Search vi | text (multi-vector) | Late interaction: encode query and document *independently* into per-token vectors; score via cheap fine-grained interac | ABSTRACT_ONLY | ★★★ Grounds the ColPali line (arm 2): late interaction is the middle path between "one vector per artifact" and "re-run the model |
| A1-29 | B9 | arXiv:2407.01449 — ColPali: Efficient Document Retrieval with Vision | page-image+text (multi-vector) | Documents are visually rich; text-extraction pipelines are "lengthy and brittle" and miss visual cues (figures, layout, | ABSTRACT_ONLY | ★★ Directly relevant to Rune's PDF flattening problem: Rune currently destroys layout at ingestion; ColPali evidence says page-ima |
| A1-30 | B10 | arXiv:2109.14084 — VideoCLIP: Contrastive Pre-training for Zero-shot | video+text | Contrastive pretraining over temporally overlapping video-text pairs with hard negatives from nearest neighbors; zero-sh | ABSTRACT_ONLY | ★★ Evidence that video→frame/sentence compression loses temporal and long-description structure. NOTE: the ID 2106.01844 sometimes |
| A1-31 | B11 | arXiv:2212.03191 — InternVideo: General Video Foundation Models via G | video+text | SOTA video foundation model by *combining* masked video modeling (generative) and video-language contrastive (discrimina | ABSTRACT_ONLY | ★★ Confirms: what is lost when video becomes frames or text is the temporal axis, and it is not recoverable from frame-level contr |
| A1-32 | B12 | (evidence cluster: B1 Tables 19–20 + B5 §2.3/§4.4) — Model-version comparability of embeddings — synthe | all | Direct verified evidence that embeddings from different CLIP versions are NOT comparable: (1) CLIP's own hyperparameter | RELEVANT_SECTIONS | ★★★ URGENT for Rune: current sqlite-vec embeddings record NO model/version. Since different initializations and dimensions make ve |
| A1-1 | PMID 1012311; doi:10.1038/264746a0 | Hearing lips and seeing voices — McGurk & MacDonald 1976, Nature 264:746–748 | audio+visual (speech) | Incongruent lip movements and heard syllables fuse into a third percept ("ga" heard when lips say "ba" and audio says "ga"); audiovisual speech is integrated pre-attentively, not just summed. | Robust phenomenon (2,100+ citations; 40-year retrospective by MacDonald 2018, Multisens Res 31, confirmed as a still-active research field), but the *rate* of the effect varies strongly by language, attention, and stimulus AV timing. | Perception-level, pre-attentive; no propositional representation; effect is stimulus-tuned. | METADATA_VERIFIED | ★★ Analogous *failure mode* to avoid: when two artifact views of the same event (e.g., screenshot + its caption) disagree, Rune must not silently "fuse" them into a plausible third interpretation — that is a reconstruction bug, not perception. |
| A1-2 | MIT Press 1993 (book); verified via its citation as ref. 1 of Stein & Stanford 2008 | The Merging of the Senses — Stein & Meredith 1993, MIT Press | multisensory (neurophysiology) | Defined the "spatial, temporal, intensity" rules of multisensory enhancement in superior colliculus neurons; cross-modal inputs combine superadditively when spatially and temporally aligned, and can *suppress* when misaligned. | The book's superadditivity emphasis was later qualified: superadditivity occurs mainly when component responses are weak (Stanford & Stein 2007, cited in Stein & Stanford 2008 refs). | Neuron-level, animal (cat/monkey) physiology. | METADATA_VERIFIED | ★★ Principle worth keeping in *informational* form: combining two artifact views helps when they are co-located/co-timed and weak individually; misaligned combination can actively hurt. |
| A1-3 | doi:10.1038/nrn2331 | Multisensory integration: current issues from the perspective of the single neuron — Stein & Stanford 2008, Nat Rev Neurosci 9:255–266 | multisensory (neurophysiology) | Review of the computational rules: nonlinearity (superadditivity) depends on stimulus efficacy; timing windows; spatial congruence; cortex-midbrain dialogue; experience-dependence. | Notes ongoing debate about using superadditivity as the fMRI integration criterion (refs 63–64: Laurienti 2005, Beauchamp 2005 argue against it) — i.e., even the *definition* of "integration happened" is contested. | Animal physiology + human imaging, not applicable mechanistically to software. | METADATA_VERIFIED | ★ Justifies checking *alignment quality* before fusing multimodal evidence in Rune; against naive "sum the scores" fusion. |
| A1-4 | PMID 14761661; doi:10.1016/j.cub.2004.01.029 | The ventriloquist effect results from near-optimal bimodal integration — Alais & Burr 2004, Curr Biol 14:257–262 | audio+visual (spatial) | Spatial ventriloquism is not "visual capture by fiat": localization behaves like statistically near-optimal weighting of the two cues by their reliability, generalizing Ernst & Banks to spatial tasks. | Confirms cue-combination framework across tasks, but near-optimality is approximate; deviations documented in later literature. | Psychophysics; stimulus-level estimates. | METADATA_VERIFIED | ★★ Supports "weight evidence by reliability" as the *shape* of good fusion, while transferring no neural mechanism. |
| A1-5 | doi:10.1038/415429a; PMID 11807554 | Humans integrate visual and haptic information in a statistically optimal fashion — Ernst & Banks 2002, Nature 415:429–433 | visual+haptic (size estimation) | Canonical cue-combination result. Abstract (read verbatim): vision usually dominates visual-haptic size judgments, but the percept is affected by haptics "in some circumstances"; the authors propose a general principle that *minimizes variance in the final estimate*, realized as maximum-likelihood estimation (MLE): they first measured the variances of unisensory visual and haptic height estimation, built an MLE integrator from those measured variances, and found the model "behaved very similarly to humans in a visual–haptic task"; visual dominance is explained as the case where visual variance is lower. | None in the paper itself (model fit was the positive result). The generality across tasks/cue types is the debated part (Alais & Burr extend it; other work documents sub-optimality). | Paywalled full text (unavailable to this arm — see honesty note); abstract-level: it is a size-estimation task with 3D-rendered stimuli and explicitly *measured* noise; the optimality claim is about first-order statistics. | ABSTRACT_ONLY | ★★★ The transferable idea is *not* "Rune should run Bayes". It is: (a) fusion quality is determined by per-source reliability measured in advance; (b) "dominance" emerges from unequal reliability, not from a hardcoded modality hierarchy. In Rune: store per-derivation confidence/provenance so a later fusion step *could* weight sources; do not build a fixed "vision wins" rule. |
| A1-6 | PMID 8346328; doi:10.1037/0033-2909.114.1.3 | Source monitoring — Johnson, Hashtroudi & Lindsay 1993, Psychol Bull 114(1):3–28 | cognitive (cross-modal memory) | Framework: people attribute memories to sources (perceived vs. imagined, internal vs. external) via *heuristic* judgment over activated features, not via an explicit tag; source errors arise when features are ambiguous or compatible with multiple sources. | The framework was built largely on *errors* (misattribution, "becoming famous overnight" paradigms); it is a framework, not a process model with quantitative fit. | Human amnesia/dementia and children studied; framework-level. | METADATA_VERIFIED | ★★★ This is the human analog of Rune's worst failure mode: an *interpretation* (LLM summary) later retrieved as if it were a *source* (file content). The human system lacks source tags; Rune must not copy that defect — source provenance should be explicit, persistent metadata, not reconstructed at retrieval. |
| A1-7 | doi:10.1037/h0020071 | Encoding specificity and retrieval processes in episodic memory — Tulving & Thomson 1973, Psychol Rev 80(5):352–373 | verbal memory | Encoding specificity: retrieval success depends on cue–trace compatibility at encoding; includes the famous "recognition failure of recallable words" — a word can be recalled with a weak cue yet not recognized as a studied item. | Recognition failure itself was a *surprising negative* result against independent-store theories; replication debates (Thomson & Tulving 1970 vs Bahrick) are documented and were revisited by Higham 2002 (Mem Cogn 30:67–80, verified in Crossref refs) who concluded "strong cues are not necessarily weak" — boundary conditions exist. | Verbal memory; task-specific cue compatibility. | METADATA_VERIFIED | ★★ Retrieval quality in Rune is a function of query–trace compatibility *as encoded*, which argues for deriving indexes at ingestion time (matching the query distribution) rather than assuming any embedding is a universal key. |
| A1-8 | PMID 9136640; doi:10.1037/0033-2909.121.3.331 | Toward a theory of episodic memory: the frontal lobes and autonoetic consciousness — Wheeler, Stuss & Tulving 1997, Psychol Bull 121(3):331–354 | cognitive/neuropsych | Episodic memory = a distinct system enabling mental time travel, dependent on autonoetic (self-knowing) consciousness and frontal-lobe function; supported by amnesia patterns and imaging. | Concept criticized as circular (consciousness defines the system that defines consciousness) — a criticism Tulving himself reviews later (Tulving 2002). | Neurological patient literature + PET; theory-level. | METADATA_VERIFIED | ★ Transfer FAILURE watch-item: Rune's "episodic" memory entries are just rows with timestamps; there is no autonoetic re-experience and should be no pretense of it. |
| A1-9 | doi:10.1146/annurev.psych.53.100901.135114 | Episodic memory: From mind to brain — Tulving 2002, Annu Rev Psychol 53:1–25 | cognitive/neuro | Refines episodic memory from "materials/tasks" to "self, subjective time, autonoetic consciousness"; reviews neuropsychological (patient) and imaging support; addresses criticisms. | The author explicitly catalogs criticisms of the episodic/semantic division; the boundary between systems is acknowledged as not sharp. | Review; definitional. | ABSTRACT_ONLY | ★★ The stable, transferable core is the *what/when/what-was-it-like* decomposition of an episode — useful as the shape of Rune's event records, not of any cognitive claim. |
| A1-10 | PMID 17338600; PMCID PMC2852534; doi:10.1037/0033-2909.133.2.273 | Event perception: a mind-brain perspective — Zacks, Speer, Swallow, Braver & Reynolds 2007, Psychol Bull 133(2):273–293 | visual+cross-modal (event understanding) | Event Segmentation Theory (EST): continuous activity is spontaneously segmented into discrete events; boundaries occur where *perceptual prediction error* transiently spikes; "event models" (what-is-happening-now) are gated, reset at boundaries; segmentation is simultaneous on multiple timescales; boundary information is preferentially encoded into long-term memory. Verified load-bearing evidence from the full text: (i) unitization (button-press marking of boundaries) is reliable within and across observers (Newtson 1976; Speer, Swallow & Zacks 2003); (ii) still frames from event boundaries are better recognized than frames from within events (Newtson & Engquist 1976); (iii) film cuts/commercials placed AT event boundaries improve recall; placed at non-boundaries they impair it (Boltz 1992; Schwan & Garsoffky 2004; Schwan et al. 2000); (iv) group-typical segmentation ability predicts later memory even controlling for dementia (Zacks, Speer, Vettel & Jacoby); (v) partial computational implementation (Reynolds, Zacks & Braver) shows boundary spikes in prediction error emerge from a prediction network without labels. | Debates noted in-paper: whether fine-grained unitization improves *recognition* (Hanson & Hirst 1991 vs Lassiter & Slaw 1991 — mixed); whether parallel segmentation of multiple activity streams occurs (explicitly an open empirical question). | Movie-watching, human, seconds-to-minutes timescales; theory's neural implementation proposals (lateral PFC, ACC) are hypotheses. | RELEVANT_SECTIONS | ★★★ Most transferable human finding in this arm: *what you can retrieve later is determined by where the boundaries were at encoding*. For Rune, event/commit/message boundaries are the natural retrieval anchors for attached screenshots, recordings, PDFs — chunk multimodal evidence by event boundaries, not by fixed token windows. Explicitly do NOT port the prediction-error *mechanism* (see rejected transfers). |
| A1-11 | PMCID PMC3314399; doi:10.1111/j.1467-8721.2007.00480.x | Event segmentation — Zacks & Swallow 2007, Curr Dir Psychol Sci 16(2):80–84 | visual (event understanding) | Short-form statement of EST and its memory consequences for a general audience. | — | Condensed version of A10. | METADATA_VERIFIED | ★ Companion citation to A10. |
| A1-12 | doi:10.1080/14640747308400340 | Learning 10,000 pictures — Standing 1973, Q J Exp Psychol 25(2):207–222 | visual | Abstract (verbatim via Crossref): memory capacity follows a power law in material presented; pictorial material shows overall superiority; "the capacity of recognition memory for pictures is almost limitless, when measured under appropriate conditions"; picture memory exceeds verbal memory even in verbal recall; verbal memory has higher retrieval speed. | The paper itself notes the limits are measurement conditions; the "almost limitless" phrasing is recognition-under-2AFC, not recall. | Forced-choice recognition; 10k-item scale; 1970s stimuli. | ABSTRACT_ONLY | ★★ Historical anchor for picture-superiority; also the origin of the "10,000 pictures" number that Brady et al. later corrects for*fidelity*. |
| A1-13 | PMCID PMC2533687; PMID 18787113; doi:10.1073/pnas.0803390105 | Visual long-term memory has a massive storage capacity for object details — Brady, Konkle, Alvarez & Oliva 2008, PNAS 105(38):14325–14329 | visual | FULL PAPER READ. 14 observers viewed 2,500 isolated objects (3 s each, 5.5 h total) with an embedded repeat-detection task, then 2-alternative forced-choice tests. Accuracy: **92.5% (novel-category foil), 87.6% (same basic-level category, different exemplar), 87.2% (same object in different state/pose)**. Repeat-detection during study: 96% hits, 1.3% false alarms; ~80% correct even with 1,023 intervening items (~2 h); error vs lag fits a power law of forgetting (r²=0.98). Authors' Landauer-style estimate: 13.8 bits/item at 93% novel performance, +2.0 bits each for exemplar and state discrimination → ~17.8 bits/item, raising a lower bound from ~14,000 to ~228,000 discriminable codes. Explicit stated conclusions/caveats (from Discussion): data "cannot reveal the format of these representations" and "should not be taken to suggest observers have a photographic-like memory"; categorical distinctiveness of the stimulus set plausibly reduced interference and helped performance; it is unresolved whether the 2AFC choices reflect familiarity or recollection (but either way the stored trace must contain foil-discriminating detail); "under conditions where we attempt to encode such details, we are capable of succeeding" — i.e., encoding was optimized (instructions, distinct categories, no interference control issues). | The paper's framing is itself a *correction* of prior belief: previous large-scale studies (Shepard 1967; Standing 1973) could not distinguish gist-only storage, and change-blindness / Loftus-misinformation lines had been read as evidence that visual memory is sparse. The finding pushes back on both extremes: memory is not gist-only, and not photographic. | 2AFC recognition only (not recall); optimal-encoding regime; unknown representational format; unknown whether performance holds for scene-level or interference-rich sets; familiarity/recollection composition unresolved. | FULL_PAPER | ★★★ Two uses for Rune: (1) evidence FOR storing detailed derived indexes of visual artifacts — detailed traces are cognitively feasible and dramatically improve *recognition*-style retrieval (which is what vector search is); (2) evidence AGAINST over-reading: Rune's analog is not "store raw pixels of everything" but "maintain exemplar-level (not gist-level) indexes" — and Rune's real bottleneck is the attention/context budget, not storage, the inverse of the human situation. |
| A1-14 | doi:10.1086/209001 | Conditions for a picture-superiority effect on consumer memory — Childers & Houston 1984, J Consumer Research 11(2):643 | verbal+visual | The picture superiority effect (better memory for pictured than verbal items) holds under specific conditions — it is conditional, not universal (conditions include encoding time and processing type). | The very title documents boundary conditions; dual-coding/picture-superiority literature contains many demonstrations of null or reversed effects depending on task (recall vs recognition), population, and stimulus meaningfulness. | Marketing/consumer context; older literature. | METADATA_VERIFIED | ★★ Explicitly cited as the anti-overgeneralization anchor: picture superiority is task/population dependent. |
| A1-15 | doi:10.1037/h0084295 | Dual coding theory: Retrospect and current status — Paivio 1991, Can J Psychol 45(3):255–287 | verbal+visual | Dual coding theory: cognition is supported by two partly independent symbol systems (verbal, imagery); concreteness effects; additive benefits of dual codes. | Known criticisms to carry forward: boundary conditions (abstract material, expert populations), failure to fully specify imagery code format, effects shrink with meaningfulness controlled; picture superiority in humans does not license "add a second representation always helps". | Theory review; no computational implementation. | METADATA_VERIFIED | ★★ Principle to transfer *in weakened form*: paired verbal+visual traces give redundant retrieval routes (robustness), not doubled memory. |
| A1-16 | doi:10.1111/j.2044-8295.1975.tb01468.x | Context-dependent memory in two natural environments: on land and underwater — Godden & Baddeley 1975, Br J Psychology 66(3):325–331 | verbal (auditory learning) | Abstract (verbatim via Crossref): divers learned word lists on land or underwater and recalled in either; "lists learnt underwater were best recalled underwater, and vice versa"; a second experiment showed the disruption of moving between environments was not the cause. | **Failed replication (negative result, verified):** Murre 2021, R Soc Open Sci 8:200724, replicated the four-condition design with 16 divers and found NO same-context advantage; a 2022 addendum (10.1098/rsos.211924) with corrected repeated-measures ANOVA "reinforces the conclusion that this study was not a successful replication". Eich's related mood-dependence work also reports weak/fragile effects (see §4 note). | Small-N field study; auditory recall; the classic status outstrips the replication record. | ABSTRACT_ONLY | ★★ With the replication record attached, this is a cautionary tale: do not design Rune features (e.g., "deployment-context keys") on the strength of this classic alone. Rune's context fields are explicit variables, not incidental reinstatement cues. |
| A1-17 | doi:10.1098/rsos.200724 (+ addendum 10.1098/rsos.211924) | The Godden and Baddeley (1975) experiment on context-dependent memory on land and underwater: a replication — Murre 2021, R Soc Open Sci 8:200724 | verbal | Direct replication with 16 divers, all four land/underwater learning×recall combinations: "Contrary to the original study, we did not find that recall in the same context where the words had been learned was better than recall in the other context." | Negative result, published; addendum re-analysis strengthens it. | One replication; power considerations discussed in-paper. | ABSTRACT_ONLY | ★★ Included deliberately as a finding: a famous multimodal-context memory effect is not robust. Transfer implications of context-dependence should be treated as unproven. |
| A1-18 | doi:10.3928/0048-5713-19951201-07 | The formation of false memories — Loftus & Pickrell 1995, Psychiatric Annals 25(12):720–725 | verbal/autobiographical ("lost in the mall") | Implanting a wholly false childhood memory (getting lost) via suggestion: a proportion of adults later elaborated the false event as real. | The paradigm is heavily debated: base rates of "false" acceptance, distinguishability from genuine-uncertain memories, demand characteristics; the strongest legal-policy claims were later moderated (see Loftus 2005 below). Original paper has only 11 references and modest sample. | Case-study scale; suggestion paradigm. | METADATA_VERIFIED | ★★★ Relevance is the *asymmetry*: human memory is reconstructive, so suggestion retroactively edits the trace. Rune's Memory is append-only with immutable source rows; the architectural answer is to never overwrite sources — corrections are new derived rows that supersede, with provenance. |
| A1-19 | PMID 16027179; doi:10.1101/lm.94705 | Planting misinformation in the human mind: a 30-year investigation of the malleability of memory — Loftus 2005, Learn Mem 12(4):361–366 | verbal/visual | Review of the misinformation-effect literature: post-event information changes later memory reports; robust across 30 years of studies. | The literature contains the McCloskey & Zaragoza (1985) counterargument that misinformation effects can reflect response bias rather than trace impairment (cited in Loftus & Pickrell's own refs) — the *mechanism* debate is unresolved even where the phenomenon is accepted. | Review, narrative. | METADATA_VERIFIED | ★★ Supports: Rune should treat any LLM-written interpretation as capable of contaminating later reasoning about the source; keep source and interpretation as separate, differently-trusted rows. |
| A1-20 | (within A13 full text) | Recognition vs recall asymmetry — synthesis row | — | Standing 1973 abstract: recognition superior, picture superiority holds in recognition; Tulving & Thomson 1973: recognition and recall dissociate (recognition failure of recallable words); Brady et al. 2008 Discussion: familiarity vs recollection unresolved but either way 2AFC taps stored detail. | Dissociation literature is large and partially contradictory; no single mechanism agreed. | NOT_RECORDED_IN_ARM as a separate field (synthesis row: limitations are carried in the negatives cell — the arm table marked this '—') | RELEVANT_SECTIONS (via A12/A13/A7 readings) | ★★★ The asymmetry transfers almost directly: Rune's memory search IS recognition (ranked similarity against stored traces), never recall (generation from a trace). Design the memory for recognition: distinctive, foil-discriminating embeddings + FTS5 terms; do not rely on the agent "re-deriving" content from gist. |
| A1-21 | arXiv:2103.00020 | Learning Transferable Visual Models From Natural Language Supervision (CLIP) — Radford, Kim, Hallacy, Ramesh, Goh, Agarwal, Sastry, Askell, Mishkin, Clark, Krueger, Sutskever 2021, arXiv (OpenAI) | image+text | 400M web (image,text) pairs (WIT); contrastive InfoNCE-style symmetric cross-entropy over N×N pairings in-batch; linear (not MLP) projection to joint embedding space; temperature learned, log-parameterized, clipped at 100; batch 32,768; zero-shot transfer: matches ResNet-50 on ImageNet (76.2% zero-shot, 95% top-5) with no ImageNet labels; robustness to natural distribution shift exceeds same-accuracy supervised models (ImageNet-R 88.9, ObjectNet 72.3, IN-Sketch 60.2, IN-Vid 95.3 PM0 — zero-shot beats supervised SOTA on 5 of 7 shift datasets). | **Section 6 Limitations (read verbatim):** zero-shot CLIP only on average matches a linear classifier on ResNet-50 features; weak on fine-grained classification (car models, flower species, aircraft variants); "struggles with more abstract and systematic tasks such as counting"; near-random on novel tasks like distance-to-nearest-car; poor OOD generalization (MNIST 88% — beaten by logistic regression on raw pixels; SVHN 51%); brittle to low-res/blurry digits and repeated characters; also: no mechanism for few-shot (zero→few-shot transition can *drop* performance); validation-set reuse during development ("not true zero-shot"); dataset suite "haphazardly assembled... co-adapted with development of CLIP". Appendix E: OCR strong only for digitally rendered text; image-retrieval direction weaker than text-retrieval. | Similarity = "which in-batch caption goes with this image" ranking; cosine scores are uncalibrated, class-prompt-dependent (prompt engineering worth ~5 points avg over 36 datasets). | RELEVANT_SECTIONS | ★★★ CLIP-style space is the default candidate for Rune's screenshot/derivation embeddings, but the verified limitation list maps 1:1 onto coding-agent assets: screenshots ARE rendered text (good), but fine-grained distinctions (v1 vs v2 of a UI, line-level diffs), counts, and spatial relations (§6) are exactly what code artifacts need. Do not let one CLIP vector be the *only* index for such distinctions. |
| A1-22 | arXiv:2102.05918 | Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision (ALIGN) — Jia, Yang, Xia, Chen, Parekh, Pham, Le, Sung, Li, Duerig 2021, ICML 2021 | image+text | 1B noisy alt-text pairs, no filtering; simple dual-encoder contrastive; SOTA image-text retrieval (Flickr30k, MSCOCO) incl. beating some cross-attention models; zero-shot classification; noisy multilingual and image+text query retrieval. | Noise-tolerant but retrieval on MS-COCO overall-SOTA not reached in zero-shot; alt-text distribution is skewed (boilerplate, SEO text) — similarity inherits alt-text semantics. | Same contrastive ranking semantics as CLIP; noisy-text training makes text side semantics broader but sloppier. | ABSTRACT_ONLY | ★★ Establishes that *scale of noisy pairs* substitutes for curation — relevant to Rune: you can build bridges from cheap derived captions of screenshots; noise is absorbed by scale, but not by 944 entries. |
| A1-23 | arXiv:2303.15343 | Sigmoid Loss for Language Image Pre-Training (SigLIP) — Zhai, Mustafa, Kolesnikov, Beyer 2023, ICCV 2023 Oral | image+text | Replaces softmax (global view) with a pairwise sigmoid loss operating only on in-pair terms; allows batch-size decoupling; SigLiT 84.5% ImageNet zero-shot with 4 TPUv4 in 2 days; benefits of batch size saturate (~32k sufficient); enables studying examples-vs-pairs and negative:positive ratio. | Empirical engineering result; notes (from abstract) that growing batch benefits diminish quickly. No fundamental semantic change vs CLIP. | Same joint-space semantics; loss shape slightly alters score calibration; still ranking-oriented. | ABSTRACT_ONLY | ★★ Two implications: (a) modern Rune-relevant CLIP-line models (SigLIP-based, e.g., in recent VLMs) exist and differ in training dynamics from classic CLIP; (b) batch-normalization dependence means score scales are model-specific — another reason cross-model score mixing is unsafe. |
| A1-24 | arXiv:2305.05665 | ImageBind: One Embedding Space To Bind Them All — Girdhar, El-Nouby, Liu, Singh, Alwala, Joulin, Misra 2023, CVPR 2023 (Highlighted) | image, text, audio, depth, thermal, IMU | FULL mechanism verified: six modalities, but training uses ONLY image-paired data (image-text from web; video-audio from Audioset; image-depth from SUN RGB-D; image-thermal from LLVIP; video-IMU from Ego4D); image+text encoders frozen from OpenCLIP ViT-H; each other modality gets a small ViT/Transformer encoder aligned to *image* embeddings via symmetric InfoNCE. **Emergent claims verified quantitatively:** text↔audio zero-shot ESC top-1 66.9 (vs supervised AudioCLIP 68.6, i.e., comparable but NOT above absolute SOTA 97.0); emergent audio→video retrieval R@1 only **6.8** vs 36.8 when audio+video combined; emergent zero-shot far below absolute SOTA on every benchmark (e.g., NYU-D 54.0 vs 76.7; K400 50.0 vs 89.9). Embedding arithmetic (image+audio → retrieval) works qualitatively. Emergent alignment improves as the image encoder scales (ViT-B→H: +4–7 points). | Limitations section (read verbatim): embeddings "lag the performance of specialist models" because trained without a specific downstream task; "research prototype and cannot be readily used for real world applications"; thermal data limited to outdoor street scenes, depth to indoor scenes (dataset coverage gaps propagate to the space); regression-to-CLIP objective underperforms contrastive. The "binds all modalities" claim is real but is *hub-mediated through images*, not pairwise-full-rank training. | Cross-modal similarity exists for unpaired modality combos but is weakest precisely for pairs with no co-occurrence in any single instance; distances not calibrated; audio↔text is emergent-approximate. | RELEVANT_SECTIONS | ★★★ Central architecture evidence. Even the most aggressive "one space" system (a) anchors everything to ONE hub modality (images) via frozen CLIP, (b) produces emergent cross-modal ties that are strictly weaker than direct pairing, and (c) relies on the hub being web-scale-trained. For Rune: a hub-anchored joint space is viable (hub = text/code), but emergent untrained-pair retrieval must be treated as approximate, never as ground truth. |
| A1-25 | arXiv:2203.02053 | Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learning — Liang, Zhang, Kwon, Yeung, Zou 2022, NeurIPS 2022 | image+text (+video, medical, protein spaces shown) | **Modality gap is real and general:** image and text embeddings occupy two separate regions of the shared space ("at arm's length"), across CLIP and other contrastive multimodal models. Verified mechanism: (1) **cone effect** — any deep-net embedding set occupies a narrow cone (mean pairwise cosine 0.56/0.47/0.51 for ResNet/ViT/text-T; <2^-512 of the 512-d hypersphere surface); this holds even with random weights (cosine 0.99 for random ResNet) and random inputs — so the gap is NOT caused by data distribution; (2) different random initializations create *different* cones (25 inits → 25 distinct cones); (3) contrastive optimization *preserves* the gap: CLIP's loss landscape at its learned temperature (1/100) has a global minimum exactly at the observed gap distance 0.82, with a repulsive structure; higher temperature closes the gap; mismatched-pair data is a forming factor. **Implications (read):** deliberately *increasing* the gap (0.82→0.97) reduced denigration bias for all races, and modified gap improves *some* zero-shot tasks — direction is task-dependent (CIFAR10 ↑, CIFAR100 ↓, EuroSAT ↓). | Authors explicitly state it is "not our goal to propose a method to close the gap, since it's not clear that it's desirable to have no modality gap." Gap-closing is not a quality metric. | **Consequence for distance meaning:** a cross-modal cosine (image-vs-text) and a within-modality cosine (text-vs-text) do NOT live on the same scale; zero cosine does not mean "unrelated"; cross-modal distance has a nonzero offset by construction. | RELEVANT_SECTIONS | ★★★ Highest-leverage ML finding for Rune: (a) never compare/mix scores across modalities without calibration; (b) "distance 0.9 across modality" ≠ "distance 0.9 within modality"; (c) the modality gap is a property of *every* dual-encoder — including future Rune text-vs-screenshot spaces — and is temperature- and initialization-dependent; (d) it is partly frozen at initialization: two encoders trained on the SAME data still separate. |
| A1-26 | arXiv:2206.04769 | CLAP: Learning Audio Concepts From Natural Language Supervision — Elizalde, Deshmukh, Al Ismail, Wang 2022, arXiv (Microsoft) | audio+text | Dual-encoder contrastive language-audio; 128k pairs; SOTA zero-shot across 16 tasks/8 domains (sound events, music, speech); removes fixed-label constraint. | Trained with *far* fewer pairs than CLIP (128k vs 400M) — abstract-level: weaker generalization surface; audio concepts are vaguer than image captions. | Ranking semantics; scale-limited calibration. | ABSTRACT_ONLY | ★ Template for audio derivation (voice notes / meeting audio → text-keyed retrieval) if Rune ever ingests audio. |
| A1-27 | arXiv:2211.06687 | Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation (LAION-CLAP) — Wu, Chen, Zhang, Hui, Nezhurina, Berg-Kirkpatrick, Dubnov 2022 (v4 2024), arXiv | audio+text | 633,526 audio-text pairs (LAION-Audio-630K); feature fusion for variable-length audio; keyword-to-caption augmentation; superior text-to-audio retrieval; SOTA zero-shot audio classification. | **Independent negative evidence on perceptual fidelity (verified):** Deng, Pardo & Pappas 2025 (arXiv:2510.14249, abstract read) evaluate MS-CLAP/LAION-CLAP/MuQ-MuLan: "the overall strength of this alignment remains limited, suggesting that current joint language-audio embeddings capture perceptual timbre semantics only partially." | Ranking semantics; timbre-level detail demonstrably under-encoded. | ABSTRACT_ONLY | ★★ Same lesson as CLIP §6: coarse semantics yes, fine perceptual structure no — for Rune, do not expect fine-grained distinctions (e.g., two similar error beeps, two similar UI chimes) from a joint space. |
| A1-28 | arXiv:2004.12832 | ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT — Khattab & Zaharia 2020, SIGIR 2020 | text (multi-vector) | Late interaction: encode query and document *independently* into per-token vectors; score via cheap fine-grained interaction (MaxSim) at query time; offline-precomputable document vectors → ANN-indexable end-to-end retrieval; competitive with cross-encoder BERT ranking at 2 orders of magnitude less latency, 4 orders fewer FLOPs. | Storage cost of multi-vector (the known trade-off, acknowledged in paper's efficiency analysis); interaction is hand-designed (MaxSim), not learned. | Similarity = max over token-pair dot products: NOT a single calibrated score; supports phrase-level and exact-match structure that single-vector models blur. | ABSTRACT_ONLY | ★★★ Grounds the ColPali line (arm 2): late interaction is the middle path between "one vector per artifact" and "re-run the model per query" — for Rune, per-patch/per-region multi-vector screenshots preserve layout detail that a single CLIP vector provably loses (B1 §6). |
| A1-29 | arXiv:2407.01449 | ColPali: Efficient Document Retrieval with Vision Language Models — Faysse, Sibille, Wu, Omrani, Viaud, Hudelot, Colombo 2024, ICLR 2025 | page-image+text (multi-vector) | Documents are visually rich; text-extraction pipelines are "lengthy and brittle" and miss visual cues (figures, layout, fonts); embed page *images* directly with a VLM producing multi-vector embeddings + late interaction; outperforms modern document-retrieval pipelines on ViDoRe; simpler and end-to-end trainable. | Abstract-level only (arm 2 owns deep read); multi-vector storage and VLM cost are the acknowledged trade-offs. | Late-interaction scores; image-side vectors; per-page. | ABSTRACT_ONLY | ★★ Directly relevant to Rune's PDF flattening problem: Rune currently destroys layout at ingestion; ColPali evidence says page-image indexing beats text-extracted indexing for visually rich docs — arm 2 decides. |
| A1-30 | arXiv:2109.14084 | VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding — Xu, Ghosh, Huang, Okhonko, Aghajanyan, Metze, Zettlemoyer, Feichtenhofer 2021, EMNLP 2021 | video+text | Contrastive pretraining over temporally overlapping video-text pairs with hard negatives from nearest neighbors; zero-shot SOTA on retrieval, VideoQA, action localization/segmentation. | Zero-shot, label-free — but built on temporally overlapping positives only (short windows); long-video/long-description gap documented by successors (VideoCLIP-XL 2024: pretraining "on brief summary texts... prevents CLIP from understanding long descriptions", verified abstract). | Ranking semantics; temporal alignment coarse. | ABSTRACT_ONLY | ★★ Evidence that video→frame/sentence compression loses temporal and long-description structure. NOTE: the ID 2106.01844 sometimes circulated for VideoCLIP is WRONG (it is a magnetism paper) — caught in verification. |
| A1-31 | arXiv:2212.03191 | InternVideo: General Video Foundation Models via Generative and Discriminative Learning — Wang, Li, Li, He, Huang, et al. 2022, arXiv technical report | video+text | SOTA video foundation model by *combining* masked video modeling (generative) and video-language contrastive (discriminative) with learnable coordination; 91.1% K400, 77.2% SSv2; SOTA on 39 video datasets. | The architecture itself is the admission: video-language contrastive ALONE is insufficient for temporal understanding — temporal supervision must come from masked video modeling. | Dual-objective; video tokens. | ABSTRACT_ONLY | ★★ Confirms: what is lost when video becomes frames or text is the temporal axis, and it is not recoverable from frame-level contrastive embeddings alone. |
| A1-32 | (evidence cluster: B1 Tables 19–20 + B5 §2.3/§4.4) | Model-version comparability of embeddings — synthesis row | all | Direct verified evidence that embeddings from different CLIP versions are NOT comparable: (1) CLIP's own hyperparameter tables (read in full): embedding dimension differs by version — RN50: 1024-d, RN101: 512-d, ViT-B/32: 512-d, ViT-L/14: 768-d — vectors are not even dimension-compatible across versions; (2) Liang et al.: the cone orientation is determined by *random initialization*, so two independently initialized encoders (even same architecture, same data) produce systematically different geometries; CLIP-vs-OpenCLIP initialization in ImageBind (B4) required frozen-image-anchor training precisely because the spaces differ. | No dedicated public benchmark of cross-version embedding interoperability was found in this arm's searches; the dimension + initialization evidence is strong but the *practical degradation curve* (how bad is score-mixing?) is not quantified by any verified source here. | Mixed-version similarity is undefined/unanchored. | RELEVANT_SECTIONS | ★★★ URGENT for Rune: current sqlite-vec embeddings record NO model/version. Since different initializations and dimensions make vectors incomparable, Rune cannot safely re-embed partially, upgrade models, or mix old and new vectors. The atlas design must stamp (model_id, model_version, space_id) per embedding row and rebuild the index wholesale on model change. |
## Arm 2 — retrieval/RAG
## Arm 2 — retrieval/RAG (A2-1 … A2-31)
| # | Source ID | Title / authors / year / venue | Key claims | Negative results | Read depth | Rune relevance |
|---|---|---|---|---|---|---|
| A2-1 | arXiv:2407.01449 | ColPali: Efficient Document Retrieval with Vision Language Models — Faysse, Sibille, Wu, Omrani, Viaud, Hudelot, Colom | Page-image → multi-vector retrieval (late interaction over PaliGemma-3B token embeddings, D=128) | (1) ViDoRe benchmark (10 tasks, page-level); (2) ColPali avg **81.3 nDCG@5** vs best OCR-pipeline baseline (Unstructured | RELEVANT_SECTIONS | **Anchor for the whole arm.** Directly quantifies what Rune's flatten-to-text PDF ingestion loses (figures/tables/layout: +10 to + |
| A2-2 | arXiv:2406.11251 | Unifying Multimodal Retrieval via Document Screenshot Embedding (DSE) — Ma, Lin, Li, Chen, Lin — 2024, EMNLP 2024 main | Single-vector screenshot embedding (EVA-CLIP + LLM layer, 1.3M Wiki-SS corpus) | Screenshot-as-unified-input removes parsing entirely; Wiki-SS (text-intensive): **+17 pts top-1 over BM25**; mixed-modal | ABSTRACT_ONLY | Second independent confirmation of the OCR-first loss number on slides (+15) and the "text-only pages ≈ parity" boundary |
| A2-3 | arXiv:2410.10594 | VisRAG: Vision-based Retrieval-augmented Generation on Multi-modality Documents — Yu, Tang, Xu, Cui, Ran, Yan, Liu, Wang, Han, Liu, | VLM-based RAG: embed page images → retrieve → feed image to VLM generator | **20–40% end-to-end gain over text-based RAG** on multi-modality docs; eliminates parsing information loss; strong gener | ABSTRACT_ONLY | Quantifies the retrieval-loss → answer-quality pipeline effect Rune's flattening causes |
| A2-4 | arXiv:2411.04952 | M3DocRAG: Multi-modal Retrieval is What You Need for Multi-page Multi-document Understanding — Cho, Mahata, Irsoy, He, Bansal — 2024 (UNC/IBM) | Open-domain multi-page DocVQA: ColPali retrieval + Qwen2-VL 7B generation | Works open-domain over **3,000+ PDFs / 40,000+ pages** (M3DocVQA benchmark); SOTA on MP-DocVQA; handles multi-hop and im | ABSTRACT_ONLY | Evidence that ColPali-style page retrieval **scales beyond toy corpora** to ~40k pages — far past Rune's needs, so scale is no obj |
| A2-5 | arXiv:2504.08748 | A Survey of Multimodal Retrieval-Augmented Generation — Mei, Mo, Yang, Chen — 2025 | Survey: MRAG components, datasets, evaluation, limitations | MRAG reduces hallucination and outperforms text-only RAG when visual+textual understanding is required; systematizes ret | ABSTRACT_ONLY | Map of the field for the Atlas; confirms visual-RAG is an established 2024–26 line, not a fad |
| A2-6 | arXiv:2210.02928 | MuRAG: Multimodal Retrieval-Augmented Generator for Open Question Answering over Images and Text — Chen, Hu, Chen, Verga, Cohen — 2022, EMNLP 2022 | One transformer accessing an external **non-parametric multimodal memory** (text + images) via frozen retrievers | First multimodal retrieval-augmented transformer; **+10–20% absolute** over existing models on WebQA and MultimodalQA un | ABSTRACT_ONLY | The "multimodal external memory" concept Rune's Memory would extend; evidence that mixed-modality retrieval into a generator works |
| A2-7 | arXiv:2311.17136 | UniIR: Training and Benchmarking Universal Multimodal Information Retrievers — Wei, Chen, Chen, Hu, Zhang, Fu, Ritter, Chen — 202 | Instruction-guided universal multimodal retriever; M-BEIR benchmark (8 tasks, 10 datasets) | Multi-task training + instruction tuning give zero-shot generalization to new retrieval tasks; single model covers text→ | ABSTRACT_ONLY | If Rune ever indexes screenshots + audio + text in one index, instruction-guided universal retrieval is the pattern; but per-lane |
| A2-8 | arXiv:2608.07006 | Does More Retrieved Evidence Help Visual RAG with Diffusion Language Models? — Wang, Gao, Zhang, Fu, Bai, Gao — 2026 | Evidence-admission for visual RAG | Retrieving more pages raises answer-page **recall** but unconditionally passing them to the generator **reduces accuracy | ABSTRACT_ONLY | Directly supports Rune's sufficiency-gated abstention / selective-context design in the multimodal case: admission control survive |
| A2-9 | arXiv:2007.00398 | DocVQA: A Dataset for VQA on Document Images — Mathew, Karatzas, Jawahar — 2020, WACV 2021 | 50k questions on 12k+ document images | Baseline VQA/reading-comprehension models show a large gap to human 94.36%; models specifically fail where **document st | ABSTRACT_ONLY | The 2020 root evidence that structure (layout) is load-bearing for document QA — the thing flattening destroys |
| A2-10 | arXiv:2204.08387 | LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking — Huang, Lv, Cui, Lu, Wei — 2022, ACM MM | Text+layout+image pre-training for Document AI (forms, receipts, DocVQA, layout analysis) | Unified masking + word-patch alignment gives SOTA on text-centric AND image-centric doc tasks | ABSTRACT_ONLY | LayoutLM family (v1 1912.13318 → v3) is the OCR-era answer to structure: it still needs the parser pipeline ColPali removes; usefu |
| A2-11 | arXiv:2004.12832 | ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT — Khattab, Zaharia — 2020, SIGIR | Token-level multi-vector passage retrieval | Late interaction ≈ BERT cross-encoder effectiveness with 2 orders-of-magnitude faster queries and 4 orders fewer FLOPs; | ABSTRACT_ONLY | The architectural license for ColPali; also supports Rune rerank-by-MaxSim at small n |
| A2-12 | arXiv:2112.01488 | ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction — Santhanam, Khattab, Saad-Falcon, Potts, Zaharia — | Compressed multi-vector (residual) retrieval | SOTA in- and out-of-domain at the time; **6–10× space reduction** makes multi-vector storage tractable | ABSTRACT_ONLY | Storage objection to multi-vector at Rune scale is small (944 pages ≈ ~1M vectors even uncompressed — trivial for a workstation) |
| A2-13 | arXiv:2212.04356 | Robust Speech Recognition via Large-Scale Weak Supervision (Whisper) — Radford, Kim, Xu, Brockman, McLeavey, Sutskever — | ASR: 680k h weakly supervised, seq2seq, timestamps at 20 ms quantization | Zero-shot robustness: avg **12.8 WER** (large-v2) vs wav2vec-2.0 29.3 on 12 datasets (**55.2% relative error reduction** | RELEVANT_SECTIONS | **Anchor.** Timestamped transcripts are the natural retrieval substrate for audio, BUT the documented hallucination/repeat-loop mo |
| A2-14 | arXiv:2303.00747 | WhisperX: Time-Accurate Speech Transcription of Long-Form Audio — Bain, Huh, Han, Zisserman — 2023, INTERSPEECH | Long-form ASR + word-level timestamps via VAD + forced alignment | Buffered/sliding-window Whisper "prone to drifting, hallucination & repetition"; VAD Cut&Merge improves transcription qu | ABSTRACT_ONLY | The engineering answer for Rune: **VAD-segmented, timestamped** transcript chunks are the retrieval unit, not raw windows |
| A2-15 | arXiv:2402.08021 | Careless Whisper: Speech-to-Text Hallucination Harms — Koenecke, Choi, Mei, Schellmann, Sloane — 2024, FA | Hallucination audit of Whisper | **~1% of transcriptions contain entirely hallucinated phrases/sentences** absent from the audio; **38% of hallucinations | ABSTRACT_ONLY | For a memory system, a 1% hallucination rate compounds silently; retrieved transcript evidence needs provenance (timestamp + audio |
| A2-16 | arXiv:2505.12969 | Calm-Whisper: Reduce Whisper Hallucination On Non-Speech By Calming Crazy Heads Down — Wang, Alhmoud, Alsahly, Alqurishi, Ravanelli — 202 | Hallucination attribution + fix | **3 of 20 decoder heads account for >75% of non-speech hallucinations** (UrbanSound); fine-tuning them cuts non-speech h | ABSTRACT_ONLY | Even patched, non-speech handling needs VAD; if Rune records meetings/music-adjacent audio this is the relevant failure class |
| A2-17 | arXiv:2606.07473 | Whisper Hallucination Detection and Mitigation via Hidden Representation Steering and Sparse AutoEncoders — Aparin, Popov, Sadekova, Yermekova — 2026 | Hallucination detection/mitigation in activation space | Hallucination is linearly separable in Whisper encoder activations; SAE steering cuts non-speech hallucination **72.63% | ABSTRACT_ONLY | The hard-audio negative result: on non-speech input Whisper hallucinates most of the time; VAD gating before ASR is mandatory, not |
| A2-18 | arXiv:2504.05702 | Evaluating Speech-to-Text Systems with PennSound — Wright, Liberman, Ryant, Fiumara — 2025 | Benchmark of AWS/Azure/Google/IBM/NeMo/Rev.ai/Whisper/Whisper.cpp on ~10 h varied poetry recordings | Rev.ai best WER; **Whisper best open-source "as long as hallucinations were avoided"**; AWS best DER; WER/DER difference | ABSTRACT_ONLY | Independent confirmation on genuinely hard audio that open Whisper is usable but hallucination-guarded |
| A2-19 | arXiv:1911.01255 | pyannote.audio: neural building blocks for speaker diarization — Bredin et al. — 2019, ICASSP 2020 | Speaker diarization toolkit: VAD, speaker change detection, overlapped-speech detection, speaker embeddings | Pre-trained models reach SOTA on most of these subtasks; pipelines are composable/jointly optimizable | ABSTRACT_ONLY | Diarization supplies **speaker labels** on timestamped transcripts — a metadata lane, not a semantic lane |
| A2-20 | docs.pyannote.ai | pyannoteAI platform docs (speaker diarization) — pyannote — current (fetched 2026) | Diarization API: "who spoke when" and "who is speaking" via voiceprints | Diarization is positioned as foundation for transcription accuracy, speaker-specific logic, downstream analytics | DOCUMENTATION_AUDITED | Confirms diarization's role is **metadata enrichment** (speaker turns) of the transcript substrate — Rune gets most value from tim |
| A2-21 | arXiv:2107.09609 | QVHighlights: Detecting Moments and Highlights in Videos via Natural Language Queries (Moment-DETR) — Lei, Berg, Bansal — 2021, NeurIPS | Temporal sentence localization / moment retrieval (NOTE: correct ID; the ID often quoted for this paper, 2108.09968, is a mis-pairing) | 10k+ YouTube videos with NL queries + moment spans + saliency; Moment-DETR casts moment retrieval as direct **set predic | ABSTRACT_ONLY | Moment retrieval = the query interface for "when did X happen in the recording"; its dependence on ASR captions reinforces transcr |
| A2-22 | arXiv:2403.15377 | InternVideo2: Scaling Foundation Models for Multimodal Video Understanding — Wang, Li, Li, et al. — 2024, ECCV 2024 | Video foundation model (masked video modeling + contrastive + next-token; up to 6B) | SOTA on 60+ video/audio tasks incl. long-video dialogue; **semantically segmenting videos and generating video-audio-spe | ABSTRACT_ONLY | Two lessons: (1) segmentation-then-caption (hierarchical) beats dense frame indexing; (2) the deployable derivative of this line f |
| A2-23 | arXiv:2109.14084 | VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding — Xu, Ghosh, Huang, Okhonko, Aghajanyan, Metze, Zett | Zero-shot video-text retrieval/QA/localization | Contrastive video-text pretraining with hard negatives from NN retrieval; zero-shot SOTA on retrieval, VideoQA, action l | ABSTRACT_ONLY | Clip-level dense semantic indexing is feasible zero-shot, but per-frame indexing is neither done nor needed anywhere in this line |
| A2-24 | arXiv:2508.09736 | Seeing, Listening, Remembering, and Reasoning: A Multimodal Agent with Long-Term Memory (M3-Agent) — Long, He, Ye, Pan, Lin, Li, Zhao, Li — 2025 | Multimodal agent memory: real-time visual+auditory input → entity-centric episodic+semantic memory; multi-turn retrieval+reasoning; M3-Bench | Entity-centric **multimodal** memory organization beats prompting-agent baseline (Gemini-1.5-pro + GPT-4o) by **+6.7 / + | ABSTRACT_ONLY | The closest published analogue to "Rune with eyes and ears": memory entries organized around **entities with multimodal bindings** |
| A2-25 | DOI 10.1109/icdar.2007.4376991 | An Overview of the Tesseract OCR Engine — R. Smith — 2007, ICDAR (Crossref-verified; 1517 ci | OCR engine architecture | Tesseract's two-pass approach (character-level + language-model pass with fuzzy space resolution) became the open-source | METADATA_VERIFIED | Origin paper of the OCR engine Rune would use OCR-first; its own design scope (prose pages) is the boundary of where OCR-first wor |
| A2-26 | tesseract-ocr.github.io/tessdoc (User Manual + ImproveQuality) | Tesseract 5.x documentation — Tesseract maintainers — current | OCR operating envelope | Best at **≥300 DPI**; internal Otsu binarization "can be suboptimal, particularly if the page background is of uneven da | DOCUMENTATION_AUDITED | Vendor-acknowledged failure classes map 1:1 onto the page types ColPali wins on (tables, infographics, layout-heavy) — OCR reliabi |
| A2-27 | arXiv:2409.01704 | General OCR Theory: Towards OCR-2.0 via a Unified End-to-end Model (GOT-OCR2.0) — Wei et al. — 2024 | Unified OCR for "all artificial optical signals": text, **math/molecular formulas, tables, charts, sheet music**, geometric shapes; 580M par | One end-to-end model outputs plain **or formatted** (markdown/tikz/smiles) results; region-level recognition guided by c | ABSTRACT_ONLY | If Rune keeps an OCR lane, "OCR-2.0" models are the upgrade path that recovers math/table structure *as text* — a middle option be |
| A2-28 | arXiv:2011.05402 | OCR Post Correction for Endangered Language Texts — Rijhwani, Anastasopoulos, Neubig — 2020, EMNLP | OCR post-correction, data-scarce scripts | General-purpose OCR tools "are not robust" in data-scarce settings; tailored post-correction cuts recognition error **34 | ABSTRACT_ONLY | Bounded evidence that OCR errors are systematic and domain-dependent → OCR output quality must be measured per Rune's actual docum |
| A2-29 | arXiv:2106.06831 | Toward the Optimized Crowdsourcing Strategy for OCR Post-Correction — Suissa, Elmalech, Zhitomirsky-Geffet — 2021/2020, | Human/crowd correction of OCR in historical collections | 753-worker experiments: accuracy best with **paragraph-size units, two-phase, image shown next to text**; efficiency bes | ABSTRACT_ONLY | **Closest found work to "user-corrected OCR propagation"**: humans correct OCR best when shown the source image alongside text — e |
| A2-30 | ResourceSpace knowledge base (metadata configuration, indexing-metadat | ResourceSpace DAM docs — Montala Ltd. — current | Production digital-asset-management pattern: metadata-first media search | Fields must be explicitly **indexed** to be searchable; **required-at-upload** metadata guarantees searchability; contro | DOCUMENTATION_AUDITED | Production media systems ship exactly the hybrid Rune already has: **exact/structured metadata lookup coexisting with free-text/se |
| A2-31 | pyannote.audio + docs (see §1c) | (Counted once in §1c) — — | — | — | UNVERIFIED | — |
| A2-1 | arXiv:2407.01449 | ColPali: Efficient Document Retrieval with Vision Language Models — Faysse, Sibille, Wu, Omrani, Viaud, Hudelot, Colombo — 2024, ICLR 2025 | Page-image → multi-vector retrieval (late interaction over PaliGemma-3B token embeddings, D=128) | (1) ViDoRe benchmark (10 tasks, page-level); (2) ColPali avg **81.3 nDCG@5** vs best OCR-pipeline baseline (Unstructured+OCR BM25 **65.5**, Unstructured+Captioning BGE-M3 **67.0**) — ~+14–16 pts; (3) biggest gains exactly where text extraction loses signal: ArxivQ (figures) 79.1 vs OCR 31.6, InfoQ (infographics) 81.8 vs 62.9, TabFQuAD (tables) 83.9 vs 70.8 (BGE-M3+OCR), Shift (charts, French, zero-shot) 73.2 vs 64.3; (4) **ColQwen2 (768) ablation in v6: 86.6 avg** vs ColPali 81.3; (5) query latency ≤22 ms on L4; ingestion much faster/simpler than parser+OCR+caption pipelines | (a) On **text-dominant** financial pages (TAT-DQA) the gap nearly vanishes: OCR 62.7 vs ColPali 65.8 (+3.1); (b) BiPali (fine-tuned bi-encoder) sometimes *hurts* vs plain SigLIP (Energy −11.7); (c) ColSigLIP variants collapsed (~3.2 avg) — training instability of naive late-interaction variants; (d) low-res input destroys it: ColPali at 224px = 56.5 avg vs 81.3 at 448px | Trained/evaluated on page-level QA-style queries, not agentic exact-match queries; multi-vector store is ~1030 vectors/page at 448px (heavier than single-vector); needs GPU for indexing; benchmark queries are VLM-generated (Claude-3 Sonnet) for practical tasks | RELEVANT_SECTIONS | **Anchor for the whole arm.** Directly quantifies what Rune's flatten-to-text PDF ingestion loses (figures/tables/layout: +10 to +48 nDCG). Also shows OCR-first is *fine* on text-dense pages — the verdict is conditional, not absolute. ColQwen2 result shows **model/version shifts scores by >5 points** → Rune's unversioned embeddings are unreproducible |
| A2-2 | arXiv:2406.11251 | Unifying Multimodal Retrieval via Document Screenshot Embedding (DSE) — Ma, Lin, Li, Chen, Lin — 2024, EMNLP 2024 main | Single-vector screenshot embedding (EVA-CLIP + LLM layer, 1.3M Wiki-SS corpus) | Screenshot-as-unified-input removes parsing entirely; Wiki-SS (text-intensive): **+17 pts top-1 over BM25**; mixed-modality slide retrieval: **+15 nDCG@10 over OCR-text retrieval** | On text-intensive Wikipedia pages DSE is only "competitive" with text retrieval — visual embeddings do not beat strong text retrieval when the page is prose | Single-vector (loses fine-grained layout matching vs late interaction); Wiki-SS is web screenshots, not scanned PDFs | ABSTRACT_ONLY | Second independent confirmation of the OCR-first loss number on slides (+15) and the "text-only pages ≈ parity" boundary |
| A2-3 | arXiv:2410.10594 | VisRAG: Vision-based Retrieval-augmented Generation on Multi-modality Documents — Yu, Tang, Xu, Cui, Ran, Yan, Liu, Wang, Han, Liu, Sun — 2024/2025, Tsinghua | VLM-based RAG: embed page images → retrieve → feed image to VLM generator | **20–40% end-to-end gain over text-based RAG** on multi-modality docs; eliminates parsing information loss; strong generalization from limited training data | (implicit) gains concentrate on visually rich docs; text RAG remains competitive on plain text | End-to-end gain bundles retrieval *and* generation improvements; generator is a VLM (GPU-heavy) | ABSTRACT_ONLY | Quantifies the retrieval-loss → answer-quality pipeline effect Rune's flattening causes |
| A2-4 | arXiv:2411.04952 | M3DocRAG: Multi-modal Retrieval is What You Need for Multi-page Multi-document Understanding — Cho, Mahata, Irsoy, He, Bansal — 2024 (UNC/IBM) | Open-domain multi-page DocVQA: ColPali retrieval + Qwen2-VL 7B generation | Works open-domain over **3,000+ PDFs / 40,000+ pages** (M3DocVQA benchmark); SOTA on MP-DocVQA; handles multi-hop and image-only evidence; comprehensive ablations of indexing/models/retrievers | Existing DocVQA methods fail when evidence is in figures or spread across pages (motivating result) | Indexing ablations show choices matter; generation still needs a 7B VLM | ABSTRACT_ONLY | Evidence that ColPali-style page retrieval **scales beyond toy corpora** to ~40k pages — far past Rune's needs, so scale is no objection |
| A2-5 | arXiv:2504.08748 | A Survey of Multimodal Retrieval-Augmented Generation — Mei, Mo, Yang, Chen — 2025 | Survey: MRAG components, datasets, evaluation, limitations | MRAG reduces hallucination and outperforms text-only RAG when visual+textual understanding is required; systematizes retriever/generator/fusion taxonomy | Identifies open problems: modality alignment, evaluation gaps | Survey-level, no new measurements | ABSTRACT_ONLY | Map of the field for the Atlas; confirms visual-RAG is an established 2024–26 line, not a fad |
| A2-6 | arXiv:2210.02928 | MuRAG: Multimodal Retrieval-Augmented Generator for Open Question Answering over Images and Text — Chen, Hu, Chen, Verga, Cohen — 2022, EMNLP 2022 | One transformer accessing an external **non-parametric multimodal memory** (text + images) via frozen retrievers | First multimodal retrieval-augmented transformer; **+10–20% absolute** over existing models on WebQA and MultimodalQA under distractor and full-wiki settings | — | Pre-LLM-era (T5/FLAN-size generator); retrieval over web QA corpora | ABSTRACT_ONLY | The "multimodal external memory" concept Rune's Memory would extend; evidence that mixed-modality retrieval into a generator works with a frozen retriever |
| A2-7 | arXiv:2311.17136 | UniIR: Training and Benchmarking Universal Multimodal Information Retrievers — Wei, Chen, Chen, Hu, Zhang, Fu, Ritter, Chen — 2023 | Instruction-guided universal multimodal retriever; M-BEIR benchmark (8 tasks, 10 datasets) | Multi-task training + instruction tuning give zero-shot generalization to new retrieval tasks; single model covers text→image, image→text, etc. | Single-task models beat the universal one per-task (universal ≠ best specialist) | CLIP-style dual-encoder granularity; coarse page/region level | ABSTRACT_ONLY | If Rune ever indexes screenshots + audio + text in one index, instruction-guided universal retrieval is the pattern; but per-lane specialists remain stronger at small n |
| A2-8 | arXiv:2608.07006 | Does More Retrieved Evidence Help Visual RAG with Diffusion Language Models? — Wang, Gao, Zhang, Fu, Bai, Gao — 2026 | Evidence-admission for visual RAG | Retrieving more pages raises answer-page **recall** but unconditionally passing them to the generator **reduces accuracy** (semantic conflict between visual sources); entropy-based candidate filter (ECF) recovers +2.62 pts over best fixed top-k | **Negative result: more visual context ≠ better answers** | Diffusion-LM-specific mechanism (ECF) | ABSTRACT_ONLY | Directly supports Rune's sufficiency-gated abstention / selective-context design in the multimodal case: admission control survives modality change |
| A2-9 | arXiv:2007.00398 | DocVQA: A Dataset for VQA on Document Images — Mathew, Karatzas, Jawahar — 2020, WACV 2021 | 50k questions on 12k+ document images | Baseline VQA/reading-comprehension models show a large gap to human 94.36%; models specifically fail where **document structure understanding is crucial** | Models "perform reasonably" only on questions not requiring structure | Pre-VLM baselines; dataset paper | ABSTRACT_ONLY | The 2020 root evidence that structure (layout) is load-bearing for document QA — the thing flattening destroys |
| A2-10 | arXiv:2204.08387 | LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking — Huang, Lv, Cui, Lu, Wei — 2022, ACM MM | Text+layout+image pre-training for Document AI (forms, receipts, DocVQA, layout analysis) | Unified masking + word-patch alignment gives SOTA on text-centric AND image-centric doc tasks | — | Requires OCR+layout input upstream (it is a *consumer* of OCR output, not a replacement); encoder, not retriever | ABSTRACT_ONLY | LayoutLM family (v1 1912.13318 → v3) is the OCR-era answer to structure: it still needs the parser pipeline ColPali removes; useful as the "OCR-first + layout model" compromise lane |
| A2-11 | arXiv:2004.12832 | ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT — Khattab, Zaharia — 2020, SIGIR | Token-level multi-vector passage retrieval | Late interaction ≈ BERT cross-encoder effectiveness with 2 orders-of-magnitude faster queries and 4 orders fewer FLOPs; supports end-to-end vector-index retrieval | — | Multi-vector space cost (pre-v2) | ABSTRACT_ONLY | The architectural license for ColPali; also supports Rune rerank-by-MaxSim at small n |
| A2-12 | arXiv:2112.01488 | ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction — Santhanam, Khattab, Saad-Falcon, Potts, Zaharia — 2021/2022, NAACL | Compressed multi-vector (residual) retrieval | SOTA in- and out-of-domain at the time; **6–10× space reduction** makes multi-vector storage tractable | — | Residual-compression index complexity | ABSTRACT_ONLY | Storage objection to multi-vector at Rune scale is small (944 pages ≈ ~1M vectors even uncompressed — trivial for a workstation) |
| A2-13 | arXiv:2212.04356 | Robust Speech Recognition via Large-Scale Weak Supervision (Whisper) — Radford, Kim, Xu, Brockman, McLeavey, Sutskever — 2022 | ASR: 680k h weakly supervised, seq2seq, timestamps at 20 ms quantization | Zero-shot robustness: avg **12.8 WER** (large-v2) vs wav2vec-2.0 29.3 on 12 datasets (**55.2% relative error reduction**); approaches human robustness; long-form WER: TED-LIUM3 3.5, Earnings-22 12.6, CORAAL 19.6 | Long-form failure modes are **non-perceptual**: repeat loops, dropped first/last words of a window, "complete hallucination where the model will output a transcript entirely unrelated to the actual audio" (§6); decoding heuristics (beam+temperature fallback, no-speech prob 0.6 + avg logprob −1.0, initial-timestamp constraint) are explicitly "a workaround for the noisy predictions" (§4.5); WER normalization itself can shift scores up to ~50% | Timestamp quantization 20 ms; 30-s window design → segmentation is external to the model; no speaker diarization (explicitly listed as external pipeline component, §2.3) | RELEVANT_SECTIONS | **Anchor.** Timestamped transcripts are the natural retrieval substrate for audio, BUT the documented hallucination/repeat-loop modes mean Rune must segment with VAD and keep non-speech probabilities — a bare 30-s window transcript is unsafe to index |
| A2-14 | arXiv:2303.00747 | WhisperX: Time-Accurate Speech Transcription of Long-Form Audio — Bain, Huh, Han, Zisserman — 2023, INTERSPEECH | Long-form ASR + word-level timestamps via VAD + forced alignment | Buffered/sliding-window Whisper "prone to drifting, hallucination & repetition"; VAD Cut&Merge improves transcription quality AND enables **12× batched speedup**; SOTA long-form + word segmentation benchmarks | Confirms Whisper's native long-form decoding degrades without segmentation | Alignment model needed for word timestamps | ABSTRACT_ONLY | The engineering answer for Rune: **VAD-segmented, timestamped** transcript chunks are the retrieval unit, not raw windows |
| A2-15 | arXiv:2402.08021 | Careless Whisper: Speech-to-Text Hallucination Harms — Koenecke, Choi, Mei, Schellmann, Sloane — 2024, FAccT (DOI 10.1145/3630106.3658996) | Hallucination audit of Whisper | **~1% of transcriptions contain entirely hallucinated phrases/sentences** absent from the audio; **38% of hallucinations include explicit harms** (violence, false authority); hallucinations disproportionately hit speakers with longer non-vocal durations (aphasia) | Direct negative: fluent-but-false transcripts at ~1% base rate even in ordinary use | English-language study; transcription-service framing | ABSTRACT_ONLY | For a memory system, a 1% hallucination rate compounds silently; retrieved transcript evidence needs provenance (timestamp + audio pointer) so hallucinated text can be checked against audio |
| A2-16 | arXiv:2505.12969 | Calm-Whisper: Reduce Whisper Hallucination On Non-Speech By Calming Crazy Heads Down — Wang, Alhmoud, Alsahly, Alqurishi, Ravanelli — 2025, Interspeech | Hallucination attribution + fix | **3 of 20 decoder heads account for >75% of non-speech hallucinations** (UrbanSound); fine-tuning them cuts non-speech hallucination >80% with <0.1% WER cost on LibriSpeech | Confirms hallucination is a systematic decoder pathology, not random noise | Fine-tuning-based fix | ABSTRACT_ONLY | Even patched, non-speech handling needs VAD; if Rune records meetings/music-adjacent audio this is the relevant failure class |
| A2-17 | arXiv:2606.07473 | Whisper Hallucination Detection and Mitigation via Hidden Representation Steering and Sparse AutoEncoders — Aparin, Popov, Sadekova, Yermekova — 2026 | Hallucination detection/mitigation in activation space | Hallucination is linearly separable in Whisper encoder activations; SAE steering cuts non-speech hallucination **72.63% → 14.11% (small)** and **86.88% → 27.33% (large-v3)** on full non-speech test sets, with small speech WER cost | Establishes baseline non-speech hallucination rates are **~73–87%** on non-speech audio — worst-case audio is unusable unmitigated | Research code, not production | ABSTRACT_ONLY | The hard-audio negative result: on non-speech input Whisper hallucinates most of the time; VAD gating before ASR is mandatory, not optional |
| A2-18 | arXiv:2504.05702 | Evaluating Speech-to-Text Systems with PennSound — Wright, Liberman, Ryant, Fiumara — 2025 | Benchmark of AWS/Azure/Google/IBM/NeMo/Rev.ai/Whisper/Whisper.cpp on ~10 h varied poetry recordings | Rev.ai best WER; **Whisper best open-source "as long as hallucinations were avoided"**; AWS best DER; WER/DER differences slim; runtime options (speed vs accuracy) materially change outcomes | Whisper's ranking is conditional on hallucination avoidance — an operational caveat, not a footnote | Poetry recordings = hard-accent/noisy speech; small sample (10 h) | ABSTRACT_ONLY | Independent confirmation on genuinely hard audio that open Whisper is usable but hallucination-guarded |
| A2-19 | arXiv:1911.01255 | pyannote.audio: neural building blocks for speaker diarization — Bredin et al. — 2019, ICASSP 2020 | Speaker diarization toolkit: VAD, speaker change detection, overlapped-speech detection, speaker embeddings | Pre-trained models reach SOTA on most of these subtasks; pipelines are composable/jointly optimizable | — | Diarization ≠ identification; DER on overlap remains hard | ABSTRACT_ONLY | Diarization supplies **speaker labels** on timestamped transcripts — a metadata lane, not a semantic lane |
| A2-20 | docs.pyannote.ai | pyannoteAI platform docs (speaker diarization) — pyannote — current (fetched 2026) | Diarization API: "who spoke when" and "who is speaking" via voiceprints | Diarization is positioned as foundation for transcription accuracy, speaker-specific logic, downstream analytics | — | Vendor docs; API service framing | DOCUMENTATION_AUDITED | Confirms diarization's role is **metadata enrichment** (speaker turns) of the transcript substrate — Rune gets most value from timestamps + speaker turns, not from embedding raw audio |
| A2-21 | arXiv:2107.09609 | QVHighlights: Detecting Moments and Highlights in Videos via Natural Language Queries (Moment-DETR) — Lei, Berg, Bansal — 2021, NeurIPS | Temporal sentence localization / moment retrieval (NOTE: correct ID; the ID often quoted for this paper, 2108.09968, is a mis-pairing) | 10k+ YouTube videos with NL queries + moment spans + saliency; Moment-DETR casts moment retrieval as direct **set prediction** (DETR-style), no human priors; ASR-caption weak pretraining substantially improves results | Strong supervised "well-engineered" baselines competitive without priors; benchmark shows localization is far from solved | Depends on extracted video features + ASR captions (i.e., text substrate again) | ABSTRACT_ONLY | Moment retrieval = the query interface for "when did X happen in the recording"; its dependence on ASR captions reinforces transcripts-as-substrate |
| A2-22 | arXiv:2403.15377 | InternVideo2: Scaling Foundation Models for Multimodal Video Understanding — Wang, Li, Li, et al. — 2024, ECCV 2024 | Video foundation model (masked video modeling + contrastive + next-token; up to 6B) | SOTA on 60+ video/audio tasks incl. long-video dialogue; **semantically segmenting videos and generating video-audio-speech captions** improves video-text alignment | — | 6B model — not workstation-deployable at full size | ABSTRACT_ONLY | Two lessons: (1) segmentation-then-caption (hierarchical) beats dense frame indexing; (2) the deployable derivative of this line for Rune is caption/ASR extraction, not the foundation model |
| A2-23 | arXiv:2109.14084 | VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding — Xu, Ghosh, Huang, Okhonko, Aghajanyan, Metze, Zettlemoyer, Feichtenhofer — 2021, EMNLP | Zero-shot video-text retrieval/QA/localization | Contrastive video-text pretraining with hard negatives from NN retrieval; zero-shot SOTA on retrieval, VideoQA, action localization/segmentation; sometimes beats supervised | — | Clip-level granularity; short clips (temporal overlap as positives) | ABSTRACT_ONLY | Clip-level dense semantic indexing is feasible zero-shot, but per-frame indexing is neither done nor needed anywhere in this line — clip/shot granularity is the norm (see §2 synthesis) |
| A2-24 | arXiv:2508.09736 | Seeing, Listening, Remembering, and Reasoning: A Multimodal Agent with Long-Term Memory (M3-Agent) — Long, He, Ye, Pan, Lin, Li, Zhao, Li — 2025 | Multimodal agent memory: real-time visual+auditory input → entity-centric episodic+semantic memory; multi-turn retrieval+reasoning; M3-Bench (100 robot videos + 920 web videos, long-video QA) | Entity-centric **multimodal** memory organization beats prompting-agent baseline (Gemini-1.5-pro + GPT-4o) by **+6.7 / +7.7 / +5.3 points** on M3-Bench-robot / -web / VideoMME-long; RL training on memory-based reasoning | Strong prompting baselines already close — the memory-organization delta is real but single-digit | Video QA domain, not code-agent domain; benchmark small (100 robot videos) | ABSTRACT_ONLY | The closest published analogue to "Rune with eyes and ears": memory entries organized around **entities with multimodal bindings**, retrieved by multi-turn reasoning — a pattern transferable to Rune's entry schema (modality-specific payload + shared entity keys) |
| A2-25 | DOI 10.1109/icdar.2007.4376991 | An Overview of the Tesseract OCR Engine — R. Smith — 2007, ICDAR (Crossref-verified; 1517 citations) | OCR engine architecture | Tesseract's two-pass approach (character-level + language-model pass with fuzzy space resolution) became the open-source standard; its layout analysis was designed for *printed prose pages* | — | Pre-LSTM (Tesseract 4/5 changed the engine); 2007 | METADATA_VERIFIED | Origin paper of the OCR engine Rune would use OCR-first; its own design scope (prose pages) is the boundary of where OCR-first works |
| A2-26 | tesseract-ocr.github.io/tessdoc (User Manual + ImproveQuality) | Tesseract 5.x documentation — Tesseract maintainers — current | OCR operating envelope | Best at **≥300 DPI**; internal Otsu binarization "can be suboptimal, particularly if the page background is of uneven darkness"; skew "reduces line segmentation significantly"; **"It is known tesseract has a problem to recognize text/data from tables … without custom segmentation/layout analysis"**; PSM modes exist because default whole-page segmentation fails on sparse/non-prose layouts; dictionaries hurt non-prose text (codes, receipts) unless disabled | The docs themselves enumerate the failure classes: low DPI, uneven background, skew, tables, non-dictionary text, missing/oversized borders, transparency | Documentation, not benchmark numbers | DOCUMENTATION_AUDITED | Vendor-acknowledged failure classes map 1:1 onto the page types ColPali wins on (tables, infographics, layout-heavy) — OCR reliability is the *mechanism* behind the OCR-first loss |
| A2-27 | arXiv:2409.01704 | General OCR Theory: Towards OCR-2.0 via a Unified End-to-end Model (GOT-OCR2.0) — Wei et al. — 2024 | Unified OCR for "all artificial optical signals": text, **math/molecular formulas, tables, charts, sheet music**, geometric shapes; 580M params | One end-to-end model outputs plain **or formatted** (markdown/tikz/smiles) results; region-level recognition guided by coordinates/colors; multi-page + dynamic resolution | Frames OCR-1.0 (incl. Tesseract-class tools) as "increasingly unable to meet usage" for non-plain-text signals | Small model — capability below frontier VLMs; no retrieval evaluation | ABSTRACT_ONLY | If Rune keeps an OCR lane, "OCR-2.0" models are the upgrade path that recovers math/table structure *as text* — a middle option between flattening and page-image retrieval |
| A2-28 | arXiv:2011.05402 | OCR Post Correction for Endangered Language Texts — Rijhwani, Anastasopoulos, Neubig — 2020, EMNLP | OCR post-correction, data-scarce scripts | General-purpose OCR tools "are not robust" in data-scarce settings; tailored post-correction cuts recognition error **34%** on average across three languages | General OCR outright fails on low-resource scripts | Niche domain (endangered languages) | ABSTRACT_ONLY | Bounded evidence that OCR errors are systematic and domain-dependent → OCR output quality must be measured per Rune's actual document population, not assumed |
| A2-29 | arXiv:2106.06831 | Toward the Optimized Crowdsourcing Strategy for OCR Post-Correction — Suissa, Elmalech, Zhitomirsky-Geffet — 2021/2020, Aslib J. of Information Management | Human/crowd correction of OCR in historical collections | 753-worker experiments: accuracy best with **paragraph-size units, two-phase, image shown next to text**; efficiency best with longer text, single-stage, no image; first systematic study of crowd OCR-correction strategy | Accuracy and efficiency optima **conflict** (image+two-phase accurate; no-image+single-stage efficient) | Historical documents; crowd settings, not agent settings | ABSTRACT_ONLY | **Closest found work to "user-corrected OCR propagation"**: humans correct OCR best when shown the source image alongside text — evidence FOR storing the page image alongside extracted text in Rune, so corrections (human or model) can be validated against the source. No work found on *propagating* user corrections through a retrieval index (→ gap, §4) |
| A2-30 | ResourceSpace knowledge base (metadata configuration, indexing-metadata, field types) | ResourceSpace DAM docs — Montala Ltd. — current | Production digital-asset-management pattern: metadata-first media search | Fields must be explicitly **indexed** to be searchable; **required-at-upload** metadata guarantees searchability; controlled vocabularies/category trees vs free text; separate "extracted text" channel exists *alongside* curated metadata | — | Vendor docs, single product | DOCUMENTATION_AUDITED | Production media systems ship exactly the hybrid Rune already has: **exact/structured metadata lookup coexisting with free-text/semantic search over extracted content**; the modality of the asset (image/audio/video) never replaces the metadata lane |
| A2-31 | cross-reference row — see A2-16 (pyannote.audio arXiv:1911.01255) and A2-17 (pyannoteAI docs) | pyannote.audio + docs counted once at their primary rows; this row is a pointer, not an independent source | cross-reference | NOT_RECORDED_IN_ARM (pointer row — no independent claims) | NOT_RECORDED_IN_ARM | NOT_RECORDED_IN_ARM | UNVERIFIED | prevents double-counting pyannote in dedupe accounting |
## Arm 3 — reliability/security
## Arm 3 — reliability/security (A3-1 … A3-45)
| # | Source ID | Title / authors / year / venue | Key claims | Negative results | Read depth | Rune relevance |
|---|---|---|---|---|---|---|
| A3-1 | arXiv:2305.10355 | Evaluating Object Hallucination in Large Vision-Language Models (POPE) — Li, Du, Zhou, Wang, Zhao, Wen — 2023, EMNLP 2023 | Image captioning / VQA hallucination measurement | First systematic object-hallucination study of LVLMs. (1) CHAIR: LLaVA CHAIR_S **32.7** vs small VLPM OSCAR-base **13.0* | FULL_PAPER | **Anchor for hallucination measurement.** If Rune ever captions/screenshots into memory, expect language-prior-driven false object |
| A3-2 | arXiv:1809.02156 | Object Hallucination in Image Captioning — Rohrbach, Hendricks, Burns, Darrell, Saenko — 2018 | Image captioning hallucination metric (CHAIR origin) | Introduces CHAIR (CHAIR_I instance-level, CHAIR_S sentence-level) using veridical ground-truth labels on MSCOCO; finds m | ABSTRACT_ONLY | Metric lineage confirmed; the "language prior beats perception" mechanism is the durable finding that persists into LVLMs (per POP |
| A3-3 | arXiv:2404.18930 | Hallucination of Multimodal Large Language Models: A Survey — Bai, Wang, Xiao, He, Han, Zhang, Shou — 2024/2025 | Survey of MLLM hallucination causes/benchmarks/mitigations | Systematizes hallucination in MLLMs: causes span training data, vision encoder, alignment module; catalogues benchmarks | ABSTRACT_ONLY | Map of the mitigation families; confirms 2024–26 the field treats hallucination as unsolved-but-measurable, i.e., Rune must design |
| A3-4 | arXiv:2309.14525 | Aligning Large Multimodal Models with Factually Augmented RLHF (LLaVA-RLHF) — Sun, Shen, Cao, Liu, et al. — 2023 | Hallucination mitigation via RLHF + factual reward augmentation | RLHF with a factually-augmented reward (captions, ground-truth options) reduces hallucination and reward hacking; +60% o | ABSTRACT_ONLY | Evidence that grounding-informed rewards help; but training-side fixes are unavailable to Rune (it consumes models, doesn't train |
| A3-5 | arXiv:2310.16045 | Woodpecker: Hallucination Correction for Multimodal Large Language Models — Yin, Fu, Zhao, Xu, Wang, Sui, Shen, Li, Sun, Chen | Training-free post-hoc hallucination correction | Five-stage verifier pipeline (concept extraction → question formulation → visual validation → claim generation → correct | ABSTRACT_ONLY | The verifier/grounding pattern with measured gains; Rune-scale adaptation: probing claims against the source image at write-time i |
| A3-6 | arXiv:2406.18521 | CharXiv: Charting Gaps in Realistic Chart Understanding in Multimodal LLMs — Wang, Xia, He, Chen, Liu, Zhu, Liang, Wu, Liu, Liu | Chart QA benchmark on 2,323 real arXiv-paper charts | On realistic charts the strongest proprietary model (GPT-4o) reaches only **47.1%** on reasoning questions and strongest | ABSTRACT_ONLY | Chart misreading is severe at realistic difficulty. If Rune indexes charts/screenshots of dashboards, chart-derived numeric claims |
| A3-7 | arXiv:2203.10244 | ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning — Masry, Long, Tan, Joty, Hoque — 2022, ACL Findings | Chart QA (human + generated questions, 9.6k human questions) | Established that chart QA needs visual AND logical reasoning; 2022-era models showed a large gap on complex reasoning qu | ABSTRACT_ONLY | Root evidence for the chart-claims-are-risky rule; superseded in difficulty by CharXiv |
| A3-8 | arXiv:2302.12173 | Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection — Greshake, Abdelnabi, Mishra, Endres, Holz, Fritz — | Indirect prompt injection (text carriers) in LLM-integrated apps | Injection via *retrieved/processed data* (web pages, documents) demonstrated against Bing/GPT-4-powered systems; taxonom | ABSTRACT_ONLY | The foundational framing: anything Rune reads is code in the instruction channel. Every later multimodal carrier paper instantiate |
| A3-9 | arXiv:2307.10490 | Abusing Images and Sounds for Indirect Instruction Injection in Multi-Modal LLMs — Bagdasaryan, Hsieh, Nassi, Shmatikov — 2023, Corne | Adversarial perturbation in images AND audio that steers MLLM output | Perturbations blended into images/audio make unmodified LLaVA/PandaGPT output attacker-chosen text and steer subsequent | ABSTRACT_ONLY | Proof that the perceptual channel itself is an injection vector even without readable text — OCR-based sanitization alone cannot c |
| A3-10 | arXiv:2309.00236 | Image Hijacks: Adversarial Images can Control Generative Models at Runtime — Bailey, Ong, Russell, Emmons — 2023/2024, UC Berke | Optimized image "hijacks" controlling VLM behavior | Behaviour/Prompt-Matching algorithms craft images that force VLMs to (1) emit attacker outputs, (2) **leak context-windo | ABSTRACT_ONLY | The context-exfiltration + false-belief attack classes map directly onto memory poisoning: an adversarial screenshot can implant f |
| A3-11 | arXiv:2306.13213 | Visual Adversarial Examples Jailbreak Aligned Large Language Models — Qi, Huang, Panda, Henderson, Wang, Mittal — 2023, | Visual adversarial examples vs aligned LLMs | Vision input is a **weak link**: a single optimized image can *universally* jailbreak an aligned vision-integrated LLM i | ABSTRACT_ONLY | Even where Rune's stack reads no text, the image encoder is the bypass; model-internal defenses cannot be assumed |
| A3-12 | arXiv:2402.19150 | Unveiling Typographic Deceptions: Insights of the Typographic Vulnerability in Large Vision-Language Models — Cheng, Xiao, Gu, Yang, Duan, Zhang, Cao, Xu, Xu — | Typographic (embedded-text) attacks on commercial + open LVLMs | Typographic attacks (text rendered into images) are **widespread across commercial and open LVLMs**; largest typographic | ABSTRACT_ONLY | Text-in-images is *transcribed and obeyed* by current models as a rule, not an edge case. Rune's screenshots regularly contain tex |
| A3-13 | arXiv:2504.04893 | SCAM: A Real-World Typographic Robustness Evaluation for Multimodal Foundation Models — Westerhoff, Purelku, Hackstein, Loos, Pinetzki, Ro | Real-world typographic attacks (1,162 handwritten/real images, hundreds of categories) | Typographic attacks **remain effective against SOTA LVLMs**, especially those with vulnerable vision encoders; larger LL | ABSTRACT_ONLY | Independent 2025 confirmation with *real-world* (not synthetic) images: the failure mode survives model generations — design for i |
| A3-14 | arXiv:2609.04533 | Repeat-After-Me: Black-Box Adaptive Visual Prompt Injection — Chen, Tsai, Evtimov, Chaudhuri, Popa, Wagner, Zhar | Black-box ADAPTIVE visual prompt injection against frontier VLMs | Adaptive black-box image injection achieves ASR **>82%** on open-weight frontier models and **47%** on commercial fronti | ABSTRACT_ONLY | **The Atlas "defenses fail under adaptive attack" lesson, in the visual domain.** Note the TOOLS.md/Memory-file overwrite pattern: |
| A3-15 | arXiv:2609.09404 | An Experimental Evaluation of Multimodal Prompt Injection Attacks on Agentic AI Frameworks (MMPIBench) — Nguyen, Husain — 2026 | Benchmark of six visual carriers (OCR text, overlays, **EXIF metadata**, **QR codes**, fake interfaces, hybrids) across 6 agent frameworks × | Attacks **attempted in 12.8%** of runs but **completed in ~1%** for vision — the gap closes at the *planning* step (mode | ABSTRACT_ONLY | Carrier taxonomy directly relevant to Rune (screenshots carry OCR text; EXIF and QR are real carriers); the vision-vs-audio defens |
| A3-16 | arXiv:2403.02691 | InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated LLM Agents — Zhan, Liang, Ying, Kang — 2024, ACL Findings | IPI benchmark: 1,054 cases, 17 user tools, 62 attacker tools, 30 LLM agents | ReAct-prompted GPT-4 executes injected instructions **24%** of the time (direct harm) / more for data exfiltration; a "h | ABSTRACT_ONLY | Tool-integrated agents (Rune's shape) are measurably injectable even in the text-only case; establishes the exfiltration-vs-direct |
| A3-17 | arXiv:2403.14720 | Defending Against Indirect Prompt Injection Attacks With Spotlighting — Hines, Lopez, Hall, Zarfati, Zunger, Kiciman (Micr | Prompt-engineering defense: delimiting/datamarking/encoding of untrusted content | Spotlighting reduces IPI ASR from **>50% to <2%** with minimal task-efficiency cost, by giving a continuous provenance s | ABSTRACT_ONLY | Cheap, evidence-backed input-hygiene defense Rune can adopt at ingestion (mark provenance of every untrusted span) — but its evalu |
| A3-18 | arXiv:2402.06363 | StruQ: Defending Against Prompt Injection with Structured Queries — Chen, Piet, Sitawarin, Wagner — 2024, USENIX Secur | Two-channel (prompt/data) structured queries + tuned LLM | Structured separation of instructions from data significantly improves resistance with little/no utility loss | ABSTRACT_ONLY | The model-side structural defense; informative as the "right" design if Rune ever trains/finetunes, but not deployable as a wrappe |
| A3-19 | arXiv:2503.18813 | Defeating Prompt Injections by Design (CaMeL) — Debenedetti, Shumailov, Fan, Hayes, Carlini, Fabia | System-level defense: control/data-flow extraction + capabilities enforced at tool calls | CaMeL creates a protective system layer around a *susceptible* LLM: untrusted data can never influence program flow; pol | ABSTRACT_ONLY | **The defense that survives adaptive evaluation** — and it works by moving enforcement OUTSIDE the model (capability checks at too |
| A3-20 | arXiv:2503.06254 | Poisoned-MRAG: Knowledge Poisoning Attacks to Multimodal Retrieval Augmented Generation — Liu, Yuan, Tie, Shi, Zhou, Sun, Gong — 2025 | Poisoned image-text pairs injected into multimodal KB | First multimodal RAG knowledge-poisoning attack: **up to 98% ASR with just 5 poisoned pairs** in a 481,782-pair DB (Info | ABSTRACT_ONLY | If Rune's memory accepts user/agent-contributed media, 5 poisoned items in a ~10⁵ corpus is enough. Defenses inside the pipeline f |
| A3-21 | arXiv:2502.17832 | MM-PoisonRAG: Disrupting Multimodal RAG with Local and Global Poisoning Attacks — Ha, Zhan, Kim, Bralios, Sanniboina, Peng, Chang, K | Localized (query-specific) vs globalized (untargeted) multimodal poisoning | Localized attack: **56% ASR under restricted attacker access**; transfers across 4 retrievers without re-optimization. G | ABSTRACT_ONLY | The one-poison-collapses-everything result argues for write-time quarantine + integrity checks on the media index; a corrupted med |
| A3-22 | arXiv:2505.18543 | Benchmarking Poisoning Attacks against Retrieval-Augmented Generation — Zhang, Xin, Li, Zhang, Fang, Liu, Nie, Liu — 2025 | First comprehensive poisoning benchmark: 15 datasets, 13 attacks, 7 defenses, across RAG architectures | Attack effectiveness drops on adversarially expanded datasets; sequential/branching/conditional/loop RAG, multi-turn RAG | ABSTRACT_ONLY | Cross-architecture confirmation that in-pipeline defenses fail; supports putting trust decisions in the tool boundary + provenance |
| A3-23 | arXiv:2106.09898 | Bad Characters: Imperceptible NLP Attacks — Boucher, Shumailov, Anderson, Papernot — 2021, IEE | Invisible characters, homoglyphs, reordering, deletion in text pipelines | A single imperceptible encoding injection degrades or breaks NLP systems (translation, search, moderation) **including d | ABSTRACT_ONLY | **Directly relevant to taint-through-transformation**: OCR of a hostile image can output invisible/bidi characters that then flow |
| A3-24 | arXiv:1708.09537 | DolphinAttack: Inaudible Voice Commands — Zhang, Yan, Ji, Zhang, Zhang, Xu — 2017, ACM CCS 2 | Ultrasonic (>20 kHz) modulated voice commands exploiting microphone nonlinearity | Inaudible commands successfully control Siri, Google Now, Alexa, Cortana, automotive navigation (FaceTime call, airplane | ABSTRACT_ONLY | Credible but *conditions-constrained*: real risk only if Rune records audio in physical spaces with hostile nearby hardware; mitig |
| A3-25 | C2PA Technical Specification 2.2 (c2pa.org/specifications/specificatio | Content Credentials: C2PA Technical Specification — C2PA — v2.2 May 2025 (2.4 current) | Media provenance standard: signed manifests (assertions + claim + signature), hard/soft content bindings, trust lists, validation states | (1) Provenance = **signed assertions bound by cryptographic hash ("hard binding")** to exact bytes; (2) §9: **"a soft bi | DOCUMENTATION_AUDITED | **The most mature normative statement of this arm's central verdicts**: (a) metadata = source-asserted, cryptographically bound, n |
| A3-26 | arXiv:2302.10174 | Towards Universal Fake Image Detectors that Generalize Across Generative Models — Ojha, Li, Lee — 2023, CVPR 2023 | AI-image detector reliability across generators | Trained real-vs-fake detectors fail on unseen generators: the "real" class becomes a **sink class** absorbing everything | ABSTRACT_ONLY | Measured unreliability of AI-image detectors → Rune must NOT store "this image is AI/synthetic" detector verdicts as memory facts; |
| A3-27 | arXiv:2308.14970 | Audio Deepfake Detection: A Survey — Yi, Wang, Tao, Zhang, Zhang, Zhao — 2023 | Survey + unified evaluation of audio deepfake detection | Unified comparison on ASVspoof 2021, ADD 2023, In-the-Wild; identifies **poor generalization to unknown attacks**, lack | ABSTRACT_ONLY | "Detection is losing" for audio; do not persist voice-authenticity verdicts as facts |
| A3-28 | arXiv:2408.08739 | ASVspoof 5: Crowdsourced Speech Data, Deepfakes, and Adversarial Attacks at Scale — Wang, Delgado, Tak, Jung, Shim, Todisco, et al. — | Large-scale spoofing/deepfake challenge: crowdsourced bona fide + attacks incl. adversarial for the first time | New database (many speakers, diverse conditions, adversarial attacks); **"attacks significantly compromise the baseline | ABSTRACT_ONLY | Speaker-verification/anti-spoofing error rates are condition-dependent single-digit-to-20%+ — nowhere near identity-authority grad |
| A3-29 | ExifTool documentation (exiftool.org, v13.59, May 2026) | ExifTool by Phil Harvey — P. Harvey — current | EXIF/GPS/IPTC/XMP/ID3/maker-notes **read, write, edit, delete** | The reference tool's core feature list *is* the spoofing proof: writing EXIF/GPS/XMP/ID3 and **deleting metadata in grou | DOCUMENTATION_AUDITED | EXIF (device, timestamps, GPS) is **source-asserted and trivially forgeable/strippable** — store it as provenance hints with expli |
| A3-30 | NIST SP 800-86, Guide to Integrating Forensic Techniques into Incident | Kent, Chevalier, Grance, Dang — Aug 2006, NIST — Digital forensics framework (files, OS, network, a | Formalizes evidence collection/examination/analysis/reporting with data-source taxonomy; positions forensics as an IT process requiring **le | Explicitly disclaims being an all-inclusive legal procedure | DOCUMENTATION_AUDITED | The forensic-standard backing for "metadata = source-asserted evidence, never verified truth": evidence value comes from process ( |
| A3-31 | GDPR Art. 9 (Regulation (EU) 2016/679, gdpr-info.eu) | Processing of special categories of personal data — EU — 2016/2018, current | Regulatory text | Prohibits processing of data revealing racial/ethnic origin, political opinions, religion, trade-union membership, **gen | DOCUMENTATION_AUDITED | The regulatory line runs through **purpose**: biometric *similarity for the user's own workflow grouping* is arguably outside "uni |
| A3-32 | NIST IR 8280, Face Recognition Vendor Test Part 3: Demographic Effects | Grother, Ngan, Hanaoka — 2019, NIST (doi:10.6028/NIST.IR.8280) — Face recognition accuracy across demographic group | Quantifies **differential error rates across sex, age, race/country-of-birth groups** in both 1:1 and 1:N systems; motivation section addres | 2019 algorithm generation; lab-style mugshot/visa datasets (3 of 4 datasets standards-compliant) | ABSTRACT_ONLY | Automated face recognition has measured demographic-differential error — reinforces that even *operational* face matching inside R |
| A3-33 | arXiv:2005.07143 | ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification — Desplanques, Thienpondt, Demuynck — 2020, INTERSPE | Speaker-embedding architecture (voiceprints) | Significantly outperforms prior SOTA on VoxCeleb test sets / VoxSRC 2019 — the reference point for the speaker-embedding | ABSTRACT_ONLY | Speaker embeddings are excellent *similarity keys* (diarization, "same voice?" hints) and terrible *identity authorities*; matches |
| A3-34 | doi:10.1037/pspa0000098 (Crossref-verified; not on arXiv) | Deep neural networks are more accurate than humans at detecting sexual orientation from facial images — Wang, Kosinski — 2018, JPSP 114(2), 246–257 | Face → sensitive-attribute inference | Classifiers from 35,326 facial images: 81% (men) / 74% (women) accuracy from a single image, **91% / 83% from five image | ABSTRACT_ONLY | The canonical demonstration that faces leak special-category traits at high accuracy → Rune must not run attribute-inference on st |
| A3-35 | arXiv:2603.20301 | Voice Privacy from an Attribute-based Perspective — Rahman, Larson, Tejedor-Garcia — 2026, Interspeech | Voice anonymization evaluated by *inferred attribute* leakage | Current voice-privacy benchmarks under-measure risk: measuring protection in attribute space shows **inferred attributes | ABSTRACT_ONLY | Even *processed/anonymized* audio keeps leaking traits — supports the "don't persist inferred traits" rule for voice-derived entri |
| A3-36 | FISWG (Facial Identification Scientific Working Group), fiswg.org | Consensus standards for image-based facial comparison; incl. FRT Implementation Guidelines v1.0 and Face Recognition Systems Operational Ass — FISWG — 2008–2026 | Forensic/operational facial identification standards body (50+ LE/intelligence/industry members) | Maintains separate disciplines: **facial identification (examiner-led comparison to standards)** vs **automated facial r | DOCUMENTATION_AUDITED | The institutional precedent for the **workflow-match vs identity-authority split**: even law enforcement separates automated match |
| A3-37 | arXiv:2209.01292 | Are Attribute Inference Attacks Just Imputation? — Jayaraman, Evans — 2022, ACM CCS 2022 | Attribute-inference attack methodology (incl. "sensitive value inference") | (1) Black-box attribute-inference attacks **rarely learn anything that cannot be learned from distribution knowledge alo | ABSTRACT_ONLY | Two-sided lesson: (a) don't over-credit what an attacker learns *from Rune's stored inferences alone*; (b) but any party with deep |
| A3-38 | arXiv:2609.12448 | GraphProfiler: Source-Linked Sensitive Attribute Inference via Personal Knowledge Graphs — Khan, He, Ma, Wachowicz, Naghizade — 2026, EMNLP 2 | LLM-based profiler aggregating cues across a user's content (age, income, occupation, etc.) | LLM profilers reach **86.7% attack success on SynthPAI (8 attributes)** and 84.6% on PANDORA **by aggregating indirect c | ABSTRACT_ONLY | Directly answers the arm question: memory is an aggregation amplifier; inferred sensitive traits become MORE harmful over time as |
| A3-39 | SQLite Limits (sqlite.org/limits.html, updated 2026) | Implementation Limits For SQLite — SQLite project — current | BLOB/DB size limits | Max string/BLOB = **SQLITE_MAX_LENGTH, default 1,000,000,000 bytes** (hard ceiling 2³¹−1 ≈ 2.1 GB; some built-ins like h | DOCUMENTATION_AUDITED | Screenshots (KB–MB) fit BLOBs fine; documents/audio fine under 1 GB; full-size video/long audio does NOT (and would bloat WAL/vacu |
| A3-40 | arXiv:2212.08035 | Hamming Distributions of Popular Perceptual Hashing Techniques — McKeown, Buchanan — 2022, DFRWS EU 2023 | Million-image evaluation of PDQ, NeuralHash, pHash under content-preserving transforms | Measures Hamming-distance distributions between unrelated images AND image variants for the three deployed hash families | ABSTRACT_ONLY | The perceptual-hash **authority** the arm asked for: treat pHash distance as a probabilistic similarity signal with known failure |
| A3-41 | arXiv:2111.06628 | Learning to Break Deep Perceptual Hashing: The Use Case NeuralHash — Struppek, Hintersdorf, Neider, Kersting — 2021, AC | Adversarial attack on deployed deep perceptual hashing (Apple NeuralHash) | Gradient-free transforms AND simple perturbations force/prevent hash collisions — enabling **hiding abusive content AND | ABSTRACT_ONLY | Hard evidence that perceptual hashes are gameable → never a deletion authority or sole enforcement mechanism; adversarial collisio |
| A3-42 | arXiv:2212.04107 | Re-purposing Perceptual Hashing based Client Side Scanning for Physical Surveillance — Hooda, Labunets, Kohno, Fernandes — 2022 | Poisoning attack on perceptual-hash matching systems | Poisoning 5% of the hash database achieves **>40% physical-surveillance rate** on target locations; stronger detection r | ABSTRACT_ONLY | Second independent result that hash-matching infrastructure is attackable → at Rune scale this is theoretical, but it kills any fu |
| A3-43 | arXiv:1912.07745 | PDQ & TMK+PDQF — A Test Drive of Facebook's Perceptual Hashing Algorithms — Dalins, Wilson, Boudry — 2019, Digital Investigati | PDQ image + TMK+PDQF video hashes on real investigation data (law-enforcement) | Open-sourced PDQ/TMK+PDQF perform well detecting commonly encountered transformations on real-world case data; reference | ABSTRACT_ONLY | Documents the *legitimate* production use: abuse/near-dup **triage inside a human-reviewed investigation workflow** — matching how |
| A3-44 | Apple Photos User Guide — "Remove duplicates" (support.apple.com/guide | Remove duplicate photos and videos — Apple — current (macOS 12→26) | Consumer photo-library dedupe practice | Photos detects duplicate photos/videos and presents them for **user-confirmed merging/deletion** — duplicates surfaced a | DOCUMENTATION_AUDITED | The shipped-consumer pattern for byte-identical vs perceptually-similar vs same-event: byte-level dedupe (fast hashes) automated; |
| A3-45 | [OI] Embeddings documentation (platform.openai.com/docs/guides/embeddi | Vector embeddings — [OI] — current | Embedding-model versioning, dimensions, distance norms | Embeddings are **model-specific artifacts** with named versions (ada-002 → text-embedding-3-small/large) and differing M | DOCUMENTATION_AUDITED | Confirms the industry framing: an index is bound to a model version; changing models ⇒ full re-embed. Rune's unversioned embedding |
| A3-1 | arXiv:2305.10355 | Evaluating Object Hallucination in Large Vision-Language Models (POPE) — Li, Du, Zhou, Wang, Zhao, Wen — 2023, EMNLP 2023 | Image captioning / VQA hallucination measurement | First systematic object-hallucination study of LVLMs. (1) CHAIR: LLaVA CHAIR_S **32.7** vs small VLPM OSCAR-base **13.0** — instruction-tuned LVLMs hallucinate MORE than smaller models; LLM-generated visual instruction data is a root cause (models hallucinate objects frequent or co-occurring in the instruction corpus: HR_A@10 ≈ 0.5, HR_C@10 ≈ 0.6 of hallucinated objects are top-10 frequent/co-occurring). (2) POPE converts evaluation to binary Yes/No polling with Random/Popular/Adversarial negative sampling; more stable than CHAIR (F1 std **±0.78** vs CHAIR_I **±3.22** across 4 prompt templates). (3) Three of five models (LLaVA, MultiModal-GPT, mPLUG-Owl) answer "Yes" ~**96–100%** of the time — F1 66.7–68.7 (Random) degrading to ~66–67 (Adversarial); InstructBLIP best (F1 89.3 → 78.5 Adversarial). Yes-response rate drops adversarially (InstructBLIP 55%→69% Yes) showing benchmark sensitivity | (a) CHAIR scores **double** between two semantically equivalent prompts (LLaVA 32.7→62.7) — caption-based hallucination measurement is instruction-unstable; (b) hallucination degree is **not** consistent with VQA performance (MiniGPT-4 better POPE F1 than LLaVA but much worse VQA) — fixing hallucination ≠ fixing capability; (c) SEEM-based (auto-segmented) POPE is harder than annotation-based (F1 71.4 vs 62.7) — measurement itself is pipeline-sensitive | Authors list: only object-level (not attributes/counts/positions); small image sample (500 imgs × 6 questions); matching-based Yes/No parsing can misfire; small model set, no closed models at time | FULL_PAPER | **Anchor for hallucination measurement.** If Rune ever captions/screenshots into memory, expect language-prior-driven false objects at high rates in unmitigated models; a polling-style verification (binary probe of stored claims against the image) is the validated cheap check pattern; never trust caption-derived entities without a probe |
| A3-2 | arXiv:1809.02156 | Object Hallucination in Image Captioning — Rohrbach, Hendricks, Burns, Darrell, Saenko — 2018, EMNLP 2018 | Image captioning hallucination metric (CHAIR origin) | Introduces CHAIR (CHAIR_I instance-level, CHAIR_S sentence-level) using veridical ground-truth labels on MSCOCO; finds models scoring best on standard sentence metrics do **not** have lower hallucination; hallucination is largely **language-prior driven** (co-occurrence statistics), not visual misclassification | Best-bleu-scored models hallucinate more — standard metrics reward hallucination | Pre-LLM models (show-and-tell era); object-level only | ABSTRACT_ONLY | Metric lineage confirmed; the "language prior beats perception" mechanism is the durable finding that persists into LVLMs (per POPE's HR@k analysis) |
| A3-3 | arXiv:2404.18930 | Hallucination of Multimodal Large Language Models: A Survey — Bai, Wang, Xiao, He, Han, Zhang, Shou — 2024/2025 | Survey of MLLM hallucination causes/benchmarks/mitigations | Systematizes hallucination in MLLMs: causes span training data, vision encoder, alignment module; catalogues benchmarks and mitigation families (instruction tuning, RLHF, decoding interventions, post-hoc correction) | Notes detection/mitigation remains open; benchmarks disagree | Survey; no new measurements | ABSTRACT_ONLY | Map of the mitigation families; confirms 2024–26 the field treats hallucination as unsolved-but-measurable, i.e., Rune must design for residual hallucination, not zero |
| A3-4 | arXiv:2309.14525 | Aligning Large Multimodal Models with Factually Augmented RLHF (LLaVA-RLHF) — Sun, Shen, Cao, Liu, et al. — 2023 | Hallucination mitigation via RLHF + factual reward augmentation | RLHF with a factually-augmented reward (captions, ground-truth options) reduces hallucination and reward hacking; +60% on their MMHAL-BENCH over baselines; first RLHF-trained LMM | — | Reward model still learns from human/LLM labels; benchmark self-defined | ABSTRACT_ONLY | Evidence that grounding-informed rewards help; but training-side fixes are unavailable to Rune (it consumes models, doesn't train them) — Rune's mitigation must be at ingestion/retrieval boundaries |
| A3-5 | arXiv:2310.16045 | Woodpecker: Hallucination Correction for Multimodal Large Language Models — Yin, Fu, Zhao, Xu, Wang, Sui, Shen, Li, Sun, Chen — 2023, Science China Information Sciences 2024 | Training-free post-hoc hallucination correction | Five-stage verifier pipeline (concept extraction → question formulation → visual validation → claim generation → correction); **+30.66% / +24.33% POPE accuracy** over MiniGPT-4 / mPLUG-Owl baselines with no retraining | Confirms mitigation without retraining is possible but multi-stage (costly) | Relies on the same model family for self-verification; benchmark = POPE (binary objects) | ABSTRACT_ONLY | The verifier/grounding pattern with measured gains; Rune-scale adaptation: probing claims against the source image at write-time is the evidence-backed cheap version |
| A3-6 | arXiv:2406.18521 | CharXiv: Charting Gaps in Realistic Chart Understanding in Multimodal LLMs — Wang, Xia, He, Chen, Liu, Zhu, Liang, Wu, Liu, Liu, Malladi, Chevalier, Arora, Chen — 2024, Princeton | Chart QA benchmark on 2,323 real arXiv-paper charts | On realistic charts the strongest proprietary model (GPT-4o) reaches only **47.1%** on reasoning questions and strongest open model 29.2% vs **human 80.5%**; a light stress-test (slightly different charts/questions) degrades performance by up to **34.5%** — template benchmarks were over-optimistic | (a) Open models that *appear* to beat proprietary ones on template benchmarks collapse under stress; (b) all models far below humans on genuine chart reasoning | Descriptive vs reasoning question split; 2024 model set | ABSTRACT_ONLY | Chart misreading is severe at realistic difficulty. If Rune indexes charts/screenshots of dashboards, chart-derived numeric claims are the highest-risk memory content; prefer storing the underlying data/series text over chart descriptions |
| A3-7 | arXiv:2203.10244 | ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning — Masry, Long, Tan, Joty, Hoque — 2022, ACL Findings | Chart QA (human + generated questions, 9.6k human questions) | Established that chart QA needs visual AND logical reasoning; 2022-era models showed a large gap on complex reasoning questions | Baselines weak on multi-step arithmetic over chart features | Pre-VLM baselines | ABSTRACT_ONLY | Root evidence for the chart-claims-are-risky rule; superseded in difficulty by CharXiv |
| A3-8 | arXiv:2302.12173 | Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection — Greshake, Abdelnabi, Mishra, Endres, Holz, Fritz — 2023, Saarland | Indirect prompt injection (text carriers) in LLM-integrated apps | Injection via *retrieved/processed data* (web pages, documents) demonstrated against Bing/GPT-4-powered systems; taxonomy: data theft, worming, ecosystem contamination; **"processing retrieved prompts can act as arbitrary code execution"** — retrieved content and instructions are conflated by design | Notes effective mitigations were lacking at publication | Pre-multimodal carriers (text); demos partly synthetic | ABSTRACT_ONLY | The foundational framing: anything Rune reads is code in the instruction channel. Every later multimodal carrier paper instantiates this |
| A3-9 | arXiv:2307.10490 | Abusing Images and Sounds for Indirect Instruction Injection in Multi-Modal LLMs — Bagdasaryan, Hsieh, Nassi, Shmatikov — 2023, Cornell | Adversarial perturbation in images AND audio that steers MLLM output | Perturbations blended into images/audio make unmodified LLaVA/PandaGPT output attacker-chosen text and steer subsequent dialog when the user merely asks about the image/audio — the user prompt stays benign | — | PoC on LLaVA/PandaGPT; perturbation-style (not typographic); transferability limited | ABSTRACT_ONLY | Proof that the perceptual channel itself is an injection vector even without readable text — OCR-based sanitization alone cannot catch it |
| A3-10 | arXiv:2309.00236 | Image Hijacks: Adversarial Images can Control Generative Models at Runtime — Bailey, Ong, Russell, Emmons — 2023/2024, UC Berkeley | Optimized image "hijacks" controlling VLM behavior | Behaviour/Prompt-Matching algorithms craft images that force VLMs to (1) emit attacker outputs, (2) **leak context-window information**, (3) **override safety training**, (4) believe false statements — all with **>80% success** on LLaVA from small perturbations | — | Optimized per model (white-box-ish); LLaVA-era | ABSTRACT_ONLY | The context-exfiltration + false-belief attack classes map directly onto memory poisoning: an adversarial screenshot can implant false statements or exfiltrate private memory content |
| A3-11 | arXiv:2306.13213 | Visual Adversarial Examples Jailbreak Aligned Large Language Models — Qi, Huang, Panda, Henderson, Wang, Mittal — 2023, Princeton | Visual adversarial examples vs aligned LLMs | Vision input is a **weak link**: a single optimized image can *universally* jailbreak an aligned vision-integrated LLM into harmful content beyond its optimization corpus — adversarial objectives transfer from misclassification to full behavior control | — | Requires gradient access to vision encoder; 2023 models | ABSTRACT_ONLY | Even where Rune's stack reads no text, the image encoder is the bypass; model-internal defenses cannot be assumed |
| A3-12 | arXiv:2402.19150 | Unveiling Typographic Deceptions: Insights of the Typographic Vulnerability in Large Vision-Language Models — Cheng, Xiao, Gu, Yang, Duan, Zhang, Cao, Xu, Xu — 2024, ECCV 2024 | Typographic (embedded-text) attacks on commercial + open LVLMs | Typographic attacks (text rendered into images) are **widespread across commercial and open LVLMs**; largest typographic dataset to date; mechanism analysis; a mitigation reduces performance degradation from **42.07% to 13.90%** | Even the best mitigation leaves ~14% degradation — no clean fix shown | Mitigation is training/attention-based, model-specific | ABSTRACT_ONLY | Text-in-images is *transcribed and obeyed* by current models as a rule, not an edge case. Rune's screenshots regularly contain text → this is the default attack surface |
| A3-13 | arXiv:2504.04893 | SCAM: A Real-World Typographic Robustness Evaluation for Multimodal Foundation Models — Westerhoff, Purelku, Hackstein, Loos, Pinetzki, Rodner, Hufe — 2025, CVPR 2025 Workshop / DMLR 2026 | Real-world typographic attacks (1,162 handwritten/real images, hundreds of categories) | Typographic attacks **remain effective against SOTA LVLMs**, especially those with vulnerable vision encoders; larger LLM backbones reduce (not remove) vulnerability; synthetic attacks ≈ real handwritten attacks | Vulnerability persists in 2025 SOTA; architecture of the vision encoder is a main susceptibility factor | Object-recognition task focus | ABSTRACT_ONLY | Independent 2025 confirmation with *real-world* (not synthetic) images: the failure mode survives model generations — design for it, don't wait for model fixes |
| A3-14 | arXiv:2609.04533 | Repeat-After-Me: Black-Box Adaptive Visual Prompt Injection — Chen, Tsai, Evtimov, Chaudhuri, Popa, Wagner, Zharmagambetov, (Meta/Berkeley) — 2026 | Black-box ADAPTIVE visual prompt injection against frontier VLMs | Adaptive black-box image injection achieves ASR **>82%** on open-weight frontier models and **47%** on commercial frontier VLMs (GPT-5.5-class) for harmful behavior/PII exfiltration/malicious tool calls, with benign user prompts unrelated to the injected task; transfers across commercial VLMs; demonstrated end-to-end on a real agent: injected image overwrites TOOLS.md → future RCE + secret exfiltration | — | 2026 frontier models; attack is adaptive (the hard case) | ABSTRACT_ONLY | **The Atlas "defenses fail under adaptive attack" lesson, in the visual domain.** Note the TOOLS.md/Memory-file overwrite pattern: exactly Rune's threat. Enforcement cannot live in the model |
| A3-15 | arXiv:2609.09404 | An Experimental Evaluation of Multimodal Prompt Injection Attacks on Agentic AI Frameworks (MMPIBench) — Nguyen, Husain — 2026 | Benchmark of six visual carriers (OCR text, overlays, **EXIF metadata**, **QR codes**, fake interfaces, hybrids) across 6 agent frameworks × 5 foundation models, 720 runs; audio extension | Attacks **attempted in 12.8%** of runs but **completed in ~1%** for vision — the gap closes at the *planning* step (model reads and declines). Model >> framework for safety (one model never attempts; two attempt in 23.6%). **Audio channel is far weaker defended: attack completes in 49% of cells, 75% for one model** — completion-rate reporting alone understates exposure | (a) ~99% of visual injections are caught at planning by current models — a rare *positive* result, but only for non-adaptive fixed attack sets; (b) audio bypasses that defense line almost entirely | Fixed (non-adaptive) attack set; the authors themselves warn completion-understates-exposure | ABSTRACT_ONLY | Carrier taxonomy directly relevant to Rune (screenshots carry OCR text; EXIF and QR are real carriers); the vision-vs-audio defense asymmetry matters if Rune ingests audio. Positive planning-layer result is conditional on non-adaptive attacks — pair with Repeat-After-Me |
| A3-16 | arXiv:2403.02691 | InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated LLM Agents — Zhan, Liang, Ying, Kang — 2024, ACL Findings | IPI benchmark: 1,054 cases, 17 user tools, 62 attacker tools, 30 LLM agents | ReAct-prompted GPT-4 executes injected instructions **24%** of the time (direct harm) / more for data exfiltration; a "hacking prompt" reinforcement nearly **doubles** ASR; agents broadly vulnerable | — | Text-domain; pre-2024 models | ABSTRACT_ONLY | Tool-integrated agents (Rune's shape) are measurably injectable even in the text-only case; establishes the exfiltration-vs-direct-harm taxonomy Rune's threat model needs |
| A3-17 | arXiv:2403.14720 | Defending Against Indirect Prompt Injection Attacks With Spotlighting — Hines, Lopez, Hall, Zarfati, Zunger, Kiciman (Microsoft) — 2024 | Prompt-engineering defense: delimiting/datamarking/encoding of untrusted content | Spotlighting reduces IPI ASR from **>50% to <2%** with minimal task-efficiency cost, by giving a continuous provenance signal for untrusted spans | — | GPT-family, fixed (non-adaptive) attack evaluation; authors note limits | ABSTRACT_ONLY | Cheap, evidence-backed input-hygiene defense Rune can adopt at ingestion (mark provenance of every untrusted span) — but its evaluation is non-adaptive, so treat as risk reduction, not guarantee |
| A3-18 | arXiv:2402.06363 | StruQ: Defending Against Prompt Injection with Structured Queries — Chen, Piet, Sitawarin, Wagner — 2024, USENIX Security 2025 | Two-channel (prompt/data) structured queries + tuned LLM | Structured separation of instructions from data significantly improves resistance with little/no utility loss | — | Requires a specially fine-tuned model — unavailable to Rune unless it controls its model | ABSTRACT_ONLY | The model-side structural defense; informative as the "right" design if Rune ever trains/finetunes, but not deployable as a wrapper |
| A3-19 | arXiv:2503.18813 | Defeating Prompt Injections by Design (CaMeL) — Debenedetti, Shumailov, Fan, Hayes, Carlini, Fabian, Kern, Shi, Terzis, Tramèr (Google DeepMind/ETH) — 2025 | System-level defense: control/data-flow extraction + capabilities enforced at tool calls | CaMeL creates a protective system layer around a *susceptible* LLM: untrusted data can never influence program flow; policies enforced when tools are called; **solves 77% of tasks with provable security vs 84% undefended** on AgentDojo (under adaptive DRI evaluation) | The ~7% task-completion cost is the explicit price of provable security | Requires restructured agent architecture (separate planner/querier) | ABSTRACT_ONLY | **The defense that survives adaptive evaluation** — and it works by moving enforcement OUTSIDE the model (capability checks at tool boundaries), exactly the Atlas lesson. Rune's memory-write and shell/file tools are the boundary where such policies belong |
| A3-20 | arXiv:2503.06254 | Poisoned-MRAG: Knowledge Poisoning Attacks to Multimodal Retrieval Augmented Generation — Liu, Yuan, Tie, Shi, Zhou, Sun, Gong — 2025 | Poisoned image-text pairs injected into multimodal KB | First multimodal RAG knowledge-poisoning attack: **up to 98% ASR with just 5 poisoned pairs** in a 481,782-pair DB (InfoSeek); dirty-label and clean-label cross-modal strategies; 4 defenses (paraphrasing, duplicate removal, structure-driven, purification) show **limited effectiveness and trade-offs** | Explicit negative: all evaluated defenses are inadequate | Optimization-based; assumes KB write access (realistic for user-contributed corpora) | ABSTRACT_ONLY | If Rune's memory accepts user/agent-contributed media, 5 poisoned items in a ~10⁵ corpus is enough. Defenses inside the pipeline fail; provenance/quarantine at write time is the implication |
| A3-21 | arXiv:2502.17832 | MM-PoisonRAG: Disrupting Multimodal RAG with Local and Global Poisoning Attacks — Ha, Zhan, Kim, Bralios, Sanniboina, Peng, Chang, Kang, Ji — 2025/2026 | Localized (query-specific) vs globalized (untargeted) multimodal poisoning | Localized attack: **56% ASR under restricted attacker access**; transfers across 4 retrievers without re-optimization. Globalized: **one poisoned content collapses generation to 0% accuracy**. Both bypass existing defenses | (a) A single poisoned asset can globally destroy retrieval quality — availability attack costs ~nothing; (b) existing defenses bypassed | Benchmark-scale; retriever-transfer shown | ABSTRACT_ONLY | The one-poison-collapses-everything result argues for write-time quarantine + integrity checks on the media index; a corrupted media lane can silently poison all answers |
| A3-22 | arXiv:2505.18543 | Benchmarking Poisoning Attacks against Retrieval-Augmented Generation — Zhang, Xin, Li, Zhang, Fang, Liu, Nie, Liu — 2025 | First comprehensive poisoning benchmark: 15 datasets, 13 attacks, 7 defenses, across RAG architectures | Attack effectiveness drops on adversarially expanded datasets; sequential/branching/conditional/loop RAG, multi-turn RAG, **multimodal RAG and agent RAG all remain susceptible**; "current defense techniques fail to provide robust protection" | Defense-side negative result at scale across architectures | Text-dominant datasets | ABSTRACT_ONLY | Cross-architecture confirmation that in-pipeline defenses fail; supports putting trust decisions in the tool boundary + provenance ledger, not in retrieval filters |
| A3-23 | arXiv:2106.09898 | Bad Characters: Imperceptible NLP Attacks — Boucher, Shumailov, Anderson, Papernot — 2021, IEEE S&P 2022 | Invisible characters, homoglyphs, reordering, deletion in text pipelines | A single imperceptible encoding injection degrades or breaks NLP systems (translation, search, moderation) **including deployed commercial systems (Microsoft, Google)**; three injections break most models; attacks are model-agnostic black-box | Documents that deployed systems at the time did **no input sanitization** — the trust-boundary gap is in the pipeline, not the model | Text carriers; not image OCR per se, but identical mechanism applies to OCR output containing invisible characters | ABSTRACT_ONLY | **Directly relevant to taint-through-transformation**: OCR of a hostile image can output invisible/bidi characters that then flow into Rune's text memory as "extracted text". Text extracted from media needs the same sanitization as raw internet text |
| A3-24 | arXiv:1708.09537 | DolphinAttack: Inaudible Voice Commands — Zhang, Yan, Ji, Zhang, Zhang, Xu — 2017, ACM CCS 2017 | Ultrasonic (>20 kHz) modulated voice commands exploiting microphone nonlinearity | Inaudible commands successfully control Siri, Google Now, Alexa, Cortana, automotive navigation (FaceTime call, airplane mode, navigation manipulation); SVM-based detection of ultrasonic carrier demonstrated as feasible defense | Requires proximity (attack range meters-class), specialized transducer, and alignment with the mic — demonstrated in lab-controlled conditions; practical constraints documented | Hardware-dependent; 2017 ASR stack (but the acoustic physics is durable) | ABSTRACT_ONLY | Credible but *conditions-constrained*: real risk only if Rune records audio in physical spaces with hostile nearby hardware; mitigation = band-energy anomaly detection at capture, not at ASR |
| A3-25 | C2PA Technical Specification 2.2 (c2pa.org/specifications/specifications/2.2/specs/C2PA_Specification.html) | Content Credentials: C2PA Technical Specification — C2PA — v2.2 May 2025 (2.4 current) | Media provenance standard: signed manifests (assertions + claim + signature), hard/soft content bindings, trust lists, validation states | (1) Provenance = **signed assertions bound by cryptographic hash ("hard binding")** to exact bytes; (2) §9: **"a soft binding shall not be used as a hard binding"** — perceptual fingerprints/watermarks are explicitly barred from identity/integrity roles; (3) §1.2/Principles: spec "SHOULD NOT provide value judgments… merely whether the assertions can be validated as associated with the underlying asset" — provenance asserts *source*, never *truth*; (4) §14/15: trust derives from signer identity + X.509 trust list + revocation + timestamp, with defined validation states (well-formed/valid/trusted); (5) §17: dedicated threat-modeling process + Harms Modelling; (6) covers PDF/BMFF/JPEG/PNG/ZIP/font embedding; `digitalSourceType` distinguishes AI-generated content | Soft bindings (fingerprints) fail-safe: manifest recovered via soft binding is expected to report validation failures because its hard binding won't match — similarity evidence is never integrity evidence | Ecosystem-dependent: trust requires the signer to be on a trust list and honest; provenance is opt-in; manifest strippable (soft-binding recovery exists for that case) | DOCUMENTATION_AUDITED | **The most mature normative statement of this arm's central verdicts**: (a) metadata = source-asserted, cryptographically bound, never truth; (b) perceptual similarity ≠ identity/deletion authority. Rune's media store should copy this two-tier design: cryptographic hash = identity; perceptual similarity = discovery hint only |
| A3-26 | arXiv:2302.10174 | Towards Universal Fake Image Detectors that Generalize Across Generative Models — Ojha, Li, Lee — 2023, CVPR 2023 | AI-image detector reliability across generators | Trained real-vs-fake detectors fail on unseen generators: the "real" class becomes a **sink class** absorbing everything unfamiliar — asymmetrically tuned to fake patterns, so new fakes are classified real; feature-space nearest-neighbor/probing (CLIP features) generalizes better (+15.07 mAP / +25.90% acc on unseen diffusion/AR models) but is still far from solved | Detector training paradigm itself is the failure; generalization to unseen generators is structurally unreliable | 2023 generator zoo; classification framing | ABSTRACT_ONLY | Measured unreliability of AI-image detectors → Rune must NOT store "this image is AI/synthetic" detector verdicts as memory facts; C2PA-style source assertions (when present) are the only signed evidence |
| A3-27 | arXiv:2308.14970 | Audio Deepfake Detection: A Survey — Yi, Wang, Tao, Zhang, Zhang, Zhao — 2023 | Survey + unified evaluation of audio deepfake detection | Unified comparison on ASVspoof 2021, ADD 2023, In-the-Wild; identifies **poor generalization to unknown attacks**, lack of in-the-wild data, interpretability as the unsolved core | Detection on unseen spoofing attacks degrades severely — the field's own summary | 2023 cut-off | ABSTRACT_ONLY | "Detection is losing" for audio; do not persist voice-authenticity verdicts as facts |
| A3-28 | arXiv:2408.08739 | ASVspoof 5: Crowdsourced Speech Data, Deepfakes, and Adversarial Attacks at Scale — Wang, Delgado, Tak, Jung, Shim, Todisco, et al. — 2024, ASVspoof 5 Workshop (Interspeech satellite) | Large-scale spoofing/deepfake challenge: crowdsourced bona fide + attacks incl. adversarial for the first time | New database (many speakers, diverse conditions, adversarial attacks); **"attacks significantly compromise the baseline systems"**; submissions improve but the adversarial/hidden conditions define the gap; SASV and stand-alone CM tracks with new metrics | Baseline countermeasures compromised by attack variety — measured, at scale | Challenge paper; specific EERs live in per-system papers (e.g., XMU closed-condition EER 20.45% vs open 9.36%; SZU-AFS 4.04% — condition-dependent) | ABSTRACT_ONLY | Speaker-verification/anti-spoofing error rates are condition-dependent single-digit-to-20%+ — nowhere near identity-authority grade. Voice similarity is a workflow signal only |
| A3-29 | ExifTool documentation (exiftool.org, v13.59, May 2026) | ExifTool by Phil Harvey — P. Harvey — current | EXIF/GPS/IPTC/XMP/ID3/maker-notes **read, write, edit, delete** | The reference tool's core feature list *is* the spoofing proof: writing EXIF/GPS/XMP/ID3 and **deleting metadata in groups or altogether** are one-line operations; also reads C2PA JUMBF and can delete it (`-JUMBF:all=`) on some formats; computes MD5/SHA-256/512 over image data for validation | — | Tool docs, not a study; but it is the de-facto standard both forensics and attackers use | DOCUMENTATION_AUDITED | EXIF (device, timestamps, GPS) is **source-asserted and trivially forgeable/strippable** — store it as provenance hints with explicit "unverified" status; cryptographic hash of media bytes is the only self-verifiable field |
| A3-30 | NIST SP 800-86, Guide to Integrating Forensic Techniques int | NIST SP 800-86, Guide to Integrating Forensic Techniques into Incident Response | NIST SP 800-86, Guide to Integrating Forensic Techniques into Incident Response — Kent, Chevalier, Grance, Dang — Aug 2006, NIST | Formalizes evidence collection/examination/analysis/reporting with data-source taxonomy; positions forensics as an IT process requiring **legal-counsel review for laws/regulations** — i.e., metadata and file artifacts are evidence *sources* whose weight depends on chain-of-custody and process, not self-authenticating truth | Explicitly disclaims being an all-inclusive legal procedure | 2006 (predates modern media formats; principles still cited) | DOCUMENTATION_AUDITED (landing page + abstract; full PDF not read) | The forensic-standard backing for "metadata = source-asserted evidence, never verified truth": evidence value comes from process (hashes, custody, timestamps), not from the metadata itself |
| A3-31 | GDPR Art. 9 (Regulation (EU) 2016/679, gdpr-info.eu) | Processing of special categories of personal data — EU — 2016/2018, current | Regulatory text | Prohibits processing of data revealing racial/ethnic origin, political opinions, religion, trade-union membership, **genetic data, biometric data "for the purpose of uniquely identifying a natural person," health data, sex life/sexual orientation** — unless an exception applies (explicit consent, vital interests, legal claims, substantial public interest, etc.); Art. 9(4): Member States may add conditions for biometric/health data | — | Legal text read as written; application depends on interpretation (e.g., what counts as "uniquely identifying") | DOCUMENTATION_AUDITED | The regulatory line runs through **purpose**: biometric *similarity for the user's own workflow grouping* is arguably outside "uniquely identifying", while asserting *identity* ("this is Alice") is squarely in the prohibited category absent consent. This is the legal mirror of the workflow-match vs identity-authority distinction |
| A3-32 | NIST IR 8280, Face Recognition Vendor Test Part 3: Demograph | NIST IR 8280, Face Recognition Vendor Test Part 3: Demographic Effects Grother, Ngan, Hanaoka — 2019, NIST (doi:10.6028/NIST.IR.8280) | Grother, Ngan, Hanaoka — 2019, NIST (doi:10.6028/NIST.IR.8280) — Face recognition accuracy across demographic groups (1:1 verification + 1:N search; 18.27M images of 8.49M people; 189 algorithms) | Quantifies **differential error rates across sex, age, race/country-of-birth groups** in both 1:1 and 1:N systems; motivation section addresses policing/bias impacts; recommends mitigation research; intended audience includes policymakers and system integrators "concerned with risks implied by differentials" | NOT_RECORDED_IN_ARM (negatives not stated in the abstract read) | 2019 algorithm generation; lab-style mugshot/visa datasets (3 of 4 datasets standards-compliant) | ABSTRACT_ONLY (via OpenAlex metadata + abstract; full PDF fetch returned raw bytes) | Automated face recognition has measured demographic-differential error — reinforces that even *operational* face matching inside Rune (e.g., "photos of the same person") must be presented as cluster hypotheses, never adjudicated identity |
| A3-33 | arXiv:2005.07143 | ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification — Desplanques, Thienpondt, Demuynck — 2020, INTERSPEECH | Speaker-embedding architecture (voiceprints) | Significantly outperforms prior SOTA on VoxCeleb test sets / VoxSRC 2019 — the reference point for the speaker-embedding lane (community-reported EERs on VoxCeleb1-O are sub-1% for this class; EER figures not stated in abstract) | — | Benchmark conditions (studio-ish); real-world/channel shifts degrade; EER from abstract not verifiable | ABSTRACT_ONLY | Speaker embeddings are excellent *similarity keys* (diarization, "same voice?" hints) and terrible *identity authorities*; matches ASVspoof 5's finding that spoofing attacks compromise verification systems |
| A3-34 | doi:10.1037/pspa0000098 (Crossref-verified; not on arXiv) | Deep neural networks are more accurate than humans at detecting sexual orientation from facial images — Wang, Kosinski — 2018, JPSP 114(2), 246–257 | Face → sensitive-attribute inference | Classifiers from 35,326 facial images: 81% (men) / 74% (women) accuracy from a single image, **91% / 83% from five images** vs human 61/54%; authors frame it explicitly as a **privacy threat** ("expose a threat to the privacy and safety of gay men and women") | — | Criticized methodologically (grooming/pinback-image confounds); binary orientation framing; 2018 models | ABSTRACT_ONLY | The canonical demonstration that faces leak special-category traits at high accuracy → Rune must not run attribute-inference on stored media by default, and must not persist such inferences |
| A3-35 | arXiv:2603.20301 | Voice Privacy from an Attribute-based Perspective — Rahman, Larson, Tejedor-Garcia — 2026, Interspeech 2026 | Voice anonymization evaluated by *inferred attribute* leakage | Current voice-privacy benchmarks under-measure risk: measuring protection in attribute space shows **inferred attributes still present a risk despite anonymization and despite inference errors** | Anonymization that defeats speaker-identification can still leak attributes | Interspeech-length study | ABSTRACT_ONLY | Even *processed/anonymized* audio keeps leaking traits — supports the "don't persist inferred traits" rule for voice-derived entries too |
| A3-36 | FISWG (Facial Identification Scientific Working Group), fiswg.org | Consensus standards for image-based facial comparison; incl. FRT Implementation Guidelines v1.0 and Face Recognition Systems Operational Assurance: Deployment Testing v1.0 (finalized Nov 2025) — FISWG — 2008–2026 | Forensic/operational facial identification standards body (50+ LE/intelligence/industry members) | Maintains separate disciplines: **facial identification (examiner-led comparison to standards)** vs **automated facial recognition (system operation)** — with distinct documents for implementation and operational assurance/deployment testing; recognition-system output is operational tooling, examination conclusions follow human/forensic process | — | Document bodies themselves not read (paywalled/member docs); scope verified from site + titles | DOCUMENTATION_AUDITED | The institutional precedent for the **workflow-match vs identity-authority split**: even law enforcement separates automated match (operational lead, requires assurance testing) from identification (forensic discipline with standards). Rune should mirror: face/voice similarity = lead/hint; identity assertions = out of scope |
| A3-37 | arXiv:2209.01292 | Are Attribute Inference Attacks Just Imputation? — Jayaraman, Evans — 2022, ACM CCS 2022 | Attribute-inference attack methodology (incl. "sensitive value inference") | (1) Black-box attribute-inference attacks **rarely learn anything that cannot be learned from distribution knowledge alone** — much of the claimed attack power is the dataset prior, not the model; (2) white-box attacks CAN reliably identify records with sensitive values beyond the no-model baseline; (3) DP training and removing vulnerable records do **not** mitigate the residual white-box risk | Black-box attack power ≈ imputation baseline — a key deflationary negative for naive threat models | Model-internal setting (training-data inference), not media-inference per se | ABSTRACT_ONLY | Two-sided lesson: (a) don't over-credit what an attacker learns *from Rune's stored inferences alone*; (b) but any party with deeper access (white-box embeddings/logs) can exceed baselines — so inferred sensitive traits, once persisted, extend the attack surface permanently |
| A3-38 | arXiv:2609.12448 | GraphProfiler: Source-Linked Sensitive Attribute Inference via Personal Knowledge Graphs — Khan, He, Ma, Wachowicz, Naghizade — 2026, EMNLP 2026 | LLM-based profiler aggregating cues across a user's content (age, income, occupation, etc.) | LLM profilers reach **86.7% attack success on SynthPAI (8 attributes)** and 84.6% on PANDORA **by aggregating indirect cues across ordinary posts**, with cited evidence for 98% of predictions; removing the cited posts cuts ASR far more than removing random ones | Confirms *aggregation* is the amplifier — no single entry leaks much, but the store as a whole does | Text content (not images); benchmark corpora | ABSTRACT_ONLY | Directly answers the arm question: memory is an aggregation amplifier; inferred sensitive traits become MORE harmful over time as the store grows — persistence requires explicit justification because the harm is emergent, not per-entry |
| A3-39 | SQLite Limits (sqlite.org/limits.html, updated 2026) | Implementation Limits For SQLite — SQLite project — current | BLOB/DB size limits | Max string/BLOB = **SQLITE_MAX_LENGTH, default 1,000,000,000 bytes** (hard ceiling 2³¹−1 ≈ 2.1 GB; some built-ins like hex() fail earlier); max DB ≈ **281 TB** (page-count × 64 KB pages); docs explicitly recommend *lowering* the limit in security-sensitive apps; row = single BLOB encoding during INSERT/SELECT so SQLITE_MAX_LENGTH also caps row size | — | Docs | DOCUMENTATION_AUDITED | Screenshots (KB–MB) fit BLOBs fine; documents/audio fine under 1 GB; full-size video/long audio does NOT (and would bloat WAL/vacuum) → media-as-files + content-addressed references is the right default; BLOBs only for thumbnails/small assets |
| A3-40 | arXiv:2212.08035 | Hamming Distributions of Popular Perceptual Hashing Techniques — McKeown, Buchanan — 2022, DFRWS EU 2023 | Million-image evaluation of PDQ, NeuralHash, pHash under content-preserving transforms | Measures Hamming-distance distributions between unrelated images AND image variants for the three deployed hash families; documents the collisions/tightness problems of each against seven image-variant classes — "little public evaluation" existed before | Explicitly documents that popular perceptual hashes have poorly characterized error behavior under common transforms | Hash archetypes only; no deep-perceptual-hash models | ABSTRACT_ONLY | The perceptual-hash **authority** the arm asked for: treat pHash distance as a probabilistic similarity signal with known failure modes, never a match/no-match rule |
| A3-41 | arXiv:2111.06628 | Learning to Break Deep Perceptual Hashing: The Use Case NeuralHash — Struppek, Hintersdorf, Neider, Kersting — 2021, ACM FAccT 2022 (Oral) | Adversarial attack on deployed deep perceptual hashing (Apple NeuralHash) | Gradient-free transforms AND simple perturbations force/prevent hash collisions — enabling **hiding abusive content AND framing innocent users**; hash values also leak information about device contents; conclusion: deep perceptual hashing "generally not ready for robust client-side scanning" | The safety-critical deployment of perceptual hashing was defeated by standard image transformations | Deep-hash specific (NeuralHash); classical hashes degrade differently | ABSTRACT_ONLY | Hard evidence that perceptual hashes are gameable → never a deletion authority or sole enforcement mechanism; adversarial collision = censorship/framing vector |
| A3-42 | arXiv:2212.04107 | Re-purposing Perceptual Hashing based Client Side Scanning for Physical Surveillance — Hooda, Labunets, Kohno, Fernandes — 2022 | Poisoning attack on perceptual-hash matching systems | Poisoning 5% of the hash database achieves **>40% physical-surveillance rate** on target locations; stronger detection robustness *increases* surveillance capability (robustness/abuse trade-off) | A defense-robustness improvement makes the abuse channel worse — inherent tension | Client-side scanning context | ABSTRACT_ONLY | Second independent result that hash-matching infrastructure is attackable → at Rune scale this is theoretical, but it kills any future temptation to use hash matching as an automatic enforcement tool |
| A3-43 | arXiv:1912.07745 | PDQ & TMK+PDQF — A Test Drive of Facebook's Perceptual Hashing Algorithms — Dalins, Wilson, Boudry — 2019, Digital Investigation | PDQ image + TMK+PDQF video hashes on real investigation data (law-enforcement) | Open-sourced PDQ/TMK+PDQF perform well detecting commonly encountered transformations on real-world case data; reference implementation for LE integration | — | Law-enforcement triage context; human-in-the-loop investigation assumed (hash = candidate-finding tool) | ABSTRACT_ONLY | Documents the *legitimate* production use: abuse/near-dup **triage inside a human-reviewed investigation workflow** — matching how photo platforms use perceptual hashes (candidate detection → human/secondary verification), not automated deletion |
| A3-44 | Apple Photos User Guide — "Remove duplicates" (support.apple.com/guide/photos/pht5a3157c1d/mac) | Remove duplicate photos and videos — Apple — current (macOS 12→26) | Consumer photo-library dedupe practice | Photos detects duplicate photos/videos and presents them for **user-confirmed merging/deletion** — duplicates surfaced as suggestions with human decision, and separately groups "same-event" content via People/Places/memories rather than treating perceptual similarity as identity | — | Vendor docs do not disclose the hash algorithm (public reporting indicates hash-based + embedding signals) | DOCUMENTATION_AUDITED | The shipped-consumer pattern for byte-identical vs perceptually-similar vs same-event: byte-level dedupe (fast hashes) automated; perceptual similarity surfaced as **review suggestions**; event grouping is semantic, not hash. Confirms "similarity proposes, user disposes" |
| A3-45 | [OI] Embeddings documentation (platform.openai.com/docs/guides/embeddings, fetched 2026) | Vector embeddings — [OI] — current | Embedding-model versioning, dimensions, distance norms | Embeddings are **model-specific artifacts** with named versions (ada-002 → text-embedding-3-small/large) and differing MTEB performance; dimension can be shortened (Matryoshka, arXiv:2205.13147) with re-normalization; model knowledge cutoffs and training changes are version-bound — no cross-model vector compatibility is offered or implied | — | Docs; migration mechanics not covered (that's the gap) | DOCUMENTATION_AUDITED | Confirms the industry framing: an index is bound to a model version; changing models ⇒ full re-embed. Rune's unversioned embeddings violate this — record model+dimensions per index or treat the index as disposable cache |
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
conductor committed