Dynamic Bidirectional Pattern Memory: A Production-Scale Empirical Characterisation of Inference-Time Gating in Clinical NLP

• URL: http://arxiv.org/abs/2607.00870v1
• Date: Wed, 01 Jul 2026 12:34:30 GMT
• Title: Dynamic Bidirectional Pattern Memory: A Production-Scale Empirical Characterisation of Inference-Time Gating in Clinical NLP
• Abstract summary: We study inference-time pattern-memory gating in a production-scale clinical natural language processing (NLP) pipeline.<n>The pipeline pairs a generator (Llama-3.3 70B) proposing extractions with a verifier (MMed-Llama-3.1 70B) accepting or rejecting them.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: We study inference-time pattern-memory gating in a production-scale clinical natural language processing (NLP) pipeline. The pipeline pairs a generator (Llama-3.3 70B) proposing extractions with a verifier (MMed-Llama-3.1 70B) accepting or rejecting them, over 167,034 PMC-Patients narratives, and adds a lightweight memory that learns at deployment which extractions to filter, so the verifier need not re-examine candidates already seen to fail. We report four findings. First, learning filtering rules directly from the verifier's rejections failed at full scale: the relation-extraction filter stayed empty despite 785,797 logged rejections, because they were spread too thinly across too many distinct forms to accumulate. Second, a simpler rule using a fixed clinical ontology produced the same filtering without the verifier, capturing 49,734 ontology-violating relations on a held-out 5,000-patient set. Third, of five versions of the question-answering filter, four failed for distinct, instructive reasons; the fifth succeeded by checking whether a patient's extracted entities support the question asked, and where it applies was 1.84 times likelier to flag an answer the verifier would reject than one it would accept. Fourth, one pattern held across all five: a filter is selective only when it tests the same evidence the verifier weighs, not when it imitates the verifier's output. Together these give a transferable result for any generator-verifier pipeline: the most natural memory design can fail silently at scale, and whether a pre-generation gate is selective is decided before any engineering effort, by whether its signal probes the question the verifier itself answers. Throughout, the system flags suspect extractions rather than deleting them, so every decision stays visible for clinical review. All code and test artefacts are released openly.

Related papers

• From Experiments to Decisions: Reusing Evidence in Autonomous Coding Research [0.0]
  We reconstruct how evidence is reused in a 400-task NeuroGolf campaign.<n>A numerical counterexample exposes an overbroad exclusion.<n>Other episodes distinguish what failed, incomplete and revised programs justify doing next.
  arXiv  Detail & Related papers  (2026-09-10T03:48:31Z)
• SEER: A Self-Grounded Evidence Interface for Controlled Spatial Relation Classification [71.9783246097687]
  SEER hides candidate relations during pair localization, constructs a query-specific view with explicit subject/object roles, and retains the full image and sparse box geometry as complementary evidence.<n>For relation-choice protocols with exact inverse support, an optional refinement swaps the entity roles and changes the forward decision only when exactly one visual state obeys the corresponding inverse relation.<n>The unchanged protocol yields +4.35 to +11.79 on all 2,434 filtered EmbSpatial pair-relation questions across three models.
  arXiv  Detail & Related papers  (2026-08-04T13:16:15Z)
• A multiverse-consensus pipeline for reproducible feature selection in untargeted LC-MS metabolomics [1.452875650827562]
  Untargeted LC-MS metabolomics requires a long chain of preprocessing decisions.<n>We adapt multiverse analysis to untargeted metabolomics feature selection.
  arXiv  Detail & Related papers  (2026-07-19T17:14:17Z)
• A Multi-Analyst LLM Pipeline for Auditable Rule Discovery Across 68 Public Physiological Corpora [0.0]
  Open physiological corpora can support detector design, but they do not directly specify which detector rules should be built for a new contactless monitoring platform.<n>We convert 68 public physiological corpora into an auditable library of candidate rule shapes for prospective validation.<n>The pipeline does not produce a validated clinical detector.
  arXiv  Detail & Related papers  (2026-07-07T20:56:37Z)
• Caught in the Act(ivation): Toward Pre-Output and Multi-Turn Detection of Credential Exfiltration by LLM Agents [0.0]
  We study whether activation probes can detect credential access before output tokens are emitted.<n>Third, we treat multi-turn exfiltration as a cumulative information-flow problem.<n>Results suggest credential-exfiltration defenses should combine pre-output monitoring, canary detection, and temporal leakage accounting.
  arXiv  Detail & Related papers  (2026-06-02T18:53:17Z)
• Internal Representation, Not Clinical Knowledge: Where Apparent LLM Triage Failures Originate [2.581200752140087]
  Patient-voiced clinical-triage benchmarks report high under-triage rates for consumer LLMs for constrained multiple-choice output.<n>We find the same medical features fire on the shared clinical narrative under both formats but go silent at the multiple-choice decision token in all the cases at every model.
  arXiv  Detail & Related papers  (2026-05-28T13:14:17Z)
• When Should an AI Scientist Stop? Verifiable Experiment Steering and Refusal for Autonomous Discovery [0.0]
  CARTOGRAPH is a verification layer for AI scientists.<n>It couples unresolved-subspace experiment steering (select), explicit ambiguity closure (resolve), and residual-based library inadequacy detection (refuse)
  arXiv  Detail & Related papers  (2026-05-26T18:19:16Z)
• Self-Verified Distillation: Your Language Model Is Secretly Its Own Synthetic Data Pipeline [56.53954182896384]
  We propose Self-Verified Distillation, a simple post-training refinement algorithm for large language models.<n>Self-Verified Distillation generates candidate solutions to unlabeled seed questions.<n>It filters them using prompt-based self-verification, and trains on the resulting self-curated dataset.<n>We find that sampling more candidate generations and using a larger verification budget during training data construction produces higher-quality self-curated data.
  arXiv  Detail & Related papers  (2026-05-20T17:26:10Z)
• Rethinking Dense Sequential Chains: Reasoning Language Models Can Extract Answers from Sparse, Order-Shuffling Chain-of-Thoughts [51.84894623128418]
  Modern reasoning language models generate dense, sequential chain-of-thought traces implicitly assuming that every token contributes and that steps must be consumed in order.<n>We challenge both assumptions through a systematic intervention pipeline--removal, masking, shuffling, and noise injection--applied to model-generated reasoning chains.<n>We find that answer extraction operates on a sparse, order-insensitive, and structurally robust informational substrate.
  arXiv  Detail & Related papers  (2026-05-08T06:15:50Z)
• CausalT5K: Diagnosing and Informing Refusal for Trustworthy Causal Reasoning of Skepticism, Sycophancy, Detection-Correction, and Rung Collapse [1.4608214000864057]
  CausalT5K is a diagnostic benchmark of over 5,000 cases across 10 domains.<n>Unlike synthetic benchmarks, CausalT5K embeds causal traps in realistic narratives.<n>Preliminary experiments reveal a Four-Quadrant Control Landscape where static audit policies universally fail.
  arXiv  Detail & Related papers  (2026-02-09T17:36:56Z)
• DiFR: Inference Verification Despite Nondeterminism [5.879581944824945]
  Re-running the same inference process twice often leads to different results due to benign numerical noise.<n>We introduce Token-DiFR, a method for verifying inference outputs by comparing generated tokens against predictions made by a trusted reference implementation conditioned on the same random seed.<n>We additionally introduce Activation-DiFR, a scheme that uses random projections to compress activations into compact fingerprints for subsequent verification.
  arXiv  Detail & Related papers  (2025-11-25T18:44:22Z)
• Unified Unsupervised Anomaly Detection via Matching Cost Filtering [113.43366521994396]
  Unsupervised anomaly detection (UAD) aims to identify image- and pixel-level anomalies using only normal training data.<n>We present Unified Cost Filtering (UCF), a generic post-hoc refinement framework for refining anomaly cost volume of any UAD model.
  arXiv  Detail & Related papers  (2025-10-03T03:28:18Z)
• Sensitivity, Specificity, and Consistency: A Tripartite Evaluation of Privacy Filters for Synthetic Data Generation [57.13635002340272]
  Post-hoc privacy filtering techniques have been proposed to remove samples containing personally identifiable information.<n>This work presents a rigorous evaluation of a filtering pipeline applied to chest X-ray synthesis.<n>We conclude that substantial advances in filter design are needed before these methods can be confidently deployed in sensitive applications.
  arXiv  Detail & Related papers  (2025-10-02T08:32:20Z)
• Eigen-1: Adaptive Multi-Agent Refinement with Monitor-Based RAG for Scientific Reasoning [53.45095336430027]
  We develop a unified framework that combines implicit retrieval and structured collaboration.<n>On Humanity's Last Exam (HLE) Bio/Chem Gold, our framework achieves 48.3% accuracy.<n>Results on SuperGPQA and TRQA confirm robustness across domains.
  arXiv  Detail & Related papers  (2025-09-25T14:05:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.