ClinTraceBench: Source-Verifiable Longitudinal Clinical Reasoning over EHR-Derived Dialogues

• URL: http://arxiv.org/abs/2609.01111v1
• Date: Tue, 01 Sep 2026 11:51:37 GMT
• Title: ClinTraceBench: Source-Verifiable Longitudinal Clinical Reasoning over EHR-Derived Dialogues
• Abstract summary: We introduce ClinTraceBench: 385 MIMIC-IV-derived verified dialogues with event-ID provenance, a nine-task taxonomy, and L0--L4 deterministic + L5 human-audit validation.<n>We evaluate eight history representation strategies---a no-context floor, textitlast-visit-only, textitfull-context, BGE-M3 textitdense-retrieval, two compression schemes, and two agentic-memory systems.
• Score: 11.42809060379683
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Clinical LLM assistants must reason over multi-visit patient trajectories, yet whether the compact history representations used to scale them---retrieval, structured timelines, LLM summaries, agentic memory---preserve the longitudinal signal clinical reasoning needs has not been measured. We introduce ClinTraceBench: 385 MIMIC-IV-derived verified dialogues with event-ID provenance, a nine-task taxonomy (T1--T9), and L0--L4 deterministic + L5 human-audit validation (98.92\% agreement). We evaluate eight history representation strategies---a no-context floor, \textit{last-visit-only}, \textit{full-context}, BGE-M3 \textit{dense-retrieval}, two compression schemes, and two agentic-memory systems (\textit{Mem0}, \textit{A-Mem})---across four backbones (DeepSeek-V3, GPT-4o-mini, Haiku~4.5, Sonnet~4.6) on 6{,}271 questions: 32 cells, 200{,}672 predictions. Four findings: (SP4) a controlled T3 injection probe isolates compression-induced \textit{relation} loss---with the attribution sentence present \textit{before} construction, \textit{Mem0}, \textit{A-Mem} and \textit{llm-summary} still recover only 0--5.3\% of the injected positives; (SP1) compressed strategies pay an aggregation tax on multi-visit trends and cross-patient comparisons; (SP2) the blind-to-full gap spans $+29.8$~pp (GPT-4o-mini) to $+62.7$~pp (Haiku); (SP3) abstention scales non-monotonically with context length. On the Pareto frontier Haiku dominates Sonnet under \textit{full-context} (\$25.76 vs.\ \$106.21), inverting the ``biggest backbone wins'' heuristic.

Related papers

• Synergistic Perception-Reasoning Governance: Grounding Medical MLLMs with Verifiable Anatomical Evidence [45.78409219344429]
  Multimodal large language models (MLLMs) show strong promise for clinical VQA and radiology report generation, yet inference-time hallucinations undermine trustworthy use.<n>We propose a holistic, training-free evidence-injection framework that systematically mitigates hallucinations through dual-side evidence injection.<n>We conduct systematic evaluations on 2 tasks and 5 datasets using textttLLaVA-1.5-7B, texttLLaVA-Med-1.5-7B, texttQwen3-VL-8B/32B, and textttInternVL
  arXiv  Detail & Related papers  (2026-06-30T08:07:25Z)
• Now You (Still) See Me: Detecting Evasive Steganographic Payloads in LLMs [52.149036302760386]
  Large language models can be fine-tuned to encode prompt-borne secrets into fluent, seemingly benign outputs.<n>Recent work proposes detection using linear probes that recover the secret from internal activations.<n>We show that this defense can be systematically evaded, but that detectability can be recovered through a targeted data-level intervention.
  arXiv  Detail & Related papers  (2026-06-08T12:27:11Z)
• TokenMizer: Graph-Structured Session Memory for Long-Horizon LLM Context Management [0.0]
  We present TokenMizer, an open-source proxy system that models large language model sessions as a typed knowledge graph.<n>It produces resume blocks averaging 78 tokens (range: 42-124) - 2x smaller than evaluated baselines.<n>Across all sessions, TokenMizer mean task recall 51.0%, decision recall 46.6%, and file recall 58.7%.
  arXiv  Detail & Related papers  (2026-06-04T16:12:28Z)
• Benchmarking Speech-to-Speech Translation Models [55.00303727199927]
  Speech-to-speech translation (S2ST) has advanced rapidly, but offline evaluation lacks a unified protocol.<n>We introduce a benchmarking framework integrating 46 metrics across eight dimensions.<n>We deploy it on 1,248 model-language configurations from FLEURS and CVSS.
  arXiv  Detail & Related papers  (2026-06-02T07:01:33Z)
• ATLAS: All-round Testing of Long-context Abilities across Scales [42.83200405769639]
  ATLAS is a benchmarking framework that redefines long-context evaluation as length-dependent capability profiling.<n> ATLAS contributes three methodological principles: (i) a layered taxonomy separating foundational operations from application workloads so failures can be attributed, (ii) length-aware AUC scoring that integrates score-length curves over a fixed 8K-1M grid, replacing single-point metrics with full degradation profiles, and (iii) ATLAScore, a harmonic-mean aggregate over taxonomy categories that penalizes imbalanced profiles.
  arXiv  Detail & Related papers  (2026-05-27T07:33:47Z)
• When Do LLM Agents Treat Surface Noise Differently from Semantic Noise? A 68-Cell Measurement Study with a Held-Out Trace-Level Validation [9.055086193088083]
  We document an empirical phenomenon in chain-of-thought and ReAct agents driven by ten large language models.<n>Mean-bearing perturbations alter final answers more often than presentation perturbations of comparable severity.
  arXiv  Detail & Related papers  (2026-05-25T15:57:11Z)
• Perception or Prejudice: Can MLLMs Go Beyond First Impressions of Personality? [74.69723255239663]
  Grounded Personality Reasoning requires MLLMs to anchor each Big Five rating in observable evidence through a chain of rating, reasoning, and grounding.<n> MM-OCEAN is produced by a multi-agent pipeline with human verification, with timestamped behavioral observations, evidence-grounded trait analyses, and seven categories of cue-grounding MCQs.<n>The analysis uncovers a striking Prejudice Gap: across the field, 51% of correct ratings are not grounded in retrieved cues, and the Holistic-Grounding Rate spans only 0-33.5%.
  arXiv  Detail & Related papers  (2026-05-21T07:42:47Z)
• Rethinking How to Remember: Beyond Atomic Facts in Lifelong LLM Agent Memory [88.81430082035617]
  LLM agents require a memory system that can faithfully store, efficiently retrieve, and deeply reason over accumulated dialogue history.<n>We propose TriMem, which maintains three coexisting representation granularities, including raw dialogue segments anchored by source identifiers for storage fidelity.
  arXiv  Detail & Related papers  (2026-05-19T15:05:06Z)
• What Do EEG Foundation Models Capture from Human Brain Signals? [64.48249643001402]
  Modern EEG foundation models learn directly from raw signals via self-supervised pretraining.<n>We decompose into three sub-questions: emphwhat does the model learn, emphwhat does the model use, and emphhow much can be explained<n>The audit covers three foundation models (CSBrain, CBraMod, LaBraM), five clinical tasks (MDD, Stress, ISRUC-Sleep, TUSL, Siena), and a 6-family 63-feature lexicon.
  arXiv  Detail & Related papers  (2026-05-12T01:57:53Z)
• ProactBench: Beyond What The User Asked For [5.422521416406412]
  ProactBench decomposes it into three phase-tied types: textscEmergent, inference from a single disclosed anchor; textscCritical, synthesis across multiple anchors; and textscRecovery, grounded forward-looking value after task completion.<n>Our information asymmetries defend against style-confounding scoring, leakage, external-context contamination, and information dumps.
  arXiv  Detail & Related papers  (2026-05-09T23:56:04Z)
• The Structural Attention Tax: How Retrieval Format Hijacks In-Context Learning Independent of Content [11.226669792850645]
  We formalise a phenomenon we term the structural attention tax: knowledge graph (KG) triples.<n>KG triples capture 2-3x more attention per token than semantically equivalent natural-language text.<n>We develop a formal framework decomposing attention scores into semantic and structural components.
  arXiv  Detail & Related papers  (2026-04-21T08:47:00Z)
• One Token Away from Collapse: The Fragility of Instruction-Tuned Helpfulness [12.183451602438753]
  We show that simple lexical constraints (banning a single punctuation character or common word) cause instruction-tuned LLMs to collapse their responses.<n>We show that base models show no systematic collapse under identical constraints, with effects that are small, noisy, and bidirectional.
  arXiv  Detail & Related papers  (2026-04-14T17:40:01Z)
• Cooperative Memory Paging with Keyword Bookmarks for Long-Horizon LLM Conversations [2.6382975801439836]
  We propose cooperative paging: segments are replaced with minimal keyword bookmarks, and the model is given a recall() tool to retrieve full content on demand.<n>On the LoCoMo benchmark, cooperative paging achieves the highest answer quality among six methods -- outperforming truncation, BM25, word-overlap retrieval, a search-tool baseline, and full context.<n>We then study the paging design space with a 5x4 ablation over boundary strategies and eviction policies (3,176 synthetic probes, 1,600 LoCoMo probes)
  arXiv  Detail & Related papers  (2026-04-14T07:06:35Z)
• When Self-Reference Fails to Close: Matrix-Level Dynamics in Large Language Models [0.0]
  Self-referential statements and meta-cognitive prompts are more stable than paradoxical self-reference on key collapse-related metrics.<n>We propose, as a conjecture, that NCTR forces finite-depth transformers toward dynamical regimes where these problems concentrate.
  arXiv  Detail & Related papers  (2026-04-13T23:23:02Z)
• APEX-EM: Non-Parametric Online Learning for Autonomous Agents via Structured Procedural-Episodic Experience Replay [7.370176470430802]
  LLM-based autonomous agents lack persistent procedural memory.<n>We present APEX-EM, a non-parametric online learning framework that accumulates, retrieves, and reuses structured procedural plans.
  arXiv  Detail & Related papers  (2026-03-31T00:24:56Z)
• ClaimFlow: Tracing the Evolution of Scientific Claims in NLP [67.23189226608389]
  We introduce $texttClaimFlow$, a claim-centric view of the NLP literature.<n>We define a new task $-$ $textitClaim Relation Classification$ $-$ which requires models to infer the scientific stance toward a cited claim.<n>Our analysis reveals that $63.5$% claims are never reused; only $11.1$% are ever challenged.
  arXiv  Detail & Related papers  (2026-03-17T02:43:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.