Internal Representation, Not Clinical Knowledge: Where Apparent LLM Triage Failures Originate

• URL: http://arxiv.org/abs/2605.29889v1
• Date: Thu, 28 May 2026 13:14:17 GMT
• Title: Internal Representation, Not Clinical Knowledge: Where Apparent LLM Triage Failures Originate
• Abstract summary: Patient-voiced clinical-triage benchmarks report high under-triage rates for consumer LLMs for constrained multiple-choice output.<n>We find the same medical features fire on the shared clinical narrative under both formats but go silent at the multiple-choice decision token in all the cases at every model.
• Score: 2.581200752140087
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Patient-voiced clinical-triage benchmarks report high under-triage rates for consumer LLMs for constrained multiple-choice output, yet the same cases score differently with free-text. We ask whether output format changes the model's \emph{clinical representation} or only the mapping from a preserved representation to an answer. Using sparse-autoencoder (SAE) features in Gemma 3 4B/12B IT and Qwen3-8B, we find the same medical features fire on the shared clinical narrative under both formats but go {silent} at the multiple-choice decision token in all the cases at every model. Three independent methods (natural-language autoencoder verbalization, decision-token logit attribution, and top-feature characterization) agree that scaffold and format features, but not medical features, drive the decision logits. Behaviorally, the multiple-choice penalty inverts under both structured and natural-language input, option-order shuffle rules out positional bias, and the gap is dominated by off-by-one decision (the model picks an adjacent acuity letter to the gold answer) rather than knowledge failure. Thus, the failure originates in the output format and not in the clinical representation.

Related papers

• Loud or Silent? A Reusable Framework for Per-Modality Failure Analysis in Multimodal Clinical AI [3.7524666944787644]
  Two questions matter beyond the size of the accuracy loss: which modality was responsible, and whether the model fails loudly or silently once that modality is dropped.<n>We present a model-agnostic modality-failure framework: given N modality embeddings, any mask-aware probe, and labels, it returns a per-example failure taxonomy.<n>It reports per-modality loud-vs-silent rates, and scales to a three-modality complementarity matrix.
  arXiv  Detail & Related papers  (2026-08-02T19:38:56Z)
• When Correct Decisions Hide Internal Stress: Decision-State Probing in Multimodal Language Models [18.50038551163168]
  We use S$3$E to analyze behavior-internal decoupling in multimodal language models.<n>We measure whether semantic-conflict candidates induce excess decision-state displacement.<n>Our results suggest forced-choice correctness alone is not a sufficient certificate of invariant internal decision geometry.
  arXiv  Detail & Related papers  (2026-06-07T01:11:36Z)
• Measuring the sensitivity of LLM-based structured extraction to prompt, model, and schema choices in clinical discharge summaries [0.0]
  Large language models are increasingly used for structured extraction from clinical free-text notes.<n>This work measures that sensitivity without human-annotated ground truth, by holding the extraction task fixed and varying one choice at a time.<n>Cross-prompt agreement was measured by Cohen's kappa on ICD-stratified subsets.
  arXiv  Detail & Related papers  (2026-06-04T10:14:12Z)
• Judge Circuits [24.717818484378583]
  We investigate the internal mechanism in Gemma-3, Qwen2.5, and Llama-3.<n>We find that judgments across structured understanding and open-ended preference tasks share a sparse, generalized Latent Evaluator sub-graph.<n>We provide a mechanistic account of format-induced inconsistency on the open-weight models we study.
  arXiv  Detail & Related papers  (2026-05-15T14:57:21Z)
• CLEAR: Revealing How Noise and Ambiguity Degrade Reliability in LLMs for Medicine [9.346299965194746]
  We introduce the CLinical Evaluation of Ambiguity and Reliability (CLEAR) framework.<n>CLEAR perturbs the number of plausible answer options, the presence of a ground truth or abstention option, and the semantic framing of answer options.<n>Applying CLEAR on three benchmarks evaluated across 17 medical large language models reveals three notable limitations.
  arXiv  Detail & Related papers  (2026-05-01T18:23:24Z)
• Statistics, Not Scale: Modular Medical Dialogue with Bayesian Belief Engine [51.722324399751294]
  Large language models are increasingly deployed as autonomous diagnostic agents, yet they conflate two fundamentally different capabilities.<n>We introduce BMBE, a modular diagnostic dialogue framework that enforces a strict separation between language and reasoning.
  arXiv  Detail & Related papers  (2026-04-21T21:59:57Z)
• One Panel Does Not Fit All: Case-Adaptive Multi-Agent Deliberation for Clinical Prediction [10.760929240872564]
  We propose CAMP (Case-Adaptive Multi-agent Panel), where an attending-physician agent dynamically assembles a specialist panel tailored to each case's diagnostic uncertainty.<n>A hybrid router directs each diagnosis through strong consensus, fallback to the attending physician's judgment, or evidence-based arbitration that weighs argument quality over vote counts.<n> CAMP consistently outperforms strong baselines while consuming fewer tokens than most competing multi-agent methods, with voting records and arbitration traces offering transparent decision audits.
  arXiv  Detail & Related papers  (2026-03-31T18:00:34Z)
• DiffVP: Differential Visual Semantic Prompting for LLM-Based CT Report Generation [18.257492970454898]
  We propose Differential Visual Prompting (DiffVP), which conditions report generation on explicit, high-level semantic scan-to-reference differences.<n>On two large-scale benchmarks, DiffVP consistently outperforms prior methods, improving the average BLEU-1-4 by +10.98 and +4.36, respectively.
  arXiv  Detail & Related papers  (2026-03-18T13:38:26Z)
• Dependence-Aware Label Aggregation for LLM-as-a-Judge via Ising Models [55.94503936470247]
  Large-scale AI evaluation increasingly relies on aggregating binary judgments from $K$ annotators, including judges.<n>Most classical methods assume annotators are conditionally independent given the true label $Yin0,1$, an assumption often violated by LLM judges.<n>We study label aggregation through a hierarchy of dependence-aware models based on Ising graphical models and latent factors.
  arXiv  Detail & Related papers  (2026-01-29T21:26:50Z)
• MEDEQUALQA: Evaluating Biases in LLMs with Counterfactual Reasoning [7.167933033102407]
  We introduce MEDEQUALQA, a counterfactual benchmark that perturbs only patient pronouns while holding critical symptoms and conditions constant.<n>We evaluate a GPT-4.1 model and compute Semantic Textual Similarity (STS) between reasoning traces to measure stability across pronoun variants.<n>Our results show overall high similarity (mean STS >0.80), but reveal consistent localized divergences in cited risk factors, guideline anchors, and differential ordering.
  arXiv  Detail & Related papers  (2025-10-09T22:12:58Z)
• Self-ensemble: Mitigating Confidence Mis-calibration for Large Language Models [67.62810111789338]
  Large Language Models exhibit a confidence distortion problem on multi-choice question-answering.<n>We propose Self-ensemble to solve this problem.<n> Experimental results on three LLMs and datasets demonstrate that Self-ensemble comprehensively addresses the confidence distortion problem.
  arXiv  Detail & Related papers  (2025-06-02T17:59:29Z)
• Answer, Assemble, Ace: Understanding How LMs Answer Multiple Choice Questions [103.20281438405111]
  Multiple-choice question answering (MCQA) is a key competence of performant transformer language models.<n>We employ vocabulary projection and activation patching methods to localize key hidden states that encode relevant information for predicting the correct answer.<n>We show that subsequent layers increase the probability of the predicted answer symbol in vocabulary space, and that this probability increase is associated with a sparse set of attention heads with unique roles.
  arXiv  Detail & Related papers  (2024-07-21T00:10:23Z)
• Large Language Models Are Not Robust Multiple Choice Selectors [117.72712117510953]
  Multiple choice questions (MCQs) serve as a common yet important task format in the evaluation of large language models (LLMs) This work shows that modern LLMs are vulnerable to option position changes due to their inherent "selection bias" We propose a label-free, inference-time debiasing method, called PriDe, which separates the model's prior bias for option IDs from the overall prediction distribution.
  arXiv  Detail & Related papers  (2023-09-07T17:44:56Z)
• SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models [55.60306377044225]
  "SelfCheckGPT" is a simple sampling-based approach to fact-check the responses of black-box models. We investigate this approach by using GPT-3 to generate passages about individuals from the WikiBio dataset.
  arXiv  Detail & Related papers  (2023-03-15T19:31:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.