Readable but Not Controllable: Neuron-Level Evidence for Medical LLM Hallucination

• URL: http://arxiv.org/abs/2607.00158v1
• Date: Tue, 30 Jun 2026 20:34:45 GMT
• Title: Readable but Not Controllable: Neuron-Level Evidence for Medical LLM Hallucination
• Abstract summary: We show that a simple, carefully conditioned probe can reliably detect hallucination.<n>We also show that this signal is distributed and redundant rather than narrowly localized.<n>Our results suggest hallucination mitigation is not simply a matter of identifying the right neurons.
• Score: 2.123088044860656
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Hallucination remains one of the central obstacles to deploying medical LLMs. Yet, even when hallucination can be detected, it is still unclear whether the internal representations associated with it can be used for control rather than detection alone. Using four open-source models across a suite of medical question-answering datasets, we show that a simple, carefully conditioned probe can reliably detect hallucination, with AUROC scores between 0.77 and 0.86 in our case. We further show that this signal is distributed and redundant rather than narrowly localized. Systematically selected neurons outperform random neurons only at very small subset sizes, whereas random subsets of a few hundred neurons recover nearly the full signal, and low-dimensional random projections preserve most of the detection performance. Beyond detection, we test whether this representation is causally actionable. Across 16 model--dataset combinations, our results reveal a sharp gap between decodability and controllability. The same internal structure that makes hallucination easy to detect does not translate into reliable neuron-level control. These findings show that medical hallucination seems to be readily visible in internal activations, but not easily corrected by steering the neurons most associated with it. More broadly, our results suggest that hallucination mitigation is not simply a matter of identifying the right neurons, and point to a deeper separation between what representations reveal and what they allow us to change.

Related papers

• NeuroActiSep: Detecting Factual Hallucinations from Feed-Forward Neurons in a Single Pass [0.17205106391379021]
  A less-studied approach is to identify feed-forward neurons correlated with hallucination.<n>We propose a method to rank feed-forward neurons at the final prompt token using a custom neuron selection dataset.<n>Our work provides empirical evidence that probes trained using the features from the selected neurons perform on par with probes trained on internal states.
  arXiv  Detail & Related papers  (2026-09-13T11:42:40Z)
• RAGognizer: Hallucination-Aware Fine-Tuning via Detection Head Integration [0.2696472814555309]
  We introduce RAGognizer, a hallucination-aware fine-tuning approach that integrates a detection head into an Large Language Models.<n> RAGognizer achieves state-of-the-art token-level hallucination detection while substantially reducing hallucination rates during generation.
  arXiv  Detail & Related papers  (2026-04-17T11:07:32Z)
• Steering the Verifiability of Multimodal AI Hallucinations [115.51077572812862]
  multimodal large language models (MLLMs) are prone to hallucinations and pose considerable risks to human users.<n>We propose an activation-space intervention method that learns separate probes for obvious and elusive hallucinations.<n>We reveal that obvious and elusive hallucinations elicit different intervention probes, allowing for fine-grained control over the model's verifiability.
  arXiv  Detail & Related papers  (2026-04-08T06:13:16Z)
• Neural Uncertainty Principle: A Unified View of Adversarial Fragility and LLM Hallucination [60.197429875410286]
  Adversarial vulnerability in vision and hallucination in large language models are conventionally viewed as separate problems.<n>Formalizing a Neural Uncertainty Principle (NUP) under a loss-induced state, we find that in near-bound regimes, further compression must be accompanied by increased sensitivity dispersion.<n>In vision, masking highly coupled components improves robustness without costly adversarial training.<n>In language, the same prefill-stage probe detects hallucination risk before generating any answer tokens.
  arXiv  Detail & Related papers  (2026-03-20T02:07:10Z)
• Towards Interpretable Hallucination Analysis and Mitigation in LVLMs via Contrastive Neuron Steering [60.23509717784518]
  Existing mitigation methods predominantly focus on output-level adjustments, leaving internal mechanisms that give rise to hallucinations largely unexplored.<n>We propose Contrastive Neuron Steering ( CNS), which identifies image-specific neurons via contrastive analysis between clean and noisy inputs.<n> CNS selectively amplifies informative neurons while suppressing perturbation-induced activations, producing more robust and semantically grounded visual representations.
  arXiv  Detail & Related papers  (2026-01-31T09:21:04Z)
• H-Neurons: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs [56.31565301428888]
  We identify hallucination-associated neurons (H-Neurons) in large language models (LLMs)<n>In terms of identification, we demonstrate that a remarkably sparse subset of neurons can reliably predict hallucination occurrences.<n>In terms of behavioral impact, controlled interventions reveal that these neurons are causally linked to over-compliance behaviors.
  arXiv  Detail & Related papers  (2025-12-01T15:32:14Z)
• Attention Head Embeddings with Trainable Deep Kernels for Hallucination Detection in LLMs [47.18623962083962]
  We present a novel approach for detecting hallucinations in large language models.<n>We find that hallucinated responses exhibit smaller deviations from their prompts compared to grounded responses.<n>We propose a model-intrinsic detection method that uses distributional distances as principled hallucination scores.
  arXiv  Detail & Related papers  (2025-06-11T15:59:15Z)
• HalluciDoctor: Mitigating Hallucinatory Toxicity in Visual Instruction Data [102.56792377624927]
  hallucinations inherent in machine-generated data remain under-explored. We present a novel hallucination detection and elimination framework, HalluciDoctor, based on the cross-checking paradigm. Our method successfully mitigates 44.6% hallucinations relatively and maintains competitive performance compared to LLaVA.
  arXiv  Detail & Related papers  (2023-11-22T04:52:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.