Adaptive Activation Cancellation for Hallucination Mitigation in Large Language Models

• URL: http://arxiv.org/abs/2603.10195v1
• Date: Tue, 10 Mar 2026 19:43:18 GMT
• Title: Adaptive Activation Cancellation for Hallucination Mitigation in Large Language Models
• Abstract summary: We propose a real-time inference-time framework that treats hallucination-associated neural activations as structured interference within the transformer residual stream.<n>The framework identifies Hallucination Nodes (H-Nodes) via layer-wise linear probing and suppresses them using a confidence-weighted forward hook.
• Score: 6.53417787720275
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Language Models frequently generate fluent but factually incorrect text. We propose Adaptive Activation Cancellation (AAC), a real-time inference-time framework that treats hallucination-associated neural activations as structured interference within the transformer residual stream, drawing an explicit analogy to classical adaptive noise cancellation from signal processing. The framework identifies Hallucination Nodes (H-Nodes) via layer-wise linear probing and suppresses them using a confidence-weighted forward hook during auto-regressive generation -- requiring no external knowledge, no fine-tuning, and no additional inference passes. Evaluated across OPT-125M, Phi-3-mini, and LLaMA 3-8B on TruthfulQA and HaluEval, the real-time hook is the only intervention that consistently improves downstream accuracy on all three scales. Critically, the method is strictly surgical: WikiText-103 perplexity and MMLU reasoning accuracy are preserved at exactly 0.0% degradation across all three model scales, a property that distinguishes AAC from interventions that trade fluency or general capability for factual improvement. On the LLaMA 3-8B scale, the hook additionally yields positive generation-level gains (MC1 +0.04; MC2 +0.003; Token-F1 +0.003) while achieving probe-space selectivity 5.94x - 3.5x higher than the ITI baseline -- demonstrating that targeted neuron-level suppression can simultaneously improve factual accuracy and preserve model capability.

Related papers

• Actionable Hallucination Detection: Translating Latent Uncertainty into Agentic Critique [1.1172382217477128]
  Large Language Models (LLMs) deployed as AI agents frequently exhibit user specification-grounding failures.<n>Existing detection methods fail to provide actionable, real-time correction as they either do not localize the hallucinations, or incur prohibitive inference latency.<n>We introduce the Latent Critic, a lightweight low-rank adapter (LoRA) that operates concurrently with a frozen base LLM's generation.
  arXiv  Detail & Related papers  (2026-08-11T03:26:16Z)
• Parameter-efficient Prompt Tuning of Vision Foundation Model With Adaptive Focal Loss for Interpretable MCI Screening [1.0195618602298684]
  We propose a parameter-efficient framework utilizing frozen DINOv2-adapted model adapted via three modality-specific learnable prompt tokens.<n>The proposed architecture yields an MCI-class F1 of 0.641 and an AUC of 0.795, outperforming the computationally heavier ResViT baseline by 0.110 in MCI-class F1.
  arXiv  Detail & Related papers  (2026-07-16T14:26:22Z)
• Ring-Zero: Scaling Zero RL to a Trillion Parameters for Emergent Reasoning [55.99621402404222]
  Reinforcement learning with verifiable rewards without human-annotated data, often referred to as zero RL, has emerged as a powerful paradigm for eliciting chain-of-thought reasoning.<n>We aim to elicit high-quality reasoning behaviors from the model.<n>We find that naive scaling often suffers from poor readability, token redundancy, and a lack of adaptive reasoning depth.
  arXiv  Detail & Related papers  (2026-07-14T06:14:55Z)
• Information Fidelity in Tool-Using LLM Agents: A Martingale Analysis of the Model Context Protocol [69.11739400975445]
  We introduce the first theoretical framework for analyzing error accumulation in Model Context Protocol (MCP) agents.<n>We show that cumulative distortion exhibits linear growth and high-probability deviations bounded by $O(sqrtT)$.<n>Key findings include: semantic weighting reduces distortion by 80%, and periodic re-grounding approximately every 9 steps suffices for error control.
  arXiv  Detail & Related papers  (2026-02-10T21:08:53Z)
• Representation-Aware Unlearning via Activation Signatures: From Suppression to Knowledge-Signature Erasure [2.0017902634527194]
  We introduce Knowledge Immunization Framework (KIF), a representation-aware architecture that distinguishes genuine erasure from obfuscation.<n>Our approach combines dynamic suppression of subject-specific representations with parameter-efficient adaptation, enabling durable unlearning without full model retraining.
  arXiv  Detail & Related papers  (2026-01-15T16:28:14Z)
• Noise-Robust Tiny Object Localization with Flows [63.60972031108944]
  We propose a noise-robust localization framework leveraging normalizing flows for flexible error modeling and uncertainty-guided optimization.<n>Our method captures complex, non-Gaussian prediction distributions through flow-based error modeling, enabling robust learning under noisy supervision.<n>An uncertainty-aware gradient modulation mechanism further suppresses learning from high-uncertainty, noise-prone samples, mitigating overfitting while stabilizing training.
  arXiv  Detail & Related papers  (2026-01-02T09:16:55Z)
• Mitigating LLM Hallucination via Behaviorally Calibrated Reinforcement Learning [32.32593439144886]
  Behavior-calibrated reinforcement learning allows smaller models to surpass frontier models in uncertainty quantification.<n>Our model's log-scale Accuracy-to-Hallucination Ratio gain (0.806) exceeds GPT-5's (0.207) in a challenging in-domain evaluation.
  arXiv  Detail & Related papers  (2025-12-22T22:51:48Z)
• Detecting AI Hallucinations in Finance: An Information-Theoretic Method Cuts Hallucination Rate by 92% [4.693270291878929]
  Large language models (LLMs) produce fluent but unsupported answers - hallucinations.<n>We propose ECLIPSE, a framework that treats hallucination as a mismatch between a model's semantic entropy and the capacity of available evidence.
  arXiv  Detail & Related papers  (2025-12-02T05:25:48Z)
• Fine-Tuned LLMs Know They Don't Know: A Parameter-Efficient Approach to Recovering Honesty [48.593888815135934]
  Large Language Models (LLMs) are increasingly important for safe deployment in high-stakes domains.<n>Existing recovery methods rely on data-intensive global parameter adjustments, implicitly assuming that SFT deeply corrupts the models' ability to recognize their knowledge boundaries.<n>We propose Honesty-Critical Neurons Restoration (HCNR) to surgically repair this suppressed capacity.
  arXiv  Detail & Related papers  (2025-11-17T05:30:48Z)
• NIRVANA: Structured pruning reimagined for large language models compression [50.651730342011014]
  We introduce NIRVANA, a novel pruning method designed to balance immediate zero-shot preservation accuracy with robust fine-tuning.<n>To further address the unique challenges posed by structured pruning, NIRVANA incorporates an adaptive sparsity allocation mechanism across layers and modules.<n>Experiments conducted on Llama3, Qwen, T5 models demonstrate that NIRVANA outperforms existing structured pruning methods under equivalent sparsity constraints.
  arXiv  Detail & Related papers  (2025-09-17T17:59:00Z)
• Mitigating Hallucinations in Large Vision-Language Models by Self-Injecting Hallucinations [73.37711261605271]
  hallucination mitigation methods are mainly based on preference alignment and require external human annotations or auxiliary models for preference data collection.<n>We propose Autonomous Preference Alignment via Self-Injection (APASI), a novel and generalizable method that mitigates hallucinations without external dependencies.<n>APASI leverages the target LVLM to self-inject hallucinations into a generated response, creating a pair of responses with varying preference levels.
  arXiv  Detail & Related papers  (2025-09-14T14:26:53Z)
• GrAInS: Gradient-based Attribution for Inference-Time Steering of LLMs and VLMs [56.93583799109029]
  GrAInS is an inference-time steering approach that operates across both language-only and vision-language models and tasks.<n>During inference, GrAInS hidden activations at transformer layers guided by token-level attribution signals, and normalizes activations to preserve representational scale.<n>It consistently outperforms both fine-tuning and existing steering baselines.
  arXiv  Detail & Related papers  (2025-07-24T02:34:13Z)
• Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free [81.65559031466452]
  We conduct experiments to investigate gating-augmented softmax attention variants.<n>We find that a simple modification-applying a head-specific sigmoid gate after the Scaled Dot-Product Attention (SDPA)-consistently improves performance.
  arXiv  Detail & Related papers  (2025-05-10T17:15:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.