Resonant Context Anchoring: Decoupling Attention Routing and Signal Gain at Inference Time

• URL: http://arxiv.org/abs/2606.01923v1
• Date: Mon, 01 Jun 2026 08:57:51 GMT
• Title: Resonant Context Anchoring: Decoupling Attention Routing and Signal Gain at Inference Time
• Abstract summary: Large Language Models (LLMs) frequently exhibit "contextual disregard" when faced with input evidence that conflicts with their internal parametric memory.<n>We propose Resonant Context Anchoring (RCA), a lightweight inference-time intervention method grounded in the perspective of residual stream signal dynamics.
• Score: 3.988157662292324
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large Language Models (LLMs) frequently exhibit "contextual disregard" when faced with input evidence that conflicts with their internal parametric memory, leading to persistent factual hallucinations. Existing mitigation strategies primarily rely on suppressing specific neuron activations or employing computationally expensive contrastive decoding mechanisms, which often result in increased perplexity or significantly elevated inference latency. To address these limitations, we propose Resonant Context Anchoring (RCA), a lightweight inference-time intervention method grounded in the perspective of residual stream signal dynamics. RCA aims to resolve the signal attenuation of external evidence during its propagation through deep networks. The core mechanism involves the orthogonal decoupling of routing logic and information magnitude within the self-attention module. By utilizing raw pre-softmax attention scores as an instantaneous metric of semantic alignment, we construct a dynamic gain field via non-linear rectification to selectively amplify the norms of value vectors corresponding to context tokens, without altering the attention probability distribution. This mechanism effectively elevates the signal-to-noise ratio (SNR) of input evidence within the residual stream mixture, thereby robustly anchoring the generation trajectory to the truthful context during inference. Extensive experiments on the Llama-3 model series demonstrate that RCA significantly improves contextual faithfulness across multiple factual consistency and strong knowledge-conflict tasks, effectively suppressing parametric hallucinations. Furthermore, results confirm that as a training-free and computationally negligible plug-and-play module, RCA achieves a Pareto improvement in faithfulness and fluency while maintaining the model's general language understanding capabilities.

Related papers

• UniNDM: A Unified Noise-driven Detection and Mitigation Framework Against Sexual Content in Text-to-Image Generation [90.0896070825457]
  We propose a unified noise-driven framework that rethinks safety mechanisms through the lens of noise dynamics in diffusion processes.<n>Our key insight is that early-stage predicted noise exhibits inherent separability between normal and sexually explicit content.<n>We develop a lightweight noise-based detector achieving superior accuracy with virtually no computational overhead.
  arXiv  Detail & Related papers  (2026-07-18T14:03:39Z)
• Spatiotemporal Hidden-State Dynamics as a Signature of Internal Reasoning in Large Language Models [0.0]
  Internal representations carry correctness-related signals, but their coarse aggregations may obscure the token and layer structure underlying reasoning.<n>We investigate hidden-state transitions across decoding steps and layers, and identify a pattern in large reasoning models (LRMs)<n>We formalize this characteristic as Stemporal Amplitude of Latent Transition (StALT), a statistic trajectory that summarizes temporal changes between adjacent tokens weighted by within-token layer saliency.
  arXiv  Detail & Related papers  (2026-05-03T12:46:41Z)
• Hypergraph and Latent ODE Learning for Multimodal Root Cause Localization in Microservices [25.49822040345663]
  Root cause localization in cloud native microservice systems requires modeling complex service dependencies, irregular temporal dynamics, and heterogeneous observability data.<n>We present HyperODE RCA, a unified framework that combines hypergraph attention learning, latent ordinary differential equations, and multimodal cross attention fusion for fine grained root cause analysis.
  arXiv  Detail & Related papers  (2026-05-01T02:18:57Z)
• Adaptive Guidance for Retrieval-Augmented Masked Diffusion Models [60.25003813232697]
  We propose Adaptive Retrieval-Augmented Masked Diffusion (ARAM) as a training-free adaptive guidance framework for Masked Diffusion Models (MDMs)<n>ARAM calibrates the guidance scale during denoising according to the Signal-to-Noise Ratio (SNR) of the distributional shift induced by retrieved context.<n>Experiments on multiple knowledge-intensive QA benchmarks show that ARAM improves overall QA performance over competitive RAG baselines.
  arXiv  Detail & Related papers  (2026-03-18T12:54:50Z)
• Hit-RAG: Learning to Reason with Long Contexts via Preference Alignment [97.55382322103986]
  Hit-RAG is a multi-stage preference alignment framework designed to resolve cognitive bottlenecks.<n>Our approach systematically refines the utilization of external evidence via three distinct stages.
  arXiv  Detail & Related papers  (2026-03-07T04:05:27Z)
• Wireless Federated Multi-Task LLM Fine-Tuning via Sparse-and-Orthogonal LoRA [61.12136997430116]
  Decentralized federated learning (DFL) based on low-rank adaptation (LoRA) enables mobile devices with multi-task datasets to collaboratively fine-tune a large language model (LLM) by exchanging locally updated parameters with a subset of neighboring devices via wireless connections for knowledge integration.<n> directly aggregating parameters fine-tuned on heterogeneous datasets induces three primary issues across the DFL life-cycle: (i) catastrophic knowledge forgetting during fine-tuning process, arising from conflicting update directions caused by data heterogeneity; (ii) textitinefficient communication and convergence during model aggregation process,
  arXiv  Detail & Related papers  (2026-02-24T02:45:32Z)
• Causal Autoregressive Diffusion Language Model [70.7353007255797]
  CARD reformulates the diffusion process within a strictly causal attention mask, enabling dense, per-token supervision in a single forward pass.<n>Our results demonstrate that CARD achieves ARM-level data efficiency while unlocking the latency benefits of parallel generation.
  arXiv  Detail & Related papers  (2026-01-29T17:38:29Z)
• Lost in the Noise: How Reasoning Models Fail with Contextual Distractors [57.31788955167306]
  Recent advances in reasoning models and agentic AI systems have led to an increased reliance on diverse external information.<n>We introduce NoisyBench, a comprehensive benchmark that systematically evaluates model robustness across 11 datasets in RAG, reasoning, alignment, and tool-use tasks.<n>Our evaluation reveals a catastrophic performance drop of up to 80% in state-of-the-art models when faced with contextual distractors.
  arXiv  Detail & Related papers  (2026-01-12T05:43:51Z)
• DynaCausal: Dynamic Causality-Aware Root Cause Analysis for Distributed Microservices [17.058900957896864]
  DynaCausal is a dynamic causality-aware framework for cause analysis in distributed microservice systems.<n>We show how DynaCausal consistently surpasses state-of-the-art methods, attaining an average AC@1 of 0.63 with absolute gains from 0.25 to 0.46.
  arXiv  Detail & Related papers  (2025-10-26T10:13:18Z)
• Latent Diffusion Model Based Denoising Receiver for 6G Semantic Communication: From Stochastic Differential Theory to Application [11.385703484113552]
  We propose a novel semantic communication framework empowered by generative artificial intelligence (GAI)<n>A latent diffusion model (LDM)-based semantic communication framework is proposed that combines a variational autoencoder for semantic features extraction.<n>The proposed system is a training-free framework that supports zero-shot generalization, and achieves superior performance under low-SNR and out-of-distribution conditions.
  arXiv  Detail & Related papers  (2025-06-06T03:20:32Z)
• Dynamic Attention-Guided Context Decoding for Mitigating Context Faithfulness Hallucinations in Large Language Models [26.51079570548107]
  Large language models (LLMs) often exhibit Context Faithfulness Hallucinations.<n>We propose Dynamic Attention-Guided Context Decoding (DAGCD), a lightweight framework that leverages attention distributions and uncertainty signals in a single-pass decoding.
  arXiv  Detail & Related papers  (2025-01-02T05:07:06Z)
• Disentangled Noisy Correspondence Learning [56.06801962154915]
  Cross-modal retrieval is crucial in understanding latent correspondences across modalities. DisNCL is a novel information-theoretic framework for feature Disentanglement in Noisy Correspondence Learning.
  arXiv  Detail & Related papers  (2024-08-10T09:49:55Z)
• Improving auditory attention decoding performance of linear and non-linear methods using state-space model [21.40315235087551]
  Recent advances in electroencephalography have shown that it is possible to identify the target speaker from single-trial EEG recordings. AAD methods reconstruct the attended speech envelope from EEG recordings, based on a linear least-squares cost function or non-linear neural networks. We investigate a state-space model using correlation coefficients obtained with a small correlation window to improve the decoding performance.
  arXiv  Detail & Related papers  (2020-04-02T09:56:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.