Where Relevance Emerges: A Layer-Wise Study of Internal Attention for Zero-Shot Re-Ranking

• URL: http://arxiv.org/abs/2602.22591v1
• Date: Thu, 26 Feb 2026 03:51:31 GMT
• Title: Where Relevance Emerges: A Layer-Wise Study of Internal Attention for Zero-Shot Re-Ranking
• Authors: Haodong Chen, Shengyao Zhuang, Zheng Yao, Guido Zuccon, Teerapong Leelanupab,
• Abstract summary: In-Context Re-ranking (ICR) has recently been proposed as an $O(1)$ alternative method.<n>ICR extracts internal attention signals directly, avoiding the overhead of text generation.<n>No unified study has compared internal attention with traditional generative and likelihood-based mechanisms.
• Score: 40.652380579951206
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Zero-shot document re-ranking with Large Language Models (LLMs) has evolved from Pointwise methods to Listwise and Setwise approaches that optimize computational efficiency. Despite their success, these methods predominantly rely on generative scoring or output logits, which face bottlenecks in inference latency and result consistency. In-Context Re-ranking (ICR) has recently been proposed as an $O(1)$ alternative method. ICR extracts internal attention signals directly, avoiding the overhead of text generation. However, existing ICR methods simply aggregate signals across all layers; layer-wise contributions and their consistency across architectures have been left unexplored. Furthermore, no unified study has compared internal attention with traditional generative and likelihood-based mechanisms across diverse ranking frameworks under consistent conditions. In this paper, we conduct an orthogonal evaluation of generation, likelihood, and internal attention mechanisms across multiple ranking frameworks. We further identify a universal "bell-curve" distribution of relevance signals across transformer layers, which motivates the proposed Selective-ICR strategy that reduces inference latency by 30%-50% without compromising effectiveness. Finally, evaluation on the reasoning-intensive BRIGHT benchmark shows that precisely capturing high-quality in-context attention signals fundamentally reduces the need for model scaling and reinforcement learning: a zero-shot 8B model matches the performance of 14B reinforcement-learned re-rankers, while even a 0.6B model outperforms state-of-the-art generation-based approaches. These findings redefine the efficiency-effectiveness frontier for LLM-based re-ranking and highlight the latent potential of internal signals for complex reasoning ranking tasks. Our code and results are publicly available at https://github.com/ielab/Selective-ICR.

Related papers

• Search-P1: Path-Centric Reward Shaping for Stable and Efficient Agentic RAG Training [11.136092421166097]
  Agentic RAG enhances large language models by incorporating external knowledge.<n>Current RL-based training methods suffer from sparse outcome rewards that discard intermediate signals.<n>We propose Search-P1, a framework that introduces path-centric reward shaping for agentic RAG training.
  arXiv  Detail & Related papers  (2026-02-26T03:31:00Z)
• From Absolute to Relative: Rethinking Reward Shaping in Group-Based Reinforcement Learning [7.6602542594279335]
  We propose Reinforcement Learning with Relative Rewards to shift reward shaping from absolute scoring to relative ranking.<n>We show that RLRR yields consistent performance improvements over standard group-based baselines across reasoning benchmarks and open-ended generation tasks.
  arXiv  Detail & Related papers  (2026-01-30T15:07:06Z)
• SoliReward: Mitigating Susceptibility to Reward Hacking and Annotation Noise in Video Generation Reward Models [53.19726629537694]
  Post-training alignment of video generation models with human preferences is a critical goal.<n>Current data collection paradigms, reliant on in-prompt pairwise annotations, suffer from labeling noise.<n>We propose SoliReward, a systematic framework for video RM training.
  arXiv  Detail & Related papers  (2025-12-17T14:28:23Z)
• Efficient Thought Space Exploration through Strategic Intervention [54.35208611253168]
  We propose a novel Hint-Practice Reasoning (HPR) framework that operationalizes this insight through two synergistic components.<n>The framework's core innovation lies in Distributional Inconsistency Reduction (DIR), which dynamically identifies intervention points.<n> Experiments across arithmetic and commonsense reasoning benchmarks demonstrate HPR's state-of-the-art efficiency-accuracy tradeoffs.
  arXiv  Detail & Related papers  (2025-11-13T07:26:01Z)
• Towards Robust Zero-Shot Reinforcement Learning [22.262048244005296]
  Recent development of zero-shot reinforcement learning (RL) has opened a new avenue for learning pre-trained generalist policies that can adapt to arbitrary new tasks in a zero-shot manner.<n>While the popular Forward-Backward representations (FB) and related methods have shown promise in zero-shot RL, we empirically found that their modeling lacks expressivity and that extrapolation errors caused suboptimal performance.<n>We propose an upgraded FB-based framework that simultaneously enhances learning stability, policy extraction capability, and representation learning quality.
  arXiv  Detail & Related papers  (2025-10-17T07:33:19Z)
• Resource-Aware Neural Network Pruning Using Graph-based Reinforcement Learning [0.8890833546984916]
  This paper presents a novel approach to neural network pruning by integrating a graph-based observation space into an AutoML framework.<n>Our framework transforms the pruning process by introducing a graph representation of the target neural network.<n>For the action space we transition from continuous pruning ratios to fine-grained binary action spaces.
  arXiv  Detail & Related papers  (2025-09-04T15:05:05Z)
• Lightweight and Direct Document Relevance Optimization for Generative Information Retrieval [49.669503570350166]
  Generative information retrieval (GenIR) is a promising neural retrieval paradigm that formulates document retrieval as a document identifier (docid) generation task.<n>Existing GenIR models suffer from token-level misalignment, where models trained to predict the next token often fail to capture document-level relevance effectively.<n>We propose direct document relevance optimization (DDRO), which aligns token-level docid generation with document-level relevance estimation through direct optimization via pairwise ranking.
  arXiv  Detail & Related papers  (2025-04-07T15:27:37Z)
• ACTRESS: Active Retraining for Semi-supervised Visual Grounding [52.08834188447851]
  A previous study, RefTeacher, makes the first attempt to tackle this task by adopting the teacher-student framework to provide pseudo confidence supervision and attention-based supervision. This approach is incompatible with current state-of-the-art visual grounding models, which follow the Transformer-based pipeline. Our paper proposes the ACTive REtraining approach for Semi-Supervised Visual Grounding, abbreviated as ACTRESS.
  arXiv  Detail & Related papers  (2024-07-03T16:33:31Z)
• Energy-based Out-of-Distribution Detection for Graph Neural Networks [76.0242218180483]
  We propose a simple, powerful and efficient OOD detection model for GNN-based learning on graphs, which we call GNNSafe. GNNSafe achieves up to $17.0%$ AUROC improvement over state-of-the-arts and it could serve as simple yet strong baselines in such an under-developed area.
  arXiv  Detail & Related papers  (2023-02-06T16:38:43Z)
• Robust Locality-Aware Regression for Labeled Data Classification [5.432221650286726]
  We propose a new discriminant feature extraction framework, namely Robust Locality-Aware Regression (RLAR) In our model, we introduce a retargeted regression to perform the marginal representation learning adaptively instead of using the general average inter-class margin. To alleviate the disturbance of outliers and prevent overfitting, we measure the regression term and locality-aware term together with the regularization term by the L2,1 norm.
  arXiv  Detail & Related papers  (2020-06-15T11:36:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.