One Model Is Enough: Native Retrieval Embeddings from LLM Agent Hidden States

• URL: http://arxiv.org/abs/2603.08429v1
• Date: Mon, 09 Mar 2026 14:25:35 GMT
• Title: One Model Is Enough: Native Retrieval Embeddings from LLM Agent Hidden States
• Abstract summary: This paper introduces a lightweight projection head that maps hidden states directly into the embedding space, eliminating the need for a separate embedding model.<n> Experiments on the QReCC conversational search benchmark show competitive Recall@10 and MRR@10 compared to the standard generate-then-encode pipeline.
• Score: 5.647839536820347
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: LLM agents that retrieve external knowledge typically generate a search query as text, then run a separate embedding model to encode it into a vector. This two-model pipeline adds infrastructure complexity and latency, yet is redundant: the LLM already encodes the full conversational context in its hidden states. We propose equipping LLM agents with native retrieval capability by adding a lightweight projection head that maps hidden states directly into the embedding space, eliminating the need for a separate embedding model. Trained with a combination of alignment, contrastive, and rank distillation losses, our method retains 97\% of baseline retrieval quality while enabling the LLM agent to search with its own representations. Experiments on the QReCC conversational search benchmark show competitive Recall@10 and MRR@10 compared to the standard generate-then-encode pipeline, with systematic ablations confirming the contribution of each loss component.

Related papers

• Latent Abstraction for Retrieval-Augmented Generation [4.6096940605642915]
  Existing systems rely on generating natural language queries at each hop.<n>We propose textbfLAnR (Latent Abstraction for RAG), a unified framework in which a single LLM jointly performs encoding, retrieval, and generation entirely within its own latent space.
  arXiv  Detail & Related papers  (2026-04-20T06:26:13Z)
• Align Documents to Questions: Question-Oriented Document Rewriting for Retrieval-Augmented Generation [51.55755193937205]
  We propose QREAM, a style-controlled rewriter that aligns retrieved documents with a question-oriented style while preserving facts.<n>Our framework consists of two stages: (1) QREAM-ICL, which uses stylistic seeds to guide iterative rewriting exploration; and (2) QREAM-FT, a lightweight student model distilled from denoised ICL outputs.
  arXiv  Detail & Related papers  (2026-04-19T08:39:21Z)
• LLM as a Neural Architect: Controlled Generation of Image Captioning Models Under Strict API Contracts [48.83701310501069]
  We present NN-Caption, an LLM-guided neural architecture search pipeline.<n>It generates runnable image-captioning models by composing CNN encoders from LEMUR's classification backbones.<n>This work presents a pipeline that integrates prompt-based code generation with automatic evaluation.
  arXiv  Detail & Related papers  (2025-12-07T10:47:28Z)
• Hint-Augmented Re-ranking: Efficient Product Search using LLM-Based Query Decomposition [20.966359103135762]
  We show that LLMs can uncover latent intent behind superlatives in e-commerce queries.<n>Our approach decomposes queries into attribute-value hints generated concurrently with retrieval.<n>Our method improves search performanc eby 10.9 points in MAP and ranking by 5.9 points in MRR over baselines.
  arXiv  Detail & Related papers  (2025-11-17T23:53:25Z)
• LLM-guided Hierarchical Retrieval [54.73080745446999]
  LATTICE is a hierarchical retrieval framework that enables an LLM to reason over and navigate large corpora with logarithmic search complexity.<n>A central challenge in such LLM-guided search is that the model's relevance judgments are noisy, context-dependent, and unaware of the hierarchy.<n>Our framework achieves state-of-the-art zero-shot performance on the reasoning-intensive BRIGHT benchmark.
  arXiv  Detail & Related papers  (2025-10-15T07:05:17Z)
• Iterative Self-Incentivization Empowers Large Language Models as Agentic Searchers [74.17516978246152]
  Large language models (LLMs) have been widely integrated into information retrieval to advance traditional techniques.<n>We propose EXSEARCH, an agentic search framework, where the LLM learns to retrieve useful information as the reasoning unfolds.<n>Experiments on four knowledge-intensive benchmarks show that EXSEARCH substantially outperforms baselines.
  arXiv  Detail & Related papers  (2025-05-26T15:27:55Z)
• Invar-RAG: Invariant LLM-aligned Retrieval for Better Generation [43.630437906898635]
  We propose a novel two-stage fine-tuning architecture called Invar-RAG. In the retrieval stage, an LLM-based retriever is constructed by integrating LoRA-based representation learning. In the generation stage, a refined fine-tuning method is employed to improve LLM accuracy in generating answers based on retrieved information.
  arXiv  Detail & Related papers  (2024-11-11T14:25:37Z)
• Refiner: Restructure Retrieval Content Efficiently to Advance Question-Answering Capabilities [30.1331670544648]
  Large Language Models (LLMs) are limited by their parametric knowledge, leading to hallucinations in knowledge-extensive tasks. We propose $textitRefiner$, an end-to-end extract-and-restructure paradigm that operates in the post-retrieval process of RAG.
  arXiv  Detail & Related papers  (2024-06-17T09:25:10Z)
• PromptReps: Prompting Large Language Models to Generate Dense and Sparse Representations for Zero-Shot Document Retrieval [76.50690734636477]
  We propose PromptReps, which combines the advantages of both categories: no need for training and the ability to retrieve from the whole corpus. The retrieval system harnesses both dense text embedding and sparse bag-of-words representations.
  arXiv  Detail & Related papers  (2024-04-29T04:51:30Z)
• Description-Based Text Similarity [59.552704474862004]
  We identify the need to search for texts based on abstract descriptions of their content. We propose an alternative model that significantly improves when used in standard nearest neighbor search.
  arXiv  Detail & Related papers  (2023-05-21T17:14:31Z)
• Large Language Models are Strong Zero-Shot Retriever [89.16756291653371]
  We propose a simple method that applies a large language model (LLM) to large-scale retrieval in zero-shot scenarios. Our method, the Language language model as Retriever (LameR), is built upon no other neural models but an LLM.
  arXiv  Detail & Related papers  (2023-04-27T14:45:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.