Bridging the Semantic-Utility Gap in Multimodal RAG via Generator-in-the-Loop Alignment

• URL: http://arxiv.org/abs/2609.08188v1
• Date: Tue, 08 Sep 2026 03:18:58 GMT
• Title: Bridging the Semantic-Utility Gap in Multimodal RAG via Generator-in-the-Loop Alignment
• Abstract summary: Vision-language models (VLMs) augmented with retrieval-augmented generation (RAG) benefit from access to external evidence.<n>Standard retrievers and rerankers optimize for semantic similarity rather than answer utility, creating a preference gap.<n>We propose a two-stage generator-in-the-loop alignment framework that closes this gap without human document-level relevance annotations.
• Score: 37.61076424435405
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-language models (VLMs) augmented with retrieval-augmented generation (RAG) benefit from access to external evidence. However, standard retrievers and rerankers optimize for semantic similarity rather than answer utility, creating a preference gap: documents that appear relevant may not help the generator produce a correct answer. Motivated by this, we propose a two-stage generator-in-the-loop alignment framework that closes this gap without human document-level relevance annotations. Our framework consists of two stages: in Stage 1, a VLM generates a hypothetical text passage from the image-query pair, which is used as the retrieval query for dense text search, bridging the image-to-text modality gap. In Stage 2, a cross-encoder reranker adapted with low-rank adaptation (LoRA) is fine-tuned using answer-supervised preference pairs mined from the frozen VLM: given the dataset answer label, a candidate document is labeled positive if the VLM produces the correct answer when given that document as context, and negative otherwise. This generator-guided signal is compatible with multiple alignment loss functions, including contrastive (triplet) loss, pairwise direct preference optimization (DPO), and supervised fine-tuning (SFT), and supports periodic re-mining to refresh preference pairs as the reranker improves. Experiments on VQA-X and A-OKVQA with Qwen3.5-2B and Qwen3-VL-4B-Instruct show that our proposed framework consistently outperforms rank-order, random, and REPLUG-style likelihood baselines under various alignment losses and pool size settings, suggesting that answer-level generator feedback is an effective supervision signal for preference alignment.

Related papers

• Gradient-free Task-Conditioned Retrieval for On-Device In-Context Learning [35.54770997926101]
  Conditional Retrieval Alignment (CoRA) is a gradient-free framework that converts a frozen encoder into a task-conditioned retriever.<n>We show that CoRA's rank-constrained basis is the optimal low-rank compression of the output-conditioned fitted representation.<n>We extend the framework to multimodal retrieval by incorporating visual representations into the conditioning and retrieval spaces.
  arXiv  Detail & Related papers  (2026-07-30T07:03:18Z)
• GRADRAG: Cross-Component Prompt Adaptation for Coordinated Multi-Agent RAG [0.5566372336864309]
  We introduce GRADRAG, a framework for cross-component prompt adaptation.<n>An Evaluator critiques downstream answers and supporting evidence, producing actionable feedback.<n>We evaluate GRADRAG on the SQUALITY and QMSUM benchmarks.
  arXiv  Detail & Related papers  (2026-07-23T13:54:38Z)
• F-GRPO: Factorized Group-Relative Policy Optimization for Unified Candidate Generation and Ranking [79.49893545611779]
  Large Language Models (LLMs) can generate a subset and order it within a single autoregressive pass.<n>This flexibility introduces a new optimization challenge: the model must search an output space while receiving utility feedback only after the full ranked list is generated.<n>This credit assignment gap makes end-to-end optimization unstable and sample-inefficient.<n>We propose a unified framework that performs both within a single autoregressive rollout and optimize them end-to-end via factorized group-relative policy optimization.
  arXiv  Detail & Related papers  (2026-05-13T04:52:33Z)
• Reward-Guided Semantic Evolution for Test-time Adaptive Object Detection [82.2968697030677]
  Open-vocabulary object detection with vision-language models (VLMs) such as Grounding DINO suffers from performance degradation under test-time distribution shifts.<n>We propose Reward-Guided Semantic Evolution (RGSE), a training-free framework that directly refines the text embeddings at test time.
  arXiv  Detail & Related papers  (2026-05-06T06:17:41Z)
• Align Documents to Questions: Question-Oriented Document Rewriting for Retrieval-Augmented Generation [51.55755193937205]
  We propose QREAM, a style-controlled rewriter that aligns retrieved documents with a question-oriented style while preserving facts.<n>Our framework consists of two stages: (1) QREAM-ICL, which uses stylistic seeds to guide iterative rewriting exploration; and (2) QREAM-FT, a lightweight student model distilled from denoised ICL outputs.
  arXiv  Detail & Related papers  (2026-04-19T08:39:21Z)
• RefAlign: Representation Alignment for Reference-to-Video Generation [53.368296137314225]
  RefAlign is a representation alignment framework that aligns DiT reference-branch features to the semantic space of a visual foundation model (VFM)<n>Experiments on the OpenS2V-Eval benchmark demonstrate that RefAlign outperforms current state-of-the-art methods in TotalScore.
  arXiv  Detail & Related papers  (2026-03-26T17:59:57Z)
• ROAP: A Reading-Order and Attention-Prior Pipeline for Optimizing Layout Transformers in Key Information Extraction [5.594845708011402]
  This paper presents ROAP, a lightweight and architecture-agnostic pipeline designed to optimize attention distributions in layout Transformers.<n> experiments on the FUNSD and CORD benchmarks demonstrate that ROAP consistently improves the performance of backbones.
  arXiv  Detail & Related papers  (2026-01-09T02:02:37Z)
• RAPO++: Cross-Stage Prompt Optimization for Text-to-Video Generation via Data Alignment and Test-Time Scaling [59.088798018184235]
  textbfRAPO++ is a cross-stage prompt optimization framework.<n>It unifies training-data-aligned refinement, test-time iterative scaling, and large language model fine-tuning.<n> RAPO++ achieves significant gains in semantic alignment, compositional reasoning, temporal stability, and physical plausibility.
  arXiv  Detail & Related papers  (2025-10-23T04:45:09Z)
• REAL: Response Embedding-based Alignment for LLMs [1.9513983244114355]
  We propose a strategy for constructing a high-quality training dataset that focuses on acquiring the less ambiguous preference pairs.<n>Experiments show that choosing dissimilar response pairs enhances the direct alignment of LLMs.<n>Findings suggest that focusing on distinct pairs can reduce the label error and improve LLM alignment efficiency.
  arXiv  Detail & Related papers  (2024-09-17T22:40:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.