MARVEL: Multimodal Adaptive Reasoning-intensiVe Expand-rerank and retrievaL

• URL: http://arxiv.org/abs/2604.07079v1
• Date: Wed, 08 Apr 2026 13:35:09 GMT
• Title: MARVEL: Multimodal Adaptive Reasoning-intensiVe Expand-rerank and retrievaL
• Abstract summary: We introduce textbfMARVEL (textbfMultimodal textbfAdaptive textbfReasoning-intensitextbfVe textbfExpand-rerank and retrievatextbfL), a unified pipeline that combines LLM-driven query expansion, textbfMARVEL-Retriever and GPT-4o-based chain-of-thought reranking.
• Score: 5.5504253907902275
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multimodal retrieval over text corpora remains a fundamental challenge: the best vision-language encoder achieves only 27.6 nDCG@10 on MM-BRIGHT, a reasoning-intensive multimodal retrieval benchmark, underperforming strong text-only systems. We argue that effective multimodal retrieval requires three tightly integrated capabilities that existing approaches address only in isolation: expanding the query's latent intent, retrieving with a model trained for complex reasoning, and reranking via explicit step-by-step reasoning over candidates. We introduce \textbf{MARVEL} (\textbf{M}ultimodal \textbf{A}daptive \textbf{R}easoning-intensi\textbf{V}e \textbf{E}xpand-rerank and retrieva\textbf{L}), a unified pipeline that combines LLM-driven query expansion, \textbf{MARVEL-Retriever} -- a reasoning-enhanced dense retriever fine-tuned for complex multimodal queries -- and GPT-4o-based chain-of-thought reranking with optional multi-pass reciprocal rank fusion. Evaluated on MM-BRIGHT across 29 technical domains, MARVEL achieves \textbf{37.9} nDCG@10, surpassing the best multimodal encoder by \textbf{+10.3 points} and outperforming all single-stage baselines in 27 of 29 domains and matching or approaching the best baseline in the remaining two highly-specialized domains (Crypto, Quantum Computing), demonstrating that reasoning-intensive multimodal retrieval is best addressed through a unified expand-retrieve-rerank framework. https://github.com/mm-bright/multimodal-reasoning-retrieval

Related papers

• Generative Universal Multimodal Retrieval with Dual-role Identifiers [1.0981910170846492]
  We propose DrIG, a novel Generative framework for universal multimodal retrieval featuring Dual-role Identifiers.<n>DrIG consistently outperforms state-of-the-art generative multimodal baselines across diverse tasks, while hybrid reranking achieves a favorable efficiency-effectiveness trade-off against strong dense retrievers.
  arXiv  Detail & Related papers  (2026-08-13T09:10:09Z)
• HIVE: Query, Hypothesize, Verify An LLM Framework for Multimodal Reasoning-Intensive Retrieval [5.5504253907902275]
  Multimodal retrieval models fail on reasoning-intensive queries where images must be deeply integrated with text to identify relevant documents.<n>We introduce textbfHIVE (textbfHypothesis-driven textbfVisual textbfEvidence Retrieval), a plug-and-play framework that injects explicit visual-text reasoning into a retriever via LLMs.
  arXiv  Detail & Related papers  (2026-04-08T15:41:42Z)
• BRIDGE: Multimodal-to-Text Retrieval via Reinforcement-Learned Query Alignment [5.285385905661152]
  Best vision-language encoder achieves only 27.6 nDCG@10 on MM-BRIGHT, underperforming strong text-only retrievers.<n>We present textbfBRIDGE, a two-component system that resolves this mismatch without multimodal encoders.<n>textbfFORGE is a query alignment model trained via reinforcement learning, which distills noisy multimodal queries into compact, retrieval-optimized search strings.<n>textbfLENS is a reasoning-enhanced dense retriever fine-tuned on reasoning-intensive retrieval data to handle the intent-rich queries FORGE produces.
  arXiv  Detail & Related papers  (2026-04-08T15:28:21Z)
• PLUME: Latent Reasoning Based Universal Multimodal Embedding [52.35354073629127]
  Universal multimodal embedding (UME) maps heterogeneous inputs into a shared retrieval space with a single model.<n>Recent approaches improve UME by generating explicit chain-of-thought (CoT) rationales before extracting embeddings.<n>We propose PLUME, a latent reasoning framework that advances UME by replacing verbalized CoT with a short autoregressive rollout of continuous latent states.
  arXiv  Detail & Related papers  (2026-04-02T14:04:53Z)
• Beyond Global Similarity: Towards Fine-Grained, Multi-Condition Multimodal Retrieval [27.493644447594367]
  MCMR (Multi-Conditional Multimodal Retrieval) is a large-scale benchmark designed to evaluate fine-grained, multi-condition cross-modal retrieval under natural-language queries.<n>It spans five product domains: upper and bottom clothing, jewelry, shoes, and furniture.<n>We benchmark a diverse suite of MLLM-based multimodal retrievers and vision-language rerankers to assess their condition-aware reasoning abilities.
  arXiv  Detail & Related papers  (2026-03-01T12:53:47Z)
• MM-BRIGHT: A Multi-Task Multimodal Benchmark for Reasoning-Intensive Retrieval [18.53521844184766]
  MM-BRIGHT is the first multimodal benchmark for reasoning-intensive retrieval.<n>Our dataset consists of 2,803 real-world queries spanning 29 diverse technical domains.
  arXiv  Detail & Related papers  (2026-01-14T15:31:54Z)
• M$^3$KG-RAG: Multi-hop Multimodal Knowledge Graph-enhanced Retrieval-Augmented Generation [20.170643730917963]
  M$3$KG-RAG is a Multi-hop Multimodal Knowledge Graph-enhanced RAG.<n>It retrieves query-aligned audio-visual knowledge from MMKGs.<n>It improves reasoning depth and answer faithfulness in MLLMs.
  arXiv  Detail & Related papers  (2025-12-23T07:54:03Z)
• Q-RAG: Long Context Multi-step Retrieval via Value-based Embedder Training [50.37345200692884]
  We propose Q-RAG, a novel approach that fine-tunes Embedder model for multi-step retrieval using reinforcement learning (RL)<n>Q-RAG offers a competitive, resource-efficient alternative to existing multi-step retrieval methods for open-domain question answering.
  arXiv  Detail & Related papers  (2025-11-10T17:31:02Z)
• MRMR: A Realistic and Expert-Level Multidisciplinary Benchmark for Reasoning-Intensive Multimodal Retrieval [87.24221266746686]
  We introduce MRMR, the first expert-level multidisciplinary multimodal retrieval benchmark requiring intensive reasoning.<n>It challenges retrieval systems across diverse areas of expertise, enabling fine-grained model comparison across domains.<n> queries are reasoning-intensive, with images requiring deeper interpretation such as diagnosing microscopic slides.
  arXiv  Detail & Related papers  (2025-10-10T16:14:56Z)
• UNIDOC-BENCH: A Unified Benchmark for Document-Centric Multimodal RAG [82.84014669683863]
  Multimodal retrieval-augmented generation (MM-RAG) is a key approach for applying large language models to real-world knowledge bases.<n>UniDoc-Bench is the first large-scale, realistic benchmark for MM-RAG built from 70k real-world PDF pages.<n>Our experiments show that multimodal text-image fusion RAG systems consistently outperform both unimodal and jointly multimodal embedding-based retrieval.
  arXiv  Detail & Related papers  (2025-10-04T04:30:13Z)
• Progressive Multimodal Reasoning via Active Retrieval [64.74746997923967]
  Multi-step multimodal reasoning tasks pose significant challenges for large language models (MLLMs)<n>We propose AR-MCTS, a universal framework designed to progressively improve the reasoning capabilities of MLLMs.<n>We show that AR-MCTS can optimize sampling diversity and accuracy, yielding reliable multimodal reasoning.
  arXiv  Detail & Related papers  (2024-12-19T13:25:39Z)
• MM-Embed: Universal Multimodal Retrieval with Multimodal LLMs [78.5013630951288]
  This paper introduces techniques for advancing information retrieval with multimodal large language models (MLLMs)<n>We first study fine-tuning an MLLM as a bi-encoder retriever on 10 datasets with 16 retrieval tasks.<n>Our model, MM-Embed, achieves state-of-the-art performance on the multimodal retrieval benchmark M-BEIR.
  arXiv  Detail & Related papers  (2024-11-04T20:06:34Z)
• CART: A Generative Cross-Modal Retrieval Framework with Coarse-To-Fine Semantic Modeling [53.97609687516371]
  Cross-modal retrieval aims to search for instances, which are semantically related to the query through the interaction of different modal data.<n>Traditional solutions utilize a single-tower or dual-tower framework to explicitly compute the score between queries and candidates.<n>We propose a generative cross-modal retrieval framework (CART) based on coarse-to-fine semantic modeling.
  arXiv  Detail & Related papers  (2024-06-25T12:47:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.