UMER: Unifying Embedding and Ranking via Pair-Aware Discriminative Reasoning for Universal Multimodal Retrieval

• URL: http://arxiv.org/abs/2608.18504v2
• Date: Wed, 26 Aug 2026 06:46:11 GMT
• Title: UMER: Unifying Embedding and Ranking via Pair-Aware Discriminative Reasoning for Universal Multimodal Retrieval
• Abstract summary: We propose a Unified Multimodal Embedding and Ranking framework for universal multimodal retrieval.<n>UMER replaces item-wise reflection with Pair-Aware Discriminative Reasoning.<n>On the MMEB-V2 benchmark, UMER achieves state-of-the-art performance.
• Score: 7.404190239665035
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Universal multimodal retrieval aims to support diverse instruction-aware retrieval tasks, demanding both efficient corpus-scale matching and fine-grained semantic reasoning. Recent MLLM-based embedding methods typically derive representations from hidden states, while Chain-of-Thought (CoT) reasoning is emerging as a promising strategy for embedding enhancement by encoding intermediate semantic evidence into the representation space. However, existing CoT methods typically use item-wise reasoning over queries and candidates in isolation, providing no explicit evidence to distinguish a positive from a semantically confusable hard negative. Moreover, contrastive embeddings capture global similarity but struggle with meta-tasks requiring answer verification, category judgment or fine-grained reasoning. In this paper, we propose UMER, a Unified Multimodal Embedding and Ranking framework for universal multimodal retrieval. UMER replaces item-wise reflection with Pair-Aware Discriminative Reasoning, which compares query--candidate pairs to identify instruction-relevant matching and discrepancy evidence. UMER jointly learns contrastive embeddings for efficient global matching and discriminative ranking for explicit pairwise relevance judgment within a single MLLM. A complementary mutual distillation strategy further transfers reliable pairwise preferences between the embedding and ranking functions. On the MMEB-V2 benchmark, UMER achieves state-of-the-art performance under comparable experimental settings while supporting budget-adjustable inference.

Related papers

• Reason What Matters: Retrieval-Grounded Reasoning for Universal Multimodal Embeddings [18.705254311226565]
  Retrieval-aware Self-Distillation (RASD) constructs privileged guidance from input-supported evidence that distinguishes the positive item from retrieved hard negatives.<n>Retrieval-adaptive Inference (RAI) uses a retrieval confidence head to estimate the remaining retrieval utility of a partial CoT.<n>ReWAM achieves state-of-the-art retrieval performance while delivering up to 5x the inference throughput of competitive explicit-CoT UME methods.
  arXiv  Detail & Related papers  (2026-09-14T09:48:05Z)
• Action-Aligned Retrieval with Pairwise Multimodal Reranking for Text-Based Person Anomaly Search [10.617164505348283]
  Text-based person anomaly search requires distinguishing individuals based on fine-grained, context-dependent behaviors rather than mere appearance.<n>Existing methods struggle to capture these context-conditioned actions.<n>We propose a unified three-stage coarse-to-fine framework that combines action-aligned retrieval with pairwise multimodal reranking.
  arXiv  Detail & Related papers  (2026-08-24T17:04:34Z)
• Learning from Failures: Retrieval-Centric CoT via Hard Negatives for Unified Multimodal Retrieval [17.695551248974215]
  UniME-R1 is an embedder-adviser framework that learns to reason over initially retrieved candidates.<n>UniME-R1 consistently improves retrieval performance over strong baselines.
  arXiv  Detail & Related papers  (2026-08-06T14:04:56Z)
• AgentMap: Joint Equivalence and Subsumption Discovery for Ontology Matching [15.085939995703667]
  We introduce Hybrid Ontology Matching (HOM), a new OM task that unifies equivalence and subsumption discovery.<n>We propose a Large Language Model (LLM)-based multi-agent OM framework AgentMap that is implemented by a series of interdependent semantic decisions.<n> Experimental results show that AgentMap achieves promising performance on the hybrid setting.
  arXiv  Detail & Related papers  (2026-07-29T16:58:10Z)
• Beyond Overlap Metrics: Rewarding Reasoning and Preferences for Faithful Multi-Role Dialogue Summarization [5.909152930998754]
  Multi-role dialogue summarization requires modeling complex interactions among multiple speakers.<n>Most existing methods optimize for automatic metrics such as ROUGE and BERTScore.<n>We propose a novel framework that couples explicit cognitive-style reasoning with reward-based optimization.
  arXiv  Detail & Related papers  (2026-04-19T01:27:41Z)
• UniME-V2: MLLM-as-a-Judge for Universal Multimodal Embedding Learning [101.62386137855704]
  We present a novel Universal Multimodal Embedding (UniME-V2) model.<n>Our approach first constructs a potential hard negative set through global retrieval.<n>We then introduce the MLLM-as-a-Judge mechanism, which utilizes MLLMs to assess the semantic alignment of query-candidate pairs.<n>These scores serve as a foundation for hard negative mining, mitigating the impact of false negatives and enabling the identification of diverse, high-quality hard negatives.
  arXiv  Detail & Related papers  (2025-10-15T13:07:00Z)
• CompassVerifier: A Unified and Robust Verifier for LLMs Evaluation and Outcome Reward [50.97588334916863]
  We develop CompassVerifier, an accurate and robust lightweight verifier model for evaluation and outcome reward.<n>It demonstrates multi-domain competency spanning math, knowledge, and diverse reasoning tasks, with the capability to process various answer types.<n>We introduce VerifierBench benchmark comprising model outputs collected from multiple data sources, augmented through manual analysis of metaerror patterns to enhance CompassVerifier.
  arXiv  Detail & Related papers  (2025-08-05T17:55:24Z)
• Estimating Commonsense Plausibility through Semantic Shifts [66.06254418551737]
  We propose ComPaSS, a novel discriminative framework that quantifies commonsense plausibility by measuring semantic shifts.<n> Evaluations on two types of fine-grained commonsense plausibility estimation tasks show that ComPaSS consistently outperforms baselines.
  arXiv  Detail & Related papers  (2025-02-19T06:31:06Z)
• CART: A Generative Cross-Modal Retrieval Framework with Coarse-To-Fine Semantic Modeling [53.97609687516371]
  Cross-modal retrieval aims to search for instances, which are semantically related to the query through the interaction of different modal data.<n>Traditional solutions utilize a single-tower or dual-tower framework to explicitly compute the score between queries and candidates.<n>We propose a generative cross-modal retrieval framework (CART) based on coarse-to-fine semantic modeling.
  arXiv  Detail & Related papers  (2024-06-25T12:47:04Z)
• RankCSE: Unsupervised Sentence Representations Learning via Learning to Rank [54.854714257687334]
  We propose a novel approach, RankCSE, for unsupervised sentence representation learning. It incorporates ranking consistency and ranking distillation with contrastive learning into a unified framework. An extensive set of experiments are conducted on both semantic textual similarity (STS) and transfer (TR) tasks.
  arXiv  Detail & Related papers  (2023-05-26T08:27:07Z)
• Synergies between Disentanglement and Sparsity: Generalization and Identifiability in Multi-Task Learning [79.83792914684985]
  We prove a new identifiability result that provides conditions under which maximally sparse base-predictors yield disentangled representations. Motivated by this theoretical result, we propose a practical approach to learn disentangled representations based on a sparsity-promoting bi-level optimization problem.
  arXiv  Detail & Related papers  (2022-11-26T21:02:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.