Overview of the EReL@MIR 2025 Multimodal Document Retrieval Challenge (Track 1)

• URL: http://arxiv.org/abs/2606.04240v1
• Date: Tue, 02 Jun 2026 21:39:32 GMT
• Title: Overview of the EReL@MIR 2025 Multimodal Document Retrieval Challenge (Track 1)
• Abstract summary: The MIR Challenge was held at the first EReL@MIR workshop, co-located with The Web Conference 2025.<n>It asked participants to build a emphsingle retrieval system that handles two complementary regimes.<n>This report describes the challenge design, datasets, and evaluation protocol; reports the final standings; and analyses the three winning teams' systems.
• Score: 4.198923768732269
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Retrieval over visually-rich documents, pages that interleave text with figures, tables, and charts, is essential for multimodal retrieval-augmented generation, yet most retrievers still discard the visual channel. The \emph{Multimodal Document Retrieval Challenge}, Track~1 of the MIR Challenge at the first EReL@MIR workshop, co-located with The Web Conference 2025, asks participants to build a \emph{single} retrieval system that handles two complementary regimes: closed-set document page retrieval within long documents from a text query (MMDocIR), and open-domain retrieval of Wikipedia-style passages from an image or image-plus-text query (M2KR). Systems are ranked by the macro-average of mean Recall@$\{1,3,5\}$ over the two tasks. The challenge drew 455 entrants and 586 submissions across 22 teams. This report describes the challenge design, datasets, and evaluation protocol; reports the final standings; and analyses the three winning teams' systems. All three build on decoder-based Multimodal-LLM embedders from the Qwen2-VL family rather than on CLIP-style encoders, and differ chiefly in whether they reach the top through fine-tuned ensembles, training-free multi-route fusion with a strong vision-language re-ranker, or zero-shot late interaction. The training-free system finished within $0.1$ point of the fine-tuned winner.

Related papers

• What the Reranker Sees: Multi-Aspect Page Annotation for Long-Document Multimodal Question Answering [21.941823782270557]
  Trident is a visual-question-answering system with a visual caption, section path, entity tags, multi-axis concept hits, and a text snippet.<n>On two long-document datasets, the annotation+rerank protocol substantially improves retrieval F1 across five datasets.<n>The best Trident configuration is the strongest downstream QA pipeline in our evaluation.
  arXiv  Detail & Related papers  (2026-08-14T19:29:42Z)
• TempRet: Temporal Enhancement and Two-Stage Reranking for CVPR 2026 EPIC-KITCHENS-100 Multi-Instance Retrieval Challenge [71.10535279591527]
  We present our solution, termed TempRet, to the CVPR 2026 EPIC-KITCHENS-100 MIR challenge.<n>Our approach builds upon a CLIP-based dual-encoder backbone and introduces two key components to address the temporal and cross-modal challenges.<n>Our method achieves 67.97% average mAP and 82.92% average nDCG on the EK-100 MIR benchmark.
  arXiv  Detail & Related papers  (2026-05-23T08:37:39Z)
• BRIDGE: Multimodal-to-Text Retrieval via Reinforcement-Learned Query Alignment [5.285385905661152]
  Best vision-language encoder achieves only 27.6 nDCG@10 on MM-BRIGHT, underperforming strong text-only retrievers.<n>We present textbfBRIDGE, a two-component system that resolves this mismatch without multimodal encoders.<n>textbfFORGE is a query alignment model trained via reinforcement learning, which distills noisy multimodal queries into compact, retrieval-optimized search strings.<n>textbfLENS is a reasoning-enhanced dense retriever fine-tuned on reasoning-intensive retrieval data to handle the intent-rich queries FORGE produces.
  arXiv  Detail & Related papers  (2026-04-08T15:28:21Z)
• ICDAR 2025 Competition on End-to-End Document Image Machine Translation Towards Complex Layouts [47.61983500021015]
  Document Image Machine Translation (DIMT) seeks to translate text embedded in document images from one language to another.<n>The DIMT 2025 Challenge advances research on end-to-end document image translation.<n>The competition features two tracks, OCR-free and OCR-based, each with two subtasks for small (less than 1B parameters) and large (greater than 1B parameters) models.
  arXiv  Detail & Related papers  (2026-03-10T09:04:38Z)
• $G^2$-Reader: Dual Evolving Graphs for Multimodal Document QA [53.491241153213565]
  $G2$-Reader is a dual-graph system for multimodal question answering.<n>$G2$-Reader with Qwen3-VL-32B-Instruct reaches 66.21% average accuracy, outperforming strong baselines and a standalone GPT-5 (53.08%)<n>On VisDoMBench across five multimodal domains, $G2$-Reader with Qwen3-VL-32B-Instruct reaches 66.21% average accuracy, outperforming strong baselines and a standalone GPT-5 (53.08%)
  arXiv  Detail & Related papers  (2026-01-29T17:52:54Z)
• SDS KoPub VDR: A Benchmark Dataset for Visual Document Retrieval in Korean Public Documents [10.146296597660598]
  Existing benchmarks for visual document retrieval (VDR) largely overlook non-English languages and the structural complexity of official publications.<n>We introduce SDS KoPub VDR, the first large-scale, public benchmark for retrieving and understanding Korean public documents.<n>The benchmark is built upon 361 real-world documents, including 256 files under the KOGL Type 1 license and 105 from official legal portals.
  arXiv  Detail & Related papers  (2025-11-07T01:16:07Z)
• Recurrence Meets Transformers for Universal Multimodal Retrieval [59.92546492752452]
  ReT-2 is a unified retrieval model that supports multimodal queries composed of both images and text.<n>We evaluate ReT-2 on the challenging M2KR and M-BEIR benchmarks across different retrieval configurations.<n>When integrated into retrieval-augmented generation pipelines, ReT-2 also improves downstream performance on Encyclopedic-VQA and InfoSeek datasets.
  arXiv  Detail & Related papers  (2025-09-10T18:00:29Z)
• Zero-Shot Document Understanding using Pseudo Table of Contents-Guided Retrieval-Augmented Generation [4.875345207589195]
  DocsRay is a training-free document understanding system.<n>It integrates pseudo Table of Contents (TOC) generation with hierarchical Retrieval-Augmented Generation (RAG)
  arXiv  Detail & Related papers  (2025-07-31T03:14:45Z)
• DocMMIR: A Framework for Document Multi-modal Information Retrieval [35.41540195822167]
  We introduce DocMMIR, a novel multi-modal document retrieval framework.<n>We construct a large-scale cross-domain multimodal benchmark, comprising 450K samples.<n>Results show a +31% improvement in MRR@10 compared to the zero-shot baseline.
  arXiv  Detail & Related papers  (2025-05-25T20:58:58Z)
• mPLUG-DocOwl2: High-resolution Compressing for OCR-free Multi-page Document Understanding [103.05835688963947]
  We propose a High-resolution DocCompressor module to compress each high-resolution document image into 324 tokens. DocOwl2 sets a new state-of-the-art across multi-page document understanding benchmarks and reduces first token latency by more than 50%. Compared to single-image MLLMs trained on similar data, our DocOwl2 achieves comparable single-page understanding performance with less than 20% of the visual tokens.
  arXiv  Detail & Related papers  (2024-09-05T11:09:00Z)
• End-to-end Knowledge Retrieval with Multi-modal Queries [50.01264794081951]
  ReMuQ requires a system to retrieve knowledge from a large corpus by integrating contents from both text and image queries. We introduce a retriever model ReViz'' that can directly process input text and images to retrieve relevant knowledge in an end-to-end fashion. We demonstrate superior performance in retrieval on two datasets under zero-shot settings.
  arXiv  Detail & Related papers  (2023-06-01T08:04:12Z)
• Sequence-aware multimodal page classification of Brazilian legal documents [0.21204495827342434]
  We train and evaluate our methods on a novel multimodal dataset of 6,510 lawsuits. Each lawsuit is an ordered sequence of pages, which are stored both as an image and as a corresponding text. We use them as extractors of visual and textual features, which are then combined through our proposed Fusion Module.
  arXiv  Detail & Related papers  (2022-07-02T06:23:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.