From Easy to Hard: The MIR Benchmark for Progressive Interleaved Multi-Image Reasoning

• URL: http://arxiv.org/abs/2509.17040v2
• Date: Thu, 16 Oct 2025 02:56:19 GMT
• Title: From Easy to Hard: The MIR Benchmark for Progressive Interleaved Multi-Image Reasoning
• Authors: Hang Du, Jiayang Zhang, Guoshun Nan, Wendi Deng, Zhenyan Chen, Chenyang Zhang, Wang Xiao, Shan Huang, Yuqi Pan, Tao Qi, Sicong Leng,
• Abstract summary: Multi-image Interleaved Reasoning aims to improve Multi-modal Large Language Models (MLLMs) ability to jointly comprehend and reason across multiple images and their associated textual contexts.<n>MIR requires joint reasoning over multiple images accompanied by interleaved textual contexts to accurately associate image regions with corresponding texts.<n>Our method significantly enhances models reasoning performance on MIR benchmarking and other established benchmarks.
• Score: 27.117102717184597
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-image Interleaved Reasoning aims to improve Multi-modal Large Language Models (MLLMs) ability to jointly comprehend and reason across multiple images and their associated textual contexts, introducing unique challenges beyond single-image or non-interleaved multi-image tasks. While current multi-image benchmarks overlook interleaved textual contexts and neglect distinct relationships between individual images and their associated texts, enabling models to reason over multi-image interleaved data may significantly enhance their comprehension of complex scenes and better capture cross-modal correlations. To bridge this gap, we introduce a novel benchmark MIR, requiring joint reasoning over multiple images accompanied by interleaved textual contexts to accurately associate image regions with corresponding texts and logically connect information across images. To enhance MLLMs ability to comprehend multi-image interleaved data, we introduce reasoning steps for each instance within the benchmark and propose a stage-wise curriculum learning strategy. This strategy follows an "easy to hard" approach, progressively guiding models from simple to complex scenarios, thereby enhancing their ability to handle challenging tasks. Extensive experiments benchmarking multiple MLLMs demonstrate that our method significantly enhances models reasoning performance on MIR and other established benchmarks. We believe that MIR will encourage further research into multi-image interleaved reasoning, facilitating advancements in MLLMs capability to handle complex inter-modal tasks.

Related papers

• Query-Kontext: An Unified Multimodal Model for Image Generation and Editing [53.765351127477224]
  Unified Multimodal Models (UMMs) have demonstrated remarkable performance in text-to-image generation (T2I) and editing (TI2I)<n>We introduce Query-Kontext, a novel approach that bridges the VLM and diffusion model via a multimodal kontext'' composed of semantic cues and coarse-grained image conditions encoded from multimodal inputs.<n> Experiments show that our approach matches strong unified baselines and even outperforms task-specific state-of-the-art methods in several cases.
  arXiv  Detail & Related papers  (2025-09-30T17:59:46Z)
• Human-centered Interactive Learning via MLLMs for Text-to-Image Person Re-identification [26.689580621314576]
  We propose an Interactive Cross-modal Learning framework (ICL) to enhance the discriminability of text queries through external multimodal knowledge.<n>To achieve this, we propose a plug-and-play Test-time Humane-centered Interaction (THI) module.<n> THI refines user queries based on the multimodal large language model (MLLM) to reduce the gap to the best-matching images.
  arXiv  Detail & Related papers  (2025-05-21T02:26:17Z)
• CoLLM: A Large Language Model for Composed Image Retrieval [76.29725148964368]
  Composed Image Retrieval (CIR) is a complex task that aims to retrieve images based on a multimodal query.<n>We present CoLLM, a one-stop framework that generates triplets on-the-fly from image-caption pairs.<n>We leverage Large Language Models (LLMs) to generate joint embeddings of reference images and modification texts.
  arXiv  Detail & Related papers  (2025-03-25T17:59:50Z)
• VOILA: Evaluation of MLLMs For Perceptual Understanding and Analogical Reasoning [63.0285363282581]
  Multimodal Large Language Models (MLLMs) have become a powerful tool for integrating visual and textual information.<n>We introduce VOILA, a benchmark designed to evaluate MLLMs' perceptual understanding and abstract relational reasoning.<n>We reveal that current MLLMs struggle to comprehend inter-image relationships and exhibit limited capabilities in high-level relational reasoning.
  arXiv  Detail & Related papers  (2025-02-25T23:36:19Z)
• Towards Text-Image Interleaved Retrieval [49.96332254241075]
  We introduce the text-image interleaved retrieval (TIIR) task, where the query and document are interleaved text-image sequences.<n>We construct a TIIR benchmark based on naturally interleaved wikiHow tutorials, where a specific pipeline is designed to generate interleaved queries.<n>We propose a novel Matryoshka Multimodal Embedder (MME), which compresses the number of visual tokens at different granularity.
  arXiv  Detail & Related papers  (2025-02-18T12:00:47Z)
• SMIR: Efficient Synthetic Data Pipeline To Improve Multi-Image Reasoning [26.986638043619397]
  We introduce SMiR, a synthetic data-generation pipeline for multi-image reasoning.<n>We produce 160K synthetic training samples, offering a cost-effective alternative to closed-source solutions.<n>We also present SMiR-Bench, a multi-image reasoning benchmark comprising 200 diverse examples.
  arXiv  Detail & Related papers  (2025-01-07T10:21:21Z)
• Compositional Image Retrieval via Instruction-Aware Contrastive Learning [40.54022628032561]
  Composed Image Retrieval (CIR) involves retrieving a target image based on a composed query of an image paired with text that specifies modifications or changes to the visual reference.<n>In practice, due to the scarcity of annotated data in downstream tasks, Zero-Shot CIR (ZS-CIR) is desirable.<n>We propose a novel embedding method utilizing an instruction-tuned Multimodal LLM (MLLM) to generate composed representation.
  arXiv  Detail & Related papers  (2024-12-07T22:46:52Z)
• MC-Bench: A Benchmark for Multi-Context Visual Grounding in the Era of MLLMs [61.56904387052982]
  This paper proposes a new visual grounding task called multi-context visual grounding.<n>It aims to localize instances of interest across multiple images based on open-ended text prompts.<n>We benchmark over 20 state-of-the-art MLLMs and foundation models with potential multi-context visual grounding capabilities.
  arXiv  Detail & Related papers  (2024-10-16T07:52:57Z)
• MMRA: A Benchmark for Evaluating Multi-Granularity and Multi-Image Relational Association Capabilities in Large Visual Language Models [29.795942154703642]
  We propose the multi-image relation association task and a meticulously curated Multi-granularity Multi-image Association benchmark. Our experiments reveal that on the MMRA benchmark, current multi-image LVLMs exhibit distinct advantages and disadvantages across various subtasks. Our findings indicate that while LVLMs demonstrate a strong capability to perceive image details, enhancing their ability to associate information across multiple images hinges on improving the reasoning capabilities of their language model component.
  arXiv  Detail & Related papers  (2024-07-24T15:59:01Z)
• Browse and Concentrate: Comprehending Multimodal Content via prior-LLM Context Fusion [70.9767518332692]
  Multimodal Large Language Models (MLLMs) that incorporate LLMs with pre-trained vision models have recently demonstrated impressive performance across diverse vision-language tasks. However, they fall short to comprehend context involving multiple images. We propose a two phase paradigm, browse-and-concentrate, to enable in-depth multimodal context fusion.
  arXiv  Detail & Related papers  (2024-02-19T14:59:07Z)
• MMICL: Empowering Vision-language Model with Multi-Modal In-Context Learning [42.68425777473114]
  Vision-language models (VLMs) enhanced by large language models (LLMs) have grown exponentially in popularity. We introduce vision-language Model with Multi-Modal In-Context Learning (MMICL), a new approach to allow the VLM to deal with multi-modal inputs efficiently. Our experiments confirm that MMICL achieves new state-of-the-art zero-shot performance on a wide range of general vision-language tasks.
  arXiv  Detail & Related papers  (2023-09-14T17:59:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.