DiCoBench: Benchmarking Multi-Image Fine-Grained Perception via Differential and Commonality Visual Cues
- URL: http://arxiv.org/abs/2606.26602v1
- Date: Thu, 25 Jun 2026 05:02:38 GMT
- Title: DiCoBench: Benchmarking Multi-Image Fine-Grained Perception via Differential and Commonality Visual Cues
- Abstract summary: DiCoBench is a comprehensive, multi-image high-resolution benchmark for fine-grained perception.<n>Our evaluation reveals a striking performance gap compared to human accuracy (98.3%)<n>We believe DiCoBench will serve as a challenging testbed to drive future research in autonomous, high-resolution multi-image perception.
- Score: 54.31138496553705
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Recent advancements in Multimodal Large Language Models (MLLMs) have demonstrated impressive fine-grained perception capabilities. However, existing benchmarks predominantly rely on explicit textual cues or low-resolution inputs, failing to evaluate a model's ability to autonomously perceive implicit visual cues in high-resolution. To bridge this gap, we introduce DiCoBench, a comprehensive, multi-image high-resolution benchmark designed for cross-image fine-grained perception. DiCoBench consists of 765 meticulously curated samples categorized into two progressive tracks: Differential Visual Cues and Commonality Visual Cues, covering 8 distinct perception tasks. By formulating the benchmark as a multiple-choice question task and utilizing high-resolution imagery (approaching 2K), we eliminate evaluation metric bias and pose a substantial challenge to current state-of-the-art MLLMs. Our extensive evaluation of 18 diverse MLLMs reveals a striking performance gap compared to human accuracy (98.3\%), with top-performing models struggling significantly with micro-scale detail capture. We believe DiCoBench will serve as a challenging testbed to drive future research in autonomous, high-resolution multi-image perception.
Related papers
- WorldBench: A Challenging and Visually Diverse Multimodal Reasoning Benchmark [85.74548332724977]
We build a taxonomy of thousands of visual concepts across multiple domains.<n>We manually design challenging questions that frontier MLLMs fail to answer.<n>WorldBench achieves higher visual diversity than any existing diverse benchmark.
arXiv Detail & Related papers (2026-06-04T01:11:21Z) - DiffCap-Bench: A Comprehensive, Challenging, Robust Benchmark for Image Difference Captioning [26.88130913151649]
Image Difference Captioning (IDC) generates language descriptions that precisely identify differences between two images.<n>We introduce DiffCap-Bench, a comprehensive IDC benchmark covering ten distinct difference categories.<n>We also propose an LLM-as-a-Judge evaluation protocol grounded in human-validated Difference Lists.
arXiv Detail & Related papers (2026-05-06T05:12:41Z) - Indexing Multimodal Language Models for Large-scale Image Retrieval [15.19064519873711]
Multimodal Large Language Models (MLLMs) have demonstrated strong cross-modal reasoning capabilities, yet their potential for vision-only tasks remains underexplored.<n>We investigate MLLMs as training-free similarity estimators for instance-level image-to-image retrieval.<n>Our approach prompts the model with paired images and converts next-token probabilities into similarity scores, enabling zero-shot re-ranking within large-scale retrieval pipelines.
arXiv Detail & Related papers (2026-04-14T19:59:36Z) - MICON-Bench: Benchmarking and Enhancing Multi-Image Context Image Generation in Unified Multimodal Models [89.89575486159795]
We introduce textbfMICON-Bench, a benchmark for multi-image context generation.<n>We propose an MLLM-driven Evaluation-by-Checkpoint framework for automatic verification of semantic and visual consistency.<n>We also present textbfDynamic Attention Rebalancing (DAR), a training-free, plug-and-play mechanism that dynamically adjusts attention during inference to enhance coherence and reduce hallucinations.
arXiv Detail & Related papers (2026-02-23T04:32:52Z) - More Images, More Problems? A Controlled Analysis of VLM Failure Modes [80.64323947730905]
Large Vision Language Models (LVLMs) have demonstrated remarkable capabilities, yet their proficiency in understanding and reasoning over multiple images remains largely unexplored.<n>We introduce MIMIC, a new benchmark designed to rigorously evaluate the multi-image capabilities of LVLMs.
arXiv Detail & Related papers (2026-01-12T18:45:13Z) - QG-CoC: Question-Guided Chain-of-Captions for Large Multimodal Models [50.51641024244313]
We investigate how current prompting methods perceive fine-grained visual details and process visual information when dealing with multiple images.<n>Inspired by the findings, we propose a new zero-shot prompting method, Question-Guided Chain-of-Captions (QG-CoC)<n>We evaluate our method on various open-source and closed-source MLLMs for multi-image and single-image benchmarks.
arXiv Detail & Related papers (2025-11-05T05:49:48Z) - Calibrating MLLM-as-a-judge via Multimodal Bayesian Prompt Ensembles [20.7718577645105]
Multimodal large language models (MLLMs) are increasingly used to evaluate text-to-image (TTI) generation systems.<n>These "judge" models often suffer from biases, overconfidence, and inconsistent performance across diverse image domains.<n>We propose a new multimodal-aware method called Multimodal Mixture-of-Bayesian Prompt Ensembles (MMB)
arXiv Detail & Related papers (2025-09-10T17:06:47Z) - VOILA: Evaluation of MLLMs For Perceptual Understanding and Analogical Reasoning [63.0285363282581]
Multimodal Large Language Models (MLLMs) have become a powerful tool for integrating visual and textual information.<n>We introduce VOILA, a benchmark designed to evaluate MLLMs' perceptual understanding and abstract relational reasoning.<n>We reveal that current MLLMs struggle to comprehend inter-image relationships and exhibit limited capabilities in high-level relational reasoning.
arXiv Detail & Related papers (2025-02-25T23:36:19Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.