Enhancing Multi-Image Understanding through Delimiter Token Scaling

• URL: http://arxiv.org/abs/2602.01984v1
• Date: Mon, 02 Feb 2026 11:38:01 GMT
• Title: Enhancing Multi-Image Understanding through Delimiter Token Scaling
• Authors: Minyoung Lee, Yeji Park, Dongjun Hwang, Yejin Kim, Seong Joon Oh, Junsuk Choe,
• Abstract summary: Large Vision-Language Models (LVLMs) achieve strong performance on single-image tasks, but their performance declines when multiple images are provided as input.<n>One major reason is the cross-image information leakage, where the model struggles to distinguish information across different images.<n>Existing LVLMs already employ tokens to mark the start and end of each image, yet our analysis reveals that these tokens fail to effectively block cross-image information leakage.<n>We propose a method that scales the hidden states of tokens. This enhances the model's ability to preserve image-specific information by reinforcing intra-image interaction and limiting undesired cross-
• Score: 25.247506519133406
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large Vision-Language Models (LVLMs) achieve strong performance on single-image tasks, but their performance declines when multiple images are provided as input. One major reason is the cross-image information leakage, where the model struggles to distinguish information across different images. Existing LVLMs already employ delimiter tokens to mark the start and end of each image, yet our analysis reveals that these tokens fail to effectively block cross-image information leakage. To enhance their effectiveness, we propose a method that scales the hidden states of delimiter tokens. This enhances the model's ability to preserve image-specific information by reinforcing intra-image interaction and limiting undesired cross-image interactions. Consequently, the model is better able to distinguish between images and reason over them more accurately. Experiments show performance gains on multi-image benchmarks such as Mantis, MuirBench, MIRB, and QBench2. We further evaluate our method on text-only tasks that require clear distinction. The method improves performance on multi-document and multi-table understanding benchmarks, including TQABench, MultiNews, and WCEP-10. Notably, our method requires no additional training or inference cost.

Related papers

• More Images, More Problems? A Controlled Analysis of VLM Failure Modes [80.64323947730905]
  Large Vision Language Models (LVLMs) have demonstrated remarkable capabilities, yet their proficiency in understanding and reasoning over multiple images remains largely unexplored.<n>We introduce MIMIC, a new benchmark designed to rigorously evaluate the multi-image capabilities of LVLMs.
  arXiv  Detail & Related papers  (2026-01-12T18:45:13Z)
• Mitigating Cross-Image Information Leakage in LVLMs for Multi-Image Tasks [22.873506907783153]
  We propose a training-free and architecture-agnostic decoding strategy that mitigates cross-image information leakage during inference.<n> FOCUS consistently improves performance across four multi-image benchmarks and diverse LVLM families.<n>This demonstrates that FOCUS offers a general and practical solution for enhancing multi-image reasoning without additional training or architectural modifications.
  arXiv  Detail & Related papers  (2025-08-19T11:31:39Z)
• CATP: Contextually Adaptive Token Pruning for Efficient and Enhanced Multimodal In-Context Learning [15.733788584792388]
  We propose Contextually Adaptive Token Pruning (CATP), a training-free pruning method targeted at multimodal in-context learning (ICL)<n>After removing 77.8% of the image tokens, CATP produces an average performance gain of 0.6% over the vanilla model on four LVLMs and eight benchmarks.<n>It effectively improves efficiency by achieving an average reduction of 10.78% in latency.
  arXiv  Detail & Related papers  (2025-08-11T11:41:51Z)
• ToFu: Visual Tokens Reduction via Fusion for Multi-modal, Multi-patch, Multi-image Task [34.269081635534526]
  We propose ToFu, a visual encoder-agnostic, training-free Token Fusion strategy for high-resolution, multi-image, tasks.<n>We validate our approach on the well-established LLaVA-Interleave Bench, which covers challenging multi-image tasks.
  arXiv  Detail & Related papers  (2025-03-06T14:00:59Z)
• Towards Text-Image Interleaved Retrieval [49.96332254241075]
  We introduce the text-image interleaved retrieval (TIIR) task, where the query and document are interleaved text-image sequences.<n>We construct a TIIR benchmark based on naturally interleaved wikiHow tutorials, where a specific pipeline is designed to generate interleaved queries.<n>We propose a novel Matryoshka Multimodal Embedder (MME), which compresses the number of visual tokens at different granularity.
  arXiv  Detail & Related papers  (2025-02-18T12:00:47Z)
• LoTLIP: Improving Language-Image Pre-training for Long Text Understanding [71.04947115945349]
  We relabel the data with long captions, however, directly learning with which may lead to performance degradation in understanding short text. We then help the model catch up to its original level of short text understanding yet greatly enhance its capability of long text understanding. Our method demonstrates superior performance in long-text-image retrieval tasks.
  arXiv  Detail & Related papers  (2024-10-07T17:52:56Z)
• Token-level Correlation-guided Compression for Efficient Multimodal Document Understanding [54.532578213126065]
  Most document understanding methods preserve all tokens within sub-images and treat them equally. This neglects their different informativeness and leads to a significant increase in the number of image tokens. We propose Token-level Correlation-guided Compression, a parameter-free and plug-and-play methodology to optimize token processing.
  arXiv  Detail & Related papers  (2024-07-19T16:11:15Z)
• MVAM: Multi-View Attention Method for Fine-grained Image-Text Matching [65.87255122130188]
  We propose a Multi-view Attention Method (MVAM) for image-text matching.<n>We also incorporate an objective to explicitly encourage attention heads to focus on distinct aspects of the input data.<n>Our method allows models to encode images and text from different perspectives and focus on more critical details, leading to better matching performance.
  arXiv  Detail & Related papers  (2024-02-27T06:11:54Z)
• Align before Fuse: Vision and Language Representation Learning with Momentum Distillation [52.40490994871753]
  We introduce a contrastive loss to representations BEfore Fusing (ALBEF) through cross-modal attention. We propose momentum distillation, a self-training method which learns from pseudo-targets produced by a momentum model. ALBEF achieves state-of-the-art performance on multiple downstream vision-language tasks.
  arXiv  Detail & Related papers  (2021-07-16T00:19:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.