Mitigating Cross-Image Information Leakage in LVLMs for Multi-Image Tasks

• URL: http://arxiv.org/abs/2508.13744v1
• Date: Tue, 19 Aug 2025 11:31:39 GMT
• Title: Mitigating Cross-Image Information Leakage in LVLMs for Multi-Image Tasks
• Authors: Yeji Park, Minyoung Lee, Sanghyuk Chun, Junsuk Choe,
• Abstract summary: We propose a training-free and architecture-agnostic decoding strategy that mitigates cross-image information leakage during inference.<n> FOCUS consistently improves performance across four multi-image benchmarks and diverse LVLM families.<n>This demonstrates that FOCUS offers a general and practical solution for enhancing multi-image reasoning without additional training or architectural modifications.
• Score: 22.873506907783153
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Vision-Language Models (LVLMs) demonstrate strong performance on single-image tasks. However, we observe that their performance degrades significantly when handling multi-image inputs. This occurs because visual cues from different images become entangled in the model's output. We refer to this phenomenon as cross-image information leakage. To address this issue, we propose FOCUS, a training-free and architecture-agnostic decoding strategy that mitigates cross-image information leakage during inference. FOCUS sequentially masks all but one image with random noise, guiding the model to focus on the single clean image. We repeat this process across all target images to obtain logits under partially masked contexts. These logits are aggregated and then contrastively refined using a noise-only reference input, which suppresses the leakage and yields more accurate outputs. FOCUS consistently improves performance across four multi-image benchmarks and diverse LVLM families. This demonstrates that FOCUS offers a general and practical solution for enhancing multi-image reasoning without additional training or architectural modifications.

Related papers

• Enhancing Multi-Image Understanding through Delimiter Token Scaling [25.247506519133406]
  Large Vision-Language Models (LVLMs) achieve strong performance on single-image tasks, but their performance declines when multiple images are provided as input.<n>One major reason is the cross-image information leakage, where the model struggles to distinguish information across different images.<n>Existing LVLMs already employ tokens to mark the start and end of each image, yet our analysis reveals that these tokens fail to effectively block cross-image information leakage.<n>We propose a method that scales the hidden states of tokens. This enhances the model's ability to preserve image-specific information by reinforcing intra-image interaction and limiting undesired cross-
  arXiv  Detail & Related papers  (2026-02-02T11:38:01Z)
• More Images, More Problems? A Controlled Analysis of VLM Failure Modes [80.64323947730905]
  Large Vision Language Models (LVLMs) have demonstrated remarkable capabilities, yet their proficiency in understanding and reasoning over multiple images remains largely unexplored.<n>We introduce MIMIC, a new benchmark designed to rigorously evaluate the multi-image capabilities of LVLMs.
  arXiv  Detail & Related papers  (2026-01-12T18:45:13Z)
• ViCO: A Training Strategy towards Semantic Aware Dynamic High-Resolution [71.69364653858447]
  Existing Multimodal Large Language Models (MLLMs) suffer from increased inference costs due to the additional vision tokens introduced by image inputs.<n>We propose Visual Consistency Learning (ViCO), a novel training algorithm that enables the model to represent images of varying complexities using different numbers of vision tokens.<n> Experimental results demonstrate that our method can reduce the number of vision tokens by up to 50% while maintaining the model's perception, reasoning, and OCR capabilities.
  arXiv  Detail & Related papers  (2025-10-14T17:58:10Z)
• Learning Only with Images: Visual Reinforcement Learning with Reasoning, Rendering, and Visual Feedback [33.127607245587576]
  We introduce a framework that enables MLLMs to learn complex visual reasoning from only raw images.<n>We demonstrate that this relative ease provides an ideal reward signal for optimization via Reinforcement Learning.<n>The RRVF-trained model not only outperforms existing MLLMs and supervised fine-tuning baselines but also exhibits superior generalization.
  arXiv  Detail & Related papers  (2025-07-28T12:21:19Z)
• Language Integration in Fine-Tuning Multimodal Large Language Models for Image-Based Regression [2.9998889086656586]
  We show how Transformer-Based Classification (RvTC) replaces vocabulary-constrained classification with a flexible bin-based approach.<n>Unlike generic task descriptions, prompts containing semantic information about specific images enable MLLMs to leverage cross-modal understanding.
  arXiv  Detail & Related papers  (2025-07-20T15:05:24Z)
• EasyRef: Omni-Generalized Group Image Reference for Diffusion Models via Multimodal LLM [38.8308841469793]
  This paper introduces EasyRef, a novel plug-and-play adaptation method that enables diffusion models to be conditioned on multiple reference images and the text prompt.<n>We leverage the multi-image comprehension and instruction-following capabilities of the multimodal large language model (MLLM) to exploit consistent visual elements within multiple images.<n> Experimental results demonstrate EasyRef surpasses both tuning-free methods like IP-Adapter and tuning-based methods like LoRA, achieving superior aesthetic quality and robust zero-shot generalization across diverse domains.
  arXiv  Detail & Related papers  (2024-12-12T18:59:48Z)
• CricaVPR: Cross-image Correlation-aware Representation Learning for Visual Place Recognition [73.51329037954866]
  We propose a robust global representation method with cross-image correlation awareness for visual place recognition. Our method uses the attention mechanism to correlate multiple images within a batch. Our method outperforms state-of-the-art methods by a large margin with significantly less training time.
  arXiv  Detail & Related papers  (2024-02-29T15:05:11Z)
• Jack of All Tasks, Master of Many: Designing General-purpose Coarse-to-Fine Vision-Language Model [83.85856356798531]
  VistaLLM is a visual system that addresses coarse- and fine-grained vision-language tasks. It employs a gradient-aware adaptive sampling technique to represent binary segmentation masks as sequences. We also introduce a novel task, AttCoSeg, which boosts the model's reasoning and grounding capability over multiple input images.
  arXiv  Detail & Related papers  (2023-12-19T18:53:01Z)
• Viewpoint Integration and Registration with Vision Language Foundation Model for Image Change Understanding [15.392243642628387]
  We show that existing vision language foundation models (VLFMs) perform poorly when applied directly to image change understanding (ICU) ICU requires models to capture actual changes between multiple images and describe them in language. We propose a Viewpoint Integration and Registration method to address these problems.
  arXiv  Detail & Related papers  (2023-09-15T17:41:29Z)
• Learning to Mask and Permute Visual Tokens for Vision Transformer Pre-Training [55.12082817901671]
  We propose a new self-supervised pre-training approach, named Masked and Permuted Vision Transformer (MaPeT)<n>MaPeT employs autoregressive and permuted predictions to capture intra-patch dependencies.<n>Our results demonstrate that MaPeT achieves competitive performance on ImageNet, compared to baselines and competitors under the same model setting.
  arXiv  Detail & Related papers  (2023-06-12T18:12:19Z)
• Improving Masked Autoencoders by Learning Where to Mask [65.89510231743692]
  Masked image modeling is a promising self-supervised learning method for visual data. We present AutoMAE, a framework that uses Gumbel-Softmax to interlink an adversarially-trained mask generator and a mask-guided image modeling process. In our experiments, AutoMAE is shown to provide effective pretraining models on standard self-supervised benchmarks and downstream tasks.
  arXiv  Detail & Related papers  (2023-03-12T05:28:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.