CARES: Context-Aware Resolution Selector for VLMs

• URL: http://arxiv.org/abs/2510.19496v1
• Date: Wed, 22 Oct 2025 11:44:31 GMT
• Title: CARES: Context-Aware Resolution Selector for VLMs
• Abstract summary: Large vision-language models (VLMs) commonly process images at native or high resolution to remain effective across tasks.<n>We introduce emphCARES, a lightweight preprocessing module that predicts the emphminimal sufficient input resolution.<n> CARES preserves task performance while reducing compute by up to 80%.
• Score: 29.734101330721263
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large vision-language models (VLMs) commonly process images at native or high resolution to remain effective across tasks. This inflates visual tokens ofter to 97-99% of total tokens, resulting in high compute and latency, even when low-resolution images would suffice. We introduce \emph{CARES}-a \textbf{C}ontext-\textbf{A}ware \textbf{R}esolution \textbf{S}elector, a lightweight preprocessing module that, given an image-query pair, predicts the \emph{minimal} sufficient input resolution. CARES uses a compact VLM (350M) to extract features and predict when a target pretrained VLM's response converges to its peak ability to answer correctly. Though trained as a discrete classifier over a set of optional resolutions, CARES interpolates continuous resolutions at inference for fine-grained control. Across five multimodal benchmarks spanning documents and natural images, as well as diverse target VLMs, CARES preserves task performance while reducing compute by up to 80%.

Related papers

• MAViE: A Multi-scale Adaptive Vision Encoder for Fine-grained Visual Perception and Efficient Multimodal Reasoning [0.0]
  method fuses shallow, intermediate, and deep features from a vision Transformer to preserve global semantics.<n>It then performs question-conditioned token routing according to question relevance, local information content, global semantics, and spatial coverage.<n>It reduces the average number of SigLIP-SO400M visual tokens from 729 to 146 (approximately 80.0%) and improves the mean score on VQAv2, GQA, TextVQA, ScienceQA-IMG, and MMBench by 2.2 percentage points.
  arXiv  Detail & Related papers  (2026-07-27T13:36:08Z)
• Self-supervised pretraining for an iterative image size agnostic vision transformer [60.09702846704075]
  Vision Transformers (ViTs) dominate self-supervised learning (SSL)<n>We introduce a novel sequential-to-global SSL framework based on DINO's self-distillation objective.<n>We achieve competitive performance on ImageNet-1K and downstream classification tasks.
  arXiv  Detail & Related papers  (2026-04-22T09:53:28Z)
• ViCO: A Training Strategy towards Semantic Aware Dynamic High-Resolution [71.69364653858447]
  Existing Multimodal Large Language Models (MLLMs) suffer from increased inference costs due to the additional vision tokens introduced by image inputs.<n>We propose Visual Consistency Learning (ViCO), a novel training algorithm that enables the model to represent images of varying complexities using different numbers of vision tokens.<n> Experimental results demonstrate that our method can reduce the number of vision tokens by up to 50% while maintaining the model's perception, reasoning, and OCR capabilities.
  arXiv  Detail & Related papers  (2025-10-14T17:58:10Z)
• Mitigating Cross-Image Information Leakage in LVLMs for Multi-Image Tasks [22.873506907783153]
  We propose a training-free and architecture-agnostic decoding strategy that mitigates cross-image information leakage during inference.<n> FOCUS consistently improves performance across four multi-image benchmarks and diverse LVLM families.<n>This demonstrates that FOCUS offers a general and practical solution for enhancing multi-image reasoning without additional training or architectural modifications.
  arXiv  Detail & Related papers  (2025-08-19T11:31:39Z)
• VisionThink: Smart and Efficient Vision Language Model via Reinforcement Learning [95.89543460132413]
  Vision-language models (VLMs) have improved performance by increasing the number of visual tokens.<n>However, most real-world scenarios do not require such an extensive number of visual tokens.<n>We present a new paradigm for visual token compression, namely, VisionThink.
  arXiv  Detail & Related papers  (2025-07-17T17:59:55Z)
• A Training-Free, Task-Agnostic Framework for Enhancing MLLM Performance on High-Resolution Images [19.549498712690404]
  Multimodal Large Language Models (MLLMs) have demonstrated remarkable capabilities in vision-language understanding, reasoning, and generation.<n>They struggle with tasks requiring fine-grained localization and reasoning in high-resolution images.<n>We propose Extract Candidate then Predict (ECP), a novel training-free, task-agnostic two-stage framework designed to enhance MLLM performance on high-resolution images.
  arXiv  Detail & Related papers  (2025-07-14T12:14:53Z)
• HMAR: Efficient Hierarchical Masked Auto-Regressive Image Generation [91.08481618973111]
  Visual Auto-Regressive modeling ( VAR) has shown promise in bridging the speed and quality gap between autoregressive image models and diffusion models.<n>We introduce Hierarchical Masked Auto-Regressive modeling (HMAR) to generate high-quality images with fast sampling.<n>HMAR reformulates next-scale prediction as a Markovian process, wherein the prediction of each resolution scale is conditioned only on tokens in its immediate predecessor.
  arXiv  Detail & Related papers  (2025-06-04T20:08:07Z)
• DyMU: Dynamic Merging and Virtual Unmerging for Efficient VLMs [124.52164183968145]
  We present DyMU, an efficient, training-free framework that reduces the computational burden of vision-language models (VLMs)<n>Our approach comprises two key components. First, Dynamic Token Merging (DToMe) reduces the number of visual token embeddings by merging similar tokens based on image complexity.<n>Second, Virtual Token Unmerging (VTU) simulates the expected token sequence for large language models (LLMs) by efficiently reconstructing the attention dynamics of a full sequence.
  arXiv  Detail & Related papers  (2025-04-23T18:38:18Z)
• When Large Vision-Language Model Meets Large Remote Sensing Imagery: Coarse-to-Fine Text-Guided Token Pruning [31.696397337675847]
  Large Vision-Language Models (LVLMs) typically employ limited pre-defined grids to process images.<n>We propose a text-guided token pruning method with Dynamic Image Pyramid (DIP) integration.<n>Our method outperforms existing high-resolution strategies on four datasets using the same data.
  arXiv  Detail & Related papers  (2025-03-10T17:51:16Z)
• Memory Efficient Matting with Adaptive Token Routing [73.09131141304984]
  Transformer-based models have recently achieved outstanding performance in image matting.<n>MeMatte is a textbfmemory-textbfefficient textbfmatting framework for processing high-resolution images.
  arXiv  Detail & Related papers  (2024-12-14T06:21:24Z)
• ConvLLaVA: Hierarchical Backbones as Visual Encoder for Large Multimodal Models [77.59651787115546]
  High-resolution Large Multimodal Models (LMMs) encounter the challenges of excessive visual tokens and quadratic visual complexity. We propose ConvLLaVA, which employs ConvNeXt, a hierarchical backbone, as the visual encoder of LMM. ConvLLaVA compresses high-resolution images into information-rich visual features, effectively preventing the generation of excessive visual tokens.
  arXiv  Detail & Related papers  (2024-05-24T17:34:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.