Q-Zoom: Query-Aware Adaptive Perception for Efficient Multimodal Large Language Models

• URL: http://arxiv.org/abs/2604.06912v1
• Date: Wed, 08 Apr 2026 10:12:30 GMT
• Title: Q-Zoom: Query-Aware Adaptive Perception for Efficient Multimodal Large Language Models
• Abstract summary: Q-Zoom is a query-aware adaptive high-resolution perception framework.<n>A lightweight Gating Network safely bypasses high-resolution processing when coarse global features suffice.<n>A Self-Distilled Region Network precisely localizes the task-relevant Region-of-Interest from intermediate feature spaces.
• Score: 41.32939197803798
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: MLLMs require high-resolution visual inputs for fine-grained tasks like document understanding and dense scene perception. However, current global resolution scaling paradigms indiscriminately flood the quadratic self-attention mechanism with visually redundant tokens, severely bottlenecking inference throughput while ignoring spatial sparsity and query intent. To overcome this, we propose Q-Zoom, a query-aware adaptive high-resolution perception framework that operates in an efficient coarse-to-fine manner. First, a lightweight Dynamic Gating Network safely bypasses high-resolution processing when coarse global features suffice. Second, for queries demanding fine-grained perception, a Self-Distilled Region Proposal Network (SD-RPN) precisely localizes the task-relevant Region-of-Interest (RoI) directly from intermediate feature spaces. To optimize these modules efficiently, the gating network uses a consistency-aware generation strategy to derive deterministic routing labels, while the SD-RPN employs a fully self-supervised distillation paradigm. A continuous spatio-temporal alignment scheme and targeted fine-tuning then seamlessly fuse the dense local RoI with the coarse global layout. Extensive experiments demonstrate that Q-Zoom establishes a dominant Pareto frontier. Using Qwen2.5-VL-7B as a primary testbed, Q-Zoom accelerates inference by 2.52 times on Document & OCR benchmarks and 4.39 times in High-Resolution scenarios while matching the baseline's peak accuracy. Furthermore, when configured for maximum perceptual fidelity, Q-Zoom surpasses the baseline's peak performance by 1.1% and 8.1% on these respective benchmarks. These robust improvements transfer seamlessly to Qwen3-VL, LLaVA, and emerging RL-based thinking-with-image models. Project page is available at https://yuhengsss.github.io/Q-Zoom/.

Related papers

• TRaM-VSR: Importance-Aware Token Routing and Merging for One-Step Diffusion Video Super-Resolution [52.96050253456634]
  TRaM-VSR is a framework for adaptive token allocation using context-aware video priors and network-levels.<n>It is shown that TRaM-VSR accelerates inference significantly while preserving state-of-the-art reconstruction quality and robust temporal consistency.
  arXiv  Detail & Related papers  (2026-07-24T11:57:26Z)
• ReHARK: Refined Hybrid Adaptive RBF Kernels for Robust One-Shot Vision-Language Adaptation [3.8707695363745214]
  ReHARK is a training-free framework that reinterprets few-shot adaptation through global proximal regularization.<n>A new state-of-the-art for one-shot adaptation is established by ReHARK, which achieves an average accuracy of 65.83%.
  arXiv  Detail & Related papers  (2026-03-12T04:59:09Z)
• SMR-Net:Robot Snap Detection Based on Multi-Scale Features and Self-Attention Network [0.0]
  Traditional visual methods suffer from poor robustness and large localization errors when handling complex scenarios.<n>This paper proposes SMR-Net, a self-attention-based multi-scale object detection algorithm.<n> Experimental results on Type A and Type B snap datasets show SMR-Net outperforms traditional Faster R-CNN significantly.
  arXiv  Detail & Related papers  (2026-03-01T10:28:01Z)
• AdaSpot: Spend Resolution Where It Matters for Precise Event Spotting [59.31340724915079]
  Event Spotting is a key task for applications in sports analytics, robotics, and autonomous systems.<n>bfAdaSpot achieves state-of-the-art performance under strict evaluation metrics.
  arXiv  Detail & Related papers  (2026-02-25T16:24:48Z)
• Real-Time LiDAR Super-Resolution via Frequency-Aware Multi-Scale Fusion [0.4078247440919472]
  FLASH (Frequency-aware LiDAR Adaptive Super-resolution with Hierarchical fusion) is a novel framework that overcomes limitations through dual-domain processing.<n> FLASH integrates two key innovations: (i) Frequency-Aware Window Attention that combines local spatial attention with global frequency-domain analysis via FFT, capturing both fine-grained geometry and periodic scanning patterns at log-linear complexity, and (ii) Adaptive Multi-Scale Fusion that replaces conventional skip connections with learned position-specific feature aggregation, enhanced by CBAM attention for dynamic feature selection.
  arXiv  Detail & Related papers  (2025-11-10T18:38:15Z)
• Scale-DiT: Ultra-High-Resolution Image Generation with Hierarchical Local Attention [50.391914489898774]
  Scale-DiT is a new diffusion framework that introduces hierarchical local attention with low-resolution global guidance.<n>A lightweight LoRA adaptation bridges global and local pathways during denoising, ensuring consistency across structure and detail.<n>Experiments demonstrate that Scale-DiT achieves more than $2times$ faster inference and lower memory usage.
  arXiv  Detail & Related papers  (2025-10-18T03:15:26Z)
• Catching the Details: Self-Distilled RoI Predictors for Fine-Grained MLLM Perception [43.40677219802473]
  Multimodal Large Language Models (MLLMs) require high-resolution visual information to perform fine-grained perception.<n>Recent methods leverage a Region-of-Interest (RoI) mechanism to focus on salient areas.<n>We propose an efficient, annotation-free Self-Distilled Region Proposal Network (SD-RPN) that resolves this trade-off.
  arXiv  Detail & Related papers  (2025-09-21T06:54:04Z)
• Inter2Former: Dynamic Hybrid Attention for Efficient High-Precision Interactive [58.0729162588429]
  Interactive segmentation improves annotation efficiency by segmenting target regions from user prompts.<n>Current approaches face a critical trade-off: dense-token methods achieve superior accuracy but suffer from prohibitively slow processing on CPU devices.<n>We propose Inter2Former to address this challenge by optimizing computation allocation in dense-token processing.
  arXiv  Detail & Related papers  (2025-07-13T12:33:37Z)
• Collaborative Intelligent Reflecting Surface Networks with Multi-Agent Reinforcement Learning [63.83425382922157]
  Intelligent reflecting surface (IRS) is envisioned to be widely applied in future wireless networks. In this paper, we investigate a multi-user communication system assisted by cooperative IRS devices with the capability of energy harvesting.
  arXiv  Detail & Related papers  (2022-03-26T20:37:14Z)
• Higher Performance Visual Tracking with Dual-Modal Localization [106.91097443275035]
  Visual Object Tracking (VOT) has synchronous needs for both robustness and accuracy. We propose a dual-modal framework for target localization, consisting of robust localization suppressingors via ONR and the accurate localization attending to the target center precisely via OFC.
  arXiv  Detail & Related papers  (2021-03-18T08:47:56Z)
• Spatial-Scale Aligned Network for Fine-Grained Recognition [42.71878867504503]
  Existing approaches for fine-grained visual recognition focus on learning marginal region-based representations. We propose the spatial-scale aligned network (SSANET) and implicitly address misalignments during the recognition process.
  arXiv  Detail & Related papers  (2020-01-05T11:12:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.