SPARC: Separating Perception And Reasoning Circuits for Test-time Scaling of VLMs

• URL: http://arxiv.org/abs/2602.06566v2
• Date: Tue, 10 Feb 2026 15:30:46 GMT
• Title: SPARC: Separating Perception And Reasoning Circuits for Test-time Scaling of VLMs
• Abstract summary: We introduce (Separating Perception And Reasoning Circuits) a modular framework that explicitly decouples visual perception from reasoning.<n>Inspired by sequential sensory-to-cognitive processing in the brain, it implements a two-stage pipeline where the model first performs explicit visual search to localize question-relevant regions, then conditions its reasoning on those regions to produce the final answer.<n>Across challenging visual reasoning benchmarks, it outperforms monolithic baselines and strong visual-grounding approaches.
• Score: 34.330422211810685
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Despite recent successes, test-time scaling - i.e., dynamically expanding the token budget during inference as needed - remains brittle for vision-language models (VLMs): unstructured chains-of-thought about images entangle perception and reasoning, leading to long, disorganized contexts where small perceptual mistakes may cascade into completely wrong answers. Moreover, expensive reinforcement learning with hand-crafted rewards is required to achieve good performance. Here, we introduce SPARC (Separating Perception And Reasoning Circuits), a modular framework that explicitly decouples visual perception from reasoning. Inspired by sequential sensory-to-cognitive processing in the brain, SPARC implements a two-stage pipeline where the model first performs explicit visual search to localize question-relevant regions, then conditions its reasoning on those regions to produce the final answer. This separation enables independent test-time scaling with asymmetric compute allocation (e.g., prioritizing perceptual processing under distribution shift), supports selective optimization (e.g., improving the perceptual stage alone when it is the bottleneck for end-to-end performance), and accommodates compressed contexts by running global search at lower image resolutions and allocating high-resolution processing only to selected regions, thereby reducing total visual tokens count and compute. Across challenging visual reasoning benchmarks, SPARC outperforms monolithic baselines and strong visual-grounding approaches. For instance, SPARC improves the accuracy of Qwen3VL-4B on the $V^*$ VQA benchmark by 6.7 percentage points, and it surpasses "thinking with images" by 4.6 points on a challenging OOD task despite requiring a 200$\times$ lower token budget.

Related papers

• ENCORE: Entropy-Guided Cropping and Attention Regularization for Robust Vision--Language Understanding [40.664306877870715]
  Vision-Language Models (VLMs) perform well on diverse vision-language tasks, but transformer-based visual encoders split images into fixed-resolution sub-images.<n>Existing methods only focus on the visual modality and fail to dynamically preserve the integrity of prompt-relevant regions.<n>We propose textbfENCORE, an entropy-guided framework with two components.
  arXiv  Detail & Related papers  (2026-08-24T09:00:02Z)
• Are All Tokens Necessary for Visual Place Recognition? An Empirical Study of Token Reduction for Efficient Inference [27.84020098999677]
  We present the first systematic benchmark of token reduction for efficient visual place recognition.<n>Our benchmark comprehensively evaluates representative token pruning, token merging, and hybrid pruning-merging methods.<n>Our work establishes a comprehensive foundation for future research on token-efficient VPR and efficient visual retrieval systems.
  arXiv  Detail & Related papers  (2026-07-17T02:13:19Z)
• Perceive-to-Reason: Decoupling Perception and Reasoning for Fine-Grained Visual Reasoning [73.40085256022111]
  We propose Perceive-to-Reason (P2R), a unified framework that formulates fine-grained visual reasoning as a two-stage process.<n>P2R localizes question-relevant evidence as a Perceiver, and then answers the question as a Reasoner based on the annotated image and cropped regions.<n>PRA-GRPO is a role-aware reinforcement learning strategy that alternates between perception-focused and reasoning-focused updates.
  arXiv  Detail & Related papers  (2026-07-01T17:24:26Z)
• P$^2$-DPO: Grounding Hallucination in Perceptual Processing via Calibration Direct Preference Optimization [55.74731799669337]
  Hallucination has recently garnered significant research attention in Large Vision-Language Models (LVLMs)<n>Direct Preference Optimization (DPO) aims to learn directly from the corrected preferences provided by humans.<n>Existing preference pairs are often vision-agnostic and their inherently off-policy nature limits their effectiveness in guiding model learning.<n>We propose Perceptual Processing Direct Preference Optimization (P$2$-DPO), a novel training paradigm in which the model generates and learns from its own preference pairs.
  arXiv  Detail & Related papers  (2026-06-02T09:22:53Z)
• Evading Visual Aphasia: Contrastive Adaptive Semantic Token Pruning for Vision-Language Models [52.78477729846771]
  We introduce COAST (COntrastive Adaptive Semantic Token Pruning), a training-free pruning framework that casts compression as adaptive semantic routing.<n> COAST consistently outperforms strong pruning baselines across token budgets and generalizes across multiple LVLM families.
  arXiv  Detail & Related papers  (2026-05-10T09:07:04Z)
• Q-Zoom: Query-Aware Adaptive Perception for Efficient Multimodal Large Language Models [41.32939197803798]
  Q-Zoom is a query-aware adaptive high-resolution perception framework.<n>A lightweight Gating Network safely bypasses high-resolution processing when coarse global features suffice.<n>A Self-Distilled Region Network precisely localizes the task-relevant Region-of-Interest from intermediate feature spaces.
  arXiv  Detail & Related papers  (2026-04-08T10:12:30Z)
• ResPrune: Text-Conditioned Subspace Reconstruction for Visual Token Pruning in Large Vision-Language Models [59.94664910790462]
  ResPrune is a training-free visual token pruning framework for large vision-language models.<n>It selects a compact yet informative subset of visual tokens.<n>It achieves effective reductions in computation, memory consumption, and inference latency.
  arXiv  Detail & Related papers  (2026-03-22T07:44:45Z)
• AdaSpot: Spend Resolution Where It Matters for Precise Event Spotting [59.31340724915079]
  Event Spotting is a key task for applications in sports analytics, robotics, and autonomous systems.<n>bfAdaSpot achieves state-of-the-art performance under strict evaluation metrics.
  arXiv  Detail & Related papers  (2026-02-25T16:24:48Z)
• Nüwa: Mending the Spatial Integrity Torn by VLM Token Pruning [82.39668822222386]
  Vision token pruning has proven to be an effective acceleration technique for the efficient Vision Language Model (VLM)<n>We propose $textNwa$, a two-stage token pruning framework that enables efficient feature aggregation while maintaining spatial integrity.<n>Experiments demonstrate that $textNwa$ achieves SOTA performance on multiple VQA benchmarks (from 94% to 95%) and yields substantial improvements on visual grounding tasks (from 7% to 47%)
  arXiv  Detail & Related papers  (2026-02-03T00:51:03Z)
• Head-Aware Visual Cropping: Enhancing Fine-Grained VQA with Attention-Guided Subimage [4.771792258699647]
  We propose textbfHead Aware Visual Cropping (HAVC), a training-free method that improves visual grounding by leveraging a selectively refined subset of attention heads.<n> experiments on multiple fine-grained VQA benchmarks demonstrate that HAVC consistently outperforms state-of-the-art cropping strategies.
  arXiv  Detail & Related papers  (2026-01-30T02:46:55Z)
• Look in the Middle: Structural Anchor Pruning for Scalable Visual RAG Indexing [11.750217913598057]
  Recent Vision-Language Models (e.g., ColPali) enable fine-grained Visual Document Retrieval (VDR) but incur prohibitive index vector size overheads.<n>We propose Structural Anchor Pruning (SAP), a training-free pruning method that identifies key visual patches from middle layers to achieve high performance compression.
  arXiv  Detail & Related papers  (2026-01-27T22:50:11Z)
• Pixel-Grounded Retrieval for Knowledgeable Large Multimodal Models [58.46663983451155]
  PixSearch is an end-to-end Segmenting Large Multimodal Model (LMM) that unifies region-level perception and retrieval-augmented reasoning.<n>During encoding, PixSearch emits search> tokens to trigger retrieval, selects query modalities (text, image, or region), and generates pixel-level masks that directly serve as visual queries.<n>On egocentric and entity-centric VQA benchmarks, PixSearch substantially improves factual consistency and generalization.
  arXiv  Detail & Related papers  (2026-01-27T00:46:08Z)
• CoV: Chain-of-View Prompting for Spatial Reasoning [64.77921266428824]
  Chain-of-View (CoV) prompting transforms a vision--language model into an active viewpoint reasoner.<n>We evaluate CoV on OpenEQA and obtain an average +11.56% improvement in LLM-Match, with a maximum gain of +13.62% on Qwen3-VL-Flash.
  arXiv  Detail & Related papers  (2026-01-08T17:59:42Z)
• VOGUE: Guiding Exploration with Visual Uncertainty Improves Multimodal Reasoning [62.09195763860549]
  Reinforcement learning with verifiable rewards (RLVR) improves reasoning in large language models (LLMs) but struggles with exploration.<n>We introduce $textbfVOGUE (Visual Uncertainty Guided Exploration)$, a novel method that shifts exploration from the output (text) to the input (visual) space.<n>Our work shows that grounding exploration in the inherent uncertainty of visual inputs is an effective strategy for improving multimodal reasoning.
  arXiv  Detail & Related papers  (2025-10-01T20:32:08Z)
• ERGO: Efficient High-Resolution Visual Understanding for Vision-Language Models [11.263321053154364]
  ERGO is a reasoning-driven perception-leveraging multimodal context to determine where to focus.<n>We develop simple yet effective reward components in a reinforcement learning framework for coarse-to-fine perception.<n>Our approach delivers higher accuracy than the original model and competitive methods, with greater efficiency.
  arXiv  Detail & Related papers  (2025-09-26T07:15:19Z)
• Saccadic Vision for Fine-Grained Visual Classification [10.681604440788854]
  Fine-grained visual classification (FGVC) requires distinguishing between visually similar categories through subtle, localized features.<n>Existing part-based methods rely on complex localization networks that learn mappings from pixel to sample space.<n>We propose a two-stage process that first extracts peripheral features and generates a sample map.<n>We employ contextualized selective attention to weigh the impact of each fixation patch before fusing peripheral and focus representations.
  arXiv  Detail & Related papers  (2025-09-19T07:03:37Z)
• $\texttt{SPECS}$: Faster Test-Time Scaling through Speculative Drafts [55.231201692232894]
  $textttSPECS$ is a latency-aware test-time scaling method inspired by speculative decoding.<n>Our results show that $textttSPECS$matches or surpasses beam search accuracy while reducing latency by up to $sim$19.1%.
  arXiv  Detail & Related papers  (2025-06-15T05:50:05Z)
• Activation-aware Probe-Query: Effective Key-Value Retrieval for Long-Context LLMs Inference [56.71209737306054]
  We propose textbfActQKV, a training-free, textbfActivation-aware approach that dynamically determines probe-textbfQuery and leverages it to retrieve the relevant textbfKV pairs for inference.<n>Experiments on the Long-Bench and $infty$ Benchmarks demonstrate its state-of-the-art performance with competitive inference quality and resource efficiency.
  arXiv  Detail & Related papers  (2025-02-19T08:50:44Z)
• LoLep: Single-View View Synthesis with Locally-Learned Planes and Self-Attention Occlusion Inference [66.45326873274908]
  We propose a novel method, LoLep, which regresses Locally-Learned planes from a single RGB image to represent scenes accurately. Compared to MINE, our approach has an LPIPS reduction of 4.8%-9.0% and an RV reduction of 73.9%-83.5%.
  arXiv  Detail & Related papers  (2023-07-23T03:38:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.