ProFocus: Proactive Perception and Focused Reasoning in Vision-and-Language Navigation

• URL: http://arxiv.org/abs/2603.05530v1
• Date: Sun, 01 Mar 2026 04:13:18 GMT
• Title: ProFocus: Proactive Perception and Focused Reasoning in Vision-and-Language Navigation
• Abstract summary: We propose a training-free progressive framework that unifies underlineProactive Perception and underlineFocused Reasoning.<n>For proactive perception, ProFocus transforms panoramic observations into structured ego-centric semantic maps.<n>For focused reasoning, we propose Branch-Diverse Monte Carlo Tree Search (BD-MCTS) to identify top-$k$ high-value waypoints.
• Score: 57.65093237897077
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-and-Language Navigation (VLN) requires agents to accurately perceive complex visual environments and reason over navigation instructions and histories. However, existing methods passively process redundant visual inputs and treat all historical contexts indiscriminately, resulting in inefficient perception and unfocused reasoning. To address these challenges, we propose \textbf{ProFocus}, a training-free progressive framework that unifies \underline{Pro}active Perception and \underline{Focus}ed Reasoning through collaboration between large language models (LLMs) and vision-language models (VLMs). For proactive perception, ProFocus transforms panoramic observations into structured ego-centric semantic maps, enabling the orchestration agent to identify missing visual information needed for reliable decision-making, and to generate targeted visual queries with corresponding focus regions that guide the perception agent to acquire the required observations. For focused reasoning, we propose Branch-Diverse Monte Carlo Tree Search (BD-MCTS) to identify top-$k$ high-value waypoints from extensive historical candidates. The decision agent focuses reasoning on the historical contexts associated with these waypoints, rather than considering all historical waypoints equally. Extensive experiments validate the effectiveness of ProFocus, achieving state-of-the-art performance among zero-shot methods on R2R and REVERIE benchmarks.

Related papers

• LaCoVL-FER: Landmark-Guided Contrastive Learning Network with Vision-Language Enhancement for Facial Expression Recognition [51.70817823155725]
  We propose a novel landmark-guided contrastive learning network with vision-language enhancement for Facial Expression Recognition (FER)<n>LaCoVL-FER integrates geometric priors from facial landmarks and semantic priors from a vision-language model.<n>Experiments demonstrate that LaCoVL-FER outperforms state-of-the-art methods on three representative real-world FER datasets.
  arXiv  Detail & Related papers  (2026-05-19T13:15:41Z)
• CAVE: A Structured Credit Assignment Approach for Fragmented Visual Evidence Reasoning [13.024838183478858]
  We propose Credit Assignment for Visual Evidence (CAVE) for interleaved visual reasoning.<n>CAVE evaluates the contribution of intermediate steps at the action level via three complementary reasoning process signals.<n>We construct TRACER-Bench, which covers four nonlocal and semantically confusable reasoning dimensions.
  arXiv  Detail & Related papers  (2026-05-13T16:50:24Z)
• Saliency-R1: Enforcing Interpretable and Faithful Vision-language Reasoning via Saliency-map Alignment Reward [26.150136674969605]
  We propose Saliency-R1, a framework for improving the interpretability and faithfulness of vision-language models (VLMs)<n>We introduce a novel saliency map technique that efficiently highlights critical image regions contributing to generated tokens without additional computational overhead.<n>Experiments show Saliency-R1 improves reasoning faithfulness, interpretability, and overall task performance.
  arXiv  Detail & Related papers  (2026-04-06T07:51:59Z)
• Focus-Scan-Refine: From Human Visual Perception to Efficient Visual Token Pruning [78.75062483648243]
  Vision-language models (VLMs) often generate massive visual tokens that greatly increase inference latency and memory footprint.<n>We propose Focus-Scan-Refine (FSR), a human-inspired, plug-and-play pruning framework that mimics how humans answer visual questions.<n>FSR consistently improves the accuracy-efficiency trade-off over existing state-of-the-art pruning methods.
  arXiv  Detail & Related papers  (2026-02-05T16:02:48Z)
• ViThinker: Active Vision-Language Reasoning via Dynamic Perceptual Querying [15.728211622542267]
  ViThinker is a framework that enables vision-language models to autonomously generate decision tokens triggering the synthesis of expert-aligned visual features on demand.<n>ViThinker internalizes vision-expert capabilities during training, performing generative mental simulation during inference without external tool calls.
  arXiv  Detail & Related papers  (2026-02-02T22:29:57Z)
• A Multimodal Depth-Aware Method For Embodied Reference Understanding [56.30142869506262]
  Embodied Reference Understanding requires identifying a target object in a visual scene based on both language instructions and pointing cues.<n>We propose a novel ERU framework that jointly leverages data augmentation, depth-map modality, and a depth-aware decision module.
  arXiv  Detail & Related papers  (2025-10-09T14:32:21Z)
• Focusing by Contrastive Attention: Enhancing VLMs' Visual Reasoning [79.34909830834464]
  Vision-Language Models (VLMs) have demonstrated remarkable success across diverse visual tasks, yet their performance degrades in complex visual environments.<n>We show that visual complexity strongly correlates with attention entropy, negatively impacting reasoning performance.<n>We propose Contrastive Attention Refinement for Visual Enhancement (CARVE), a training-free method that extracts task-relevant visual signals through attention contrasting at the pixel level.
  arXiv  Detail & Related papers  (2025-09-08T09:20:04Z)
• Generalized Decoupled Learning for Enhancing Open-Vocabulary Dense Perception [71.26728044621458]
  DeCLIP is a novel framework that enhances CLIP by decoupling the self-attention module to obtain content'' and context'' features respectively.<n>It consistently achieves state-of-the-art performance across a broad spectrum of tasks, including 2D detection and segmentation, 3D instance segmentation, video instance segmentation, and 6D object pose estimation.
  arXiv  Detail & Related papers  (2025-08-15T06:43:51Z)
• VisRL: Intention-Driven Visual Perception via Reinforced Reasoning [22.907814548315468]
  We propose VisRL, the first framework that applies reinforcement learning (RL) to the problem of intention-driven visual perception.<n>By treating intermediate focus selection as an internal decision optimized through trial-and-error, our method eliminates the need for costly region annotations.<n>Our method consistently outperforms strong baselines, demonstrating both its effectiveness and its strong generalization across different LMMs.
  arXiv  Detail & Related papers  (2025-03-10T16:49:35Z)
• Referencing Where to Focus: Improving VisualGrounding with Referential Query [30.33315985826623]
  We propose a novel visual grounding method called RefFormer.<n>It consists of the query adaption module that can be seamlessly integrated into CLIP.<n>Our proposed query adaption module can also act as an adapter, preserving the rich knowledge within CLIP without the need to tune the parameters of the backbone network.
  arXiv  Detail & Related papers  (2024-12-26T10:19:20Z)
• Open-Vocabulary Animal Keypoint Detection with Semantic-feature Matching [74.75284453828017]
  Open-Vocabulary Keypoint Detection (OVKD) task is innovatively designed to use text prompts for identifying arbitrary keypoints across any species. We have developed a novel framework named Open-Vocabulary Keypoint Detection with Semantic-feature Matching (KDSM) This framework combines vision and language models, creating an interplay between language features and local keypoint visual features.
  arXiv  Detail & Related papers  (2023-10-08T07:42:41Z)
• Exploring Predicate Visual Context in Detecting Human-Object Interactions [44.937383506126274]
  We study how best to re-introduce image features via cross-attention. Our model with enhanced predicate visual context (PViC) outperforms state-of-the-art methods on the HICO-DET and V-COCO benchmarks.
  arXiv  Detail & Related papers  (2023-08-11T15:57:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.