Where to Look Matters: Learning Influential Views for VLM-based 3D Visual Grounding

• URL: http://arxiv.org/abs/2609.04741v1
• Date: Fri, 04 Sep 2026 05:17:38 GMT
• Title: Where to Look Matters: Learning Influential Views for VLM-based 3D Visual Grounding
• Abstract summary: IVS is a framework that learns Influential View Selection for VLM-based 3D visual grounding.<n>We show that IVS consistently improves grounding accuracy over existing zero-shot pipelines.
• Score: 20.234831906157154
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent zero-shot 3D visual grounding methods leverage vision-language models (VLMs) to localize objects in 3D scenes from natural language queries. However, these methods typically rely on heuristic rules to select which camera views are provided to the VLM, often prioritizing object visibility rather than grounding relevance. We present IVSGround, a framework that learns Influential View Selection for VLM-based 3D visual grounding. Instead of using fixed heuristics, a lightweight view selector is trained to identify views that provide discriminative evidence for grounding. To obtain supervision signals, we generate training signals using feedback from a reasoning VLM through a two-stage rejection sampling process. During inference, the learned selector predicts query-conditioned influential views for each candidate object, which are then evaluated by a frozen reasoning VLM through comparative grounding. Experiments on ScanRefer and NR3D show that IVSGround consistently improves grounding accuracy over existing zero-shot pipelines, demonstrating that selecting where to look is crucial for effective 3D visual grounding. Project page: https://ivsground.github.io/

Related papers

• Training-Free Open-Vocabulary Visual Grounding for Remote Sensing Images and Videos [52.516842108549135]
  Remote sensing visual grounding aims to localize a referred target in a remote sensing image or video according to a natural language expression.<n>Existing RSVG methods usually rely on task-specific manual annotations, which are costly to collect and inevitably limited in covering the diversity of real-world geospatial scenarios.<n>We propose RSVGZeroOV, a training-free framework that leverages frozen generic foundation models for zero-shot open-vocabulary RSVG.
  arXiv  Detail & Related papers  (2026-06-15T02:31:32Z)
• Any3D-VLA: Enhancing VLA Robustness via Diverse Point Clouds [57.024495128182195]
  We conduct a pilot study across different observation spaces and visual representations.<n>Results show that explicitly lifting visual input into point clouds yields representations that better complement their corresponding 2D representations.<n>We propose Any3D-VLA to address the challenges of (1) scarce 3D data and (2) the domain gap induced by cross-environment differences and depth-scale biases.
  arXiv  Detail & Related papers  (2026-01-31T16:34:52Z)
• View-on-Graph: Zero-shot 3D Visual Grounding via Vision-Language Reasoning on Scene Graphs [19.27758108925572]
  3D visual grounding identifies objects in 3D scenes from language descriptions.<n>Existing zero-shot approaches leverage 2D vision-language models (VLMs) by converting 3D spatial information (SI) into forms to VLM processing.<n>We propose a new VLM x SI paradigm that externalizes the 3D SI into a form enabling the VLM to incrementally retrieve only what it needs during reasoning.
  arXiv  Detail & Related papers  (2025-12-10T00:59:17Z)
• iFinder: Structured Zero-Shot Vision-Based LLM Grounding for Dash-Cam Video Reasoning [51.15353027471834]
  iFinder is a semantic grounding framework that translates dash-cam videos into a hierarchical, interpretable data structure for large language models.<n>iFinder operates as a training-free pipeline that employs pretrained vision models to extract critical cues.<n>It significantly outperforms end-to-end V-VLMs on four zero-shot driving benchmarks.
  arXiv  Detail & Related papers  (2025-09-23T20:25:53Z)
• SeqVLM: Proposal-Guided Multi-View Sequences Reasoning via VLM for Zero-Shot 3D Visual Grounding [40.60812160987424]
  3D Visual Grounding (3DVG) aims to localize objects in 3D scenes using natural language descriptions.<n>We propose SeqVLM, a novel zero-shot 3DVG framework that leverages multi-view real-world scene images with spatial information for target object reasoning.<n> Experiments on the ScanRefer and Nr3D benchmarks demonstrate state-of-the-art performance, surpassing previous zero-shot methods by 4.0% and 5.2%, respectively.
  arXiv  Detail & Related papers  (2025-08-28T13:15:37Z)
• VLM-Grounder: A VLM Agent for Zero-Shot 3D Visual Grounding [57.04804711488706]
  3D visual grounding is crucial for robots, requiring integration of natural language and 3D scene understanding. We present VLM-Grounder, a novel framework using vision-language models (VLMs) for zero-shot 3D visual grounding based solely on 2D images.
  arXiv  Detail & Related papers  (2024-10-17T17:59:55Z)
• 3D Object Detection with a Self-supervised Lidar Scene Flow Backbone [10.341296683155973]
  We propose using a self-supervised training strategy to learn a general point cloud backbone model for downstream 3D vision tasks. Our main contribution leverages learned flow and motion representations and combines a self-supervised backbone with a 3D detection head. Experiments on KITTI and nuScenes benchmarks show that the proposed self-supervised pre-training increases 3D detection performance significantly.
  arXiv  Detail & Related papers  (2022-05-02T07:53:29Z)
• Multi-View Transformer for 3D Visual Grounding [64.30493173825234]
  We propose a Multi-View Transformer (MVT) for 3D visual grounding. We project the 3D scene to a multi-view space, in which the position information of the 3D scene under different views are modeled simultaneously and aggregated together.
  arXiv  Detail & Related papers  (2022-04-05T12:59:43Z)
• PreViTS: Contrastive Pretraining with Video Tracking Supervision [53.73237606312024]
  PreViTS is an unsupervised SSL framework for selecting clips containing the same object. PreViTS spatially constrains the frame regions to learn from and trains the model to locate meaningful objects. We train a momentum contrastive (MoCo) encoder on VGG-Sound and Kinetics-400 datasets with PreViTS.
  arXiv  Detail & Related papers  (2021-12-01T19:49:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.