Geometry-Guided 3D Visual Token Pruning for Video-Language Models

• URL: http://arxiv.org/abs/2604.18260v1
• Date: Mon, 20 Apr 2026 13:33:50 GMT
• Title: Geometry-Guided 3D Visual Token Pruning for Video-Language Models
• Abstract summary: We propose Geo3DPruner, a Geometry-Guided 3D Visual Token Pruning framework.<n>Geo3DPruner first models cross-frame relevance through geometry-aware global attention, and then performs a two-stage pruning process.<n>Experiments on multiple 3D scene understanding benchmarks demonstrate that Geo3DPruner retains over 90% of the original performance while pruning 90% of visual tokens, significantly outperforming existing text-guided and vision-guided pruning methods.
• Score: 35.786221225043455
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multimodal large language models have demonstrated remarkable capabilities in 2D vision, motivating their extension to 3D scene understanding. Recent studies represent 3D scenes as 3D spatial videos composed of image sequences with depth and camera pose information, enabling pre-trained video-language models to perform 3D reasoning tasks. However, the large number of visual tokens in spatial videos remains a major bottleneck for efficient inference and context management. Existing pruning methods overlook the view consistency of spatial videos and the spatial diversity of the remaining tokens, which prevents them from effectively removing inter-frame redundancy and preserving scene completeness. In this paper, we propose Geo3DPruner, a Geometry-Guided 3D Visual Token Pruning framework. Geo3DPruner first models cross-frame relevance through geometry-aware global attention, and then performs a two-stage pruning process. The intra-voxel stage selects representative multi-view features within each voxel, while the inter-voxel stage preserves spatial diversity by selecting a globally distributed subset of voxels. Extensive experiments on multiple 3D scene understanding benchmarks demonstrate that Geo3DPruner retains over 90% of the original performance while pruning 90% of visual tokens, significantly outperforming existing text-guided and vision-guided pruning methods.

Related papers

• Qwen-3D: A Generalist 3D Vision-Language Model for Spatial Understanding [24.745098191502233]
  3D geometry provides a natural compression mechanism for visual streams.<n>Qwen-3D is a geometry-aware LMM that compresses visual information within the Qwen backbone using multi-view geometric cues.<n>Qwen-3D surpasses existing 3D LMMs and outperforms several large proprietary 2D models.
  arXiv  Detail & Related papers  (2026-08-04T00:32:09Z)
• Proxy3D: Efficient 3D Representations for Vision-Language Models via Semantic Clustering and Alignment [50.02189698630855]
  We propose a Proxy3D method with compact yet comprehensive 3D proxy representations for the vision modality.<n>Our method achieves competitive or state-of-the-art performance in 3D visual question answering, visual grounding and general spatial intelligence benchmarks.
  arXiv  Detail & Related papers  (2026-05-08T17:50:47Z)
• DriveTok: 3D Driving Scene Tokenization for Unified Multi-View Reconstruction and Understanding [75.41918200553525]
  DriveTok is an efficient 3D driving scene tokenizer for unified multi-view reconstruction and understanding.<n>For decoding, we employ a multi-view transformer to reconstruct multi-view features from the scene tokens.<n>We also add a 3D head directly on the scene tokens for 3D semantic occupancy prediction.
  arXiv  Detail & Related papers  (2026-03-19T17:58:22Z)
• 3D Aware Region Prompted Vision Language Model [99.4106711584584]
  SR-3D connects single-view 2D images and multi-view 3D data through a shared visual token space.<n> SR-3D supports flexible region prompting, allowing users to annotate regions with bounding boxes, segmentation masks on any frame, or directly in 3D.
  arXiv  Detail & Related papers  (2025-09-16T17:59:06Z)
• Unleashing the Multi-View Fusion Potential: Noise Correction in VLM for Open-Vocabulary 3D Scene Understanding [15.86865606131156]
  We introduce MVOV3D, a novel approach aimed at unleashing the potential of 2D multi-view fusion for open-vocabulary 3D scene understanding.<n>Specifically, MVOV3D improves multi-view 2D features by leveraging precise region-level image features and text features encoded by CLIP encoders.<n>Our method achieves a new record with 14.7% mIoU on ScanNet200 and 16.2% mIoU on Matterport160 for challenge open-vocabulary semantic segmentation.
  arXiv  Detail & Related papers  (2025-06-28T08:40:42Z)
• NVSMask3D: Hard Visual Prompting with Camera Pose Interpolation for 3D Open Vocabulary Instance Segmentation [14.046423852723615]
  We introduce a novel 3D Gaussian Splatting based hard visual prompting approach to generate diverse viewpoints around target objects.<n>Our method simulates realistic 3D perspectives, effectively augmenting existing hard visual prompts.<n>This training-free strategy integrates seamlessly with prior hard visual prompts, enriching object-descriptive features.
  arXiv  Detail & Related papers  (2025-04-20T14:39:27Z)
• MLLM-For3D: Adapting Multimodal Large Language Model for 3D Reasoning Segmentation [91.94869042117621]
  Reasoning segmentation aims to segment target objects in complex scenes based on human intent and spatial reasoning.<n>Recent multimodal large language models (MLLMs) have demonstrated impressive 2D image reasoning segmentation.<n>We introduce MLLM-For3D, a framework that transfers knowledge from 2D MLLMs to 3D scene understanding.
  arXiv  Detail & Related papers  (2025-03-23T16:40:20Z)
• WildSeg3D: Segment Any 3D Objects in the Wild from 2D Images [16.107027445270887]
  We introduce WildSeg3D, an efficient approach that enables the segmentation of arbitrary 3D objects across diverse environments.<n>A key challenge of this feed-forward approach lies in the accumulation of 3D alignment errors across multiple 2D views.<n>We also propose Dynamic Global Aligning (DGA) and Multi-view Group Mapping (MGM) for real-time interactive segmentation.
  arXiv  Detail & Related papers  (2025-03-11T13:10:41Z)
• g3D-LF: Generalizable 3D-Language Feature Fields for Embodied Tasks [62.74304008688472]
  Generalizable 3D-Language Feature Fields (g3D-LF) is a 3D representation model pre-trained on large-scale 3D-language dataset for embodied tasks.
  arXiv  Detail & Related papers  (2024-11-26T01:54:52Z)
• Volumetric Environment Representation for Vision-Language Navigation [66.04379819772764]
  Vision-language navigation (VLN) requires an agent to navigate through a 3D environment based on visual observations and natural language instructions. We introduce a Volumetric Environment Representation (VER), which voxelizes the physical world into structured 3D cells. VER predicts 3D occupancy, 3D room layout, and 3D bounding boxes jointly.
  arXiv  Detail & Related papers  (2024-03-21T06:14:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.