3DZip: Spatial-Aware Feature Diversity-Guided Token Compression for 3D Question Answering

• URL: http://arxiv.org/abs/2608.01185v1
• Date: Sun, 02 Aug 2026 12:11:51 GMT
• Title: 3DZip: Spatial-Aware Feature Diversity-Guided Token Compression for 3D Question Answering
• Abstract summary: Recent 3D vision-language models construct geometry aware tokens by projecting 2D visual features into world coordinates.<n>This design generates thousands of tokens per scene, resulting in substantial computational and memory overhead.<n>We propose 3DZip, a three-stage token compression framework that first applies coarse voxelization to remove point-level redundancy, then selects anchor tokens based on feature-space diversity, and finally merges remaining tokens under spatial constraints to preserve geometric coherence.
• Score: 8.79576142391319
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent 3D vision-language models (3D VLMs) construct geometry aware tokens by projecting 2D visual features into world coordinates, enabling spatial reasoning for tasks such as 3D question answering. However, this design generates thousands of tokens per scene, resulting in substantial computational and memory overhead. While token compression has been extensively studied in 2D VLMs, existing approaches rely on semantic relevance or attention-based selection that overlook the structured spatial nature of 3D tokens. Moreover, redundancy in 3D representations cannot be resolved by spatial proximity alone, as object-level token imbalance persists even after spatial aggregation. To address this, we propose 3DZip, a three-stage token compression framework that first applies coarse voxelization to remove point-level redundancy, then selects anchor tokens based on feature-space diversity via a Determinantal Point Process, and finally merges remaining tokens under spatial constraints to preserve geometric coherence. Experiments on three 3D question answering benchmarks demonstrate that 3DZip consistently outperforms existing compression methods, retaining 94.7% of the original performance with only 128 tokens, achieving a $1.92\times$ faster inference speed.

Related papers

• CoVeR: Coverage-Based Token Pruning for Multi-View 3D Reasoning in VLMs [58.22241532542095]
  Representing a 3D scene as multi-view images allows 2D VLMs to reason in 3D by reusing priors from pre-training, sidestepping the scarcity of annotated 3D data.<n>Existing visual token pruners fall into two families, each limited in the 3D multi-view setting.<n>We show that spatial coverage is associated with 3D reasoning performance and introduce CoVeR, a deterministic, training-free selector that uses only token coordinates, with no learned signals.
  arXiv  Detail & Related papers  (2026-09-08T07:16:37Z)
• ZipTok3D: High-Fidelity 3D Tokenization with Compact Token Prefixes [38.735159381505014]
  ZipTok3D is a 3D tokenizer designed for high-fidelity reconstruction from extremely short token sequences.<n>With the same token dimension, ZipTok3D achieves reconstruction quality comparable to the 32-token COD-VAE baseline.
  arXiv  Detail & Related papers  (2026-09-01T18:07:49Z)
• CoverPrune: Coverage-Driven Token Pruning for 3D VLMs via Optimal Transport [50.2610873651284]
  Existing token pruning methods rely on diversity-based selection, discarding similar tokens to maximize dispersion.<n>In 3D environments, this approach frequently drops representative prototype tokens in favor of outliers.<n>We propose a paradigm shift for 3D VLM token pruning: from maximizing OT to preserving visual evidence coverage.
  arXiv  Detail & Related papers  (2026-08-13T13:29:15Z)
• SToRe3D: Sparse Token Relevance in ViTs for Efficient Multi-View 3D Object Detection [2.148411555253816]
  Vision Transformers (ViTs) enable strong multi-view 3D detection but are limited by high inference latency from dense token and query processing across multiple views and large 3D regions.<n>We introduce SToRe3D, a relevance-aligned sparsity framework that jointly selects 2D image tokens and 3D object queries while storing filtered features for reactivation.
  arXiv  Detail & Related papers  (2026-05-13T20:53:03Z)
• SeGPruner: Semantic-Geometric Visual Token Pruner for 3D Question Answering [18.541639027941198]
  SeGPruner is a semantic-aware and geometry-guided token reduction framework for efficient 3D with multi-view images.<n>It substantially improves inference efficiency, reducing the visual token budget by 91% and inference latency by 86%.
  arXiv  Detail & Related papers  (2026-03-31T08:44:04Z)
• Cog3DMap: Multi-View Vision-Language Reasoning with 3D Cognitive Maps [77.63233146945718]
  We introduce Cog3DMap, a framework that recurrently constructs an explicit 3D memory from multi-view images.<n>Our framework enables direct reasoning over a spatially structured 3D map, achieving state-of-the-art performance on various spatial reasoning benchmarks.
  arXiv  Detail & Related papers  (2026-03-24T10:05:32Z)
• VolumeDP: Modeling Volumetric Representation for Manipulation Policy Learning [29.62906091681386]
  VolumeDP is a policy architecture that restores spatial alignment by explicitly reasoning in 3D.<n>It achieves a state-of-the-art average success rate of 88.8% on the LIBERO simulation benchmark.
  arXiv  Detail & Related papers  (2026-03-18T13:40:24Z)
• HCC-3D: Hierarchical Compensatory Compression for 98% 3D Token Reduction in Vision-Language Models [19.63602285036466]
  3D understanding has drawn significant attention recently, leveraging Vision-Language Models (VLMs) to enable multi-modal reasoning between point cloud and text data.<n>Current 3D-VLMs directly embed the 3D point clouds into 3D tokens, following large 2D-VLMs with powerful reasoning capabilities.<n>This framework has a great computational cost limiting its application, where we identify that the bottleneck lies in processing all 3D tokens in the Large Language Model (LLM) part.<n>This raises the question: how can we reduce the computational overhead introduced by 3D tokens while preserving the integrity of their essential information?<n>
  arXiv  Detail & Related papers  (2025-11-13T02:28:10Z)
• SeqAffordSplat: Scene-level Sequential Affordance Reasoning on 3D Gaussian Splatting [85.87902260102652]
  We introduce the novel task of Sequential 3D Gaussian Affordance Reasoning.<n>We then propose SeqSplatNet, an end-to-end framework that directly maps an instruction to a sequence of 3D affordance masks.<n>Our method sets a new state-of-the-art on our challenging benchmark, effectively advancing affordance reasoning from single-step interactions to complex, sequential tasks at the scene level.
  arXiv  Detail & Related papers  (2025-07-31T17:56:55Z)
• Self-Supervised and Generalizable Tokenization for CLIP-Based 3D Understanding [87.68271178167373]
  We present a universal 3D tokenizer designed for scale-invariant representation learning with a frozen CLIP backbone.<n>S4Token is a tokenization pipeline that produces semantically-informed tokens regardless of scene scale.
  arXiv  Detail & Related papers  (2025-05-24T18:26:30Z)
• NDC-Scene: Boost Monocular 3D Semantic Scene Completion in Normalized Device Coordinates Space [77.6067460464962]
  Monocular 3D Semantic Scene Completion (SSC) has garnered significant attention in recent years due to its potential to predict complex semantics and geometry shapes from a single image, requiring no 3D inputs. We identify several critical issues in current state-of-the-art methods, including the Feature Ambiguity of projected 2D features in the ray to the 3D space, the Pose Ambiguity of the 3D convolution, and the Imbalance in the 3D convolution across different depth levels. We devise a novel Normalized Device Coordinates scene completion network (NDC-Scene) that directly extends the 2
  arXiv  Detail & Related papers  (2023-09-26T02:09:52Z)
• EP2P-Loc: End-to-End 3D Point to 2D Pixel Localization for Large-Scale Visual Localization [44.05930316729542]
  We propose EP2P-Loc, a novel large-scale visual localization method for 3D point clouds. To increase the number of inliers, we propose a simple algorithm to remove invisible 3D points in the image. For the first time in this task, we employ a differentiable for end-to-end training.
  arXiv  Detail & Related papers  (2023-09-14T07:06:36Z)
• Learning Joint 2D-3D Representations for Depth Completion [90.62843376586216]
  We design a simple yet effective neural network block that learns to extract joint 2D and 3D features. Specifically, the block consists of two domain-specific sub-networks that apply 2D convolution on image pixels and continuous convolution on 3D points.
  arXiv  Detail & Related papers  (2020-12-22T22:58:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.