HiSC: Hierarchical Spatial Clustering Token Compression for Efficient 3D Scene Understanding

• URL: http://arxiv.org/abs/2608.04610v1
• Date: Wed, 05 Aug 2026 09:15:43 GMT
• Title: HiSC: Hierarchical Spatial Clustering Token Compression for Efficient 3D Scene Understanding
• Abstract summary: 3D vision-language models (3D VLMs) enable spatial reasoning over multi-view scenes but suffer from substantial token redundancy.<n>We propose textbfHiSC, a training-free framework for hierarchical spatial clustering token compression in 3D VLMs.<n>HiSC lifts token compression from token-level selection to cluster-level processing by organizing tokens into spatially grounded clusters.
• Score: 16.080049512272026
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: 3D vision-language models (3D VLMs) enable spatial reasoning over multi-view scenes but suffer from substantial token redundancy due to duplicated observations and large uninformative regions, leading to high computational cost. Although visual token compression has shown promise in accelerating 2D VLMs, it fails to capture the structured nature of 3D scenes and leads to incomplete spatial coverage and loss of fine-grained details. In this paper, we propose \textbf{HiSC}, a training-free framework for hierarchical spatial clustering token compression in 3D VLMs. HiSC lifts token compression from token-level selection to cluster-level processing by organizing tokens into spatially grounded clusters using joint geometric and semantic cues. Specifically, we first introduce a \textbf{spatial graph-based merging (SGraM) strategy} that models cross-view redundancy as spatial connectivity and consolidates physically consistent regions, effectively merging extremely similar redundant tokens prior to LLM inference. We then propose a \textbf{spatial clustering-based pruning (SCluP) paradigm} within LLM inference, which performs hierarchical compression across clusters and within clusters, preserving object instance completeness while retaining fine-grained details for important regions. Extensive experiments on diverse 3D reasoning benchmarks show validate the effectiveness of HiSC, particularly under high visual token pruning ratios. Besides, HiSC achieves over 90\% token reduction with minimal performance degradation. Code is accessible at https://github.com/elecreak/HiSC.

Related papers

• CoverPrune: Coverage-Driven Token Pruning for 3D VLMs via Optimal Transport [50.2610873651284]
  Existing token pruning methods rely on diversity-based selection, discarding similar tokens to maximize dispersion.<n>In 3D environments, this approach frequently drops representative prototype tokens in favor of outliers.<n>We propose a paradigm shift for 3D VLM token pruning: from maximizing OT to preserving visual evidence coverage.
  arXiv  Detail & Related papers  (2026-08-13T13:29:15Z)
• Unfolding 3D Gaussian Splatting via Iterative Gaussian Synopsis [48.743921535513635]
  Iterative Gaussian Synopsis is a novel framework for compact and progressive rendering through a top-down "unfolding" scheme.<n>Our approach begins with a full-resolution 3DGS model and iteratively derives coarser LODs using an adaptive, learnable mask-based pruning mechanism.<n>This combination produces a compact yet expressive feature representation, designed to minimize redundancy and support efficient, level-specific adaptation.
  arXiv  Detail & Related papers  (2026-04-13T16:28:35Z)
• Towards Foundation Models for 3D Scene Understanding: Instance-Aware Self-Supervised Learning for Point Clouds [53.82500407523346]
  PointINS is an instance-oriented self-supervised framework that enriches point cloud representations through geometry-aware learning.<n>PointINS achieves on average +3.5% mAP improvement for indoor instance segmentation and +4.1% PQ gain for outdoor panoptic segmentation.
  arXiv  Detail & Related papers  (2026-03-26T08:31:06Z)
• LaSSM: Efficient Semantic-Spatial Query Decoding via Local Aggregation and State Space Models for 3D Instance Segmentation [21.566771922153027]
  We introduce LaSSM, which prioritizes simplicity and efficiency while maintaining competitive performance.<n>We also present a coordinate-guided state space model (SSM) decoder that progressively refines queries.<n>LaSSM ranks first place on the latest ScanNet++ V2 leaderboard, outperforming the previous best method by 2.5% mAP with only 1/3 FLOPs.
  arXiv  Detail & Related papers  (2026-02-11T16:34:12Z)
• CSGaussian: Progressive Rate-Distortion Compression and Segmentation for 3D Gaussian Splatting [57.73006852239138]
  We present the first unified framework for rate-distortion-optimized compression and segmentation of 3D Gaussian Splatting (3DGS)<n>Inspired by recent advances in rate-distortion-optimized 3DGS compression, this work integrates semantic learning into the compression pipeline to support decoder-side applications.<n>Our scheme features a lightweight implicit neural representation-based hyperprior, enabling efficient entropy coding of both color and semantic attributes.
  arXiv  Detail & Related papers  (2026-01-19T08:21:45Z)
• Deep Global Clustering for Hyperspectral Image Segmentation: Concepts, Applications, and Open Challenges [1.9116784879310027]
  Hyperspectral imaging (HSI) analysis faces computational bottlenecks due to massive data volumes that exceed available memory.<n>This report presents Deep Global Clustering (DGC), a conceptual framework for memory-efficient HSI segmentation.<n>DGC operates on small patches with overlapping regions to enforce consistency, enabling training in under 30 minutes on consumer hardware.
  arXiv  Detail & Related papers  (2025-12-30T12:10:43Z)
• C3G: Learning Compact 3D Representations with 2K Gaussians [55.04010158339562]
  Recent approaches use per-pixel 3D Gaussian Splatting for reconstruction, followed by a 2D-to-3D feature lifting stage for scene understanding.<n>We propose C3G, a novel feed-forward framework that estimates compact 3D Gaussians only at essential spatial locations.
  arXiv  Detail & Related papers  (2025-12-03T17:59:05Z)
• CORE: Compact Object-centric REpresentations as a New Paradigm for Token Merging in LVLMs [29.08277140543501]
  We introduce CORE (Compact Object-centric REpresentations), a new paradigm for visual token compression.<n> CORE leverages an efficient segmentation decoder to generate object masks, which serve as a high-level semantic prior to guide the merging of visual tokens.<n>Experiments show that CORE not only establishes a new state-of-the-art on six authoritative benchmarks for fixed-rate compression, but also achieves dramatic efficiency gains in adaptive-rate settings.
  arXiv  Detail & Related papers  (2025-11-18T03:02:23Z)
• HCC-3D: Hierarchical Compensatory Compression for 98% 3D Token Reduction in Vision-Language Models [19.63602285036466]
  3D understanding has drawn significant attention recently, leveraging Vision-Language Models (VLMs) to enable multi-modal reasoning between point cloud and text data.<n>Current 3D-VLMs directly embed the 3D point clouds into 3D tokens, following large 2D-VLMs with powerful reasoning capabilities.<n>This framework has a great computational cost limiting its application, where we identify that the bottleneck lies in processing all 3D tokens in the Large Language Model (LLM) part.<n>This raises the question: how can we reduce the computational overhead introduced by 3D tokens while preserving the integrity of their essential information?<n>
  arXiv  Detail & Related papers  (2025-11-13T02:28:10Z)
• Have We Scene It All? Scene Graph-Aware Deep Point Cloud Compression [18.40946383877556]
  We propose a deep compression framework based on semantic scene graphs.<n>We show that the framework achieves state-of-the-art compression rates, reducing data size by up to 98%.<n>It supports downstream applications such as multi-robot pose graph optimization and map merging.
  arXiv  Detail & Related papers  (2025-10-09T17:45:09Z)
• Point Linguist Model: Segment Any Object via Bridged Large 3D-Language Model [51.02616473941499]
  3D object segmentation with Large Language Models (LLMs) has become a prevailing paradigm due to its broad semantics, task flexibility, and strong generalization.<n>However, this paradigm is hindered by representation misalignment: LLMs process high-level semantic tokens, whereas 3D point clouds convey only dense geometric structures.<n>We present the Point Linguist Model (PLM), a general framework that bridges the representation gap between LLMs and dense 3D point clouds.
  arXiv  Detail & Related papers  (2025-09-09T15:01:28Z)
• Self-Supervised and Generalizable Tokenization for CLIP-Based 3D Understanding [87.68271178167373]
  We present a universal 3D tokenizer designed for scale-invariant representation learning with a frozen CLIP backbone.<n>S4Token is a tokenization pipeline that produces semantically-informed tokens regardless of scene scale.
  arXiv  Detail & Related papers  (2025-05-24T18:26:30Z)
• ClusteringSDF: Self-Organized Neural Implicit Surfaces for 3D Decomposition [32.99080359375706]
  ClusteringSDF is a novel approach to achieve both segmentation and reconstruction in 3D via the neural implicit surface representation. We introduce a high-efficient clustering mechanism for lifting the 2D labels to 3D and the experimental results on the challenging scenes from ScanNet and Replica datasets show that ClusteringSDF can achieve competitive performance.
  arXiv  Detail & Related papers  (2024-03-21T17:59:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.