CLM: Removing the GPU Memory Barrier for 3D Gaussian Splatting

• URL: http://arxiv.org/abs/2511.04951v1
• Date: Fri, 07 Nov 2025 03:30:28 GMT
• Title: CLM: Removing the GPU Memory Barrier for 3D Gaussian Splatting
• Authors: Hexu Zhao, Xiwen Min, Xiaoteng Liu, Moonjun Gong, Yiming Li, Ang Li, Saining Xie, Jinyang Li, Aurojit Panda,
• Abstract summary: CLM is a system that allows 3DGS to render large scenes using a single consumer-grade GPU.<n>It does so by offloading Gaussians to CPU memory, and loading them into GPU memory only when necessary.<n>To reduce performance and communication overheads, CLM uses a novel offloading strategy.
• Score: 34.933663925174635
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: 3D Gaussian Splatting (3DGS) is an increasingly popular novel view synthesis approach due to its fast rendering time, and high-quality output. However, scaling 3DGS to large (or intricate) scenes is challenging due to its large memory requirement, which exceed most GPU's memory capacity. In this paper, we describe CLM, a system that allows 3DGS to render large scenes using a single consumer-grade GPU, e.g., RTX4090. It does so by offloading Gaussians to CPU memory, and loading them into GPU memory only when necessary. To reduce performance and communication overheads, CLM uses a novel offloading strategy that exploits observations about 3DGS's memory access pattern for pipelining, and thus overlap GPU-to-CPU communication, GPU computation and CPU computation. Furthermore, we also exploit observation about the access pattern to reduce communication volume. Our evaluation shows that the resulting implementation can render a large scene that requires 100 million Gaussians on a single RTX4090 and achieve state-of-the-art reconstruction quality.

Related papers

• Quantile Rendering: Efficiently Embedding High-dimensional Feature on 3D Gaussian Splatting [52.18697134979677]
  Recent advancements in computer vision have successfully extended Open-vocabulary segmentation (OVS) to the 3D domain by leveraging 3D Gaussian Splatting (3D-GS)<n>Existing methods employ codebooks or feature compression, causing information loss, thereby degrading segmentation quality.<n>We introduce Quantile Rendering (Q-Render), a novel rendering strategy for 3D Gaussians that efficiently handles high-dimensional features while maintaining high fidelity.<n>Our framework outperforms state-of-the-art methods, while enabling real-time rendering with an approximate 43.7x speedup on 512-D feature maps.
  arXiv  Detail & Related papers  (2025-12-24T04:16:18Z)
• GS-Scale: Unlocking Large-Scale 3D Gaussian Splatting Training via Host Offloading [9.776813771006358]
  3D Gaussian Splatting has revolutionized graphics rendering by delivering high visual quality and fast rendering speeds.<n>Training large-scale scenes at high quality remains challenging due to substantial memory demands.<n>We propose GS-Scale, a fast and memory-efficient training system for 3D Gaussian Splatting.
  arXiv  Detail & Related papers  (2025-09-19T06:13:28Z)
• ContraGS: Codebook-Condensed and Trainable Gaussian Splatting for Fast, Memory-Efficient Reconstruction [9.155819295255212]
  3D Gaussian Splatting (3DGS) is a technique to model real-world scenes with high quality and real-time rendering.<n>We introduce ContraGS, a method to enable training directly on compressed 3DGS representations without reducing the Gaussian Counts.<n>We show that ContraGS significantly reduces the peak memory during training (on average 3.49X) and accelerated training and rendering (1.36X and 1.88X on average, respectively) while retraining close to state-of-art quality.
  arXiv  Detail & Related papers  (2025-09-03T23:40:17Z)
• FlexGS: Train Once, Deploy Everywhere with Many-in-One Flexible 3D Gaussian Splatting [57.97160965244424]
  3D Gaussian splatting (3DGS) has enabled various applications in 3D scene representation and novel view synthesis.<n>Previous approaches have focused on pruning less important Gaussians, effectively compressing 3DGS.<n>We present an elastic inference method for 3DGS, achieving substantial rendering performance without additional fine-tuning.
  arXiv  Detail & Related papers  (2025-06-04T17:17:57Z)
• LODGE: Level-of-Detail Large-Scale Gaussian Splatting with Efficient Rendering [75.67501939005119]
  We present a novel level-of-detail (LOD) method for 3D Gaussian Splatting on memory-constrained devices.<n>Our approach iteratively selects optimal subsets of Gaussians based on camera distance.<n>Our method achieves state-of-the-art performance on both outdoor (Hierarchical 3DGS) and indoor (Zip-NeRF) datasets.
  arXiv  Detail & Related papers  (2025-05-29T06:50:57Z)
• SLAG: Scalable Language-Augmented Gaussian Splatting [19.643023058839603]
  Language-augmented scene representations hold great promise for large-scale robotics applications such as search-and-rescue, smart cities, and mining.<n>Many of these scenarios are time-sensitive, requiring rapid scene encoding while also being data-intensive, necessitating scalable solutions.<n>We introduce SLAG, a multi-GPU framework for language-augmented Gaussian splatting that enhances the speed and scalability of embedding large scenes.
  arXiv  Detail & Related papers  (2025-05-12T23:32:24Z)
• MEGA: Memory-Efficient 4D Gaussian Splatting for Dynamic Scenes [49.36091070642661]
  This paper introduces a memory-efficient framework for 4DGS.<n>It achieves a storage reduction by approximately 190$times$ and 125$times$ on the Technicolor and Neural 3D Video datasets.<n>It maintains comparable rendering speeds and scene representation quality, setting a new standard in the field.
  arXiv  Detail & Related papers  (2024-10-17T14:47:08Z)
• On Scaling Up 3D Gaussian Splatting Training [31.161086345038424]
  3DGS is increasingly popular for 3D reconstruction due to its superior visual quality and rendering speed.<n>Currently, 3DGS training occurs on a single GPU, limiting its ability to handle high-resolution and large-scale 3D reconstruction tasks.<n>We introduce Grendel, a distributed system designed to partition 3DGS parameters and parallelize across multiple GPU.
  arXiv  Detail & Related papers  (2024-06-26T17:59:28Z)
• PUP 3D-GS: Principled Uncertainty Pruning for 3D Gaussian Splatting [59.277480452459315]
  We propose a principled sensitivity pruning score that preserves visual fidelity and foreground details at significantly higher compression ratios.<n>We also propose a multi-round prune-refine pipeline that can be applied to any pretrained 3D-GS model without changing its training pipeline.
  arXiv  Detail & Related papers  (2024-06-14T17:53:55Z)
• Practical offloading for fine-tuning LLM on commodity GPU via learned sparse projectors [11.127604539303373]
  Fine-tuning large language models (LLMs) requires significant memory, often exceeding the capacity of a single GPU.<n>A common solution to this memory challenge is offloading compute and data from the GPU to the CPU.<n>We present an offloading framework, LSP-Offload, that enables near-native speed LLM fine-tuning on commodity hardware.
  arXiv  Detail & Related papers  (2024-06-14T16:59:11Z)
• EAGLES: Efficient Accelerated 3D Gaussians with Lightweight EncodingS [40.94643885302646]
  3D Gaussian splatting (3D-GS) has gained popularity in novel-view scene synthesis. It addresses the challenges of lengthy training times and slow rendering speeds associated with Radiance Neural Fields (NeRFs) We present a technique utilizing quantized embeddings to significantly reduce per-point memory storage requirements.
  arXiv  Detail & Related papers  (2023-12-07T18:59:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.